Accurate traffic flow prediction has a profound impact on modern traffic management. Traffic flow has complex spatial-temporal correlations and periodicity, which poses difficulties for precise prediction. To address ...Accurate traffic flow prediction has a profound impact on modern traffic management. Traffic flow has complex spatial-temporal correlations and periodicity, which poses difficulties for precise prediction. To address this problem, a Multi-head Self-attention and Spatial-Temporal Graph Convolutional Network (MSSTGCN) for multiscale traffic flow prediction is proposed. Firstly, to capture the hidden traffic periodicity of traffic flow, traffic flow is divided into three kinds of periods, including hourly, daily, and weekly data. Secondly, a graph attention residual layer is constructed to learn the global spatial features across regions. Local spatial-temporal dependence is captured by using a T-GCN module. Thirdly, a transformer layer is introduced to learn the long-term dependence in time. A position embedding mechanism is introduced to label position information for all traffic sequences. Thus, this multi-head self-attention mechanism can recognize the sequence order and allocate weights for different time nodes. Experimental results on four real-world datasets show that the MSSTGCN performs better than the baseline methods and can be successfully adapted to traffic prediction tasks.展开更多
Video emotion recognition is widely used due to its alignment with the temporal characteristics of human emotional expression,but existingmodels have significant shortcomings.On the one hand,Transformermultihead self-...Video emotion recognition is widely used due to its alignment with the temporal characteristics of human emotional expression,but existingmodels have significant shortcomings.On the one hand,Transformermultihead self-attention modeling of global temporal dependency has problems of high computational overhead and feature similarity.On the other hand,fixed-size convolution kernels are often used,which have weak perception ability for emotional regions of different scales.Therefore,this paper proposes a video emotion recognition model that combines multi-scale region-aware convolution with temporal interactive sampling.In terms of space,multi-branch large-kernel stripe convolution is used to perceive emotional region features at different scales,and attention weights are generated for each scale feature.In terms of time,multi-layer odd-even down-sampling is performed on the time series,and oddeven sub-sequence interaction is performed to solve the problem of feature similarity,while reducing computational costs due to the linear relationship between sampling and convolution overhead.This paper was tested on CMU-MOSI,CMU-MOSEI,and Hume Reaction.The Acc-2 reached 83.4%,85.2%,and 81.2%,respectively.The experimental results show that the model can significantly improve the accuracy of emotion recognition.展开更多
This study proposes a multi-scale simplified residual convolutional neural network(MS-SRCNN)for the precise prediction of Mg-Nd binary alloy compositions from scanning electron microscope(SEM)images.A multi-scale data...This study proposes a multi-scale simplified residual convolutional neural network(MS-SRCNN)for the precise prediction of Mg-Nd binary alloy compositions from scanning electron microscope(SEM)images.A multi-scale data structure is established by spatially aligning and stacking SEM images at different magnifications.The MS-SRCNN significantly reduces computational runtime by over 90%compared to traditional architectures like ResNet50,VGG16,and VGG19,without compromising prediction accuracy.The model demonstrates more excellent predictive performance,achieving a>5%increase in R2 compared to single-scale models.Furthermore,the MS-SRCNN exhibits robust composition prediction capability across other Mg-based binary alloys,including Mg-La,Mg-Sn,Mg-Ce,Mg-Sm,Mg-Ag,and Mg-Y,thereby emphasizing its generalization and extrapolation potential.This research establishes a non-destructive,microstructure-informed composition analysis framework,reduces characterization time compared to traditional experiment methods and provides insights into the composition-microstructure relationship in diverse material systems.展开更多
Traffic flow prediction is of great importance in traffic planning,road resource management,and congestion mitigation.However,existing prediction have significant limitations in modeling multi-scale spatial-temporal f...Traffic flow prediction is of great importance in traffic planning,road resource management,and congestion mitigation.However,existing prediction have significant limitations in modeling multi-scale spatial-temporal features,particularly in capturing temporal periodicity and spatial dependency in dynamically evolving traffic networks.This paper proposes a novel framework of traffic flow prediction,referred to as Adaptive Graph Fusion Dual-scale Convolutional Network(AGFDCN),which integrates spatial-temporal dynamic graphs with dual-scale convolutional networks.Specifically,we introduce a Dual-Scale Temporal Network,which combines long-and short-term dilated causal convolutions with a temporal decay-aware attention mechanism to efficiently capture traffic patterns across multiple temporal scales.Furthermore,we design a Dynamic Adaptive Graph Module,which models complex spatial dependencies in traffic networks through an adaptive graph fusion mechanism and a dual-path attention-gated module.Finally,the temporal and spatial representations are integrated by employing a gated fusion mechanism,enhancing the overall prediction performance.Experimental results obtained based on three highway datasets(i.e.,PEMS04,PEMS07 and PEMS08)verify that the proposed model outperforms several state-of-the-art baselines in various evaluation metrics.Compared to the spatial-temporal graph model AGCRN with best performance in the baseline models,the proposed model exhibits significant improvements across all datasets:it achieves reduces of MAE by 42.07%and RMSE by 35.43%on PEMS04;MAE by 28.35%and RMSE by 29.28%on PEMS07;and MAE by 30.52%and RMSE by 30.73%on PEMS08,respectively,validating its effectiveness in modeling complex spatial-temporal traffic data and its robustness in handling sudden traffic changes.展开更多
In doped two-dimensional nanomaterials,magnetism is one of the important physical properties.By introducing foreign doping atoms or molecules,the electronic structure of the material can be effectively regulated,leadi...In doped two-dimensional nanomaterials,magnetism is one of the important physical properties.By introducing foreign doping atoms or molecules,the electronic structure of the material can be effectively regulated,leading to changes in magnetic behavior.Currently,magnetic property prediction has achieved considerable results with the help of traditional CNNs,but there are still obvious limitations:(1)The feature extraction of dopant sites is constrained by fixed receptive fields,making it difficult to characterize local structural perturbations in the vicinity of dopant atoms and their spatial influence propagating to surrounding regions;(2)CNNs lack the capability to model long-range dependencies between non-neighboring atoms and their chemical bonds,thereby weakening the representation of long-range interactions within the material.In this study,we propose Multi-Scale and Attention ConvNeXt(MSA-ConvNeXt)based on multi-scale convolution and attention mechanisms,which consists of the following two core modules:(1)The Multi-scale Convolution Attention Block(MCAB),which models local structural perturbations around dopant atoms and their spatial effects via parallelmulti-scale convolutions.It uses a serial channel and spatial attention mechanism to adaptively recalibrate multi-scale features,highlighting the response of doping related regions and enhancing the ability to express dopant-site information;(2)The Visual Geometry Group–Swin Transformer(VGG-Swin)architecture extracts structural features of dopant sites using VGG convolutions to prevent the attenuation of structural information during global relationship modeling.Subsequently,the Swin Transformer is introduced,which uses the self-attention mechanism to dynamically weight and globally associate features at different spatial locations,in order to depict the long-range correlations between non-neighboring atoms and their chemical bonds with the dopant-site.Experiments conducted on a doped two-dimensional nanomaterial dataset constructed from the CMR database demonstrate that the proposed model outperforms existing methods in terms of accuracy and F1-score.Specifically,MSA-ConvNeXt achieves an accuracy of 91.66%,representing an improvement of 1.65%over the next best model.In addition,all experimental results are averaged over multiple independent runs(with five different random seeds),demonstrating the stability and reliability of themodel’s performance.Ablation studies further validate the effectiveness of each module design.展开更多
Black ice formation on road surfaces presents a serious hazard due to its low visibility and high slipperiness,underscoring the critical need for timely and accurate detection in intelligent transportation systems.In ...Black ice formation on road surfaces presents a serious hazard due to its low visibility and high slipperiness,underscoring the critical need for timely and accurate detection in intelligent transportation systems.In this paper,we propose AdaMsDCNet,an adaptive multi-scale dilated convolution network designed for real-time black-ice semantic segmentation on resource-constrained edge platforms,applying a Convolutional Neural Network(CNN)with an adaptive Multi-Scale Dilated Convolution(MsDC)feature fusion encoder-decoder architecture.The key concept of AdaMsDCNet is to employ an encoder-decoder architecture with parallel multi-scale dilated convolutional paths that adjust dilation rates at different encoder depths using a systematic 4→2→1 progression,optimally capturing a wide range of receptive fields while mitigating checkerboard artifacts.The encoder dynamically fuses features from multiple dilation rates at each stage,enhancing segmentation accuracy.Simultaneously,the decoder uses transposed convolutions and skip connections to preserve fine spatial details.Experimental validation on a proprietary thermal infrared dataset of 1156 annotated images show that AdaMsDCNet_9 achieves 96.47%mIoU,95.48%Black-Ice IoU,97.55%Precision,97.82%Recall,and 97.69%F1-Score,outperforming U-Net(+26.78 pp mIoU,+29.88 pp Recall),DeepLabv3+(+2.82 pp mIoU),and LinkNet(+1.08 pp mIoU)while requiring only 1.86M parameters and maintaining real-time inference speeds of 3.94~5.63 FPS on the NVIDIA Jetson Nano embedded GPU.Ablation studies confirm the benefits of adaptive dilation,parallel feature fusion,and controlled channel growth for the accuracy–efficiency trade-off.Limitations including dataset generalization to uncontrolled outdoor conditions and the evaluation of imbalance-aware loss functions are identified as directions for future work.展开更多
In recent years,gait-based emotion recognition has been widely applied in the field of computer vision.However,existing gait emotion recognition methods typically rely on complete human skeleton data,and their accurac...In recent years,gait-based emotion recognition has been widely applied in the field of computer vision.However,existing gait emotion recognition methods typically rely on complete human skeleton data,and their accuracy significantly declines when the data is occluded.To enhance the accuracy of gait emotion recognition under occlusion,this paper proposes a Multi-scale Suppression Graph ConvolutionalNetwork(MS-GCN).TheMS-GCN consists of three main components:Joint Interpolation Module(JI Moudle),Multi-scale Temporal Convolution Network(MS-TCN),and Suppression Graph Convolutional Network(SGCN).The JI Module completes the spatially occluded skeletal joints using the(K-Nearest Neighbors)KNN interpolation method.The MS-TCN employs convolutional kernels of various sizes to comprehensively capture the emotional information embedded in the gait,compensating for the temporal occlusion of gait information.The SGCN extracts more non-prominent human gait features by suppressing the extraction of key body part features,thereby reducing the negative impact of occlusion on emotion recognition results.The proposed method is evaluated on two comprehensive datasets:Emotion-Gait,containing 4227 real gaits from sources like BML,ICT-Pollick,and ELMD,and 1000 synthetic gaits generated using STEP-Gen technology,and ELMB,consisting of 3924 gaits,with 1835 labeled with emotions such as“Happy,”“Sad,”“Angry,”and“Neutral.”On the standard datasets Emotion-Gait and ELMB,the proposed method achieved accuracies of 0.900 and 0.896,respectively,attaining performance comparable to other state-ofthe-artmethods.Furthermore,on occlusion datasets,the proposedmethod significantly mitigates the performance degradation caused by occlusion compared to other methods,the accuracy is significantly higher than that of other methods.展开更多
Robust motion similarity retrieval from monocular 2D pose sequences is challenged by body-scale variation,viewpoint inconsistency,translation drift,and temporal misalignment.Existing contrastive skeleton learning meth...Robust motion similarity retrieval from monocular 2D pose sequences is challenged by body-scale variation,viewpoint inconsistency,translation drift,and temporal misalignment.Existing contrastive skeleton learning methods primarily address action recognition and rarely integrate explicit geometric canonicalization for retrievaloriented metric learning.This paper proposes a spatial-temporal normalized contrastive embedding framework that unifies structured nuisance suppression with scalable similarity representation learning.A four-stage normalization pipeline—torso-scale normalization,pelvis-centered alignment,posture-axis alignment,and phase-synchronized temporal resampling—removes geometric and temporal distortions prior to embedding.The normalized sequences are encoded using an acausal dilated temporal convolutional network trained with a hybrid contrastive objective combining NT-Xent and semi-hard triplet loss,enabling both global separation and fine-grained stylistic discrimination.A prototype-based representation further supports interpretable amateur-to-professional style mapping.Experiments on a golf swing benchmark achieve a Top-1 accuracy of 91.3%,outperforming BiLSTM and Dynamic Time Warping baselines.The framework establishes an invariant and interpretable paradigmfor motion similarity retrieval applicable to broader human movement analysis tasks.展开更多
Although the Convolutional Neural Network(CNN)has shown great potential for land cover classification,the frequently used single-scale convolution kernel limits the scope of informa-tion extraction.Therefore,we propos...Although the Convolutional Neural Network(CNN)has shown great potential for land cover classification,the frequently used single-scale convolution kernel limits the scope of informa-tion extraction.Therefore,we propose a Multi-Scale Fully Convolutional Network(MSFCN)with a multi-scale convolutional kernel as well as a Channel Attention Block(CAB)and a Global Pooling Module(GPM)in this paper to exploit discriminative representations from two-dimensional(2D)satellite images.Meanwhile,to explore the ability of the proposed MSFCN for spatio-temporal images,we expand our MSFCN to three-dimension using three-dimensional(3D)CNN,capable of harnessing each land cover category’s time series interac-tion from the reshaped spatio-temporal remote sensing images.To verify the effectiveness of the proposed MSFCN,we conduct experiments on two spatial datasets and two spatio-temporal datasets.The proposed MSFCN achieves 60.366%on the WHDLD dataset and 75.127%on the GID dataset in terms of mIoU index while the figures for two spatio-temporal datasets are 87.753%and 77.156%.Extensive comparative experiments and abla-tion studies demonstrate the effectiveness of the proposed MSFCN.展开更多
Aiming at the difficulty of fault identification caused by manual extraction of fault features of rotating machinery,a one-dimensional multi-scale convolutional auto-encoder fault diagnosis model is proposed,based on ...Aiming at the difficulty of fault identification caused by manual extraction of fault features of rotating machinery,a one-dimensional multi-scale convolutional auto-encoder fault diagnosis model is proposed,based on the standard convolutional auto-encoder.In this model,the parallel convolutional and deconvolutional kernels of different scales are used to extract the features from the input signal and reconstruct the input signal;then the feature map extracted by multi-scale convolutional kernels is used as the input of the classifier;and finally the parameters of the whole model are fine-tuned using labeled data.Experiments on one set of simulation fault data and two sets of rolling bearing fault data are conducted to validate the proposed method.The results show that the model can achieve 99.75%,99.3%and 100%diagnostic accuracy,respectively.In addition,the diagnostic accuracy and reconstruction error of the one-dimensional multi-scale convolutional auto-encoder are compared with traditional machine learning,convolutional neural networks and a traditional convolutional auto-encoder.The final results show that the proposed model has a better recognition effect for rolling bearing fault data.展开更多
Fall behavior is closely related to high mortality in the elderly,so fall detection becomes an important and urgent research area.However,the existing fall detection methods are difficult to be applied in daily life d...Fall behavior is closely related to high mortality in the elderly,so fall detection becomes an important and urgent research area.However,the existing fall detection methods are difficult to be applied in daily life due to a large amount of calculation and poor detection accuracy.To solve the above problems,this paper proposes a dense spatial-temporal graph convolutional network based on lightweight OpenPose.Lightweight OpenPose uses MobileNet as a feature extraction network,and the prediction layer uses bottleneck-asymmetric structure,thus reducing the amount of the network.The bottleneck-asymmetrical structure compresses the number of input channels of feature maps by 1×1 convolution and replaces the 7×7 convolution structure with the asymmetric structure of 1×7 convolution,7×1 convolution,and 7×7 convolution in parallel.The spatial-temporal graph convolutional network divides the multi-layer convolution into dense blocks,and the convolutional layers in each dense block are connected,thus improving the feature transitivity,enhancing the network’s ability to extract features,thus improving the detection accuracy.Two representative datasets,Multiple Cameras Fall dataset(MCF),and Nanyang Technological University Red Green Blue+Depth Action Recognition dataset(NTU RGB+D),are selected for our experiments,among which NTU RGB+D has two evaluation benchmarks.The results show that the proposed model is superior to the current fall detection models.The accuracy of this network on the MCF dataset is 96.3%,and the accuracies on the two evaluation benchmarks of the NTU RGB+D dataset are 85.6%and 93.5%,respectively.展开更多
The prediction for Multivariate Time Series(MTS)explores the interrelationships among variables at historical moments,extracts their relevant characteristics,and is widely used in finance,weather,complex industries an...The prediction for Multivariate Time Series(MTS)explores the interrelationships among variables at historical moments,extracts their relevant characteristics,and is widely used in finance,weather,complex industries and other fields.Furthermore,it is important to construct a digital twin system.However,existing methods do not take full advantage of the potential properties of variables,which results in poor predicted accuracy.In this paper,we propose the Adaptive Fused Spatial-Temporal Graph Convolutional Network(AFSTGCN).First,to address the problem of the unknown spatial-temporal structure,we construct the Adaptive Fused Spatial-Temporal Graph(AFSTG)layer.Specifically,we fuse the spatial-temporal graph based on the interrelationship of spatial graphs.Simultaneously,we construct the adaptive adjacency matrix of the spatial-temporal graph using node embedding methods.Subsequently,to overcome the insufficient extraction of disordered correlation features,we construct the Adaptive Fused Spatial-Temporal Graph Convolutional(AFSTGC)module.The module forces the reordering of disordered temporal,spatial and spatial-temporal dependencies into rule-like data.AFSTGCN dynamically and synchronously acquires potential temporal,spatial and spatial-temporal correlations,thereby fully extracting rich hierarchical feature information to enhance the predicted accuracy.Experiments on different types of MTS datasets demonstrate that the model achieves state-of-the-art single-step and multi-step performance compared with eight other deep learning models.展开更多
Pedestrian attribute classification from a pedestrian image captured in surveillance scenarios is challenging due to diverse clothing appearances,varied poses and different camera views. A multiscale and multi-label c...Pedestrian attribute classification from a pedestrian image captured in surveillance scenarios is challenging due to diverse clothing appearances,varied poses and different camera views. A multiscale and multi-label convolutional neural network( MSMLCNN) is proposed to predict multiple pedestrian attributes simultaneously. The pedestrian attribute classification problem is firstly transformed into a multi-label problem including multiple binary attributes needed to be classified. Then,the multi-label problem is solved by fully connecting all binary attributes to multi-scale features with logistic regression functions. Moreover,the multi-scale features are obtained by concatenating those featured maps produced from multiple pooling layers of the MSMLCNN at different scales. Extensive experiment results show that the proposed MSMLCNN outperforms state-of-the-art pedestrian attribute classification methods with a large margin.展开更多
Aiming at the problems of low target pixels and intricate background in small target detection in infrared scenes,a target detection model based on multi-scale feature extraction with YOLOv8 was proposed.Firstly,all d...Aiming at the problems of low target pixels and intricate background in small target detection in infrared scenes,a target detection model based on multi-scale feature extraction with YOLOv8 was proposed.Firstly,all downsampling convolutions in the network were replaced with the Haar wavelet downsampling(HWD)module to better preserve fine-grained details in infrared imagery during downsampling.Secondly,the spatial pyramid pooling-fast(SPPF)module was improved by introducing separable convolutions,which expanded the receptive field in both horizontal and vertical directions,enabling more comprehensive spatial information capture.Furthermore,a novel C2f_CDWR module was designed using dilated convolutions with varying dilation rates to achieve adaptive feature extraction across multiple receptive fields,thus enhancing detection performance for objects of different sizes.Finally,to improve localization accuracy,the original CIoU loss in YOLOv8 was replaced with Inner-SIoU,which effectively improved bounding box regression accuracy and significantly boosted the model’s capability in detecting small infrared targets.The experimental evaluation on the HIT-UAV dataset shows that the precision of the enhanced YOLOv8 model is 90.5%,the recall rate is 75.9%,and the mean average precision is 85.7%.In terms of infrared target detection,its performance was significantly better than that of the baseline YOLOv8 model and other benchmark models.展开更多
As an integrated application of modern information technologies and artificial intelligence,Prognostic and Health Management(PHM)is important for machine health monitoring.Prediction of tool wear is one of the symboli...As an integrated application of modern information technologies and artificial intelligence,Prognostic and Health Management(PHM)is important for machine health monitoring.Prediction of tool wear is one of the symbolic applications of PHM technology in modern manufacturing systems and industry.In this paper,a multi-scale Convolutional Gated Recurrent Unit network(MCGRU)is proposed to address raw sensory data for tool wear prediction.At the bottom of MCGRU,six parallel and independent branches with different kernel sizes are designed to form a multi-scale convolutional neural network,which augments the adaptability to features of different time scales.These features of different scales extracted from raw data are then fed into a Deep Gated Recurrent Unit network to capture long-term dependencies and learn significant representations.At the top of the MCGRU,a fully connected layer and a regression layer are built for cutting tool wear prediction.Two case studies are performed to verify the capability and effectiveness of the proposed MCGRU network and results show that MCGRU outperforms several state-of-the-art baseline models.展开更多
Background The use of remote photoplethysmography(rPPG)to estimate blood volume pulse in a noncontact manner has been an active research topic in recent years.Existing methods are primarily based on a singlescale regi...Background The use of remote photoplethysmography(rPPG)to estimate blood volume pulse in a noncontact manner has been an active research topic in recent years.Existing methods are primarily based on a singlescale region of interest(ROI).However,some noise signals that are not easily separated in a single-scale space can be easily separated in a multi-scale space.Also,existing spatiotemporal networks mainly focus on local spatiotemporal information and do not emphasize temporal information,which is crucial in pulse extraction problems,resulting in insufficient spatiotemporal feature modelling.Methods Here,we propose a multi-scale facial video pulse extraction network based on separable spatiotemporal convolution(SSTC)and dimension separable attention(DSAT).First,to solve the problem of a single-scale ROI,we constructed a multi-scale feature space for initial signal separation.Second,SSTC and DSAT were designed for efficient spatiotemporal correlation modeling,which increased the information interaction between the long-span time and space dimensions;this placed more emphasis on temporal features.Results The signal-to-noise ratio(SNR)of the proposed network reached 9.58dB on the PURE dataset and 6.77dB on the UBFC-rPPG dataset,outperforming state-of-the-art algorithms.Conclusions The results showed that fusing multi-scale signals yielded better results than methods based on only single-scale signals.The proposed SSTC and dimension-separable attention mechanism will contribute to more accurate pulse signal extraction.展开更多
Convolutional neural networks (CNNs) are widely used in image classification tasks, but their increasing model size and computation make them challenging to implement on embedded systems with constrained hardware reso...Convolutional neural networks (CNNs) are widely used in image classification tasks, but their increasing model size and computation make them challenging to implement on embedded systems with constrained hardware resources. To address this issue, the MobileNetV1 network was developed, which employs depthwise convolution to reduce network complexity. MobileNetV1 employs a stride of 2 in several convolutional layers to decrease the spatial resolution of feature maps, thereby lowering computational costs. However, this stride setting can lead to a loss of spatial information, particularly affecting the detection and representation of smaller objects or finer details in images. To maintain the trade-off between complexity and model performance, a lightweight convolutional neural network with hierarchical multi-scale feature fusion based on the MobileNetV1 network is proposed. The network consists of two main subnetworks. The first subnetwork uses a depthwise dilated separable convolution (DDSC) layer to learn imaging features with fewer parameters, which results in a lightweight and computationally inexpensive network. Furthermore, depthwise dilated convolution in DDSC layer effectively expands the field of view of filters, allowing them to incorporate a larger context. The second subnetwork is a hierarchical multi-scale feature fusion (HMFF) module that uses parallel multi-resolution branches architecture to process the input feature map in order to extract the multi-scale feature information of the input image. Experimental results on the CIFAR-10, Malaria, and KvasirV1 datasets demonstrate that the proposed method is efficient, reducing the network parameters and computational cost by 65.02% and 39.78%, respectively, while maintaining the network performance compared to the MobileNetV1 baseline.展开更多
Convolutional neural networks(CNNs)-based medical image segmentation technologies have been widely used in medical image segmentation because of their strong representation and generalization abilities.However,due to ...Convolutional neural networks(CNNs)-based medical image segmentation technologies have been widely used in medical image segmentation because of their strong representation and generalization abilities.However,due to the inability to effectively capture global information from images,CNNs can easily lead to loss of contours and textures in segmentation results.Notice that the transformer model can effectively capture the properties of long-range dependencies in the image,and furthermore,combining the CNN and the transformer can effectively extract local details and global contextual features of the image.Motivated by this,we propose a multi-branch and multi-scale attention network(M2ANet)for medical image segmentation,whose architecture consists of three components.Specifically,in the first component,we construct an adaptive multi-branch patch module for parallel extraction of image features to reduce information loss caused by downsampling.In the second component,we apply residual block to the well-known convolutional block attention module to enhance the network’s ability to recognize important features of images and alleviate the phenomenon of gradient vanishing.In the third component,we design a multi-scale feature fusion module,in which we adopt adaptive average pooling and position encoding to enhance contextual features,and then multi-head attention is introduced to further enrich feature representation.Finally,we validate the effectiveness and feasibility of the proposed M2ANet method through comparative experiments on four benchmark medical image segmentation datasets,particularly in the context of preserving contours and textures.展开更多
Energy expenditure estimation can be used to measure the exercise load and physical condition of different individuals, such as soldiers, athletes, firemen, etc., during their training and work. Energy expenditure est...Energy expenditure estimation can be used to measure the exercise load and physical condition of different individuals, such as soldiers, athletes, firemen, etc., during their training and work. Energy expenditure estimation methods based on computer vision have rapidly developed in recent years. Compared with sensor-based methods, such methods are capable of monitoring several target persons at the same time, and the subjects do not need to wear different sensor devices that hamper their movement. In this paper, we propose a cross-attention spatial–temporal convolutional neural network to predict the energy expenditure of people under different exercise intensities. The model explores the relationship between changes in the human skeleton and energy expenditure intensity. In addition, a cross-attention correction module is used to reduce the negative effects of individual physical fitness characteristics during energy expenditure estimation. The experimental results show that our proposed method achieves high accuracy for energy expenditure estimation and performs better than existing computer vision-based energy expenditure estimation methods do. The proposed method can be widely used in various physical activity scenarios to measure energy expenditure, increasing the convenience of usage.展开更多
Semantic segmentation plays a foundational role in biomedical image analysis, providing precise information about cellular, tissue, and organ structures in both biological and medical imaging modalities. Traditional a...Semantic segmentation plays a foundational role in biomedical image analysis, providing precise information about cellular, tissue, and organ structures in both biological and medical imaging modalities. Traditional approaches often fail in the face of challenges such as low contrast, morphological variability, and densely packed structures. Recent advancements in deep learning have transformed segmentation capabilities through the integration of fine-scale detail preservation, coarse-scale contextual modeling, and multi-scale feature fusion. This work provides a comprehensive analysis of state-of-the-art deep learning models, including U-Net variants, attention-based frameworks, and Transformer-integrated networks, highlighting innovations that improve accuracy, generalizability, and computational efficiency. Key architectural components such as convolution operations, shallow and deep blocks, skip connections, and hybrid encoders are examined for their roles in enhancing spatial representation and semantic consistency. We further discuss the importance of hierarchical and instance-aware segmentation and annotation in interpreting complex biological scenes and multiplexed medical images. By bridging methodological developments with diverse application domains, this paper outlines current trends and future directions for semantic segmentation, emphasizing its critical role in facilitating annotation, diagnosis, and discovery in biomedical research.展开更多
基金supported by the National Natural Science Foundation of China(Grant Nos.62472149,62376089,62202147)Hubei Provincial Science and Technology Plan Project(2023BCB04100).
摘要Accurate traffic flow prediction has a profound impact on modern traffic management. Traffic flow has complex spatial-temporal correlations and periodicity, which poses difficulties for precise prediction. To address this problem, a Multi-head Self-attention and Spatial-Temporal Graph Convolutional Network (MSSTGCN) for multiscale traffic flow prediction is proposed. Firstly, to capture the hidden traffic periodicity of traffic flow, traffic flow is divided into three kinds of periods, including hourly, daily, and weekly data. Secondly, a graph attention residual layer is constructed to learn the global spatial features across regions. Local spatial-temporal dependence is captured by using a T-GCN module. Thirdly, a transformer layer is introduced to learn the long-term dependence in time. A position embedding mechanism is introduced to label position information for all traffic sequences. Thus, this multi-head self-attention mechanism can recognize the sequence order and allocate weights for different time nodes. Experimental results on four real-world datasets show that the MSSTGCN performs better than the baseline methods and can be successfully adapted to traffic prediction tasks.
基金supported,in part,by the National Nature Science Foundation of China under Grant 62272236,62376128in part,by the Natural Science Foundation of Jiangsu Province under Grant BK20201136,BK20191401.
摘要Video emotion recognition is widely used due to its alignment with the temporal characteristics of human emotional expression,but existingmodels have significant shortcomings.On the one hand,Transformermultihead self-attention modeling of global temporal dependency has problems of high computational overhead and feature similarity.On the other hand,fixed-size convolution kernels are often used,which have weak perception ability for emotional regions of different scales.Therefore,this paper proposes a video emotion recognition model that combines multi-scale region-aware convolution with temporal interactive sampling.In terms of space,multi-branch large-kernel stripe convolution is used to perceive emotional region features at different scales,and attention weights are generated for each scale feature.In terms of time,multi-layer odd-even down-sampling is performed on the time series,and oddeven sub-sequence interaction is performed to solve the problem of feature similarity,while reducing computational costs due to the linear relationship between sampling and convolution overhead.This paper was tested on CMU-MOSI,CMU-MOSEI,and Hume Reaction.The Acc-2 reached 83.4%,85.2%,and 81.2%,respectively.The experimental results show that the model can significantly improve the accuracy of emotion recognition.
基金funded by the National Natural Science Foundation of China(No.52204407)the Natural Science Foundation of Jiangsu Province(No.BK20220595)the China Postdoctoral Science Foundation(No.2022M723689).
摘要This study proposes a multi-scale simplified residual convolutional neural network(MS-SRCNN)for the precise prediction of Mg-Nd binary alloy compositions from scanning electron microscope(SEM)images.A multi-scale data structure is established by spatially aligning and stacking SEM images at different magnifications.The MS-SRCNN significantly reduces computational runtime by over 90%compared to traditional architectures like ResNet50,VGG16,and VGG19,without compromising prediction accuracy.The model demonstrates more excellent predictive performance,achieving a>5%increase in R2 compared to single-scale models.Furthermore,the MS-SRCNN exhibits robust composition prediction capability across other Mg-based binary alloys,including Mg-La,Mg-Sn,Mg-Ce,Mg-Sm,Mg-Ag,and Mg-Y,thereby emphasizing its generalization and extrapolation potential.This research establishes a non-destructive,microstructure-informed composition analysis framework,reduces characterization time compared to traditional experiment methods and provides insights into the composition-microstructure relationship in diverse material systems.
基金supported in part by the National Nature Science Foundation of China under Grants 62476216 and 62006184in part by the Key Research and Development Program of Shaanxi Province under Grant 2024GX-YBXM-146+1 种基金in part by the Scientific Research ProgramFunded by EducationDepartment of the Shaanxi Provincial Government under Grant 23JP091the Youth Innovation Team of Shaanxi Universities.
摘要Traffic flow prediction is of great importance in traffic planning,road resource management,and congestion mitigation.However,existing prediction have significant limitations in modeling multi-scale spatial-temporal features,particularly in capturing temporal periodicity and spatial dependency in dynamically evolving traffic networks.This paper proposes a novel framework of traffic flow prediction,referred to as Adaptive Graph Fusion Dual-scale Convolutional Network(AGFDCN),which integrates spatial-temporal dynamic graphs with dual-scale convolutional networks.Specifically,we introduce a Dual-Scale Temporal Network,which combines long-and short-term dilated causal convolutions with a temporal decay-aware attention mechanism to efficiently capture traffic patterns across multiple temporal scales.Furthermore,we design a Dynamic Adaptive Graph Module,which models complex spatial dependencies in traffic networks through an adaptive graph fusion mechanism and a dual-path attention-gated module.Finally,the temporal and spatial representations are integrated by employing a gated fusion mechanism,enhancing the overall prediction performance.Experimental results obtained based on three highway datasets(i.e.,PEMS04,PEMS07 and PEMS08)verify that the proposed model outperforms several state-of-the-art baselines in various evaluation metrics.Compared to the spatial-temporal graph model AGCRN with best performance in the baseline models,the proposed model exhibits significant improvements across all datasets:it achieves reduces of MAE by 42.07%and RMSE by 35.43%on PEMS04;MAE by 28.35%and RMSE by 29.28%on PEMS07;and MAE by 30.52%and RMSE by 30.73%on PEMS08,respectively,validating its effectiveness in modeling complex spatial-temporal traffic data and its robustness in handling sudden traffic changes.
基金supported by the Heilongjiang Provincial Discipline Innovation Project(No.LJGXCG2024-F10).
摘要In doped two-dimensional nanomaterials,magnetism is one of the important physical properties.By introducing foreign doping atoms or molecules,the electronic structure of the material can be effectively regulated,leading to changes in magnetic behavior.Currently,magnetic property prediction has achieved considerable results with the help of traditional CNNs,but there are still obvious limitations:(1)The feature extraction of dopant sites is constrained by fixed receptive fields,making it difficult to characterize local structural perturbations in the vicinity of dopant atoms and their spatial influence propagating to surrounding regions;(2)CNNs lack the capability to model long-range dependencies between non-neighboring atoms and their chemical bonds,thereby weakening the representation of long-range interactions within the material.In this study,we propose Multi-Scale and Attention ConvNeXt(MSA-ConvNeXt)based on multi-scale convolution and attention mechanisms,which consists of the following two core modules:(1)The Multi-scale Convolution Attention Block(MCAB),which models local structural perturbations around dopant atoms and their spatial effects via parallelmulti-scale convolutions.It uses a serial channel and spatial attention mechanism to adaptively recalibrate multi-scale features,highlighting the response of doping related regions and enhancing the ability to express dopant-site information;(2)The Visual Geometry Group–Swin Transformer(VGG-Swin)architecture extracts structural features of dopant sites using VGG convolutions to prevent the attenuation of structural information during global relationship modeling.Subsequently,the Swin Transformer is introduced,which uses the self-attention mechanism to dynamically weight and globally associate features at different spatial locations,in order to depict the long-range correlations between non-neighboring atoms and their chemical bonds with the dopant-site.Experiments conducted on a doped two-dimensional nanomaterial dataset constructed from the CMR database demonstrate that the proposed model outperforms existing methods in terms of accuracy and F1-score.Specifically,MSA-ConvNeXt achieves an accuracy of 91.66%,representing an improvement of 1.65%over the next best model.In addition,all experimental results are averaged over multiple independent runs(with five different random seeds),demonstrating the stability and reliability of themodel’s performance.Ablation studies further validate the effectiveness of each module design.
基金funded by the Institute of Information&Communications Technology Planning&Evaluation(IITP)-Innovative Human Resource Development for Local Intellectualization Program.
摘要Black ice formation on road surfaces presents a serious hazard due to its low visibility and high slipperiness,underscoring the critical need for timely and accurate detection in intelligent transportation systems.In this paper,we propose AdaMsDCNet,an adaptive multi-scale dilated convolution network designed for real-time black-ice semantic segmentation on resource-constrained edge platforms,applying a Convolutional Neural Network(CNN)with an adaptive Multi-Scale Dilated Convolution(MsDC)feature fusion encoder-decoder architecture.The key concept of AdaMsDCNet is to employ an encoder-decoder architecture with parallel multi-scale dilated convolutional paths that adjust dilation rates at different encoder depths using a systematic 4→2→1 progression,optimally capturing a wide range of receptive fields while mitigating checkerboard artifacts.The encoder dynamically fuses features from multiple dilation rates at each stage,enhancing segmentation accuracy.Simultaneously,the decoder uses transposed convolutions and skip connections to preserve fine spatial details.Experimental validation on a proprietary thermal infrared dataset of 1156 annotated images show that AdaMsDCNet_9 achieves 96.47%mIoU,95.48%Black-Ice IoU,97.55%Precision,97.82%Recall,and 97.69%F1-Score,outperforming U-Net(+26.78 pp mIoU,+29.88 pp Recall),DeepLabv3+(+2.82 pp mIoU),and LinkNet(+1.08 pp mIoU)while requiring only 1.86M parameters and maintaining real-time inference speeds of 3.94~5.63 FPS on the NVIDIA Jetson Nano embedded GPU.Ablation studies confirm the benefits of adaptive dilation,parallel feature fusion,and controlled channel growth for the accuracy–efficiency trade-off.Limitations including dataset generalization to uncontrolled outdoor conditions and the evaluation of imbalance-aware loss functions are identified as directions for future work.
基金supported by the National Natural Science Foundation of China(62272049,62236006,62172045)the Key Projects of Beijing Union University(ZKZD202301).
摘要In recent years,gait-based emotion recognition has been widely applied in the field of computer vision.However,existing gait emotion recognition methods typically rely on complete human skeleton data,and their accuracy significantly declines when the data is occluded.To enhance the accuracy of gait emotion recognition under occlusion,this paper proposes a Multi-scale Suppression Graph ConvolutionalNetwork(MS-GCN).TheMS-GCN consists of three main components:Joint Interpolation Module(JI Moudle),Multi-scale Temporal Convolution Network(MS-TCN),and Suppression Graph Convolutional Network(SGCN).The JI Module completes the spatially occluded skeletal joints using the(K-Nearest Neighbors)KNN interpolation method.The MS-TCN employs convolutional kernels of various sizes to comprehensively capture the emotional information embedded in the gait,compensating for the temporal occlusion of gait information.The SGCN extracts more non-prominent human gait features by suppressing the extraction of key body part features,thereby reducing the negative impact of occlusion on emotion recognition results.The proposed method is evaluated on two comprehensive datasets:Emotion-Gait,containing 4227 real gaits from sources like BML,ICT-Pollick,and ELMD,and 1000 synthetic gaits generated using STEP-Gen technology,and ELMB,consisting of 3924 gaits,with 1835 labeled with emotions such as“Happy,”“Sad,”“Angry,”and“Neutral.”On the standard datasets Emotion-Gait and ELMB,the proposed method achieved accuracies of 0.900 and 0.896,respectively,attaining performance comparable to other state-ofthe-artmethods.Furthermore,on occlusion datasets,the proposedmethod significantly mitigates the performance degradation caused by occlusion compared to other methods,the accuracy is significantly higher than that of other methods.
基金supported by Incheon National University Research Grant(2020).
摘要Robust motion similarity retrieval from monocular 2D pose sequences is challenged by body-scale variation,viewpoint inconsistency,translation drift,and temporal misalignment.Existing contrastive skeleton learning methods primarily address action recognition and rarely integrate explicit geometric canonicalization for retrievaloriented metric learning.This paper proposes a spatial-temporal normalized contrastive embedding framework that unifies structured nuisance suppression with scalable similarity representation learning.A four-stage normalization pipeline—torso-scale normalization,pelvis-centered alignment,posture-axis alignment,and phase-synchronized temporal resampling—removes geometric and temporal distortions prior to embedding.The normalized sequences are encoded using an acausal dilated temporal convolutional network trained with a hybrid contrastive objective combining NT-Xent and semi-hard triplet loss,enabling both global separation and fine-grained stylistic discrimination.A prototype-based representation further supports interpretable amateur-to-professional style mapping.Experiments on a golf swing benchmark achieve a Top-1 accuracy of 91.3%,outperforming BiLSTM and Dynamic Time Warping baselines.The framework establishes an invariant and interpretable paradigmfor motion similarity retrieval applicable to broader human movement analysis tasks.
基金supported by the National Natural Science Foundation of China[grant number 41671452].
摘要Although the Convolutional Neural Network(CNN)has shown great potential for land cover classification,the frequently used single-scale convolution kernel limits the scope of informa-tion extraction.Therefore,we propose a Multi-Scale Fully Convolutional Network(MSFCN)with a multi-scale convolutional kernel as well as a Channel Attention Block(CAB)and a Global Pooling Module(GPM)in this paper to exploit discriminative representations from two-dimensional(2D)satellite images.Meanwhile,to explore the ability of the proposed MSFCN for spatio-temporal images,we expand our MSFCN to three-dimension using three-dimensional(3D)CNN,capable of harnessing each land cover category’s time series interac-tion from the reshaped spatio-temporal remote sensing images.To verify the effectiveness of the proposed MSFCN,we conduct experiments on two spatial datasets and two spatio-temporal datasets.The proposed MSFCN achieves 60.366%on the WHDLD dataset and 75.127%on the GID dataset in terms of mIoU index while the figures for two spatio-temporal datasets are 87.753%and 77.156%.Extensive comparative experiments and abla-tion studies demonstrate the effectiveness of the proposed MSFCN.
基金The National Natural Science Foundation of China(No.51675098)
摘要Aiming at the difficulty of fault identification caused by manual extraction of fault features of rotating machinery,a one-dimensional multi-scale convolutional auto-encoder fault diagnosis model is proposed,based on the standard convolutional auto-encoder.In this model,the parallel convolutional and deconvolutional kernels of different scales are used to extract the features from the input signal and reconstruct the input signal;then the feature map extracted by multi-scale convolutional kernels is used as the input of the classifier;and finally the parameters of the whole model are fine-tuned using labeled data.Experiments on one set of simulation fault data and two sets of rolling bearing fault data are conducted to validate the proposed method.The results show that the model can achieve 99.75%,99.3%and 100%diagnostic accuracy,respectively.In addition,the diagnostic accuracy and reconstruction error of the one-dimensional multi-scale convolutional auto-encoder are compared with traditional machine learning,convolutional neural networks and a traditional convolutional auto-encoder.The final results show that the proposed model has a better recognition effect for rolling bearing fault data.
基金supported,in part,by the National Nature Science Foundation of China under Grant Numbers 62272236,62376128in part,by the Natural Science Foundation of Jiangsu Province under Grant Numbers BK20201136,BK20191401.
摘要Fall behavior is closely related to high mortality in the elderly,so fall detection becomes an important and urgent research area.However,the existing fall detection methods are difficult to be applied in daily life due to a large amount of calculation and poor detection accuracy.To solve the above problems,this paper proposes a dense spatial-temporal graph convolutional network based on lightweight OpenPose.Lightweight OpenPose uses MobileNet as a feature extraction network,and the prediction layer uses bottleneck-asymmetric structure,thus reducing the amount of the network.The bottleneck-asymmetrical structure compresses the number of input channels of feature maps by 1×1 convolution and replaces the 7×7 convolution structure with the asymmetric structure of 1×7 convolution,7×1 convolution,and 7×7 convolution in parallel.The spatial-temporal graph convolutional network divides the multi-layer convolution into dense blocks,and the convolutional layers in each dense block are connected,thus improving the feature transitivity,enhancing the network’s ability to extract features,thus improving the detection accuracy.Two representative datasets,Multiple Cameras Fall dataset(MCF),and Nanyang Technological University Red Green Blue+Depth Action Recognition dataset(NTU RGB+D),are selected for our experiments,among which NTU RGB+D has two evaluation benchmarks.The results show that the proposed model is superior to the current fall detection models.The accuracy of this network on the MCF dataset is 96.3%,and the accuracies on the two evaluation benchmarks of the NTU RGB+D dataset are 85.6%and 93.5%,respectively.
基金supported by the China Scholarship Council and the CERNET Innovation Project under grant No.20170111.
摘要The prediction for Multivariate Time Series(MTS)explores the interrelationships among variables at historical moments,extracts their relevant characteristics,and is widely used in finance,weather,complex industries and other fields.Furthermore,it is important to construct a digital twin system.However,existing methods do not take full advantage of the potential properties of variables,which results in poor predicted accuracy.In this paper,we propose the Adaptive Fused Spatial-Temporal Graph Convolutional Network(AFSTGCN).First,to address the problem of the unknown spatial-temporal structure,we construct the Adaptive Fused Spatial-Temporal Graph(AFSTG)layer.Specifically,we fuse the spatial-temporal graph based on the interrelationship of spatial graphs.Simultaneously,we construct the adaptive adjacency matrix of the spatial-temporal graph using node embedding methods.Subsequently,to overcome the insufficient extraction of disordered correlation features,we construct the Adaptive Fused Spatial-Temporal Graph Convolutional(AFSTGC)module.The module forces the reordering of disordered temporal,spatial and spatial-temporal dependencies into rule-like data.AFSTGCN dynamically and synchronously acquires potential temporal,spatial and spatial-temporal correlations,thereby fully extracting rich hierarchical feature information to enhance the predicted accuracy.Experiments on different types of MTS datasets demonstrate that the model achieves state-of-the-art single-step and multi-step performance compared with eight other deep learning models.
基金Supported by the National Natural Science Foundation of China(No.61602191,61672521,61375037,61473291,61572501,61572536,61502491,61372107,61401167)the Natural Science Foundation of Fujian Province(No.2016J01308)+3 种基金the Scientific and Technology Funds of Quanzhou(No.2015Z114)the Scientific and Technology Funds of Xiamen(No.3502Z20173045)the Promotion Program for Young and Middle aged Teacher in Science and Technology Research of Huaqiao University(No.ZQN-PY418,ZQN-YX403)the Scientific Research Funds of Huaqiao University(No.16BS108)
摘要Pedestrian attribute classification from a pedestrian image captured in surveillance scenarios is challenging due to diverse clothing appearances,varied poses and different camera views. A multiscale and multi-label convolutional neural network( MSMLCNN) is proposed to predict multiple pedestrian attributes simultaneously. The pedestrian attribute classification problem is firstly transformed into a multi-label problem including multiple binary attributes needed to be classified. Then,the multi-label problem is solved by fully connecting all binary attributes to multi-scale features with logistic regression functions. Moreover,the multi-scale features are obtained by concatenating those featured maps produced from multiple pooling layers of the MSMLCNN at different scales. Extensive experiment results show that the proposed MSMLCNN outperforms state-of-the-art pedestrian attribute classification methods with a large margin.
基金supported by the National Natural Science Foundation of China(No.62361034)。
摘要Aiming at the problems of low target pixels and intricate background in small target detection in infrared scenes,a target detection model based on multi-scale feature extraction with YOLOv8 was proposed.Firstly,all downsampling convolutions in the network were replaced with the Haar wavelet downsampling(HWD)module to better preserve fine-grained details in infrared imagery during downsampling.Secondly,the spatial pyramid pooling-fast(SPPF)module was improved by introducing separable convolutions,which expanded the receptive field in both horizontal and vertical directions,enabling more comprehensive spatial information capture.Furthermore,a novel C2f_CDWR module was designed using dilated convolutions with varying dilation rates to achieve adaptive feature extraction across multiple receptive fields,thus enhancing detection performance for objects of different sizes.Finally,to improve localization accuracy,the original CIoU loss in YOLOv8 was replaced with Inner-SIoU,which effectively improved bounding box regression accuracy and significantly boosted the model’s capability in detecting small infrared targets.The experimental evaluation on the HIT-UAV dataset shows that the precision of the enhanced YOLOv8 model is 90.5%,the recall rate is 75.9%,and the mean average precision is 85.7%.In terms of infrared target detection,its performance was significantly better than that of the baseline YOLOv8 model and other benchmark models.
基金Supported in part by Natural Science Foundation of China(Grant Nos.51835009,51705398)Shaanxi Province 2020 Natural Science Basic Research Plan(Grant No.2020JQ-042)Aeronautical Science Foundation(Grant No.2019ZB070001).
摘要As an integrated application of modern information technologies and artificial intelligence,Prognostic and Health Management(PHM)is important for machine health monitoring.Prediction of tool wear is one of the symbolic applications of PHM technology in modern manufacturing systems and industry.In this paper,a multi-scale Convolutional Gated Recurrent Unit network(MCGRU)is proposed to address raw sensory data for tool wear prediction.At the bottom of MCGRU,six parallel and independent branches with different kernel sizes are designed to form a multi-scale convolutional neural network,which augments the adaptability to features of different time scales.These features of different scales extracted from raw data are then fed into a Deep Gated Recurrent Unit network to capture long-term dependencies and learn significant representations.At the top of the MCGRU,a fully connected layer and a regression layer are built for cutting tool wear prediction.Two case studies are performed to verify the capability and effectiveness of the proposed MCGRU network and results show that MCGRU outperforms several state-of-the-art baseline models.
基金Supported by the National Natural Science Foundation of China(61903336,61976190)the Natural Science Foundation of Zhejiang Province(LY21F030015)。
摘要Background The use of remote photoplethysmography(rPPG)to estimate blood volume pulse in a noncontact manner has been an active research topic in recent years.Existing methods are primarily based on a singlescale region of interest(ROI).However,some noise signals that are not easily separated in a single-scale space can be easily separated in a multi-scale space.Also,existing spatiotemporal networks mainly focus on local spatiotemporal information and do not emphasize temporal information,which is crucial in pulse extraction problems,resulting in insufficient spatiotemporal feature modelling.Methods Here,we propose a multi-scale facial video pulse extraction network based on separable spatiotemporal convolution(SSTC)and dimension separable attention(DSAT).First,to solve the problem of a single-scale ROI,we constructed a multi-scale feature space for initial signal separation.Second,SSTC and DSAT were designed for efficient spatiotemporal correlation modeling,which increased the information interaction between the long-span time and space dimensions;this placed more emphasis on temporal features.Results The signal-to-noise ratio(SNR)of the proposed network reached 9.58dB on the PURE dataset and 6.77dB on the UBFC-rPPG dataset,outperforming state-of-the-art algorithms.Conclusions The results showed that fusing multi-scale signals yielded better results than methods based on only single-scale signals.The proposed SSTC and dimension-separable attention mechanism will contribute to more accurate pulse signal extraction.
摘要Convolutional neural networks (CNNs) are widely used in image classification tasks, but their increasing model size and computation make them challenging to implement on embedded systems with constrained hardware resources. To address this issue, the MobileNetV1 network was developed, which employs depthwise convolution to reduce network complexity. MobileNetV1 employs a stride of 2 in several convolutional layers to decrease the spatial resolution of feature maps, thereby lowering computational costs. However, this stride setting can lead to a loss of spatial information, particularly affecting the detection and representation of smaller objects or finer details in images. To maintain the trade-off between complexity and model performance, a lightweight convolutional neural network with hierarchical multi-scale feature fusion based on the MobileNetV1 network is proposed. The network consists of two main subnetworks. The first subnetwork uses a depthwise dilated separable convolution (DDSC) layer to learn imaging features with fewer parameters, which results in a lightweight and computationally inexpensive network. Furthermore, depthwise dilated convolution in DDSC layer effectively expands the field of view of filters, allowing them to incorporate a larger context. The second subnetwork is a hierarchical multi-scale feature fusion (HMFF) module that uses parallel multi-resolution branches architecture to process the input feature map in order to extract the multi-scale feature information of the input image. Experimental results on the CIFAR-10, Malaria, and KvasirV1 datasets demonstrate that the proposed method is efficient, reducing the network parameters and computational cost by 65.02% and 39.78%, respectively, while maintaining the network performance compared to the MobileNetV1 baseline.
基金supported by the Natural Science Foundation of the Anhui Higher Education Institutions of China(Grant Nos.2023AH040149 and 2024AH051915)the Anhui Provincial Natural Science Foundation(Grant No.2208085MF168)+1 种基金the Science and Technology Innovation Tackle Plan Project of Maanshan(Grant No.2024RGZN001)the Scientific Research Fund Project of Anhui Medical University(Grant No.2023xkj122).
摘要Convolutional neural networks(CNNs)-based medical image segmentation technologies have been widely used in medical image segmentation because of their strong representation and generalization abilities.However,due to the inability to effectively capture global information from images,CNNs can easily lead to loss of contours and textures in segmentation results.Notice that the transformer model can effectively capture the properties of long-range dependencies in the image,and furthermore,combining the CNN and the transformer can effectively extract local details and global contextual features of the image.Motivated by this,we propose a multi-branch and multi-scale attention network(M2ANet)for medical image segmentation,whose architecture consists of three components.Specifically,in the first component,we construct an adaptive multi-branch patch module for parallel extraction of image features to reduce information loss caused by downsampling.In the second component,we apply residual block to the well-known convolutional block attention module to enhance the network’s ability to recognize important features of images and alleviate the phenomenon of gradient vanishing.In the third component,we design a multi-scale feature fusion module,in which we adopt adaptive average pooling and position encoding to enhance contextual features,and then multi-head attention is introduced to further enrich feature representation.Finally,we validate the effectiveness and feasibility of the proposed M2ANet method through comparative experiments on four benchmark medical image segmentation datasets,particularly in the context of preserving contours and textures.
基金supported by the National Key Research and Development Program of China (Grant No.2022YFC3600403)Emerging Interdisciplinary Platform for Medicine and Engineering in Sports (EIPMES),Beijing,China。
摘要Energy expenditure estimation can be used to measure the exercise load and physical condition of different individuals, such as soldiers, athletes, firemen, etc., during their training and work. Energy expenditure estimation methods based on computer vision have rapidly developed in recent years. Compared with sensor-based methods, such methods are capable of monitoring several target persons at the same time, and the subjects do not need to wear different sensor devices that hamper their movement. In this paper, we propose a cross-attention spatial–temporal convolutional neural network to predict the energy expenditure of people under different exercise intensities. The model explores the relationship between changes in the human skeleton and energy expenditure intensity. In addition, a cross-attention correction module is used to reduce the negative effects of individual physical fitness characteristics during energy expenditure estimation. The experimental results show that our proposed method achieves high accuracy for energy expenditure estimation and performs better than existing computer vision-based energy expenditure estimation methods do. The proposed method can be widely used in various physical activity scenarios to measure energy expenditure, increasing the convenience of usage.
基金Open Access funding provided by the National Institutes of Health(NIH)The funding for this project was provided by NCATS Intramural Fund.
摘要Semantic segmentation plays a foundational role in biomedical image analysis, providing precise information about cellular, tissue, and organ structures in both biological and medical imaging modalities. Traditional approaches often fail in the face of challenges such as low contrast, morphological variability, and densely packed structures. Recent advancements in deep learning have transformed segmentation capabilities through the integration of fine-scale detail preservation, coarse-scale contextual modeling, and multi-scale feature fusion. This work provides a comprehensive analysis of state-of-the-art deep learning models, including U-Net variants, attention-based frameworks, and Transformer-integrated networks, highlighting innovations that improve accuracy, generalizability, and computational efficiency. Key architectural components such as convolution operations, shallow and deep blocks, skip connections, and hybrid encoders are examined for their roles in enhancing spatial representation and semantic consistency. We further discuss the importance of hierarchical and instance-aware segmentation and annotation in interpreting complex biological scenes and multiplexed medical images. By bridging methodological developments with diverse application domains, this paper outlines current trends and future directions for semantic segmentation, emphasizing its critical role in facilitating annotation, diagnosis, and discovery in biomedical research.