Recently,large-scale deep learning models have been increasingly adopted for point cloud classification.However,thesemethods typically require collecting extensive datasets frommultiple clients,which may lead to priva...Recently,large-scale deep learning models have been increasingly adopted for point cloud classification.However,thesemethods typically require collecting extensive datasets frommultiple clients,which may lead to privacy leaks.Federated learning provides an effective solution to data leakage by eliminating the need for data transmission,relying instead on the exchange of model parameters.However,the uneven distribution of client data can still affect the model’s ability to generalize effectively.To address these challenges,we propose a new framework for point cloud classification called Federated Dynamic Aggregation Selection Strategy-based Multi-Receptive Field Fusion Classification Framework(FDASS-MRFCF).Specifically,we tackle these challenges with two key innovations:(1)During the client local training phase,we propose a Multi-Receptive Field Fusion Classification Model(MRFCM),which captures local and global structures in point cloud data through dynamic convolution and multi-scale feature fusion,enhancing the robustness of point cloud classification.(2)In the server aggregation phase,we introduce a Federated Dynamic Aggregation Selection Strategy(FDASS),which employs a hybrid strategy to average client model parameters,skip aggregation,or reallocate local models to different clients,thereby balancing global consistency and local diversity.We evaluate our framework using the ModelNet40 and ShapeNetPart benchmarks,demonstrating its effectiveness.The proposed method is expected to significantly advance the field of point cloud classification in a secure environment.展开更多
Among numerous 3D data acquisition methods, oblique photogrammetry and 3D laser scanning stand out with their irreplaceable technological advantages and form highly complementary technical characteristics. Tilt photog...Among numerous 3D data acquisition methods, oblique photogrammetry and 3D laser scanning stand out with their irreplaceable technological advantages and form highly complementary technical characteristics. Tilt photogrammetry relies on unmanned aerial vehicle platforms equipped with multi angle cameras to reconstruct three-dimensional geometric information from optical images through motion recovery structure algorithms and automatically map real textures. It has significant advantages such as wide coverage, high collection efficiency, and strong texture realism. However, due to limitations in flight angles and lighting conditions, there are often geometric deficiencies or texture distortions in areas near the ground, under eaves, and in vegetation covered areas of buildings. 3D laser scanning directly obtains high-precision 3D point clouds of the target surface by actively emitting laser pulses, which is not affected by lighting conditions and can accurately record the 3D coordinates of objects with millimeter level accuracy. However, its point cloud data lacks color and texture information, resulting in relatively low acquisition efficiency, large data volume, and long processing cycle.展开更多
The location where a robot grasps an object is closely related to the task type.For the same object,different user requirements may necessitate different grasping strategies.Visual affordance serves as a reliable sour...The location where a robot grasps an object is closely related to the task type.For the same object,different user requirements may necessitate different grasping strategies.Visual affordance serves as a reliable source of prior knowledge for manipulation.Existing methods learn affordance from images or videos,but planar affordance lacks the spatial information required for 6-degree-of-freedom(6-DoF)manipulation.Furthermore,current approaches are limited to affordances associated with predefined categories and cannot directly infer affordances from user instructions.To address such limitations,we propose a novel task:instruction-driven three-dimensional(3D)object affordance segmentation.To support this research,we introduce an instruction–affordance dataset(IAD),a challenging dataset consisting of 7190 object instances across 20 common object categories,paired with 624 manipulation instructions that specify the corresponding affordances.To evaluate generalization to novel commands,our dataset includes both seen and unseen settings.Building on this,we design an instruction-driven 3D affordance segmentation(IDAS)network,which extracts point cloud features and integrates instruction features layer by layer.Given a user instruction,our method segments suggested manipulation regions on the object’s point cloud,thereby guiding the selection of optimal grasp poses.Experimental results show that our method outperforms other related approaches under both seen and unseen settings,demonstrating generalization ability to diverse user commands and unknown affordances.展开更多
Among various data collection methods, oblique photogrammetry and 3D laser scanning stand out with their respective technological advantages. Tilt photogrammetry uses a multi angle camera system mounted on low altitud...Among various data collection methods, oblique photogrammetry and 3D laser scanning stand out with their respective technological advantages. Tilt photogrammetry uses a multi angle camera system mounted on low altitude platforms such as drones to synchronously obtain multi angle optical images of the target scene, and generates a three-dimensional model based on the principle of motion recovery structure. This technology has the advantages of wide coverage, high collection efficiency, and strong texture authenticity, especially suitable for rapid modeling of large-scale urban scenes. However, due to the influence of flight perspective, lighting conditions, and shading effects, oblique photography often suffers from geometric deficiencies or texture distortions in areas such as under eaves, near ground of buildings, and under tree crowns.展开更多
Point cloud compression is critical to deploy 3D representation of the physical world such as 3D immersive telepresence,autonomous driving,and cultural heritage preservation.However,point cloud data are distributed ir...Point cloud compression is critical to deploy 3D representation of the physical world such as 3D immersive telepresence,autonomous driving,and cultural heritage preservation.However,point cloud data are distributed irregularly and discontinuously in spatial and temporal domains,where redundant unoccupied voxels and weak correlations in 3D space make achieving efficient compression a challenging problem.In this paper,we propose a spatio-temporal context-guided algorithm for lossless point cloud geometry compression.The proposed scheme starts with dividing the point cloud into sliced layers of unit thickness along the longest axis.Then,it introduces a prediction method where both intraframe and inter-frame point clouds are available,by determining correspondences between adjacent layers and estimating the shortest path using the travelling salesman algorithm.Finally,the few prediction residual is efficiently compressed with optimal context-guided and adaptive fastmode arithmetic coding techniques.Experiments prove that the proposed method can effectively achieve low bit rate lossless compression of point cloud geometric information,and is suitable for 3D point cloud compression applicable to various types of scenes.展开更多
Accurately extracting plant point clouds from complex agricultural environments is essential for high-throughput phenotyping in smart farming.However,existing methods face significant challenges when processing large-...Accurately extracting plant point clouds from complex agricultural environments is essential for high-throughput phenotyping in smart farming.However,existing methods face significant challenges when processing large-scale agricultural point clouds owing to high noise levels,dense spatial distribution,and blurred structural boundaries between plant and non-plant regions.To address these issues,this study proposes PlaneSegNet,a voxel-based semantic segmentation network that incorporates an innovative plane attention module.This module aggregates projection features from the XZ and YZ planes,enhancing the model's ability to detect vertical geometric variations and thereby improving segmentation performance in boundary regions.Extensive experiments across representative agricultural scenarios at multiple scales,including open-field populations,greenhouse cultivation environments,and large-scale rural landscapes,demonstrate that PlaneSegNet significantly outperforms traditional geometry-based approaches and deep-learning models in plant and non-plant separation.By directly generating high-quality plant-only point clouds,PlaneSegNet significantly reduces reliance on manual pre-processing,offering a practical and generalisable solution for automated plant extraction across a wide range of agricultural applications.展开更多
基金supported in part by the National Key Research and Development Program of Chinaunder(Grant 2021YFB3101100)in part by the National Natural Science Foundation of Chinaunder(Grant 42461057),(Grant 62272123),and(Grant 42371470)+1 种基金in part by the Fundamental Research Program of Shanxi Province under(Grant 202303021212164)in part by the Postgraduate Education Innovation Program of Shanxi Province under(Grant 2024KY474).
摘要Recently,large-scale deep learning models have been increasingly adopted for point cloud classification.However,thesemethods typically require collecting extensive datasets frommultiple clients,which may lead to privacy leaks.Federated learning provides an effective solution to data leakage by eliminating the need for data transmission,relying instead on the exchange of model parameters.However,the uneven distribution of client data can still affect the model’s ability to generalize effectively.To address these challenges,we propose a new framework for point cloud classification called Federated Dynamic Aggregation Selection Strategy-based Multi-Receptive Field Fusion Classification Framework(FDASS-MRFCF).Specifically,we tackle these challenges with two key innovations:(1)During the client local training phase,we propose a Multi-Receptive Field Fusion Classification Model(MRFCM),which captures local and global structures in point cloud data through dynamic convolution and multi-scale feature fusion,enhancing the robustness of point cloud classification.(2)In the server aggregation phase,we introduce a Federated Dynamic Aggregation Selection Strategy(FDASS),which employs a hybrid strategy to average client model parameters,skip aggregation,or reallocate local models to different clients,thereby balancing global consistency and local diversity.We evaluate our framework using the ModelNet40 and ShapeNetPart benchmarks,demonstrating its effectiveness.The proposed method is expected to significantly advance the field of point cloud classification in a secure environment.
摘要Among numerous 3D data acquisition methods, oblique photogrammetry and 3D laser scanning stand out with their irreplaceable technological advantages and form highly complementary technical characteristics. Tilt photogrammetry relies on unmanned aerial vehicle platforms equipped with multi angle cameras to reconstruct three-dimensional geometric information from optical images through motion recovery structure algorithms and automatically map real textures. It has significant advantages such as wide coverage, high collection efficiency, and strong texture realism. However, due to limitations in flight angles and lighting conditions, there are often geometric deficiencies or texture distortions in areas near the ground, under eaves, and in vegetation covered areas of buildings. 3D laser scanning directly obtains high-precision 3D point clouds of the target surface by actively emitting laser pulses, which is not affected by lighting conditions and can accurately record the 3D coordinates of objects with millimeter level accuracy. However, its point cloud data lacks color and texture information, resulting in relatively low acquisition efficiency, large data volume, and long processing cycle.
基金supported by the National Natural Science Foundation of China(No.61973192).
摘要The location where a robot grasps an object is closely related to the task type.For the same object,different user requirements may necessitate different grasping strategies.Visual affordance serves as a reliable source of prior knowledge for manipulation.Existing methods learn affordance from images or videos,but planar affordance lacks the spatial information required for 6-degree-of-freedom(6-DoF)manipulation.Furthermore,current approaches are limited to affordances associated with predefined categories and cannot directly infer affordances from user instructions.To address such limitations,we propose a novel task:instruction-driven three-dimensional(3D)object affordance segmentation.To support this research,we introduce an instruction–affordance dataset(IAD),a challenging dataset consisting of 7190 object instances across 20 common object categories,paired with 624 manipulation instructions that specify the corresponding affordances.To evaluate generalization to novel commands,our dataset includes both seen and unseen settings.Building on this,we design an instruction-driven 3D affordance segmentation(IDAS)network,which extracts point cloud features and integrates instruction features layer by layer.Given a user instruction,our method segments suggested manipulation regions on the object’s point cloud,thereby guiding the selection of optimal grasp poses.Experimental results show that our method outperforms other related approaches under both seen and unseen settings,demonstrating generalization ability to diverse user commands and unknown affordances.
摘要Among various data collection methods, oblique photogrammetry and 3D laser scanning stand out with their respective technological advantages. Tilt photogrammetry uses a multi angle camera system mounted on low altitude platforms such as drones to synchronously obtain multi angle optical images of the target scene, and generates a three-dimensional model based on the principle of motion recovery structure. This technology has the advantages of wide coverage, high collection efficiency, and strong texture authenticity, especially suitable for rapid modeling of large-scale urban scenes. However, due to the influence of flight perspective, lighting conditions, and shading effects, oblique photography often suffers from geometric deficiencies or texture distortions in areas such as under eaves, near ground of buildings, and under tree crowns.
摘要Point cloud compression is critical to deploy 3D representation of the physical world such as 3D immersive telepresence,autonomous driving,and cultural heritage preservation.However,point cloud data are distributed irregularly and discontinuously in spatial and temporal domains,where redundant unoccupied voxels and weak correlations in 3D space make achieving efficient compression a challenging problem.In this paper,we propose a spatio-temporal context-guided algorithm for lossless point cloud geometry compression.The proposed scheme starts with dividing the point cloud into sliced layers of unit thickness along the longest axis.Then,it introduces a prediction method where both intraframe and inter-frame point clouds are available,by determining correspondences between adjacent layers and estimating the shortest path using the travelling salesman algorithm.Finally,the few prediction residual is efficiently compressed with optimal context-guided and adaptive fastmode arithmetic coding techniques.Experiments prove that the proposed method can effectively achieve low bit rate lossless compression of point cloud geometric information,and is suitable for 3D point cloud compression applicable to various types of scenes.
基金supported by Funds from the National key research and development program[2022YFD2002303-01].
摘要Accurately extracting plant point clouds from complex agricultural environments is essential for high-throughput phenotyping in smart farming.However,existing methods face significant challenges when processing large-scale agricultural point clouds owing to high noise levels,dense spatial distribution,and blurred structural boundaries between plant and non-plant regions.To address these issues,this study proposes PlaneSegNet,a voxel-based semantic segmentation network that incorporates an innovative plane attention module.This module aggregates projection features from the XZ and YZ planes,enhancing the model's ability to detect vertical geometric variations and thereby improving segmentation performance in boundary regions.Extensive experiments across representative agricultural scenarios at multiple scales,including open-field populations,greenhouse cultivation environments,and large-scale rural landscapes,demonstrate that PlaneSegNet significantly outperforms traditional geometry-based approaches and deep-learning models in plant and non-plant separation.By directly generating high-quality plant-only point clouds,PlaneSegNet significantly reduces reliance on manual pre-processing,offering a practical and generalisable solution for automated plant extraction across a wide range of agricultural applications.