A gradient descent algorithm with adjustable parameter for attitude estimation is developed,aiming at the attitude measurement for small unmanned aerial vehicle(UAV)in real-time flight conditions.The accelerometer and...A gradient descent algorithm with adjustable parameter for attitude estimation is developed,aiming at the attitude measurement for small unmanned aerial vehicle(UAV)in real-time flight conditions.The accelerometer and magnetometer are introduced to construct an error equation with the gyros,thus the drifting characteristics of gyroscope can be compensated by solving the error equation utilized by the gradient descent algorithm.Performance of the presented algorithm is evaluated using a self-proposed micro-electro-mechanical system(MEMS)based attitude heading reference system which is mounted on a tri-axis turntable.The on-ground,turntable and flight experiments indicate that the estimation attitude has a good accuracy.Also,the presented system is compared with an open-source flight control system which runs extended Kalman filter(EKF),and the results show that the attitude control system using the gradient descent method can estimate the attitudes for UAV effectively.展开更多
Under some assumptions, the solution set of a nonlinear complementarity problem coincides with the set of local minima of the corresponding minimization problem. This paper uses a family of new merit functions to deal...Under some assumptions, the solution set of a nonlinear complementarity problem coincides with the set of local minima of the corresponding minimization problem. This paper uses a family of new merit functions to deal with nonlinear complementarity problem where the underlying function is assumed to be a continuous but not necessarily locally Lipschitzian map and gives a descent algorithm for solving the nonsmooth continuous complementarity problems. In addition, the global convergence of the derivative free descent algorithm is also proved.展开更多
In the field of calculating the attack area of air-to-air missiles in modern air combat scenarios,the limitations of existing research,including real-time calculation,accuracy efficiency trade-off,and the absence of t...In the field of calculating the attack area of air-to-air missiles in modern air combat scenarios,the limitations of existing research,including real-time calculation,accuracy efficiency trade-off,and the absence of the three-dimensional attack area model,restrict their practical applications.To address these issues,an improved backtracking algorithm is proposed to improve calculation efficiency.A significant reduction in solution time and maintenance of accuracy in the three-dimensional attack area are achieved by using the proposed algorithm.Furthermore,the age-layered population structure genetic programming(ALPS-GP)algorithm is introduced to determine an analytical polynomial model of the three-dimensional attack area,considering real-time requirements.The accuracy of the polynomial model is enhanced through the coefficient correction using an improved gradient descent algorithm.The study reveals a remarkable combination of high accuracy and efficient real-time computation,with a mean error of 91.89 m using the analytical polynomial model of the three-dimensional attack area solved in just 10-4s,thus meeting the requirements of real-time combat scenarios.展开更多
Some properties of a class of quasi-differentiable functions(the difference of two finite convex functions) are considered in this paper. And the convergence of the steepest descent algorithm for unconstrained and c...Some properties of a class of quasi-differentiable functions(the difference of two finite convex functions) are considered in this paper. And the convergence of the steepest descent algorithm for unconstrained and constrained quasi-differentiable programming is proved.展开更多
The distributed nonconvex optimization problem of minimizing a global cost function formed by a sum of n local cost functions by using local information exchange is considered.This problem is an important component of...The distributed nonconvex optimization problem of minimizing a global cost function formed by a sum of n local cost functions by using local information exchange is considered.This problem is an important component of many machine learning techniques with data parallelism,such as deep learning and federated learning.We propose a distributed primal-dual stochastic gradient descent(SGD)algorithm,suitable for arbitrarily connected communication networks and any smooth(possibly nonconvex)cost functions.We show that the proposed algorithm achieves the linear speedup convergence rate O(1/(√nT))for general nonconvex cost functions and the linear speedup convergence rate O(1/(nT)) when the global cost function satisfies the Polyak-Lojasiewicz(P-L)condition,where T is the total number of iterations.We also show that the output of the proposed algorithm with constant parameters linearly converges to a neighborhood of a global optimum.We demonstrate through numerical experiments the efficiency of our algorithm in comparison with the baseline centralized SGD and recently proposed distributed SGD algorithms.展开更多
A recommender system(RS)relying on latent factor analysis usually adopts stochastic gradient descent(SGD)as its learning algorithm.However,owing to its serial mechanism,an SGD algorithm suffers from low efficiency and...A recommender system(RS)relying on latent factor analysis usually adopts stochastic gradient descent(SGD)as its learning algorithm.However,owing to its serial mechanism,an SGD algorithm suffers from low efficiency and scalability when handling large-scale industrial problems.Aiming at addressing this issue,this study proposes a momentum-incorporated parallel stochastic gradient descent(MPSGD)algorithm,whose main idea is two-fold:a)implementing parallelization via a novel datasplitting strategy,and b)accelerating convergence rate by integrating momentum effects into its training process.With it,an MPSGD-based latent factor(MLF)model is achieved,which is capable of performing efficient and high-quality recommendations.Experimental results on four high-dimensional and sparse matrices generated by industrial RS indicate that owing to an MPSGD algorithm,an MLF model outperforms the existing state-of-the-art ones in both computational efficiency and scalability.展开更多
Integrated optical phased arrays(OPAs),owing to their high integration level and wide-angle beamsteering capability,are a key promising component for 3D/4D sensing and free-space optical communication.However,integrat...Integrated optical phased arrays(OPAs),owing to their high integration level and wide-angle beamsteering capability,are a key promising component for 3D/4D sensing and free-space optical communication.However,integrated OPAs typically employ a single-wavelength laser as the input light source,which limits their application in spectral imaging and secure communication.In this work,we propose and experimentally demonstrate a dual-wavelength coherent beam combining(CBC)scheme using a 64-element integrated OPA.A multiwavelength CBC model is established and validated with two wavelength pairs(1545/1555 and 1535∕1565 nm).Both simulation and experimental results confirm that the beam dispersion intensifies with the steering angle and wavelength separation.To overcome this inherent dispersion issue,we employ a stochastic parallel gradient descent algorithm with two avalanche photodiodes as feedback sensors,achieving an arbitrarily configurable angular separation between the two wavelength beams.Finally,we demonstrate the wide field of view beam-steering capability of the combined beam across a 60 deg(±30 deg)range.We provide a foundational framework for advancing multiwavelength OPA toward applications in spectral imaging and secure optical communications.展开更多
准确求解各支承点的承载力分布是静压导轨结构设计与性能优化的关键。针对多点支承超静定系统的支承力求解问题,传统代数方法的计算维度与复杂度随支承点数量增加而显著上升。为此,提出一种基于梯度下降算法的超静定载荷分布迭代求解方...准确求解各支承点的承载力分布是静压导轨结构设计与性能优化的关键。针对多点支承超静定系统的支承力求解问题,传统代数方法的计算维度与复杂度随支承点数量增加而显著上升。为此,提出一种基于梯度下降算法的超静定载荷分布迭代求解方法。该方法的核心思想,基于刚体运动学假设和刚体静力学线性分布假设,将各支承点的未知反力映射为关于分布斜率与截距的线性函数,从而将高维的力矢量求解降维为极低维的特征参数优化。在此基础上,构建了表征静力学平衡偏差的损失函数,利用梯度下降算法迭代搜索最优参数以最小化平衡偏差。研究表明,该方法有效规避了对高维线性方程组的依赖,实现了计算复杂度与支承点数量的解耦。同时,通过对比工程案例的算法计算结果与有限元仿真(finite element analysis,FEA)结果,证实了该方法兼具良好的精度与可靠性,能够满足工程设计需求。展开更多
Coordinate descent method is a unconstrained optimization technique. When it is applied to support vector machine (SVM), at each step the method updates one component of w by solving a one-variable sub-problem while...Coordinate descent method is a unconstrained optimization technique. When it is applied to support vector machine (SVM), at each step the method updates one component of w by solving a one-variable sub-problem while fixing other components. All components of w update after one iteration. Then go to next iteration. Though the method converges and converges fast in the beginning, it converges slow for final convergence. To improve the speed of final convergence of coordinate descent method, Hooke and Jeeves algorithm which adds pattern search after every iteration in coordinate descent method was applied to SVM and a global Newton algorithm was used to solve one-variable subproblems. We proved the convergence of the algorithm. Experimental results show Hooke and Jeeves' method does accelerate convergence specially for final convergence and achieves higher testing accuracy more quickly in classification.展开更多
基金supported by the Fundamental Research Funds for the Central Universities(No.56XAA17075)
摘要A gradient descent algorithm with adjustable parameter for attitude estimation is developed,aiming at the attitude measurement for small unmanned aerial vehicle(UAV)in real-time flight conditions.The accelerometer and magnetometer are introduced to construct an error equation with the gyros,thus the drifting characteristics of gyroscope can be compensated by solving the error equation utilized by the gradient descent algorithm.Performance of the presented algorithm is evaluated using a self-proposed micro-electro-mechanical system(MEMS)based attitude heading reference system which is mounted on a tri-axis turntable.The on-ground,turntable and flight experiments indicate that the estimation attitude has a good accuracy.Also,the presented system is compared with an open-source flight control system which runs extended Kalman filter(EKF),and the results show that the attitude control system using the gradient descent method can estimate the attitudes for UAV effectively.
基金Supported by the National Science foundation of China(10671126, 40771095)the Key Project for Fundamental Research of STCSM(06JC14057)+1 种基金Shanghai Leading Academic Discipline Project(S30501)the Innovation Fund Project for Graduate Students of Shanghai(JWCXSL0801)
摘要Under some assumptions, the solution set of a nonlinear complementarity problem coincides with the set of local minima of the corresponding minimization problem. This paper uses a family of new merit functions to deal with nonlinear complementarity problem where the underlying function is assumed to be a continuous but not necessarily locally Lipschitzian map and gives a descent algorithm for solving the nonsmooth continuous complementarity problems. In addition, the global convergence of the derivative free descent algorithm is also proved.
基金National Natural Science Foundation of China(62373187)Forward-looking Layout Special Projects(ILA220591A22)。
摘要In the field of calculating the attack area of air-to-air missiles in modern air combat scenarios,the limitations of existing research,including real-time calculation,accuracy efficiency trade-off,and the absence of the three-dimensional attack area model,restrict their practical applications.To address these issues,an improved backtracking algorithm is proposed to improve calculation efficiency.A significant reduction in solution time and maintenance of accuracy in the three-dimensional attack area are achieved by using the proposed algorithm.Furthermore,the age-layered population structure genetic programming(ALPS-GP)algorithm is introduced to determine an analytical polynomial model of the three-dimensional attack area,considering real-time requirements.The accuracy of the polynomial model is enhanced through the coefficient correction using an improved gradient descent algorithm.The study reveals a remarkable combination of high accuracy and efficient real-time computation,with a mean error of 91.89 m using the analytical polynomial model of the three-dimensional attack area solved in just 10-4s,thus meeting the requirements of real-time combat scenarios.
基金Supported by the State Foundations of Ph.D.Units(20020141013)Supported by the NSF of China(10001007)
摘要Some properties of a class of quasi-differentiable functions(the difference of two finite convex functions) are considered in this paper. And the convergence of the steepest descent algorithm for unconstrained and constrained quasi-differentiable programming is proved.
基金supported by the Knut and Alice Wallenberg Foundationthe Swedish Foundation for Strategic Research+1 种基金the Swedish Research Councilthe National Natural Science Foundation of China(62133003,61991403,61991404,61991400)。
摘要The distributed nonconvex optimization problem of minimizing a global cost function formed by a sum of n local cost functions by using local information exchange is considered.This problem is an important component of many machine learning techniques with data parallelism,such as deep learning and federated learning.We propose a distributed primal-dual stochastic gradient descent(SGD)algorithm,suitable for arbitrarily connected communication networks and any smooth(possibly nonconvex)cost functions.We show that the proposed algorithm achieves the linear speedup convergence rate O(1/(√nT))for general nonconvex cost functions and the linear speedup convergence rate O(1/(nT)) when the global cost function satisfies the Polyak-Lojasiewicz(P-L)condition,where T is the total number of iterations.We also show that the output of the proposed algorithm with constant parameters linearly converges to a neighborhood of a global optimum.We demonstrate through numerical experiments the efficiency of our algorithm in comparison with the baseline centralized SGD and recently proposed distributed SGD algorithms.
基金supported in part by the National Natural Science Foundation of China(61772493)the Deanship of Scientific Research(DSR)at King Abdulaziz University(RG-48-135-40)+1 种基金Guangdong Province Universities and College Pearl River Scholar Funded Scheme(2019)the Natural Science Foundation of Chongqing(cstc2019jcyjjqX0013)。
摘要A recommender system(RS)relying on latent factor analysis usually adopts stochastic gradient descent(SGD)as its learning algorithm.However,owing to its serial mechanism,an SGD algorithm suffers from low efficiency and scalability when handling large-scale industrial problems.Aiming at addressing this issue,this study proposes a momentum-incorporated parallel stochastic gradient descent(MPSGD)algorithm,whose main idea is two-fold:a)implementing parallelization via a novel datasplitting strategy,and b)accelerating convergence rate by integrating momentum effects into its training process.With it,an MPSGD-based latent factor(MLF)model is achieved,which is capable of performing efficient and high-quality recommendations.Experimental results on four high-dimensional and sparse matrices generated by industrial RS indicate that owing to an MPSGD algorithm,an MLF model outperforms the existing state-of-the-art ones in both computational efficiency and scalability.
基金supported by the National Natural Science Foundation of China(Grant Nos.62575305,62305275,12574333,11974258,and 62305388)the Natural Science Foundation of Hunan Province of China(Grant No.2024JJ6473)the Innovation Research Foundation of National University of Defense Technology(Innovation Research Foundation of(NUDT)(Grant No.24-ZZCX-JDZ-18).
摘要Integrated optical phased arrays(OPAs),owing to their high integration level and wide-angle beamsteering capability,are a key promising component for 3D/4D sensing and free-space optical communication.However,integrated OPAs typically employ a single-wavelength laser as the input light source,which limits their application in spectral imaging and secure communication.In this work,we propose and experimentally demonstrate a dual-wavelength coherent beam combining(CBC)scheme using a 64-element integrated OPA.A multiwavelength CBC model is established and validated with two wavelength pairs(1545/1555 and 1535∕1565 nm).Both simulation and experimental results confirm that the beam dispersion intensifies with the steering angle and wavelength separation.To overcome this inherent dispersion issue,we employ a stochastic parallel gradient descent algorithm with two avalanche photodiodes as feedback sensors,achieving an arbitrarily configurable angular separation between the two wavelength beams.Finally,we demonstrate the wide field of view beam-steering capability of the combined beam across a 60 deg(±30 deg)range.We provide a foundational framework for advancing multiwavelength OPA toward applications in spectral imaging and secure optical communications.
摘要准确求解各支承点的承载力分布是静压导轨结构设计与性能优化的关键。针对多点支承超静定系统的支承力求解问题,传统代数方法的计算维度与复杂度随支承点数量增加而显著上升。为此,提出一种基于梯度下降算法的超静定载荷分布迭代求解方法。该方法的核心思想,基于刚体运动学假设和刚体静力学线性分布假设,将各支承点的未知反力映射为关于分布斜率与截距的线性函数,从而将高维的力矢量求解降维为极低维的特征参数优化。在此基础上,构建了表征静力学平衡偏差的损失函数,利用梯度下降算法迭代搜索最优参数以最小化平衡偏差。研究表明,该方法有效规避了对高维线性方程组的依赖,实现了计算复杂度与支承点数量的解耦。同时,通过对比工程案例的算法计算结果与有限元仿真(finite element analysis,FEA)结果,证实了该方法兼具良好的精度与可靠性,能够满足工程设计需求。
基金supported by the National Natural Science Foundation of China (6057407560705004)
摘要Coordinate descent method is a unconstrained optimization technique. When it is applied to support vector machine (SVM), at each step the method updates one component of w by solving a one-variable sub-problem while fixing other components. All components of w update after one iteration. Then go to next iteration. Though the method converges and converges fast in the beginning, it converges slow for final convergence. To improve the speed of final convergence of coordinate descent method, Hooke and Jeeves algorithm which adds pattern search after every iteration in coordinate descent method was applied to SVM and a global Newton algorithm was used to solve one-variable subproblems. We proved the convergence of the algorithm. Experimental results show Hooke and Jeeves' method does accelerate convergence specially for final convergence and achieves higher testing accuracy more quickly in classification.