arXiv subjects

Guangming Wang

Publications and source records attributed to Guangming Wang.

At least 73 records · Page 4Linked to original sources

arXiv2022

Efficient 3D Deep LiDAR Odometry

An efficient 3D point cloud learning architecture, named EfficientLO-Net, for LiDAR odometry is first proposed in this paper. In this architecture, the projection-aware representation of the 3D point cloud is proposed to organize the raw 3D point cloud into an ordered data form to achieve efficiency. The Pyramid, Warping, and Cost volume (PWC) structure for the LiDAR odometry task is built to estimate and refine the pose in a coarse-to-fine approach. A projection-aware attentive cost volume is built to directly associate two discrete point clouds and obtain embedding motion patterns. Then, a trainable embedding mask is proposed to weigh the local motion patterns to regress the overall pose and filter outlier points. The trainable pose warp-refinement module is iteratively used with embedding mask optimized hierarchically to make the pose estimation more robust for outliers. The entire architecture is holistically optimized end-to-end to achieve adaptive learning of cost volume and mask, and all operations involving point cloud sampling and grouping are accelerated by projection-aware 3D feature learning methods. The superior performance and effectiveness of our LiDAR odometry architecture are demonstrated on KITTI, M2DGR, and Argoverse datasets. Our method outperforms all recent learning-based methods and even the geometry-based approach, LOAM with mapping optimization, on most sequences of KITTI odometry dataset. We open sourced our codes at: https://github.com/IRMVLab/EfficientLO-Net.

cs.CV↗

arXiv2022

Unsupervised Learning of 3D Scene Flow with 3D Odometry Assistance

Scene flow represents the 3D motion of each point in the scene, which explicitly describes the distance and the direction of each point's movement. Scene flow estimation is used in various applications such as autonomous driving fields, activity recognition, and virtual reality fields. As it is challenging to annotate scene flow with ground truth for real-world data, this leaves no real-world dataset available to provide a large amount of data with ground truth for scene flow estimation. Therefore, many works use synthesized data to pre-train their network and real-world LiDAR data to finetune. Unlike the previous unsupervised learning of scene flow in point clouds, we propose to use odometry information to assist the unsupervised learning of scene flow and use real-world LiDAR data to train our network. Supervised odometry provides more accurate shared cost volume for scene flow. In addition, the proposed network has mask-weighted warp layers to get a more accurate predicted point cloud. The warp operation means applying an estimated pose transformation or scene flow to a source point cloud to obtain a predicted point cloud and is the key to refining scene flow from coarse to fine. When performing warp operations, the points in different states use different weights for the pose transformation and scene flow transformation. We classify the states of points as static, dynamic, and occluded, where the static masks are used to divide static and dynamic points, and the occlusion masks are used to divide occluded points. The mask-weighted warp layer indicates that static masks and occlusion masks are used as weights when performing warp operations. Our designs are proved to be effective in ablation experiments. The experiment results show the promising prospect of an odometry-assisted unsupervised learning method for 3D scene flow in real-world data.

cs.CV↗

arXiv2022

Pseudo-LiDAR for Visual Odometry

In the existing methods, LiDAR odometry shows superior performance, but visual odometry is still widely used for its price advantage. Conventionally, the task of visual odometry mainly rely on the input of continuous images. However, it is very complicated for the odometry network to learn the epipolar geometry information provided by the images. In this paper, the concept of pseudo-LiDAR is introduced into the odometry to solve this problem. The pseudo-LiDAR point cloud back-projects the depth map generated by the image into the 3D point cloud, which changes the way of image representation. Compared with the stereo images, the pseudo-LiDAR point cloud generated by the stereo matching network can get the explicit 3D coordinates. Since the 6 Degrees of Freedom (DoF) pose transformation occurs in 3D space, the 3D structure information provided by the pseudo-LiDAR point cloud is more direct than the image. Compared with sparse LiDAR, the pseudo-LiDAR has a denser point cloud. In order to make full use of the rich point cloud information provided by the pseudo-LiDAR, a projection-aware dense odometry pipeline is adopted. Most previous LiDAR-based algorithms sampled 8192 points from the point cloud as input to the odometry network. The projection-aware dense odometry pipeline takes all the pseudo-LiDAR point clouds generated from the images except for the error points as the input to the network. While making full use of the 3D geometric information in the images, the semantic information in the images is also used in the odometry task. The fusion of 2D-3D is achieved in an image-only based odometry. Experiments on the KITTI dataset prove the effectiveness of our method. To the best of our knowledge, this is the first visual odometry method using pseudo-LiDAR.

cs.CV↗

arXiv2022

Electronic structure of $\boldsymbolα$-RuCl$_3$ by fixed-node and fixed-phase diffusion Monte Carlo methods

Layered material $α$-RuCl$_3$ has caught wide attention due to its possible realization of Kitaev's spin liquid and its electronic structure that involves the interplay of electron-electron correlations and spin-orbit effects. Several DFT$+U$ studies have suggested that both electron-electron correlations and spin-orbit effects are crucial for accurately describing the band gap. This work studies the importance of these two effects using fixed-node and fixed-phase diffusion Monte Carlo calculations both in spin-averaged and explicit spin-orbit formalisms. In the latter, the Slater-Jastrow trial function is constructed from two-component spin-orbitals using our recent quantum Monte Carlo (QMC) developments and thoroughly tested effective core potentials. Our results show that the gap in the ideal crystal is already accurately described by the spin-averaged case, with the dominant role being played by the magnetic ground state with significant exchange and electron correlation effects. We find qualitative agreement between hybrid DFT, DFT+$U$, and QMC. In addition, QMC results agree very well with available experiments, and we identify the values of exact Fock exchange mixing that provide comparable gaps. Explicit spin-orbit QMC calculations reveal that the effect of spin-orbit coupling on the gap is minor, of the order of 0.2 eV, which corresponds to the strength of the spin-orbit of the Ru atom.

cond-mat.str-el↗

arXiv2022

What Matters for 3D Scene Flow Network

3D scene flow estimation from point clouds is a low-level 3D motion perception task in computer vision. Flow embedding is a commonly used technique in scene flow estimation, and it encodes the point motion between two consecutive frames. Thus, it is critical for the flow embeddings to capture the correct overall direction of the motion. However, previous works only search locally to determine a soft correspondence, ignoring the distant points that turn out to be the actual matching ones. In addition, the estimated correspondence is usually from the forward direction of the adjacent point clouds, and may not be consistent with the estimated correspondence acquired from the backward direction. To tackle these problems, we propose a novel all-to-all flow embedding layer with backward reliability validation during the initial scene flow estimation. Besides, we investigate and compare several design choices in key components of the 3D scene flow network, including the point similarity calculation, input elements of predictor, and predictor & refinement level design. After carefully choosing the most effective designs, we are able to present a model that achieves the state-of-the-art performance on FlyingThings3D and KITTI Scene Flow datasets. Our proposed model surpasses all existing methods by at least 38.2% on FlyingThings3D dataset and 24.7% on KITTI Scene Flow dataset for EPE3D metric. We release our codes at https://github.com/IRMVLab/3DFlow.

cs.CV↗

arXiv2022

Correlation consistent effective core potentials for late 3d transition metals adapted for plane wave calculations

We construct a new modification of correlation consistent effective potentials (ccECPs) for late $3d$ elements Cr-Zn with Ne-core that are adapted for efficiency and low energy cut-offs in plane wave calculations. The decrease in accuracy is rather minor so that the constructions are in the same overall accuracy class as the original ccECPs. The resulting new constructions work with energy cut-offs at or below $\approx$ 400 Ry and thus make calculations of large systems with transition metals feasible for plane wave codes. We provide also the basic benchmarks for atomic spectra and molecular tests of this modified option that we denote as ccECP-soft.

cond-mat.mtrl-sci↗

arXiv2022

3D Hierarchical Refinement and Augmentation for Unsupervised Learning of Depth and Pose from Monocular Video

Depth and ego-motion estimations are essential for the localization and navigation of autonomous robots and autonomous driving. Recent studies make it possible to learn the per-pixel depth and ego-motion from the unlabeled monocular video. A novel unsupervised training framework is proposed with 3D hierarchical refinement and augmentation using explicit 3D geometry. In this framework, the depth and pose estimations are hierarchically and mutually coupled to refine the estimated pose layer by layer. The intermediate view image is proposed and synthesized by warping the pixels in an image with the estimated depth and coarse pose. Then, the residual pose transformation can be estimated from the new view image and the image of the adjacent frame to refine the coarse pose. The iterative refinement is implemented in a differentiable manner in this paper, making the whole framework optimized uniformly. Meanwhile, a new image augmentation method is proposed for the pose estimation by synthesizing a new view image, which creatively augments the pose in 3D space but gets a new augmented 2D image. The experiments on KITTI demonstrate that our depth estimation achieves state-of-the-art performance and even surpasses recent approaches that utilize other auxiliary tasks. Our visual odometry outperforms all recent unsupervised monocular learning-based methods and achieves competitive performance to the geometry-based method, ORB-SLAM2 with back-end optimization.

cs.CV↗

arXiv2022

Unsupervised Learning of 3D Scene Flow from Monocular Camera

Scene flow represents the motion of points in the 3D space, which is the counterpart of the optical flow that represents the motion of pixels in the 2D image. However, it is difficult to obtain the ground truth of scene flow in the real scenes, and recent studies are based on synthetic data for training. Therefore, how to train a scene flow network with unsupervised methods based on real-world data shows crucial significance. A novel unsupervised learning method for scene flow is proposed in this paper, which utilizes the images of two consecutive frames taken by monocular camera without the ground truth of scene flow for training. Our method realizes the goal that training scene flow network with real-world data, which bridges the gap between training data and test data and broadens the scope of available data for training. Unsupervised learning of scene flow in this paper mainly consists of two parts: (i) depth estimation and camera pose estimation, and (ii) scene flow estimation based on four different loss functions. Depth estimation and camera pose estimation obtain the depth maps and camera pose between two consecutive frames, which provide further information for the next scene flow estimation. After that, we used depth consistency loss, dynamic-static consistency loss, Chamfer loss, and Laplacian regularization loss to carry out unsupervised training of the scene flow network. To our knowledge, this is the first paper that realizes the unsupervised learning of 3D scene flow from monocular camera. The experiment results on KITTI show that our method for unsupervised learning of scene flow meets great performance compared to traditional methods Iterative Closest Point (ICP) and Fast Global Registration (FGR). The source code is available at: https://github.com/IRMVLab/3DUnMonoFlow.

cs.CV↗

arXiv2022

Magnetic measures of purity for MnBi$_2$Te$_4$

The intrinsically anti-ferromagnetic topological insulator, MnBi$_2$Te$_4$ (MBT), has garnered significant attention recently. The excitement for this layered van der Waals bonded compound stems from its potential to host numerous exotic topological quantum states. For instance, quantum anomalous Hall states are predicted for odd-layer compounds and axion insulator states for the even-layer compounds. Unfortunately, the realization of these phenomena has been hindered by experimental challenges such as the existence of negative charge carriers, i.e., electron doping, which have been linked to anti-site defects among the Mn and Bi sub-lattices. Based on high level diffusion Monte Carlo (DMC) and DMC-tuned DFT+U calculations, we provide benchmark quality results for the bulk Mn magnetization as well as for Mn$_{Bi}$ and Bi$_{Mn}$ defects. We use this information to refine and extend models that estimate the anti-site defect concentration in actual MBT samples when combined with data from magnetic susceptibility and intermediate field magnetization measurements. Our models are validated through favorable comparison with prior experimental studies that obtained both magnetic and site occupancy data. We then extend our estimates to a larger set of prior samples to identify a probable zone of low defect density that has yet to be reached in synthesis. We anticipate our theoretically based magnetic purity measures may be used as minimization targets in the cycle of refinement needed to synthesize MBT samples with low anti-site defect concentrations and more reproducible topological properties.

cond-mat.mtrl-sci↗

arXiv2022

Interactive Multi-scale Fusion of 2D and 3D Features for Multi-object Tracking

Multiple object tracking (MOT) is a significant task in achieving autonomous driving. Traditional works attempt to complete this task, either based on point clouds (PC) collected by LiDAR, or based on images captured from cameras. However, relying on one single sensor is not robust enough, because it might fail during the tracking process. On the other hand, feature fusion from multiple modalities contributes to the improvement of accuracy. As a result, new techniques based on different sensors integrating features from multiple modalities are being developed. Texture information from RGB cameras and 3D structure information from Lidar have respective advantages under different circumstances. However, it's not easy to achieve effective feature fusion because of completely distinct information modalities. Previous fusion methods usually fuse the top-level features after the backbones extract the features from different modalities. In this paper, we first introduce PointNet++ to obtain multi-scale deep representations of point cloud to make it adaptive to our proposed Interactive Feature Fusion between multi-scale features of images and point clouds. Specifically, through multi-scale interactive query and fusion between pixel-level and point-level features, our method, can obtain more distinguishing features to improve the performance of multiple object tracking. Besides, we explore the effectiveness of pre-training on each single modality and fine-tuning on the fusion-based model. The experimental results demonstrate that our method can achieve good performance on the KITTI benchmark and outperform other approaches without using multi-scale feature fusion. Moreover, the ablation studies indicates the effectiveness of multi-scale feature fusion and pre-training on single modality.

cs.CV↗

arXiv2022

DetFlowTrack: 3D Multi-object Tracking based on Simultaneous Optimization of Object Detection and Scene Flow Estimation

3D Multi-Object Tracking (MOT) is an important part of the unmanned vehicle perception module. Most methods optimize object detection and data association independently. These methods make the network structure complicated and limit the improvement of MOT accuracy. we proposed a 3D MOT framework based on simultaneous optimization of object detection and scene flow estimation. In the framework, a detection-guidance scene flow module is proposed to relieve the problem of incorrect inter-frame assocation. For more accurate scene flow label especially in the case of motion with rotation, a box-transformation-based scene flow ground truth calculation method is proposed. Experimental results on the KITTI MOT dataset show competitive results over the state-of-the-arts and the robustness under extreme motion with rotation.

cs.CV↗

arXiv2022

A new generation of effective core potentials from correlated and spin-orbit calculations: selected heavy elements

We introduce new correlation consistent effective core potentials (ccECPs) for the elements I, Te, Bi, Ag, Au, Pd, Ir, Mo, and W with $4d$, $5d$, $6s$ and $6p$ valence spaces. These ccECPs are given as a sum of spin-orbit averaged relativistic effective potential (AREP) and effective spin-orbit (SO) terms. The construction involves several steps with increasing refinements from more simple to fully correlated methods. The optimizations are carried out with objective functions that include weighted many-body atomic spectra, norm-conservation criteria, and spin-orbit splittings. Transferability tests involve molecular binding curves of corresponding hydride and oxide dimers. The constructed ccECPs are systematically better and in a few cases on par with previous effective core potential (ECP) tables on all tested criteria and provide a significant increase in accuracy for valence-only calculations with these elements. Our study confirms the importance of the AREP part in determining the overall quality of the ECP even in the presence of sizable spin-orbit effects. The subsequent quantum Monte Carlo (QMC) calculations point out the importance of accurate trial wave functions which in some cases (mid series transition elements) require treatment well beyond single-reference.

cond-mat.mtrl-sci↗

arXiv2022

Origin of Metal-Insulator Transitions in Correlated Perovskite Metals

The mechanisms that drive metal-to-insulator transitions (MIT) in correlated solids are not fully understood. For example, the perovskite (PV) SrCoO3 is a FM metal while the oxygen-deficient (n-doped) brownmillerite (BM) SrCoO2.5 is an anti-ferromagnetic (AFM) insulator. Given the magnetic and structural transitions that accompany the MIT, the driver for such a MIT transition is unclear. We also observe that the perovskite metals LaNiO3, SrFeO3, and SrCoO3 also undergo MIT when n-doped via high-to-low valence compositional changes. Also, pressurizing the insulating BM SrCoO2.5 phase, drives a gap closing. Using DFT and correlated diffusion Monte Carlo approaches we demonstrate that the ABO3 perovskites most prone to MIT are self hole-doped materials, reminiscent of a negative charge-transfer system. Upon n-doping away from the negative-charge transfer metallic phase, an underlying charge-lattice (or e-phonon) coupling drives the system to a bond-disproportionated gapped state, thereby achieving ligand hole passivation at certain sites only, leading to charge-disproportionated states. The size of the gap opened is correlated with the size of the hole-filling at these ligand sites. This suggests that the interactions driving the gap opening to realize a MIT even in correlated metals is the charge-transfer energy, but it couples with the underlying phonons to enable the transition to the insulating phase. Other orderings (magnetic, charge, etc.) driven by weaker interactions are secondary and may assist gap openings at small dopings, but its the charge-transfer energy that predominantly determines the bandgap, with a negative energy preferring the metallic phase. This n-doping can be achieved by modulations in stoichiometry or composition or pressure. Hence, controlling the amount of the ligand-hole is key in controlling MIT. We compare our predictions to experiments where possible.

cond-mat.mtrl-sci↗

arXiv2022

Residual 3D Scene Flow Learning with Context-Aware Feature Extraction

Scene flow estimation is the task to predict the point-wise or pixel-wise 3D displacement vector between two consecutive frames of point clouds or images, which has important application in fields such as service robots and autonomous driving. Although many previous works have explored greatly on scene flow estimation based on point clouds, there are two problems that have not been noticed or well solved before: 1) Points of adjacent frames in repetitive patterns may be wrongly associated due to similar spatial structure in their neighbourhoods; 2) Scene flow between adjacent frames of point clouds with long-distance movement may be inaccurately estimated. To solve the first problem, a novel context-aware set convolution layer is proposed in this paper to exploit contextual structure information of Euclidean space and learn soft aggregation weights for local point features. This design is inspired by human perception of contextual structure information during scene understanding with repetitive patterns. The context-aware set convolution layer is incorporated in a context-aware point feature pyramid module of 3D point clouds for scene flow estimation. For the second problem, an explicit residual flow learning structure is proposed in the residual flow refinement layer to cope with long-distance movement. The experiments and ablation study on FlyingThings3D and KITTI scene flow datasets demonstrate the effectiveness of each proposed component. The qualitative results show that the problems of ambiguous inter-frame association and long-distance movement estimation are well handled. Quantitative results on both FlyingThings3D and KITTI scene flow datasets show that the proposed method achieves state-of-the-art performance, surpassing all other previous works to the best of our knowledge by at least 25%.

cs.CV↗

arXiv2021

Learning of Long-Horizon Sparse-Reward Robotic Manipulator Tasks with Base Controllers

Deep Reinforcement Learning (DRL) enables robots to perform some intelligent tasks end-to-end. However, there are still many challenges for long-horizon sparse-reward robotic manipulator tasks. On the one hand, a sparse-reward setting causes exploration inefficient. On the other hand, exploration using physical robots is of high cost and unsafe. In this paper, we propose a method of learning long-horizon sparse-reward tasks utilizing one or more existing traditional controllers named base controllers in this paper. Built upon Deep Deterministic Policy Gradients (DDPG), our algorithm incorporates the existing base controllers into stages of exploration, value learning, and policy update. Furthermore, we present a straightforward way of synthesizing different base controllers to integrate their strengths. Through experiments ranging from stacking blocks to cups, it is demonstrated that the learned state-based or image-based policies steadily outperform base controllers. Compared to previous works of learning from demonstrations, our method improves sample efficiency by orders of magnitude and improves the performance. Overall, our method bears the potential of leveraging existing industrial robot manipulation systems to build more flexible and intelligent controllers.

cs.RO↗

arXiv2021

A quantum Monte Carlo study of systems with effective core potentials and node nonlinearities

We study beryllium dihydride (BeH$_2$) and acetylene (C$_2$H$_2$) molecules using real-space diffusion Monte Carlo (DMC) method. The molecules serve as perhaps the simplest prototypes that illustrate the difficulties with biases in the fixed-node DMC calculations that might appear with the use of effective core potentials (ECPs) or other nonlocal operators. This is especially relevant for the recently introduced correlation consistent ECPs (ccECPs) for $2s2p$ elements. Corresponding ccECPs exhibit deeper potential functions due to higher fidelity to all-electron counterparts, which could lead to larger local energy fluctuations. We point out that the difficulties stem from issues that are straightforward to address by upgrades of basis sets, use of T-moves for nonlocal terms, inclusion of a few configurations into the trial function and similar. The resulting accuracy corresponds to the ccECP target (chemical accuracy) and it is in consistent agreement with independent correlated calculations. Further possibilities for upgrading the reliability of the DMC algorithm and considerations for better adapted and more robust Jastrow factors are discussed as well.

physics.comp-ph↗

arXiv2021

Anchor-Based Spatio-Temporal Attention 3D Convolutional Networks for Dynamic 3D Point Cloud Sequences

With the rapid development of measurement technology, LiDAR and depth cameras are widely used in the perception of the 3D environment. Recent learning based methods for robot perception most focus on the image or video, but deep learning methods for dynamic 3D point cloud sequences are underexplored. Therefore, developing efficient and accurate perception method compatible with these advanced instruments is pivotal to autonomous driving and service robots. An Anchor-based Spatio-Temporal Attention 3D Convolution operation (ASTA3DConv) is proposed in this paper to process dynamic 3D point cloud sequences. The proposed convolution operation builds a regular receptive field around each point by setting several virtual anchors around each point. The features of neighborhood points are firstly aggregated to each anchor based on the spatio-temporal attention mechanism. Then, anchor-based 3D convolution is adopted to aggregate these anchors' features to the core points. The proposed method makes better use of the structured information within the local region and learns spatio-temporal embedding features from dynamic 3D point cloud sequences. Anchor-based Spatio-Temporal Attention 3D Convolutional Neural Networks (ASTA3DCNNs) are built for classification and segmentation tasks based on the proposed ASTA3DConv and evaluated on action recognition and semantic segmentation tasks. The experiments and ablation studies on MSRAction3D and Synthia datasets demonstrate the superior performance and effectiveness of our method for dynamic 3D point cloud sequences. Our method achieves the state-of-the-art performance among the methods with dynamic 3D point cloud sequences as input on MSRAction3D and Synthia datasets.

cs.CV↗

arXiv2021

NccFlow: Unsupervised Learning of Optical Flow With Non-occlusion from Geometry

Optical flow estimation is a fundamental problem of computer vision and has many applications in the fields of robot learning and autonomous driving. This paper reveals novel geometric laws of optical flow based on the insight and detailed definition of non-occlusion. Then, two novel loss functions are proposed for the unsupervised learning of optical flow based on the geometric laws of non-occlusion. Specifically, after the occlusion part of the images are masked, the flowing process of pixels is carefully considered and geometric constraints are conducted based on the geometric laws of optical flow. First, neighboring pixels in the first frame will not intersect during the pixel displacement to the second frame. Secondly, when the cluster containing adjacent four pixels in the first frame moves to the second frame, no other pixels will flow into the quadrilateral formed by them. According to the two geometrical constraints, the optical flow non-intersection loss and the optical flow non-blocking loss in the non-occlusion regions are proposed. Two loss functions punish the irregular and inexact optical flows in the non-occlusion regions. The experiments on datasets demonstrated that the proposed unsupervised losses of optical flow based on the geometric laws in non-occlusion regions make the estimated optical flow more refined in detail, and improve the performance of unsupervised learning of optical flow. In addition, the experiments training on synthetic data and evaluating on real data show that the generalization ability of optical flow network is improved by our proposed unsupervised approach.

cs.CV↗