Showing 1–50 of 1,134 results for author: Cao, Z

Distributed multi-robot potential-field-based exploration with submap-based mapping and noise-augmented strategy

Authors: Khattiya Pongsirijinda, Zhiqiang Cao, Kaushik Bhowmik, Muhammad Shalihan, Billy Pik Lik Lau, Ran Liu, Chau Yuen, U-Xuan Tan

Abstract: Multi-robot collaboration has become a needed component in unknown environment exploration due to its ability to accomplish various challenging situations. Potential-field-based methods are widely used for autonomous exploration because of their high efficiency and low travel cost. However, exploration speed and collaboration ability are still challenging topics. Therefore, we propose a Distribute… ▽ More Multi-robot collaboration has become a needed component in unknown environment exploration due to its ability to accomplish various challenging situations. Potential-field-based methods are widely used for autonomous exploration because of their high efficiency and low travel cost. However, exploration speed and collaboration ability are still challenging topics. Therefore, we propose a Distributed Multi-Robot Potential-Field-Based Exploration (DMPF-Explore). In particular, we first present a Distributed Submap-Based Multi-Robot Collaborative Mapping Method (DSMC-Map), which can efficiently estimate the robot trajectories and construct the global map by merging the local maps from each robot. Second, we introduce a Potential-Field-Based Exploration Strategy Augmented with Modified Wave-Front Distance and Colored Noises (MWF-CN), in which the accessible frontier neighborhood is extended, and the colored noise provokes the enhancement of exploration performance. The proposed exploration method is deployed for simulation and real-world scenarios. The results show that our approach outperforms the existing ones regarding exploration speed and collaboration ability. △ Less

Submitted 10 July, 2024; originally announced July 2024.

Comments: This paper has been accepted by Robotics and Autonomous Systems

Dynamic Neural Radiance Field From Defocused Monocular Video

Authors: Xianrui Luo, Huiqiang Sun, Juewen Peng, Zhiguo Cao

Abstract: Dynamic Neural Radiance Field (NeRF) from monocular videos has recently been explored for space-time novel view synthesis and achieved excellent results. However, defocus blur caused by depth variation often occurs in video capture, compromising the quality of dynamic reconstruction because the lack of sharp details interferes with modeling temporal consistency between input views. To tackle this… ▽ More Dynamic Neural Radiance Field (NeRF) from monocular videos has recently been explored for space-time novel view synthesis and achieved excellent results. However, defocus blur caused by depth variation often occurs in video capture, compromising the quality of dynamic reconstruction because the lack of sharp details interferes with modeling temporal consistency between input views. To tackle this issue, we propose D2RF, the first dynamic NeRF method designed to restore sharp novel views from defocused monocular videos. We introduce layered Depth-of-Field (DoF) volume rendering to model the defocus blur and reconstruct a sharp NeRF supervised by defocused views. The blur model is inspired by the connection between DoF rendering and volume rendering. The opacity in volume rendering aligns with the layer visibility in DoF rendering.To execute the blurring, we modify the layered blur kernel to the ray-based kernel and employ an optimized sparse kernel to gather the input rays efficiently and render the optimized rays with our layered DoF volume rendering. We synthesize a dataset with defocused dynamic scenes for our task, and extensive experiments on our dataset show that our method outperforms existing approaches in synthesizing all-in-focus novel views from defocus blur while maintaining spatial-temporal consistency in the scene. △ Less

Submitted 7 July, 2024; originally announced July 2024.

Comments: Accepted by ECCV 2024

Explicitly Guided Information Interaction Network for Cross-modal Point Cloud Completion

Authors: Hang Xu, Chen Long, Wenxiao Zhang, Yuan Liu, Zhen Cao, Zhen Dong, Bisheng Yang

Abstract: In this paper, we explore a novel framework, EGIInet (Explicitly Guided Information Interaction Network), a model for View-guided Point cloud Completion (ViPC) task, which aims to restore a complete point cloud from a partial one with a single view image. In comparison with previous methods that relied on the global semantics of input images, EGIInet efficiently combines the information from two m… ▽ More In this paper, we explore a novel framework, EGIInet (Explicitly Guided Information Interaction Network), a model for View-guided Point cloud Completion (ViPC) task, which aims to restore a complete point cloud from a partial one with a single view image. In comparison with previous methods that relied on the global semantics of input images, EGIInet efficiently combines the information from two modalities by leveraging the geometric nature of the completion task. Specifically, we propose an explicitly guided information interaction strategy supported by modal alignment for point cloud completion. First, in contrast to previous methods which simply use 2D and 3D backbones to encode features respectively, we unified the encoding process to promote modal alignment. Second, we propose a novel explicitly guided information interaction strategy that could help the network identify critical information within images, thus achieving better guidance for completion. Extensive experiments demonstrate the effectiveness of our framework, and we achieved a new state-of-the-art (+16% CD over XMFnet) in benchmark datasets despite using fewer parameters than the previous methods. The pre-trained model and code and are available at https://github.com/WHU-USI3DV/EGIInet. △ Less

Submitted 4 July, 2024; v1 submitted 3 July, 2024; originally announced July 2024.

Comments: ECCV 2024

WildAvatar: Web-scale In-the-wild Video Dataset for 3D Avatar Creation

Authors: Zihao Huang, Shoukang Hu, Guangcong Wang, Tianqi Liu, Yuhang Zang, Zhiguo Cao, Wei Li, Ziwei Liu

Abstract: Existing human datasets for avatar creation are typically limited to laboratory environments, wherein high-quality annotations (e.g., SMPL estimation from 3D scans or multi-view images) can be ideally provided. However, their annotating requirements are impractical for real-world images or videos, posing challenges toward real-world applications on current avatar creation methods. To this end, we… ▽ More Existing human datasets for avatar creation are typically limited to laboratory environments, wherein high-quality annotations (e.g., SMPL estimation from 3D scans or multi-view images) can be ideally provided. However, their annotating requirements are impractical for real-world images or videos, posing challenges toward real-world applications on current avatar creation methods. To this end, we propose the WildAvatar dataset, a web-scale in-the-wild human avatar creation dataset extracted from YouTube, with $10,000+$ different human subjects and scenes. WildAvatar is at least $10\times$ richer than previous datasets for 3D human avatar creation. We evaluate several state-of-the-art avatar creation methods on our dataset, highlighting the unexplored challenges in real-world applications on avatar creation. We also demonstrate the potential for generalizability of avatar creation methods, when provided with data at scale. We publicly release our data source links and annotations, to push forward 3D human avatar creation and other related fields for real-world applications. △ Less

Submitted 14 July, 2024; v1 submitted 2 July, 2024; originally announced July 2024.

Comments: Project page: https://wildavatar.github.io/

Pseudo-Labeling by Multi-Policy Viewfinder Network for Image Cropping

Authors: Zhiyu Pan, Kewei Wang, Yizheng Wu, Liwen Xiao, Jiahao Cui, Zhicheng Wang, Zhiguo Cao

Abstract: Automatic image cropping models predict reframing boxes to enhance image aesthetics. Yet, the scarcity of labeled data hinders the progress of this task. To overcome this limitation, we explore the possibility of utilizing both labeled and unlabeled data together to expand the scale of training data for image cropping models. This idea can be implemented in a pseudo-labeling way: producing pseudo… ▽ More Automatic image cropping models predict reframing boxes to enhance image aesthetics. Yet, the scarcity of labeled data hinders the progress of this task. To overcome this limitation, we explore the possibility of utilizing both labeled and unlabeled data together to expand the scale of training data for image cropping models. This idea can be implemented in a pseudo-labeling way: producing pseudo labels for unlabeled data by a teacher model and training a student model with these pseudo labels. However, the student may learn from teacher's mistakes. To address this issue, we propose the multi-policy viewfinder network (MPV-Net) that offers diverse refining policies to rectify the mistakes in original pseudo labels from the teacher. The most reliable policy is selected to generate trusted pseudo labels. The reliability of policies is evaluated via the robustness against box jittering. The efficacy of our method can be evaluated by the improvement compared to the supervised baseline which only uses labeled data. Notably, our MPV-Net outperforms off-the-shelf pseudo-labeling methods, yielding the most substantial improvement over the supervised baseline. Furthermore, our approach achieves state-of-the-art results on both the FCDB and FLMS datasets, signifying the superiority of our approach. △ Less

Submitted 4 July, 2024; v1 submitted 2 July, 2024; originally announced July 2024.

Comments: 18 pages, 8figures

EquiBot: SIM(3)-Equivariant Diffusion Policy for Generalizable and Data Efficient Learning

Authors: Jingyun Yang, Zi-ang Cao, Congyue Deng, Rika Antonova, Shuran Song, Jeannette Bohg

Abstract: Building effective imitation learning methods that enable robots to learn from limited data and still generalize across diverse real-world environments is a long-standing problem in robot learning. We propose EquiBot, a robust, data-efficient, and generalizable approach for robot manipulation task learning. Our approach combines SIM(3)-equivariant neural network architectures with diffusion models… ▽ More Building effective imitation learning methods that enable robots to learn from limited data and still generalize across diverse real-world environments is a long-standing problem in robot learning. We propose EquiBot, a robust, data-efficient, and generalizable approach for robot manipulation task learning. Our approach combines SIM(3)-equivariant neural network architectures with diffusion models. This ensures that our learned policies are invariant to changes in scale, rotation, and translation, enhancing their applicability to unseen environments while retaining the benefits of diffusion-based policy learning such as multi-modality and robustness. We show in a suite of 6 simulation tasks that our proposed method reduces the data requirements and improves generalization to novel scenarios. In the real world, we show with in total 10 variations of 6 mobile manipulation tasks that our method can easily generalize to novel objects and scenes after learning from just 5 minutes of human demonstrations in each task. △ Less

Submitted 1 July, 2024; originally announced July 2024.

Comments: The first two authors contributed equally

DICE: End-to-end Deformation Capture of Hand-Face Interactions from a Single Image

Authors: Qingxuan Wu, Zhiyang Dou, Sirui Xu, Soshi Shimada, Chen Wang, Zhengming Yu, Yuan Liu, Cheng Lin, Zeyu Cao, Taku Komura, Vladislav Golyanik, Christian Theobalt, Wenping Wang, Lingjie Liu

Abstract: Reconstructing 3D hand-face interactions with deformations from a single image is a challenging yet crucial task with broad applications in AR, VR, and gaming. The challenges stem from self-occlusions during single-view hand-face interactions, diverse spatial relationships between hands and face, complex deformations, and the ambiguity of the single-view setting. The first and only method for hand… ▽ More Reconstructing 3D hand-face interactions with deformations from a single image is a challenging yet crucial task with broad applications in AR, VR, and gaming. The challenges stem from self-occlusions during single-view hand-face interactions, diverse spatial relationships between hands and face, complex deformations, and the ambiguity of the single-view setting. The first and only method for hand-face interaction recovery, Decaf, introduces a global fitting optimization guided by contact and deformation estimation networks trained on studio-collected data with 3D annotations. However, Decaf suffers from a time-consuming optimization process and limited generalization capability due to its reliance on 3D annotations of hand-face interaction data. To address these issues, we present DICE, the first end-to-end method for Deformation-aware hand-face Interaction reCovEry from a single image. DICE estimates the poses of hands and faces, contacts, and deformations simultaneously using a Transformer-based architecture. It features disentangling the regression of local deformation fields and global mesh vertex locations into two network branches, enhancing deformation and contact estimation for precise and robust hand-face mesh recovery. To improve generalizability, we propose a weakly-supervised training approach that augments the training set using in-the-wild images without 3D ground-truth annotations, employing the depths of 2D keypoints estimated by off-the-shelf models and adversarial priors of poses for supervision. Our experiments demonstrate that DICE achieves state-of-the-art performance on a standard benchmark and in-the-wild data in terms of accuracy and physical plausibility. Additionally, our method operates at an interactive rate (20 fps) on an Nvidia 4090 GPU, whereas Decaf requires more than 15 seconds for a single image. Our code will be publicly available upon publication. △ Less

Submitted 25 June, 2024; originally announced June 2024.

Comments: 23 pages, 9 figures, 3 tables

Optimising Random Forest Machine Learning Algorithms for User VR Experience Prediction Based on Iterative Local Search-Sparrow Search Algorithm

Authors: Xirui Tang, Feiyang Li, Zinan Cao, Qixuan Yu, Yulu Gong

Abstract: In this paper, an improved method for VR user experience prediction is investigated by introducing a sparrow search algorithm and a random forest algorithm improved by an iterative local search-optimised sparrow search algorithm. The study firstly conducted a statistical analysis of the data, and then trained and tested using the traditional random forest model, the random forest model improved by… ▽ More In this paper, an improved method for VR user experience prediction is investigated by introducing a sparrow search algorithm and a random forest algorithm improved by an iterative local search-optimised sparrow search algorithm. The study firstly conducted a statistical analysis of the data, and then trained and tested using the traditional random forest model, the random forest model improved by the sparrow search algorithm, and the random forest algorithm improved based on the iterative local search-sparrow search algorithm, respectively. The results show that the traditional random forest model has a prediction accuracy of 93% on the training set but only 73.3% on the test set, which is poor in generalisation; whereas the model improved by the sparrow search algorithm has a prediction accuracy of 94% on the test set, which is improved compared with the traditional model. What is more noteworthy is that the improved model based on the iterative local search-sparrow search algorithm achieves 100% accuracy on both the training and test sets, which is significantly better than the other two methods. These research results provide new ideas and methods for VR user experience prediction, especially the improved model based on the iterative local search-sparrow search algorithm performs well and is able to more accurately predict and classify the user's VR experience. In the future, the application of this method in other fields can be further explored, and its effectiveness can be verified through real cases to promote the development of AI technology in the field of user experience. △ Less

Submitted 3 June, 2024; originally announced June 2024.

Instance Consistency Regularization for Semi-Supervised 3D Instance Segmentation

Authors: Yizheng Wu, Zhiyu Pan, Kewei Wang, Xingyi Li, Jiahao Cui, Liwen Xiao, Guosheng Lin, Zhiguo Cao

Abstract: Large-scale datasets with point-wise semantic and instance labels are crucial to 3D instance segmentation but also expensive. To leverage unlabeled data, previous semi-supervised 3D instance segmentation approaches have explored self-training frameworks, which rely on high-quality pseudo labels for consistency regularization. They intuitively utilize both instance and semantic pseudo labels in a j… ▽ More Large-scale datasets with point-wise semantic and instance labels are crucial to 3D instance segmentation but also expensive. To leverage unlabeled data, previous semi-supervised 3D instance segmentation approaches have explored self-training frameworks, which rely on high-quality pseudo labels for consistency regularization. They intuitively utilize both instance and semantic pseudo labels in a joint learning manner. However, semantic pseudo labels contain numerous noise derived from the imbalanced category distribution and natural confusion of similar but distinct categories, which leads to severe collapses in self-training. Motivated by the observation that 3D instances are non-overlapping and spatially separable, we ask whether we can solely rely on instance consistency regularization for improved semi-supervised segmentation. To this end, we propose a novel self-training network InsTeacher3D to explore and exploit pure instance knowledge from unlabeled data. We first build a parallel base 3D instance segmentation model DKNet, which distinguishes each instance from the others via discriminative instance kernels without reliance on semantic segmentation. Based on DKNet, we further design a novel instance consistency regularization framework to generate and leverage high-quality instance pseudo labels. Experimental results on multiple large-scale datasets show that the InsTeacher3D significantly outperforms prior state-of-the-art semi-supervised approaches. Code is available: https://github.com/W1zheng/InsTeacher3D. △ Less

Submitted 24 June, 2024; originally announced June 2024.

Comments: 14 pages, 10 figures

SNR-Progressive Model with Harmonic Compensation for Low-SNR Speech Enhancement

Authors: Zhongshu Hou, Qinwen Hu, Zhanzhong Cao, Ming Tang, Jing Lu

Abstract: Despite significant progress made in the last decade, deep neural network (DNN) based speech enhancement (SE) still faces the challenge of notable degradation in the quality of recovered speech under low signal-to-noise ratio (SNR) conditions. In this letter, we propose an SNR-progressive speech enhancement model with harmonic compensation for low-SNR SE. Reliable pitch estimation is obtained from… ▽ More Despite significant progress made in the last decade, deep neural network (DNN) based speech enhancement (SE) still faces the challenge of notable degradation in the quality of recovered speech under low signal-to-noise ratio (SNR) conditions. In this letter, we propose an SNR-progressive speech enhancement model with harmonic compensation for low-SNR SE. Reliable pitch estimation is obtained from the intermediate output, which has the benefit of retaining more speech components than the coarse estimate while possessing a significant higher SNR than the input noisy speech. An effective harmonic compensation mechanism is introduced for better harmonic recovery. Extensive ex-periments demonstrate the advantage of our proposed model. A multi-modal speech extraction system based on the proposed backbone model ranks first in the ICASSP 2024 MISP Challenge: https://mispchallenge.github.io/mispchallenge2023/index.html. △ Less

Submitted 24 June, 2024; originally announced June 2024.

Extraction of binary neutron star gravitational wave waveforms from Einstein Telescope using deep learning

Authors: Cunliang Ma, Xinyao Yu, Zhoujian Cao, Mingzhen Jia

Abstract: In the future, the third generation (3G) gravitational wave (GW) detectors, exemplified by the Einstein Telescope (ET), will be operational. The detection rate of GW from binary neutron star (BNS) is expected to reach approximately $10^4$ per year. To address the challenges posed by BNS GW data processing for 3G GW detectors, this paper explores the extraction of BNS waveforms from ET. Drawing ins… ▽ More In the future, the third generation (3G) gravitational wave (GW) detectors, exemplified by the Einstein Telescope (ET), will be operational. The detection rate of GW from binary neutron star (BNS) is expected to reach approximately $10^4$ per year. To address the challenges posed by BNS GW data processing for 3G GW detectors, this paper explores the extraction of BNS waveforms from ET. Drawing inspiration from SPIIR's matched filtering approach, we introduce a novel framework leveraging deep learning for BNS waveform extraction. By integrating denoised outputs of time-delayed strain, we can reconstruct the embedded BNS waveform. We have established three distinct BNS GW denoising models, each tailored to address the early inspiral, later inspiral, and merger phases of BNS GW, respectively. To further regulate the waveform shape, we propose the Amplitude Regularity Model that takes denoised output as input and regulated waveform as output. The experiments conducted on test data demonstrate the efficacy of the denoising models, the Amplitude Regularity Models, as well as the overall waveform construction method. To the best of our knowledge, this marks the first instance of deep learning being utilized for the task of BNS waveform extraction. We believe that the proposed method holds promise for early warning, searching, and localization of BNS GWs. △ Less

Submitted 22 June, 2024; originally announced June 2024.

Comments: 14 pages, 16 figures

ICLEval: Evaluating In-Context Learning Ability of Large Language Models

Authors: Wentong Chen, Yankai Lin, ZhenHao Zhou, HongYun Huang, Yantao Jia, Zhao Cao, Ji-Rong Wen

Abstract: In-Context Learning (ICL) is a critical capability of Large Language Models (LLMs) as it empowers them to comprehend and reason across interconnected inputs. Evaluating the ICL ability of LLMs can enhance their utilization and deepen our understanding of how this ability is acquired at the training stage. However, existing evaluation frameworks primarily focus on language abilities and knowledge,… ▽ More In-Context Learning (ICL) is a critical capability of Large Language Models (LLMs) as it empowers them to comprehend and reason across interconnected inputs. Evaluating the ICL ability of LLMs can enhance their utilization and deepen our understanding of how this ability is acquired at the training stage. However, existing evaluation frameworks primarily focus on language abilities and knowledge, often overlooking the assessment of ICL ability. In this work, we introduce the ICLEval benchmark to evaluate the ICL abilities of LLMs, which encompasses two key sub-abilities: exact copying and rule learning. Through the ICLEval benchmark, we demonstrate that ICL ability is universally present in different LLMs, and model size is not the sole determinant of ICL efficacy. Surprisingly, we observe that ICL abilities, particularly copying, develop early in the pretraining process and stabilize afterward. Our source codes and benchmark are released at https://github.com/yiye3/ICLEval. △ Less

Submitted 21 June, 2024; originally announced June 2024.

New QEC codes and EAQEC codes from repeated-root cyclic codes of length $2^rp^s$

Authors: Lanqiang Li, Ziwen Cao, Tingting Wu, Li Liu

Abstract: Let $p$ be an odd prime and $r,s,m$ be positive integers. In this study, we initiate our exploration by delving into the intricate structure of all repeated-root cyclic codes and their duals with a length of $2^rp^s$ over the finite field $\mathbb{F}_{p^m}$. Through the utilization of CSS and Steane's constructions, a series of new quantum error-correcting (QEC) codes are constructed with paramete… ▽ More Let $p$ be an odd prime and $r,s,m$ be positive integers. In this study, we initiate our exploration by delving into the intricate structure of all repeated-root cyclic codes and their duals with a length of $2^rp^s$ over the finite field $\mathbb{F}_{p^m}$. Through the utilization of CSS and Steane's constructions, a series of new quantum error-correcting (QEC) codes are constructed with parameters distinct from all previous constructions. Furthermore, we provide all maximum distance separable (MDS) cyclic codes of length $2^rp^s$, which are further utilized in the construction of QEC MDS codes. Finally, we introduce a significant number of novel entanglement-assisted quantum error-correcting (EAQEC) codes derived from these repeated-root cyclic codes. Notably, these newly constructed codes exhibit parameters distinct from those of previously known constructions. △ Less

Submitted 19 June, 2024; originally announced June 2024.

MSC Class: 94B15 (Primary) 94B05; 11T71(Secondary)

Any360D: Towards 360 Depth Anything with Unlabeled 360 Data and Möbius Spatial Augmentation

Authors: Zidong Cao, Jinjing Zhu, Weiming Zhang, Lin Wang

Abstract: Recently, Depth Anything Model (DAM) - a type of depth foundation model - reveals impressive zero-shot capacity for diverse perspective images. Despite its success, it remains an open question regarding DAM's performance on 360 images that enjoy a large field-of-view (180x360) but suffer from spherical distortions. To this end, we establish, to our knowledge, the first benchmark that aims to 1) ev… ▽ More Recently, Depth Anything Model (DAM) - a type of depth foundation model - reveals impressive zero-shot capacity for diverse perspective images. Despite its success, it remains an open question regarding DAM's performance on 360 images that enjoy a large field-of-view (180x360) but suffer from spherical distortions. To this end, we establish, to our knowledge, the first benchmark that aims to 1) evaluate the performance of DAM on 360 images and 2) develop a powerful 360 DAM for the benefit of the community. For this, we conduct a large suite of experiments that consider the key properties of 360 images, e.g., different 360 representations, various spatial transformations, and diverse indoor and outdoor scenes. This way, our benchmark unveils some key findings, e.g., DAM is less effective for diverse 360 scenes and sensitive to spatial transformations. To address these challenges, we first collect a large-scale unlabeled dataset including diverse indoor and outdoor scenes. We then propose a semi-supervised learning (SSL) framework to learn a 360 DAM, dubbed Any360D. Under the umbrella of SSL, Any360D first learns a teacher model by fine-tuning DAM via metric depth supervision. Then, we train the student model by uncovering the potential of large-scale unlabeled data with pseudo labels from the teacher model. Möbius transformation-based spatial augmentation (MTSA) is proposed to impose consistency regularization between the unlabeled data and spatially transformed ones. This subtly improves the student model's robustness to various spatial transformations even under severe distortions. Extensive experiments demonstrate that Any360D outperforms DAM and many prior data-specific models, e.g., PanoFormer, across diverse scenes, showing impressive zero-shot capacity for being a 360 depth foundation model. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Beam test results of the prototype of the multi wire drift chamber for the CSR external-target experiment

Authors: Zhi Qin, Zhoubo He, Zhe Cao, Tao Chen, Zhi Deng, Limin Duan, Dong Guo, Rongjiang Hu, Jie Kong, Canwen Liu, Peng Ma, Xianglun Wei, Shihai Wen, Xiangjie Wen, Junwei Yan, Herun Yang, Zuoqiao Yang, Yuhong Yu, Zhigang Xiao

Abstract: The half-size prototype of the multi wire drift chamber (MWDC) for the cooling storage ring (CSR) external-target experiment (CEE) was assembled and tested in 350 MeV/u Kr+Fe reactions on the heavy ion research facility in Lanzhou (HIRFL). The prototype consists of 6 sense layers, where the sense wires are stretched in three directions X, U and V, meeting $0^\circ$, $30^\circ$ and $-30^\circ$ with… ▽ More The half-size prototype of the multi wire drift chamber (MWDC) for the cooling storage ring (CSR) external-target experiment (CEE) was assembled and tested in 350 MeV/u Kr+Fe reactions on the heavy ion research facility in Lanzhou (HIRFL). The prototype consists of 6 sense layers, where the sense wires are stretched in three directions X, U and V, meeting $0^\circ$, $30^\circ$ and $-30^\circ$ with respect to the vertical axis, respectively. The sensitive area of the prototype is $76 {\rm cm} \times 76 {\rm cm}$. The amplified and shaped signals from the anode wires are digitized in a serial capacity array. Being operated with 1500 V high voltage on the anode wires, the efficiency for each layer is beyond 95\%. The tracking residual is about $301 \pm 2 \rm μm$. The performance meets the requirements of CEE. △ Less

Submitted 15 May, 2024; originally announced June 2024.

The projected sensitivity of SCEP experiment to Magnetic Monopole

Authors: Changqing Ye, Beige Liu, Zhe Cao, Lingzhi Han, Xinming Huang, Min Jiang, Dong Liu, Qing Lin, Shitian Wan, Yusheng Wu, Lei Zhao, Yue Zhang, Xinhua Peng, Zhengguo Zhao

Abstract: The investigation of beyond-Standard-Model particles is a compelling direction in the pursuit of new physics. One such hypothetical particle, the magnetic monopole, has garnered considerable attention due to its strong theoretical motivation and potential to unveil profound physical phenomena. The magnetic monopole is intricately linked to the long-standing enigma surrounding the quantization of e… ▽ More The investigation of beyond-Standard-Model particles is a compelling direction in the pursuit of new physics. One such hypothetical particle, the magnetic monopole, has garnered considerable attention due to its strong theoretical motivation and potential to unveil profound physical phenomena. The magnetic monopole is intricately linked to the long-standing enigma surrounding the quantization of electric charge. In this manuscript, we propose a novel detection scenario for magnetic monopoles by employing a coincidence measurement technique that combines a room-temperature magnetometer with plastic scintillators. This setup allows for the collection of both the induction and scintillation signals generated by the passage of a monopole. The estimation of the sensitivity using a simple benchmark setup is given. △ Less

Submitted 19 June, 2024; v1 submitted 18 June, 2024; originally announced June 2024.

Monolithic Multi-parameter Terahertz Nano-micro Detector Based on Plasmon Polariton Atomic Cavity

Authors: Huanjun Chen, Ximiao Wang, Shaojing Liu, Zhaolong Cao, Jinyang Li, Hongjia Zhu, Shangdong Li, Ningsheng Xu, Shaozhi Deng

Abstract: Terahertz signals hold significant potential for ultra-wideband communication and high-resolution radar, necessitating miniaturized detectors capable of multi-parameter detection of intensity, frequency, polarization, and phase. Conventional detectors cannot meet these requirements. Here, we propose plasmon polariton atomic cavities (PPAC) made from single-atom-thick graphene, demonstrating the mo… ▽ More Terahertz signals hold significant potential for ultra-wideband communication and high-resolution radar, necessitating miniaturized detectors capable of multi-parameter detection of intensity, frequency, polarization, and phase. Conventional detectors cannot meet these requirements. Here, we propose plasmon polariton atomic cavities (PPAC) made from single-atom-thick graphene, demonstrating the monolithic multifunctional miniaturized detector. With a footprint one-tenth the incident wavelength, the detector offers benchmarking intensity-, frequency-, and polarization-sensitive detection, rapid response, and sub-diffraction spatial resolution, all operating at room temperature across 0.22 to 4.24 THz. We present the monolithic detection applications for free-space THz polarization-coded communication and stealth imaging of physical properties. These results showcase the PPAC's unique ability to achieve strong absorption and weak signal detection with a thickness of only 10^-5 of the excitation wavelength, which is inaccessible with other approaches. △ Less

Submitted 17 June, 2024; originally announced June 2024.

Mask-coding-assisted continuous-variable quantum direct communication with orbital angular momentum multiplexing

Authors: Zhengwen Cao, Yujie Wang, Geng Chai, Xinlei Chen, Yuan Lu

Abstract: Quantum secure direct communication (QSDC) is a approach of communication to transmit secret messages base on quantum mechanics. Different from the quantum key distribution, secret messages can be transmitted directly on quantum channel with QSDC. Higher channel capacity and noise suppression capabilities are key to achieving long-distance quantum communication. Here we report a continuous-variabl… ▽ More Quantum secure direct communication (QSDC) is a approach of communication to transmit secret messages base on quantum mechanics. Different from the quantum key distribution, secret messages can be transmitted directly on quantum channel with QSDC. Higher channel capacity and noise suppression capabilities are key to achieving long-distance quantum communication. Here we report a continuous-variable QSDC scheme based on mask-coding and orbital angular momentum, in which the mask-coding is employed to protect the security of the transmitting messages and to suppression the influence of excess noise. And the combination of orbital angular momentum and the information block transmission can effectively improve the secrecy capacity. In the 800 information blocks * 1310 bits length 10-km experiment, the results show that the statistical average of bit error rate has decreased by 36% and reached 0.38% comparing to existing solutions, and the system excess noise is 0.0184 SNU, and the final secrecy capacity is 6.319 * 106 bps. Therefore, this scheme reduces error bits while increasing secrecy capacity, providing a solution for long-distance large-scale quantum communication. △ Less

Submitted 17 June, 2024; originally announced June 2024.

Quantized Andreev conductance in semiconductor nanowires

Authors: Yichun Gao, Wenyu Song, Yuhao Wang, Zuhan Geng, Zhan Cao, Zehao Yu, Shuai Yang, Jiaye Xu, Fangting Chen, Zonglin Li, Ruidong Li, Lining Yang, Zhaoyu Wang, Shan Zhang, Xiao Feng, Tiantian Wang, Yunyi Zang, Lin Li, Dong E. Liu, Runan Shang, Qi-Kun Xue, Ke He, Hao Zhang

Abstract: Clean one-dimensional electron systems can exhibit quantized conductance. The plateau conductance doubles if the transport is dominated by Andreev reflection. Here, we report quantized conductance observed in both Andreev and normal-state transports in PbTe-Pb and PbTe-In hybrid nanowires. The Andreev plateau is observed at $4e^2/h$, twice of the normal plateau value of $2e^2/h$. In comparison, An… ▽ More Clean one-dimensional electron systems can exhibit quantized conductance. The plateau conductance doubles if the transport is dominated by Andreev reflection. Here, we report quantized conductance observed in both Andreev and normal-state transports in PbTe-Pb and PbTe-In hybrid nanowires. The Andreev plateau is observed at $4e^2/h$, twice of the normal plateau value of $2e^2/h$. In comparison, Andreev conductance in the best-optimized III-V nanowires is non-quantized due to mode-mixing induced dips (a disorder effect), despite the quantization of normal-state transport. The negligible mode mixing in PbTe hybrids indicates an unprecedented low-disorder transport regime for nanowire devices, beneficial for Majorana researches. △ Less

Submitted 17 June, 2024; originally announced June 2024.

PWDFT-SW: Extending the Limit of Plane-Wave DFT Calculations to 16K Atoms on the New Sunway Supercomputer

Authors: Qingcai Jiang, Zhenwei Cao, Junshi Chen, Xinming Qin, Wei Hu, Hong An, Jinlong Yang

Abstract: First-principles density functional theory (DFT) with plane wave (PW) basis set is the most widely used method in quantum mechanical material simulations due to its advantages in accuracy and universality. However, a perceived drawback of PW-based DFT calculations is their substantial computational cost and memory usage, which currently limits their ability to simulate large-scale complex systems… ▽ More First-principles density functional theory (DFT) with plane wave (PW) basis set is the most widely used method in quantum mechanical material simulations due to its advantages in accuracy and universality. However, a perceived drawback of PW-based DFT calculations is their substantial computational cost and memory usage, which currently limits their ability to simulate large-scale complex systems containing thousands of atoms. This situation is exacerbated in the new Sunway supercomputer, where each process is limited to a mere 16 GB of memory. Herein, we present a novel parallel implementation of plane wave density functional theory on the new Sunway supercomputer (PWDFT-SW). PWDFT-SW fully extracts the benefits of Sunway supercomputer by extensively refactoring and calibrating our algorithms to align with the system characteristics of the Sunway system. Through extensive numerical experiments, we demonstrate that our methods can substantially decrease both computational costs and memory usage. Our optimizations translate to a speedup of 64.8x for a physical system containing 4,096 silicon atoms, enabling us to push the limit of PW-based DFT calculations to large-scale systems containing 16,384 carbon atoms. △ Less

Submitted 15 June, 2024; originally announced June 2024.

Constraints on Ultra Heavy Dark Matter Properties from Dwarf Spheroidal Galaxies with LHAASO Observations

Authors: Zhen Cao, F. Aharonian, Q. An, Axikegu, Y. X. Bai, Y. W. Bao, D. Bastieri, X. J. Bi, Y. J. Bi, J. T. Cai, Q. Cao, W. Y. Cao, Zhe Cao, J. Chang, J. F. Chang, A. M. Chen, E. S. Chen, Liang Chen, Lin Chen, Long Chen, M. J. Chen, M. L. Chen, Q. H. Chen, S. H. Chen, S. Z. Chen , et al. (255 additional authors not shown)

Abstract: In this work we try to search for signals generated by ultra-heavy dark matter at the Large High Altitude Air Shower Observatory (LHAASO) data. We look for possible gamma-ray by dark matter annihilation or decay from 16 dwarf spheroidal galaxies in the field of view of LHAASO. Dwarf spheroidal galaxies are among the most promising targets for indirect detection of dark matter which have low fluxes… ▽ More In this work we try to search for signals generated by ultra-heavy dark matter at the Large High Altitude Air Shower Observatory (LHAASO) data. We look for possible gamma-ray by dark matter annihilation or decay from 16 dwarf spheroidal galaxies in the field of view of LHAASO. Dwarf spheroidal galaxies are among the most promising targets for indirect detection of dark matter which have low fluxes of astrophysical $γ$-ray background while large amount of dark matter. By analyzing more than 700 days observational data at LHAASO, no significant dark matter signal from 1 TeV to 1 EeV is detected. Accordingly we derive the most stringent constraints on the ultra-heavy dark matter annihilation cross-section up to EeV. The constraints on the lifetime of dark matter in decay mode are also derived. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: 17 pages, 12 figures, accepted by PRL

dx2-y2-wave Bose Metal induced by the next-nearest-neighbor hopping t'

Authors: Zhangkai Cao, Jianyu Li, Jiahao Su, Tao Ying, Ho-Kin Tang

Abstract: Superconductivity arises when electrons form Cooper pairs with phase coherence. In contrast, a lack of phase coherence in Cooper pairs can lead to an uncondensed metallic ground state known as the Bose metal state. In this study, we investigate an attractively interacting fermionic system with nearest-neighbor (NN) hopping (t) and next-nearest-neighbor (NNN) hopping (t') anisotropy between two spe… ▽ More Superconductivity arises when electrons form Cooper pairs with phase coherence. In contrast, a lack of phase coherence in Cooper pairs can lead to an uncondensed metallic ground state known as the Bose metal state. In this study, we investigate an attractively interacting fermionic system with nearest-neighbor (NN) hopping (t) and next-nearest-neighbor (NNN) hopping (t') anisotropy between two species of spins in a two-dimensional (2D) lattice. Utilizing the constrained path quantum Monte Carlo (CPQMC) method, we demonstrate the existence of a dx2-y2-wave Cooper pair Bose metal (CPBM) phase with t'/t > 0.7. The CPBM phase exhibits a dome-like structure in the phase diagram of filling n~0.65, with the maximal region around an optimal t'/t ~ 0.2, suggesting that an appropriate value of t' facilitates the formation of the Bose metal. Furthermore, we find that a Bose metal formed by fermions with a closed Fermi surface confirms that the crucial condition for this exotic phenomenon is primarily the anisotropy of the Fermi surface, rather than its topology. Our finding of the dx2-y2-wave CPBM demonstrates the same pairing symmetry as the pseudogap behavior in cuprates, and its experimental realization in ultracold atom systems is also feasible. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: 13 pages, 11 figures. arXiv admin note: substantial text overlap with arXiv:2405.13405. arXiv admin note: substantial text overlap with arXiv:2405.13405

AIM: Let Any Multi-modal Large Language Models Embrace Efficient In-Context Learning

Authors: Jun Gao, Qian Qiao, Ziqiang Cao, Zili Wang, Wenjie Li

Abstract: In-context learning (ICL) facilitates Large Language Models (LLMs) exhibiting emergent ability on downstream tasks without updating billions of parameters. However, in the area of multi-modal Large Language Models (MLLMs), two problems hinder the application of multi-modal ICL: (1) Most primary MLLMs are only trained on single-image datasets, making them unable to read multi-modal demonstrations.… ▽ More In-context learning (ICL) facilitates Large Language Models (LLMs) exhibiting emergent ability on downstream tasks without updating billions of parameters. However, in the area of multi-modal Large Language Models (MLLMs), two problems hinder the application of multi-modal ICL: (1) Most primary MLLMs are only trained on single-image datasets, making them unable to read multi-modal demonstrations. (2) With the demonstrations increasing, thousands of visual tokens highly challenge hardware and degrade ICL performance. During preliminary explorations, we discovered that the inner LLM tends to focus more on the linguistic modality within multi-modal demonstrations to generate responses. Therefore, we propose a general and light-weighted framework \textbf{AIM} to tackle the mentioned problems through \textbf{A}ggregating \textbf{I}mage information of \textbf{M}ultimodal demonstrations to the dense latent space of the corresponding linguistic part. Specifically, AIM first uses the frozen backbone MLLM to read each image-text demonstration and extracts the vector representations on top of the text. These vectors naturally fuse the information of the image-text pair, and AIM transforms them into fused virtual tokens acceptable for the inner LLM via a trainable projection layer. Ultimately, these fused tokens function as variants of multi-modal demonstrations, fed into the MLLM to direct its response to the current query as usual. Because these fused tokens stem from the textual component of the image-text pair, a multi-modal demonstration is nearly reduced to a pure textual demonstration, thus seamlessly applying to any MLLMs. With its de facto MLLM frozen, AIM is parameter-efficient and we train it on public multi-modal web corpora which have nothing to do with downstream test tasks. △ Less

Submitted 30 June, 2024; v1 submitted 11 June, 2024; originally announced June 2024.

Efficient k-Nearest-Neighbor Machine Translation with Dynamic Retrieval

Authors: Yan Gao, Zhiwei Cao, Zhongjian Miao, Baosong Yang, Shiyu Liu, Min Zhang, Jinsong Su

Abstract: To achieve non-parametric NMT domain adaptation, $k$-Nearest-Neighbor Machine Translation ($k$NN-MT) constructs an external datastore to store domain-specific translation knowledge, which derives a $k$NN distribution to interpolate the prediction distribution of the NMT model via a linear interpolation coefficient $λ$. Despite its success, $k$NN retrieval at each timestep leads to substantial time… ▽ More To achieve non-parametric NMT domain adaptation, $k$-Nearest-Neighbor Machine Translation ($k$NN-MT) constructs an external datastore to store domain-specific translation knowledge, which derives a $k$NN distribution to interpolate the prediction distribution of the NMT model via a linear interpolation coefficient $λ$. Despite its success, $k$NN retrieval at each timestep leads to substantial time overhead. To address this issue, dominant studies resort to $k$NN-MT with adaptive retrieval ($k$NN-MT-AR), which dynamically estimates $λ$ and skips $k$NN retrieval if $λ$ is less than a fixed threshold. Unfortunately, $k$NN-MT-AR does not yield satisfactory results. In this paper, we first conduct a preliminary study to reveal two key limitations of $k$NN-MT-AR: 1) the optimization gap leads to inaccurate estimation of $λ$ for determining $k$NN retrieval skipping, and 2) using a fixed threshold fails to accommodate the dynamic demands for $k$NN retrieval at different timesteps. To mitigate these limitations, we then propose $k$NN-MT with dynamic retrieval ($k$NN-MT-DR) that significantly extends vanilla $k$NN-MT in two aspects. Firstly, we equip $k$NN-MT with a MLP-based classifier for determining whether to skip $k$NN retrieval at each timestep. Particularly, we explore several carefully-designed scalar features to fully exert the potential of the classifier. Secondly, we propose a timestep-aware threshold adjustment method to dynamically generate the threshold, which further improves the efficiency of our model. Experimental results on the widely-used datasets demonstrate the effectiveness and generality of our model.\footnote{Our code is available at \url{https://github.com/DeepLearnXMU/knn-mt-dr}. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Comments: Accepted to ACL 2024 Findings

ProMotion: Prototypes As Motion Learners

Authors: Yawen Lu, Dongfang Liu, Qifan Wang, Cheng Han, Yiming Cui, Zhiwen Cao, Xueling Zhang, Yingjie Victor Chen, Heng Fan

Abstract: In this work, we introduce ProMotion, a unified prototypical framework engineered to model fundamental motion tasks. ProMotion offers a range of compelling attributes that set it apart from current task-specific paradigms. We adopt a prototypical perspective, establishing a unified paradigm that harmonizes disparate motion learning approaches. This novel paradigm streamlines the architectural desi… ▽ More In this work, we introduce ProMotion, a unified prototypical framework engineered to model fundamental motion tasks. ProMotion offers a range of compelling attributes that set it apart from current task-specific paradigms. We adopt a prototypical perspective, establishing a unified paradigm that harmonizes disparate motion learning approaches. This novel paradigm streamlines the architectural design, enabling the simultaneous assimilation of diverse motion information. We capitalize on a dual mechanism involving the feature denoiser and the prototypical learner to decipher the intricacies of motion. This approach effectively circumvents the pitfalls of ambiguity in pixel-wise feature matching, significantly bolstering the robustness of motion representation. We demonstrate a profound degree of transferability across distinct motion patterns. This inherent versatility reverberates robustly across a comprehensive spectrum of both 2D and 3D downstream tasks. Empirical results demonstrate that ProMotion outperforms various well-known specialized architectures, achieving 0.54 and 0.054 Abs Rel error on the Sintel and KITTI depth datasets, 1.04 and 2.01 average endpoint error on the clean and final pass of Sintel flow benchmark, and 4.30 F1-all error on the KITTI flow benchmark. For its efficacy, we hope our work can catalyze a paradigm shift in universal models in computer vision. △ Less

Submitted 7 June, 2024; originally announced June 2024.

Comments: 11 pages

Mini Honor of Kings: A Lightweight Environment for Multi-Agent Reinforcement Learning

Authors: Lin Liu, Jian Zhao, Cheng Hu, Zhengtao Cao, Youpeng Zhao, Zhenbin Ye, Meng Meng, Wenjun Wang, Zhaofeng He, Houqiang Li, Xia Lin, Lanxiao Huang

Abstract: Games are widely used as research environments for multi-agent reinforcement learning (MARL), but they pose three significant challenges: limited customization, high computational demands, and oversimplification. To address these issues, we introduce the first publicly available map editor for the popular mobile game Honor of Kings and design a lightweight environment, Mini Honor of Kings (Mini Ho… ▽ More Games are widely used as research environments for multi-agent reinforcement learning (MARL), but they pose three significant challenges: limited customization, high computational demands, and oversimplification. To address these issues, we introduce the first publicly available map editor for the popular mobile game Honor of Kings and design a lightweight environment, Mini Honor of Kings (Mini HoK), for researchers to conduct experiments. Mini HoK is highly efficient, allowing experiments to be run on personal PCs or laptops while still presenting sufficient challenges for existing MARL algorithms. We have tested our environment on common MARL algorithms and demonstrated that these algorithms have yet to find optimal solutions within this environment. This facilitates the dissemination and advancement of MARL methods within the research community. Additionally, we hope that more researchers will leverage the Honor of Kings map editor to develop innovative and scientifically valuable new maps. Our code and user manual are available at: https://github.com/tencent-ailab/mini-hok. △ Less

Submitted 16 June, 2024; v1 submitted 6 June, 2024; originally announced June 2024.

Retaining Key Information under High Compression Ratios: Query-Guided Compressor for LLMs

Authors: Zhiwei Cao, Qian Cao, Yu Lu, Ningxin Peng, Luyang Huang, Shanbo Cheng, Jinsong Su

Abstract: The growing popularity of Large Language Models has sparked interest in context compression for Large Language Models (LLMs). However, the performance of previous methods degrades dramatically as compression ratios increase, sometimes even falling to the closed-book level. This decline can be attributed to the loss of key information during the compression process. Our preliminary study supports t… ▽ More The growing popularity of Large Language Models has sparked interest in context compression for Large Language Models (LLMs). However, the performance of previous methods degrades dramatically as compression ratios increase, sometimes even falling to the closed-book level. This decline can be attributed to the loss of key information during the compression process. Our preliminary study supports this hypothesis, emphasizing the significance of retaining key information to maintain model performance under high compression ratios. As a result, we introduce Query-Guided Compressor (QGC), which leverages queries to guide the context compression process, effectively preserving key information within the compressed context. Additionally, we employ a dynamic compression strategy. We validate the effectiveness of our proposed QGC on the Question Answering task, including NaturalQuestions, TriviaQA, and HotpotQA datasets. Experimental results show that QGC can consistently perform well even at high compression ratios, which also offers significant benefits in terms of inference cost and throughput. △ Less

Submitted 17 June, 2024; v1 submitted 4 June, 2024; originally announced June 2024.

Comments: Accepted to ACL 2024

Prototypical Transformer as Unified Motion Learners

Authors: Cheng Han, Yawen Lu, Guohao Sun, James C. Liang, Zhiwen Cao, Qifan Wang, Qiang Guan, Sohail A. Dianat, Raghuveer M. Rao, Tong Geng, Zhiqiang Tao, Dongfang Liu

Abstract: In this work, we introduce the Prototypical Transformer (ProtoFormer), a general and unified framework that approaches various motion tasks from a prototype perspective. ProtoFormer seamlessly integrates prototype learning with Transformer by thoughtfully considering motion dynamics, introducing two innovative designs. First, Cross-Attention Prototyping discovers prototypes based on signature moti… ▽ More In this work, we introduce the Prototypical Transformer (ProtoFormer), a general and unified framework that approaches various motion tasks from a prototype perspective. ProtoFormer seamlessly integrates prototype learning with Transformer by thoughtfully considering motion dynamics, introducing two innovative designs. First, Cross-Attention Prototyping discovers prototypes based on signature motion patterns, providing transparency in understanding motion scenes. Second, Latent Synchronization guides feature representation learning via prototypes, effectively mitigating the problem of motion uncertainty. Empirical results demonstrate that our approach achieves competitive performance on popular motion tasks such as optical flow and scene depth. Furthermore, it exhibits generality across various downstream tasks, including object tracking and video stabilization. △ Less

Submitted 3 June, 2024; originally announced June 2024.

Comments: 21 pages, 10 figures

Guiding ChatGPT to Generate Salient Domain Summaries

Authors: Jun Gao, Ziqiang Cao, Shaoyao Huang, Luozheng Qin, Chunhui Ai

Abstract: ChatGPT is instruct-tuned to generate general and human-expected content to align with human preference through Reinforcement Learning from Human Feedback (RLHF), meanwhile resulting in generated responses not salient enough. Therefore, in this case, ChatGPT may fail to satisfy domain requirements in zero-shot settings, leading to poor ROUGE scores. Inspired by the In-Context Learning (ICL) and re… ▽ More ChatGPT is instruct-tuned to generate general and human-expected content to align with human preference through Reinforcement Learning from Human Feedback (RLHF), meanwhile resulting in generated responses not salient enough. Therefore, in this case, ChatGPT may fail to satisfy domain requirements in zero-shot settings, leading to poor ROUGE scores. Inspired by the In-Context Learning (ICL) and retelling ability of ChatGPT, this paper proposes PADS, a \textbf{P}ipeline for \textbf{A}ssisting ChatGPT in \textbf{D}omain \textbf{S}ummarization. PADS consists of a retriever to retrieve similar examples from corpora and a rank model to rerank the multiple candidate summaries generated by ChatGPT. Specifically, given an inference document, we first retrieve an in-context demonstration via the retriever. Then, we require ChatGPT to generate $k$ candidate summaries for the inference document at a time under the guidance of the retrieved demonstration. Finally, the rank model independently scores the $k$ candidate summaries according to their quality and selects the optimal one. We extensively explore dense and sparse retrieval methods to select effective demonstrations for reference and efficiently train the rank model to reflect the quality of candidate summaries for each given summarized document. Additionally, PADS contains merely 400M trainable parameters originating from the rank model and we merely collect 2.5k data to train it. We evaluate PADS on five datasets from different domains, and the result indicates that each module in PADS is committed to effectively guiding ChatGPT to generate salient summaries fitting different domain requirements. Specifically, in the popular summarization dataset Gigaword, PADS achieves over +8 gain on ROUGE-L, compared with the naive ChatGPT in the zero-shot setting. \footnote{Our code are available at \url{https://github.com/jungao1106/PADS}} △ Less

Submitted 3 June, 2024; originally announced June 2024.

All-sky Guide Star Catalog for CSST

Authors: Hui-Mei Feng, Zi-Huang Cao, Man I Lam, Ran Li, Hao Tian, Da-Yi Yin, Yuan-Yu Yang, Xin Zhang, Dong-Wei Fan, Yi-Qiao Dong, Xin-Feng Li, Wei Wang, Long Li, Hugh R. A. Jones, Yi-Han Tao, Jia-Lu Nie, Pei-Pei Wang, Mao-Yuan Liu, He-jun Yang, Chao Liu

Abstract: The China Space Station Telescope (CSST) is a two-meter space telescope with multiple back-end instruments. The Fine Guidance Sensor (FGS) is an essential subsystem of the CSST Precision Image Stability System to ensure the required absolute pointing accuracy and line-of-sight stabilization. In this study, we construct the Main Guide Star Catalog for FGS. To accomplish this, we utilize the informa… ▽ More The China Space Station Telescope (CSST) is a two-meter space telescope with multiple back-end instruments. The Fine Guidance Sensor (FGS) is an essential subsystem of the CSST Precision Image Stability System to ensure the required absolute pointing accuracy and line-of-sight stabilization. In this study, we construct the Main Guide Star Catalog for FGS. To accomplish this, we utilize the information about the FGS and object information from the Gaia Data Release 3. We provide an FGS instrument magnitude and exclude variables, binaries, and high proper motion stars from the catalog to ensure uniform FGS guidance capabilities. Subsequently, we generate a HEALPix index, which provides a hierarchical tessellation of the celestial sphere, and employ the Voronoi algorithm to achieve a homogeneous distribution of stars across the catalog. This distribution ensures adequate coverage and sampling of the sky. The performance of the CSST guide star catalog was assessed by simulating the field of view of the FGS according to the CSST mock survey strategy catalog. The analysis of the results indicates that this catalog provides adequate coverage and accuracy. The catalog's performance meets the FGS requirements, ensuring the functioning of the FGS and its guidance capabilities. △ Less

Submitted 3 June, 2024; originally announced June 2024.

Comments: published on RAA

Prompt Chaining or Stepwise Prompt? Refinement in Text Summarization

Authors: Shichao Sun, Ruifeng Yuan, Ziqiang Cao, Wenjie Li, Pengfei Liu

Abstract: Large language models (LLMs) have demonstrated the capacity to improve summary quality by mirroring a human-like iterative process of critique and refinement starting from the initial draft. Two strategies are designed to perform this iterative process: Prompt Chaining and Stepwise Prompt. Prompt chaining orchestrates the drafting, critiquing, and refining phases through a series of three discrete… ▽ More Large language models (LLMs) have demonstrated the capacity to improve summary quality by mirroring a human-like iterative process of critique and refinement starting from the initial draft. Two strategies are designed to perform this iterative process: Prompt Chaining and Stepwise Prompt. Prompt chaining orchestrates the drafting, critiquing, and refining phases through a series of three discrete prompts, while Stepwise prompt integrates these phases within a single prompt. However, the relative effectiveness of the two methods has not been extensively studied. This paper is dedicated to examining and comparing these two methods in the context of text summarization to ascertain which method stands out as the most effective. Experimental results show that the prompt chaining method can produce a more favorable outcome. This might be because stepwise prompt might produce a simulated refinement process according to our various experiments. Since refinement is adaptable to diverse tasks, our conclusions have the potential to be extrapolated to other applications, thereby offering insights that may contribute to the broader development of LLMs. △ Less

Submitted 1 June, 2024; originally announced June 2024.

Comments: Accepted to Findings of ACL 2024

Deciphering Human Mobility: Inferring Semantics of Trajectories with Large Language Models

Authors: Yuxiao Luo, Zhongcai Cao, Xin Jin, Kang Liu, Ling Yin

Abstract: Understanding human mobility patterns is essential for various applications, from urban planning to public safety. The individual trajectory such as mobile phone location data, while rich in spatio-temporal information, often lacks semantic detail, limiting its utility for in-depth mobility analysis. Existing methods can infer basic routine activity sequences from this data, lacking depth in under… ▽ More Understanding human mobility patterns is essential for various applications, from urban planning to public safety. The individual trajectory such as mobile phone location data, while rich in spatio-temporal information, often lacks semantic detail, limiting its utility for in-depth mobility analysis. Existing methods can infer basic routine activity sequences from this data, lacking depth in understanding complex human behaviors and users' characteristics. Additionally, they struggle with the dependency on hard-to-obtain auxiliary datasets like travel surveys. To address these limitations, this paper defines trajectory semantic inference through three key dimensions: user occupation category, activity sequence, and trajectory description, and proposes the Trajectory Semantic Inference with Large Language Models (TSI-LLM) framework to leverage LLMs infer trajectory semantics comprehensively and deeply. We adopt spatio-temporal attributes enhanced data formatting (STFormat) and design a context-inclusive prompt, enabling LLMs to more effectively interpret and infer the semantics of trajectory data. Experimental validation on real-world trajectory datasets demonstrates the efficacy of TSI-LLM in deciphering complex human mobility patterns. This study explores the potential of LLMs in enhancing the semantic analysis of trajectory data, paving the way for more sophisticated and accessible human mobility research. △ Less

Submitted 30 May, 2024; originally announced May 2024.

Universal deterministic patterns in stochastic count data

Authors: Zhixing Cao, Yiling Wang, Ramon Grima

Abstract: We report the existence of deterministic patterns in plots showing the relationship between the mean and the Fano factor (ratio of variance and mean) of stochastic count data. These patterns are found in a wide variety of datasets, including those from genomics, paper citations, commerce, ecology, disease outbreaks, and employment statistics. We develop a theory showing that the patterns naturally… ▽ More We report the existence of deterministic patterns in plots showing the relationship between the mean and the Fano factor (ratio of variance and mean) of stochastic count data. These patterns are found in a wide variety of datasets, including those from genomics, paper citations, commerce, ecology, disease outbreaks, and employment statistics. We develop a theory showing that the patterns naturally emerge when data sampled from discrete probability distributions is organised in matrix form. The theory precisely predicts the patterns and shows that they are a function of only one variable - the sample size. △ Less

Submitted 27 May, 2024; originally announced May 2024.

Comments: 13 pages, 5 figures

Unifying Demonstration Selection and Compression for In-Context Learning

Authors: Jun Gao, Ziqiang Cao, Wenjie Li

Abstract: In-context learning (ICL) facilitates large language models (LLMs) exhibiting spectacular emergent capabilities in various scenarios. Unfortunately, introducing demonstrations easily makes the prompt length explode, bringing a significant burden to hardware. In addition, random demonstrations usually achieve limited improvements in ICL, necessitating demonstration selection among accessible candid… ▽ More In-context learning (ICL) facilitates large language models (LLMs) exhibiting spectacular emergent capabilities in various scenarios. Unfortunately, introducing demonstrations easily makes the prompt length explode, bringing a significant burden to hardware. In addition, random demonstrations usually achieve limited improvements in ICL, necessitating demonstration selection among accessible candidates. Previous studies introduce extra modules to perform demonstration compression or selection independently. In this paper, we propose an ICL framework UniICL, which Unifies demonstration selection and compression, and final response generation via a single frozen LLM. Specifically, UniICL first projects actual demonstrations and inference text inputs into short virtual tokens, respectively. Then, virtual tokens are applied to select suitable demonstrations by measuring semantic similarity within latent space among candidate demonstrations and inference input. Finally, inference text inputs together with selected virtual demonstrations are fed into the same frozen LLM for response generation. Notably, UniICL is a parameter-efficient framework that only contains 17M trainable parameters originating from the projection layer. We conduct experiments and analysis over in- and out-domain datasets of both generative and understanding tasks, encompassing ICL scenarios with plentiful and limited demonstration candidates. Results show that UniICL effectively unifies $12 \times$ compression, demonstration selection, and response generation, efficiently scaling up the baseline from 4-shot to 64-shot ICL in IMDb with 24 GB CUDA allocation △ Less

Submitted 15 June, 2024; v1 submitted 27 May, 2024; originally announced May 2024.

SelfCP: Compressing Over-Limit Prompt via the Frozen Large Language Model Itself

Authors: Jun Gao, Ziqiang Cao, Wenjie Li

Abstract: Long prompt leads to huge hardware costs when using transformer-based Large Language Models (LLMs). Unfortunately, many tasks, such as summarization, inevitably introduce long documents, and the wide application of in-context learning easily makes the prompt length explode. This paper proposes a Self-Compressor (SelfCP), which employs the target LLM itself to compress over-limit prompts into dense… ▽ More Long prompt leads to huge hardware costs when using transformer-based Large Language Models (LLMs). Unfortunately, many tasks, such as summarization, inevitably introduce long documents, and the wide application of in-context learning easily makes the prompt length explode. This paper proposes a Self-Compressor (SelfCP), which employs the target LLM itself to compress over-limit prompts into dense vectors while keeping the allowed prompts unmodified. Dense vectors are then projected into dense tokens via a learnable connector to make the same LLM unburden to understand. The connector is supervised-tuned under the language modeling objective of the LLM on relatively long texts selected from publicly accessed datasets, involving an instruction dataset to make SelfCP respond to various prompts, while the target LLM keeps frozen during training. We build the lightweight SelfCP upon 2 different backbones with merely 17M learnable parameters originating from the connector and a learnable embedding. Evaluation on both English and Chinese benchmarks demonstrate that SelfCP effectively substitutes 12$\times$ over-limit prompts with dense tokens to reduce memory costs and booster inference throughputs, yet improving response quality. The outstanding performance brings an efficient solution for LLMs to tackle long prompts without training LLMs from scratch. △ Less

Submitted 18 June, 2024; v1 submitted 27 May, 2024; originally announced May 2024.

AutoCV: Empowering Reasoning with Automated Process Labeling via Confidence Variation

Authors: Jianqiao Lu, Zhiyang Dou, Hongru Wang, Zeyu Cao, Jianbo Dai, Yingjia Wan, Yinya Huang, Zhijiang Guo

Abstract: In this work, we propose a novel method named \textbf{Auto}mated Process Labeling via \textbf{C}onfidence \textbf{V}ariation (\textbf{\textsc{AutoCV}}) to enhance the reasoning capabilities of large language models (LLMs) by automatically annotating the reasoning steps. Our approach begins by training a verification model on the correctness of final answers, enabling it to generate automatic proce… ▽ More In this work, we propose a novel method named \textbf{Auto}mated Process Labeling via \textbf{C}onfidence \textbf{V}ariation (\textbf{\textsc{AutoCV}}) to enhance the reasoning capabilities of large language models (LLMs) by automatically annotating the reasoning steps. Our approach begins by training a verification model on the correctness of final answers, enabling it to generate automatic process annotations. This verification model assigns a confidence score to each reasoning step, indicating the probability of arriving at the correct final answer from that point onward. We detect relative changes in the verification's confidence scores across reasoning steps to automatically annotate the reasoning process. This alleviates the need for numerous manual annotations or the high computational costs associated with model-induced annotation approaches. We experimentally validate that the confidence variations learned by the verification model trained on the final answer correctness can effectively identify errors in the reasoning steps. Subsequently, we demonstrate that the process annotations generated by \textsc{AutoCV} can improve the accuracy of the verification model in selecting the correct answer from multiple outputs generated by LLMs. Notably, we achieve substantial improvements across five datasets in mathematics and commonsense reasoning. The source code of \textsc{AutoCV} is available at \url{https://github.com/rookie-joe/AUTOCV}. △ Less

Submitted 28 May, 2024; v1 submitted 26 May, 2024; originally announced May 2024.

Comments: 20 pages, 1 figure, 13 tables

Exotic d-wave Bose Metal in two dimensions

Authors: Zhangkai Cao, Jiahao Su, Jianyu Li, Tao Ying, WanSheng Wang, Jin-Hua Sun, Ho-Kin Tang, Haiqing Lin

Abstract: The Landau Fermi liquid theory, a cornerstone in condensed matter physics, encounters limitations in explaining certain phenomena, like the peculiar behavior of strange metals in high-temperature superconductors. Non-Fermi liquids, like Bose metals with uncondensed bosonic ground state, offer potential explanations, yet constructing an elusive Bose metal phase in two dimensions (2D) remains a form… ▽ More The Landau Fermi liquid theory, a cornerstone in condensed matter physics, encounters limitations in explaining certain phenomena, like the peculiar behavior of strange metals in high-temperature superconductors. Non-Fermi liquids, like Bose metals with uncondensed bosonic ground state, offer potential explanations, yet constructing an elusive Bose metal phase in two dimensions (2D) remains a formidable challenge. Utilizing constraint path quantum Monte Carlo and functional renormalization group methods on a fermionic system with spin anisotropy in a 2D lattice, we reveal the emergence of a Cooper pair Bose metal in a highly anisotropic regime (a < 0.30) with wide range of filling, most notably at a filling fraction of n~0.8. Our findings exhibit a visible nonzero momentum Bose surface in the Cooper-pair distribution function, accompanied by a distinct signal of dxy correlation between pairs. Our results highlight that spin-dependent anisotropy in the Fermi surface leads to versatile pairing forms. Platforms such as ultracold atoms in optical lattices and recently proposed altermagnets hold promise for realizing this intriguing phase. △ Less

Submitted 24 May, 2024; v1 submitted 22 May, 2024; originally announced May 2024.

Comments: 15 pages, 13 figures

An Improved Robust Total Logistic Distance Metric algorithm for Generalized Gaussian Noise and Noisy Input

Authors: Haiquan Zhao, Yi Peng, Zian Cao

Abstract: Although the known maximum total generalized correntropy (MTGC) and generalized maximum blakezisserman total correntropy (GMBZTC) algorithms can maintain good performance under the errors-in-variables (EIV) model disrupted by generalized Gaussian noise, their requirement for manual ad-justment of parameters is excessive, greatly increasing the practical difficulty of use. To solve this problem, th… ▽ More Although the known maximum total generalized correntropy (MTGC) and generalized maximum blakezisserman total correntropy (GMBZTC) algorithms can maintain good performance under the errors-in-variables (EIV) model disrupted by generalized Gaussian noise, their requirement for manual ad-justment of parameters is excessive, greatly increasing the practical difficulty of use. To solve this problem, the total arctangent based on logical distance metric (TACLDM) algo-rithm is proposed by utilizing the advantage of few parameters in logical distance metric (LDM) theory and the convergence behavior is improved by the arctangent function. Compared with other competing algorithms, the TACLDM algorithm not only has fewer parameters, but also has better robustness to generalized Gaussian noise and significantly reduces the steady-state error. Furthermore, the analysis of the algorithm in the generalized Gaussian noise environment is analyzed in detail in this paper. Finally, computer simulations demonstrate the outstanding performance of the TACLDM algorithm and the rigorous theoretical deduction in this paper. △ Less

Submitted 21 May, 2024; originally announced May 2024.

Comments: 10 page

MSC Class: 94 ACM Class: C.2; F.2; H.4

MVSGaussian: Fast Generalizable Gaussian Splatting Reconstruction from Multi-View Stereo

Authors: Tianqi Liu, Guangcong Wang, Shoukang Hu, Liao Shen, Xinyi Ye, Yuhang Zang, Zhiguo Cao, Wei Li, Ziwei Liu

Abstract: We present MVSGaussian, a new generalizable 3D Gaussian representation approach derived from Multi-View Stereo (MVS) that can efficiently reconstruct unseen scenes. Specifically, 1) we leverage MVS to encode geometry-aware Gaussian representations and decode them into Gaussian parameters. 2) To further enhance performance, we propose a hybrid Gaussian rendering that integrates an efficient volume… ▽ More We present MVSGaussian, a new generalizable 3D Gaussian representation approach derived from Multi-View Stereo (MVS) that can efficiently reconstruct unseen scenes. Specifically, 1) we leverage MVS to encode geometry-aware Gaussian representations and decode them into Gaussian parameters. 2) To further enhance performance, we propose a hybrid Gaussian rendering that integrates an efficient volume rendering design for novel view synthesis. 3) To support fast fine-tuning for specific scenes, we introduce a multi-view geometric consistent aggregation strategy to effectively aggregate the point clouds generated by the generalizable model, serving as the initialization for per-scene optimization. Compared with previous generalizable NeRF-based methods, which typically require minutes of fine-tuning and seconds of rendering per image, MVSGaussian achieves real-time rendering with better synthesis quality for each scene. Compared with the vanilla 3D-GS, MVSGaussian achieves better view synthesis with less training computational cost. Extensive experiments on DTU, Real Forward-facing, NeRF Synthetic, and Tanks and Temples datasets validate that MVSGaussian attains state-of-the-art performance with convincing generalizability, real-time rendering speed, and fast per-scene optimization. △ Less

Submitted 15 July, 2024; v1 submitted 20 May, 2024; originally announced May 2024.

Comments: ECCV2024, Project page: https://mvsgaussian.github.io/ , Code: https://github.com/TQTQliu/MVSGaussian

Data quality control system and long-term performance monitor of the LHAASO-KM2A

Authors: Zhen Cao, F. Aharonian, Axikegu, Y. X. Bai, Y. W. Bao, D. Bastieri, X. J. Bi, Y. J. Bi, W. Bian, A. V. Bukevich, Q. Cao, W. Y. Cao, Zhe Cao, J. Chang, J. F. Chang, A. M. Chen, E. S. Chen, H. X. Chen, Liang Chen, Lin Chen, Long Chen, M. J. Chen, M. L. Chen, Q. H. Chen, S. Chen , et al. (263 additional authors not shown)

Abstract: The KM2A is the largest sub-array of the Large High Altitude Air Shower Observatory (LHAASO). It consists of 5216 electromagnetic particle detectors (EDs) and 1188 muon detectors (MDs). The data recorded by the EDs and MDs are used to reconstruct primary information of cosmic ray and gamma-ray showers. This information is used for physical analysis in gamma-ray astronomy and cosmic ray physics. To… ▽ More The KM2A is the largest sub-array of the Large High Altitude Air Shower Observatory (LHAASO). It consists of 5216 electromagnetic particle detectors (EDs) and 1188 muon detectors (MDs). The data recorded by the EDs and MDs are used to reconstruct primary information of cosmic ray and gamma-ray showers. This information is used for physical analysis in gamma-ray astronomy and cosmic ray physics. To ensure the reliability of the LHAASO-KM2A data, a three-level quality control system has been established. It is used to monitor the status of detector units, stability of reconstructed parameters and the performance of the array based on observations of the Crab Nebula and Moon shadow. This paper will introduce the control system and its application on the LHAASO-KM2A data collected from August 2021 to July 2023. During this period, the pointing and angular resolution of the array were stable. From the observations of the Moon shadow and Crab Nebula, the results achieved using the two methods are consistent with each other. According to the observation of the Crab Nebula at energies from 25 TeV to 100 TeV, the time averaged pointing errors are estimated to be $-0.003^{\circ} \pm 0.005^{\circ}$ and $0.001^{\circ} \pm 0.006^{\circ}$ in the R.A. and Dec directions, respectively. △ Less

Submitted 13 June, 2024; v1 submitted 20 May, 2024; originally announced May 2024.

Comments: 15 pages, 9 figures

CRF360D: Monocular 360 Depth Estimation via Spherical Fully-Connected CRFs

Authors: Zidong Cao, Lin Wang

Abstract: Monocular 360 depth estimation is challenging due to the inherent distortion of the equirectangular projection (ERP). This distortion causes a problem: spherical adjacent points are separated after being projected to the ERP plane, particularly in the polar regions. To tackle this problem, recent methods calculate the spherical neighbors in the tangent domain. However, as the tangent patch and sph… ▽ More Monocular 360 depth estimation is challenging due to the inherent distortion of the equirectangular projection (ERP). This distortion causes a problem: spherical adjacent points are separated after being projected to the ERP plane, particularly in the polar regions. To tackle this problem, recent methods calculate the spherical neighbors in the tangent domain. However, as the tangent patch and sphere only have one common point, these methods construct neighboring spherical relationships around the common point. In this paper, we propose spherical fully-connected CRFs (SF-CRFs). We begin by evenly partitioning an ERP image with regular windows, where windows at the equator involve broader spherical neighbors than those at the poles. To improve the spherical relationships, our SF-CRFs enjoy two key components. Firstly, to involve sufficient spherical neighbors, we propose a Spherical Window Transform (SWT) module. This module aims to replicate the equator window's spherical relationships to all other windows, leveraging the rotational invariance of the sphere. Remarkably, the transformation process is highly efficient, completing the transformation of all windows in a 512X1024 ERP with 0.038 seconds on CPU. Secondly, we propose a Planar-Spherical Interaction (PSI) module to facilitate the relationships between regular and transformed windows, which not only preserves the local details but also captures global structures. By building a decoder based on the SF-CRFs blocks, we propose CRF360D, a novel 360 depth estimation framework that achieves state-of-the-art performance across diverse datasets. Our CRF360D is compatible with different perspective image-trained backbones (e.g., EfficientNet), serving as the encoder. △ Less

Submitted 19 May, 2024; originally announced May 2024.

High Discrimination Ratio, Broadband Circularly Polarized Light Photodetector Using Dielectric Achiral Nanostructures

Authors: Guanyu Zhang, Xiaying Lyu, Yulu Qin, Yaolong Li, Zipu Fan, Xianghan Meng, Yuqing Cheng, Zini Cao, Yixuan Xu, Dong Sun, Yunan Gao, Qihuang Gong, Guowei Lu

Abstract: The on-chip measurement of polarization states plays an increasingly crucial role in modern sensing and imaging applications. While high-performance monolithic linearly polarized photodetectors have been extensively studied, integrated circularly polarized light (CPL) photodetectors are still hindered by inadequate discrimination capability. In this study, we employ achiral all-dielectric nanostru… ▽ More The on-chip measurement of polarization states plays an increasingly crucial role in modern sensing and imaging applications. While high-performance monolithic linearly polarized photodetectors have been extensively studied, integrated circularly polarized light (CPL) photodetectors are still hindered by inadequate discrimination capability. In this study, we employ achiral all-dielectric nanostructures to develop a broadband CPL photodetector with an impressive discrimination ratio of ~107 at the wavelength of 405 nm, significantly surpassing its counterparts by two orders of magnitude. Our device shows outstanding CPL discrimination capability across the visible band without requiring intensity calibration. Its function mechanism is based on the CPL-dependent near-field modes within achiral structures: under left or right CPL illumination, distinct near-field modes are excited, resulting in asymmetric irradiation of the two electrodes and generating a photovoltage with directions determined by the chirality of the incident light field. The proposed design strategy facilitates the realization of ultra-compact CPL detection across diverse materials, structures, and spectral ranges, presenting a novel avenue for achieving high-performance monolithic CPL detection. △ Less

Submitted 19 May, 2024; originally announced May 2024.

Comments: 20 pages, 4 figures

Adaptive Stabilization Based on Machine Learning for Column Generation

Authors: Yunzhuang Shen, Yuan Sun, Xiaodong Li, Zhiguang Cao, Andrew Eberhard, Guangquan Zhang

Abstract: Column generation (CG) is a well-established method for solving large-scale linear programs. It involves iteratively optimizing a subproblem containing a subset of columns and using its dual solution to generate new columns with negative reduced costs. This process continues until the dual values converge to the optimal dual solution to the original problem. A natural phenomenon in CG is the heavy… ▽ More Column generation (CG) is a well-established method for solving large-scale linear programs. It involves iteratively optimizing a subproblem containing a subset of columns and using its dual solution to generate new columns with negative reduced costs. This process continues until the dual values converge to the optimal dual solution to the original problem. A natural phenomenon in CG is the heavy oscillation of the dual values during iterations, which can lead to a substantial slowdown in the convergence rate. Stabilization techniques are devised to accelerate the convergence of dual values by using information beyond the state of the current subproblem. However, there remains a significant gap in obtaining more accurate dual values at an earlier stage. To further narrow this gap, this paper introduces a novel approach consisting of 1) a machine learning approach for accurate prediction of optimal dual solutions and 2) an adaptive stabilization technique that effectively capitalizes on accurate predictions. On the graph coloring problem, we show that our method achieves a significantly improved convergence rate compared to traditional methods. △ Less

Submitted 18 May, 2024; originally announced May 2024.

Comments: Accepted by ICML'24

The Future of Large Language Model Pre-training is Federated

Authors: Lorenzo Sani, Alex Iacob, Zeyu Cao, Bill Marino, Yan Gao, Tomas Paulik, Wanru Zhao, William F. Shen, Preslav Aleksandrov, Xinchi Qiu, Nicholas D. Lane

Abstract: Generative pre-trained large language models (LLMs) have demonstrated impressive performance over a wide range of tasks, thanks to the unprecedented amount of data they have been trained on. As established scaling laws indicate, LLMs' future performance improvement depends on the amount of computing and data sources we can leverage for pre-training. Federated learning (FL) has the potential to unl… ▽ More Generative pre-trained large language models (LLMs) have demonstrated impressive performance over a wide range of tasks, thanks to the unprecedented amount of data they have been trained on. As established scaling laws indicate, LLMs' future performance improvement depends on the amount of computing and data sources we can leverage for pre-training. Federated learning (FL) has the potential to unleash the majority of the planet's data and computational resources, which are underutilized by the data-center-focused training methodology of current LLM practice. Our work presents a robust, flexible, reproducible FL approach that enables large-scale collaboration across institutions to train LLMs. This would mobilize more computational and data resources while matching or potentially exceeding centralized performance. We further show the effectiveness of the federated training scales with model size and present our approach for training a billion-scale federated LLM using limited resources. This will help data-rich actors to become the protagonists of LLMs pre-training instead of leaving the stage to compute-rich actors alone. △ Less

Submitted 17 May, 2024; originally announced May 2024.

Comments: 10 pages, 4 figures, pre-print

The RoboDrive Challenge: Drive Anytime Anywhere in Any Condition

Authors: Lingdong Kong, Shaoyuan Xie, Hanjiang Hu, Yaru Niu, Wei Tsang Ooi, Benoit R. Cottereau, Lai Xing Ng, Yuexin Ma, Wenwei Zhang, Liang Pan, Kai Chen, Ziwei Liu, Weichao Qiu, Wei Zhang, Xu Cao, Hao Lu, Ying-Cong Chen, Caixin Kang, Xinning Zhou, Chengyang Ying, Wentao Shang, Xingxing Wei, Yinpeng Dong, Bo Yang, Shengyin Jiang , et al. (66 additional authors not shown)

Abstract: In the realm of autonomous driving, robust perception under out-of-distribution conditions is paramount for the safe deployment of vehicles. Challenges such as adverse weather, sensor malfunctions, and environmental unpredictability can severely impact the performance of autonomous systems. The 2024 RoboDrive Challenge was crafted to propel the development of driving perception technologies that c… ▽ More In the realm of autonomous driving, robust perception under out-of-distribution conditions is paramount for the safe deployment of vehicles. Challenges such as adverse weather, sensor malfunctions, and environmental unpredictability can severely impact the performance of autonomous systems. The 2024 RoboDrive Challenge was crafted to propel the development of driving perception technologies that can withstand and adapt to these real-world variabilities. Focusing on four pivotal tasks -- BEV detection, map segmentation, semantic occupancy prediction, and multi-view depth estimation -- the competition laid down a gauntlet to innovate and enhance system resilience against typical and atypical disturbances. This year's challenge consisted of five distinct tracks and attracted 140 registered teams from 93 institutes across 11 countries, resulting in nearly one thousand submissions evaluated through our servers. The competition culminated in 15 top-performing solutions, which introduced a range of innovative approaches including advanced data augmentation, multi-sensor fusion, self-supervised learning for error correction, and new algorithmic strategies to enhance sensor robustness. These contributions significantly advanced the state of the art, particularly in handling sensor inconsistencies and environmental variability. Participants, through collaborative efforts, pushed the boundaries of current technologies, showcasing their potential in real-world scenarios. Extensive evaluations and analyses provided insights into the effectiveness of these solutions, highlighting key trends and successful strategies for improving the resilience of driving perception systems. This challenge has set a new benchmark in the field, providing a rich repository of techniques expected to guide future research in this field. △ Less

Submitted 29 May, 2024; v1 submitted 14 May, 2024; originally announced May 2024.

Comments: ICRA 2024; 32 pages, 24 figures, 5 tables; Code at https://robodrive-24.github.io/

DiffTF++: 3D-aware Diffusion Transformer for Large-Vocabulary 3D Generation

Authors: Ziang Cao, Fangzhou Hong, Tong Wu, Liang Pan, Ziwei Liu

Abstract: Generating diverse and high-quality 3D assets automatically poses a fundamental yet challenging task in 3D computer vision. Despite extensive efforts in 3D generation, existing optimization-based approaches struggle to produce large-scale 3D assets efficiently. Meanwhile, feed-forward methods often focus on generating only a single category or a few categories, limiting their generalizability. The… ▽ More Generating diverse and high-quality 3D assets automatically poses a fundamental yet challenging task in 3D computer vision. Despite extensive efforts in 3D generation, existing optimization-based approaches struggle to produce large-scale 3D assets efficiently. Meanwhile, feed-forward methods often focus on generating only a single category or a few categories, limiting their generalizability. Therefore, we introduce a diffusion-based feed-forward framework to address these challenges with a single model. To handle the large diversity and complexity in geometry and texture across categories efficiently, we 1) adopt improved triplane to guarantee efficiency; 2) introduce the 3D-aware transformer to aggregate the generalized 3D knowledge with specialized 3D features; and 3) devise the 3D-aware encoder/decoder to enhance the generalized 3D knowledge. Building upon our 3D-aware Diffusion model with TransFormer, DiffTF, we propose a stronger version for 3D generation, i.e., DiffTF++. It boils down to two parts: multi-view reconstruction loss and triplane refinement. Specifically, we utilize multi-view reconstruction loss to fine-tune the diffusion model and triplane decoder, thereby avoiding the negative influence caused by reconstruction errors and improving texture synthesis. By eliminating the mismatch between the two stages, the generative performance is enhanced, especially in texture. Additionally, a 3D-aware refinement process is introduced to filter out artifacts and refine triplanes, resulting in the generation of more intricate and reasonable details. Extensive experiments on ShapeNet and OmniObject3D convincingly demonstrate the effectiveness of our proposed modules and the state-of-the-art 3D object generation performance with large diversity, rich semantics, and high quality. △ Less

Submitted 13 May, 2024; originally announced May 2024.

Comments: arXiv admin note: substantial text overlap with arXiv:2309.07920

Discovery of Very-high-energy Gamma-ray Emissions from the Low Luminosity AGN NGC 4278 by LHAASO

Authors: Zhen Cao, F. Aharonian, Q. An, Axikegu, Y. X. Bai, Y. W. Bao, D. Bastieri, X. J. Bi, Y. J. Bi, J. T. Cai, Q. Cao, W. Y. Cao, Zhe Cao, J. Chang, J. F. Chang, A. M. Chen, E. S. Chen, Liang Chen, Lin Chen, Long Chen, M. J. Chen, M. L. Chen, Q. H. Chen, S. H. Chen, S. Z. Chen , et al. (255 additional authors not shown)

Abstract: The first source catalog of Large High Altitude Air Shower Observatory reported the detection of a very-high-energy gamma ray source, 1LHAASO J1219+2915. In this paper a further detailed study of the spectral and temporal behavior of this point-like source have been carried. The best-fit position of the TeV source ($\rm{RA}=185.05^{\circ}\pm0.04^{\circ}$, $\rm{Dec}=29.25^{\circ}\pm0.03^{\circ}$) i… ▽ More The first source catalog of Large High Altitude Air Shower Observatory reported the detection of a very-high-energy gamma ray source, 1LHAASO J1219+2915. In this paper a further detailed study of the spectral and temporal behavior of this point-like source have been carried. The best-fit position of the TeV source ($\rm{RA}=185.05^{\circ}\pm0.04^{\circ}$, $\rm{Dec}=29.25^{\circ}\pm0.03^{\circ}$) is compatible with NGC 4278 within $\sim0.03$ degree. Variation analysis shows an indication of the variability at a few months level in the TeV band, which is consistent with low frequency observations. Based on these observations, we report the detection of TeV $γ$-ray emissions from this low-luminosity AGN NGC 4278. The observations by LHAASO-WCDA during active period has a significance level of 8.8\,$σ$ with best-fit photon spectral index $\varGamma=2.56\pm0.14$ and a flux $f_{1-10\,\rm{TeV}}=(7.0\pm1.1_{\rm{sta}}\pm0.35_{\rm{syst}})\times10^{-13}\,\rm{photons\,cm^{-2}\,s^{-1}}$, or approximately $5\%$ of the Crab Nebula. The discovery of VHE from NGC 4278 indicates that the compact, weak radio jet can efficiently accelerate particles and emit TeV photons. △ Less

Submitted 13 May, 2024; originally announced May 2024.

Comments: 11 pages, 5 figures

Tidal disruption event AT2020ocn: early-time X-ray flares caused by a possible disc alignment process

Authors: Z. Cao, P. G. Jonker, D. R. Pasham, S. Wen, N. C. Stone, A. I. Zabludoff

Abstract: A tidal disruption event (TDE) may occur when a star is torn apart by the tidal force of a black hole (BH). Eventually, an accretion disc is thought to form out of stellar debris falling back towards the BH. If the star's orbital angular momentum vector prior to disruption is not aligned with the BH spin angular momentum vector, the disc will be tilted with respect to the BH equatorial plane. The… ▽ More A tidal disruption event (TDE) may occur when a star is torn apart by the tidal force of a black hole (BH). Eventually, an accretion disc is thought to form out of stellar debris falling back towards the BH. If the star's orbital angular momentum vector prior to disruption is not aligned with the BH spin angular momentum vector, the disc will be tilted with respect to the BH equatorial plane. The disc will eventually be drawn into the BH equatorial plane due to a combination of the Bardeen-Petterson effect and internal torques. Here, we analyse the X-ray and UV observations of the TDE AT2020ocn obtained by Swift, XMM-Newton, and NICER. The X-ray light curve shows strong flares during the first $\approx100$ days, while, over the same period, the UV emission decays gradually. We find that the X-ray flares can be explained by a model that also explains the spectral evolution. This model includes a slim disc viewed under a variable inclination plus an inverse-Comptonisation component processing the slim disc emission. A scenario where the ongoing Lense-Thirring precession during the disc alignment process is responsible for the observed inclination variations is consistent with the data. In later observations, we find that the X-ray spectrum of AT2020ocn becomes harder, while the mass accretion rate remains at super-Eddington levels, suggesting the formation of a corona in line with accretion onto other compact objects. We constrain the BH mass to be $(7^{+13}_{-3})\times10^{5}$ M$_\odot$ at the 1$σ$ (68%) confidence level. △ Less

Submitted 13 May, 2024; originally announced May 2024.

Comments: 14 pages, 8 figures, 5 tables, with supplementary materials. Accepted for publication in ApJ

Large Language Model in Financial Regulatory Interpretation

Authors: Zhiyu Cao, Zachary Feinstein

Abstract: This study explores the innovative use of Large Language Models (LLMs) as analytical tools for interpreting complex financial regulations. The primary objective is to design effective prompts that guide LLMs in distilling verbose and intricate regulatory texts, such as the Basel III capital requirement regulations, into a concise mathematical framework that can be subsequently translated into acti… ▽ More This study explores the innovative use of Large Language Models (LLMs) as analytical tools for interpreting complex financial regulations. The primary objective is to design effective prompts that guide LLMs in distilling verbose and intricate regulatory texts, such as the Basel III capital requirement regulations, into a concise mathematical framework that can be subsequently translated into actionable code. This novel approach aims to streamline the implementation of regulatory mandates within the financial reporting and risk management systems of global banking institutions. A case study was conducted to assess the performance of various LLMs, demonstrating that GPT-4 outperforms other models in processing and collecting necessary information, as well as executing mathematical calculations. The case study utilized numerical simulations with asset holdings -- including fixed income, equities, currency pairs, and commodities -- to demonstrate how LLMs can effectively implement the Basel III capital adequacy requirements. Keywords: Large Language Models, Prompt Engineering, LLMs in Finance, Basel III, Minimum Capital Requirements, LLM Ethics △ Less

Submitted 10 July, 2024; v1 submitted 10 May, 2024; originally announced May 2024.

Few-Shot Class Incremental Learning via Robust Transformer Approach

Authors: Naeem Paeedeh, Mahardhika Pratama, Sunu Wibirama, Wolfgang Mayer, Zehong Cao, Ryszard Kowalczyk

Abstract: Few-Shot Class-Incremental Learning presents an extension of the Class Incremental Learning problem where a model is faced with the problem of data scarcity while addressing the catastrophic forgetting problem. This problem remains an open problem because all recent works are built upon the convolutional neural networks performing sub-optimally compared to the transformer approaches. Our paper pre… ▽ More Few-Shot Class-Incremental Learning presents an extension of the Class Incremental Learning problem where a model is faced with the problem of data scarcity while addressing the catastrophic forgetting problem. This problem remains an open problem because all recent works are built upon the convolutional neural networks performing sub-optimally compared to the transformer approaches. Our paper presents Robust Transformer Approach built upon the Compact Convolution Transformer. The issue of overfitting due to few samples is overcome with the notion of the stochastic classifier, where the classifier's weights are sampled from a distribution with mean and variance vectors, thus increasing the likelihood of correct classifications, and the batch-norm layer to stabilize the training process. The issue of CF is dealt with the idea of delta parameters, small task-specific trainable parameters while keeping the backbone networks frozen. A non-parametric approach is developed to infer the delta parameters for the model's predictions. The prototype rectification approach is applied to avoid biased prototype calculations due to the issue of data scarcity. The advantage of ROBUSTA is demonstrated through a series of experiments in the benchmark problems where it is capable of outperforming prior arts with big margins without any data augmentation protocols. △ Less

Submitted 7 May, 2024; originally announced May 2024.

Comments: Under Review in Information Sciences