Showing 1–50 of 918 results for author: Ren, X

First Measurement of Solar $^8$B Neutrino Flux through Coherent Elastic Neutrino-Nucleus Scattering in PandaX-4T

Authors: PandaX Collaboration, Zihao Bo, Wei Chen, Xun Chen, Yunhua Chen, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Zhixing Gao, Lisheng Geng, Karl Giboni, Xunan Guo, Xuyuan Guo, Zichao Guo, Chencheng Han, Ke Han, Changda He, Jinrong He, Di Huang, Houqi Huang, Junting Huang, Ruquan Hou, Yu Hou, Xiangdong Ji , et al. (77 additional authors not shown)

Abstract: The PandaX-4T liquid xenon detector at the China Jinping Underground Laboratory is used to measure the solar $^8$B neutrino flux by detecting neutrinos through coherent scattering with xenon nuclei. Data samples requiring the coincidence of scintillation and ionization signals (paired), as well as unpaired ionization-only signals (US2), are selected with energy threshold of approximately 1.1 keV (… ▽ More The PandaX-4T liquid xenon detector at the China Jinping Underground Laboratory is used to measure the solar $^8$B neutrino flux by detecting neutrinos through coherent scattering with xenon nuclei. Data samples requiring the coincidence of scintillation and ionization signals (paired), as well as unpaired ionization-only signals (US2), are selected with energy threshold of approximately 1.1 keV (0.33 keV) nuclear recoil energy. Combining the commissioning run and the first science run of PandaX-4T, a total exposure of 1.25 and 1.04 tonne$\cdot$year are collected for the paired and US2, respectively. After unblinding, 3 and 332 events are observed with an expectation of 2.8$\pm$0.5 and 251$\pm$32 background events, for the paired and US2 data, respectively. A combined analysis yields a best-fit $^8$B neutrino signal of 3.5 (75) events from the paired (US2) data sample, with $\sim$37\% uncertainty, and the background-only hypothesis is disfavored at 2.64$σ$ significance. This gives a solar $^8$B neutrino flux of ($8.4\pm3.1$)$\times$10$^6$ cm$^{-2}$s$^{-1}$, consistent with the standard solar model prediction. This is the first indication of solar $^8$B neutrino ``fog'' in a dark matter direct detection experiment. △ Less

Submitted 15 July, 2024; originally announced July 2024.

Qwen2 Technical Report

Authors: An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang , et al. (34 additional authors not shown)

Abstract: This report introduces the Qwen2 series, the latest addition to our large language models and large multimodal models. We release a comprehensive suite of foundational and instruction-tuned language models, encompassing a parameter range from 0.5 to 72 billion, featuring dense models and a Mixture-of-Experts model. Qwen2 surpasses most prior open-weight models, including its predecessor Qwen1.5, a… ▽ More This report introduces the Qwen2 series, the latest addition to our large language models and large multimodal models. We release a comprehensive suite of foundational and instruction-tuned language models, encompassing a parameter range from 0.5 to 72 billion, featuring dense models and a Mixture-of-Experts model. Qwen2 surpasses most prior open-weight models, including its predecessor Qwen1.5, and exhibits competitive performance relative to proprietary models across diverse benchmarks on language understanding, generation, multilingual proficiency, coding, mathematics, and reasoning. The flagship model, Qwen2-72B, showcases remarkable performance: 84.2 on MMLU, 37.9 on GPQA, 64.6 on HumanEval, 89.5 on GSM8K, and 82.4 on BBH as a base language model. The instruction-tuned variant, Qwen2-72B-Instruct, attains 9.1 on MT-Bench, 48.1 on Arena-Hard, and 35.7 on LiveCodeBench. Moreover, Qwen2 demonstrates robust multilingual capabilities, proficient in approximately 30 languages, spanning English, Chinese, Spanish, French, German, Arabic, Russian, Korean, Japanese, Thai, Vietnamese, and more, underscoring its versatility and global reach. To foster community innovation and accessibility, we have made the Qwen2 model weights openly available on Hugging Face and ModelScope, and the supplementary materials including example code on GitHub. These platforms also include resources for quantization, fine-tuning, and deployment, facilitating a wide range of applications and research endeavors. △ Less

Submitted 16 July, 2024; v1 submitted 15 July, 2024; originally announced July 2024.

Comments: 25 pages, 1 figure

Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance

Authors: Kaitlyn Zhou, Jena D. Hwang, Xiang Ren, Nouha Dziri, Dan Jurafsky, Maarten Sap

Abstract: The reconfiguration of human-LM interactions from simple sentence completions to complex, multi-domain, humanlike engagements necessitates new methodologies to understand how humans choose to rely on LMs. In our work, we contend that reliance is influenced by numerous factors within the interactional context of a generation, a departure from prior work that used verbalized confidence (e.g., "I'm c… ▽ More The reconfiguration of human-LM interactions from simple sentence completions to complex, multi-domain, humanlike engagements necessitates new methodologies to understand how humans choose to rely on LMs. In our work, we contend that reliance is influenced by numerous factors within the interactional context of a generation, a departure from prior work that used verbalized confidence (e.g., "I'm certain the answer is...") as the key determinant of reliance. Here, we introduce Rel-A.I., an in situ, system-level evaluation approach to measure human reliance on LM-generated epistemic markers (e.g., "I think it's..", "Undoubtedly it's..."). Using this methodology, we measure reliance rates in three emergent human-LM interaction settings: long-term interactions, anthropomorphic generations, and variable subject matter. Our findings reveal that reliance is not solely based on verbalized confidence but is significantly affected by other features of the interaction context. Prior interactions, anthropomorphic cues, and subject domain all contribute to reliance variability. An expression such as, "I'm pretty sure it's...", can vary up to 20% in reliance frequency depending on its interactional context. Our work underscores the importance of context in understanding human reliance and offers future designers and researchers with a methodology to conduct such measurements. △ Less

Submitted 10 July, 2024; originally announced July 2024.

Comments: Preprint

StoryDiffusion: How to Support UX Storyboarding With Generative-AI

Authors: Zhaohui Liang, Xiaoyu Zhang, Kevin Ma, Zhao Liu, Xipei Ren, Kosa Goucher-Lambert, Can Liu

Abstract: Storyboarding is an established method for designing user experiences. Generative AI can support this process by helping designers quickly create visual narratives. However, existing tools only focus on accurate text-to-image generation. Currently, it is not clear how to effectively support the entire creative process of storyboarding and how to develop AI-powered tools to support designers' indiv… ▽ More Storyboarding is an established method for designing user experiences. Generative AI can support this process by helping designers quickly create visual narratives. However, existing tools only focus on accurate text-to-image generation. Currently, it is not clear how to effectively support the entire creative process of storyboarding and how to develop AI-powered tools to support designers' individual workflows. In this work, we iteratively developed and implemented StoryDiffusion, a system that integrates text-to-text and text-to-image models, to support the generation of narratives and images in a single pipeline. With a user study, we observed 12 UX designers using the system for both concept ideation and illustration tasks. Our findings identified AI-directed vs. user-directed creative strategies in both tasks and revealed the importance of supporting the interchange between narrative iteration and image generation. We also found effects of the design tasks on their strategies and preferences, providing insights for future development. △ Less

Submitted 10 July, 2024; originally announced July 2024.

PAPM: A Physics-aware Proxy Model for Process Systems

Authors: Pengwei Liu, Zhongkai Hao, Xingyu Ren, Hangjie Yuan, Jiayang Ren, Dong Ni

Abstract: In the context of proxy modeling for process systems, traditional data-driven deep learning approaches frequently encounter significant challenges, such as substantial training costs induced by large amounts of data, and limited generalization capabilities. As a promising alternative, physics-aware models incorporate partial physics knowledge to ameliorate these challenges. Although demonstrating… ▽ More In the context of proxy modeling for process systems, traditional data-driven deep learning approaches frequently encounter significant challenges, such as substantial training costs induced by large amounts of data, and limited generalization capabilities. As a promising alternative, physics-aware models incorporate partial physics knowledge to ameliorate these challenges. Although demonstrating efficacy, they fall short in terms of exploration depth and universality. To address these shortcomings, we introduce a physics-aware proxy model (PAPM) that fully incorporates partial prior physics of process systems, which includes multiple input conditions and the general form of conservation relations, resulting in better out-of-sample generalization. Additionally, PAPM contains a holistic temporal-spatial stepping module for flexible adaptation across various process systems. Through systematic comparisons with state-of-the-art pure data-driven and physics-aware models across five two-dimensional benchmarks in nine generalization tasks, PAPM notably achieves an average performance improvement of 6.7%, while requiring fewer FLOPs, and just 1% of the parameters compared to the prior leading method. The code is available at https://github.com/pengwei07/PAPM. △ Less

Submitted 6 July, 2024; originally announced July 2024.

Comments: ICML 2024

Global calibration of large-scale photonic integrated circuits

Authors: Jin-Hao Zheng, Qin-Qin Wang, Lan-Tian Feng, Yu-Yang Ding, Xiao-Ye Xu, Xi-Feng Ren, Chuan-Feng Li, Guang-Can Guo

Abstract: The advancing maturity of photonic integrated circuit (PIC) fabrication technology enables the high integration of an increasing number of optical components onto a single chip. With the incremental circuit complexity, the calibration of active phase shifters in a large-scale PIC becomes a crucially important issue. The traditional one-by-one calibration techniques encounter significant hurdles wi… ▽ More The advancing maturity of photonic integrated circuit (PIC) fabrication technology enables the high integration of an increasing number of optical components onto a single chip. With the incremental circuit complexity, the calibration of active phase shifters in a large-scale PIC becomes a crucially important issue. The traditional one-by-one calibration techniques encounter significant hurdles with the propagation of calibration errors, and achieving the decoupling of all phase shifters for independent calibration is not straightforward. To address this issue, we propose a machine-learning approach for globally calibrating the large-scale PIC. Our method utilizes a custom network to simultaneously learn the nonlinear phase-current relations for all thermo-optic phase shifters on the PIC by minimizing the negative likelihood of the measurement datasets. Moreover, the reflectivities of all static beamsplitter components can also be synchronizedly extracted using this calibration method. As an example, a quantum walk PIC with a circuit depth of 12 is calibrated, and a programmable discrete-time quantum walk is experimentally demonstrated. These results will greatly benefit the applications of large-scale PICs in photonic quantum information processing. △ Less

Submitted 2 July, 2024; originally announced July 2024.

Comments: 9 pages, 5 figures, and comments are welcome

fVDB: A Deep-Learning Framework for Sparse, Large-Scale, and High-Performance Spatial Intelligence

Authors: Francis Williams, Jiahui Huang, Jonathan Swartz, Gergely Klár, Vijay Thakkar, Matthew Cong, Xuanchi Ren, Ruilong Li, Clement Fuji-Tsang, Sanja Fidler, Eftychios Sifakis, Ken Museth

Abstract: We present fVDB, a novel GPU-optimized framework for deep learning on large-scale 3D data. fVDB provides a complete set of differentiable primitives to build deep learning architectures for common tasks in 3D learning such as convolution, pooling, attention, ray-tracing, meshing, etc. fVDB simultaneously provides a much larger feature set (primitives and operators) than established frameworks wi… ▽ More We present fVDB, a novel GPU-optimized framework for deep learning on large-scale 3D data. fVDB provides a complete set of differentiable primitives to build deep learning architectures for common tasks in 3D learning such as convolution, pooling, attention, ray-tracing, meshing, etc. fVDB simultaneously provides a much larger feature set (primitives and operators) than established frameworks with no loss in efficiency: our operators match or exceed the performance of other frameworks with narrower scope. Furthermore, fVDB can process datasets with much larger footprint and spatial resolution than prior works, while providing a competitive memory footprint on small inputs. To achieve this combination of versatility and performance, fVDB relies on a single novel VDB index grid acceleration structure paired with several key innovations including GPU accelerated sparse grid construction, convolution using tensorcores, fast ray tracing kernels using a Hierarchical Digital Differential Analyzer algorithm (HDDA), and jagged tensors. Our framework is fully integrated with PyTorch enabling interoperability with existing pipelines, and we demonstrate its effectiveness on a number of representative tasks such as large-scale point-cloud segmentation, high resolution 3D generative modeling, unbounded scale Neural Radiance Fields, and large-scale point cloud reconstruction. △ Less

Submitted 1 July, 2024; originally announced July 2024.

VertiMRF: Differentially Private Vertical Federated Data Synthesis

Authors: Fangyuan Zhao, Zitao Li, Xuebin Ren, Bolin Ding, Shusen Yang, Yaliang Li

Abstract: Data synthesis is a promising solution to share data for various downstream analytic tasks without exposing raw data. However, without a theoretical privacy guarantee, a synthetic dataset would still leak some sensitive information. Differential privacy is thus widely adopted to safeguard data synthesis by strictly limiting the released information. This technique is advantageous yet presents sign… ▽ More Data synthesis is a promising solution to share data for various downstream analytic tasks without exposing raw data. However, without a theoretical privacy guarantee, a synthetic dataset would still leak some sensitive information. Differential privacy is thus widely adopted to safeguard data synthesis by strictly limiting the released information. This technique is advantageous yet presents significant challenges in the vertical federated setting, where data attributes are distributed among different data parties. The main challenge lies in maintaining privacy while efficiently and precisely reconstructing the correlation among cross-party attributes. In this paper, we propose a novel algorithm called VertiMRF, designed explicitly for generating synthetic data in the vertical setting and providing differential privacy protection for all information shared from data parties. We introduce techniques based on the Flajolet-Martin sketch (or frequency oracle) for encoding local data satisfying differential privacy and estimating cross-party marginals. We provide theoretical privacy and utility proof for encoding in this multi-attribute data. Collecting the locally generated private Markov Random Field (MRF) and the sketches, a central server can reconstruct a global MRF, maintaining the most useful information. Additionally, we introduce two techniques tailored for datasets with large attribute domain sizes, namely dimension reduction and consistency enforcement. These two techniques allow flexible and inconsistent binning strategies of local private MRF and the data sketching module, which can preserve information to the greatest extent. We conduct extensive experiments on four real-world datasets to evaluate the effectiveness of VertiMRF. End-to-end comparisons demonstrate the superiority of VertiMRF, and ablation studies validate the effectiveness of each component. △ Less

Submitted 27 June, 2024; originally announced June 2024.

CAVE: Controllable Authorship Verification Explanations

Authors: Sahana Ramnath, Kartik Pandey, Elizabeth Boschee, Xiang Ren

Abstract: Authorship Verification (AV) (do two documents have the same author?) is essential for many sensitive real-life applications. AV is often used in proprietary domains that require a private, offline model, making SOTA online models like ChatGPT undesirable. Other SOTA systems use methods, e.g. Siamese Networks, that are uninterpretable, and hence cannot be trusted in high-stakes applications. In th… ▽ More Authorship Verification (AV) (do two documents have the same author?) is essential for many sensitive real-life applications. AV is often used in proprietary domains that require a private, offline model, making SOTA online models like ChatGPT undesirable. Other SOTA systems use methods, e.g. Siamese Networks, that are uninterpretable, and hence cannot be trusted in high-stakes applications. In this work, we take the first step to address the above challenges with our model CAVE (Controllable Authorship Verification Explanations): CAVE generates free-text AV explanations that are controlled to be 1) structured (can be decomposed into sub-explanations with respect to relevant linguistic features), and 2) easily verified for explanation-label consistency (via intermediate labels in sub-explanations). In this work, we train a Llama-3-8B as CAVE; since there are no human-written corpora for AV explanations, we sample silver-standard explanations from GPT-4-TURBO and distill them into a pretrained Llama-3-8B. Results on three difficult AV datasets IMdB2, Blog-Auth, and FanFiction show that CAVE generates high quality explanations (as measured by automatic and human evaluation) as well as competitive task accuracies. △ Less

Submitted 24 June, 2024; originally announced June 2024.

FutureNet-LOF: Joint Trajectory Prediction and Lane Occupancy Field Prediction with Future Context Encoding

Authors: Mingkun Wang, Xiaoguang Ren, Ruochun Jin, Minglong Li, Xiaochuan Zhang, Changqian Yu, Mingxu Wang, Wenjing Yang

Abstract: Most prior motion prediction endeavors in autonomous driving have inadequately encoded future scenarios, leading to predictions that may fail to accurately capture the diverse movements of agents (e.g., vehicles or pedestrians). To address this, we propose FutureNet, which explicitly integrates initially predicted trajectories into the future scenario and further encodes these future contexts to e… ▽ More Most prior motion prediction endeavors in autonomous driving have inadequately encoded future scenarios, leading to predictions that may fail to accurately capture the diverse movements of agents (e.g., vehicles or pedestrians). To address this, we propose FutureNet, which explicitly integrates initially predicted trajectories into the future scenario and further encodes these future contexts to enhance subsequent forecasting. Additionally, most previous motion forecasting works have focused on predicting independent futures for each agent. However, safe and smooth autonomous driving requires accurately predicting the diverse future behaviors of numerous surrounding agents jointly in complex dynamic environments. Given that all agents occupy certain potential travel spaces and possess lane driving priority, we propose Lane Occupancy Field (LOF), a new representation with lane semantics for motion forecasting in autonomous driving. LOF can simultaneously capture the joint probability distribution of all road participants' future spatial-temporal positions. Due to the high compatibility between lane occupancy field prediction and trajectory prediction, we propose a novel network with future context encoding for the joint prediction of these two tasks. Our approach ranks 1st on two large-scale motion forecasting benchmarks: Argoverse 1 and Argoverse 2. △ Less

Submitted 20 June, 2024; originally announced June 2024.

Comments: 10 pages

Demystifying Forgetting in Language Model Fine-Tuning with Statistical Analysis of Example Associations

Authors: Xisen Jin, Xiang Ren

Abstract: Language models (LMs) are known to suffer from forgetting of previously learned examples when fine-tuned, breaking stability of deployed LM systems. Despite efforts on mitigating forgetting, few have investigated whether, and how forgotten upstream examples are associated with newly learned tasks. Insights on such associations enable efficient and targeted mitigation of forgetting. In this paper,… ▽ More Language models (LMs) are known to suffer from forgetting of previously learned examples when fine-tuned, breaking stability of deployed LM systems. Despite efforts on mitigating forgetting, few have investigated whether, and how forgotten upstream examples are associated with newly learned tasks. Insights on such associations enable efficient and targeted mitigation of forgetting. In this paper, we empirically analyze forgetting that occurs in $N$ upstream examples while the model learns $M$ new tasks and visualize their associations with a $M \times N$ matrix. We empirically demonstrate that the degree of forgetting can often be approximated by simple multiplicative contributions of the upstream examples and newly learned tasks. We also reveal more complicated patterns where specific subsets of examples are forgotten with statistics and visualization. Following our analysis, we predict forgetting that happens on upstream examples when learning a new task with matrix completion over the empirical associations, outperforming prior approaches that rely on trainable LMs. Project website: https://inklab.usc.edu/lm-forgetting-prediction/ △ Less

Submitted 20 June, 2024; originally announced June 2024.

Comments: 5 pages

High-Fidelity Facial Albedo Estimation via Texture Quantization

Authors: Zimin Ran, Xingyu Ren, Xiang An, Kaicheng Yang, Xiangzi Dai, Ziyong Feng, Jia Guo, Linchao Zhu, Jiankang Deng

Abstract: Recent 3D face reconstruction methods have made significant progress in shape estimation, but high-fidelity facial albedo reconstruction remains challenging. Existing methods depend on expensive light-stage captured data to learn facial albedo maps. However, a lack of diversity in subjects limits their ability to recover high-fidelity results. In this paper, we present a novel facial albedo recons… ▽ More Recent 3D face reconstruction methods have made significant progress in shape estimation, but high-fidelity facial albedo reconstruction remains challenging. Existing methods depend on expensive light-stage captured data to learn facial albedo maps. However, a lack of diversity in subjects limits their ability to recover high-fidelity results. In this paper, we present a novel facial albedo reconstruction model, HiFiAlbedo, which recovers the albedo map directly from a single image without the need for captured albedo data. Our key insight is that the albedo map is the illumination invariant texture map, which enables us to use inexpensive texture data to derive an albedo estimation by eliminating illumination. To achieve this, we first collect large-scale ultra-high-resolution facial images and train a high-fidelity facial texture codebook. By using the FFHQ dataset and limited UV textures, we then fine-tune the encoder for texture reconstruction from the input image with adversarial supervision in both image and UV space. Finally, we train a cross-attention module and utilize group identity loss to learn the adaptation from facial texture to the albedo domain. Extensive experimentation has demonstrated that our method exhibits excellent generalizability and is capable of achieving high-fidelity results for in-the-wild facial albedo recovery. Our code, pre-trained weights, and training data will be made publicly available at https://hifialbedo.github.io/. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Self and Cross-Model Distillation for LLMs: Effective Methods for Refusal Pattern Alignment

Authors: Jie Li, Yi Liu, Chongyang Liu, Xiaoning Ren, Ling Shi, Weisong Sun, Yinxing Xue

Abstract: Large Language Models (LLMs) like OpenAI's GPT series, Anthropic's Claude, and Meta's LLaMa have shown remarkable capabilities in text generation. However, their susceptibility to toxic prompts presents significant security challenges. This paper investigates alignment techniques, including Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF), to mitigate these risks.… ▽ More Large Language Models (LLMs) like OpenAI's GPT series, Anthropic's Claude, and Meta's LLaMa have shown remarkable capabilities in text generation. However, their susceptibility to toxic prompts presents significant security challenges. This paper investigates alignment techniques, including Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF), to mitigate these risks. We conduct an empirical study on refusal patterns across nine LLMs, revealing that models with uniform refusal patterns, such as Claude3, exhibit higher security. Based on these findings, we propose self-distilling and cross-model distilling methods to enhance LLM security. Our results show that these methods significantly improve refusal rates and reduce unsafe content, with cross-model distilling achieving refusal rates close to Claude3's 94.51%. These findings underscore the potential of distillation-based alignment in securing LLMs against toxic prompts. △ Less

Submitted 17 June, 2024; originally announced June 2024.

All-electron $BSE@GW$ method with Numeric Atom-Centered Orbitals for Extended Systems

Authors: Ruiyi Zhou, Yi Yao, Volker Blum, Xinguo Ren, Yosuke Kanai

Abstract: Green's function theory has emerged as a powerful many-body approach not only in condensed matter physics but also in quantum chemistry in recent years. We have developed a new all-electron implementation of the BSE@GW formalism using numeric atom-centered orbital basis sets (Liu et al., J. Chem. Phys. 152, 044105 (2020)). We present our recent developments in implementing this formalism for exten… ▽ More Green's function theory has emerged as a powerful many-body approach not only in condensed matter physics but also in quantum chemistry in recent years. We have developed a new all-electron implementation of the BSE@GW formalism using numeric atom-centered orbital basis sets (Liu et al., J. Chem. Phys. 152, 044105 (2020)). We present our recent developments in implementing this formalism for extended systems with periodic boundary conditions. We discuss its numerical implementation and various convergence tests pertaining to numerical atom-centered orbitals, auxiliary basis sets for the resolution-of-identity formalism, and Brillouin zone sampling. Proof-of-principle examples are presented to compare with other formalisms, illustrating the new all-electron BSE@GW method for extended systems. △ Less

Submitted 16 June, 2024; originally announced June 2024.

Comments: 8 figures

EdgeTimer: Adaptive Multi-Timescale Scheduling in Mobile Edge Computing with Deep Reinforcement Learning

Authors: Yijun Hao, Shusen Yang, Fang Li, Yifan Zhang, Shibo Wang, Xuebin Ren

Abstract: In mobile edge computing (MEC), resource scheduling is crucial to task requests' performance and service providers' cost, involving multi-layer heterogeneous scheduling decisions. Existing schedulers typically adopt static timescales to regularly update scheduling decisions of each layer, without adaptive adjustment of timescales for different layers, resulting in potentially poor performance in p… ▽ More In mobile edge computing (MEC), resource scheduling is crucial to task requests' performance and service providers' cost, involving multi-layer heterogeneous scheduling decisions. Existing schedulers typically adopt static timescales to regularly update scheduling decisions of each layer, without adaptive adjustment of timescales for different layers, resulting in potentially poor performance in practice. We notice that the adaptive timescales would significantly improve the trade-off between the operation cost and delay performance. Based on this insight, we propose EdgeTimer, the first work to automatically generate adaptive timescales to update multi-layer scheduling decisions using deep reinforcement learning (DRL). First, EdgeTimer uses a three-layer hierarchical DRL framework to decouple the multi-layer decision-making task into a hierarchy of independent sub-tasks for improving learning efficiency. Second, to cope with each sub-task, EdgeTimer adopts a safe multi-agent DRL algorithm for decentralized scheduling while ensuring system reliability. We apply EdgeTimer to a wide range of Kubernetes scheduling rules, and evaluate it using production traces with different workload patterns. Extensive trace-driven experiments demonstrate that EdgeTimer can learn adaptive timescales, irrespective of workload patterns and built-in scheduling rules. It obtains up to 9.1x more profit than existing approaches without sacrificing the delay performance. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Optimal Batched Linear Bandits

Authors: Xuanfei Ren, Tianyuan Jin, Pan Xu

Abstract: We introduce the E$^4$ algorithm for the batched linear bandit problem, incorporating an Explore-Estimate-Eliminate-Exploit framework. With a proper choice of exploration rate, we prove E$^4$ achieves the finite-time minimax optimal regret with only $O(\log\log T)$ batches, and the asymptotically optimal regret with only $3$ batches as $T\rightarrow\infty$, where $T$ is the time horizon. We furthe… ▽ More We introduce the E$^4$ algorithm for the batched linear bandit problem, incorporating an Explore-Estimate-Eliminate-Exploit framework. With a proper choice of exploration rate, we prove E$^4$ achieves the finite-time minimax optimal regret with only $O(\log\log T)$ batches, and the asymptotically optimal regret with only $3$ batches as $T\rightarrow\infty$, where $T$ is the time horizon. We further prove a lower bound on the batch complexity of linear contextual bandits showing that any asymptotically optimal algorithm must require at least $3$ batches in expectation as $T\rightarrow\infty$, which indicates E$^4$ achieves the asymptotic optimality in regret and batch complexity simultaneously. To the best of our knowledge, E$^4$ is the first algorithm for linear bandits that simultaneously achieves the minimax and asymptotic optimality in regret with the corresponding optimal batch complexities. In addition, we show that with another choice of exploration rate E$^4$ achieves an instance-dependent regret bound requiring at most $O(\log T)$ batches, and maintains the minimax optimality and asymptotic optimality. We conduct thorough experiments to evaluate our algorithm on randomly generated instances and the challenging \textit{End of Optimism} instances \citep{lattimore2017end} which were shown to be hard to learn for optimism based algorithms. Empirical results show that E$^4$ consistently outperforms baseline algorithms with respect to regret minimization, batch complexity, and computational efficiency. △ Less

Submitted 6 June, 2024; originally announced June 2024.

Comments: 26 pages, 6 figures, 4 tables. To appear in the proceedings of the 41st International Conference on Machine Learning (ICML 2024)

Data-driven Explainable Controller for Soft Robots based on Recurrent Neural Networks

Authors: Zixi Chen, Xuyang Ren, Gastone Ciuti, Cesare Stefanini

Abstract: The nonlinearity and hysteresis of soft robot motions have posed challenges in accurate soft robot control. Neural networks, especially recurrent neural networks (RNNs), have been widely leveraged for this issue due to their nonlinear activation functions and recurrent structures. Although they have shown satisfying accuracy in most tasks, these black-box approaches are not explainable, and hence,… ▽ More The nonlinearity and hysteresis of soft robot motions have posed challenges in accurate soft robot control. Neural networks, especially recurrent neural networks (RNNs), have been widely leveraged for this issue due to their nonlinear activation functions and recurrent structures. Although they have shown satisfying accuracy in most tasks, these black-box approaches are not explainable, and hence, they are unsuitable for areas with high safety requirements, like robot-assisted surgery. Based on the RNN controllers, we propose a data-driven explainable controller (DDEC) whose parameters can be updated online. We discuss the Jacobian controller and kinematics controller in theory and demonstrate that they are only special cases of DDEC. Moreover, we utilize RNN, the Jacobian controller, the kinematics controller, and DDECs for trajectory following tasks. Experimental results have shown that our approach outperforms the other controllers considering trajectory following errors while being explainable. We also conduct a study to explore and explain the functions of each DDEC component. This is the first interpretable soft robot controller that overcomes the shortcomings of both NN controllers and interpretable controllers. Future work may involve proposing different DDECs based on different RNN controllers and exploiting them for high-safety-required applications. △ Less

Submitted 6 June, 2024; originally announced June 2024.

Comments: 10 pages, 8 figures, 5 tables

Refactoring to Pythonic Idioms: A Hybrid Knowledge-Driven Approach Leveraging Large Language Models

Authors: Zejun Zhang, Zhenchang Xing, Xiaoxue Ren, Qinghua Lu, Xiwei Xu

Abstract: Pythonic idioms are highly valued and widely used in the Python programming community. However, many Python users find it challenging to use Pythonic idioms. Adopting a rule-based approach or LLM-only approach is not sufficient to overcome three persistent challenges of code idiomatization including code miss, wrong detection and wrong refactoring. Motivated by the determinism of rules and adaptab… ▽ More Pythonic idioms are highly valued and widely used in the Python programming community. However, many Python users find it challenging to use Pythonic idioms. Adopting a rule-based approach or LLM-only approach is not sufficient to overcome three persistent challenges of code idiomatization including code miss, wrong detection and wrong refactoring. Motivated by the determinism of rules and adaptability of LLMs, we propose a hybrid approach consisting of three modules. We not only write prompts to instruct LLMs to complete tasks, but we also invoke Analytic Rule Interfaces (ARIs) to accomplish tasks. The ARIs are Python code generated by prompting LLMs to generate code. We first construct a knowledge module with three elements including ASTscenario, ASTcomponent and Condition, and prompt LLMs to generate Python code for incorporation into an ARI library for subsequent use. After that, for any syntax-error-free Python code, we invoke ARIs from the ARI library to extract ASTcomponent from the ASTscenario, and then filter out ASTcomponent that does not meet the condition. Finally, we design prompts to instruct LLMs to abstract and idiomatize code, and then invoke ARIs from the ARI library to rewrite non-idiomatic code into the idiomatic code. Next, we conduct a comprehensive evaluation of our approach, RIdiom, and Prompt-LLM on nine established Pythonic idioms in RIdiom. Our approach exhibits superior accuracy, F1-score, and recall, while maintaining precision levels comparable to RIdiom, all of which consistently exceed or come close to 90% for each metric of each idiom. Lastly, we extend our evaluation to encompass four new Pythonic idioms. Our approach consistently outperforms Prompt-LLM, achieving metrics with values consistently exceeding 90% for accuracy, F1-score, precision, and recall. △ Less

Submitted 5 June, 2024; originally announced June 2024.

Comments: Accepted by FSE 2024,22 pages

XRec: Large Language Models for Explainable Recommendation

Authors: Qiyao Ma, Xubin Ren, Chao Huang

Abstract: Recommender systems help users navigate information overload by providing personalized recommendations aligned with their preferences. Collaborative Filtering (CF) is a widely adopted approach, but while advanced techniques like graph neural networks (GNNs) and self-supervised learning (SSL) have enhanced CF models for better user representations, they often lack the ability to provide explanation… ▽ More Recommender systems help users navigate information overload by providing personalized recommendations aligned with their preferences. Collaborative Filtering (CF) is a widely adopted approach, but while advanced techniques like graph neural networks (GNNs) and self-supervised learning (SSL) have enhanced CF models for better user representations, they often lack the ability to provide explanations for the recommended items. Explainable recommendations aim to address this gap by offering transparency and insights into the recommendation decision-making process, enhancing users' understanding. This work leverages the language capabilities of Large Language Models (LLMs) to push the boundaries of explainable recommender systems. We introduce a model-agnostic framework called XRec, which enables LLMs to provide comprehensive explanations for user behaviors in recommender systems. By integrating collaborative signals and designing a lightweight collaborative adaptor, the framework empowers LLMs to understand complex patterns in user-item interactions and gain a deeper understanding of user preferences. Our extensive experiments demonstrate the effectiveness of XRec, showcasing its ability to generate comprehensive and meaningful explanations that outperform baseline approaches in explainable recommender systems. We open-source our model implementation at https://github.com/HKUDS/XRec. △ Less

Submitted 4 June, 2024; originally announced June 2024.

Differentially Private Fine-Tuning of Diffusion Models

Authors: Yu-Lin Tsai, Yizhe Li, Zekai Chen, Po-Yu Chen, Chia-Mu Yu, Xuebin Ren, Francois Buet-Golfouse

Abstract: The integration of Differential Privacy (DP) with diffusion models (DMs) presents a promising yet challenging frontier, particularly due to the substantial memorization capabilities of DMs that pose significant privacy risks. Differential privacy offers a rigorous framework for safeguarding individual data points during model training, with Differential Privacy Stochastic Gradient Descent (DP-SGD)… ▽ More The integration of Differential Privacy (DP) with diffusion models (DMs) presents a promising yet challenging frontier, particularly due to the substantial memorization capabilities of DMs that pose significant privacy risks. Differential privacy offers a rigorous framework for safeguarding individual data points during model training, with Differential Privacy Stochastic Gradient Descent (DP-SGD) being a prominent implementation. Diffusion method decomposes image generation into iterative steps, theoretically aligning well with DP's incremental noise addition. Despite the natural fit, the unique architecture of DMs necessitates tailored approaches to effectively balance privacy-utility trade-off. Recent developments in this field have highlighted the potential for generating high-quality synthetic data by pre-training on public data (i.e., ImageNet) and fine-tuning on private data, however, there is a pronounced gap in research on optimizing the trade-offs involved in DP settings, particularly concerning parameter efficiency and model scalability. Our work addresses this by proposing a parameter-efficient fine-tuning strategy optimized for private diffusion models, which minimizes the number of trainable parameters to enhance the privacy-utility trade-off. We empirically demonstrate that our method achieves state-of-the-art performance in DP synthesis, significantly surpassing previous benchmarks on widely studied datasets (e.g., with only 0.47M trainable parameters, achieving a more than 35% improvement over the previous state-of-the-art with a small privacy budget on the CelebA-64 dataset). Anonymous codes available at https://anonymous.4open.science/r/DP-LORA-F02F. △ Less

Submitted 3 June, 2024; originally announced June 2024.

Comments: 16 pages, 5 figures, 11 tables

Topo4D: Topology-Preserving Gaussian Splatting for High-Fidelity 4D Head Capture

Authors: Xuanchen Li, Yuhao Cheng, Xingyu Ren, Haozhe Jia, Di Xu, Wenhan Zhu, Yichao Yan

Abstract: 4D head capture aims to generate dynamic topological meshes and corresponding texture maps from videos, which is widely utilized in movies and games for its ability to simulate facial muscle movements and recover dynamic textures in pore-squeezing. The industry often adopts the method involving multi-view stereo and non-rigid alignment. However, this approach is prone to errors and heavily reliant… ▽ More 4D head capture aims to generate dynamic topological meshes and corresponding texture maps from videos, which is widely utilized in movies and games for its ability to simulate facial muscle movements and recover dynamic textures in pore-squeezing. The industry often adopts the method involving multi-view stereo and non-rigid alignment. However, this approach is prone to errors and heavily reliant on time-consuming manual processing by artists. To simplify this process, we propose Topo4D, a novel framework for automatic geometry and texture generation, which optimizes densely aligned 4D heads and 8K texture maps directly from calibrated multi-view time-series images. Specifically, we first represent the time-series faces as a set of dynamic 3D Gaussians with fixed topology in which the Gaussian centers are bound to the mesh vertices. Afterward, we perform alternative geometry and texture optimization frame-by-frame for high-quality geometry and texture learning while maintaining temporal topology stability. Finally, we can extract dynamic facial meshes in regular wiring arrangement and high-fidelity textures with pore-level details from the learned Gaussians. Extensive experiments show that our method achieves superior results than the current SOTA face reconstruction methods both in the quality of meshes and textures. Project page: https://xuanchenli.github.io/Topo4D/. △ Less

Submitted 15 July, 2024; v1 submitted 1 June, 2024; originally announced June 2024.

Observational test for $f(Q)$ gravity with weak gravitational lensing

Authors: Qingqing Wang, Xin Ren, Yi-Fu Cai, Wentao Luo, Emmanuel N. Saridakis

Abstract: In this article we confront a class of $f(Q)$ gravity models with observational data of galaxy-galaxy lensing. Specifically, we consider the $f(Q)$ gravity models containing a small quadratic correction when compared with General Relativity (GR), and quantify this correction by a model parameter $α$. To derive the observational constraints, we start by extracting the spherically symmetric solution… ▽ More In this article we confront a class of $f(Q)$ gravity models with observational data of galaxy-galaxy lensing. Specifically, we consider the $f(Q)$ gravity models containing a small quadratic correction when compared with General Relativity (GR), and quantify this correction by a model parameter $α$. To derive the observational constraints, we start by extracting the spherically symmetric solutions which correspond to the deviations from the Schwarzschild solution that depends on the model parameter in a two-fold way, i.e., a renormalized mass and a new term proportional to $r^{-2}$. Then, we calculate the effective lensing potential, the deflection angle, the shear component, and the effective Excess Surface Density (ESD) profile. After that, we employ the group catalog and shape catalog from the SDSS DR7 for the lens and source samples respectively. Moreover, we handle the off-center radius as a free parameter and constrain it using the MCMC. Concerning the deviation parameter from GR we derive $α=1.202^{+0.277}_{-0.179}\times 10^{-6} {\rm Mpc}^{-2}$ at 1 $σ$ confidence level, and then compare the fitting efficiency with the standard $Λ$CDM paradigm by applying the AIC and BIC information criteria. Our results indicate that the $f(Q)$ corrections alongside off-center effects yield a scenario that is slightly favored. △ Less

Submitted 31 May, 2024; originally announced June 2024.

Comments: 12pages,2figures

Global solutions to the Euler-Coriolis system

Authors: Xiao Ren, Gang Tian

Abstract: We prove the global well-posedness and scattering for the 3D incompressible Euler-Coriolis system with sufficiently small, regular and suitably localized initial data. Equivalently, we obtain the asymptotic stability for "rigid body" rotational solutions to the pure Euler equations. This extends the recent work of Guo, Pausader and Widmayer to the general non-axisymmetric setting. We prove the global well-posedness and scattering for the 3D incompressible Euler-Coriolis system with sufficiently small, regular and suitably localized initial data. Equivalently, we obtain the asymptotic stability for "rigid body" rotational solutions to the pure Euler equations. This extends the recent work of Guo, Pausader and Widmayer to the general non-axisymmetric setting. △ Less

Submitted 28 May, 2024; originally announced May 2024.

On basic velocity estimates for the plane steady-state Navier-Stokes system and its applications

Authors: Mikhail Korobkov, Xiao Ren

Abstract: We consider some new estimates for general steady Navier-Stokes solutions in plane domains. According to our main result, if the domain is convex, then the difference between mean values of the velocity over two concentric circles is bounded (up to a constant factor) by the square-root of the Dirichlet integral in the annulus between the circles. The constant factor in this inequality is universal… ▽ More We consider some new estimates for general steady Navier-Stokes solutions in plane domains. According to our main result, if the domain is convex, then the difference between mean values of the velocity over two concentric circles is bounded (up to a constant factor) by the square-root of the Dirichlet integral in the annulus between the circles. The constant factor in this inequality is universal and does not depend on the ratio of the circle radii. Several applications of these formulas are discussed. △ Less

Submitted 28 May, 2024; originally announced May 2024.

MSC Class: 76D05; 35Q30

CAPE: Context-Adaptive Positional Encoding for Length Extrapolation

Authors: Chuanyang Zheng, Yihang Gao, Han Shi, Minbin Huang, Jingyao Li, Jing Xiong, Xiaozhe Ren, Michael Ng, Xin Jiang, Zhenguo Li, Yu Li

Abstract: Positional encoding plays a crucial role in transformers, significantly impacting model performance and length generalization. Prior research has introduced absolute positional encoding (APE) and relative positional encoding (RPE) to distinguish token positions in given sequences. However, both APE and RPE remain fixed after model training regardless of input data, limiting their adaptability and… ▽ More Positional encoding plays a crucial role in transformers, significantly impacting model performance and length generalization. Prior research has introduced absolute positional encoding (APE) and relative positional encoding (RPE) to distinguish token positions in given sequences. However, both APE and RPE remain fixed after model training regardless of input data, limiting their adaptability and flexibility. Hence, we expect that the desired positional encoding should be context-adaptive and can be dynamically adjusted with the given attention. In this paper, we propose a Context-Adaptive Positional Encoding (CAPE) method, which dynamically and semantically adjusts based on input context and learned fixed priors. Experimental validation on real-world datasets (Arxiv, Books3, and CHE) demonstrates that CAPE enhances model performances in terms of trained length and length generalization, where the improvements are statistically significant. The model visualization suggests that our model can keep both local and anti-local information. Finally, we successfully train the model on sequence length 128 and achieve better performance at evaluation sequence length 8192, compared with other static positional encoding methods, revealing the benefit of the adaptive positional encoding method. △ Less

Submitted 23 May, 2024; originally announced May 2024.

Comments: Technical Report

Fast Estimation of Relative Transformation Based on Fusion of Odometry and UWB Ranging Data

Authors: Yuan Fu, Zheng Zhang, Guangyang Zeng, Chun Liu, Junfeng Wu, Xiaoqiang Ren

Abstract: In this paper, we investigate the problem of estimating the 4-DOF (three-dimensional position and orientation) robot-robot relative frame transformation using odometers and distance measurements between robots. Firstly, we apply a two-step estimation method based on maximum likelihood estimation. Specifically, a good initial value is obtained through unconstrained least squares and projection, fol… ▽ More In this paper, we investigate the problem of estimating the 4-DOF (three-dimensional position and orientation) robot-robot relative frame transformation using odometers and distance measurements between robots. Firstly, we apply a two-step estimation method based on maximum likelihood estimation. Specifically, a good initial value is obtained through unconstrained least squares and projection, followed by a more accurate estimate achieved through one-step Gauss-Newton iteration. Additionally, the optimal installation positions of Ultra-Wideband (UWB) are provided, and the minimum operating time under different quantities of UWB devices is determined. Simulation demonstrates that the two-step approach offers faster computation with guaranteed accuracy while effectively addressing the relative transformation estimation problem within limited space constraints. Furthermore, this method can be applied to real-time relative transformation estimation when a specific number of UWB devices are installed. △ Less

Submitted 21 May, 2024; originally announced May 2024.

Comments: 15 pages, 4 figures

MSC Class: 93J08 ACM Class: G.m

Ab initio intermolecular interactions mediate thermochemically real-fluid effects that affect system reactivity

Authors: Mingrui Wang, Ruoyue Tang, Xinrui Ren, Yanqing Cui, Song Cheng

Abstract: The properties of supercritical fluids are dictated by intermolecular interactions that involve two or more molecules. Such intermolecular interactions were described via intermolecular potentials in historical supercritical combustion modeling studies, but have been treated empirically and with no consideration of radical interactions or multi-body interactions involving more than two molecules.… ▽ More The properties of supercritical fluids are dictated by intermolecular interactions that involve two or more molecules. Such intermolecular interactions were described via intermolecular potentials in historical supercritical combustion modeling studies, but have been treated empirically and with no consideration of radical interactions or multi-body interactions involving more than two molecules. This approach has been adopted long ago, assuming sufficient characterization of real-fluid effects during supercritical combustion. Here, with data from ab initio multi-body intermolecular potentials, non-empirical Virial Equation of State (EoS), and real-fluid thermochemical and kinetic simulations, we reveal that empirical intermolecular potentials can lead to significant errors in representing supercritical fluids under common combustion situations, which can be impressively described by ab initio intermolecular potentials. These interactions are also found to greatly influence autoignition delay times, a common measure of global reactivity, with significant contributions from radical interactions and multi-body interactions. It is therefore of necessity to incorporate ab initio intermolecular interactions in studying supercritical combustion and various dynamic systems involving supercritical fluids, which has now been enabled through the new framework developed in the present study. △ Less

Submitted 19 May, 2024; originally announced May 2024.

Quantum CZ Gate based on Single Gradient Metasurface

Authors: Qi Liu, Yu Tian, Zhaohua Tian, Guixin Li, Xi-Feng Ren, Qihuang Gong, Ying Gu

Abstract: We propose a scheme to realize quantum controlled-Z (CZ) gates through single gradient metasurface. Using its unique parallel beam-splitting feature, i.e., a series of connected beam splitters with the same splitting ratio, one metasurface can support a CZ gate, several independent CZ gates, or a cascaded CZ gates. Taking advantage of the input polarization determined output path-locking feature,… ▽ More We propose a scheme to realize quantum controlled-Z (CZ) gates through single gradient metasurface. Using its unique parallel beam-splitting feature, i.e., a series of connected beam splitters with the same splitting ratio, one metasurface can support a CZ gate, several independent CZ gates, or a cascaded CZ gates. Taking advantage of the input polarization determined output path-locking feature, both polarization-encoded and path-encoded CZ gates can be demonstrated on the same metasurface, which further improves the integration level of quantum devices. Our research paves the way for integrating quantum logical function through the metasurface. △ Less

Submitted 16 May, 2024; originally announced May 2024.

A Survey of Large Language Models for Graphs

Authors: Xubin Ren, Jiabin Tang, Dawei Yin, Nitesh Chawla, Chao Huang

Abstract: Graphs are an essential data structure utilized to represent relationships in real-world scenarios. Prior research has established that Graph Neural Networks (GNNs) deliver impressive outcomes in graph-centric tasks, such as link prediction and node classification. Despite these advancements, challenges like data sparsity and limited generalization capabilities continue to persist. Recently, Large… ▽ More Graphs are an essential data structure utilized to represent relationships in real-world scenarios. Prior research has established that Graph Neural Networks (GNNs) deliver impressive outcomes in graph-centric tasks, such as link prediction and node classification. Despite these advancements, challenges like data sparsity and limited generalization capabilities continue to persist. Recently, Large Language Models (LLMs) have gained attention in natural language processing. They excel in language comprehension and summarization. Integrating LLMs with graph learning techniques has attracted interest as a way to enhance performance in graph learning tasks. In this survey, we conduct an in-depth review of the latest state-of-the-art LLMs applied in graph learning and introduce a novel taxonomy to categorize existing methods based on their framework design. We detail four unique designs: i) GNNs as Prefix, ii) LLMs as Prefix, iii) LLMs-Graphs Integration, and iv) LLMs-Only, highlighting key methodologies within each category. We explore the strengths and limitations of each framework, and emphasize potential avenues for future research, including overcoming current integration challenges between LLMs and graph learning techniques, and venturing into new application areas. This survey aims to serve as a valuable resource for researchers and practitioners eager to leverage large language models in graph learning, and to inspire continued progress in this dynamic field. We consistently maintain the related open-source materials at \url{https://github.com/HKUDS/Awesome-LLM4Graph-Papers}. △ Less

Submitted 24 June, 2024; v1 submitted 10 May, 2024; originally announced May 2024.

Comments: Published as a KDD'24 survey paper

Testing Cotton gravity as dark matter substitute with weak lensing

Authors: Geyu Mo, Qingqing Wang, Xin Ren, Weitong Yan, Yen Chin Ong, Wentao Luo

Abstract: Harada proposed a modified theory of gravity called Cotton gravity, and argued that it successfully explains the rotation curves of $84$ galaxies without the need of dark matter. In this work we use galaxy-galaxy lensing technique to test whether the modification effect of Cotton gravity can indeed be a viable substitute for dark matter. Using the spherically symmetric solution of Cotton gravity,… ▽ More Harada proposed a modified theory of gravity called Cotton gravity, and argued that it successfully explains the rotation curves of $84$ galaxies without the need of dark matter. In this work we use galaxy-galaxy lensing technique to test whether the modification effect of Cotton gravity can indeed be a viable substitute for dark matter. Using the spherically symmetric solution of Cotton gravity, we obtain the deflection angle via Gauss-Bonnet theorem and the weak lensing shear. We use five galaxy catalogs divided in 5 stellar mass bins from the Sloan Digital Sky Survey Data Release 7 (SDSS DR7), each of which is further divided into blue star forming galaxy and red passive galaxy sub-catalogs. We find that Cotton gravity on its own has significant deviation from the measured galaxy-galaxy lensing signals, thus it cannot replace the role of dark matter. If we consider the combination of dark matter and Cotton gravity, the modification is tightly constrained. Our analysis also applies to other modified gravity theories whose an additional linear term appears in the Schwarzschild solution. △ Less

Submitted 12 May, 2024; originally announced May 2024.

Comments: 16 pages, 3 figures

Constrain the linear scalar perturbation theory of Cotton gravity

Authors: Pengbo Xia, Dongdong Zhang, Xin Ren, Bo Wang, Yen Chin Ong

Abstract: We perform a cosmological test of Cotton gravity, which describes gravity by cotton tensor. The model we consider allows for the same background evolution as the $Λ$CDM model. We derive the cosmological perturbation theory of the scalar mode at the linear level, where the difference from the $Λ$CDM model is characterized by the parameter $β$. We incorporate Cotton gravity with a neutrino model and… ▽ More We perform a cosmological test of Cotton gravity, which describes gravity by cotton tensor. The model we consider allows for the same background evolution as the $Λ$CDM model. We derive the cosmological perturbation theory of the scalar mode at the linear level, where the difference from the $Λ$CDM model is characterized by the parameter $β$. We incorporate Cotton gravity with a neutrino model and perform a Monte Carlo Markov Chain (MCMC) analysis using data from the Cosmic Microwave Background (CMB) and Sloan Digital Sky Survey (SDSS). The analysis constrains parameter $β=-0.00008^{+0.00080}_{-0.00104}$ at the 1-$σ$ confidence level. We conclude that currently, there is no obvious deviation between Cotton gravity and the $Λ$CDM model in the linear cosmological perturbation level for observations. △ Less

Submitted 12 May, 2024; originally announced May 2024.

Comments: 12 pages, 15 figures

Robust field-free switching using large unconventional spin-orbit torque in an all-van der Waals heterostructure

Authors: Yiyang Zhang, Xiaolin Ren, Ruizi Liu, Zehan Chen, Xuezhao Wu, Jie Pang, Wei Wang, Guibin Lan, Kenji Watanabe, Takashi Taniguchi, Youguo Shi, Guoqiang Yu, Qiming Shao

Abstract: The emerging all-van der Waals (vdW) magnetic heterostructure provides a new platform to control the magnetization by the electric field beyond the traditional spintronics devices. One promising strategy is using unconventional spin-orbit torque (SOT) exerted by the out-of-plane polarized spin current to enable deterministic magnetization switching and enhance the switching efficiency. However, in… ▽ More The emerging all-van der Waals (vdW) magnetic heterostructure provides a new platform to control the magnetization by the electric field beyond the traditional spintronics devices. One promising strategy is using unconventional spin-orbit torque (SOT) exerted by the out-of-plane polarized spin current to enable deterministic magnetization switching and enhance the switching efficiency. However, in all-vdW heterostructures, large unconventional SOT remains elusive and the robustness of the field-free switching against external magnetic field hasn't been examined, which hinder further applications. Here we demonstrate the field-free switching in an all-vdW heterostructure combining a type-II Weyl semimetal TaIrTe4 and above-room-temperature ferromagnet Fe3GaTe2. The fully field-free switching can be achieved at 2.56 x 10^10 A per m2 at 300K and a large SOT efficiency of the out-of-plane polarized spin current generated by TaIrTe4 is determined to be 0.37. Moreover, we find that the switching polarity cannot be changed until the external in-plane magnetic field reaches 252mT, indicating a robust switching against the magnetic field. The numerical simulation suggests the large unconventional SOT reduces the switching current density and enhances the robustness of the switching. Our work shows that all-vdW heterostructures are promising candidates for future highly efficient and stable SOT-based devices. △ Less

Submitted 10 May, 2024; originally announced May 2024.

Thermodynamic stability in relativistic viscous and spin hydrodynamics

Authors: Xiang Ren, Chen Yang, Dong-Lin Wang, Shi Pu

Abstract: We have applied thermodynamic stability analysis to derive the stability and causality conditions for conventional relativistic viscous hydrodynamics and spin hydrodynamics. We obtain the thermodynamic stability conditions for second-order relativistic hydrodynamics with shear and bulk viscous tensors, finding them identical to those derived from linear mode analysis. We then derive the thermodyna… ▽ More We have applied thermodynamic stability analysis to derive the stability and causality conditions for conventional relativistic viscous hydrodynamics and spin hydrodynamics. We obtain the thermodynamic stability conditions for second-order relativistic hydrodynamics with shear and bulk viscous tensors, finding them identical to those derived from linear mode analysis. We then derive the thermodynamic stability conditions for minimal causal extended second-order spin hydrodynamics in canonical form, both with and without viscous tensors. Without viscous tensors, the constraints from thermodynamic stability exactly match those from linear mode analysis. In the presence of viscous tensors, the thermodynamic stability imposes more stringent constraints than those obtained from linear mode analysis. Our results suggest that conditions derived from thermodynamic stability analysis can guarantee both causality and stability in linear mode analysis. △ Less

Submitted 5 May, 2024; originally announced May 2024.

Comments: 30 pages

WildChat: 1M ChatGPT Interaction Logs in the Wild

Authors: Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, Yuntian Deng

Abstract: Chatbots such as GPT-4 and ChatGPT are now serving millions of users. Despite their widespread use, there remains a lack of public datasets showcasing how these tools are used by a population of users in practice. To bridge this gap, we offered free access to ChatGPT for online users in exchange for their affirmative, consensual opt-in to anonymously collect their chat transcripts and request head… ▽ More Chatbots such as GPT-4 and ChatGPT are now serving millions of users. Despite their widespread use, there remains a lack of public datasets showcasing how these tools are used by a population of users in practice. To bridge this gap, we offered free access to ChatGPT for online users in exchange for their affirmative, consensual opt-in to anonymously collect their chat transcripts and request headers. From this, we compiled WildChat, a corpus of 1 million user-ChatGPT conversations, which consists of over 2.5 million interaction turns. We compare WildChat with other popular user-chatbot interaction datasets, and find that our dataset offers the most diverse user prompts, contains the largest number of languages, and presents the richest variety of potentially toxic use-cases for researchers to study. In addition to timestamped chat transcripts, we enrich the dataset with demographic data, including state, country, and hashed IP addresses, alongside request headers. This augmentation allows for more detailed analysis of user behaviors across different geographical regions and temporal dimensions. Finally, because it captures a broad range of use cases, we demonstrate the dataset's potential utility in fine-tuning instruction-following models. WildChat is released at https://wildchat.allen.ai under AI2 ImpACT Licenses. △ Less

Submitted 2 May, 2024; originally announced May 2024.

Comments: accepted by ICLR 2024

Quintom cosmology and modified gravity after DESI 2024

Authors: Yuhang Yang, Xin Ren, Qingqing Wang, Zhiyu Lu, Dongdong Zhang, Yi-Fu Cai, Emmanuel N. Saridakis

Abstract: We reconstruct the cosmological background evolution under the scenario of dynamical dark energy through the Gaussian process approach, using the latest Dark Energy Spectroscopic Instrument (DESI) baryon acoustic oscillations (BAO) \cite{DESI:2024mwx} combined with other observations. Our results reveal that the reconstructed dark-energy equation-of-state (EoS) parameter $w(z)$ exhibits the so-cal… ▽ More We reconstruct the cosmological background evolution under the scenario of dynamical dark energy through the Gaussian process approach, using the latest Dark Energy Spectroscopic Instrument (DESI) baryon acoustic oscillations (BAO) \cite{DESI:2024mwx} combined with other observations. Our results reveal that the reconstructed dark-energy equation-of-state (EoS) parameter $w(z)$ exhibits the so-called quintom-B behavior, crossing $-1$ from phantom to quintessence regime as the universe expands. We investigate under what situation this type of evolution could be achieved from the perspectives of field theories and modified gravity. In particular, we reconstruct the corresponding actions for $f(R)$, $f(T)$, and $f(Q)$ gravity, respectively. We explicitly show that, certain modified gravity can exhibit the quintom dynamics and fit the recent DESI data efficiently, and for all cases the quadratic deviation from the $Λ$CDM scenario is mildly favored. △ Less

Submitted 30 April, 2024; originally announced April 2024.

Comments: 10 pages, 3 figures

Belt and Brace: When Federated Learning Meets Differential Privacy

Authors: Xuebin Ren, Shusen Yang, Cong Zhao, Julie McCann, Zongben Xu

Abstract: Federated learning (FL) has great potential for large-scale machine learning (ML) without exposing raw data.Differential privacy (DP) is the de facto standard of privacy protection with provable guarantees.Advances in ML suggest that DP would be a perfect fit for FL with comprehensive privacy preservation. Hence, extensive efforts have been devoted to achieving practically usable FL with DP, which… ▽ More Federated learning (FL) has great potential for large-scale machine learning (ML) without exposing raw data.Differential privacy (DP) is the de facto standard of privacy protection with provable guarantees.Advances in ML suggest that DP would be a perfect fit for FL with comprehensive privacy preservation. Hence, extensive efforts have been devoted to achieving practically usable FL with DP, which however is still challenging.Practitioners often not only are not fully aware of its development and categorization, but also face a hard choice between privacy and utility. Therefore, it calls for a holistic review of current advances and an investigation on the challenges and opportunities for highly usable FL systems with a DP guarantee. In this article, we first introduce the primary concepts of FL and DP, and highlight the benefits of integration. We then review the current developments by categorizing different paradigms and notions. Aiming at usable FL with DP, we present the optimization principles to seek a better tradeoff between model utility and privacy loss. Finally, we discuss future challenges in the emergent areas and relevant research topics. △ Less

Submitted 29 April, 2024; originally announced April 2024.

Comments: 10 pages, 4 figures, accepted by and to appear in Communications of the ACM (CACM)

Discovery of Giant Unit-Cell Super-Structure in the Infinite-Layer Nickelate PrNiO$_2$

Authors: J. Oppliger, J. Küspert, A. -C. Dippel, M. v. Zimmermann, O. Gutowski, X. Ren, X. J. Zhou, Z. Zhu, R. Frison, Q. Wang, L. Martinelli, I. Biało, J. Chang

Abstract: Spectacular quantum phenomena such as superconductivity often emerge in flat-band systems where Coulomb interactions overpower electron kinetics. Engineering strategies for flat-band physics is therefore of great importance. Here, using high-energy grazing-incidence x-ray diffraction, we demonstrate how in-situ temperature annealing of the infinite-layer nickelate PrNiO$_2$ induces a giant superla… ▽ More Spectacular quantum phenomena such as superconductivity often emerge in flat-band systems where Coulomb interactions overpower electron kinetics. Engineering strategies for flat-band physics is therefore of great importance. Here, using high-energy grazing-incidence x-ray diffraction, we demonstrate how in-situ temperature annealing of the infinite-layer nickelate PrNiO$_2$ induces a giant superlattice structure. The annealing effect has a maximum well above room temperature. By covering a large scattering volume, we show a rare period-six in-plane (bi-axial) symmetry and a period-four symmetry in the out-of-plane direction. This giant unit-cell superstructure likely stems from ordering of diffusive oxygen. The stability of this superlattice structure suggests a connection to an energetically favorable electronic state of matter. As such, our study provides a new pathway - different from Moiré structures - to ultra-small Brillouin zone electronics. △ Less

Submitted 27 April, 2024; originally announced April 2024.

Comments: Main: 7 pages, 4 figures. Supplementary: 2 pages, 3 figures

Machine Unlearning in Large Language Models

Authors: Kongyang Chen, Zixin Wang, Bing Mi, Waixi Liu, Shaowei Wang, Xiaojun Ren, Jiaxing Shen

Abstract: Recently, large language models (LLMs) have emerged as a notable field, attracting significant attention for its ability to automatically generate intelligent contents for various application domains. However, LLMs still suffer from significant security and privacy issues. For example, LLMs might expose user privacy from hacking attacks or targeted prompts. To address this problem, this paper intr… ▽ More Recently, large language models (LLMs) have emerged as a notable field, attracting significant attention for its ability to automatically generate intelligent contents for various application domains. However, LLMs still suffer from significant security and privacy issues. For example, LLMs might expose user privacy from hacking attacks or targeted prompts. To address this problem, this paper introduces a novel machine unlearning framework into LLMs. Our objectives are to make LLMs not produce harmful, hallucinatory, or privacy-compromising responses, while retaining their standard output capabilities. To accomplish this, we use an evaluative model to pinpoint dialogues needing unlearning. We also establish a distance loss to function as the model's negative loss, diverting it from previous undesirable outputs. Furthermore, we determine the expected output's cluster mean to formulate a positive loss, directing the model's outputs toward preferable outcomes without compromising its reasoning abilities and performance. Experimental results show that our approach effectively meets unlearning objectives without substantially compromising model performance. △ Less

Submitted 3 February, 2024; originally announced April 2024.

OccGen: Generative Multi-modal 3D Occupancy Prediction for Autonomous Driving

Authors: Guoqing Wang, Zhongdao Wang, Pin Tang, Jilai Zheng, Xiangxuan Ren, Bailan Feng, Chao Ma

Abstract: Existing solutions for 3D semantic occupancy prediction typically treat the task as a one-shot 3D voxel-wise segmentation perception problem. These discriminative methods focus on learning the mapping between the inputs and occupancy map in a single step, lacking the ability to gradually refine the occupancy map and the reasonable scene imaginative capacity to complete the local regions somewhere.… ▽ More Existing solutions for 3D semantic occupancy prediction typically treat the task as a one-shot 3D voxel-wise segmentation perception problem. These discriminative methods focus on learning the mapping between the inputs and occupancy map in a single step, lacking the ability to gradually refine the occupancy map and the reasonable scene imaginative capacity to complete the local regions somewhere. In this paper, we introduce OccGen, a simple yet powerful generative perception model for the task of 3D semantic occupancy prediction. OccGen adopts a ''noise-to-occupancy'' generative paradigm, progressively inferring and refining the occupancy map by predicting and eliminating noise originating from a random Gaussian distribution. OccGen consists of two main components: a conditional encoder that is capable of processing multi-modal inputs, and a progressive refinement decoder that applies diffusion denoising using the multi-modal features as conditions. A key insight of this generative pipeline is that the diffusion denoising process is naturally able to model the coarse-to-fine refinement of the dense 3D occupancy map, therefore producing more detailed predictions. Extensive experiments on several occupancy benchmarks demonstrate the effectiveness of the proposed method compared to the state-of-the-art methods. For instance, OccGen relatively enhances the mIoU by 9.5%, 6.3%, and 13.3% on nuScenes-Occupancy dataset under the muli-modal, LiDAR-only, and camera-only settings, respectively. Moreover, as a generative perception model, OccGen exhibits desirable properties that discriminative models cannot achieve, such as providing uncertainty estimates alongside its multiple-step predictions. △ Less

Submitted 23 April, 2024; originally announced April 2024.

Data reconstruction of the dynamical connection function in $f(Q)$ cosmology

Authors: Yuhang Yang, Xin Ren, Bo Wang, Yi-Fu Cai, Emmanuel N. Saridakis

Abstract: We employ Hubble data and Gaussian Processes in order to reconstruct the dynamical connection function in $f(Q)$ cosmology beyond the coincident gauge. In particular, there exist three branches of connections that satisfy the torsionless and curvatureless conditions, parameterized by a new dynamical function $γ$. We express the redshift dependence of $γ$ in terms of the $H(z)$ function and the… ▽ More We employ Hubble data and Gaussian Processes in order to reconstruct the dynamical connection function in $f(Q)$ cosmology beyond the coincident gauge. In particular, there exist three branches of connections that satisfy the torsionless and curvatureless conditions, parameterized by a new dynamical function $γ$. We express the redshift dependence of $γ$ in terms of the $H(z)$ function and the $f(Q)$ form and parameters, and then we reconstruct it using 55 $H(z)$ observation data. Firstly, we investigate the case where ordinary conservation law holds, and we reconstruct the $f(Q)$ function, which is very well described by a quadratic correction on top of Symmetric Teleparallel Equivalent of General Relativity. Proceeding to the general case, we consider two of the most studied $f(Q)$ models of the literature, namely the square-root and the exponential one. In both cases we reconstruct $γ(z)$, and we show that according to AIC and BIC information criteria its inclusion is favoured compared to both $Λ$CDM paradigm, as well as to the same $f(Q)$ models under the coincident gauge. This feature acts as an indication that $f(Q)$ cosmology should be studied beyond the coincident gauge. △ Less

Submitted 18 April, 2024; originally announced April 2024.

Comments: 19 pages, 5 figures

Efficient structural relaxation based on the random phase approximation: Applications to the water clusters

Authors: Muhammad N. Tahir, Honghui Shang, Jia Li, Xinguo Ren

Abstract: We report an improved implementation for evaluating the analytical gradients of the random phase approximation (RPA) electron-correlation energy based on atomic orbitals and the localized resolution of identity scheme. The more efficient RPA force calculations allow us to relax structures of medium-size water clusters. Particular attention is paid to the structures and energy orderings of the low-… ▽ More We report an improved implementation for evaluating the analytical gradients of the random phase approximation (RPA) electron-correlation energy based on atomic orbitals and the localized resolution of identity scheme. The more efficient RPA force calculations allow us to relax structures of medium-size water clusters. Particular attention is paid to the structures and energy orderings of the low-energy isomers of (H$_2$O)$_n$ clusters with $n=21$, 22, and 25. It is found that the energy ordering of the low-energy isomers of these water clusters are rather sensitive to how their structures are determined. For the five low-energy isomers of (H$_2$O)$_{25}$, the RPA energy ordering based on the RPA geometries is quite different from that based on the geometries relaxed by lower-level theories, in contrast with the situation of small water clusters like the water hexamer. The standard RPA underbinds the water clusters, and this underbinding behavior gets more pronounced as the complete basis set (CBS) limit is approached. The renormalized single excitation (rSE) correction remedies this underbinding, giving rise to a noticeable overbinding behavior at finite basis sets. However, as the CBS limit is approached, RPA+rSE yields an accuracy for the binding energies that is comparable to the best available double hybrid functionals, as demonstrated for the WATER27 testset. △ Less

Submitted 16 April, 2024; originally announced April 2024.

CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language Prompting

Authors: Huihan Li, Liwei Jiang, Jena D. Huang, Hyunwoo Kim, Sebastin Santy, Taylor Sorensen, Bill Yuchen Lin, Nouha Dziri, Xiang Ren, Yejin Choi

Abstract: As the utilization of large language models (LLMs) has proliferated worldwide, it is crucial for them to have adequate knowledge and fair representation for diverse global cultures. In this work, we uncover culture perceptions of three SOTA models on 110 countries and regions on 8 culture-related topics through culture-conditioned generations, and extract symbols from these generations that are as… ▽ More As the utilization of large language models (LLMs) has proliferated worldwide, it is crucial for them to have adequate knowledge and fair representation for diverse global cultures. In this work, we uncover culture perceptions of three SOTA models on 110 countries and regions on 8 culture-related topics through culture-conditioned generations, and extract symbols from these generations that are associated to each culture by the LLM. We discover that culture-conditioned generation consist of linguistic "markers" that distinguish marginalized cultures apart from default cultures. We also discover that LLMs have an uneven degree of diversity in the culture symbols, and that cultures from different geographic regions have different presence in LLMs' culture-agnostic generation. Our findings promote further research in studying the knowledge and fairness of global culture perception in LLMs. Code and Data can be found in: https://github.com/huihanlhh/Culture-Gen/ △ Less

Submitted 26 April, 2024; v1 submitted 15 April, 2024; originally announced April 2024.

SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Prediction

Authors: Pin Tang, Zhongdao Wang, Guoqing Wang, Jilai Zheng, Xiangxuan Ren, Bailan Feng, Chao Ma

Abstract: Vision-based perception for autonomous driving requires an explicit modeling of a 3D space, where 2D latent representations are mapped and subsequent 3D operators are applied. However, operating on dense latent spaces introduces a cubic time and space complexity, which limits scalability in terms of perception range or spatial resolution. Existing approaches compress the dense representation using… ▽ More Vision-based perception for autonomous driving requires an explicit modeling of a 3D space, where 2D latent representations are mapped and subsequent 3D operators are applied. However, operating on dense latent spaces introduces a cubic time and space complexity, which limits scalability in terms of perception range or spatial resolution. Existing approaches compress the dense representation using projections like Bird's Eye View (BEV) or Tri-Perspective View (TPV). Although efficient, these projections result in information loss, especially for tasks like semantic occupancy prediction. To address this, we propose SparseOcc, an efficient occupancy network inspired by sparse point cloud processing. It utilizes a lossless sparse latent representation with three key innovations. Firstly, a 3D sparse diffuser performs latent completion using spatially decomposed 3D sparse convolutional kernels. Secondly, a feature pyramid and sparse interpolation enhance scales with information from others. Finally, the transformer head is redesigned as a sparse variant. SparseOcc achieves a remarkable 74.9% reduction on FLOPs over the dense baseline. Interestingly, it also improves accuracy, from 12.8% to 14.1% mIOU, which in part can be attributed to the sparse representation's ability to avoid hallucinations on empty voxels. △ Less

Submitted 15 April, 2024; originally announced April 2024.

Comments: 10 pages, 4 figures, accepted by CVPR 2024

Journal ref: IEEE Conference on Computer Vision and Pattern Recognition 2024 (CVPR 2024)

LoopAnimate: Loopable Salient Object Animation

Authors: Fanyi Wang, Peng Liu, Haotian Hu, Dan Meng, Jingwen Su, Jinjin Xu, Yanhao Zhang, Xiaoming Ren, Zhiwang Zhang

Abstract: Research on diffusion model-based video generation has advanced rapidly. However, limitations in object fidelity and generation length hinder its practical applications. Additionally, specific domains like animated wallpapers require seamless looping, where the first and last frames of the video match seamlessly. To address these challenges, this paper proposes LoopAnimate, a novel method for gene… ▽ More Research on diffusion model-based video generation has advanced rapidly. However, limitations in object fidelity and generation length hinder its practical applications. Additionally, specific domains like animated wallpapers require seamless looping, where the first and last frames of the video match seamlessly. To address these challenges, this paper proposes LoopAnimate, a novel method for generating videos with consistent start and end frames. To enhance object fidelity, we introduce a framework that decouples multi-level image appearance and textual semantic information. Building upon an image-to-image diffusion model, our approach incorporates both pixel-level and feature-level information from the input image, injecting image appearance and textual semantic embeddings at different positions of the diffusion model. Existing UNet-based video generation models require to input the entire videos during training to encode temporal and positional information at once. However, due to limitations in GPU memory, the number of frames is typically restricted to 16. To address this, this paper proposes a three-stage training strategy with progressively increasing frame numbers and reducing fine-tuning modules. Additionally, we introduce the Temporal E nhanced Motion Module(TEMM) to extend the capacity for encoding temporal and positional information up to 36 frames. The proposed LoopAnimate, which for the first time extends the single-pass generation length of UNet-based video generation models to 35 frames while maintaining high-quality video generation. Experiments demonstrate that LoopAnimate achieves state-of-the-art performance in both objective metrics, such as fidelity and temporal consistency, and subjective evaluation results. △ Less

Submitted 16 April, 2024; v1 submitted 14 April, 2024; originally announced April 2024.

Almost Optimal Time Lower Bound for Approximating Parameterized Clique, CSP, and More, under ETH

Authors: Venkatesan Guruswami, Bingkai Lin, Xuandi Ren, Yican Sun, Kewen Wu

Abstract: The Parameterized Inapproximability Hypothesis (PIH), which is an analog of the PCP theorem in parameterized complexity, asserts that, there is a constant $\varepsilon> 0$ such that for any computable function $f:\mathbb{N}\to\mathbb{N}$, no $f(k)\cdot n^{O(1)}$-time algorithm can, on input a $k$-variable CSP instance with domain size $n$, find an assignment satisfying $1-\varepsilon$ fraction of… ▽ More The Parameterized Inapproximability Hypothesis (PIH), which is an analog of the PCP theorem in parameterized complexity, asserts that, there is a constant $\varepsilon> 0$ such that for any computable function $f:\mathbb{N}\to\mathbb{N}$, no $f(k)\cdot n^{O(1)}$-time algorithm can, on input a $k$-variable CSP instance with domain size $n$, find an assignment satisfying $1-\varepsilon$ fraction of the constraints. A recent work by Guruswami, Lin, Ren, Sun, and Wu (STOC'24) established PIH under the Exponential Time Hypothesis (ETH). In this work, we improve the quantitative aspects of PIH and prove (under ETH) that approximating sparse parameterized CSPs within a constant factor requires $n^{k^{1-o(1)}}$ time. This immediately implies that, assuming ETH, finding a $(k/2)$-clique in an $n$-vertex graph with a $k$-clique requires $n^{k^{1-o(1)}}$ time. We also prove almost optimal time lower bounds for approximating $k$-ExactCover and Max $k$-Coverage. Our proof follows the blueprint of the previous work to identify a "vector-structured" ETH-hard CSP whose satisfiability can be checked via an appropriate form of "parallel" PCP. Using further ideas in the reduction, we guarantee additional structures for constraints in the CSP. We then leverage this to design a parallel PCP of almost linear size based on Reed-Muller codes and derandomized low degree testing. △ Less

Submitted 11 June, 2024; v1 submitted 12 April, 2024; originally announced April 2024.

Mitigating Receiver Impact on Radio Frequency Fingerprint Identification via Domain Adaptation

Authors: Liu Yang, Qiang Li, Xiaoyang Ren, Yi Fang, Shafei Wang

Abstract: Radio Frequency Fingerprint Identification (RFFI), which exploits non-ideal hardware-induced unique distortion resident in the transmit signals to identify an emitter, is emerging as a means to enhance the security of communication systems. Recently, machine learning has achieved great success in developing state-of-the-art RFFI models. However, few works consider cross-receiver RFFI problems, whe… ▽ More Radio Frequency Fingerprint Identification (RFFI), which exploits non-ideal hardware-induced unique distortion resident in the transmit signals to identify an emitter, is emerging as a means to enhance the security of communication systems. Recently, machine learning has achieved great success in developing state-of-the-art RFFI models. However, few works consider cross-receiver RFFI problems, where the RFFI model is trained and deployed on different receivers. Due to altered receiver characteristics, direct deployment of RFFI model on a new receiver leads to significant performance degradation. To address this issue, we formulate the cross-receiver RFFI as a model adaptation problem, which adapts the trained model to unlabeled signals from a new receiver. We first develop a theoretical generalization error bound for the adaptation model. Motivated by the bound, we propose a novel method to solve the cross-receiver RFFI problem, which includes domain alignment and adaptive pseudo-labeling. The former aims at finding a feature space where both domains exhibit similar distributions, effectively reducing the domain discrepancy. Meanwhile, the latter employs a dynamic pseudo-labeling scheme to implicitly transfer the label information from the labeled receiver to the new receiver. Experimental results indicate that the proposed method can effectively mitigate the receiver impact and improve the cross-receiver RFFI performance. △ Less

Submitted 12 April, 2024; originally announced April 2024.

Comments: Accepted by IEEE Internet of Things Journal

LRR: Language-Driven Resamplable Continuous Representation against Adversarial Tracking Attacks

Authors: Jianlang Chen, Xuhong Ren, Qing Guo, Felix Juefei-Xu, Di Lin, Wei Feng, Lei Ma, Jianjun Zhao

Abstract: Visual object tracking plays a critical role in visual-based autonomous systems, as it aims to estimate the position and size of the object of interest within a live video. Despite significant progress made in this field, state-of-the-art (SOTA) trackers often fail when faced with adversarial perturbations in the incoming frames. This can lead to significant robustness and security issues when the… ▽ More Visual object tracking plays a critical role in visual-based autonomous systems, as it aims to estimate the position and size of the object of interest within a live video. Despite significant progress made in this field, state-of-the-art (SOTA) trackers often fail when faced with adversarial perturbations in the incoming frames. This can lead to significant robustness and security issues when these trackers are deployed in the real world. To achieve high accuracy on both clean and adversarial data, we propose building a spatial-temporal continuous representation using the semantic text guidance of the object of interest. This novel continuous representation enables us to reconstruct incoming frames to maintain semantic and appearance consistency with the object of interest and its clean counterparts. As a result, our proposed method successfully defends against different SOTA adversarial tracking attacks while maintaining high accuracy on clean data. In particular, our method significantly increases tracking accuracy under adversarial attacks with around 90% relative improvement on UAV123, which is even higher than the accuracy on clean data. △ Less

Submitted 9 April, 2024; originally announced April 2024.

A Comprehensive Survey on Self-Supervised Learning for Recommendation

Authors: Xubin Ren, Wei Wei, Lianghao Xia, Chao Huang

Abstract: Recommender systems play a crucial role in tackling the challenge of information overload by delivering personalized recommendations based on individual user preferences. Deep learning techniques, such as RNNs, GNNs, and Transformer architectures, have significantly propelled the advancement of recommender systems by enhancing their comprehension of user behaviors and preferences. However, supervi… ▽ More Recommender systems play a crucial role in tackling the challenge of information overload by delivering personalized recommendations based on individual user preferences. Deep learning techniques, such as RNNs, GNNs, and Transformer architectures, have significantly propelled the advancement of recommender systems by enhancing their comprehension of user behaviors and preferences. However, supervised learning methods encounter challenges in real-life scenarios due to data sparsity, resulting in limitations in their ability to learn representations effectively. To address this, self-supervised learning (SSL) techniques have emerged as a solution, leveraging inherent data structures to generate supervision signals without relying solely on labeled data. By leveraging unlabeled data and extracting meaningful representations, recommender systems utilizing SSL can make accurate predictions and recommendations even when confronted with data sparsity. In this paper, we provide a comprehensive review of self-supervised learning frameworks designed for recommender systems, encompassing a thorough analysis of over 170 papers. We conduct an exploration of nine distinct scenarios, enabling a comprehensive understanding of SSL-enhanced recommenders in different contexts. For each domain, we elaborate on different self-supervised learning paradigms, namely contrastive learning, generative learning, and adversarial learning, so as to present technical details of how SSL enhances recommender systems in various contexts. We consistently maintain the related open-source materials at https://github.com/HKUDS/Awesome-SSLRec-Papers. △ Less

Submitted 7 April, 2024; v1 submitted 4 April, 2024; originally announced April 2024.

Light-quark mass dependence of the $Λ(1405)$ resonance

Authors: Xiu-Lei Ren

Abstract: We present the light-quark mass dependence of the $Λ(1405)$ resonance at leading order in a renormalizable framework of covariant chiral effective field theory. The meson-baryon scattering amplitudes, which are obtained by solving the scattering equation within time-ordered perturbation theory, follow the quark mass trajectory of the Coordinated Lattice Simulations consortium. At $M_π\approx 200$… ▽ More We present the light-quark mass dependence of the $Λ(1405)$ resonance at leading order in a renormalizable framework of covariant chiral effective field theory. The meson-baryon scattering amplitudes, which are obtained by solving the scattering equation within time-ordered perturbation theory, follow the quark mass trajectory of the Coordinated Lattice Simulations consortium. At $M_π\approx 200$ MeV and $M_K\approx 487$ MeV, our parameter-free prediction of $Λ(1405)$ poles is consistent with the recent lattice results of BaSc Collaboration [Phys. Rev. Lett. 132, 051901 (2024)]. Varying the pion mass from $135$ MeV to $400$ MeV, we present the evolution of double-pole positions of $Λ(1405)$: the higher pole remains a resonance around the $\bar{K}N$ threshold; whereas the lower pole undergoes a transition from resonance to a virtual state, and ultimately to a bound state of the $πΣ$ system, which could be verified by the forthcoming lattice QCD simulations. △ Less

Submitted 10 June, 2024; v1 submitted 3 April, 2024; originally announced April 2024.

Comments: 9 pages, 4 figures, 1 table; version to appear in Phys. Lett. B

Novel_Authentication_Protocols_Tailored_for_Ambient_IoT_Devices_in_3GPP_5G_Networks

Authors: Xiongpeng Ren, Jin Cao, Hui Li, Yinghui Zhang

Abstract: AIoT devices have attracted significant attention within the 3GPP organization. These devices, distinguished from conventional IoT devices, do not rely on additional batteries or have extremely small battery capacities, offering features such as low cost, easy deployment, and maintenance-free operation. Authentication and secure transmission are fundamental security requirements for AIoT devices.… ▽ More AIoT devices have attracted significant attention within the 3GPP organization. These devices, distinguished from conventional IoT devices, do not rely on additional batteries or have extremely small battery capacities, offering features such as low cost, easy deployment, and maintenance-free operation. Authentication and secure transmission are fundamental security requirements for AIoT devices. However, existing standard security mechanisms are not specifically designed for AIoT devices due to their complex key hierarchies and multi-round interactions, making them unsuitable. Besides, AIoT devices would have more various communication topologies. Therefore, we propose dedicated ultra-lightweight access authentication protocols based on various technologies and algorithms to serve as a forward-looking reference for future research and standardization. Analysis and simulation experiments using chips that closely resemble real AIoT devices, demonstrate that the existing standard protocols are indeed not suitable for such devices, and our protocols outperform existing standard protocols in terms of computational time and energy consumption. After the successful execution of proposed protocols, they can achieve secure transmission of application data, striking a balance between performance and security. △ Less

Submitted 2 April, 2024; originally announced April 2024.