-
How Do Developers Structure Unit Test Cases? An Empirical Study from the "AAA" Perspective
Authors:
Chenhao Wei,
Lu Xiao,
Tingting Yu,
Sunny Wong,
Abigail Clune
Abstract:
The AAA pattern, i.e. arrange, act, and assert, provides a unified structure for unit test cases, which benefits comprehension and maintenance. However, there is little understanding regarding whether and how common real-life developers structure unit test cases following AAA in practice. In particular, are there recurring anti-patterns that deviate from the AAA structure and merit refactoring? An…
▽ More
The AAA pattern, i.e. arrange, act, and assert, provides a unified structure for unit test cases, which benefits comprehension and maintenance. However, there is little understanding regarding whether and how common real-life developers structure unit test cases following AAA in practice. In particular, are there recurring anti-patterns that deviate from the AAA structure and merit refactoring? And, if test cases follow the AAA structure, could they contain design flaws in the A blocks? If we propose refactoring to fix the design of test cases following the AAA, how do developers receive the proposals? Do they favor refactoring? If not, what are their considerations? This study presents an empirical study on 435 real-life unit test cases randomly selected from four open-source projects. Overall, the majority (71.5%) of test cases follow the AAA structure. And, we observed three recurring anti-patterns that deviate from the AAA structure, as well as four design flaws that may reside inside of the A blocks. Each issue type has its drawbacks and merits corresponding refactoring resolutions. We sent a total of 18 refactoring proposals as issue tickets for fixing these problems. We received 78% positive feedback favoring the refactoring. From the rejections, we learned that return-on-investment is a key consideration for developers. The findings provide insights for practitioners to structure unit test cases with AAA in mind, and for researchers to develop related techniques for enforcing AAA in test cases.
△ Less
Submitted 10 July, 2024;
originally announced July 2024.
-
TrAME: Trajectory-Anchored Multi-View Editing for Text-Guided 3D Gaussian Splatting Manipulation
Authors:
Chaofan Luo,
Donglin Di,
Yongjia Ma,
Zhou Xue,
Chen Wei,
Xun Yang,
Yebin Liu
Abstract:
Despite significant strides in the field of 3D scene editing, current methods encounter substantial challenge, particularly in preserving 3D consistency in multi-view editing process. To tackle this challenge, we propose a progressive 3D editing strategy that ensures multi-view consistency via a Trajectory-Anchored Scheme (TAS) with a dual-branch editing mechanism. Specifically, TAS facilitates a…
▽ More
Despite significant strides in the field of 3D scene editing, current methods encounter substantial challenge, particularly in preserving 3D consistency in multi-view editing process. To tackle this challenge, we propose a progressive 3D editing strategy that ensures multi-view consistency via a Trajectory-Anchored Scheme (TAS) with a dual-branch editing mechanism. Specifically, TAS facilitates a tightly coupled iterative process between 2D view editing and 3D updating, preventing error accumulation yielded from text-to-image process. Additionally, we explore the relationship between optimization-based methods and reconstruction-based methods, offering a unified perspective for selecting superior design choice, supporting the rationale behind the designed TAS. We further present a tuning-free View-Consistent Attention Control (VCAC) module that leverages cross-view semantic and geometric reference from the source branch to yield aligned views from the target branch during the editing of 2D views. To validate the effectiveness of our method, we analyze 2D examples to demonstrate the improved consistency with the VCAC module. Further extensive quantitative and qualitative results in text-guided 3D scene editing indicate that our method achieves superior editing quality compared to state-of-the-art methods. We will make the complete codebase publicly available following the conclusion of the double-blind review process.
△ Less
Submitted 2 July, 2024;
originally announced July 2024.
-
Mind's Eye: Image Recognition by EEG via Multimodal Similarity-Keeping Contrastive Learning
Authors:
Chi-Sheng Chen,
Chun-Shu Wei
Abstract:
Decoding images from non-invasive electroencephalographic (EEG) signals has been a grand challenge in understanding how the human brain process visual information in real-world scenarios. To cope with the issues of signal-to-noise ratio and nonstationarity, this paper introduces a MUltimodal Similarity-keeping contrastivE learning (MUSE) framework for zero-shot EEG-based image classification. We d…
▽ More
Decoding images from non-invasive electroencephalographic (EEG) signals has been a grand challenge in understanding how the human brain process visual information in real-world scenarios. To cope with the issues of signal-to-noise ratio and nonstationarity, this paper introduces a MUltimodal Similarity-keeping contrastivE learning (MUSE) framework for zero-shot EEG-based image classification. We develop a series of multivariate time-series encoders tailored for EEG signals and assess the efficacy of regularized contrastive EEG-Image pretraining using an extensive visual EEG dataset. Our method achieves state-of-the-art performance, with a top-1 accuracy of 19.3% and a top-5 accuracy of 48.8% in 200-way zero-shot image classification. Furthermore, we visualize neural patterns via model interpretation, shedding light on the visual processing dynamics in the human brain. The code repository for this work is available at: https://github.com/ChiShengChen/MUSE_EEG.
△ Less
Submitted 5 June, 2024;
originally announced June 2024.
-
Proving Olympiad Algebraic Inequalities without Human Demonstrations
Authors:
Chenrui Wei,
Mengzhou Sun,
Wei Wang
Abstract:
Solving Olympiad-level mathematical problems represents a significant advancement in machine intelligence and automated reasoning. Current machine learning methods, however, struggle to solve Olympiad-level problems beyond Euclidean plane geometry due to a lack of large-scale, high-quality datasets. The challenge is even greater in algebraic systems, which involve infinite reasoning spaces within…
▽ More
Solving Olympiad-level mathematical problems represents a significant advancement in machine intelligence and automated reasoning. Current machine learning methods, however, struggle to solve Olympiad-level problems beyond Euclidean plane geometry due to a lack of large-scale, high-quality datasets. The challenge is even greater in algebraic systems, which involve infinite reasoning spaces within finite conditions. To address these issues, we propose AIPS, an Algebraic Inequality Proving System capable of autonomously generating complex inequality theorems and effectively solving Olympiad-level inequality problems without requiring human demonstrations. During proof search in a mixed reasoning manner, a value curriculum learning strategy on generated datasets is implemented to improve proving performance, demonstrating strong mathematical intuitions. On a test set of 20 International Mathematical Olympiad-level inequality problems, AIPS successfully solved 10, outperforming state-of-the-art methods. Furthermore, AIPS automatically generated a vast array of non-trivial theorems without human intervention, some of which have been evaluated by professional contestants and deemed to reach the level of the International Mathematical Olympiad. Notably, one theorem was selected as a competition problem in a major city 2024 Mathematical Olympiad.
△ Less
Submitted 20 June, 2024;
originally announced June 2024.
-
Exploiting Diffusion Prior for Out-of-Distribution Detection
Authors:
Armando Zhu,
Jiabei Liu,
Keqin Li,
Shuying Dai,
Bo Hong,
Peng Zhao,
Changsong Wei
Abstract:
Out-of-distribution (OOD) detection is crucial for deploying robust machine learning models, especially in areas where security is critical. However, traditional OOD detection methods often fail to capture complex data distributions from large scale date. In this paper, we present a novel approach for OOD detection that leverages the generative ability of diffusion models and the powerful feature…
▽ More
Out-of-distribution (OOD) detection is crucial for deploying robust machine learning models, especially in areas where security is critical. However, traditional OOD detection methods often fail to capture complex data distributions from large scale date. In this paper, we present a novel approach for OOD detection that leverages the generative ability of diffusion models and the powerful feature extraction capabilities of CLIP. By using these features as conditional inputs to a diffusion model, we can reconstruct the images after encoding them with CLIP. The difference between the original and reconstructed images is used as a signal for OOD identification. The practicality and scalability of our method is increased by the fact that it does not require class-specific labeled ID data, as is the case with many other methods. Extensive experiments on several benchmark datasets demonstrates the robustness and effectiveness of our method, which have significantly improved the detection accuracy.
△ Less
Submitted 16 June, 2024;
originally announced June 2024.
-
Augmenting Biomedical Named Entity Recognition with General-domain Resources
Authors:
Yu Yin,
Hyunjae Kim,
Xiao Xiao,
Chih Hsuan Wei,
Jaewoo Kang,
Zhiyong Lu,
Hua Xu,
Meng Fang,
Qingyu Chen
Abstract:
Training a neural network-based biomedical named entity recognition (BioNER) model usually requires extensive and costly human annotations. While several studies have employed multi-task learning with multiple BioNER datasets to reduce human effort, this approach does not consistently yield performance improvements and may introduce label ambiguity in different biomedical corpora. We aim to tackle…
▽ More
Training a neural network-based biomedical named entity recognition (BioNER) model usually requires extensive and costly human annotations. While several studies have employed multi-task learning with multiple BioNER datasets to reduce human effort, this approach does not consistently yield performance improvements and may introduce label ambiguity in different biomedical corpora. We aim to tackle those challenges through transfer learning from easily accessible resources with fewer concept overlaps with biomedical datasets. In this paper, we proposed GERBERA, a simple-yet-effective method that utilized a general-domain NER dataset for training. Specifically, we performed multi-task learning to train a pre-trained biomedical language model with both the target BioNER dataset and the general-domain dataset. Subsequently, we fine-tuned the models specifically for the BioNER dataset. We systematically evaluated GERBERA on five datasets of eight entity types, collectively consisting of 81,410 instances. Despite using fewer biomedical resources, our models demonstrated superior performance compared to baseline models trained with multiple additional BioNER datasets. Specifically, our models consistently outperformed the baselines in six out of eight entity types, achieving an average improvement of 0.9% over the best baseline performance across eight biomedical entity types sourced from five different corpora. Our method was especially effective in amplifying performance on BioNER datasets characterized by limited data, with a 4.7% improvement in F1 scores on the JNLPBA-RNA dataset.
△ Less
Submitted 18 June, 2024; v1 submitted 15 June, 2024;
originally announced June 2024.
-
LiSD: An Efficient Multi-Task Learning Framework for LiDAR Segmentation and Detection
Authors:
Jiahua Xu,
Si Zuo,
Chenfeng Wei,
Wei Zhou
Abstract:
With the rapid proliferation of autonomous driving, there has been a heightened focus on the research of lidar-based 3D semantic segmentation and object detection methodologies, aiming to ensure the safety of traffic participants. In recent decades, learning-based approaches have emerged, demonstrating remarkable performance gains in comparison to conventional algorithms. However, the segmentation…
▽ More
With the rapid proliferation of autonomous driving, there has been a heightened focus on the research of lidar-based 3D semantic segmentation and object detection methodologies, aiming to ensure the safety of traffic participants. In recent decades, learning-based approaches have emerged, demonstrating remarkable performance gains in comparison to conventional algorithms. However, the segmentation and detection tasks have traditionally been examined in isolation to achieve the best precision. To this end, we propose an efficient multi-task learning framework named LiSD which can address both segmentation and detection tasks, aiming to optimize the overall performance. Our proposed LiSD is a voxel-based encoder-decoder framework that contains a hierarchical feature collaboration module and a holistic information aggregation module. Different integration methods are adopted to keep sparsity in segmentation while densifying features for query initialization in detection. Besides, cross-task information is utilized in an instance-aware refinement module to obtain more accurate predictions. Experimental results on the nuScenes dataset and Waymo Open Dataset demonstrate the effectiveness of our proposed model. It is worth noting that LiSD achieves the state-of-the-art performance of 83.3% mIoU on the nuScenes segmentation benchmark for lidar-only methods.
△ Less
Submitted 11 June, 2024; v1 submitted 11 June, 2024;
originally announced June 2024.
-
Async Learned User Embeddings for Ads Delivery Optimization
Authors:
Mingwei Tang,
Meng Liu,
Hong Li,
Junjie Yang,
Chenglin Wei,
Boyang Li,
Dai Li,
Rengan Xu,
Yifan Xu,
Zehua Zhang,
Xiangyu Wang,
Linfeng Liu,
Yuelei Xie,
Chengye Liu,
Labib Fawaz,
Li Li,
Hongnan Wang,
Bill Zhu,
Sri Reddy
Abstract:
In recommendation systems, high-quality user embeddings can capture subtle preferences, enable precise similarity calculations, and adapt to changing preferences over time to maintain relevance. The effectiveness of recommendation systems depends on the quality of user embedding. We propose to asynchronously learn high fidelity user embeddings for billions of users each day from sequence based mul…
▽ More
In recommendation systems, high-quality user embeddings can capture subtle preferences, enable precise similarity calculations, and adapt to changing preferences over time to maintain relevance. The effectiveness of recommendation systems depends on the quality of user embedding. We propose to asynchronously learn high fidelity user embeddings for billions of users each day from sequence based multimodal user activities through a Transformer-like large scale feature learning module. The async learned user representations embeddings (ALURE) are further converted to user similarity graphs through graph learning and then combined with user realtime activities to retrieval highly related ads candidates for the ads delivery system. Our method shows significant gains in both offline and online experiments.
△ Less
Submitted 23 June, 2024; v1 submitted 9 June, 2024;
originally announced June 2024.
-
Confidence-Aware Sub-Structure Beam Search (CABS): Mitigating Hallucination in Structured Data Generation with Large Language Models
Authors:
Chengwei Wei,
Kee Kiat Koo,
Amir Tavanaei,
Karim Bouyarmane
Abstract:
Large Language Models (LLMs) have facilitated structured data generation, with applications in domains like tabular data, document databases, product catalogs, etc. However, concerns persist about generation veracity due to incorrect references or hallucinations, necessitating the incorporation of some form of model confidence for mitigation. Existing confidence estimation methods on LLM generatio…
▽ More
Large Language Models (LLMs) have facilitated structured data generation, with applications in domains like tabular data, document databases, product catalogs, etc. However, concerns persist about generation veracity due to incorrect references or hallucinations, necessitating the incorporation of some form of model confidence for mitigation. Existing confidence estimation methods on LLM generations primarily focus on the confidence at the individual token level or the entire output sequence level, limiting their applicability to structured data generation, which consists of an intricate mix of both independent and correlated entries at the sub-structure level. In this paper, we first investigate confidence estimation methods for generated sub-structure-level data. We introduce the concept of Confidence Network that applies on the hidden state of the LLM transformer, as a more targeted estimate than the traditional token conditional probability. We further propose Confidence-Aware sub-structure Beam Search (CABS), a novel decoding method operating at the sub-structure level in structured data generation. CABS enhances the faithfulness of structured data generation by considering confidence scores from the Confidence Network for each sub-structure-level data and iteratively refining the prompts. Results show that CABS outperforms traditional token-level beam search for structured data generation by 16.7% Recall at 90% precision averagely on the problem of product attribute generation.
△ Less
Submitted 30 May, 2024;
originally announced June 2024.
-
Context Injection Attacks on Large Language Models
Authors:
Cheng'an Wei,
Kai Chen,
Yue Zhao,
Yujia Gong,
Lu Xiang,
Shenchen Zhu
Abstract:
Large Language Models (LLMs) such as ChatGPT and Llama-2 have become prevalent in real-world applications, exhibiting impressive text generation performance. LLMs are fundamentally developed from a scenario where the input data remains static and lacks a clear structure. To behave interactively over time, LLM-based chat systems must integrate additional contextual information (i.e., chat history)…
▽ More
Large Language Models (LLMs) such as ChatGPT and Llama-2 have become prevalent in real-world applications, exhibiting impressive text generation performance. LLMs are fundamentally developed from a scenario where the input data remains static and lacks a clear structure. To behave interactively over time, LLM-based chat systems must integrate additional contextual information (i.e., chat history) into their inputs, following a pre-defined structure. This paper identifies how such integration can expose LLMs to misleading context from untrusted sources and fail to differentiate between system and user inputs, allowing users to inject context. We present a systematic methodology for conducting context injection attacks aimed at eliciting disallowed responses by introducing fabricated context. This could lead to illegal actions, inappropriate content, or technology misuse. Our context fabrication strategies, acceptance elicitation and word anonymization, effectively create misleading contexts that can be structured with attacker-customized prompt templates, achieving injection through malicious user messages. Comprehensive evaluations on real-world LLMs such as ChatGPT and Llama-2 confirm the efficacy of the proposed attack with success rates reaching 97%. We also discuss potential countermeasures that can be adopted for attack detection and developing more secure models. Our findings provide insights into the challenges associated with the real-world deployment of LLMs for interactive and structured data scenarios.
△ Less
Submitted 30 May, 2024;
originally announced May 2024.
-
GeneAgent: Self-verification Language Agent for Gene Set Knowledge Discovery using Domain Databases
Authors:
Zhizheng Wang,
Qiao Jin,
Chih-Hsuan Wei,
Shubo Tian,
Po-Ting Lai,
Qingqing Zhu,
Chi-Ping Day,
Christina Ross,
Zhiyong Lu
Abstract:
Gene set knowledge discovery is essential for advancing human functional genomics. Recent studies have shown promising performance by harnessing the power of Large Language Models (LLMs) on this task. Nonetheless, their results are subject to several limitations common in LLMs such as hallucinations. In response, we present GeneAgent, a first-of-its-kind language agent featuring self-verification…
▽ More
Gene set knowledge discovery is essential for advancing human functional genomics. Recent studies have shown promising performance by harnessing the power of Large Language Models (LLMs) on this task. Nonetheless, their results are subject to several limitations common in LLMs such as hallucinations. In response, we present GeneAgent, a first-of-its-kind language agent featuring self-verification capability. It autonomously interacts with various biological databases and leverages relevant domain knowledge to improve accuracy and reduce hallucination occurrences. Benchmarking on 1,106 gene sets from different sources, GeneAgent consistently outperforms standard GPT-4 by a significant margin. Moreover, a detailed manual review confirms the effectiveness of the self-verification module in minimizing hallucinations and generating more reliable analytical narratives. To demonstrate its practical utility, we apply GeneAgent to seven novel gene sets derived from mouse B2905 melanoma cell lines, with expert evaluations showing that GeneAgent offers novel insights into gene functions and subsequently expedites knowledge discovery.
△ Less
Submitted 25 May, 2024;
originally announced May 2024.
-
ARVideo: Autoregressive Pretraining for Self-Supervised Video Representation Learning
Authors:
Sucheng Ren,
Hongru Zhu,
Chen Wei,
Yijiang Li,
Alan Yuille,
Cihang Xie
Abstract:
This paper presents a new self-supervised video representation learning framework, ARVideo, which autoregressively predicts the next video token in a tailored sequence order. Two key designs are included. First, we organize autoregressive video tokens into clusters that span both spatially and temporally, thereby enabling a richer aggregation of contextual information compared to the standard spat…
▽ More
This paper presents a new self-supervised video representation learning framework, ARVideo, which autoregressively predicts the next video token in a tailored sequence order. Two key designs are included. First, we organize autoregressive video tokens into clusters that span both spatially and temporally, thereby enabling a richer aggregation of contextual information compared to the standard spatial-only or temporal-only clusters. Second, we adopt a randomized spatiotemporal prediction order to facilitate learning from multi-dimensional data, addressing the limitations of a handcrafted spatial-first or temporal-first sequence order. Extensive experiments establish ARVideo as an effective paradigm for self-supervised video representation learning. For example, when trained with the ViT-B backbone, ARVideo competitively attains 81.2% on Kinetics-400 and 70.9% on Something-Something V2, which are on par with the strong benchmark set by VideoMAE. Importantly, ARVideo also demonstrates higher training efficiency, i.e., it trains 14% faster and requires 58% less GPU memory compared to VideoMAE.
△ Less
Submitted 23 May, 2024;
originally announced May 2024.
-
Enhancing Fine-Grained Image Classifications via Cascaded Vision Language Models
Authors:
Canshi Wei
Abstract:
Fine-grained image classification, particularly in zero/few-shot scenarios, presents a significant challenge for vision-language models (VLMs), such as CLIP. These models often struggle with the nuanced task of distinguishing between semantically similar classes due to limitations in their pre-trained recipe, which lacks supervision signals for fine-grained categorization. This paper introduces Ca…
▽ More
Fine-grained image classification, particularly in zero/few-shot scenarios, presents a significant challenge for vision-language models (VLMs), such as CLIP. These models often struggle with the nuanced task of distinguishing between semantically similar classes due to limitations in their pre-trained recipe, which lacks supervision signals for fine-grained categorization. This paper introduces CascadeVLM, an innovative framework that overcomes the constraints of previous CLIP-based methods by effectively leveraging the granular knowledge encapsulated within large vision-language models (LVLMs). Experiments across various fine-grained image datasets demonstrate that CascadeVLM significantly outperforms existing models, specifically on the Stanford Cars dataset, achieving an impressive 85.6% zero-shot accuracy. Performance gain analysis validates that LVLMs produce more accurate predictions for challenging images that CLIPs are uncertain about, bringing the overall accuracy boost. Our framework sheds light on a holistic integration of VLMs and LVLMs for effective and efficient fine-grained image classification.
△ Less
Submitted 18 May, 2024;
originally announced May 2024.
-
Neural Optimization with Adaptive Heuristics for Intelligent Marketing System
Authors:
Changshuai Wei,
Benjamin Zelditch,
Joyce Chen,
Andre Assuncao Silva T Ribeiro,
Jingyi Kenneth Tay,
Borja Ocejo Elizondo,
Keerthi Selvaraj,
Aman Gupta,
Licurgo Benemann De Almeida
Abstract:
Computational marketing has become increasingly important in today's digital world, facing challenges such as massive heterogeneous data, multi-channel customer journeys, and limited marketing budgets. In this paper, we propose a general framework for marketing AI systems, the Neural Optimization with Adaptive Heuristics (NOAH) framework. NOAH is the first general framework for marketing optimizat…
▽ More
Computational marketing has become increasingly important in today's digital world, facing challenges such as massive heterogeneous data, multi-channel customer journeys, and limited marketing budgets. In this paper, we propose a general framework for marketing AI systems, the Neural Optimization with Adaptive Heuristics (NOAH) framework. NOAH is the first general framework for marketing optimization that considers both to-business (2B) and to-consumer (2C) products, as well as both owned and paid channels. We describe key modules of the NOAH framework, including prediction, optimization, and adaptive heuristics, providing examples for bidding and content optimization. We then detail the successful application of NOAH to LinkedIn's email marketing system, showcasing significant wins over the legacy ranking system. Additionally, we share details and insights that are broadly useful, particularly on: (i) addressing delayed feedback with lifetime value, (ii) performing large-scale linear programming with randomization, (iii) improving retrieval with audience expansion, (iv) reducing signal dilution in targeting tests, and (v) handling zero-inflated heavy-tail metrics in statistical testing.
△ Less
Submitted 25 June, 2024; v1 submitted 16 May, 2024;
originally announced May 2024.
-
SEvenLLM: Benchmarking, Eliciting, and Enhancing Abilities of Large Language Models in Cyber Threat Intelligence
Authors:
Hangyuan Ji,
Jian Yang,
Linzheng Chai,
Chaoren Wei,
Liqun Yang,
Yunlong Duan,
Yunli Wang,
Tianzhen Sun,
Hongcheng Guo,
Tongliang Li,
Changyu Ren,
Zhoujun Li
Abstract:
To address the increasing complexity and frequency of cybersecurity incidents emphasized by the recent cybersecurity threat reports with over 10 billion instances, cyber threat intelligence (CTI) plays a critical role in the modern cybersecurity landscape by offering the insights required to understand and combat the constantly evolving nature of cyber threats. Inspired by the powerful capability…
▽ More
To address the increasing complexity and frequency of cybersecurity incidents emphasized by the recent cybersecurity threat reports with over 10 billion instances, cyber threat intelligence (CTI) plays a critical role in the modern cybersecurity landscape by offering the insights required to understand and combat the constantly evolving nature of cyber threats. Inspired by the powerful capability of large language models (LLMs) in handling complex tasks, in this paper, we introduce a framework to benchmark, elicit, and improve cybersecurity incident analysis and response abilities in LLMs for Security Events (SEvenLLM). Specifically, we create a high-quality bilingual instruction corpus by crawling cybersecurity raw text from cybersecurity websites to overcome the lack of effective data for information extraction. Then, we design a pipeline to auto-select tasks from the tasks pool and convert the raw text into supervised corpora comprised of question and response. The instruction dataset SEvenLLM-Instruct is used to train cybersecurity LLMs with the multi-task learning objective (27 well-designed tasks) for augmenting the analysis of cybersecurity events. Extensive experiments in our curated benchmark (SEvenLLM-bench) demonstrate that SEvenLLM performs more sophisticated threat analysis and fortifies defenses against the evolving landscape of cyber threats.
△ Less
Submitted 3 June, 2024; v1 submitted 6 May, 2024;
originally announced May 2024.
-
CRAFT: Extracting and Tuning Cultural Instructions from the Wild
Authors:
Bin Wang,
Geyu Lin,
Zhengyuan Liu,
Chengwei Wei,
Nancy F. Chen
Abstract:
Large language models (LLMs) have rapidly evolved as the foundation of various natural language processing (NLP) applications. Despite their wide use cases, their understanding of culturally-related concepts and reasoning remains limited. Meantime, there is a significant need to enhance these models' cultural reasoning capabilities, especially concerning underrepresented regions. This paper introd…
▽ More
Large language models (LLMs) have rapidly evolved as the foundation of various natural language processing (NLP) applications. Despite their wide use cases, their understanding of culturally-related concepts and reasoning remains limited. Meantime, there is a significant need to enhance these models' cultural reasoning capabilities, especially concerning underrepresented regions. This paper introduces a novel pipeline for extracting high-quality, culturally-related instruction tuning datasets from vast unstructured corpora. We utilize a self-instruction generation pipeline to identify cultural concepts and trigger instruction. By integrating with a general-purpose instruction tuning dataset, our model demonstrates enhanced capabilities in recognizing and understanding regional cultural nuances, thereby enhancing its reasoning capabilities. We conduct experiments across three regions: Singapore, the Philippines, and the United States, achieving performance improvement of up to 6%. Our research opens new avenues for extracting cultural instruction tuning sets directly from unstructured data, setting a precedent for future innovations in the field.
△ Less
Submitted 9 July, 2024; v1 submitted 5 May, 2024;
originally announced May 2024.
-
MANTIS: Interleaved Multi-Image Instruction Tuning
Authors:
Dongfu Jiang,
Xuan He,
Huaye Zeng,
Cong Wei,
Max Ku,
Qian Liu,
Wenhu Chen
Abstract:
Large multimodal models (LMMs) have shown great results in single-image vision language tasks. However, their abilities to solve multi-image visual language tasks is yet to be improved. The existing LMMs like OpenFlamingo, Emu2, Idefics gain their multi-image ability through pre-training on hundreds of millions of noisy interleaved image-text data from the web, which is neither efficient nor effec…
▽ More
Large multimodal models (LMMs) have shown great results in single-image vision language tasks. However, their abilities to solve multi-image visual language tasks is yet to be improved. The existing LMMs like OpenFlamingo, Emu2, Idefics gain their multi-image ability through pre-training on hundreds of millions of noisy interleaved image-text data from the web, which is neither efficient nor effective. In this paper, we aim to build strong multi-image LMMs via instruction tuning with academic-level resources. Therefore, we meticulously construct Mantis-Instruct containing 721K multi-image instruction data to train a family of models Mantis. The instruction tuning empowers Mantis with different multi-image skills like co-reference, comparison, reasoning, and temporal understanding. We evaluate Mantis on five multi-image benchmarks and seven single-image benchmarks. Mantis-SigLIP can achieve SoTA results on all the multi-image benchmarks and beat the strongest multi-image baseline, Idefics2-8B by an average of 11 absolute points. Notably, Idefics2-8B was pre-trained on 140M interleaved multi-image data, which is 200x larger than Mantis-Instruct. We observe that Mantis performs equivalently well on the held-in and held-out benchmarks, which shows its generalization ability. Notably, we found that Mantis can even match the performance of GPT-4V on multi-image benchmarks. We further evaluate Mantis on single-image benchmarks and demonstrate that Mantis also maintains a strong single-image performance on par with CogVLM and Emu2. Our results show that multi-image abilities are not necessarily gained through massive pre-training, instead, it can be gained by the low-cost instruction tuning. Our work provides new perspectives on how to improve LMMs' multi-image abilities.
△ Less
Submitted 23 May, 2024; v1 submitted 2 May, 2024;
originally announced May 2024.
-
CoCoG: Controllable Visual Stimuli Generation based on Human Concept Representations
Authors:
Chen Wei,
Jiachen Zou,
Dietmar Heinke,
Quanying Liu
Abstract:
A central question for cognitive science is to understand how humans process visual objects, i.e, to uncover human low-dimensional concept representation space from high-dimensional visual stimuli. Generating visual stimuli with controlling concepts is the key. However, there are currently no generative models in AI to solve this problem. Here, we present the Concept based Controllable Generation…
▽ More
A central question for cognitive science is to understand how humans process visual objects, i.e, to uncover human low-dimensional concept representation space from high-dimensional visual stimuli. Generating visual stimuli with controlling concepts is the key. However, there are currently no generative models in AI to solve this problem. Here, we present the Concept based Controllable Generation (CoCoG) framework. CoCoG consists of two components, a simple yet efficient AI agent for extracting interpretable concept and predicting human decision-making in visual similarity judgment tasks, and a conditional generation model for generating visual stimuli given the concepts. We quantify the performance of CoCoG from two aspects, the human behavior prediction accuracy and the controllable generation ability. The experiments with CoCoG indicate that 1) the reliable concept embeddings in CoCoG allows to predict human behavior with 64.07\% accuracy in the THINGS-similarity dataset; 2) CoCoG can generate diverse objects through the control of concepts; 3) CoCoG can manipulate human similarity judgment behavior by intervening key concepts. CoCoG offers visual objects with controlling concepts to advance our understanding of causality in human cognition. The code of CoCoG is available at \url{https://github.com/ncclab-sustech/CoCoG}.
△ Less
Submitted 25 April, 2024;
originally announced April 2024.
-
EnzChemRED, a rich enzyme chemistry relation extraction dataset
Authors:
Po-Ting Lai,
Elisabeth Coudert,
Lucila Aimo,
Kristian Axelsen,
Lionel Breuza,
Edouard de Castro,
Marc Feuermann,
Anne Morgat,
Lucille Pourcel,
Ivo Pedruzzi,
Sylvain Poux,
Nicole Redaschi,
Catherine Rivoire,
Anastasia Sveshnikova,
Chih-Hsuan Wei,
Robert Leaman,
Ling Luo,
Zhiyong Lu,
Alan Bridge
Abstract:
Expert curation is essential to capture knowledge of enzyme functions from the scientific literature in FAIR open knowledgebases but cannot keep pace with the rate of new discoveries and new publications. In this work we present EnzChemRED, for Enzyme Chemistry Relation Extraction Dataset, a new training and benchmarking dataset to support the development of Natural Language Processing (NLP) metho…
▽ More
Expert curation is essential to capture knowledge of enzyme functions from the scientific literature in FAIR open knowledgebases but cannot keep pace with the rate of new discoveries and new publications. In this work we present EnzChemRED, for Enzyme Chemistry Relation Extraction Dataset, a new training and benchmarking dataset to support the development of Natural Language Processing (NLP) methods such as (large) language models that can assist enzyme curation. EnzChemRED consists of 1,210 expert curated PubMed abstracts in which enzymes and the chemical reactions they catalyze are annotated using identifiers from the UniProt Knowledgebase (UniProtKB) and the ontology of Chemical Entities of Biological Interest (ChEBI). We show that fine-tuning pre-trained language models with EnzChemRED can significantly boost their ability to identify mentions of proteins and chemicals in text (Named Entity Recognition, or NER) and to extract the chemical conversions in which they participate (Relation Extraction, or RE), with average F1 score of 86.30% for NER, 86.66% for RE for chemical conversion pairs, and 83.79% for RE for chemical conversion pairs and linked enzymes. We combine the best performing methods after fine-tuning using EnzChemRED to create an end-to-end pipeline for knowledge extraction from text and apply this to abstracts at PubMed scale to create a draft map of enzyme functions in literature to guide curation efforts in UniProtKB and the reaction knowledgebase Rhea. The EnzChemRED corpus is freely available at https://ftp.expasy.org/databases/rhea/nlp/.
△ Less
Submitted 22 April, 2024;
originally announced April 2024.
-
Practical GHz single-cavity all-fiber dual-comb laser for high-speed spectroscopy
Authors:
Lin Ling,
Wei Lin,
Zhaoheng Liang,
Minjie Pan,
Chiyi Wei,
Xuewen Chen,
Yang Yang,
Zhijin Xiong,
Yuankai Guo,
Xiaoming Wei,
Zhongmin Yang
Abstract:
Dual-comb spectroscopy (DCS) with few-GHz tooth spacing that provides the optimal trade-off between spectral resolution and refresh rate is a powerful tool for measuring and analyzing rapidly evolving transient events. Despite such an exciting opportunity, existing technologies compromise either the spectral resolution or refresh rate, leaving few-GHz DCS with robust design largely unmet for front…
▽ More
Dual-comb spectroscopy (DCS) with few-GHz tooth spacing that provides the optimal trade-off between spectral resolution and refresh rate is a powerful tool for measuring and analyzing rapidly evolving transient events. Despite such an exciting opportunity, existing technologies compromise either the spectral resolution or refresh rate, leaving few-GHz DCS with robust design largely unmet for frontier applications. In this work, we demonstrate a novel GHz DCS by exploring the multimode interference-mediated spectral filtering effect in an all-fiber ultrashort cavity configuration. The GHz single-cavity all-fiber dual-comb source is seeded by a dual-wavelength mode-locked fiber laser operating at fundamental repetition rates of about 1.0 GHz differing by 148 kHz, which has an excellent stability in the free-running state that the Allan deviation is only 101.7 mHz for an average time of 1 second. Thanks to the large repetition rate difference between the asynchronous dichromatic pulse trains, the GHz DCS enables a refresh time as short as 6.75 us, making it promising for studying nonrepeatable transient phenomena in real time. To this end, the practicality of the present GHz DCS is validated by successfully capturing the 'shock waves' of balloon and firecracker explosions outdoors. This GHz single-cavity all-fiber dual-comb system promises a noteworthy improvement in acquisition speed and reliability without sacrificing measurement accuracy, anticipated as a practical tool for high-speed applications.
△ Less
Submitted 18 April, 2024;
originally announced April 2024.
-
Feature Corrective Transfer Learning: End-to-End Solutions to Object Detection in Non-Ideal Visual Conditions
Authors:
Chuheng Wei,
Guoyuan Wu,
Matthew J. Barth
Abstract:
A significant challenge in the field of object detection lies in the system's performance under non-ideal imaging conditions, such as rain, fog, low illumination, or raw Bayer images that lack ISP processing. Our study introduces "Feature Corrective Transfer Learning", a novel approach that leverages transfer learning and a bespoke loss function to facilitate the end-to-end detection of objects in…
▽ More
A significant challenge in the field of object detection lies in the system's performance under non-ideal imaging conditions, such as rain, fog, low illumination, or raw Bayer images that lack ISP processing. Our study introduces "Feature Corrective Transfer Learning", a novel approach that leverages transfer learning and a bespoke loss function to facilitate the end-to-end detection of objects in these challenging scenarios without the need to convert non-ideal images into their RGB counterparts. In our methodology, we initially train a comprehensive model on a pristine RGB image dataset. Subsequently, non-ideal images are processed by comparing their feature maps against those from the initial ideal RGB model. This comparison employs the Extended Area Novel Structural Discrepancy Loss (EANSDL), a novel loss function designed to quantify similarities and integrate them into the detection loss. This approach refines the model's ability to perform object detection across varying conditions through direct feature map correction, encapsulating the essence of Feature Corrective Transfer Learning. Experimental validation on variants of the KITTI dataset demonstrates a significant improvement in mean Average Precision (mAP), resulting in a 3.8-8.1% relative enhancement in detection under non-ideal conditions compared to the baseline model, and a less marginal performance difference within 1.3% of the mAP@[0.5:0.95] achieved under ideal conditions by the standard Faster RCNN algorithm.
△ Less
Submitted 19 April, 2024; v1 submitted 17 April, 2024;
originally announced April 2024.
-
KI-GAN: Knowledge-Informed Generative Adversarial Networks for Enhanced Multi-Vehicle Trajectory Forecasting at Signalized Intersections
Authors:
Chuheng Wei,
Guoyuan Wu,
Matthew J. Barth,
Amr Abdelraouf,
Rohit Gupta,
Kyungtae Han
Abstract:
Reliable prediction of vehicle trajectories at signalized intersections is crucial to urban traffic management and autonomous driving systems. However, it presents unique challenges, due to the complex roadway layout at intersections, involvement of traffic signal controls, and interactions among different types of road users. To address these issues, we present in this paper a novel model called…
▽ More
Reliable prediction of vehicle trajectories at signalized intersections is crucial to urban traffic management and autonomous driving systems. However, it presents unique challenges, due to the complex roadway layout at intersections, involvement of traffic signal controls, and interactions among different types of road users. To address these issues, we present in this paper a novel model called Knowledge-Informed Generative Adversarial Network (KI-GAN), which integrates both traffic signal information and multi-vehicle interactions to predict vehicle trajectories accurately. Additionally, we propose a specialized attention pooling method that accounts for vehicle orientation and proximity at intersections. Based on the SinD dataset, our KI-GAN model is able to achieve an Average Displacement Error (ADE) of 0.05 and a Final Displacement Error (FDE) of 0.12 for a 6-second observation and 6-second prediction cycle. When the prediction window is extended to 9 seconds, the ADE and FDE values are further reduced to 0.11 and 0.26, respectively. These results demonstrate the effectiveness of the proposed KI-GAN model in vehicle trajectory prediction under complex scenarios at signalized intersections, which represents a significant advancement in the target field.
△ Less
Submitted 19 April, 2024; v1 submitted 17 April, 2024;
originally announced April 2024.
-
Resilience of Large Language Models for Noisy Instructions
Authors:
Bin Wang,
Chengwei Wei,
Zhengyuan Liu,
Geyu Lin,
Nancy F. Chen
Abstract:
As the rapidly advancing domain of natural language processing (NLP), large language models (LLMs) have emerged as powerful tools for interpreting human commands and generating text across various tasks. Nonetheless, the resilience of LLMs to handle text containing inherent errors, stemming from human interactions and collaborative systems, has not been thoroughly explored. Our study investigates…
▽ More
As the rapidly advancing domain of natural language processing (NLP), large language models (LLMs) have emerged as powerful tools for interpreting human commands and generating text across various tasks. Nonetheless, the resilience of LLMs to handle text containing inherent errors, stemming from human interactions and collaborative systems, has not been thoroughly explored. Our study investigates the resilience of LLMs against five common types of disruptions including 1) ASR (Automatic Speech Recognition) errors, 2) OCR (Optical Character Recognition) errors, 3) grammatical mistakes, 4) typographical errors, and 5) distractive content. We aim to investigate how these models react by deliberately embedding these errors into instructions. Our findings reveal that while some LLMs show a degree of resistance to certain types of noise, their overall performance significantly suffers. This emphasizes the importance of further investigation into enhancing model resilience. In response to the observed decline in performance, our study also evaluates a "re-pass" strategy, designed to purify the instructions of noise before the LLMs process them. Our analysis indicates that correcting noisy instructions, particularly for open-source LLMs, presents significant challenges.
△ Less
Submitted 15 April, 2024;
originally announced April 2024.
-
LaSagnA: Language-based Segmentation Assistant for Complex Queries
Authors:
Cong Wei,
Haoxian Tan,
Yujie Zhong,
Yujiu Yang,
Lin Ma
Abstract:
Recent advancements have empowered Large Language Models for Vision (vLLMs) to generate detailed perceptual outcomes, including bounding boxes and masks. Nonetheless, there are two constraints that restrict the further application of these vLLMs: the incapability of handling multiple targets per query and the failure to identify the absence of query objects in the image. In this study, we acknowle…
▽ More
Recent advancements have empowered Large Language Models for Vision (vLLMs) to generate detailed perceptual outcomes, including bounding boxes and masks. Nonetheless, there are two constraints that restrict the further application of these vLLMs: the incapability of handling multiple targets per query and the failure to identify the absence of query objects in the image. In this study, we acknowledge that the main cause of these problems is the insufficient complexity of training queries. Consequently, we define the general sequence format for complex queries. Then we incorporate a semantic segmentation task in the current pipeline to fulfill the requirements of training data. Furthermore, we present three novel strategies to effectively handle the challenges arising from the direct integration of the proposed format. The effectiveness of our model in processing complex queries is validated by the comparable results with conventional methods on both close-set and open-set semantic segmentation datasets. Additionally, we outperform a series of vLLMs in reasoning and referring segmentation, showcasing our model's remarkable capabilities. We release the code at https://github.com/congvvc/LaSagnA.
△ Less
Submitted 12 April, 2024;
originally announced April 2024.
-
FuSeBMC AI: Acceleration of Hybrid Approach through Machine Learning
Authors:
Kaled M. Alshmrany,
Mohannad Aldughaim,
Chenfeng Wei,
Tom Sweet,
Richard Allmendinger,
Lucas C. Cordeiro
Abstract:
We present FuSeBMC-AI, a test generation tool grounded in machine learning techniques. FuSeBMC-AI extracts various features from the program and employs support vector machine and neural network models to predict a hybrid approach optimal configuration. FuSeBMC-AI utilizes Bounded Model Checking and Fuzzing as back-end verification engines. FuSeBMC-AI outperforms the default configuration of the u…
▽ More
We present FuSeBMC-AI, a test generation tool grounded in machine learning techniques. FuSeBMC-AI extracts various features from the program and employs support vector machine and neural network models to predict a hybrid approach optimal configuration. FuSeBMC-AI utilizes Bounded Model Checking and Fuzzing as back-end verification engines. FuSeBMC-AI outperforms the default configuration of the underlying verification engine in certain cases while concurrently diminishing resource consumption.
△ Less
Submitted 9 April, 2024;
originally announced April 2024.
-
BOLD v4: A Centralized Bioinformatics Platform for DNA-based Biodiversity Data
Authors:
Sujeevan Ratnasingham,
Catherine Wei,
Dean Chan,
Jireh Agda,
Josh Agda,
Liliana Ballesteros-Mejia,
Hamza Ait Boutou,
Zak Mohammad El Bastami,
Eddie Ma,
Ramya Manjunath,
Dana Rea,
Chris Ho,
Angela Telfer,
Jaclyn McKeowan,
Miduna Rahulan,
Claudia Steinke,
Justin Dorsheimer,
Megan Milton,
Paul D. N. Hebert
Abstract:
BOLD, the Barcode of Life Data System, supports the acquisition, storage, validation, analysis, and publication of DNA barcodes, activities requiring the integration of molecular, morphological, and distributional data. Its pivotal role in curating the reference library of DNA barcodes, coupled with its data management and analysis capabilities, make it a central resource for biodiversity science.…
▽ More
BOLD, the Barcode of Life Data System, supports the acquisition, storage, validation, analysis, and publication of DNA barcodes, activities requiring the integration of molecular, morphological, and distributional data. Its pivotal role in curating the reference library of DNA barcodes, coupled with its data management and analysis capabilities, make it a central resource for biodiversity science. It enables rapid, accurate identification of specimens and also reveals patterns of genetic diversity and evolutionary relationships among taxa. Launched in 2005, BOLD has become an increasingly powerful tool for advancing understanding of planetary biodiversity. It currently hosts 17 million specimen records and 14 million barcodes that provide coverage for more than a million species from every continent and ocean. The platform has the long-term goal of providing a consistent, accurate system for identifying all species of eukaryotes. BOLD's integrated analytical tools, full data lifecycle support, and secure collaboration framework distinguish it from other biodiversity platforms. BOLD v4 brought enhanced data management and analysis capabilities as well as novel functionality for data dissemination and publication. Its next version will include features to strengthen its utility to the research community, governments, industry, and society-at-large.
△ Less
Submitted 5 May, 2024; v1 submitted 8 April, 2024;
originally announced April 2024.
-
CSST Strong Lensing Preparation: a Framework for Detecting Strong Lenses in the Multi-color Imaging Survey by the China Survey Space Telescope (CSST)
Authors:
Xu Li,
Ruiqi Sun,
Jiameng Lv,
Peng Jia,
Nan Li,
Chengliang Wei,
Zou Hu,
Xinzhong Er,
Yun Chen,
Zhang Ban,
Yuedong Fang,
Qi Guo,
Dezi Liu,
Guoliang Li,
Lin Lin,
Ming Li,
Ran Li,
Xiaobo Li,
Yu Luo,
Xianmin Meng,
Jundan Nie,
Zhaoxiang Qi,
Yisheng Qiu,
Li Shao,
Hao Tian
, et al. (7 additional authors not shown)
Abstract:
Strong gravitational lensing is a powerful tool for investigating dark matter and dark energy properties. With the advent of large-scale sky surveys, we can discover strong lensing systems on an unprecedented scale, which requires efficient tools to extract them from billions of astronomical objects. The existing mainstream lens-finding tools are based on machine learning algorithms and applied to…
▽ More
Strong gravitational lensing is a powerful tool for investigating dark matter and dark energy properties. With the advent of large-scale sky surveys, we can discover strong lensing systems on an unprecedented scale, which requires efficient tools to extract them from billions of astronomical objects. The existing mainstream lens-finding tools are based on machine learning algorithms and applied to cut-out-centered galaxies. However, according to the design and survey strategy of optical surveys by CSST, preparing cutouts with multiple bands requires considerable efforts. To overcome these challenges, we have developed a framework based on a hierarchical visual Transformer with a sliding window technique to search for strong lensing systems within entire images. Moreover, given that multi-color images of strong lensing systems can provide insights into their physical characteristics, our framework is specifically crafted to identify strong lensing systems in images with any number of channels. As evaluated using CSST mock data based on an Semi-Analytic Model named CosmoDC2, our framework achieves precision and recall rates of 0.98 and 0.90, respectively. To evaluate the effectiveness of our method in real observations, we have applied it to a subset of images from the DESI Legacy Imaging Surveys and media images from Euclid Early Release Observations. 61 new strong lensing system candidates are discovered by our method. However, we also identified false positives arising primarily from the simplified galaxy morphology assumptions within the simulation. This underscores the practical limitations of our approach while simultaneously highlighting potential avenues for future improvements.
△ Less
Submitted 2 April, 2024;
originally announced April 2024.
-
Simulating emission line galaxies for the next generation of large-scale structure surveys
Authors:
Wenxiang Pei,
Qi Guo,
Ming Li,
Qiao Wang,
Jiaxin Han,
Jia Hu,
Tong Su,
Liang Gao,
Jie Wang,
Yu Luo,
Chengliang Wei
Abstract:
We investigate emission line galaxies across cosmic time by combining the modified L-Galaxies semi-analytical galaxy formation model with the JiuTian cosmological simulation. We improve the tidal disruption model of satellite galaxies in L-Galaxies to address the time dependence problem. We utilise the public code CLOUDY to compute emission line ratios for a grid of HII region models. The emission…
▽ More
We investigate emission line galaxies across cosmic time by combining the modified L-Galaxies semi-analytical galaxy formation model with the JiuTian cosmological simulation. We improve the tidal disruption model of satellite galaxies in L-Galaxies to address the time dependence problem. We utilise the public code CLOUDY to compute emission line ratios for a grid of HII region models. The emission line models assume the same initial mass function as that used to generate the spectral energy distribution of semi-analytical galaxies, ensuring a coherent treatment for modelling the full galaxy spectrum. By incorporating these emission line ratios with galaxy properties, we reproduce observed luminosity functions for H$α$, H$β$, [OII], and [OIII] in the local Universe and at high redshifts. We also find good agreement between model predictions and observations for auto-correlation and cross-correlation functions of [OII]-selected galaxies, as well as their luminosity dependence. The bias of emission line galaxies depends on both luminosity and redshift. At lower redshifts, it remains constant with increasing luminosity up to around $\sim 10^{42.5}\rm \, erg\,s^{-1}$ and then rises steeply for higher luminosities. The transition luminosity increases with redshift and becomes insignificant above $z$=1.5. Generally, galaxy bias shows an increasing trend with redshift. However, for luminous galaxies, the bias is higher at low redshifts, as the strong luminosity dependence observed at low redshifts diminishes at higher redshifts. We provide a fitting formula for the bias of emission line galaxies as a function of luminosity and redshift, which can be utilised for large-scale structure studies with future galaxy surveys.
△ Less
Submitted 29 March, 2024;
originally announced April 2024.
-
Low-cost adaptive obstacle avoidance trajectory control for express delivery drone
Authors:
Yanhui Zhang,
Caisheng Wei,
Yifan Zhang,
Congcong Tian,
Weifang Chen
Abstract:
This paper studies quadcopters obstacle avoidance trajectory control (OATC) problem for express delivery. A new nonlinear adaptive learning controller that is low-cost and portable to different wheelbase sizes is proposed to adapt to large-angle maneuvers and load changes in UAV delivery missions. The controller consists of a nonlinear variable gain (NLVG) function and an extreme value search (ES)…
▽ More
This paper studies quadcopters obstacle avoidance trajectory control (OATC) problem for express delivery. A new nonlinear adaptive learning controller that is low-cost and portable to different wheelbase sizes is proposed to adapt to large-angle maneuvers and load changes in UAV delivery missions. The controller consists of a nonlinear variable gain (NLVG) function and an extreme value search (ES) algorithm to reduce overshoot and settling time. Finally, simulations were conducted on a quadcopter to verify the effectiveness of the proposed control scheme under two typical collision-free trajectories.
△ Less
Submitted 28 March, 2024;
originally announced March 2024.
-
AiOS: All-in-One-Stage Expressive Human Pose and Shape Estimation
Authors:
Qingping Sun,
Yanjun Wang,
Ailing Zeng,
Wanqi Yin,
Chen Wei,
Wenjia Wang,
Haiyi Mei,
Chi Sing Leung,
Ziwei Liu,
Lei Yang,
Zhongang Cai
Abstract:
Expressive human pose and shape estimation (a.k.a. 3D whole-body mesh recovery) involves the human body, hand, and expression estimation. Most existing methods have tackled this task in a two-stage manner, first detecting the human body part with an off-the-shelf detection model and inferring the different human body parts individually. Despite the impressive results achieved, these methods suffer…
▽ More
Expressive human pose and shape estimation (a.k.a. 3D whole-body mesh recovery) involves the human body, hand, and expression estimation. Most existing methods have tackled this task in a two-stage manner, first detecting the human body part with an off-the-shelf detection model and inferring the different human body parts individually. Despite the impressive results achieved, these methods suffer from 1) loss of valuable contextual information via cropping, 2) introducing distractions, and 3) lacking inter-association among different persons and body parts, inevitably causing performance degradation, especially for crowded scenes. To address these issues, we introduce a novel all-in-one-stage framework, AiOS, for multiple expressive human pose and shape recovery without an additional human detection step. Specifically, our method is built upon DETR, which treats multi-person whole-body mesh recovery task as a progressive set prediction problem with various sequential detection. We devise the decoder tokens and extend them to our task. Specifically, we first employ a human token to probe a human location in the image and encode global features for each instance, which provides a coarse location for the later transformer block. Then, we introduce a joint-related token to probe the human joint in the image and encoder a fine-grained local feature, which collaborates with the global feature to regress the whole-body mesh. This straightforward but effective model outperforms previous state-of-the-art methods by a 9% reduction in NMVE on AGORA, a 30% reduction in PVE on EHF, a 10% reduction in PVE on ARCTIC, and a 3% reduction in PVE on EgoBody.
△ Less
Submitted 26 March, 2024;
originally announced March 2024.
-
Offline Reinforcement Learning: Role of State Aggregation and Trajectory Data
Authors:
Zeyu Jia,
Alexander Rakhlin,
Ayush Sekhari,
Chen-Yu Wei
Abstract:
We revisit the problem of offline reinforcement learning with value function realizability but without Bellman completeness. Previous work by Xie and Jiang (2021) and Foster et al. (2022) left open the question whether a bounded concentrability coefficient along with trajectory-based offline data admits a polynomial sample complexity. In this work, we provide a negative answer to this question for…
▽ More
We revisit the problem of offline reinforcement learning with value function realizability but without Bellman completeness. Previous work by Xie and Jiang (2021) and Foster et al. (2022) left open the question whether a bounded concentrability coefficient along with trajectory-based offline data admits a polynomial sample complexity. In this work, we provide a negative answer to this question for the task of offline policy evaluation. In addition to addressing this question, we provide a rather complete picture for offline policy evaluation with only value function realizability. Our primary findings are threefold: 1) The sample complexity of offline policy evaluation is governed by the concentrability coefficient in an aggregated Markov Transition Model jointly determined by the function class and the offline data distribution, rather than that in the original MDP. This unifies and generalizes the ideas of Xie and Jiang (2021) and Foster et al. (2022), 2) The concentrability coefficient in the aggregated Markov Transition Model may grow exponentially with the horizon length, even when the concentrability coefficient in the original MDP is small and the offline data is admissible (i.e., the data distribution equals the occupancy measure of some policy), 3) Under value function realizability, there is a generic reduction that can convert any hard instance with admissible data to a hard instance with trajectory data, implying that trajectory data offers no extra benefits over admissible data. These three pieces jointly resolve the open problem, though each of them could be of independent interest.
△ Less
Submitted 25 March, 2024;
originally announced March 2024.
-
AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Tasks
Authors:
Max Ku,
Cong Wei,
Weiming Ren,
Harry Yang,
Wenhu Chen
Abstract:
In the dynamic field of digital content creation using generative models, state-of-the-art video editing models still do not offer the level of quality and control that users desire. Previous works on video editing either extended from image-based generative models in a zero-shot manner or necessitated extensive fine-tuning, which can hinder the production of fluid video edits. Furthermore, these…
▽ More
In the dynamic field of digital content creation using generative models, state-of-the-art video editing models still do not offer the level of quality and control that users desire. Previous works on video editing either extended from image-based generative models in a zero-shot manner or necessitated extensive fine-tuning, which can hinder the production of fluid video edits. Furthermore, these methods frequently rely on textual input as the editing guidance, leading to ambiguities and limiting the types of edits they can perform. Recognizing these challenges, we introduce AnyV2V, a novel tuning-free paradigm designed to simplify video editing into two primary steps: (1) employing an off-the-shelf image editing model to modify the first frame, (2) utilizing an existing image-to-video generation model to generate the edited video through temporal feature injection. AnyV2V can leverage any existing image editing tools to support an extensive array of video editing tasks, including prompt-based editing, reference-based style transfer, subject-driven editing, and identity manipulation, which were unattainable by previous methods. AnyV2V can also support any video length. Our evaluation indicates that AnyV2V significantly outperforms other baseline methods in automatic and human evaluations by significant margin, maintaining visual consistency with the source video while achieving high-quality edits across all the editing tasks.
△ Less
Submitted 10 June, 2024; v1 submitted 21 March, 2024;
originally announced March 2024.
-
WHAC: World-grounded Humans and Cameras
Authors:
Wanqi Yin,
Zhongang Cai,
Ruisi Wang,
Fanzhou Wang,
Chen Wei,
Haiyi Mei,
Weiye Xiao,
Zhitao Yang,
Qingping Sun,
Atsushi Yamashita,
Ziwei Liu,
Lei Yang
Abstract:
Estimating human and camera trajectories with accurate scale in the world coordinate system from a monocular video is a highly desirable yet challenging and ill-posed problem. In this study, we aim to recover expressive parametric human models (i.e., SMPL-X) and corresponding camera poses jointly, by leveraging the synergy between three critical players: the world, the human, and the camera. Our a…
▽ More
Estimating human and camera trajectories with accurate scale in the world coordinate system from a monocular video is a highly desirable yet challenging and ill-posed problem. In this study, we aim to recover expressive parametric human models (i.e., SMPL-X) and corresponding camera poses jointly, by leveraging the synergy between three critical players: the world, the human, and the camera. Our approach is founded on two key observations. Firstly, camera-frame SMPL-X estimation methods readily recover absolute human depth. Secondly, human motions inherently provide absolute spatial cues. By integrating these insights, we introduce a novel framework, referred to as WHAC, to facilitate world-grounded expressive human pose and shape estimation (EHPS) alongside camera pose estimation, without relying on traditional optimization techniques. Additionally, we present a new synthetic dataset, WHAC-A-Mole, which includes accurately annotated humans and cameras, and features diverse interactive human motions as well as realistic camera trajectories. Extensive experiments on both standard and newly established benchmarks highlight the superiority and efficacy of our framework. We will make the code and dataset publicly available.
△ Less
Submitted 19 March, 2024;
originally announced March 2024.
-
CSST large-scale structure analysis pipeline: I. constructing reference mock galaxy redshift surveys
Authors:
Yizhou Gu,
Xiaohu Yang,
Jiaxin Han,
Yirong Wang,
Qingyang Li,
Zhenlin Tan,
Wenkang Jiang,
Yaru Wang,
Jiaqi Wang,
Antonios Katsianis,
Xiaoju Xu,
Haojie Xu,
Wensheng Hong,
Houjun Mo,
Run Wen,
Xianzhong Zheng,
Feng Shi,
Pengjie Zhang,
Zhongxu Zhai,
Chengze Liu,
Wenting Wang,
Ying Zu,
Hong Guo,
Youcai Zhang,
Yi Lu
, et al. (7 additional authors not shown)
Abstract:
In this paper, we set out to construct a set of reference mock galaxy redshift surveys (MGRSs) for the future Chinese Space-station Survey Telescope (CSST) observation, where subsequent survey selection effects can be added and evaluated. This set of MGRSs is generated using the dark matter subhalos extracted from a high-resolution Jiutian $N$-body simulation of the standard $Λ$CDM cosmogony with…
▽ More
In this paper, we set out to construct a set of reference mock galaxy redshift surveys (MGRSs) for the future Chinese Space-station Survey Telescope (CSST) observation, where subsequent survey selection effects can be added and evaluated. This set of MGRSs is generated using the dark matter subhalos extracted from a high-resolution Jiutian $N$-body simulation of the standard $Λ$CDM cosmogony with $Ω_m=0.3111$, $Ω_Λ=0.6889$, and $σ_8=0.8102$. The simulation has a boxsize of $1~h^{-1} {\rm Gpc}$, and consists of $6144^3$ particles with mass resolution $3.723 \times 10^{8} h^{-1} M_\odot $. In order to take into account the effect of redshift evolution, we first use all 128 snapshots in the Jiutian simulation to generate a light-cone halo/subhalo catalog. Next, galaxy luminosities are assigned to the main and subhalo populations using the subhalo abundance matching (SHAM) method with the DESI $z$-band luminosity functions at different redshifts. Multi-band photometries, as well as images, are then assigned to each mock galaxy using a 3-dimensional parameter space nearest neighbor sampling of the DESI LS observational galaxies and groups. Finally, the CSST and DESI LS survey geometry and magnitude limit cuts are applied to generate the required MGRSs. As we have checked, this set of MGRSs can generally reproduce the observed galaxy luminosity/mass functions within 0.1 dex for galaxies with $L > 10^8 L_\odot$ (or $M_* > 10^{8.5} M_\odot$) and within 1-$σ$ level for galaxies with $L < 10^8L_\odot$ (or $M_* < 10^{8.5} M_\odot$). Together with the CSST slitless spectra and redshifts for our DESI LS seed galaxies that are under construction, we will set out to test various slitless observational selection effects in subsequent probes.
△ Less
Submitted 15 March, 2024;
originally announced March 2024.
-
On Tractable $Φ$-Equilibria in Non-Concave Games
Authors:
Yang Cai,
Constantinos Daskalakis,
Haipeng Luo,
Chen-Yu Wei,
Weiqiang Zheng
Abstract:
While Online Gradient Descent and other no-regret learning procedures are known to efficiently converge to a coarse correlated equilibrium in games where each agent's utility is concave in their own strategy, this is not the case when utilities are non-concave -- a common scenario in machine learning applications involving strategies parameterized by deep neural networks, or when agents' utilities…
▽ More
While Online Gradient Descent and other no-regret learning procedures are known to efficiently converge to a coarse correlated equilibrium in games where each agent's utility is concave in their own strategy, this is not the case when utilities are non-concave -- a common scenario in machine learning applications involving strategies parameterized by deep neural networks, or when agents' utilities are computed by neural networks, or both. Non-concave games introduce significant game-theoretic and optimization challenges: (i) Nash equilibria may not exist; (ii) local Nash equilibria, though existing, are intractable; and (iii) mixed Nash, correlated, and coarse correlated equilibria generally have infinite support and are intractable. To sidestep these challenges, we revisit the classical solution concept of $Φ$-equilibria introduced by Greenwald and Jafari [2003], which is guaranteed to exist for an arbitrary set of strategy modifications $Φ$ even in non-concave games [Stoltz and Lugosi, 2007]. However, the tractability of $Φ$-equilibria in such games remains elusive. In this paper, we initiate the study of tractable $Φ$-equilibria in non-concave games and examine several natural families of strategy modifications. We show that when $Φ$ is finite, there exists an efficient uncoupled learning algorithm that converges to the corresponding $Φ$-equilibria. Additionally, we explore cases where $Φ$ is infinite but consists of local modifications, showing that Online Gradient Descent can efficiently approximate $Φ$-equilibria in non-trivial regimes.
△ Less
Submitted 2 July, 2024; v1 submitted 12 March, 2024;
originally announced March 2024.
-
Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion
Authors:
Dongyang Li,
Chen Wei,
Shiying Li,
Jiachen Zou,
Quanying Liu
Abstract:
How to decode human vision through neural signals has attracted a long-standing interest in neuroscience and machine learning. Modern contrastive learning and generative models improved the performance of fMRI-based visual decoding and reconstruction. However, the high cost and low temporal resolution of fMRI limit their applications in brain-computer interfaces (BCIs), prompting a high need for E…
▽ More
How to decode human vision through neural signals has attracted a long-standing interest in neuroscience and machine learning. Modern contrastive learning and generative models improved the performance of fMRI-based visual decoding and reconstruction. However, the high cost and low temporal resolution of fMRI limit their applications in brain-computer interfaces (BCIs), prompting a high need for EEG-based visual reconstruction. In this study, we present an EEG-based visual reconstruction framework. It consists of a plug-and-play EEG encoder called the Adaptive Thinking Mapper (ATM), which is aligned with image embeddings, and a two-stage EEG guidance image generator that first transforms EEG features into image priors and then reconstructs the visual stimuli with a pre-trained image generator. Our approach allows EEG embeddings to achieve superior performance in image classification and retrieval tasks. Our two-stage image generation strategy vividly reconstructs images seen by humans. Furthermore, we analyzed the impact of signals from different time windows and brain regions on decoding and reconstruction. The versatility of our framework is demonstrated in the magnetoencephalogram (MEG) data modality. We report that EEG-based visual decoding achieves SOTA performance, highlighting the portability, low cost, and high temporal resolution of EEG, enabling a wide range of BCI applications. The code of ATM is available at https://github.com/dongyangli-del/EEG_Image_decode.
△ Less
Submitted 4 April, 2024; v1 submitted 12 March, 2024;
originally announced March 2024.
-
Suppression of flux jumps in high-$J_c$ Nb$_3$Sn conductors by ferromagnetic layer
Authors:
Cun Xue,
Kai-Wei Cao,
Tian He,
Chong Wei,
Wei Liu,
Jun-Yi Ge
Abstract:
Flux jumps observed in high-$J_c$ Nb$_3$Sn conductors are urgent problems to construct high field superconducting magnets. The low-field instabilities usually reduce the current-carrying capability and thus cause the premature quench of Nb$_3$Sn coils at low magnetic field. In this paper, we explore suppressing the flux jumps by ferromagnetic (FM) layer. Firstly, we experimentally and theoreticall…
▽ More
Flux jumps observed in high-$J_c$ Nb$_3$Sn conductors are urgent problems to construct high field superconducting magnets. The low-field instabilities usually reduce the current-carrying capability and thus cause the premature quench of Nb$_3$Sn coils at low magnetic field. In this paper, we explore suppressing the flux jumps by ferromagnetic (FM) layer. Firstly, we experimentally and theoretically investigate the flux jumps of Nb$_3$Sn/FM hybrid wires exposed to a magnetic field loop with constant sweeping rate. Comparing with bare Nb$_3$Sn and Nb$_3$Sn/Cu wires, we reveal two underlying mechanisms that the suppression of flux jumps is mainly attributed to the thermal effect of FM layer for the case of lower sweeping rate, whereas both thermal and electromagnetic effects play a crucial role for the case of higher sweeping rate. Furthermore, we explore the flux jumps of Nb$_3$Sn/FM hybrid wires exposed to AC magnetic fields with amplitude $B_{a0}$ and frequency $\rmω$. We build up the phase diagrams of flux jumps in the plane $\rmω$-$B_{a0}$ for bare Nb$_{3}$Sn wire, Nb$_{3}$Sn/Cu wire and Nb$_{3}$Sn/FM wire, respectively. We stress that the region of flux jumps of Nb$_{3}$Sn/FM wire is much smaller than the other two wires, which indicates that the Nb$_{3}$Sn/FM wire has significant advantage over merely increasing the heat capacity. The findings shed light on suppression of the flux jumps by utilizing FM materials, which is useful for developing new type of high-$J_c$ Nb$_{3}$Sn conductors.
△ Less
Submitted 4 June, 2024; v1 submitted 10 March, 2024;
originally announced March 2024.
-
Towards Generalizable Tumor Synthesis
Authors:
Qi Chen,
Xiaoxi Chen,
Haorui Song,
Zhiwei Xiong,
Alan Yuille,
Chen Wei,
Zongwei Zhou
Abstract:
Tumor synthesis enables the creation of artificial tumors in medical images, facilitating the training of AI models for tumor detection and segmentation. However, success in tumor synthesis hinges on creating visually realistic tumors that are generalizable across multiple organs and, furthermore, the resulting AI models being capable of detecting real tumors in images sourced from different domai…
▽ More
Tumor synthesis enables the creation of artificial tumors in medical images, facilitating the training of AI models for tumor detection and segmentation. However, success in tumor synthesis hinges on creating visually realistic tumors that are generalizable across multiple organs and, furthermore, the resulting AI models being capable of detecting real tumors in images sourced from different domains (e.g., hospitals). This paper made a progressive stride toward generalizable tumor synthesis by leveraging a critical observation: early-stage tumors (< 2cm) tend to have similar imaging characteristics in computed tomography (CT), whether they originate in the liver, pancreas, or kidneys. We have ascertained that generative AI models, e.g., Diffusion Models, can create realistic tumors generalized to a range of organs even when trained on a limited number of tumor examples from only one organ. Moreover, we have shown that AI models trained on these synthetic tumors can be generalized to detect and segment real tumors from CT volumes, encompassing a broad spectrum of patient demographics, imaging protocols, and healthcare facilities.
△ Less
Submitted 28 March, 2024; v1 submitted 29 February, 2024;
originally announced February 2024.
-
OPDAI at SemEval-2024 Task 6: Small LLMs can Accelerate Hallucination Detection with Weakly Supervised Data
Authors:
Chengcheng Wei,
Ze Chen,
Songtan Fang,
Jiarong He,
Max Gao
Abstract:
This paper mainly describes a unified system for hallucination detection of LLMs, which wins the second prize in the model-agnostic track of the SemEval-2024 Task 6, and also achieves considerable results in the model-aware track. This task aims to detect hallucination with LLMs for three different text-generation tasks without labeled training data. We utilize prompt engineering and few-shot lear…
▽ More
This paper mainly describes a unified system for hallucination detection of LLMs, which wins the second prize in the model-agnostic track of the SemEval-2024 Task 6, and also achieves considerable results in the model-aware track. This task aims to detect hallucination with LLMs for three different text-generation tasks without labeled training data. We utilize prompt engineering and few-shot learning to verify the performance of different LLMs on the validation data. Then we select the LLMs with better performance to generate high-quality weakly supervised training data, which not only satisfies the consistency of different LLMs, but also satisfies the consistency of the optimal LLM with different sampling parameters. Furthermore, we finetune different LLMs by using the constructed training data, and finding that a relatively small LLM can achieve a competitive level of performance in hallucination detection, when compared to the large LLMs and the prompt-based approaches using GPT-4.
△ Less
Submitted 20 February, 2024;
originally announced February 2024.
-
Both Matter: Enhancing the Emotional Intelligence of Large Language Models without Compromising the General Intelligence
Authors:
Weixiang Zhao,
Zhuojun Li,
Shilong Wang,
Yang Wang,
Yulin Hu,
Yanyan Zhao,
Chen Wei,
Bing Qin
Abstract:
Emotional Intelligence (EI), consisting of emotion perception, emotion cognition and emotion expression, plays the critical roles in improving user interaction experience for the current large language model (LLM) based conversational general AI assistants. Previous works mainly focus on raising the emotion perception ability of them via naive fine-tuning on EI-related classification or regression…
▽ More
Emotional Intelligence (EI), consisting of emotion perception, emotion cognition and emotion expression, plays the critical roles in improving user interaction experience for the current large language model (LLM) based conversational general AI assistants. Previous works mainly focus on raising the emotion perception ability of them via naive fine-tuning on EI-related classification or regression tasks. However, this leads to the incomplete enhancement of EI and catastrophic forgetting of the general intelligence (GI). To this end, we first introduce \textsc{EiBench}, a large-scale collection of EI-related tasks in the text-to-text formation with task instructions that covers all three aspects of EI, which lays a solid foundation for the comprehensive EI enhancement of LLMs. Then a novel \underline{\textbf{Mo}}dular \underline{\textbf{E}}motional \underline{\textbf{I}}ntelligence enhancement method (\textbf{MoEI}), consisting of Modular Parameter Expansion and intra-inter modulation, is proposed to comprehensively enhance the EI of LLMs without compromise their GI. Extensive experiments on two representative LLM-based assistants, Flan-T5 and LLaMA-2-Chat, demonstrate the effectiveness of MoEI to improving EI while maintain GI.
△ Less
Submitted 12 June, 2024; v1 submitted 15 February, 2024;
originally announced February 2024.
-
Cosmological Forecast of the Void Size Function Measurement from the CSST Spectroscopic Survey
Authors:
Yingxiao Song,
Qi Xiong,
Yan Gong,
Furen Deng,
Kwan Chuen Chan,
Xuelei Chen,
Qi Guo,
Jiaxin Han,
Guoliang Li,
Ming Li,
Yun Liu,
Yu Luo,
Wenxiang Pei,
Chengliang Wei
Abstract:
Void size function (VSF) contains information of the cosmic large-scale structure (LSS), and can be used to derive the properties of dark energy and dark matter. We predict the VSFs measured from the spectroscopic galaxy survey operated by the China Space Station Telescope (CSST), and study the strength of cosmological constraint. We employ a high-resolution Jiutian simulation to get CSST galaxy m…
▽ More
Void size function (VSF) contains information of the cosmic large-scale structure (LSS), and can be used to derive the properties of dark energy and dark matter. We predict the VSFs measured from the spectroscopic galaxy survey operated by the China Space Station Telescope (CSST), and study the strength of cosmological constraint. We employ a high-resolution Jiutian simulation to get CSST galaxy mock samples based on an improved semi-analytical model. We identify voids from this galaxy catalog using the watershed algorithm without assuming a spherical shape, and estimate the VSFs at different redshift bins from $z=0.5$ to 1.1. We propose a void selection method based on the ellipticity, and assume the void linear underdensity threshold $δ_{\rm v}$ in the theoretical model is redshift-dependent and set it as a free parameter in each redshift bin. The Markov Chain Monte Carlo (MCMC) method is adopted to implement the constraints on the cosmological and void parameters. We find that the CSST VSF measurement can constrain the cosmological parameters to a few percent level. The best-fit values of $δ_{\rm v}$ are ranging from $\sim-0.4$ to $-0.1$ as the redshift increases from 0.5 to 1.1, which has a distinct difference from the theoretical calculation with $δ_{\rm v}\simeq-2.7$ assuming the spherical evolution and using particles as tracer. Our method can provide a good reference for void identification and selection in the VSF analysis of the spectroscopic galaxy surveys.
△ Less
Submitted 24 June, 2024; v1 submitted 8 February, 2024;
originally announced February 2024.
-
ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generation
Authors:
Weiming Ren,
Huan Yang,
Ge Zhang,
Cong Wei,
Xinrun Du,
Wenhao Huang,
Wenhu Chen
Abstract:
Image-to-video (I2V) generation aims to use the initial frame (alongside a text prompt) to create a video sequence. A grand challenge in I2V generation is to maintain visual consistency throughout the video: existing methods often struggle to preserve the integrity of the subject, background, and style from the first frame, as well as ensure a fluid and logical progression within the video narrati…
▽ More
Image-to-video (I2V) generation aims to use the initial frame (alongside a text prompt) to create a video sequence. A grand challenge in I2V generation is to maintain visual consistency throughout the video: existing methods often struggle to preserve the integrity of the subject, background, and style from the first frame, as well as ensure a fluid and logical progression within the video narrative. To mitigate these issues, we propose ConsistI2V, a diffusion-based method to enhance visual consistency for I2V generation. Specifically, we introduce (1) spatiotemporal attention over the first frame to maintain spatial and motion consistency, (2) noise initialization from the low-frequency band of the first frame to enhance layout consistency. These two approaches enable ConsistI2V to generate highly consistent videos. We also extend the proposed approaches to show their potential to improve consistency in auto-regressive long video generation and camera motion control. To verify the effectiveness of our method, we propose I2V-Bench, a comprehensive evaluation benchmark for I2V generation. Our automatic and human evaluation results demonstrate the superiority of ConsistI2V over existing methods.
△ Less
Submitted 30 June, 2024; v1 submitted 6 February, 2024;
originally announced February 2024.
-
Planar Graphs with Homomorphisms to the 9-cycle
Authors:
Daniel W. Cranston,
Jiaao Li,
Zhouningxin Wang,
Chunyan Wei
Abstract:
We study the problem of finding homomorphisms into odd cycles from planar graphs with high odd-girth. The Jaeger-Zhang conjecture states that every planar graph of odd-girth at least $4k+1$ admits a homomorphism to the odd cycle $C_{2k+1}$. The $k=1$ case is the well-known Grötzsch's $3$-coloring theorem. For general $k$, in 2013 Lovász, Thomassen, Wu, and Zhang showed that it suffices to have odd…
▽ More
We study the problem of finding homomorphisms into odd cycles from planar graphs with high odd-girth. The Jaeger-Zhang conjecture states that every planar graph of odd-girth at least $4k+1$ admits a homomorphism to the odd cycle $C_{2k+1}$. The $k=1$ case is the well-known Grötzsch's $3$-coloring theorem. For general $k$, in 2013 Lovász, Thomassen, Wu, and Zhang showed that it suffices to have odd-girth at least $6k+1$. Improvements are known for $C_5$ and $C_7$ in [Combinatorica 2017, SIDMA 2020, Combinatorica 2022]. For $C_9$ we improve this hypothesis by showing that it suffices to have odd-girth 23. Our main tool is a variation on the potential method applied to modular orientations. This allows more flexibility when seeking reducible configurations. The same techniques also prove some results on circular coloring of signed planar graphs.
△ Less
Submitted 4 February, 2024;
originally announced February 2024.
-
Integration of cognitive tasks into artificial general intelligence test for large models
Authors:
Youzhi Qu,
Chen Wei,
Penghui Du,
Wenxin Che,
Chi Zhang,
Wanli Ouyang,
Yatao Bian,
Feiyang Xu,
Bin Hu,
Kai Du,
Haiyan Wu,
Jia Liu,
Quanying Liu
Abstract:
During the evolution of large models, performance evaluation is necessarily performed to assess their capabilities and ensure safety before practical application. However, current model evaluations mainly rely on specific tasks and datasets, lacking a united framework for assessing the multidimensional intelligence of large models. In this perspective, we advocate for a comprehensive framework of…
▽ More
During the evolution of large models, performance evaluation is necessarily performed to assess their capabilities and ensure safety before practical application. However, current model evaluations mainly rely on specific tasks and datasets, lacking a united framework for assessing the multidimensional intelligence of large models. In this perspective, we advocate for a comprehensive framework of cognitive science-inspired artificial general intelligence (AGI) tests, aimed at fulfilling the testing needs of large models with enhanced capabilities. The cognitive science-inspired AGI tests encompass the full spectrum of intelligence facets, including crystallized intelligence, fluid intelligence, social intelligence, and embodied intelligence. To assess the multidimensional intelligence of large models, the AGI tests consist of a battery of well-designed cognitive tests adopted from human intelligence tests, and then naturally encapsulates into an immersive virtual community. We propose increasing the complexity of AGI testing tasks commensurate with advancements in large models and emphasizing the necessity for the interpretation of test results to avoid false negatives and false positives. We believe that cognitive science-inspired AGI tests will effectively guide the targeted improvement of large models in specific dimensions of intelligence and accelerate the integration of large models into human society.
△ Less
Submitted 5 March, 2024; v1 submitted 4 February, 2024;
originally announced February 2024.
-
Advancing EEG/MEG Source Imaging with Geometric-Informed Basis Functions
Authors:
Song Wang,
Chen Wei,
Kexin Lou,
Dongfeng Gu,
Quanying Liu
Abstract:
Electroencephalography (EEG) and Magnetoencephalography (MEG) are pivotal in understanding brain activity but are limited by their poor spatial resolution. EEG/MEG source imaging (ESI) infers the high-resolution electric field distribution in the brain based on the low-resolution scalp EEG/MEG observations. However, the ESI problem is ill-posed, and how to bring neuroscience priors into ESI method…
▽ More
Electroencephalography (EEG) and Magnetoencephalography (MEG) are pivotal in understanding brain activity but are limited by their poor spatial resolution. EEG/MEG source imaging (ESI) infers the high-resolution electric field distribution in the brain based on the low-resolution scalp EEG/MEG observations. However, the ESI problem is ill-posed, and how to bring neuroscience priors into ESI method is the key. Here, we present a novel method which utilizes the Brain Geometric-informed Basis Functions (GBFs) as priors to enhance EEG/MEG source imaging. Through comprehensive experiments on both synthetic data and real task EEG data, we demonstrate the superiority of GBFs over traditional spatial basis functions (e.g., Harmonic and MSP), as well as existing ESI methods (e.g., dSPM, MNE, sLORETA, eLORETA). GBFs provide robust ESI results under different noise levels, and result in biologically interpretable EEG sources. We believe the high-resolution EEG source imaging from GBFs will greatly advance neuroscience research.
△ Less
Submitted 31 January, 2024;
originally announced January 2024.
-
WGAN-AFL: Seed Generation Augmented Fuzzer with Wasserstein-GAN
Authors:
Liqun Yang,
Chunan Li,
Yongxin Qiu,
Chaoren Wei,
Jian Yang,
Hongcheng Guo,
Jinxin Ma,
Zhoujun Li
Abstract:
The importance of addressing security vulnerabilities is indisputable, with software becoming crucial in sectors such as national defense and finance. Consequently, The security issues caused by software vulnerabilities cannot be ignored. Fuzz testing is an automated software testing technology that can detect vulnerabilities in the software. However, most previous fuzzers encounter challenges tha…
▽ More
The importance of addressing security vulnerabilities is indisputable, with software becoming crucial in sectors such as national defense and finance. Consequently, The security issues caused by software vulnerabilities cannot be ignored. Fuzz testing is an automated software testing technology that can detect vulnerabilities in the software. However, most previous fuzzers encounter challenges that fuzzing performance is sensitive to initial input seeds. In the absence of high-quality initial input seeds, fuzzers may expend significant resources on program path exploration, leading to a substantial decrease in the efficiency of vulnerability detection. To address this issue, we propose WGAN-AFL. By collecting high-quality testcases, we train a generative adversarial network (GAN) to learn their features, thereby obtaining high-quality initial input seeds. To overcome drawbacks like mode collapse and training instability inherent in GANs, we utilize the Wasserstein GAN (WGAN) architecture for training, further enhancing the quality of the generated seeds. Experimental results demonstrate that WGAN-AFL significantly outperforms the original AFL in terms of code coverage, new paths, and vulnerability discovery, demonstrating the effective enhancement of seed quality by WGAN-AFL.
△ Less
Submitted 30 January, 2024;
originally announced January 2024.
-
Hometronics: Accessible production of graphene suspensions for health sensing applications using only household items
Authors:
Adel K. A. Aljarid,
Jasper Winder,
Cencen Wei,
Arvind Venkatraman,
Oliver Tomes,
Aaron Soul,
Dimitrios G. Papageorgiou,
Matthias E. Möbius,
Conor S. Boland
Abstract:
Nanoscience at times can seem out of reach to the developing world and the general public, with much of the equipment expensive and knowledge seemingly esoteric to nonexperts. Using only cheap, everyday household items, accessible research with real applications can be shown. Here, graphene suspensions were produced using pencil lead, tap water, kitchen appliances, soaps and coffee filters, with a…
▽ More
Nanoscience at times can seem out of reach to the developing world and the general public, with much of the equipment expensive and knowledge seemingly esoteric to nonexperts. Using only cheap, everyday household items, accessible research with real applications can be shown. Here, graphene suspensions were produced using pencil lead, tap water, kitchen appliances, soaps and coffee filters, with a childrens glue based graphene nanocomposite for highly sensitive pulse measurements demonstrated.
△ Less
Submitted 27 January, 2024;
originally announced January 2024.
-
Near-Optimal Policy Optimization for Correlated Equilibrium in General-Sum Markov Games
Authors:
Yang Cai,
Haipeng Luo,
Chen-Yu Wei,
Weiqiang Zheng
Abstract:
We study policy optimization algorithms for computing correlated equilibria in multi-player general-sum Markov Games. Previous results achieve $O(T^{-1/2})$ convergence rate to a correlated equilibrium and an accelerated $O(T^{-3/4})$ convergence rate to the weaker notion of coarse correlated equilibrium. In this paper, we improve both results significantly by providing an uncoupled policy optimiz…
▽ More
We study policy optimization algorithms for computing correlated equilibria in multi-player general-sum Markov Games. Previous results achieve $O(T^{-1/2})$ convergence rate to a correlated equilibrium and an accelerated $O(T^{-3/4})$ convergence rate to the weaker notion of coarse correlated equilibrium. In this paper, we improve both results significantly by providing an uncoupled policy optimization algorithm that attains a near-optimal $\tilde{O}(T^{-1})$ convergence rate for computing a correlated equilibrium. Our algorithm is constructed by combining two main elements (i) smooth value updates and (ii) the optimistic-follow-the-regularized-leader algorithm with the log barrier regularizer.
△ Less
Submitted 1 May, 2024; v1 submitted 26 January, 2024;
originally announced January 2024.
-
CUI@CHI 2024: Building Trust in CUIs-From Design to Deployment
Authors:
Smit Desai,
Christina Wei,
Jaisie Sin,
Mateusz Dubiel,
Nima Zargham,
Shashank Ahire,
Martin Porcheron,
Anastasia Kuzminykh,
Minha Lee,
Heloisa Candello,
Joel Fischer,
Cosmin Munteanu,
Benjamin R Cowan
Abstract:
Conversational user interfaces (CUIs) have become an everyday technology for people the world over, as well as a booming area of research. Advances in voice synthesis and the emergence of chatbots powered by large language models (LLMs), notably ChatGPT, have pushed CUIs to the forefront of human-computer interaction (HCI) research and practice. Now that these technologies enable an elemental leve…
▽ More
Conversational user interfaces (CUIs) have become an everyday technology for people the world over, as well as a booming area of research. Advances in voice synthesis and the emergence of chatbots powered by large language models (LLMs), notably ChatGPT, have pushed CUIs to the forefront of human-computer interaction (HCI) research and practice. Now that these technologies enable an elemental level of usability and user experience (UX), we must turn our attention to higher-order human factors: trust and reliance. In this workshop, we aim to bring together a multidisciplinary group of researchers and practitioners invested in the next phase of CUI design. Through keynotes, presentations, and breakout sessions, we will share our knowledge, identify cutting-edge resources, and fortify an international network of CUI scholars. In particular, we will engage with the complexity of trust and reliance as attitudes and behaviours that emerge when people interact with conversational agents.
△ Less
Submitted 25 January, 2024;
originally announced January 2024.
-
PubTator 3.0: an AI-powered Literature Resource for Unlocking Biomedical Knowledge
Authors:
Chih-Hsuan Wei,
Alexis Allot,
Po-Ting Lai,
Robert Leaman,
Shubo Tian,
Ling Luo,
Qiao Jin,
Zhizheng Wang,
Qingyu Chen,
Zhiyong Lu
Abstract:
PubTator 3.0 (https://www.ncbi.nlm.nih.gov/research/pubtator3/) is a biomedical literature resource using state-of-the-art AI techniques to offer semantic and relation searches for key concepts like proteins, genetic variants, diseases, and chemicals. It currently provides over one billion entity and relation annotations across approximately 36 million PubMed abstracts and 6 million full-text arti…
▽ More
PubTator 3.0 (https://www.ncbi.nlm.nih.gov/research/pubtator3/) is a biomedical literature resource using state-of-the-art AI techniques to offer semantic and relation searches for key concepts like proteins, genetic variants, diseases, and chemicals. It currently provides over one billion entity and relation annotations across approximately 36 million PubMed abstracts and 6 million full-text articles from the PMC open access subset, updated weekly. PubTator 3.0's online interface and API utilize these precomputed entity relations and synonyms to provide advanced search capabilities and enable large-scale analyses, streamlining many complex information needs. We showcase the retrieval quality of PubTator 3.0 using a series of entity pair queries, demonstrating that PubTator 3.0 retrieves a greater number of articles than either PubMed or Google Scholar, with higher precision in the top 20 results. We further show that integrating ChatGPT (GPT-4) with PubTator APIs dramatically improves the factuality and verifiability of its responses. In summary, PubTator 3.0 offers a comprehensive set of features and tools that allow researchers to navigate the ever-expanding wealth of biomedical literature, expediting research and unlocking valuable insights for scientific discovery.
△ Less
Submitted 19 January, 2024;
originally announced January 2024.