-
Multi-Label Learning with Stronger Consistency Guarantees
Authors:
Anqi Mao,
Mehryar Mohri,
Yutao Zhong
Abstract:
We present a detailed study of surrogate losses and algorithms for multi-label learning, supported by $H$-consistency bounds. We first show that, for the simplest form of multi-label loss (the popular Hamming loss), the well-known consistent binary relevance surrogate suffers from a sub-optimal dependency on the number of labels in terms of $H$-consistency bounds, when using smooth losses such as…
▽ More
We present a detailed study of surrogate losses and algorithms for multi-label learning, supported by $H$-consistency bounds. We first show that, for the simplest form of multi-label loss (the popular Hamming loss), the well-known consistent binary relevance surrogate suffers from a sub-optimal dependency on the number of labels in terms of $H$-consistency bounds, when using smooth losses such as logistic losses. Furthermore, this loss function fails to account for label correlations. To address these drawbacks, we introduce a novel surrogate loss, multi-label logistic loss, that accounts for label correlations and benefits from label-independent $H$-consistency bounds. We then broaden our analysis to cover a more extensive family of multi-label losses, including all common ones and a new extension defined based on linear-fractional functions with respect to the confusion matrix. We also extend our multi-label logistic losses to more comprehensive multi-label comp-sum losses, adapting comp-sum losses from standard classification to the multi-label learning. We prove that this family of surrogate losses benefits from $H$-consistency bounds, and thus Bayes-consistency, across any general multi-label loss. Our work thus proposes a unified surrogate loss framework benefiting from strong consistency guarantees for any multi-label loss, significantly expanding upon previous work which only established Bayes-consistency and for specific loss functions. Additionally, we adapt constrained losses from standard classification to multi-label constrained losses in a similar way, which also benefit from $H$-consistency bounds and thus Bayes-consistency for any multi-label loss. We further describe efficient gradient computation algorithms for minimizing the multi-label logistic loss.
△ Less
Submitted 18 July, 2024;
originally announced July 2024.
-
Realizable $H$-Consistent and Bayes-Consistent Loss Functions for Learning to Defer
Authors:
Anqi Mao,
Mehryar Mohri,
Yutao Zhong
Abstract:
We present a comprehensive study of surrogate loss functions for learning to defer. We introduce a broad family of surrogate losses, parameterized by a non-increasing function $Ψ$, and establish their realizable $H$-consistency under mild conditions. For cost functions based on classification error, we further show that these losses admit $H$-consistency bounds when the hypothesis set is symmetric…
▽ More
We present a comprehensive study of surrogate loss functions for learning to defer. We introduce a broad family of surrogate losses, parameterized by a non-increasing function $Ψ$, and establish their realizable $H$-consistency under mild conditions. For cost functions based on classification error, we further show that these losses admit $H$-consistency bounds when the hypothesis set is symmetric and complete, a property satisfied by common neural network and linear function hypothesis sets. Our results also resolve an open question raised in previous work (Mozannar et al., 2023) by proving the realizable $H$-consistency and Bayes-consistency of a specific surrogate loss. Furthermore, we identify choices of $Ψ$ that lead to $H$-consistent surrogate losses for any general cost function, thus achieving Bayes-consistency, realizable $H$-consistency, and $H$-consistency bounds simultaneously. We also investigate the relationship between $H$-consistency bounds and realizable $H$-consistency in learning to defer, highlighting key differences from standard classification. Finally, we empirically evaluate our proposed surrogate losses and compare them with existing baselines.
△ Less
Submitted 18 July, 2024;
originally announced July 2024.
-
Enhanced $H$-Consistency Bounds
Authors:
Anqi Mao,
Mehryar Mohri,
Yutao Zhong
Abstract:
Recent research has introduced a key notion of $H$-consistency bounds for surrogate losses. These bounds offer finite-sample guarantees, quantifying the relationship between the zero-one estimation error (or other target loss) and the surrogate loss estimation error for a specific hypothesis set. However, previous bounds were derived under the condition that a lower bound of the surrogate loss con…
▽ More
Recent research has introduced a key notion of $H$-consistency bounds for surrogate losses. These bounds offer finite-sample guarantees, quantifying the relationship between the zero-one estimation error (or other target loss) and the surrogate loss estimation error for a specific hypothesis set. However, previous bounds were derived under the condition that a lower bound of the surrogate loss conditional regret is given as a convex function of the target conditional regret, without non-constant factors depending on the predictor or input instance. Can we derive finer and more favorable $H$-consistency bounds? In this work, we relax this condition and present a general framework for establishing enhanced $H$-consistency bounds based on more general inequalities relating conditional regrets. Our theorems not only subsume existing results as special cases but also enable the derivation of more favorable bounds in various scenarios. These include standard multi-class classification, binary and multi-class classification under Tsybakov noise conditions, and bipartite ranking.
△ Less
Submitted 18 July, 2024;
originally announced July 2024.
-
Spontaneous Style Text-to-Speech Synthesis with Controllable Spontaneous Behaviors Based on Language Models
Authors:
Weiqin Li,
Peiji Yang,
Yicheng Zhong,
Yixuan Zhou,
Zhisheng Wang,
Zhiyong Wu,
Xixin Wu,
Helen Meng
Abstract:
Spontaneous style speech synthesis, which aims to generate human-like speech, often encounters challenges due to the scarcity of high-quality data and limitations in model capabilities. Recent language model-based TTS systems can be trained on large, diverse, and low-quality speech datasets, resulting in highly natural synthesized speech. However, they are limited by the difficulty of simulating v…
▽ More
Spontaneous style speech synthesis, which aims to generate human-like speech, often encounters challenges due to the scarcity of high-quality data and limitations in model capabilities. Recent language model-based TTS systems can be trained on large, diverse, and low-quality speech datasets, resulting in highly natural synthesized speech. However, they are limited by the difficulty of simulating various spontaneous behaviors and capturing prosody variations in spontaneous speech. In this paper, we propose a novel spontaneous speech synthesis system based on language models. We systematically categorize and uniformly model diverse spontaneous behaviors. Moreover, fine-grained prosody modeling is introduced to enhance the model's ability to capture subtle prosody variations in spontaneous speech.Experimental results show that our proposed method significantly outperforms the baseline methods in terms of prosody naturalness and spontaneous behavior naturalness.
△ Less
Submitted 18 July, 2024;
originally announced July 2024.
-
Diffusion Model-based Incentive Mechanism with Prospect Theory for Edge AIGC Services in 6G IoT
Authors:
Jinbo Wen,
Jiangtian Nie,
Yue Zhong,
Changyan Yi,
Xiaohuan Li,
Jiangming Jin,
Yang Zhang,
Dusit Niyato
Abstract:
The fusion of Internet of Things (IoT) with Sixth-Generation (6G) technology has significant potential to revolutionize the IoT landscape. Utilizing the ultra-reliable and low-latency communication capabilities of 6G, 6G-IoT networks can transmit high-quality and diverse data to enhance edge learning. Artificial Intelligence-Generated Content (AIGC) harnesses advanced AI algorithms to automaticall…
▽ More
The fusion of Internet of Things (IoT) with Sixth-Generation (6G) technology has significant potential to revolutionize the IoT landscape. Utilizing the ultra-reliable and low-latency communication capabilities of 6G, 6G-IoT networks can transmit high-quality and diverse data to enhance edge learning. Artificial Intelligence-Generated Content (AIGC) harnesses advanced AI algorithms to automatically generate various types of content. The emergence of edge AIGC integrates with edge networks, facilitating real-time provision of customized AIGC services by deploying AIGC models on edge devices. However, the current practice of edge devices as AIGC Service Providers (ASPs) lacks incentives, hindering the sustainable provision of high-quality edge AIGC services amidst information asymmetry. In this paper, we develop a user-centric incentive mechanism framework for edge AIGC services in 6G-IoT networks. Specifically, we first propose a contract theory model for incentivizing ASPs to provide AIGC services to clients. Recognizing the irrationality of clients towards personalized AIGC services, we utilize Prospect Theory (PT) to better capture the subjective utility of clients. Finally, we adopt the generative diffusion model to generate the optimal contract design under PT, outperforming traditional deep reinforcement learning algorithms, i.e., soft actor-critic algorithms. Our numerical results demonstrate the effectiveness of the proposed scheme.
△ Less
Submitted 10 June, 2024;
originally announced July 2024.
-
Low-Rank Interconnected Adaptation Across Layers
Authors:
Yibo Zhong,
Yao Zhou
Abstract:
Low-rank adaptation (LoRA), as one of the most well-known representative methods of parameter-efficient fine-tuning, freezes the backbone model and introduces parallel adapter modules to each layer of the model. These modules consist of two low-rank trainable matrices: a low-dimension projector (LP) and a high-dimension projector (HP) with their product approximating the change for updating the mo…
▽ More
Low-rank adaptation (LoRA), as one of the most well-known representative methods of parameter-efficient fine-tuning, freezes the backbone model and introduces parallel adapter modules to each layer of the model. These modules consist of two low-rank trainable matrices: a low-dimension projector (LP) and a high-dimension projector (HP) with their product approximating the change for updating the model weight. However, LoRA's paired LP and HP per layer limit learned weights to specific features, ignoring the varied information extracted by stacked layers in models like Transformers. By considering the differences between layers and establishing connections across them when learning the weights, we enhance the capture of relevant information for downstream tasks using this interconnected adaptation when fine-tuning. Meanwhile, preserving the unique characteristics of each layer and thus selectively mix the learning traits of various layers according to a specific ratio can also be crucial in certain tasks. In this paper, we propose Low-rank Interconnected adaptation across layers (Lily). Specifically, we retain layer-specific LPs (local LPs) for low-dimensional feature projection and unify all HPs into a model-wide global HP, thereby overcoming layer-specific constraints in LoRA. The global HP, layer-independent, supports multiple HP sub-modules, or inspired by Mixture of Experts (MoE), HP experts capturing learning traits across all layer depths. For the ratio to mix all the experts, we use a router inspired by MoE to selectively adapt the features of different layers, thus obtaining a unique expert distribution. We evaluated Lily on a wide range of downstream tasks and achieved state-of-the-art results, outperforming LoRA and a range of competitive methods. Code will be available at https://github.com/blameitonme1/lily.
△ Less
Submitted 13 July, 2024;
originally announced July 2024.
-
Convergences of Combinatorial Ricci Flows to Degenerated Circle Packings in Hyperbolic Background Geometry
Authors:
Guangming Hu,
Sicheng Lu,
Dong Tan,
Youliang Zhong,
Puchun Zhou
Abstract:
This paper investigates a kind of degenerated circle packings in hyperbolic background geometry. A main problem is whether a prescribed total geodesic curvature data can be realized by a degenerated circle packing or not. We fully characterize the sufficient and necessary conditions and show the uniqueness. Furthermore, we introduce the combinatoral Ricci flow to find the desired degenerated circl…
▽ More
This paper investigates a kind of degenerated circle packings in hyperbolic background geometry. A main problem is whether a prescribed total geodesic curvature data can be realized by a degenerated circle packing or not. We fully characterize the sufficient and necessary conditions and show the uniqueness. Furthermore, we introduce the combinatoral Ricci flow to find the desired degenerated circle packed surface, analougus to the methods of Chow-Luo and Takatsu.
△ Less
Submitted 11 July, 2024;
originally announced July 2024.
-
Label-anticipated Event Disentanglement for Audio-Visual Video Parsing
Authors:
Jinxing Zhou,
Dan Guo,
Yuxin Mao,
Yiran Zhong,
Xiaojun Chang,
Meng Wang
Abstract:
Audio-Visual Video Parsing (AVVP) task aims to detect and temporally locate events within audio and visual modalities. Multiple events can overlap in the timeline, making identification challenging. While traditional methods usually focus on improving the early audio-visual encoders to embed more effective features, the decoding phase -- crucial for final event classification, often receives less…
▽ More
Audio-Visual Video Parsing (AVVP) task aims to detect and temporally locate events within audio and visual modalities. Multiple events can overlap in the timeline, making identification challenging. While traditional methods usually focus on improving the early audio-visual encoders to embed more effective features, the decoding phase -- crucial for final event classification, often receives less attention. We aim to advance the decoding phase and improve its interpretability. Specifically, we introduce a new decoding paradigm, \underline{l}abel s\underline{e}m\underline{a}ntic-based \underline{p}rojection (LEAP), that employs labels texts of event categories, each bearing distinct and explicit semantics, for parsing potentially overlapping events.LEAP works by iteratively projecting encoded latent features of audio/visual segments onto semantically independent label embeddings. This process, enriched by modeling cross-modal (audio/visual-label) interactions, gradually disentangles event semantics within video segments to refine relevant label embeddings, guaranteeing a more discriminative and interpretable decoding process. To facilitate the LEAP paradigm, we propose a semantic-aware optimization strategy, which includes a novel audio-visual semantic similarity loss function. This function leverages the Intersection over Union of audio and visual events (EIoU) as a novel metric to calibrate audio-visual similarities at the feature level, accommodating the varied event densities across modalities. Extensive experiments demonstrate the superiority of our method, achieving new state-of-the-art performance for AVVP and also enhancing the relevant audio-visual event localization task.
△ Less
Submitted 10 July, 2024;
originally announced July 2024.
-
Weakly-supervised Medical Image Segmentation with Gaze Annotations
Authors:
Yuan Zhong,
Chenhui Tang,
Yumeng Yang,
Ruoxi Qi,
Kang Zhou,
Yuqi Gong,
Pheng Ann Heng,
Janet H. Hsiao,
Qi Dou
Abstract:
Eye gaze that reveals human observational patterns has increasingly been incorporated into solutions for vision tasks. Despite recent explorations on leveraging gaze to aid deep networks, few studies exploit gaze as an efficient annotation approach for medical image segmentation which typically entails heavy annotating costs. In this paper, we propose to collect dense weak supervision for medical…
▽ More
Eye gaze that reveals human observational patterns has increasingly been incorporated into solutions for vision tasks. Despite recent explorations on leveraging gaze to aid deep networks, few studies exploit gaze as an efficient annotation approach for medical image segmentation which typically entails heavy annotating costs. In this paper, we propose to collect dense weak supervision for medical image segmentation with a gaze annotation scheme. To train with gaze, we propose a multi-level framework that trains multiple networks from discriminative human attention, simulated with a set of pseudo-masks derived by applying hierarchical thresholds on gaze heatmaps. Furthermore, to mitigate gaze noise, a cross-level consistency is exploited to regularize overfitting noisy labels, steering models toward clean patterns learned by peer networks. The proposed method is validated on two public medical datasets of polyp and prostate segmentation tasks. We contribute a high-quality gaze dataset entitled GazeMedSeg as an extension to the popular medical segmentation datasets. To the best of our knowledge, this is the first gaze dataset for medical image segmentation. Our experiments demonstrate that gaze annotation outperforms previous label-efficient annotation schemes in terms of both performance and annotation time. Our collected gaze data and code are available at: https://github.com/med-air/GazeMedSeg.
△ Less
Submitted 10 July, 2024;
originally announced July 2024.
-
Cardinality-Aware Set Prediction and Top-$k$ Classification
Authors:
Corinna Cortes,
Anqi Mao,
Christopher Mohri,
Mehryar Mohri,
Yutao Zhong
Abstract:
We present a detailed study of cardinality-aware top-$k$ classification, a novel approach that aims to learn an accurate top-$k$ set predictor while maintaining a low cardinality. We introduce a new target loss function tailored to this setting that accounts for both the classification error and the cardinality of the set predicted. To optimize this loss function, we propose two families of surrog…
▽ More
We present a detailed study of cardinality-aware top-$k$ classification, a novel approach that aims to learn an accurate top-$k$ set predictor while maintaining a low cardinality. We introduce a new target loss function tailored to this setting that accounts for both the classification error and the cardinality of the set predicted. To optimize this loss function, we propose two families of surrogate losses: cost-sensitive comp-sum losses and cost-sensitive constrained losses. Minimizing these loss functions leads to new cardinality-aware algorithms that we describe in detail in the case of both top-$k$ and threshold-based classifiers. We establish $H$-consistency bounds for our cardinality-aware surrogate loss functions, thereby providing a strong theoretical foundation for our algorithms. We report the results of extensive experiments on CIFAR-10, CIFAR-100, ImageNet, and SVHN datasets demonstrating the effectiveness and benefits of our cardinality-aware algorithms.
△ Less
Submitted 9 July, 2024;
originally announced July 2024.
-
ERQ: Error Reduction for Post-Training Quantization of Vision Transformers
Authors:
Yunshan Zhong,
Jiawei Hu,
You Huang,
Yuxin Zhang,
Rongrong Ji
Abstract:
Post-training quantization (PTQ) for vision transformers (ViTs) has garnered significant attention due to its efficiency in compressing models. However, existing methods typically overlook the intricate interdependence between quantized weight and activation, leading to considerable quantization error. In this paper, we propose ERQ, a two-step PTQ approach meticulously crafted to sequentially redu…
▽ More
Post-training quantization (PTQ) for vision transformers (ViTs) has garnered significant attention due to its efficiency in compressing models. However, existing methods typically overlook the intricate interdependence between quantized weight and activation, leading to considerable quantization error. In this paper, we propose ERQ, a two-step PTQ approach meticulously crafted to sequentially reduce the quantization error arising from activation and weight quantization. ERQ first introduces Activation quantization error reduction (Aqer) that strategically formulates the minimization of activation quantization error as a Ridge Regression problem, tackling it by updating weights with full-precision. Subsequently, ERQ introduces Weight quantization error reduction (Wqer) that adopts an iterative approach to mitigate the quantization error induced by weight quantization. In each iteration, an empirically derived, efficient proxy is employed to refine the rounding directions of quantized weights, coupled with a Ridge Regression solver to curtail weight quantization error. Experimental results attest to the effectiveness of our approach. Notably, ERQ surpasses the state-of-the-art GPTQ by 22.36% in accuracy for W3A4 ViT-S.
△ Less
Submitted 9 July, 2024;
originally announced July 2024.
-
Altermagnetism in Heavy Fermion Systems
Authors:
Miaomiao Zhao,
Wei-Wei Yang,
Xueming Guo,
Hong-Gang Luo,
Yin Zhong
Abstract:
Novel collinear magnet, the altermagnet (AM) with spin-splitting energy band and zero net magnetization have attracted great interest due to its potential spintronic applications. Here, we demonstrate AM-like phases in a microscopic Kondo lattice model, widely used for heavy fermion compounds. With the framework of fermionic parton mean-field theory, we find the $d$-wave AM state can coexist with…
▽ More
Novel collinear magnet, the altermagnet (AM) with spin-splitting energy band and zero net magnetization have attracted great interest due to its potential spintronic applications. Here, we demonstrate AM-like phases in a microscopic Kondo lattice model, widely used for heavy fermion compounds. With the framework of fermionic parton mean-field theory, we find the $d$-wave AM state can coexist with the intrinsic Kondo screening effect in such itinerant-local electron system if an alternating next-nearest-neighbor-hopping (NNNH) is included. Such alternating NNNH take nonmagnetic atoms, neglected in usual antiferromagnetism study, into account when encountering real-life candidate AM materials. The AM-like states are characterized by their spin-splitting quasiparticle bands, Fermi surface, spin-resolved distribution function and conductivity. It is suggested that the magnetic quantum oscillation and charge transport measurement can detect those AM-like phases. We hope the present work may be useful for exploring AM-like phases in $f$-electron compounds.
△ Less
Submitted 6 July, 2024;
originally announced July 2024.
-
Spindown of Pulsars Interacting with Companion Winds: Impact of Magnetospheric Compression
Authors:
Yici Zhong,
Anatoly Spitkovsky,
Jens F. Mahlmann,
Hayk Hakobyan
Abstract:
The presence of a companion wind in neutron star binary systems can form a contact discontinuity well within the pulsar's light cylinder, effectively creating a waveguide that confines the pulsar's electromagnetic fields and significantly alters its spindown. We parametrize this confinement as the ratio between the equatorial position of the contact discontinuity (or standoff distance)…
▽ More
The presence of a companion wind in neutron star binary systems can form a contact discontinuity well within the pulsar's light cylinder, effectively creating a waveguide that confines the pulsar's electromagnetic fields and significantly alters its spindown. We parametrize this confinement as the ratio between the equatorial position of the contact discontinuity (or standoff distance) $R_\mathrm{m}$ and the pulsar's light cylinder $R_\mathrm{LC}$. We quantify the pulsar spindown for relativistic wind envelopes with $R_\mathrm{m}/R_\mathrm{LC} = 1/3...1$ and varying inclination angles $χ$ between magnetic and rotation axes using particle-in-cell simulations. Our strongly confined models ($R_\mathrm{m}/R_\mathrm{LC} = 1/3$) identify two distinct limits. For $χ=0^\circ$, the spindown induced by the compressed pulsar magnetosphere is enhanced by approximately three times compared to an isolated pulsar due to an increased number of open magnetic field lines. Conversely, for $χ=90^\circ$, the compressed system spins down at less than $40\%$ of the rate of an isolated reference pulsar due to the mismatch between the pulsar wind stripe wavelength and the waveguide size. We directly apply our analysis to the 2.77-second period oblique rotator ($χ=60^\circ$) in the double pulsar system PSR J0737-3039. With the numerically derived spindown estimate, we constrain its surface magnetic field to $B_* \approx (7.3 \pm 0.2) \times 10^{11}$ G. We discuss the time modulation of its period derivative, the effects of compression on its braking index, and implications for the radio eclipse in PSR J0737-3039.
△ Less
Submitted 5 July, 2024;
originally announced July 2024.
-
Performance of Medical Image Fusion in High-level Analysis Tasks: A Mutual Enhancement Framework for Unaligned PAT and MRI Image Fusion
Authors:
Yutian Zhong,
Jinchuan He,
Zhichao Liang,
Shuangyang Zhang,
Qianjin Feng,
Wufan Chen,
Li Qi
Abstract:
Photoacoustic tomography (PAT) offers optical contrast, whereas magnetic resonance imaging (MRI) excels in imaging soft tissue and organ anatomy. The fusion of PAT with MRI holds promising application prospects due to their complementary advantages. Existing image fusion have made considerable progress in pre-registered images, yet spatial deformations are difficult to avoid in medical imaging sce…
▽ More
Photoacoustic tomography (PAT) offers optical contrast, whereas magnetic resonance imaging (MRI) excels in imaging soft tissue and organ anatomy. The fusion of PAT with MRI holds promising application prospects due to their complementary advantages. Existing image fusion have made considerable progress in pre-registered images, yet spatial deformations are difficult to avoid in medical imaging scenarios. More importantly, current algorithms focus on visual quality and statistical metrics, thus overlooking the requirements of high-level tasks. To address these challenges, we proposes a unsupervised fusion model, termed PAMRFuse+, which integrates image generation and registration. Specifically, a cross-modal style transfer network is introduced to simplify cross-modal registration to single-modal registration. Subsequently, a multi-level registration network is employed to predict displacement vector fields. Furthermore, a dual-branch feature decomposition fusion network is proposed to address the challenges of cross-modal feature modeling and decomposition by integrating modality-specific and modality-shared features. PAMRFuse+ achieves satisfactory results in registering and fusing unaligned PAT-MRI datasets. Moreover, for the first time, we evaluate the performance of medical image fusion with contour segmentation and multi-organ instance segmentation. Extensive experimental demonstrations reveal the advantages of PAMRFuse+ in improving the performance of medical image analysis tasks.
△ Less
Submitted 4 July, 2024;
originally announced July 2024.
-
MindBench: A Comprehensive Benchmark for Mind Map Structure Recognition and Analysis
Authors:
Lei Chen,
Feng Yan,
Yujie Zhong,
Shaoxiang Chen,
Zequn Jie,
Lin Ma
Abstract:
Multimodal Large Language Models (MLLM) have made significant progress in the field of document analysis. Despite this, existing benchmarks typically focus only on extracting text and simple layout information, neglecting the complex interactions between elements in structured documents such as mind maps and flowcharts. To address this issue, we introduce the new benchmark named MindBench, which n…
▽ More
Multimodal Large Language Models (MLLM) have made significant progress in the field of document analysis. Despite this, existing benchmarks typically focus only on extracting text and simple layout information, neglecting the complex interactions between elements in structured documents such as mind maps and flowcharts. To address this issue, we introduce the new benchmark named MindBench, which not only includes meticulously constructed bilingual authentic or synthetic images, detailed annotations, evaluation metrics and baseline models, but also specifically designs five types of structured understanding and parsing tasks. These tasks include full parsing, partial parsing, position-related parsing, structured Visual Question Answering (VQA), and position-related VQA, covering key areas such as text recognition, spatial awareness, relationship discernment, and structured parsing. Extensive experimental results demonstrate the substantial potential and significant room for improvement in current models' ability to handle structured document information. We anticipate that the launch of MindBench will significantly advance research and application development in structured document analysis technology. MindBench is available at: https://miasanlei.github.io/MindBench.github.io/.
△ Less
Submitted 3 July, 2024;
originally announced July 2024.
-
Identifying Direct Bandgap Silicon Structures with High-throughput Search and Machine Learning Methods
Authors:
Rui Wang,
Hongyu Yu,
Yang Zhong,
Hongjun Xiang
Abstract:
Utilizations of silicon-based luminescent devices are restricted by the indirect-gap nature of diamond silicon. In this study, the high-throughput method is employed to expedite discoveries of direct-gap silicon crystals. The machine learning (ML) potential is utilized to construct a dataset comprising 2637 silicon allotropes, which is subsequently screened using an ML Hamiltonian model and densit…
▽ More
Utilizations of silicon-based luminescent devices are restricted by the indirect-gap nature of diamond silicon. In this study, the high-throughput method is employed to expedite discoveries of direct-gap silicon crystals. The machine learning (ML) potential is utilized to construct a dataset comprising 2637 silicon allotropes, which is subsequently screened using an ML Hamiltonian model and density functional theory calculations, resulting in identification of 47 direct-gap Si structures. We calculate transition dipole moments (TDM), energies, and phonon bandstructures of these structures to validate their performance. Additionally, we recalculate bandgaps of these structures employing the HSE06 functional. 22 silicon allotropes are identified as potential photovoltaic materials. Among them, the energy per atom of Si22-Pm, which has a direct bandgap of 1.27 eV, is 0.026 eV/atom higher than diamond silicon. Si18-C2/m, which has a direct bandgap of 0.796 eV, exhibits the highest TDM among identified structures. Si16-P21/c, which has a direct bandgap of 0.907 eV, has the mass density of 2.316 g/cm3, which is the highest among identified structures and higher than that of diamond silicon. The structure Si12-P1, which possesses a direct bandgap of 1.69 eV, exhibits the highest spectroscopic limited maximum efficiency (SLME) among identified structures at 32.28%, surpassing that of diamond silicon. This study offers insights into properties of silicon crystals while presenting a systematic high-throughput method for material discovery.
△ Less
Submitted 2 July, 2024;
originally announced July 2024.
-
FairMedFM: Fairness Benchmarking for Medical Imaging Foundation Models
Authors:
Ruinan Jin,
Zikang Xu,
Yuan Zhong,
Qiongsong Yao,
Qi Dou,
S. Kevin Zhou,
Xiaoxiao Li
Abstract:
The advent of foundation models (FMs) in healthcare offers unprecedented opportunities to enhance medical diagnostics through automated classification and segmentation tasks. However, these models also raise significant concerns about their fairness, especially when applied to diverse and underrepresented populations in healthcare applications. Currently, there is a lack of comprehensive benchmark…
▽ More
The advent of foundation models (FMs) in healthcare offers unprecedented opportunities to enhance medical diagnostics through automated classification and segmentation tasks. However, these models also raise significant concerns about their fairness, especially when applied to diverse and underrepresented populations in healthcare applications. Currently, there is a lack of comprehensive benchmarks, standardized pipelines, and easily adaptable libraries to evaluate and understand the fairness performance of FMs in medical imaging, leading to considerable challenges in formulating and implementing solutions that ensure equitable outcomes across diverse patient populations. To fill this gap, we introduce FairMedFM, a fairness benchmark for FM research in medical imaging.FairMedFM integrates with 17 popular medical imaging datasets, encompassing different modalities, dimensionalities, and sensitive attributes. It explores 20 widely used FMs, with various usages such as zero-shot learning, linear probing, parameter-efficient fine-tuning, and prompting in various downstream tasks -- classification and segmentation. Our exhaustive analysis evaluates the fairness performance over different evaluation metrics from multiple perspectives, revealing the existence of bias, varied utility-fairness trade-offs on different FMs, consistent disparities on the same datasets regardless FMs, and limited effectiveness of existing unfairness mitigation methods. Checkout FairMedFM's project page and open-sourced codebase, which supports extendible functionalities and applications as well as inclusive for studies on FMs in medical imaging over the long term.
△ Less
Submitted 3 July, 2024; v1 submitted 1 July, 2024;
originally announced July 2024.
-
Structured and Balanced Multi-component and Multi-layer Neural Networks
Authors:
Shijun Zhang,
Hongkai Zhao,
Yimin Zhong,
Haomin Zhou
Abstract:
In this work, we propose a balanced multi-component and multi-layer neural network (MMNN) structure to approximate functions with complex features with both accuracy and efficiency in terms of degrees of freedom and computation cost. The main idea is motivated by a multi-component, each of which can be approximated effectively by a single-layer network, and multi-layer decomposition in a "divide-a…
▽ More
In this work, we propose a balanced multi-component and multi-layer neural network (MMNN) structure to approximate functions with complex features with both accuracy and efficiency in terms of degrees of freedom and computation cost. The main idea is motivated by a multi-component, each of which can be approximated effectively by a single-layer network, and multi-layer decomposition in a "divide-and-conquer" type of strategy to deal with a complex function. While an easy modification to fully connected neural networks (FCNNs) or multi-layer perceptrons (MLPs) through the introduction of balanced multi-component structures in the network, MMNNs achieve a significant reduction of training parameters, a much more efficient training process, and a much improved accuracy compared to FCNNs or MLPs. Extensive numerical experiments are presented to illustrate the effectiveness of MMNNs in approximating high oscillatory functions and its automatic adaptivity in capturing localized features.
△ Less
Submitted 30 June, 2024;
originally announced July 2024.
-
UIO-LLMs: Unbiased Incremental Optimization for Long-Context LLMs
Authors:
Wenhao Li,
Mingbao Lin,
Yunshan Zhong,
Shuicheng Yan,
Rongrong Ji
Abstract:
Managing long texts is challenging for large language models (LLMs) due to limited context window sizes. This study introduces UIO-LLMs, an unbiased incremental optimization approach for memory-enhanced transformers under long-context settings. We initially conceptualize the process as a streamlined encoder-decoder framework where the weights-shared encoder and decoder respectively encapsulate a c…
▽ More
Managing long texts is challenging for large language models (LLMs) due to limited context window sizes. This study introduces UIO-LLMs, an unbiased incremental optimization approach for memory-enhanced transformers under long-context settings. We initially conceptualize the process as a streamlined encoder-decoder framework where the weights-shared encoder and decoder respectively encapsulate a context segment into memories and leverage these memories to predict outputs of the subsequent segment. Subsequently, by treating our memory-enhanced transformers as fully-connected recurrent neural networks (RNNs), we refine the training process using the Truncated Backpropagation Through Time (TBPTT) algorithm, which incorporates innovative incremental optimization techniques. These techniques not only diminish time complexity but also address the bias in gradient computation through an unbiased optimization process. UIO-LLMs successfully handle long context, such as extending the context window of Llama2-7b-chat from 4K to 100K tokens with minimal 2% additional parameters, while keeping the inference cost nearly linear as context length increases.
△ Less
Submitted 26 June, 2024;
originally announced June 2024.
-
A Cross Spatio-Temporal Pathology-based Lung Nodule Dataset
Authors:
Muwei Jian,
Haoran Zhang,
Mingju Shao,
Hongyu Chen,
Huihui Huang,
Yanjie Zhong,
Changlei Zhang,
Bin Wang,
Penghui Gao
Abstract:
Recently, intelligent analysis of lung nodules with the assistant of computer aided detection (CAD) techniques can improve the accuracy rate of lung cancer diagnosis. However, existing CAD systems and pulmonary datasets mainly focus on Computed Tomography (CT) images from one single period, while ignoring the cross spatio-temporal features associated with the progression of nodules contained in im…
▽ More
Recently, intelligent analysis of lung nodules with the assistant of computer aided detection (CAD) techniques can improve the accuracy rate of lung cancer diagnosis. However, existing CAD systems and pulmonary datasets mainly focus on Computed Tomography (CT) images from one single period, while ignoring the cross spatio-temporal features associated with the progression of nodules contained in imaging data from various captured periods of lung cancer. If the evolution patterns of nodules across various periods in the patients' CT sequences can be explored, it will play a crucial role in guiding the precise screening identification of lung cancer. Therefore, a cross spatio-temporal lung nodule dataset with pathological information for nodule identification and diagnosis is constructed, which contains 328 CT sequences and 362 annotated nodules from 109 patients. This comprehensive database is intended to drive research in the field of CAD towards more practical and robust methods, and also contribute to the further exploration of precision medicine related field. To ensure patient confidentiality, we have removed sensitive information from the dataset.
△ Less
Submitted 25 June, 2024;
originally announced June 2024.
-
Changen2: Multi-Temporal Remote Sensing Generative Change Foundation Model
Authors:
Zhuo Zheng,
Stefano Ermon,
Dongjun Kim,
Liangpei Zhang,
Yanfei Zhong
Abstract:
Our understanding of the temporal dynamics of the Earth's surface has been advanced by deep vision models, which often require lots of labeled multi-temporal images for training. However, collecting, preprocessing, and annotating multi-temporal remote sensing images at scale is non-trivial since it is expensive and knowledge-intensive. In this paper, we present change data generators based on gene…
▽ More
Our understanding of the temporal dynamics of the Earth's surface has been advanced by deep vision models, which often require lots of labeled multi-temporal images for training. However, collecting, preprocessing, and annotating multi-temporal remote sensing images at scale is non-trivial since it is expensive and knowledge-intensive. In this paper, we present change data generators based on generative models, which are cheap and automatic, alleviating these data problems. Our main idea is to simulate a stochastic change process over time. We describe the stochastic change process as a probabilistic graphical model (GPCM), which factorizes the complex simulation problem into two more tractable sub-problems, i.e., change event simulation and semantic change synthesis. To solve these two problems, we present Changen2, a GPCM with a resolution-scalable diffusion transformer which can generate time series of images and their semantic and change labels from labeled or unlabeled single-temporal images. Changen2 is a generative change foundation model that can be trained at scale via self-supervision, and can produce change supervisory signals from unlabeled single-temporal images. Unlike existing foundation models, Changen2 synthesizes change data to train task-specific foundation models for change detection. The resulting model possesses inherent zero-shot change detection capabilities and excellent transferability. Experiments suggest Changen2 has superior spatiotemporal scalability, e.g., Changen2 model trained on 256$^2$ pixel single-temporal images can yield time series of any length and resolutions of 1,024$^2$ pixels. Changen2 pre-trained models exhibit superior zero-shot performance (narrowing the performance gap to 3% on LEVIR-CD and approximately 10% on both S2Looking and SECOND, compared to fully supervised counterparts) and transferability across multiple types of change tasks.
△ Less
Submitted 25 June, 2024;
originally announced June 2024.
-
Efficient and Effective Implicit Dynamic Graph Neural Network
Authors:
Yongjian Zhong,
Hieu Vu,
Tianbao Yang,
Bijaya Adhikari
Abstract:
Implicit graph neural networks have gained popularity in recent years as they capture long-range dependencies while improving predictive performance in static graphs. Despite the tussle between performance degradation due to the oversmoothing of learned embeddings and long-range dependency being more pronounced in dynamic graphs, as features are aggregated both across neighborhood and time, no pri…
▽ More
Implicit graph neural networks have gained popularity in recent years as they capture long-range dependencies while improving predictive performance in static graphs. Despite the tussle between performance degradation due to the oversmoothing of learned embeddings and long-range dependency being more pronounced in dynamic graphs, as features are aggregated both across neighborhood and time, no prior work has proposed an implicit graph neural model in a dynamic setting. In this paper, we present Implicit Dynamic Graph Neural Network (IDGNN) a novel implicit neural network for dynamic graphs which is the first of its kind. A key characteristic of IDGNN is that it demonstrably is well-posed, i.e., it is theoretically guaranteed to have a fixed-point representation. We then demonstrate that the standard iterative algorithm often used to train implicit models is computationally expensive in our dynamic setting as it involves computing gradients, which themselves have to be estimated in an iterative manner. To overcome this, we pose an equivalent bilevel optimization problem and propose an efficient single-loop training algorithm that avoids iterative computation by maintaining moving averages of key components of the gradients. We conduct extensive experiments on real-world datasets on both classification and regression tasks to demonstrate the superiority of our approach over the state-of-the-art baselines. We also demonstrate that our bi-level optimization framework maintains the performance of the expensive iterative algorithm while obtaining up to \textbf{1600x} speed-up.
△ Less
Submitted 25 June, 2024;
originally announced June 2024.
-
Scaling Laws for Linear Complexity Language Models
Authors:
Xuyang Shen,
Dong Li,
Ruitao Leng,
Zhen Qin,
Weigao Sun,
Yiran Zhong
Abstract:
The interest in linear complexity models for large language models is on the rise, although their scaling capacity remains uncertain. In this study, we present the scaling laws for linear complexity language models to establish a foundation for their scalability. Specifically, we examine the scaling behaviors of three efficient linear architectures. These include TNL, a linear attention model with…
▽ More
The interest in linear complexity models for large language models is on the rise, although their scaling capacity remains uncertain. In this study, we present the scaling laws for linear complexity language models to establish a foundation for their scalability. Specifically, we examine the scaling behaviors of three efficient linear architectures. These include TNL, a linear attention model with data-independent decay; HGRN2, a linear RNN with data-dependent decay; and cosFormer2, a linear attention model without decay. We also include LLaMA as a baseline architecture for softmax attention for comparison. These models were trained with six variants, ranging from 70M to 7B parameters on a 300B-token corpus, and evaluated with a total of 1,376 intermediate checkpoints on various downstream tasks. These tasks include validation loss, commonsense reasoning, and information retrieval and generation. The study reveals that existing linear complexity language models exhibit similar scaling capabilities as conventional transformer-based models while also demonstrating superior linguistic proficiency and knowledge retention.
△ Less
Submitted 24 June, 2024;
originally announced June 2024.
-
Single-Temporal Supervised Learning for Universal Remote Sensing Change Detection
Authors:
Zhuo Zheng,
Yanfei Zhong,
Ailong Ma,
Liangpei Zhang
Abstract:
Bitemporal supervised learning paradigm always dominates remote sensing change detection using numerous labeled bitemporal image pairs, especially for high spatial resolution (HSR) remote sensing imagery. However, it is very expensive and labor-intensive to label change regions in large-scale bitemporal HSR remote sensing image pairs. In this paper, we propose single-temporal supervised learning (…
▽ More
Bitemporal supervised learning paradigm always dominates remote sensing change detection using numerous labeled bitemporal image pairs, especially for high spatial resolution (HSR) remote sensing imagery. However, it is very expensive and labor-intensive to label change regions in large-scale bitemporal HSR remote sensing image pairs. In this paper, we propose single-temporal supervised learning (STAR) for universal remote sensing change detection from a new perspective of exploiting changes between unpaired images as supervisory signals. STAR enables us to train a high-accuracy change detector only using unpaired labeled images and can generalize to real-world bitemporal image pairs. To demonstrate the flexibility and scalability of STAR, we design a simple yet unified change detector, termed ChangeStar2, capable of addressing binary change detection, object change detection, and semantic change detection in one architecture. ChangeStar2 achieves state-of-the-art performances on eight public remote sensing change detection datasets, covering above two supervised settings, multiple change types, multiple scenarios. The code is available at https://github.com/Z-Zheng/pytorch-change-models.
△ Less
Submitted 21 June, 2024;
originally announced June 2024.
-
Advanced Multimodal Deep Learning Architecture for Image-Text Matching
Authors:
Jinyin Wang,
Haijing Zhang,
Yihao Zhong,
Yingbin Liang,
Rongwei Ji,
Yiru Cang
Abstract:
Image-text matching is a key multimodal task that aims to model the semantic association between images and text as a matching relationship. With the advent of the multimedia information age, image, and text data show explosive growth, and how to accurately realize the efficient and accurate semantic correspondence between them has become the core issue of common concern in academia and industry.…
▽ More
Image-text matching is a key multimodal task that aims to model the semantic association between images and text as a matching relationship. With the advent of the multimedia information age, image, and text data show explosive growth, and how to accurately realize the efficient and accurate semantic correspondence between them has become the core issue of common concern in academia and industry. In this study, we delve into the limitations of current multimodal deep learning models in processing image-text pairing tasks. Therefore, we innovatively design an advanced multimodal deep learning architecture, which combines the high-level abstract representation ability of deep neural networks for visual information with the advantages of natural language processing models for text semantic understanding. By introducing a novel cross-modal attention mechanism and hierarchical feature fusion strategy, the model achieves deep fusion and two-way interaction between image and text feature space. In addition, we also optimize the training objectives and loss functions to ensure that the model can better map the potential association structure between images and text during the learning process. Experiments show that compared with existing image-text matching models, the optimized new model has significantly improved performance on a series of benchmark data sets. In addition, the new model also shows excellent generalization and robustness on large and diverse open scenario datasets and can maintain high matching performance even in the face of previously unseen complex situations.
△ Less
Submitted 13 June, 2024;
originally announced June 2024.
-
Unveiling the Impact of Multi-Modal Interactions on User Engagement: A Comprehensive Evaluation in AI-driven Conversations
Authors:
Lichao Zhang,
Jia Yu,
Shuai Zhang,
Long Li,
Yangyang Zhong,
Guanbao Liang,
Yuming Yan,
Qing Ma,
Fangsheng Weng,
Fayu Pan,
Jing Li,
Renjun Xu,
Zhenzhong Lan
Abstract:
Large Language Models (LLMs) have significantly advanced user-bot interactions, enabling more complex and coherent dialogues. However, the prevalent text-only modality might not fully exploit the potential for effective user engagement. This paper explores the impact of multi-modal interactions, which incorporate images and audio alongside text, on user engagement in chatbot conversations. We cond…
▽ More
Large Language Models (LLMs) have significantly advanced user-bot interactions, enabling more complex and coherent dialogues. However, the prevalent text-only modality might not fully exploit the potential for effective user engagement. This paper explores the impact of multi-modal interactions, which incorporate images and audio alongside text, on user engagement in chatbot conversations. We conduct a comprehensive analysis using a diverse set of chatbots and real-user interaction data, employing metrics such as retention rate and conversation length to evaluate user engagement. Our findings reveal a significant enhancement in user engagement with multi-modal interactions compared to text-only dialogues. Notably, the incorporation of a third modality significantly amplifies engagement beyond the benefits observed with just two modalities. These results suggest that multi-modal interactions optimize cognitive processing and facilitate richer information comprehension. This study underscores the importance of multi-modality in chatbot design, offering valuable insights for creating more engaging and immersive AI communication experiences and informing the broader AI community about the benefits of multi-modal interactions in enhancing user engagement.
△ Less
Submitted 21 June, 2024;
originally announced June 2024.
-
Towards Multi-modality Fusion and Prototype-based Feature Refinement for Clinically Significant Prostate Cancer Classification in Transrectal Ultrasound
Authors:
Hong Wu,
Juan Fu,
Hongsheng Ye,
Yuming Zhong,
Xuebin Zou,
Jianhua Zhou,
Yi Wang
Abstract:
Prostate cancer is a highly prevalent cancer and ranks as the second leading cause of cancer-related deaths in men globally. Recently, the utilization of multi-modality transrectal ultrasound (TRUS) has gained significant traction as a valuable technique for guiding prostate biopsies. In this study, we propose a novel learning framework for clinically significant prostate cancer (csPCa) classifica…
▽ More
Prostate cancer is a highly prevalent cancer and ranks as the second leading cause of cancer-related deaths in men globally. Recently, the utilization of multi-modality transrectal ultrasound (TRUS) has gained significant traction as a valuable technique for guiding prostate biopsies. In this study, we propose a novel learning framework for clinically significant prostate cancer (csPCa) classification using multi-modality TRUS. The proposed framework employs two separate 3D ResNet-50 to extract distinctive features from B-mode and shear wave elastography (SWE). Additionally, an attention module is incorporated to effectively refine B-mode features and aggregate the extracted features from both modalities. Furthermore, we utilize few shot segmentation task to enhance the capacity of classification encoder. Due to the limited availability of csPCa masks, a prototype correction module is employed to extract representative prototypes of csPCa. The performance of the framework is assessed on a large-scale dataset consisting of 512 TRUS videos with biopsy-proved prostate cancer. The results demonstrate the strong capability in accurately identifying csPCa, achieving an area under the curve (AUC) of 0.86. Moreover, the framework generates visual class activation mapping (CAM), which can serve as valuable assistance for localizing csPCa. These CAM images may offer valuable guidance during TRUS-guided targeted biopsies, enhancing the efficacy of the biopsy procedure.The code is available at https://github.com/2313595986/SmileCode.
△ Less
Submitted 20 June, 2024;
originally announced June 2024.
-
Synthesizing Multimodal Electronic Health Records via Predictive Diffusion Models
Authors:
Yuan Zhong,
Xiaochen Wang,
Jiaqi Wang,
Xiaokun Zhang,
Yaqing Wang,
Mengdi Huai,
Cao Xiao,
Fenglong Ma
Abstract:
Synthesizing electronic health records (EHR) data has become a preferred strategy to address data scarcity, improve data quality, and model fairness in healthcare. However, existing approaches for EHR data generation predominantly rely on state-of-the-art generative techniques like generative adversarial networks, variational autoencoders, and language models. These methods typically replicate inp…
▽ More
Synthesizing electronic health records (EHR) data has become a preferred strategy to address data scarcity, improve data quality, and model fairness in healthcare. However, existing approaches for EHR data generation predominantly rely on state-of-the-art generative techniques like generative adversarial networks, variational autoencoders, and language models. These methods typically replicate input visits, resulting in inadequate modeling of temporal dependencies between visits and overlooking the generation of time information, a crucial element in EHR data. Moreover, their ability to learn visit representations is limited due to simple linear mapping functions, thus compromising generation quality. To address these limitations, we propose a novel EHR data generation model called EHRPD. It is a diffusion-based model designed to predict the next visit based on the current one while also incorporating time interval estimation. To enhance generation quality and diversity, we introduce a novel time-aware visit embedding module and a pioneering predictive denoising diffusion probabilistic model (PDDPM). Additionally, we devise a predictive U-Net (PU-Net) to optimize P-DDPM.We conduct experiments on two public datasets and evaluate EHRPD from fidelity, privacy, and utility perspectives. The experimental results demonstrate the efficacy and utility of the proposed EHRPD in addressing the aforementioned limitations and advancing EHR data generation.
△ Less
Submitted 19 June, 2024;
originally announced June 2024.
-
Koebe uniformization for infinitely connected attracting Fatou domains
Authors:
Xiaoguang Wang,
Yi Zhong
Abstract:
This paper works on the structure of infinitely connected Fatou damains of rational maps in terms of Koebe uniformization. Due to the complicated boundary behavior, the existing uniformization results are failed to apply in general. We proved that if the rational map is geometrically finite, then its infinitely connected attracting Fatou damain is conformally homeomorphic to a circle domain.
This paper works on the structure of infinitely connected Fatou damains of rational maps in terms of Koebe uniformization. Due to the complicated boundary behavior, the existing uniformization results are failed to apply in general. We proved that if the rational map is geometrically finite, then its infinitely connected attracting Fatou damain is conformally homeomorphic to a circle domain.
△ Less
Submitted 19 June, 2024;
originally announced June 2024.
-
A universal bioluminescence tomography system for pre-clinical image-guided radiotherapy research
Authors:
Zhishen Tong,
Zijian Deng,
Xiangkun Xu,
Ciara Newman,
Xun Jia,
Yuncheng Zhong,
Merle Reinhart,
Paul Tsouchlos,
Tim Devling,
Hamid Dehghani,
Iulian Iordachita,
Debabrata Saha,
John W. Wong,
Ken Kang-Hsin Wang
Abstract:
CBCT-guided small animal irradiators encounter challenges in localizing soft-tissue targets due to low imaging contrast. Bioluminescence tomography (BLT) offers a promising solution, but they have largely remained in laboratorial development, limiting accessibility for researchers. In this work, we develop a universal, commercial-graded BLT-guided system (MuriGlo) designed to seamlessly integrate…
▽ More
CBCT-guided small animal irradiators encounter challenges in localizing soft-tissue targets due to low imaging contrast. Bioluminescence tomography (BLT) offers a promising solution, but they have largely remained in laboratorial development, limiting accessibility for researchers. In this work, we develop a universal, commercial-graded BLT-guided system (MuriGlo) designed to seamlessly integrate with commercial irradiators and empower researchers for translational studies. We demonstrate its capabilities in supporting in vitro and in vivo studies. The MuriGlo comprises detachable mouse bed, thermostatic control, mirrors, filters, and CCD, enabling multi-projection and multi-spectral imaging. We evaluate that the thermostatic control effectively sustains animal temperature at 37°C throughout imaging, and quantify that the system can detect as few as 61 GL261-AkaLuc cells in vitro. To illustrate how the MuriGlo can be utilized for in vivo image-guided research, we present 3 strategies, BLT-guided 5-arc, 2-field box, and BLI-guided single-beam, ranging from complicated high-conformal to simplest high-throughput plans. The high conformal BLT-guided 5-arc plan fully covers the gross tumor volume (GTV) at prescribed dose with minimal normal tissue exposure (3.9%), while the simplified, high-throughput BLT-guided 2-field box achieves 100% GTV coverage but results in higher normal tissue exposure (13.1%). Moreover, we demonstrate that the localization accuracy of MuriGlo for both widely-used SARRP and SmART irradiators is within1 mm, and the tumor coverage reaches over 97% with 0.75mm margin. The universal BLT-guided system offers seamless integration with commercial irradiators, achieving comparable localization accuracy, expected to supporting high-precision radiation research.
△ Less
Submitted 27 June, 2024; v1 submitted 18 June, 2024;
originally announced June 2024.
-
In-Context Editing: Learning Knowledge from Self-Induced Distributions
Authors:
Siyuan Qi,
Bangcheng Yang,
Kailin Jiang,
Xiaobo Wang,
Jiaqi Li,
Yifan Zhong,
Yaodong Yang,
Zilong Zheng
Abstract:
The existing fine-tuning paradigm for language models is brittle in knowledge editing scenarios, where the model must incorporate new information without extensive retraining. This brittleness often results in overfitting, reduced performance, and unnatural language generation. To address this, we propose Consistent In-Context Editing (ICE), a novel approach that leverages the model's in-context l…
▽ More
The existing fine-tuning paradigm for language models is brittle in knowledge editing scenarios, where the model must incorporate new information without extensive retraining. This brittleness often results in overfitting, reduced performance, and unnatural language generation. To address this, we propose Consistent In-Context Editing (ICE), a novel approach that leverages the model's in-context learning capability to tune toward a contextual distribution rather than a one-hot target. ICE introduces a straightforward optimization framework that includes both a target and a procedure, enhancing the robustness and effectiveness of gradient-based tuning methods. We provide analytical insights into ICE across four critical aspects of knowledge editing: accuracy, locality, generalization, and linguistic quality, showing its advantages. Experimental results across four datasets confirm the effectiveness of ICE and demonstrate its potential for continual editing, ensuring that updated information is incorporated while preserving the integrity of the model.
△ Less
Submitted 17 June, 2024;
originally announced June 2024.
-
GTR-Voice: Articulatory Phonetics Informed Controllable Expressive Speech Synthesis
Authors:
Zehua Kcriss Li,
Meiying Melissa Chen,
Yi Zhong,
Pinxin Liu,
Zhiyao Duan
Abstract:
Expressive speech synthesis aims to generate speech that captures a wide range of para-linguistic features, including emotion and articulation, though current research primarily emphasizes emotional aspects over the nuanced articulatory features mastered by professional voice actors. Inspired by this, we explore expressive speech synthesis through the lens of articulatory phonetics. Specifically,…
▽ More
Expressive speech synthesis aims to generate speech that captures a wide range of para-linguistic features, including emotion and articulation, though current research primarily emphasizes emotional aspects over the nuanced articulatory features mastered by professional voice actors. Inspired by this, we explore expressive speech synthesis through the lens of articulatory phonetics. Specifically, we define a framework with three dimensions: Glottalization, Tenseness, and Resonance (GTR), to guide the synthesis at the voice production level. With this framework, we record a high-quality speech dataset named GTR-Voice, featuring 20 Chinese sentences articulated by a professional voice actor across 125 distinct GTR combinations. We verify the framework and GTR annotations through automatic classification and listening tests, and demonstrate precise controllability along the GTR dimensions on two fine-tuned expressive TTS models. We open-source the dataset and TTS models.
△ Less
Submitted 15 June, 2024;
originally announced June 2024.
-
Noise-induced quantum synchronization and maximally entangled mixed states in superconducting circuits
Authors:
Ziyu Tao,
Finn Schmolke,
Chang-Kang Hu,
Wenhui Huang,
Yuxuan Zhou,
Jiawei Zhang,
Ji Chu,
Libo Zhang,
Xuandong Sun,
Zecheng Guo,
Jingjing Niu,
Wenle Weng,
Song Liu,
Youpeng Zhong,
Dian Tan,
Dapeng Yu,
Eric Lutz
Abstract:
Random fluctuations can lead to cooperative effects in complex systems. We here report the experimental observation of noise-induced quantum synchronization in a chain of superconducting transmon qubits with nearest-neighbor interactions. The application of Gaussian white noise to a single site leads to synchronous oscillations in the entire chain. We show that the two synchronized end qubits are…
▽ More
Random fluctuations can lead to cooperative effects in complex systems. We here report the experimental observation of noise-induced quantum synchronization in a chain of superconducting transmon qubits with nearest-neighbor interactions. The application of Gaussian white noise to a single site leads to synchronous oscillations in the entire chain. We show that the two synchronized end qubits are entangled, with nonzero concurrence, and that they belong to a class of generalized Bell states known as maximally entangled mixed states, whose entanglement cannot be increased by any global unitary. We further demonstrate the stability against frequency detuning of both synchronization and entanglement by determining the corresponding generalized Arnold tongue diagrams. Our results highlight the constructive influence of noise in a quantum many-body system and uncover the potential role of synchronization for mixed-state quantum information science.
△ Less
Submitted 14 June, 2024;
originally announced June 2024.
-
Transition from decaying to decayless kink oscillations of solar coronal loops
Authors:
Valery M. Nakariakov,
Yu Zhong,
Dmitrii Y. Kolotkov
Abstract:
The transition of an impulsively excited kink oscillation of a solar coronal loop to an oscillation with a stationary amplitude, i.e., the damping pattern, is determined using the low-dimensional self-oscillation model. In the model, the decayless kink oscillations are sustained by the interaction of the oscillating loop with an external quasi-steady flow. The analytical solution is based on the a…
▽ More
The transition of an impulsively excited kink oscillation of a solar coronal loop to an oscillation with a stationary amplitude, i.e., the damping pattern, is determined using the low-dimensional self-oscillation model. In the model, the decayless kink oscillations are sustained by the interaction of the oscillating loop with an external quasi-steady flow. The analytical solution is based on the assumption that the combined effect of the effective dissipation, for example, by resonant absorption, and interaction with an external flow, is weak. The effect is characterised by a dimensionless coupling parameter. The damping pattern is found to depend upon the initial amplitude and the coupling parameter. The approximate expression shows a good agreement with a numerical solution of the self-oscillation equation. The plausibility of the established damping pattern is demonstrated by an observational example. Notably, the damping pattern is not exponential, and the characteristic decay time is different from the time determined by the traditionally used exponential damping fit. Implications of this finding for seismology of the solar coronal plasmas are discussed. In particular, it is suggested that a very rapid, in less than the oscillation period, decay of the oscillation to the stationary level, achieved for larger values of the coupling parameter, can explain the relative rareness of the kink oscillation events.
△ Less
Submitted 11 June, 2024;
originally announced June 2024.
-
FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel Fusion
Authors:
Li-Wen Chang,
Wenlei Bao,
Qi Hou,
Chengquan Jiang,
Ningxin Zheng,
Yinmin Zhong,
Xuanrun Zhang,
Zuquan Song,
Ziheng Jiang,
Haibin Lin,
Xin Jin,
Xin Liu
Abstract:
Large deep learning models have demonstrated strong ability to solve many tasks across a wide range of applications. Those large models typically require training and inference to be distributed. Tensor parallelism is a common technique partitioning computation of an operation or layer across devices to overcome the memory capacity limitation of a single processor, and/or to accelerate computation…
▽ More
Large deep learning models have demonstrated strong ability to solve many tasks across a wide range of applications. Those large models typically require training and inference to be distributed. Tensor parallelism is a common technique partitioning computation of an operation or layer across devices to overcome the memory capacity limitation of a single processor, and/or to accelerate computation to meet a certain latency requirement. However, this kind of parallelism introduces additional communication that might contribute a significant portion of overall runtime. Thus limits scalability of this technique within a group of devices with high speed interconnects, such as GPUs with NVLinks in a node. This paper proposes a novel method, Flux, to significantly hide communication latencies with dependent computations for GPUs. Flux over-decomposes communication and computation operations into much finer-grained operations and further fuses them into a larger kernel to effectively hide communication without compromising kernel efficiency. Flux can potentially overlap up to 96% of communication given a fused kernel. Overall, it can achieve up to 1.24x speedups for training over Megatron-LM on a cluster of 128 GPUs with various GPU generations and interconnects, and up to 1.66x and 1.30x speedups for prefill and decoding inference over vLLM on a cluster with 8 GPUs with various GPU generations and interconnects.
△ Less
Submitted 18 June, 2024; v1 submitted 10 June, 2024;
originally announced June 2024.
-
Towards Detecting LLMs Hallucination via Markov Chain-based Multi-agent Debate Framework
Authors:
Xiaoxi Sun,
Jinpeng Li,
Yan Zhong,
Dongyan Zhao,
Rui Yan
Abstract:
The advent of large language models (LLMs) has facilitated the development of natural language text generation. It also poses unprecedented challenges, with content hallucination emerging as a significant concern. Existing solutions often involve expensive and complex interventions during the training process. Moreover, some approaches emphasize problem disassembly while neglecting the crucial val…
▽ More
The advent of large language models (LLMs) has facilitated the development of natural language text generation. It also poses unprecedented challenges, with content hallucination emerging as a significant concern. Existing solutions often involve expensive and complex interventions during the training process. Moreover, some approaches emphasize problem disassembly while neglecting the crucial validation process, leading to performance degradation or limited applications. To overcome these limitations, we propose a Markov Chain-based multi-agent debate verification framework to enhance hallucination detection accuracy in concise claims. Our method integrates the fact-checking process, including claim detection, evidence retrieval, and multi-agent verification. In the verification stage, we deploy multiple agents through flexible Markov Chain-based debates to validate individual claims, ensuring meticulous verification outcomes. Experimental results across three generative tasks demonstrate that our approach achieves significant improvements over baselines.
△ Less
Submitted 5 June, 2024;
originally announced June 2024.
-
Advancing Weakly-Supervised Audio-Visual Video Parsing via Segment-wise Pseudo Labeling
Authors:
Jinxing Zhou,
Dan Guo,
Yiran Zhong,
Meng Wang
Abstract:
The Audio-Visual Video Parsing task aims to identify and temporally localize the events that occur in either or both the audio and visual streams of audible videos. It often performs in a weakly-supervised manner, where only video event labels are provided, \ie, the modalities and the timestamps of the labels are unknown. Due to the lack of densely annotated labels, recent work attempts to leverag…
▽ More
The Audio-Visual Video Parsing task aims to identify and temporally localize the events that occur in either or both the audio and visual streams of audible videos. It often performs in a weakly-supervised manner, where only video event labels are provided, \ie, the modalities and the timestamps of the labels are unknown. Due to the lack of densely annotated labels, recent work attempts to leverage pseudo labels to enrich the supervision. A commonly used strategy is to generate pseudo labels by categorizing the known video event labels for each modality. However, the labels are still confined to the video level, and the temporal boundaries of events remain unlabeled. In this paper, we propose a new pseudo label generation strategy that can explicitly assign labels to each video segment by utilizing prior knowledge learned from the open world. Specifically, we exploit the large-scale pretrained models, namely CLIP and CLAP, to estimate the events in each video segment and generate segment-level visual and audio pseudo labels, respectively. We then propose a new loss function to exploit these pseudo labels by taking into account their category-richness and segment-richness. A label denoising strategy is also adopted to further improve the visual pseudo labels by flipping them whenever abnormally large forward losses occur. We perform extensive experiments on the LLP dataset and demonstrate the effectiveness of each proposed design and we achieve state-of-the-art video parsing performance on all types of event parsing, \ie, audio event, visual event, and audio-visual event. We also examine the proposed pseudo label generation strategy on a relevant weakly-supervised audio-visual event localization task and the experimental results again verify the benefits and generalization of our method.
△ Less
Submitted 2 June, 2024;
originally announced June 2024.
-
Optimizing Age of Information in Random Access Networks: A Second-Order Approach for Active/Passive Users
Authors:
Siqi Fan,
Yuxin Zhong,
I-Hong Hou,
Clement K Kam
Abstract:
In this paper, we study the moments of the Age of Information (AoI) for both active and passive users in a random access network. In this network, active users broadcast sensing data, while passive users detect in-band radio activities from out-of-network devices, such as jammers. Collisions occur when multiple active users transmit simultaneously. Passive users can detect radio activities only wh…
▽ More
In this paper, we study the moments of the Age of Information (AoI) for both active and passive users in a random access network. In this network, active users broadcast sensing data, while passive users detect in-band radio activities from out-of-network devices, such as jammers. Collisions occur when multiple active users transmit simultaneously. Passive users can detect radio activities only when no active user transmits. Each active user's transmission behavior follows a Markov process. We aim to minimize the weighted sum of any moments of AoI for both user types. To achieve this, we employ a second-order analysis of system behavior. Specifically, we characterize an active user's transmission Markov process using its mean and temporal variance. We show that any moment of the AoI can be approximated by a function of these two parameters. This insight enables us to analyze and optimize the transmission Markov process for active users. We apply this strategy to two different random access models. Simulation results show that policies derived from this strategy outperform other baseline policies.
△ Less
Submitted 1 June, 2024;
originally announced June 2024.
-
You Only Scan Once: Efficient Multi-dimension Sequential Modeling with LightNet
Authors:
Zhen Qin,
Yuxin Mao,
Xuyang Shen,
Dong Li,
Jing Zhang,
Yuchao Dai,
Yiran Zhong
Abstract:
Linear attention mechanisms have gained prominence in causal language models due to their linear computational complexity and enhanced speed. However, the inherent decay mechanism in linear attention presents challenges when applied to multi-dimensional sequence modeling tasks, such as image processing and multi-modal learning. In these scenarios, the utilization of sequential scanning to establis…
▽ More
Linear attention mechanisms have gained prominence in causal language models due to their linear computational complexity and enhanced speed. However, the inherent decay mechanism in linear attention presents challenges when applied to multi-dimensional sequence modeling tasks, such as image processing and multi-modal learning. In these scenarios, the utilization of sequential scanning to establish a global receptive field necessitates multiple scans for multi-dimensional data, thereby leading to inefficiencies. This paper identifies the inefficiency caused by a multiplicative linear recurrence and proposes an efficient alternative additive linear recurrence to avoid the issue, as it can handle multi-dimensional data within a single scan. We further develop an efficient multi-dimensional sequential modeling framework called LightNet based on the new recurrence. Moreover, we present two new multi-dimensional linear relative positional encoding methods, MD-TPE and MD-LRPE to enhance the model's ability to discern positional information in multi-dimensional scenarios. Our empirical evaluations across various tasks, including image classification, image generation, bidirectional language modeling, and autoregressive language modeling, demonstrate the efficacy of LightNet, showcasing its potential as a versatile and efficient solution for multi-dimensional sequential modeling.
△ Less
Submitted 31 May, 2024;
originally announced May 2024.
-
Unlocking the Secrets of Linear Complexity Sequence Model from A Unified Perspective
Authors:
Zhen Qin,
Xuyang Shen,
Dong Li,
Weigao Sun,
Stan Birchfield,
Richard Hartley,
Yiran Zhong
Abstract:
We present the Linear Complexity Sequence Model (LCSM), a comprehensive solution that unites various sequence modeling techniques with linear complexity, including linear attention, state space model, long convolution, and linear RNN, within a single framework. The goal is to enhance comprehension of these models by analyzing the impact of each component from a cohesive and streamlined viewpoint.…
▽ More
We present the Linear Complexity Sequence Model (LCSM), a comprehensive solution that unites various sequence modeling techniques with linear complexity, including linear attention, state space model, long convolution, and linear RNN, within a single framework. The goal is to enhance comprehension of these models by analyzing the impact of each component from a cohesive and streamlined viewpoint. Specifically, we segment the modeling processes of these models into three distinct stages: Expand, Oscillation, and Shrink (EOS), with each model having its own specific settings. The Expand stage involves projecting the input signal onto a high-dimensional memory state. This is followed by recursive operations performed on the memory state in the Oscillation stage. Finally, the memory state is projected back to a low-dimensional space in the Shrink stage. We perform comprehensive experiments to analyze the impact of different stage settings on language modeling and retrieval tasks. Our results show that data-driven methods are crucial for the effectiveness of the three stages in language modeling, whereas hand-crafted methods yield better performance in retrieval tasks.
△ Less
Submitted 27 May, 2024;
originally announced May 2024.
-
Various Lengths, Constant Speed: Efficient Language Modeling with Lightning Attention
Authors:
Zhen Qin,
Weigao Sun,
Dong Li,
Xuyang Shen,
Weixuan Sun,
Yiran Zhong
Abstract:
We present Lightning Attention, the first linear attention implementation that maintains a constant training speed for various sequence lengths under fixed memory consumption. Due to the issue with cumulative summation operations (cumsum), previous linear attention implementations cannot achieve their theoretical advantage in a casual setting. However, this issue can be effectively solved by utili…
▽ More
We present Lightning Attention, the first linear attention implementation that maintains a constant training speed for various sequence lengths under fixed memory consumption. Due to the issue with cumulative summation operations (cumsum), previous linear attention implementations cannot achieve their theoretical advantage in a casual setting. However, this issue can be effectively solved by utilizing different attention calculation strategies to compute the different parts of attention. Specifically, we split the attention calculation into intra-blocks and inter-blocks and use conventional attention computation for intra-blocks and linear attention kernel tricks for inter-blocks. This eliminates the need for cumsum in the linear attention calculation. Furthermore, a tiling technique is adopted through both forward and backward procedures to take full advantage of the GPU hardware. To enhance accuracy while preserving efficacy, we introduce TransNormerLLM (TNL), a new architecture that is tailored to our lightning attention. We conduct rigorous testing on standard and self-collected datasets with varying model sizes and sequence lengths. TNL is notably more efficient than other language models. In addition, benchmark results indicate that TNL performs on par with state-of-the-art LLMs utilizing conventional transformer structures. The source code is released at github.com/OpenNLPLab/TransnormerLLM.
△ Less
Submitted 20 June, 2024; v1 submitted 27 May, 2024;
originally announced May 2024.
-
SATSense: Multi-Satellite Collaborative Framework for Spectrum Sensing
Authors:
Haoxuan Yuan,
Zhe Chen,
Zheng Lin,
Jinbo Peng,
Zihan Fang,
Yuhang Zhong,
Zihang Song,
Yue Gao
Abstract:
Low Earth Orbit satellite Internet has recently been deployed, providing worldwide service with non-terrestrial networks. With the large-scale deployment of both non-terrestrial and terrestrial networks, limited spectrum resources will not be allocated enough. Consequently, dynamic spectrum sharing is crucial for their coexistence in the same spectrum, where accurate spectrum sensing is essential.…
▽ More
Low Earth Orbit satellite Internet has recently been deployed, providing worldwide service with non-terrestrial networks. With the large-scale deployment of both non-terrestrial and terrestrial networks, limited spectrum resources will not be allocated enough. Consequently, dynamic spectrum sharing is crucial for their coexistence in the same spectrum, where accurate spectrum sensing is essential. However, spectrum sensing in space is more challenging than in terrestrial networks due to variable channel conditions, making single-satellite sensing unstable. Therefore, we first attempt to design a collaborative sensing scheme utilizing diverse data from multiple satellites. However, it is non-trivial to achieve this collaboration due to heterogeneous channel quality, considerable raw sampling data, and packet loss. To address the above challenges, we first establish connections between the satellites by modeling their sensing data as a graph and devising a graph neural network-based algorithm to achieve effective spectrum sensing. Meanwhile, we establish a joint sub-Nyquist sampling and autoencoder data compression framework to reduce the amount of transmitted sensing data. Finally, we propose a contrastive learning-based mechanism compensates for missing packets. Extensive experiments demonstrate that our proposed strategy can achieve efficient spectrum sensing performance and outperform the conventional deep learning algorithm in spectrum sensing accuracy.
△ Less
Submitted 24 May, 2024;
originally announced May 2024.
-
PoseCrafter: One-Shot Personalized Video Synthesis Following Flexible Pose Control
Authors:
Yong Zhong,
Min Zhao,
Zebin You,
Xiaofeng Yu,
Changwang Zhang,
Chongxuan Li
Abstract:
In this paper, we introduce PoseCrafter, a one-shot method for personalized video generation following the control of flexible poses. Built upon Stable Diffusion and ControlNet, we carefully design an inference process to produce high-quality videos without the corresponding ground-truth frames. First, we select an appropriate reference frame from the training video and invert it to initialize all…
▽ More
In this paper, we introduce PoseCrafter, a one-shot method for personalized video generation following the control of flexible poses. Built upon Stable Diffusion and ControlNet, we carefully design an inference process to produce high-quality videos without the corresponding ground-truth frames. First, we select an appropriate reference frame from the training video and invert it to initialize all latent variables for generation. Then, we insert the corresponding training pose into the target pose sequences to enhance faithfulness through a trained temporal attention module. Furthermore, to alleviate the face and hand degradation resulting from discrepancies between poses of training videos and inference poses, we implement simple latent editing through an affine transformation matrix involving facial and hand landmarks. Extensive experiments on several datasets demonstrate that PoseCrafter achieves superior results to baselines pre-trained on a vast collection of videos under 8 commonly used metrics. Besides, PoseCrafter can follow poses from different individuals or artificial edits and simultaneously retain the human identity in an open-domain training video. Our project page is available at https://ml-gsai.github.io/PoseCrafter-demo/.
△ Less
Submitted 18 July, 2024; v1 submitted 23 May, 2024;
originally announced May 2024.
-
DeTox: Toxic Subspace Projection for Model Editing
Authors:
Rheeya Uppaal,
Apratim Dey,
Yiting He,
Yiqiao Zhong,
Junjie Hu
Abstract:
Recent alignment algorithms such as direct preference optimization (DPO) have been developed to improve the safety of large language models (LLMs) by training these models to match human behaviors exemplified by preference data. However, these methods are both computationally intensive and lacking in controllability and transparency, making them prone to jailbreaking and inhibiting their widesprea…
▽ More
Recent alignment algorithms such as direct preference optimization (DPO) have been developed to improve the safety of large language models (LLMs) by training these models to match human behaviors exemplified by preference data. However, these methods are both computationally intensive and lacking in controllability and transparency, making them prone to jailbreaking and inhibiting their widespread use. Furthermore, these tuning-based methods require large-scale preference data for training and are susceptible to noisy preference data. In this paper, we introduce a tuning-free alignment alternative (DeTox) and demonstrate its effectiveness under the use case of toxicity reduction. Grounded on theory from factor analysis, DeTox is a sample-efficient model editing approach that identifies a toxic subspace in the model parameter space and reduces model toxicity by projecting away the detected subspace. The toxic sub-space is identified by extracting preference data embeddings from the language model, and removing non-toxic information from these embeddings. We show that DeTox is more sample-efficient than DPO, further showcasing greater robustness to noisy data. Finally, we establish both theoretical and empirical connections between DeTox and DPO, showing that DeTox can be interpreted as a denoised version of a single DPO step.
△ Less
Submitted 28 May, 2024; v1 submitted 22 May, 2024;
originally announced May 2024.
-
Unlock the Power of Algorithm Features: A Generalization Analysis for Algorithm Selection
Authors:
Xingyu Wu,
Yan Zhong,
Jibin Wu,
Yuxiao Huang,
Sheng-hao Wu,
Kay Chen Tan
Abstract:
In the algorithm selection research, the discussion surrounding algorithm features has been significantly overshadowed by the emphasis on problem features. Although a few empirical studies have yielded evidence regarding the effectiveness of algorithm features, the potential benefits of incorporating algorithm features into algorithm selection models and their suitability for different scenarios r…
▽ More
In the algorithm selection research, the discussion surrounding algorithm features has been significantly overshadowed by the emphasis on problem features. Although a few empirical studies have yielded evidence regarding the effectiveness of algorithm features, the potential benefits of incorporating algorithm features into algorithm selection models and their suitability for different scenarios remain unclear. In this paper, we address this gap by proposing the first provable guarantee for algorithm selection based on algorithm features, taking a generalization perspective. We analyze the benefits and costs associated with algorithm features and investigate how the generalization error is affected by different factors. Specifically, we examine adaptive and predefined algorithm features under transductive and inductive learning paradigms, respectively, and derive upper bounds for the generalization error based on their model's Rademacher complexity. Our theoretical findings not only provide tight upper bounds, but also offer analytical insights into the impact of various factors, such as the training scale of problem instances and candidate algorithms, model parameters, feature values, and distributional differences between the training and test data. Notably, we demonstrate how models will benefit from algorithm features in complex scenarios involving many algorithms, and proves the positive correlation between generalization error bound and $χ^2$-divergence of distributions.
△ Less
Submitted 3 June, 2024; v1 submitted 18 May, 2024;
originally announced May 2024.
-
A Universal Growth Rate for Learning with Smooth Surrogate Losses
Authors:
Anqi Mao,
Mehryar Mohri,
Yutao Zhong
Abstract:
This paper presents a comprehensive analysis of the growth rate of $H$-consistency bounds (and excess error bounds) for various surrogate losses used in classification. We prove a square-root growth rate near zero for smooth margin-based surrogate losses in binary classification, providing both upper and lower bounds under mild assumptions. This result also translates to excess error bounds. Our l…
▽ More
This paper presents a comprehensive analysis of the growth rate of $H$-consistency bounds (and excess error bounds) for various surrogate losses used in classification. We prove a square-root growth rate near zero for smooth margin-based surrogate losses in binary classification, providing both upper and lower bounds under mild assumptions. This result also translates to excess error bounds. Our lower bound requires weaker conditions than those in previous work for excess error bounds, and our upper bound is entirely novel. Moreover, we extend this analysis to multi-class classification with a series of novel results, demonstrating a universal square-root growth rate for smooth comp-sum and constrained losses, covering common choices for training neural networks in multi-class classification. Given this universal rate, we turn to the question of choosing among different surrogate losses. We first examine how $H$-consistency bounds vary across surrogates based on the number of classes. Next, ignoring constants and focusing on behavior near zero, we identify minimizability gaps as the key differentiating factor in these bounds. Thus, we thoroughly analyze these gaps, to guide surrogate loss selection, covering: comparisons across different comp-sum losses, conditions where gaps become zero, and general conditions leading to small gaps. Additionally, we demonstrate the key role of minimizability gaps in comparing excess error bounds and $H$-consistency bounds.
△ Less
Submitted 8 July, 2024; v1 submitted 9 May, 2024;
originally announced May 2024.
-
Research on Image Recognition Technology Based on Multimodal Deep Learning
Authors:
Jinyin Wang,
Xingchen Li,
Yixuan Jin,
Yihao Zhong,
Keke Zhang,
Chang Zhou
Abstract:
This project investigates the human multi-modal behavior identification algorithm utilizing deep neural networks. According to the characteristics of different modal information, different deep neural networks are used to adapt to different modal video information. Through the integration of various deep neural networks, the algorithm successfully identifies behaviors across multiple modalities. I…
▽ More
This project investigates the human multi-modal behavior identification algorithm utilizing deep neural networks. According to the characteristics of different modal information, different deep neural networks are used to adapt to different modal video information. Through the integration of various deep neural networks, the algorithm successfully identifies behaviors across multiple modalities. In this project, multiple cameras developed by Microsoft Kinect were used to collect corresponding bone point data based on acquiring conventional images. In this way, the motion features in the image can be extracted. Ultimately, the behavioral characteristics discerned through both approaches are synthesized to facilitate the precise identification and categorization of behaviors. The performance of the suggested algorithm was evaluated using the MSR3D data set. The findings from these experiments indicate that the accuracy in recognizing behaviors remains consistently high, suggesting that the algorithm is reliable in various scenarios. Additionally, the tests demonstrate that the algorithm substantially enhances the accuracy of detecting pedestrian behaviors in video footage.
△ Less
Submitted 5 May, 2024;
originally announced May 2024.
-
Matten: Video Generation with Mamba-Attention
Authors:
Yu Gao,
Jiancheng Huang,
Xiaopeng Sun,
Zequn Jie,
Yujie Zhong,
Lin Ma
Abstract:
In this paper, we introduce Matten, a cutting-edge latent diffusion model with Mamba-Attention architecture for video generation. With minimal computational cost, Matten employs spatial-temporal attention for local video content modeling and bidirectional Mamba for global video content modeling. Our comprehensive experimental evaluation demonstrates that Matten has competitive performance with the…
▽ More
In this paper, we introduce Matten, a cutting-edge latent diffusion model with Mamba-Attention architecture for video generation. With minimal computational cost, Matten employs spatial-temporal attention for local video content modeling and bidirectional Mamba for global video content modeling. Our comprehensive experimental evaluation demonstrates that Matten has competitive performance with the current Transformer-based and GAN-based models in benchmark performance, achieving superior FVD scores and efficiency. Additionally, we observe a direct positive correlation between the complexity of our designed model and the improvement in video quality, indicating the excellent scalability of Matten.
△ Less
Submitted 10 May, 2024; v1 submitted 5 May, 2024;
originally announced May 2024.
-
AFDM Channel Estimation in Multi-Scale Multi-Lag Channels
Authors:
Rongyou Cao,
Yuheng Zhong,
Jiangbin Lyu,
Deqing Wang,
Liqun Fu
Abstract:
Affine Frequency Division Multiplexing (AFDM) is a brand new chirp-based multi-carrier (MC) waveform for high mobility communications, with promising advantages over Orthogonal Frequency Division Multiplexing (OFDM) and other MC waveforms. Existing AFDM research focuses on wireless communication at high carrier frequency (CF), which typically considers only Doppler frequency shift (DFS) as a resul…
▽ More
Affine Frequency Division Multiplexing (AFDM) is a brand new chirp-based multi-carrier (MC) waveform for high mobility communications, with promising advantages over Orthogonal Frequency Division Multiplexing (OFDM) and other MC waveforms. Existing AFDM research focuses on wireless communication at high carrier frequency (CF), which typically considers only Doppler frequency shift (DFS) as a result of mobility, while ignoring the accompanied Doppler time scaling (DTS) on waveform. However, for underwater acoustic (UWA) communication at much lower CF and propagating at speed of sound, the DTS effect could not be ignored and poses significant challenges for channel estimation. This paper analyzes the channel frequency response (CFR) of AFDM under multi-scale multi-lag (MSML) channels, where each propagating path could have different delay and DFS/DTS. Based on the newly derived input-output formula and its characteristics, two new channel estimation methods are proposed, i.e., AFDM with iterative multi-index (AFDM-IMI) estimation under low to moderate DTS, and AFDM with orthogonal matching pursuit (AFDM-OMP) estimation under high DTS. Numerical results confirm the effectiveness of the proposed methods against the original AFDM channel estimation method. Moreover, the resulted AFDM system outperforms OFDM as well as Orthogonal Chirp Division Multiplexing (OCDM) in terms of channel estimation accuracy and bit error rate (BER), which is consistent with our theoretical analysis based on CFR overlap probability (COP), mutual incoherent property (MIP) and channel diversity gain under MSML channels.
△ Less
Submitted 4 May, 2024;
originally announced May 2024.
-
Privacy-Enhanced Database Synthesis for Benchmark Publishing
Authors:
Yongrui Zhong,
Yunqing Ge,
Jianbin Qin,
Shuyuan Zheng,
Bo Tang,
Yu-Xuan Qiu,
Rui Mao,
Ye Yuan,
Makoto Onizuka,
Chuan Xiao
Abstract:
Benchmarking is crucial for evaluating a DBMS, yet existing benchmarks often fail to reflect the varied nature of user workloads. As a result, there is increasing momentum toward creating databases that incorporate real-world user data to more accurately mirror business environments. However, privacy concerns deter users from directly sharing their data, underscoring the importance of creating syn…
▽ More
Benchmarking is crucial for evaluating a DBMS, yet existing benchmarks often fail to reflect the varied nature of user workloads. As a result, there is increasing momentum toward creating databases that incorporate real-world user data to more accurately mirror business environments. However, privacy concerns deter users from directly sharing their data, underscoring the importance of creating synthesized databases for benchmarking that also prioritize privacy protection. Differential privacy has become a key method for safeguarding privacy when sharing data, but the focus has largely been on minimizing errors in aggregate queries or classification tasks, with less attention given to benchmarking factors like runtime performance. This paper delves into the creation of privacy-preserving databases specifically for benchmarking, aiming to produce a differentially private database whose query performance closely resembles that of the original data. Introducing PrivBench, an innovative synthesis framework, we support the generation of high-quality data that maintains privacy. PrivBench uses sum-product networks (SPNs) to partition and sample data, enhancing data representation while securing privacy. The framework allows users to adjust the detail of SPN partitions and privacy settings, crucial for customizing privacy levels. We validate our approach, which uses the Laplace and exponential mechanisms, in maintaining privacy. Our tests show that PrivBench effectively generates data that maintains privacy and excels in query performance, consistently reducing errors in query execution time, query cardinality, and KL divergence.
△ Less
Submitted 2 May, 2024;
originally announced May 2024.