-
Crowd-SAM: SAM as a Smart Annotator for Object Detection in Crowded Scenes
Authors:
Zhi Cai,
Yingjie Gao,
Yaoyan Zheng,
Nan Zhou,
Di Huang
Abstract:
In computer vision, object detection is an important task that finds its application in many scenarios. However, obtaining extensive labels can be challenging, especially in crowded scenes. Recently, the Segment Anything Model (SAM) has been proposed as a powerful zero-shot segmenter, offering a novel approach to instance segmentation tasks. However, the accuracy and efficiency of SAM and its vari…
▽ More
In computer vision, object detection is an important task that finds its application in many scenarios. However, obtaining extensive labels can be challenging, especially in crowded scenes. Recently, the Segment Anything Model (SAM) has been proposed as a powerful zero-shot segmenter, offering a novel approach to instance segmentation tasks. However, the accuracy and efficiency of SAM and its variants are often compromised when handling objects in crowded and occluded scenes. In this paper, we introduce Crowd-SAM, a SAM-based framework designed to enhance SAM's performance in crowded and occluded scenes with the cost of few learnable parameters and minimal labeled images. We introduce an efficient prompt sampler (EPS) and a part-whole discrimination network (PWD-Net), enhancing mask selection and accuracy in crowded scenes. Despite its simplicity, Crowd-SAM rivals state-of-the-art (SOTA) fully-supervised object detection methods on several benchmarks including CrowdHuman and CityPersons. Our code is available at https://github.com/FelixCaae/CrowdSAM.
△ Less
Submitted 16 July, 2024;
originally announced July 2024.
-
First Measurement of Solar $^8$B Neutrino Flux through Coherent Elastic Neutrino-Nucleus Scattering in PandaX-4T
Authors:
PandaX Collaboration,
Zihao Bo,
Wei Chen,
Xun Chen,
Yunhua Chen,
Zhaokan Cheng,
Xiangyi Cui,
Yingjie Fan,
Deqing Fang,
Zhixing Gao,
Lisheng Geng,
Karl Giboni,
Xunan Guo,
Xuyuan Guo,
Zichao Guo,
Chencheng Han,
Ke Han,
Changda He,
Jinrong He,
Di Huang,
Houqi Huang,
Junting Huang,
Ruquan Hou,
Yu Hou,
Xiangdong Ji
, et al. (77 additional authors not shown)
Abstract:
The PandaX-4T liquid xenon detector at the China Jinping Underground Laboratory is used to measure the solar $^8$B neutrino flux by detecting neutrinos through coherent scattering with xenon nuclei. Data samples requiring the coincidence of scintillation and ionization signals (paired), as well as unpaired ionization-only signals (US2), are selected with energy threshold of approximately 1.1 keV (…
▽ More
The PandaX-4T liquid xenon detector at the China Jinping Underground Laboratory is used to measure the solar $^8$B neutrino flux by detecting neutrinos through coherent scattering with xenon nuclei. Data samples requiring the coincidence of scintillation and ionization signals (paired), as well as unpaired ionization-only signals (US2), are selected with energy threshold of approximately 1.1 keV (0.33 keV) nuclear recoil energy. Combining the commissioning run and the first science run of PandaX-4T, a total exposure of 1.25 and 1.04 tonne$\cdot$year are collected for the paired and US2, respectively. After unblinding, 3 and 332 events are observed with an expectation of 2.8$\pm$0.5 and 251$\pm$32 background events, for the paired and US2 data, respectively. A combined analysis yields a best-fit $^8$B neutrino signal of 3.5 (75) events from the paired (US2) data sample, with $\sim$37\% uncertainty, and the background-only hypothesis is disfavored at 2.64$σ$ significance. This gives a solar $^8$B neutrino flux of ($8.4\pm3.1$)$\times$10$^6$ cm$^{-2}$s$^{-1}$, consistent with the standard solar model prediction. This is the first indication of solar $^8$B neutrino ``fog'' in a dark matter direct detection experiment.
△ Less
Submitted 15 July, 2024;
originally announced July 2024.
-
Pathology-knowledge Enhanced Multi-instance Prompt Learning for Few-shot Whole Slide Image Classification
Authors:
Linhao Qu,
Dingkang Yang,
Dan Huang,
Qinhao Guo,
Rongkui Luo,
Shaoting Zhang,
Xiaosong Wang
Abstract:
Current multi-instance learning algorithms for pathology image analysis often require a substantial number of Whole Slide Images for effective training but exhibit suboptimal performance in scenarios with limited learning data. In clinical settings, restricted access to pathology slides is inevitable due to patient privacy concerns and the prevalence of rare or emerging diseases. The emergence of…
▽ More
Current multi-instance learning algorithms for pathology image analysis often require a substantial number of Whole Slide Images for effective training but exhibit suboptimal performance in scenarios with limited learning data. In clinical settings, restricted access to pathology slides is inevitable due to patient privacy concerns and the prevalence of rare or emerging diseases. The emergence of the Few-shot Weakly Supervised WSI Classification accommodates the significant challenge of the limited slide data and sparse slide-level labels for diagnosis. Prompt learning based on the pre-trained models (\eg, CLIP) appears to be a promising scheme for this setting; however, current research in this area is limited, and existing algorithms often focus solely on patch-level prompts or confine themselves to language prompts. This paper proposes a multi-instance prompt learning framework enhanced with pathology knowledge, \ie, integrating visual and textual prior knowledge into prompts at both patch and slide levels. The training process employs a combination of static and learnable prompts, effectively guiding the activation of pre-trained models and further facilitating the diagnosis of key pathology patterns. Lightweight Messenger (self-attention) and Summary (attention-pooling) layers are introduced to model relationships between patches and slides within the same patient data. Additionally, alignment-wise contrastive losses ensure the feature-level alignment between visual and textual learnable prompts for both patches and slides. Our method demonstrates superior performance in three challenging clinical tasks, significantly outperforming comparative few-shot methods.
△ Less
Submitted 15 July, 2024;
originally announced July 2024.
-
Testing the Molecular Cloud Paradigm for Ultra-High-Energy Gamma Ray Emission from the Direction of SNR G106.3+2.7
Authors:
R. Alfaro,
C. Alvarez,
J. C. Arteaga-Velázquez,
D. Avila Rojas,
H. A. Ayala Solares,
R. Babu,
E. Belmont-Moreno,
A. Bernal,
K. S. Caballero-Mora,
T. Capistrán,
A. Carramiñana,
S. Casanova,
U. Cotti,
J. Cotzomi,
S. Coutiño de León,
E. De la Fuente,
C. de León,
D. Depaoli,
P. Desiati,
N. Di Lalla,
R. Diaz Hernandez,
B. L. Dingus,
M. A. DuVernois,
K. Engel,
T. Ergin
, et al. (65 additional authors not shown)
Abstract:
Supernova remnants (SNRs) are believed to be capable of accelerating cosmic rays (CRs) to PeV energies. SNR G106.3+2.7 is a prime PeVatron candidate. It is formed by a head region, where the pulsar J2229+6114 and its boomerang-shaped pulsar wind nebula are located, and a tail region containing SN ejecta. The lack of observed gamma ray emission from the two regions of this SNR has made it difficult…
▽ More
Supernova remnants (SNRs) are believed to be capable of accelerating cosmic rays (CRs) to PeV energies. SNR G106.3+2.7 is a prime PeVatron candidate. It is formed by a head region, where the pulsar J2229+6114 and its boomerang-shaped pulsar wind nebula are located, and a tail region containing SN ejecta. The lack of observed gamma ray emission from the two regions of this SNR has made it difficult to assess which region would be responsible for the PeV CRs. We aim to characterize the very-high-energy (VHE, 0.1-100 TeV) gamma ray emission from SNR G106.3+2.7 by determining the morphology and spectral energy distribution of the region. This is accomplished using 2565 days of data and improved reconstruction algorithms from the HAWC Observatory. We also explore possible gamma ray production mechanisms for different energy ranges. Using a multi-source fitting procedure based on a maximum-likelihood estimation method, we evaluate the complex nature of this region. We determine the morphology, spectrum, and energy range for the source found in the region. Molecular cloud information is also used to create a template and evaluate the HAWC gamma ray spectral properties at ultra-high-energies (UHE, >56 TeV). This will help probe the hadronic nature of the highest-energy emission from the region. We resolve one extended source coincident with all other gamma ray observations of the region. The emission reaches above 100~TeV and its preferred log-parabola shape in the spectrum shows a flux peak in the TeV range. The molecular cloud template fit on the higher energy data reveals that the SNR's energy budget is fully capable of producing a purely hadronic source for UHE gamma rays.
△ Less
Submitted 15 July, 2024;
originally announced July 2024.
-
CodeV: Empowering LLMs for Verilog Generation through Multi-Level Summarization
Authors:
Yang Zhao,
Di Huang,
Chongxiao Li,
Pengwei Jin,
Ziyuan Nan,
Tianyun Ma,
Lei Qi,
Yansong Pan,
Zhenxing Zhang,
Rui Zhang,
Xishan Zhang,
Zidong Du,
Qi Guo,
Xing Hu,
Yunji Chen
Abstract:
The increasing complexity and high costs associated with modern processor design have led to a surge in demand for processor design automation. Instruction-tuned large language models (LLMs) have demonstrated remarkable performance in automatically generating code for general-purpose programming languages like Python. However, these methods fail on hardware description languages (HDLs) like Verilo…
▽ More
The increasing complexity and high costs associated with modern processor design have led to a surge in demand for processor design automation. Instruction-tuned large language models (LLMs) have demonstrated remarkable performance in automatically generating code for general-purpose programming languages like Python. However, these methods fail on hardware description languages (HDLs) like Verilog due to the scarcity of high-quality instruction tuning data, as even advanced LLMs like GPT-3.5 exhibit limited performance on Verilog generation. Regarding this issue, we observe that (1) Verilog code collected from the real world has higher quality than those generated by LLMs. (2) LLMs like GPT-3.5 excel in summarizing Verilog code rather than generating it. Based on these observations, this paper introduces CodeV, a series of open-source instruction-tuned Verilog generation LLMs. Instead of generating descriptions first and then getting the corresponding code from advanced LLMs, we prompt the LLM with Verilog code and let the LLM generate the corresponding natural language description by multi-level summarization. Experimental results show that CodeV relatively surpasses the previous open-source SOTA by 14.4% (BetterV in VerilogEval) and 11.3% (RTLCoder in RTLLM) respectively, and also relatively outperforms previous commercial SOTA GPT-4 by 22.1% in VerilogEval.
△ Less
Submitted 15 July, 2024; v1 submitted 14 July, 2024;
originally announced July 2024.
-
TeV Analysis of a Source Rich Region with HAWC Observatory: Is HESS J1809-193 a Potential Hadronic PeVatron?
Authors:
A. Albert,
R. Alfaro,
C. Alvarez,
J. C. Arteaga-Velázquez,
D. Avila Rojas,
R. Babu,
E. Belmont-Moreno,
A. Bernal,
M. Breuhaus,
K. S. Caballero-Mora,
T. Capistrán,
A. Carramiñana,
S. Casanova,
J. Cotzomi,
E. De la Fuente,
D. Depaoli,
N. Di Lalla,
R. Diaz Hernandez,
B. L. Dingus,
M. A. DuVernois,
C. Espinoza,
K. L. Fan,
K. Fang,
B. Fick,
N. Fraija
, et al. (57 additional authors not shown)
Abstract:
HESS J1809-193 is an unidentified TeV source, first detected by the High Energy Stereoscopic System (H.E.S.S.) Collaboration. The emission originates in a source-rich region that includes several Supernova Remnants (SNR) and Pulsars (PSR) including SNR G11.1+0.1, SNR G11.0-0.0, and the young radio pulsar J1809-1917. Originally classified as a pulsar wind nebula (PWN) candidate, recent studies show…
▽ More
HESS J1809-193 is an unidentified TeV source, first detected by the High Energy Stereoscopic System (H.E.S.S.) Collaboration. The emission originates in a source-rich region that includes several Supernova Remnants (SNR) and Pulsars (PSR) including SNR G11.1+0.1, SNR G11.0-0.0, and the young radio pulsar J1809-1917. Originally classified as a pulsar wind nebula (PWN) candidate, recent studies show the peak of the TeV region overlapping with a system of molecular clouds. This resulted in the revision of the original leptonic scenario to look for alternate hadronic scenarios. Marked as a potential PeVatron candidate, this region has been studied extensively by H.E.S.S. due to its emission extending up-to several tens of TeV. In this work, we use 2398 days of data from the High Altitude Water Cherenkov (HAWC) observatory to carry out a systematic source search for the HESS J1809-193 region. We were able to resolve emission detected as an extended component (modelled as a Symmetric Gaussian with a 1 $σ$ radius of 0.21 $^\circ$) with no clear cutoff at high energies and emitting photons up-to 210 TeV. We model the multi-wavelength observations for the region HESS J1809-193 using a time-dependent leptonic model and a lepto-hadronic model. Our model indicates that both scenarios could explain the observed data within the region of HESS J1809-193.
△ Less
Submitted 11 July, 2024;
originally announced July 2024.
-
PredBench: Benchmarking Spatio-Temporal Prediction across Diverse Disciplines
Authors:
ZiDong Wang,
Zeyu Lu,
Di Huang,
Tong He,
Xihui Liu,
Wanli Ouyang,
Lei Bai
Abstract:
In this paper, we introduce PredBench, a benchmark tailored for the holistic evaluation of spatio-temporal prediction networks. Despite significant progress in this field, there remains a lack of a standardized framework for a detailed and comparative analysis of various prediction network architectures. PredBench addresses this gap by conducting large-scale experiments, upholding standardized and…
▽ More
In this paper, we introduce PredBench, a benchmark tailored for the holistic evaluation of spatio-temporal prediction networks. Despite significant progress in this field, there remains a lack of a standardized framework for a detailed and comparative analysis of various prediction network architectures. PredBench addresses this gap by conducting large-scale experiments, upholding standardized and appropriate experimental settings, and implementing multi-dimensional evaluations. This benchmark integrates 12 widely adopted methods with 15 diverse datasets across multiple application domains, offering extensive evaluation of contemporary spatio-temporal prediction networks. Through meticulous calibration of prediction settings across various applications, PredBench ensures evaluations relevant to their intended use and enables fair comparisons. Moreover, its multi-dimensional evaluation framework broadens the analysis with a comprehensive set of metrics, providing deep insights into the capabilities of models. The findings from our research offer strategic directions for future developments in the field. Our codebase is available at https://github.com/OpenEarthLab/PredBench.
△ Less
Submitted 11 July, 2024; v1 submitted 11 July, 2024;
originally announced July 2024.
-
InverseCoder: Unleashing the Power of Instruction-Tuned Code LLMs with Inverse-Instruct
Authors:
Yutong Wu,
Di Huang,
Wenxuan Shi,
Wei Wang,
Lingzhe Gao,
Shihao Liu,
Ziyuan Nan,
Kaizhao Yuan,
Rui Zhang,
Xishan Zhang,
Zidong Du,
Qi Guo,
Yewen Pu,
Dawei Yin,
Xing Hu,
Yunji Chen
Abstract:
Recent advancements in open-source code large language models (LLMs) have demonstrated remarkable coding abilities by fine-tuning on the data generated from powerful closed-source LLMs such as GPT-3.5 and GPT-4 for instruction tuning. This paper explores how to further improve an instruction-tuned code LLM by generating data from itself rather than querying closed-source LLMs. Our key observation…
▽ More
Recent advancements in open-source code large language models (LLMs) have demonstrated remarkable coding abilities by fine-tuning on the data generated from powerful closed-source LLMs such as GPT-3.5 and GPT-4 for instruction tuning. This paper explores how to further improve an instruction-tuned code LLM by generating data from itself rather than querying closed-source LLMs. Our key observation is the misalignment between the translation of formal and informal languages: translating formal language (i.e., code) to informal language (i.e., natural language) is more straightforward than the reverse. Based on this observation, we propose INVERSE-INSTRUCT, which summarizes instructions from code snippets instead of the reverse. Specifically, given an instruction tuning corpus for code and the resulting instruction-tuned code LLM, we ask the code LLM to generate additional high-quality instructions for the original corpus through code summarization and self-evaluation. Then, we fine-tune the base LLM on the combination of the original corpus and the self-generated one, which yields a stronger instruction-tuned LLM. We present a series of code LLMs named InverseCoder, which surpasses the performance of the original code LLMs on a wide range of benchmarks, including Python text-to-code generation, multilingual coding, and data-science code generation.
△ Less
Submitted 8 July, 2024;
originally announced July 2024.
-
BFLN: A Blockchain-based Federated Learning Model for Non-IID Data
Authors:
Yang Li,
Chunhe Xia,
Dongchi Huang,
Xiaojian Li,
Tianbo Wang
Abstract:
As the application of federated learning becomes increasingly widespread, the issue of imbalanced training data distribution has emerged as a significant challenge. Federated learning utilizes local data stored on different training clients for model training, rather than centralizing data on a server, thereby greatly enhancing the privacy and security of training data. However, the distribution o…
▽ More
As the application of federated learning becomes increasingly widespread, the issue of imbalanced training data distribution has emerged as a significant challenge. Federated learning utilizes local data stored on different training clients for model training, rather than centralizing data on a server, thereby greatly enhancing the privacy and security of training data. However, the distribution of training data across different clients may be imbalanced, with different categories of data potentially residing on different clients. This presents a challenge to traditional federated learning, which assumes data distribution is independent and identically distributed (IID). This paper proposes a Blockchain-based Federated Learning Model for Non-IID Data (BFLN), which combines federated learning with blockchain technology. By introducing a new aggregation method and incentive algorithm, BFLN enhances the model performance of federated learning on non-IID data. Experiments on public datasets demonstrate that, compared to other state-of-the-art models, BFLN improves training accuracy and provides a sustainable incentive mechanism for personalized federated learning.
△ Less
Submitted 10 July, 2024; v1 submitted 7 July, 2024;
originally announced July 2024.
-
Using Synthetic Data to Regularize Maximum Likelihood Estimation
Authors:
Weihao Li,
Dongming Huang
Abstract:
To overcome challenges in fitting complex models with small samples, catalytic priors have recently been proposed to stabilize the inference by supplementing observed data with synthetic data generated from simpler models. Based on a catalytic prior, the Maximum A Posteriori (MAP) estimator is a regularized estimator that maximizes the weighted likelihood of the combined data. This estimator is st…
▽ More
To overcome challenges in fitting complex models with small samples, catalytic priors have recently been proposed to stabilize the inference by supplementing observed data with synthetic data generated from simpler models. Based on a catalytic prior, the Maximum A Posteriori (MAP) estimator is a regularized estimator that maximizes the weighted likelihood of the combined data. This estimator is straightforward to compute, and its numerical performance is superior or comparable to other likelihood-based estimators. In this paper, we study several theoretical aspects regarding the MAP estimator in generalized linear models, with a particular focus on logistic regression. We first prove that under mild conditions, the MAP estimator exists and is stable against the randomness in synthetic data. We then establish the consistency of the MAP estimator when the dimension of covariates diverges slower than the sample size. Furthermore, we utilize the convex Gaussian min-max theorem to characterize the asymptotic behavior of the MAP estimator as the dimension grows linearly with the sample size. These theoretical results clarify the role of the tuning parameters in a catalytic prior, and provide insights in practical applications. We provide numerical studies to confirm the effective approximation of our asymptotic theory in finite samples and to illustrate adjusting inference based on the theory.
△ Less
Submitted 4 July, 2024;
originally announced July 2024.
-
Observation of the Galactic Center PeVatron Beyond 100 TeV with HAWC
Authors:
A. Albert,
R. Alfaro,
C. Alvarez,
A. Andrés,
J. C. Arteaga-Velázquez,
D. Avila Rojas,
H. A. Ayala Solares,
R. Babu,
E. Belmont-Moreno,
A. Bernal,
K. S. Caballero-Mora,
T. Capistrán,
A. Carramiñana,
S. Casanova,
U. Cotti,
J. Cotzomi,
S. Coutiño de León,
E. De la Fuente,
C. de León,
D. Depaoli,
N. Di Lalla,
N. Di Lalla,
R. Diaz Hernandez,
B. L. Dingus,
M. A. DuVernois
, et al. (78 additional authors not shown)
Abstract:
We report an observation of ultra-high energy (UHE) gamma rays from the Galactic Center region, using seven years of data collected by the High-Altitude Water Cherenkov (HAWC) Observatory. The HAWC data are best described as a point-like source (HAWC J1746-2856) with a power-law spectrum ($\mathrm{d}N/\mathrm{d}E=φ(E/26 \,\text{TeV})^γ$), where $γ=-2.88 \pm 0.15_{\text{stat}} - 0.1_{\text{sys}} $…
▽ More
We report an observation of ultra-high energy (UHE) gamma rays from the Galactic Center region, using seven years of data collected by the High-Altitude Water Cherenkov (HAWC) Observatory. The HAWC data are best described as a point-like source (HAWC J1746-2856) with a power-law spectrum ($\mathrm{d}N/\mathrm{d}E=φ(E/26 \,\text{TeV})^γ$), where $γ=-2.88 \pm 0.15_{\text{stat}} - 0.1_{\text{sys}} $ and $φ=1.5 \times 10^{-15}$ (TeV cm$^{2}$s)$^{-1}$ $\pm\, 0.3_{\text{stat}}\,^{+0.08_{\text{sys}}}_{-0.13_{\text{sys}}}$ extending from 6 to 114 TeV. We find no evidence of a spectral cutoff up to $100$ TeV using HAWC data. Two known point-like gamma-ray sources are spatially coincident with the HAWC gamma-ray excess: Sgr A$^{*}$ (HESS J1745-290) and the Arc (HESS J1746-285). We subtract the known flux contribution of these point sources from the measured flux of HAWC J1746-2856 to exclude their contamination and show that the excess observed by HAWC remains significant ($>$5$σ$) with the spectrum extending to $>$100 TeV. Our result supports that these detected UHE gamma rays can originate via hadronic interaction of PeV cosmic-ray protons with the dense ambient gas and confirms the presence of a proton PeVatron at the Galactic Center.
△ Less
Submitted 4 July, 2024;
originally announced July 2024.
-
Understanding the Emission and Morphology of the Unidentified Gamma-Ray Source TeV J2032+4130
Authors:
R. Alfaro,
C. Alvarez,
J. C. Arteaga-Velázquez,
D. Avila Rojas,
H. A. Ayala Solares,
R. Babu,
E. Belmont-Moreno,
K. S. Caballero-Mora,
T. Capistrán,
A. Carramiñana,
S. Casanova,
U. Cotti,
J. Cotzomi,
S. Coutiño de León,
E. De la Fuente,
C. de León,
D. Depaoli,
N. Di Lalla,
R. Diaz Hernandez,
B. L. Dingus,
M. A. DuVernois,
J. C. Díaz-Vélez,
K. Engel,
T. Ergin,
C. Espinoza
, et al. (56 additional authors not shown)
Abstract:
The first TeV gamma-ray source with no lower energy counterparts, TeV J2032+4130, was discovered by HEGRA. It appears in the third HAWC catalog as 3HWC J2031+415 and it is a bright TeV gamma-ray source whose emission has previously been resolved as 2 sources: HAWC J2031+415 and HAWC J2030+409. While HAWC J2030+409 has since been associated with the \emph{Fermi-LAT} Cygnus Cocoon, no such associati…
▽ More
The first TeV gamma-ray source with no lower energy counterparts, TeV J2032+4130, was discovered by HEGRA. It appears in the third HAWC catalog as 3HWC J2031+415 and it is a bright TeV gamma-ray source whose emission has previously been resolved as 2 sources: HAWC J2031+415 and HAWC J2030+409. While HAWC J2030+409 has since been associated with the \emph{Fermi-LAT} Cygnus Cocoon, no such association for HAWC J2031+415 has yet been found. In this work, we investigate the spectrum and energy-dependent morphology of HAWC J2031+415. We associate HAWC J2031+415 with the pulsar PSR J2032+4127 and perform a combined multi-wavelength analysis using radio, X-ray, and $γ$-ray emission. We conclude that HAWC J2031+415 and, by extension, TeV J2032+4130 are most probably a pulsar wind nebula (PWN) powered by PSR J2032+4127.
△ Less
Submitted 3 July, 2024;
originally announced July 2024.
-
ASSR-NeRF: Arbitrary-Scale Super-Resolution on Voxel Grid for High-Quality Radiance Fields Reconstruction
Authors:
Ding-Jiun Huang,
Zi-Ting Chou,
Yu-Chiang Frank Wang,
Cheng Sun
Abstract:
NeRF-based methods reconstruct 3D scenes by building a radiance field with implicit or explicit representations. While NeRF-based methods can perform novel view synthesis (NVS) at arbitrary scale, the performance in high-resolution novel view synthesis (HRNVS) with low-resolution (LR) optimization often results in oversmoothing. On the other hand, single-image super-resolution (SR) aims to enhance…
▽ More
NeRF-based methods reconstruct 3D scenes by building a radiance field with implicit or explicit representations. While NeRF-based methods can perform novel view synthesis (NVS) at arbitrary scale, the performance in high-resolution novel view synthesis (HRNVS) with low-resolution (LR) optimization often results in oversmoothing. On the other hand, single-image super-resolution (SR) aims to enhance LR images to HR counterparts but lacks multi-view consistency. To address these challenges, we propose Arbitrary-Scale Super-Resolution NeRF (ASSR-NeRF), a novel framework for super-resolution novel view synthesis (SRNVS). We propose an attention-based VoxelGridSR model to directly perform 3D super-resolution (SR) on the optimized volume. Our model is trained on diverse scenes to ensure generalizability. For unseen scenes trained with LR views, we then can directly apply our VoxelGridSR to further refine the volume and achieve multi-view consistent SR. We demonstrate quantitative and qualitatively that the proposed method achieves significant performance in SRNVS.
△ Less
Submitted 28 June, 2024;
originally announced June 2024.
-
SPIRONet: Spatial-Frequency Learning and Topological Channel Interaction Network for Vessel Segmentation
Authors:
De-Xing Huang,
Xiao-Hu Zhou,
Xiao-Liang Xie,
Shi-Qi Liu,
Shuang-Yi Wang,
Zhen-Qiu Feng,
Mei-Jiang Gui,
Hao Li,
Tian-Yu Xiang,
Bo-Xian Yao,
Zeng-Guang Hou
Abstract:
Automatic vessel segmentation is paramount for developing next-generation interventional navigation systems. However, current approaches suffer from suboptimal segmentation performances due to significant challenges in intraoperative images (i.e., low signal-to-noise ratio, small or slender vessels, and strong interference). In this paper, a novel spatial-frequency learning and topological channel…
▽ More
Automatic vessel segmentation is paramount for developing next-generation interventional navigation systems. However, current approaches suffer from suboptimal segmentation performances due to significant challenges in intraoperative images (i.e., low signal-to-noise ratio, small or slender vessels, and strong interference). In this paper, a novel spatial-frequency learning and topological channel interaction network (SPIRONet) is proposed to address the above issues. Specifically, dual encoders are utilized to comprehensively capture local spatial and global frequency vessel features. Then, a cross-attention fusion module is introduced to effectively fuse spatial and frequency features, thereby enhancing feature discriminability. Furthermore, a topological channel interaction module is designed to filter out task-irrelevant responses based on graph neural networks. Extensive experimental results on several challenging datasets (CADSA, CAXF, DCA1, and XCAD) demonstrate state-of-the-art performances of our method. Moreover, the inference speed of SPIRONet is 21 FPS with a 512x512 input size, surpassing clinical real-time requirements (6~12FPS). These promising outcomes indicate SPIRONet's potential for integration into vascular interventional navigation systems. Code is available at https://github.com/Dxhuang-CASIA/SPIRONet.
△ Less
Submitted 28 June, 2024;
originally announced June 2024.
-
STAL3D: Unsupervised Domain Adaptation for 3D Object Detection via Collaborating Self-Training and Adversarial Learning
Authors:
Yanan Zhang,
Chao Zhou,
Di Huang
Abstract:
Existing 3D object detection suffers from expensive annotation costs and poor transferability to unknown data due to the domain gap, Unsupervised Domain Adaptation (UDA) aims to generalize detection models trained in labeled source domains to perform robustly on unexplored target domains, providing a promising solution for cross-domain 3D object detection. Although Self-Training (ST) based cross-d…
▽ More
Existing 3D object detection suffers from expensive annotation costs and poor transferability to unknown data due to the domain gap, Unsupervised Domain Adaptation (UDA) aims to generalize detection models trained in labeled source domains to perform robustly on unexplored target domains, providing a promising solution for cross-domain 3D object detection. Although Self-Training (ST) based cross-domain 3D detection methods with the assistance of pseudo-labeling techniques have achieved remarkable progress, they still face the issue of low-quality pseudo-labels when there are significant domain disparities due to the absence of a process for feature distribution alignment. While Adversarial Learning (AL) based methods can effectively align the feature distributions of the source and target domains, the inability to obtain labels in the target domain forces the adoption of asymmetric optimization losses, resulting in a challenging issue of source domain bias. To overcome these limitations, we propose a novel unsupervised domain adaptation framework for 3D object detection via collaborating ST and AL, dubbed as STAL3D, unleashing the complementary advantages of pseudo labels and feature distribution alignment. Additionally, a Background Suppression Adversarial Learning (BS-AL) module and a Scale Filtering Module (SFM) are designed tailored for 3D cross-domain scenes, effectively alleviating the issues of the large proportion of background interference and source domain size bias. Our STAL3D achieves state-of-the-art performance on multiple cross-domain tasks and even surpasses the Oracle results on Waymo $\rightarrow$ KITTI and Waymo $\rightarrow$ KITTI-rain.
△ Less
Submitted 27 June, 2024;
originally announced June 2024.
-
Efficient, Multimodal, and Derivative-Free Bayesian Inference With Fisher-Rao Gradient Flows
Authors:
Yifan Chen,
Daniel Zhengyu Huang,
Jiaoyang Huang,
Sebastian Reich,
Andrew M. Stuart
Abstract:
In this paper, we study efficient approximate sampling for probability distributions known up to normalization constants. We specifically focus on a problem class arising in Bayesian inference for large-scale inverse problems in science and engineering applications. The computational challenges we address with the proposed methodology are: (i) the need for repeated evaluations of expensive forward…
▽ More
In this paper, we study efficient approximate sampling for probability distributions known up to normalization constants. We specifically focus on a problem class arising in Bayesian inference for large-scale inverse problems in science and engineering applications. The computational challenges we address with the proposed methodology are: (i) the need for repeated evaluations of expensive forward models; (ii) the potential existence of multiple modes; and (iii) the fact that gradient of, or adjoint solver for, the forward model might not be feasible.
While existing Bayesian inference methods meet some of these challenges individually, we propose a framework that tackles all three systematically. Our approach builds upon the Fisher-Rao gradient flow in probability space, yielding a dynamical system for probability densities that converges towards the target distribution at a uniform exponential rate. This rapid convergence is advantageous for the computational burden outlined in (i). We apply Gaussian mixture approximations with operator splitting techniques to simulate the flow numerically; the resulting approximation can capture multiple modes thus addressing (ii). Furthermore, we employ the Kalman methodology to facilitate a derivative-free update of these Gaussian components and their respective weights, addressing the issue in (iii).
The proposed methodology results in an efficient derivative-free sampler flexible enough to handle multi-modal distributions: Gaussian Mixture Kalman Inversion (GMKI). The effectiveness of GMKI is demonstrated both theoretically and numerically in several experiments with multimodal target distributions, including proof-of-concept and two-dimensional examples, as well as a large-scale application: recovering the Navier-Stokes initial condition from solution data at positive times.
△ Less
Submitted 25 June, 2024;
originally announced June 2024.
-
ClotheDreamer: Text-Guided Garment Generation with 3D Gaussians
Authors:
Yufei Liu,
Junshu Tang,
Chu Zheng,
Shijie Zhang,
Jinkun Hao,
Junwei Zhu,
Dongjin Huang
Abstract:
High-fidelity 3D garment synthesis from text is desirable yet challenging for digital avatar creation. Recent diffusion-based approaches via Score Distillation Sampling (SDS) have enabled new possibilities but either intricately couple with human body or struggle to reuse. We introduce ClotheDreamer, a 3D Gaussian-based method for generating wearable, production-ready 3D garment assets from text p…
▽ More
High-fidelity 3D garment synthesis from text is desirable yet challenging for digital avatar creation. Recent diffusion-based approaches via Score Distillation Sampling (SDS) have enabled new possibilities but either intricately couple with human body or struggle to reuse. We introduce ClotheDreamer, a 3D Gaussian-based method for generating wearable, production-ready 3D garment assets from text prompts. We propose a novel representation Disentangled Clothe Gaussian Splatting (DCGS) to enable separate optimization. DCGS represents clothed avatar as one Gaussian model but freezes body Gaussian splats. To enhance quality and completeness, we incorporate bidirectional SDS to supervise clothed avatar and garment RGBD renderings respectively with pose conditions and propose a new pruning strategy for loose clothing. Our approach can also support custom clothing templates as input. Benefiting from our design, the synthetic 3D garment can be easily applied to virtual try-on and support physically accurate animation. Extensive experiments showcase our method's superior and competitive performance. Our project page is at https://ggxxii.github.io/clothedreamer.
△ Less
Submitted 24 June, 2024;
originally announced June 2024.
-
Gravitational Wave Birefringence from Fuzzy Dark Matter
Authors:
Da Huang,
Ze-Xuan Xiong
Abstract:
Gravitational wave (GW) birefringence is a remarkable phenomenon that can be used to test the parity violation in gravity. By coupling the fuzzy dark matter (FDM) scalar to the gravitational Chern-Simons term, we explore the GW birefringence effects in the FDM background. In particular, in light of the highly oscillating granular FDM structure at the galactic scale, we are led to investigating the…
▽ More
Gravitational wave (GW) birefringence is a remarkable phenomenon that can be used to test the parity violation in gravity. By coupling the fuzzy dark matter (FDM) scalar to the gravitational Chern-Simons term, we explore the GW birefringence effects in the FDM background. In particular, in light of the highly oscillating granular FDM structure at the galactic scale, we are led to investigating the GW propagation in the Chern-Simons gravity over the general nontrivial scalar profile, which is a natural extension of previous studies on the homogeneous and isotropic configurations. As a result, it is found that GWs of both circularly polarized modes propagate in the straight line with the speed of light, and does not show any velocity birefringence. However, when considering the imaginary part of the dispersion relation, GWs exhibit the amplitude birefringence in which one circular polarization is enhanced while the other suppressed. Due to its local nature, the FDM-induced amplitude birefringence only depends on the GW frequency without any reliance on the GW event distance. More importantly, the birefringence factor shows a periodic time variation with the period reflecting the FDM scalar mass, which is the smoking gun for testing this new birefringence mechanism. Finally, we also study the extra-galactic FDM contribution to the GW birefringence, which is shown to be suppressed by the cosmological DM density and thus subdominant compared with the galactic counterpart.
△ Less
Submitted 19 June, 2024;
originally announced June 2024.
-
ARDuP: Active Region Video Diffusion for Universal Policies
Authors:
Shuaiyi Huang,
Mara Levy,
Zhenyu Jiang,
Anima Anandkumar,
Yuke Zhu,
Linxi Fan,
De-An Huang,
Abhinav Shrivastava
Abstract:
Sequential decision-making can be formulated as a text-conditioned video generation problem, where a video planner, guided by a text-defined goal, generates future frames visualizing planned actions, from which control actions are subsequently derived. In this work, we introduce Active Region Video Diffusion for Universal Policies (ARDuP), a novel framework for video-based policy learning that emp…
▽ More
Sequential decision-making can be formulated as a text-conditioned video generation problem, where a video planner, guided by a text-defined goal, generates future frames visualizing planned actions, from which control actions are subsequently derived. In this work, we introduce Active Region Video Diffusion for Universal Policies (ARDuP), a novel framework for video-based policy learning that emphasizes the generation of active regions, i.e. potential interaction areas, enhancing the conditional policy's focus on interactive areas critical for task execution. This innovative framework integrates active region conditioning with latent diffusion models for video planning and employs latent representations for direct action decoding during inverse dynamic modeling. By utilizing motion cues in videos for automatic active region discovery, our method eliminates the need for manual annotations of active regions. We validate ARDuP's efficacy via extensive experiments on simulator CLIPort and the real-world dataset BridgeData v2, achieving notable improvements in success rates and generating convincingly realistic video plans.
△ Less
Submitted 19 June, 2024;
originally announced June 2024.
-
The Design, Implementation, and Performance of the LZ Calibration Systems
Authors:
J. Aalbers,
D. S. Akerib,
A. K. Al Musalhi,
F. Alder,
C. S. Amarasinghe,
A. Ames,
T. J. Anderson,
N. Angelides,
H. M. Araújo,
J. E. Armstrong,
M. Arthurs,
A. Baker,
S. Balashov,
J. Bang,
E. E. Barillier,
J. W. Bargemann,
K. Beattie,
T. Benson,
A. Bhatti,
A. Biekert,
T. P. Biesiadzinski,
H. J. Birch,
E. Bishop,
G. M. Blockinger,
B. Boxer
, et al. (179 additional authors not shown)
Abstract:
LUX-ZEPLIN (LZ) is a tonne-scale experiment searching for direct dark matter interactions and other rare events. It is located at the Sanford Underground Research Facility (SURF) in Lead, South Dakota, USA. The core of the LZ detector is a dual-phase xenon time projection chamber (TPC), designed with the primary goal of detecting Weakly Interacting Massive Particles (WIMPs) via their induced low e…
▽ More
LUX-ZEPLIN (LZ) is a tonne-scale experiment searching for direct dark matter interactions and other rare events. It is located at the Sanford Underground Research Facility (SURF) in Lead, South Dakota, USA. The core of the LZ detector is a dual-phase xenon time projection chamber (TPC), designed with the primary goal of detecting Weakly Interacting Massive Particles (WIMPs) via their induced low energy nuclear recoils. Surrounding the TPC, two veto detectors immersed in an ultra-pure water tank enable reducing background events to enhance the discovery potential. Intricate calibration systems are purposely designed to precisely understand the responses of these three detector volumes to various types of particle interactions and to demonstrate LZ's ability to discriminate between signals and backgrounds. In this paper, we present a comprehensive discussion of the key features, requirements, and performance of the LZ calibration systems, which play a crucial role in enabling LZ's WIMP-search and its broad science program. The thorough description of these calibration systems, with an emphasis on their novel aspects, is valuable for future calibration efforts in direct dark matter and other rare-event search experiments.
△ Less
Submitted 20 June, 2024; v1 submitted 2 May, 2024;
originally announced June 2024.
-
Holistic-Motion2D: Scalable Whole-body Human Motion Generation in 2D Space
Authors:
Yuan Wang,
Zhao Wang,
Junhao Gong,
Di Huang,
Tong He,
Wanli Ouyang,
Jile Jiao,
Xuetao Feng,
Qi Dou,
Shixiang Tang,
Dan Xu
Abstract:
In this paper, we introduce a novel path to $\textit{general}$ human motion generation by focusing on 2D space. Traditional methods have primarily generated human motions in 3D, which, while detailed and realistic, are often limited by the scope of available 3D motion data in terms of both the size and the diversity. To address these limitations, we exploit extensive availability of 2D motion data…
▽ More
In this paper, we introduce a novel path to $\textit{general}$ human motion generation by focusing on 2D space. Traditional methods have primarily generated human motions in 3D, which, while detailed and realistic, are often limited by the scope of available 3D motion data in terms of both the size and the diversity. To address these limitations, we exploit extensive availability of 2D motion data. We present $\textbf{Holistic-Motion2D}$, the first comprehensive and large-scale benchmark for 2D whole-body motion generation, which includes over 1M in-the-wild motion sequences, each paired with high-quality whole-body/partial pose annotations and textual descriptions. Notably, Holistic-Motion2D is ten times larger than the previously largest 3D motion dataset. We also introduce a baseline method, featuring innovative $\textit{whole-body part-aware attention}$ and $\textit{confidence-aware modeling}$ techniques, tailored for 2D $\underline{\text T}$ext-driv$\underline{\text{EN}}$ whole-bo$\underline{\text D}$y motion gen$\underline{\text{ER}}$ation, namely $\textbf{Tender}$. Extensive experiments demonstrate the effectiveness of $\textbf{Holistic-Motion2D}$ and $\textbf{Tender}$ in generating expressive, diverse, and realistic human motions. We also highlight the utility of 2D motion for various downstream applications and its potential for lifting to 3D motion. The page link is: https://holistic-motion2d.github.io.
△ Less
Submitted 17 June, 2024;
originally announced June 2024.
-
MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformers
Authors:
Yiwen Chen,
Tong He,
Di Huang,
Weicai Ye,
Sijin Chen,
Jiaxiang Tang,
Xin Chen,
Zhongang Cai,
Lei Yang,
Gang Yu,
Guosheng Lin,
Chi Zhang
Abstract:
Recently, 3D assets created via reconstruction and generation have matched the quality of manually crafted assets, highlighting their potential for replacement. However, this potential is largely unrealized because these assets always need to be converted to meshes for 3D industry applications, and the meshes produced by current mesh extraction methods are significantly inferior to Artist-Created…
▽ More
Recently, 3D assets created via reconstruction and generation have matched the quality of manually crafted assets, highlighting their potential for replacement. However, this potential is largely unrealized because these assets always need to be converted to meshes for 3D industry applications, and the meshes produced by current mesh extraction methods are significantly inferior to Artist-Created Meshes (AMs), i.e., meshes created by human artists. Specifically, current mesh extraction methods rely on dense faces and ignore geometric features, leading to inefficiencies, complicated post-processing, and lower representation quality. To address these issues, we introduce MeshAnything, a model that treats mesh extraction as a generation problem, producing AMs aligned with specified shapes. By converting 3D assets in any 3D representation into AMs, MeshAnything can be integrated with various 3D asset production methods, thereby enhancing their application across the 3D industry. The architecture of MeshAnything comprises a VQ-VAE and a shape-conditioned decoder-only transformer. We first learn a mesh vocabulary using the VQ-VAE, then train the shape-conditioned decoder-only transformer on this vocabulary for shape-conditioned autoregressive mesh generation. Our extensive experiments show that our method generates AMs with hundreds of times fewer faces, significantly improving storage, rendering, and simulation efficiencies, while achieving precision comparable to previous methods.
△ Less
Submitted 14 June, 2024;
originally announced June 2024.
-
Steady Contiguous Vortex-Patch Dipole Solutions of the 2D Incompressible Euler Equation
Authors:
De Huang,
Jiajun Tong
Abstract:
We rigorously construct the first steady traveling wave solutions of the 2D incompressible Euler equation that take the form of a contiguous vortex-patch dipole, which can be viewed as the vortex-patch counterpart of the well-known Lamb-Chaplygin dipole. Our construction is based on a novel fixed-point approach that determines the patch boundary as the fixed point of a certain nonlinear map. Smoot…
▽ More
We rigorously construct the first steady traveling wave solutions of the 2D incompressible Euler equation that take the form of a contiguous vortex-patch dipole, which can be viewed as the vortex-patch counterpart of the well-known Lamb-Chaplygin dipole. Our construction is based on a novel fixed-point approach that determines the patch boundary as the fixed point of a certain nonlinear map. Smoothness and other properties of the patch boundary are also obtained.
△ Less
Submitted 20 June, 2024; v1 submitted 14 June, 2024;
originally announced June 2024.
-
Constraints on Ultra Heavy Dark Matter Properties from Dwarf Spheroidal Galaxies with LHAASO Observations
Authors:
Zhen Cao,
F. Aharonian,
Q. An,
Axikegu,
Y. X. Bai,
Y. W. Bao,
D. Bastieri,
X. J. Bi,
Y. J. Bi,
J. T. Cai,
Q. Cao,
W. Y. Cao,
Zhe Cao,
J. Chang,
J. F. Chang,
A. M. Chen,
E. S. Chen,
Liang Chen,
Lin Chen,
Long Chen,
M. J. Chen,
M. L. Chen,
Q. H. Chen,
S. H. Chen,
S. Z. Chen
, et al. (255 additional authors not shown)
Abstract:
In this work we try to search for signals generated by ultra-heavy dark matter at the Large High Altitude Air Shower Observatory (LHAASO) data. We look for possible gamma-ray by dark matter annihilation or decay from 16 dwarf spheroidal galaxies in the field of view of LHAASO. Dwarf spheroidal galaxies are among the most promising targets for indirect detection of dark matter which have low fluxes…
▽ More
In this work we try to search for signals generated by ultra-heavy dark matter at the Large High Altitude Air Shower Observatory (LHAASO) data. We look for possible gamma-ray by dark matter annihilation or decay from 16 dwarf spheroidal galaxies in the field of view of LHAASO. Dwarf spheroidal galaxies are among the most promising targets for indirect detection of dark matter which have low fluxes of astrophysical $γ$-ray background while large amount of dark matter. By analyzing more than 700 days observational data at LHAASO, no significant dark matter signal from 1 TeV to 1 EeV is detected. Accordingly we derive the most stringent constraints on the ultra-heavy dark matter annihilation cross-section up to EeV. The constraints on the lifetime of dark matter in decay mode are also derived.
△ Less
Submitted 12 June, 2024;
originally announced June 2024.
-
MOSA: Music Motion with Semantic Annotation Dataset for Cross-Modal Music Processing
Authors:
Yu-Fen Huang,
Nikki Moran,
Simon Coleman,
Jon Kelly,
Shun-Hwa Wei,
Po-Yin Chen,
Yun-Hsin Huang,
Tsung-Ping Chen,
Yu-Chia Kuo,
Yu-Chi Wei,
Chih-Hsuan Li,
Da-Yu Huang,
Hsuan-Kai Kao,
Ting-Wei Lin,
Li Su
Abstract:
In cross-modal music processing, translation between visual, auditory, and semantic content opens up new possibilities as well as challenges. The construction of such a transformative scheme depends upon a benchmark corpus with a comprehensive data infrastructure. In particular, the assembly of a large-scale cross-modal dataset presents major challenges. In this paper, we present the MOSA (Music m…
▽ More
In cross-modal music processing, translation between visual, auditory, and semantic content opens up new possibilities as well as challenges. The construction of such a transformative scheme depends upon a benchmark corpus with a comprehensive data infrastructure. In particular, the assembly of a large-scale cross-modal dataset presents major challenges. In this paper, we present the MOSA (Music mOtion with Semantic Annotation) dataset, which contains high quality 3-D motion capture data, aligned audio recordings, and note-by-note semantic annotations of pitch, beat, phrase, dynamic, articulation, and harmony for 742 professional music performances by 23 professional musicians, comprising more than 30 hours and 570 K notes of data. To our knowledge, this is the largest cross-modal music dataset with note-level annotations to date. To demonstrate the usage of the MOSA dataset, we present several innovative cross-modal music information retrieval (MIR) and musical content generation tasks, including the detection of beats, downbeats, phrase, and expressive contents from audio, video and motion data, and the generation of musicians' body motion from given music audio. The dataset and codes are available alongside this publication (https://github.com/yufenhuang/MOSA-Music-mOtion-and-Semantic-Annotation-dataset).
△ Less
Submitted 10 June, 2024;
originally announced June 2024.
-
Information-Theoretic Thresholds for the Alignments of Partially Correlated Graphs
Authors:
Dong Huang,
Xianwen Song,
Pengkun Yang
Abstract:
This paper studies the problem of recovering the hidden vertex correspondence between two correlated random graphs. We propose the partially correlated Erdős-Rényi graphs model, wherein a pair of induced subgraphs with a certain number are correlated. We investigate the information-theoretic thresholds for recovering the latent correlated subgraphs and the hidden vertex correspondence. We prove th…
▽ More
This paper studies the problem of recovering the hidden vertex correspondence between two correlated random graphs. We propose the partially correlated Erdős-Rényi graphs model, wherein a pair of induced subgraphs with a certain number are correlated. We investigate the information-theoretic thresholds for recovering the latent correlated subgraphs and the hidden vertex correspondence. We prove that there exists an optimal rate for partial recovery for the number of correlated nodes, above which one can correctly match a fraction of vertices and below which correctly matching any positive fraction is impossible, and we also derive an optimal rate for exact recovery. In the proof of possibility results, we propose correlated functional digraphs, which partition the edges of the intersection graph into two types of components, and bound the error probability by lower-order cumulant generating functions. The proof of impossibility results build upon the generalized Fano's inequality and the recovery thresholds settled in correlated Erdős-Rényi graphs model.
△ Less
Submitted 8 June, 2024;
originally announced June 2024.
-
Research on Tumors Segmentation based on Image Enhancement Method
Authors:
Danyi Huang,
Ziang Liu,
Yizhou Li
Abstract:
One of the most effective ways to treat liver cancer is to perform precise liver resection surgery, the key step of which includes precise digital image segmentation of the liver and its tumor. However, traditional liver parenchymal segmentation techniques often face several challenges in performing liver segmentation: lack of precision, slow processing speed, and computational burden. These short…
▽ More
One of the most effective ways to treat liver cancer is to perform precise liver resection surgery, the key step of which includes precise digital image segmentation of the liver and its tumor. However, traditional liver parenchymal segmentation techniques often face several challenges in performing liver segmentation: lack of precision, slow processing speed, and computational burden. These shortcomings limit the efficiency of surgical planning and execution. In this work, the model initially describes in detail a new image enhancement algorithm that enhances the key features of an image by adaptively adjusting the contrast and brightness of the image. Then, a deep learning-based segmentation network was introduced, which was specially trained on the enhanced images to optimize the detection accuracy of tumor regions. In addition, multi-scale analysis techniques have been incorporated into the study, allowing the model to analyze images at different resolutions to capture more nuanced tumor features. In the presentation of the experimental results, the study used the 3Dircadb dataset to test the effectiveness of the proposed method. The experimental results show that compared with the traditional image segmentation method, the new method using image enhancement technology has significantly improved the accuracy and recall rate of tumor identification.
△ Less
Submitted 7 June, 2024;
originally announced June 2024.
-
Characterizing Similarities and Divergences in Conversational Tones in Humans and LLMs by Sampling with People
Authors:
Dun-Ming Huang,
Pol Van Rijn,
Ilia Sucholutsky,
Raja Marjieh,
Nori Jacoby
Abstract:
Conversational tones -- the manners and attitudes in which speakers communicate -- are essential to effective communication. Amidst the increasing popularization of Large Language Models (LLMs) over recent years, it becomes necessary to characterize the divergences in their conversational tones relative to humans. However, existing investigations of conversational modalities rely on pre-existing t…
▽ More
Conversational tones -- the manners and attitudes in which speakers communicate -- are essential to effective communication. Amidst the increasing popularization of Large Language Models (LLMs) over recent years, it becomes necessary to characterize the divergences in their conversational tones relative to humans. However, existing investigations of conversational modalities rely on pre-existing taxonomies or text corpora, which suffer from experimenter bias and may not be representative of real-world distributions for the studies' psycholinguistic domains. Inspired by methods from cognitive science, we propose an iterative method for simultaneously eliciting conversational tones and sentences, where participants alternate between two tasks: (1) one participant identifies the tone of a given sentence and (2) a different participant generates a sentence based on that tone. We run 100 iterations of this process with human participants and GPT-4, then obtain a dataset of sentences and frequent conversational tones. In an additional experiment, humans and GPT-4 annotated all sentences with all tones. With data from 1,339 human participants, 33,370 human judgments, and 29,900 GPT-4 queries, we show how our approach can be used to create an interpretable geometric representation of relations between conversational tones in humans and GPT-4. This work demonstrates how combining ideas from machine learning and cognitive science can address challenges in human-computer interactions.
△ Less
Submitted 6 June, 2024;
originally announced June 2024.
-
Bayesian Power Steering: An Effective Approach for Domain Adaptation of Diffusion Models
Authors:
Ding Huang,
Ting Li,
Jian Huang
Abstract:
We propose a Bayesian framework for fine-tuning large diffusion models with a novel network structure called Bayesian Power Steering (BPS). We clarify the meaning behind adaptation from a \textit{large probability space} to a \textit{small probability space} and explore the task of fine-tuning pre-trained models using learnable modules from a Bayesian perspective. BPS extracts task-specific knowle…
▽ More
We propose a Bayesian framework for fine-tuning large diffusion models with a novel network structure called Bayesian Power Steering (BPS). We clarify the meaning behind adaptation from a \textit{large probability space} to a \textit{small probability space} and explore the task of fine-tuning pre-trained models using learnable modules from a Bayesian perspective. BPS extracts task-specific knowledge from a pre-trained model's learned prior distribution. It efficiently leverages large diffusion models, differentially intervening different hidden features with a head-heavy and foot-light configuration. Experiments highlight the superiority of BPS over contemporary methods across a range of tasks even with limited amount of data. Notably, BPS attains an FID score of 10.49 under the sketch condition on the COCO17 dataset.
△ Less
Submitted 5 June, 2024;
originally announced June 2024.
-
A Hybrid Deep Learning Classification of Perimetric Glaucoma Using Peripapillary Nerve Fiber Layer Reflectance and Other OCT Parameters from Three Anatomy Regions
Authors:
Ou Tan,
David S. Greenfield,
Brian A. Francis,
Rohit Varma,
Joel S. Schuman,
David Huang,
Dongseok Choi
Abstract:
Precis: A hybrid deep-learning model combines NFL reflectance and other OCT parameters to improve glaucoma diagnosis. Objective: To investigate if a deep learning model could be used to combine nerve fiber layer (NFL) reflectance and other OCT parameters for glaucoma diagnosis. Patients and Methods: This is a prospective observational study where of 106 normal subjects and 164 perimetric glaucoma…
▽ More
Precis: A hybrid deep-learning model combines NFL reflectance and other OCT parameters to improve glaucoma diagnosis. Objective: To investigate if a deep learning model could be used to combine nerve fiber layer (NFL) reflectance and other OCT parameters for glaucoma diagnosis. Patients and Methods: This is a prospective observational study where of 106 normal subjects and 164 perimetric glaucoma (PG) patients. Peripapillary NFL reflectance map, NFL thickness map, optic head analysis of disc, and macular ganglion cell complex thickness were obtained using spectral domain OCT. A hybrid deep learning model combined a fully connected network (FCN) and a convolution neural network (CNN) to develop and combine those OCT maps and parameters to distinguish normal and PG eyes. Two deep learning models were compared based on whether the NFL reflectance map was used as part of the input or not. Results: The hybrid deep learning model with reflectance achieved 0.909 sensitivity at 99% specificity and 0.926 at 95%. The overall accuracy was 0.948 with 0.893 sensitivity and 1.000 specificity, and the AROC was 0.979, which is significantly better than the logistic regression models (p < 0.001). The second best model is the hybrid deep learning model w/o reflectance, which also had significantly higher AROC than logistic regression models (p < 0.001). Logistic regression with reflectance model had slightly higher AROC or sensitivity than the other logistic regression model without reflectance (p = 0.024). Conclusions: Hybrid deep learning model significantly improved the diagnostic accuracy, without or without NFL reflectance. Hybrid deep learning model, combining reflectance/NFL thickness/GCC thickness/ONH parameter, may be a practical model for glaucoma screen purposes.
△ Less
Submitted 5 June, 2024;
originally announced June 2024.
-
Probing the Scalar WIMP-Pion Coupling with the first LUX-ZEPLIN data
Authors:
J. Aalbers,
D. S. Akerib,
A. K. Al Musalhi,
F. Alder,
C. S. Amarasinghe,
A. Ames,
T. J. Anderson,
N. Angelides,
H. M. Araújo,
J. E. Armstrong,
M. Arthurs,
A. Baker,
S. Balashov,
J. Bang,
E. E. Barillier,
J. W. Bargemann,
K. Beattie,
T. Benson,
A. Bhatti,
A. Biekert,
T. P. Biesiadzinski,
H. J. Birch,
E. J. Bishop,
G. M. Blockinger,
B. Boxer
, et al. (178 additional authors not shown)
Abstract:
Weakly interacting massive particles (WIMPs) may interact with a virtual pion that is exchanged between nucleons. This interaction channel is important to consider in models where the spin-independent isoscalar channel is suppressed. Using data from the first science run of the LUX-ZEPLIN dark matter experiment, containing 60 live days of data in a 5.5~tonne fiducial mass of liquid xenon, we repor…
▽ More
Weakly interacting massive particles (WIMPs) may interact with a virtual pion that is exchanged between nucleons. This interaction channel is important to consider in models where the spin-independent isoscalar channel is suppressed. Using data from the first science run of the LUX-ZEPLIN dark matter experiment, containing 60 live days of data in a 5.5~tonne fiducial mass of liquid xenon, we report the results on a search for WIMP-pion interactions. We observe no significant excess and set an upper limit of $1.5\times10^{-46}$~cm$^2$ at a 90\% confidence level for a WIMP mass of 33~GeV/c$^2$ for this interaction.
△ Less
Submitted 4 June, 2024;
originally announced June 2024.
-
I4VGen: Image as Stepping Stone for Text-to-Video Generation
Authors:
Xiefan Guo,
Jinlin Liu,
Miaomiao Cui,
Di Huang
Abstract:
Text-to-video generation has lagged behind text-to-image synthesis in quality and diversity due to the complexity of spatio-temporal modeling and limited video-text datasets. This paper presents I4VGen, a training-free and plug-and-play video diffusion inference framework, which enhances text-to-video generation by leveraging robust image techniques. Specifically, following text-to-image-to-video,…
▽ More
Text-to-video generation has lagged behind text-to-image synthesis in quality and diversity due to the complexity of spatio-temporal modeling and limited video-text datasets. This paper presents I4VGen, a training-free and plug-and-play video diffusion inference framework, which enhances text-to-video generation by leveraging robust image techniques. Specifically, following text-to-image-to-video, I4VGen decomposes the text-to-video generation into two stages: anchor image synthesis and anchor image-guided video synthesis. Correspondingly, a well-designed generation-selection pipeline is employed to achieve visually-realistic and semantically-faithful anchor image, and an innovative Noise-Invariant Video Score Distillation Sampling is incorporated to animate the image to a dynamic video, followed by a video regeneration process to refine the video. This inference strategy effectively mitigates the prevalent issue of non-zero terminal signal-to-noise ratio. Extensive evaluations show that I4VGen not only produces videos with higher visual realism and textual fidelity but also integrates seamlessly into existing image-to-video diffusion models, thereby improving overall video quality.
△ Less
Submitted 4 June, 2024;
originally announced June 2024.
-
AlignSAM: Aligning Segment Anything Model to Open Context via Reinforcement Learning
Authors:
Duojun Huang,
Xinyu Xiong,
Jie Ma,
Jichang Li,
Zequn Jie,
Lin Ma,
Guanbin Li
Abstract:
Powered by massive curated training data, Segment Anything Model (SAM) has demonstrated its impressive generalization capabilities in open-world scenarios with the guidance of prompts. However, the vanilla SAM is class agnostic and heavily relies on user-provided prompts to segment objects of interest. Adapting this method to diverse tasks is crucial for accurate target identification and to avoid…
▽ More
Powered by massive curated training data, Segment Anything Model (SAM) has demonstrated its impressive generalization capabilities in open-world scenarios with the guidance of prompts. However, the vanilla SAM is class agnostic and heavily relies on user-provided prompts to segment objects of interest. Adapting this method to diverse tasks is crucial for accurate target identification and to avoid suboptimal segmentation results. In this paper, we propose a novel framework, termed AlignSAM, designed for automatic prompting for aligning SAM to an open context through reinforcement learning. Anchored by an agent, AlignSAM enables the generality of the SAM model across diverse downstream tasks while keeping its parameters frozen. Specifically, AlignSAM initiates a prompting agent to iteratively refine segmentation predictions by interacting with the foundational model. It integrates a reinforcement learning policy network to provide informative prompts to the foundational models. Additionally, a semantic recalibration module is introduced to provide fine-grained labels of prompts, enhancing the model's proficiency in handling tasks encompassing explicit and implicit semantics. Experiments conducted on various challenging segmentation tasks among existing foundation models demonstrate the superiority of the proposed AlignSAM over state-of-the-art approaches. Project page: \url{https://github.com/Duojun-Huang/AlignSAM-CVPR2024}.
△ Less
Submitted 1 June, 2024;
originally announced June 2024.
-
Focal Loss Analysis of Peripapillary Nerve Fiber Layer Reflectance for Glaucoma Diagnosis
Authors:
Ou Tan,
Dongseok Choi,
Aiyin Chen,
David S. Greenfield,
Brian A. Francis,
Rohit Varma,
Joel S. Schuman,
David Huang,
Advanced Imaging for Glaucoma Study Group
Abstract:
Purpose: To evaluate nerve fiber layer (NFL) reflectance for glaucoma diagnosis using a large dataset. Methods: Participants were imaged with 4.9mm ONH scans using spectral-domain optical coherence tomography (OCT). The NFL reflectance map was reconstructed from 13 concentric rings of optic nerve head(ONH) scan, then processed by an azimuthal filter to reduce directional reflectance bias due to va…
▽ More
Purpose: To evaluate nerve fiber layer (NFL) reflectance for glaucoma diagnosis using a large dataset. Methods: Participants were imaged with 4.9mm ONH scans using spectral-domain optical coherence tomography (OCT). The NFL reflectance map was reconstructed from 13 concentric rings of optic nerve head(ONH) scan, then processed by an azimuthal filter to reduce directional reflectance bias due to variation of beam incidence angle. The peripapillary thickness and reflectance maps were both divided into 96 superpixels. Low-reflectance and low-thickness superpixels were defined as values below the 5th percentile normative reference for that location. Focal reflectance loss was measured by summing loss, relative to the normal reference average, in low-reflectance superpixels. Focal thickness loss was calculated in a similar fashion. The area under receiving characteristic curve (AROC) was used to assess diagnostic accuracy. Results: Fifty-three normal, 196 pre-perimetric, 132 early perimetric, and 59 moderate and advanced perimetric glaucoma participants were included from the Advanced Imaging for Glaucoma Study. Sixty-seven percent of glaucomatous reflectance maps showed characteristic contiguous wedge or diffuse defects. Focal NFL reflectance loss had significantly higher diagnostic accuracy than the best NFL thickness parameters (both map-based and profile-based): AROC 0.80 v. 0.75 (p<0.004) for distinguishing glaucoma eyes from healthy control eyes. The diagnostic sensitivity was also significantly higher at both 99% and 95% specificity operating points. Conclusions: Focal NFL reflectance loss improved glaucoma diagnostic accuracy compared to the standard NFL thickness parameters.
△ Less
Submitted 31 May, 2024;
originally announced June 2024.
-
Reliability for Nerve Fiber Layer Reflectance Using Spectral Domain Optical Coherence Tomography
Authors:
Kabir Hossain,
Ou Tan,
Po-Han Yeh,
Jie Wang,
Elizabeth White,
Dongseok Choi,
David Huang
Abstract:
Purpose: Reliability for Nerve Fiber Layer Reflectance Using Spectral Domain Optical Coherence Tomography (OCT) Methods: The study utilized OCT to scan participants with a cubic 6x6 mm disc scan. NFL reflectance were normalized by the average of bands below NFL and summarized. We selected several reference bands, including the pigment epithelium complex (PPEC), the band between NFL and Bruch's mem…
▽ More
Purpose: Reliability for Nerve Fiber Layer Reflectance Using Spectral Domain Optical Coherence Tomography (OCT) Methods: The study utilized OCT to scan participants with a cubic 6x6 mm disc scan. NFL reflectance were normalized by the average of bands below NFL and summarized. We selected several reference bands, including the pigment epithelium complex (PPEC), the band between NFL and Bruch's membrane (Post-NFL), and the top 50% of pixels with higher values were selected from the Post-NFL band by Post-NFL-Bright. Especially, we also included NFL attenuation coefficient (AC), which was equivalent to NFL reflectance normalized by all pixels below NFL. An experiment was designed to test the NFL reflectance against different levels of attenuation using neutral density filter (NDF). We also evaluated the within-visit and between-visit repeatability using a clinical dataset with normal and glaucoma eyes. Results: The experiment enrolled 20 healthy participants. The clinical dataset selected 22 normal and 55 glaucoma eyes with at least two visits form functional and structural OCT (FSOCT) study. The experiment showed that NFL reflectance normalized PPEC Max and Post-NFL-Bright had lowest dependence, slope=-0.77 and -1.34 dB/optical density on NDF levels, respectively. The clinical data showed that the NFL reflectance metrics normalized by Post-NFL-Bright or Post-NFL-Mean metrics had a trend of better repeatability and reproducibility than others, but the trend was not significant. All metrics demonstrated similar diagnostic accuracy (0.82-0.87), but Post-NFL-Bright provide the best result. Conclusions: The NFL reflectance normalized by the maximum in PPEC had less dependence of the global attenuation followed by Post-NFL-Bright, PPEC/Mean, Post-NFL-Mean and NFL/AC. But NFL reflectance normalized by Post-NFL-Bright had better result in two datasets.
△ Less
Submitted 31 May, 2024;
originally announced June 2024.
-
X-VILA: Cross-Modality Alignment for Large Language Model
Authors:
Hanrong Ye,
De-An Huang,
Yao Lu,
Zhiding Yu,
Wei Ping,
Andrew Tao,
Jan Kautz,
Song Han,
Dan Xu,
Pavlo Molchanov,
Hongxu Yin
Abstract:
We introduce X-VILA, an omni-modality model designed to extend the capabilities of large language models (LLMs) by incorporating image, video, and audio modalities. By aligning modality-specific encoders with LLM inputs and diffusion decoders with LLM outputs, X-VILA achieves cross-modality understanding, reasoning, and generation. To facilitate this cross-modality alignment, we curate an effectiv…
▽ More
We introduce X-VILA, an omni-modality model designed to extend the capabilities of large language models (LLMs) by incorporating image, video, and audio modalities. By aligning modality-specific encoders with LLM inputs and diffusion decoders with LLM outputs, X-VILA achieves cross-modality understanding, reasoning, and generation. To facilitate this cross-modality alignment, we curate an effective interleaved any-to-any modality instruction-following dataset. Furthermore, we identify a significant problem with the current cross-modality alignment method, which results in visual information loss. To address the issue, we propose a visual alignment mechanism with a visual embedding highway module. We then introduce a resource-efficient recipe for training X-VILA, that exhibits proficiency in any-to-any modality conversation, surpassing previous approaches by large margins. X-VILA also showcases emergent properties across modalities even in the absence of similar training data. The project will be made open-source.
△ Less
Submitted 29 May, 2024;
originally announced May 2024.
-
Hybrid-Parallel: Achieving High Performance and Energy Efficient Distributed Inference on Robots
Authors:
Zekai Sun,
Xiuxian Guan,
Junming Wang,
Haoze Song,
Yuhao Qing,
Tianxiang Shen,
Dong Huang,
Fangming Liu,
Heming Cui
Abstract:
The rapid advancements in machine learning techniques have led to significant achievements in various real-world robotic tasks. These tasks heavily rely on fast and energy-efficient inference of deep neural network (DNN) models when deployed on robots. To enhance inference performance, distributed inference has emerged as a promising approach, parallelizing inference across multiple powerful GPU d…
▽ More
The rapid advancements in machine learning techniques have led to significant achievements in various real-world robotic tasks. These tasks heavily rely on fast and energy-efficient inference of deep neural network (DNN) models when deployed on robots. To enhance inference performance, distributed inference has emerged as a promising approach, parallelizing inference across multiple powerful GPU devices in modern data centers using techniques such as data parallelism, tensor parallelism, and pipeline parallelism. However, when deployed on real-world robots, existing parallel methods fail to provide low inference latency and meet the energy requirements due to the limited bandwidth of robotic IoT. We present Hybrid-Parallel, a high-performance distributed inference system optimized for robotic IoT. Hybrid-Parallel employs a fine-grained approach to parallelize inference at the granularity of local operators within DNN layers (i.e., operators that can be computed independently with the partial input, such as the convolution kernel in the convolution layer). By doing so, Hybrid-Parallel enables different operators of different layers to be computed and transmitted concurrently, and overlap the computation and transmission phases within the same inference task. The evaluation demonstrate that Hybrid-Parallel reduces inference time by 14.9% ~41.1% and energy consumption per inference by up to 35.3% compared to the state-of-the-art baselines.
△ Less
Submitted 29 May, 2024;
originally announced May 2024.
-
Transfer Learning Under High-Dimensional Graph Convolutional Regression Model for Node Classification
Authors:
Jiachen Chen,
Danyang Huang,
Liyuan Wang,
Kathryn L. Lunetta,
Debarghya Mukherjee,
Huimin Cheng
Abstract:
Node classification is a fundamental task, but obtaining node classification labels can be challenging and expensive in many real-world scenarios. Transfer learning has emerged as a promising solution to address this challenge by leveraging knowledge from source domains to enhance learning in a target domain. Existing transfer learning methods for node classification primarily focus on integrating…
▽ More
Node classification is a fundamental task, but obtaining node classification labels can be challenging and expensive in many real-world scenarios. Transfer learning has emerged as a promising solution to address this challenge by leveraging knowledge from source domains to enhance learning in a target domain. Existing transfer learning methods for node classification primarily focus on integrating Graph Convolutional Networks (GCNs) with various transfer learning techniques. While these approaches have shown promising results, they often suffer from a lack of theoretical guarantees, restrictive conditions, and high sensitivity to hyperparameter choices. To overcome these limitations, we propose a Graph Convolutional Multinomial Logistic Regression (GCR) model and a transfer learning method based on the GCR model, called Trans-GCR. We provide theoretical guarantees of the estimate obtained under GCR model in high-dimensional settings. Moreover, Trans-GCR demonstrates superior empirical performance, has a low computational cost, and requires fewer hyperparameters than existing methods.
△ Less
Submitted 26 May, 2024;
originally announced May 2024.
-
Luban: Building Open-Ended Creative Agents via Autonomous Embodied Verification
Authors:
Yuxuan Guo,
Shaohui Peng,
Jiaming Guo,
Di Huang,
Xishan Zhang,
Rui Zhang,
Yifan Hao,
Ling Li,
Zikang Tian,
Mingju Gao,
Yutai Li,
Yiming Gan,
Shuai Liang,
Zihao Zhang,
Zidong Du,
Qi Guo,
Xing Hu,
Yunji Chen
Abstract:
Building open agents has always been the ultimate goal in AI research, and creative agents are the more enticing. Existing LLM agents excel at long-horizon tasks with well-defined goals (e.g., `mine diamonds' in Minecraft). However, they encounter difficulties on creative tasks with open goals and abstract criteria due to the inability to bridge the gap between them, thus lacking feedback for self…
▽ More
Building open agents has always been the ultimate goal in AI research, and creative agents are the more enticing. Existing LLM agents excel at long-horizon tasks with well-defined goals (e.g., `mine diamonds' in Minecraft). However, they encounter difficulties on creative tasks with open goals and abstract criteria due to the inability to bridge the gap between them, thus lacking feedback for self-improvement in solving the task. In this work, we introduce autonomous embodied verification techniques for agents to fill the gap, laying the groundwork for creative tasks. Specifically, we propose the Luban agent target creative building tasks in Minecraft, which equips with two-level autonomous embodied verification inspired by human design practices: (1) visual verification of 3D structural speculates, which comes from agent synthesized CAD modeling programs; (2) pragmatic verification of the creation by generating and verifying environment-relevant functionality programs based on the abstract criteria. Extensive multi-dimensional human studies and Elo ratings show that the Luban completes diverse creative building tasks in our proposed benchmark and outperforms other baselines ($33\%$ to $100\%$) in both visualization and pragmatism. Additional demos on the real-world robotic arm show the creation potential of the Luban in the physical world.
△ Less
Submitted 24 May, 2024;
originally announced May 2024.
-
SOAP: Enhancing Efficiency of Generated Code via Self-Optimization
Authors:
Dong Huang,
Jianbo Dai,
Han Weng,
Puzhen Wu,
Yuhao Qing,
Jie M. Zhang,
Heming Cui,
Zhijiang Guo
Abstract:
Large language models (LLMs) have shown remarkable progress in code generation, but their generated code often suffers from inefficiency, resulting in longer execution times and higher memory consumption. To address this issue, we propose Self Optimization based on OverheAd Profile (SOAP), a self-optimization framework that utilizes execution overhead profiles to improve the efficiency of LLM-gene…
▽ More
Large language models (LLMs) have shown remarkable progress in code generation, but their generated code often suffers from inefficiency, resulting in longer execution times and higher memory consumption. To address this issue, we propose Self Optimization based on OverheAd Profile (SOAP), a self-optimization framework that utilizes execution overhead profiles to improve the efficiency of LLM-generated code. SOAP first generates code using an LLM, then executes it locally to capture execution time and memory usage profiles. These profiles are fed back to the LLM, which then revises the code to reduce overhead. To evaluate the effectiveness of SOAP, we conduct extensive experiments on the EffiBench, HumanEval, and MBPP with 16 open-source and 6 closed-source models. Our evaluation results demonstrate that through iterative self-optimization, SOAP significantly enhances the efficiency of LLM-generated code. For example, the execution time (ET) of StarCoder2-15B for the EffiBench decreases from 0.93 (s) to 0.12 (s) which reduces 87.1% execution time requirement compared with the initial code. The total memory usage (TMU) of StarCoder2-15B also decreases from 22.02 (Mb*s) to 2.03 (Mb*s), which decreases 90.8% total memory consumption during the execution process. The source code of SOAP was released in https://github.com/huangd1999/SOAP.
△ Less
Submitted 23 May, 2024;
originally announced May 2024.
-
The Data Acquisition System of the LZ Dark Matter Detector: FADR
Authors:
J. Aalbers,
D. S. Akerib,
A. K. Al Musalhi,
F. Alder,
C. S. Amarasinghe,
A. Ames,
T. J. Anderson,
N. Angelides,
H. M. Araújo,
J. E. Armstrong,
M. Arthurs,
A. Baker,
S. Balashov,
J. Bang,
E. E. Barillier,
J. W. Bargemann,
K. Beattie,
T. Benson,
A. Bhatti,
A. Biekert,
T. P. Biesiadzinski,
H. J. Birch,
E. Bishop,
G. M. Blockinger,
B. Boxer
, et al. (191 additional authors not shown)
Abstract:
The Data Acquisition System (DAQ) for the LUX-ZEPLIN (LZ) dark matter detector is described. The signals from 745 PMTs, distributed across three subsystems, are sampled with 100-MHz 32-channel digitizers (DDC-32s). A basic waveform analysis is carried out on the on-board Field Programmable Gate Arrays (FPGAs) to extract information about the observed scintillation and electroluminescence signals.…
▽ More
The Data Acquisition System (DAQ) for the LUX-ZEPLIN (LZ) dark matter detector is described. The signals from 745 PMTs, distributed across three subsystems, are sampled with 100-MHz 32-channel digitizers (DDC-32s). A basic waveform analysis is carried out on the on-board Field Programmable Gate Arrays (FPGAs) to extract information about the observed scintillation and electroluminescence signals. This information is used to determine if the digitized waveforms should be preserved for offline analysis.
The system is designed around the Kintex-7 FPGA. In addition to digitizing the PMT signals and providing basic event selection in real time, the flexibility provided by the use of FPGAs allows us to monitor the performance of the detector and the DAQ in parallel to normal data acquisition.
The hardware and software/firmware of this FPGA-based Architecture for Data acquisition and Realtime monitoring (FADR) are discussed and performance measurements are described.
△ Less
Submitted 12 July, 2024; v1 submitted 23 May, 2024;
originally announced May 2024.
-
MAGIC: Map-Guided Few-Shot Audio-Visual Acoustics Modeling
Authors:
Diwei Huang,
Kunyang Lin,
Peihao Chen,
Qing Du,
Mingkui Tan
Abstract:
Few-shot audio-visual acoustics modeling seeks to synthesize the room impulse response in arbitrary locations with few-shot observations. To sufficiently exploit the provided few-shot data for accurate acoustic modeling, we present a *map-guided* framework by constructing acoustic-related visual semantic feature maps of the scenes. Visual features preserve semantic details related to sound and map…
▽ More
Few-shot audio-visual acoustics modeling seeks to synthesize the room impulse response in arbitrary locations with few-shot observations. To sufficiently exploit the provided few-shot data for accurate acoustic modeling, we present a *map-guided* framework by constructing acoustic-related visual semantic feature maps of the scenes. Visual features preserve semantic details related to sound and maps provide explicit structural regularities of sound propagation, which are valuable for modeling environment acoustics. We thus extract pixel-wise semantic features derived from observations and project them into a top-down map, namely the **observation semantic map**. This map contains the relative positional information among points and the semantic feature information associated with each point. Yet, limited information extracted by few-shot observations on the map is not sufficient for understanding and modeling the whole scene. We address the challenge by generating a **scene semantic map** via diffusing features and anticipating the observation semantic map. The scene semantic map then interacts with echo encoding by a transformer-based encoder-decoder to predict RIR for arbitrary speaker-listener query pairs. Extensive experiments on Matterport3D and Replica dataset verify the efficacy of our framework.
△ Less
Submitted 22 May, 2024;
originally announced May 2024.
-
Estimating the Impact of Social Distance Policy in Mitigating COVID-19 Spread with Factor-Based Imputation Approach
Authors:
Difang Huang,
Ying Liang,
Boyao Wu,
Yanyi Ye
Abstract:
We identify the effectiveness of social distancing policies in reducing the transmission of the COVID-19 spread. We build a model that measures the relative frequency and geographic distribution of the virus growth rate and provides hypothetical infection distribution in the states that enacted the social distancing policies, where we control time-varying, observed and unobserved, state-level hete…
▽ More
We identify the effectiveness of social distancing policies in reducing the transmission of the COVID-19 spread. We build a model that measures the relative frequency and geographic distribution of the virus growth rate and provides hypothetical infection distribution in the states that enacted the social distancing policies, where we control time-varying, observed and unobserved, state-level heterogeneities. Using panel data on infection and deaths in all US states from February 20 to April 20, 2020, we find that stay-at-home orders and other types of social distancing policies significantly reduced the growth rate of infection and deaths. We show that the effects are time-varying and range from the weakest at the beginning of policy intervention to the strongest by the end of our sample period. We also found that social distancing policies were more effective in states with higher income, better education, more white people, more democratic voters, and higher CNN viewership.
△ Less
Submitted 20 May, 2024;
originally announced May 2024.
-
Data quality control system and long-term performance monitor of the LHAASO-KM2A
Authors:
Zhen Cao,
F. Aharonian,
Axikegu,
Y. X. Bai,
Y. W. Bao,
D. Bastieri,
X. J. Bi,
Y. J. Bi,
W. Bian,
A. V. Bukevich,
Q. Cao,
W. Y. Cao,
Zhe Cao,
J. Chang,
J. F. Chang,
A. M. Chen,
E. S. Chen,
H. X. Chen,
Liang Chen,
Lin Chen,
Long Chen,
M. J. Chen,
M. L. Chen,
Q. H. Chen,
S. Chen
, et al. (263 additional authors not shown)
Abstract:
The KM2A is the largest sub-array of the Large High Altitude Air Shower Observatory (LHAASO). It consists of 5216 electromagnetic particle detectors (EDs) and 1188 muon detectors (MDs). The data recorded by the EDs and MDs are used to reconstruct primary information of cosmic ray and gamma-ray showers. This information is used for physical analysis in gamma-ray astronomy and cosmic ray physics. To…
▽ More
The KM2A is the largest sub-array of the Large High Altitude Air Shower Observatory (LHAASO). It consists of 5216 electromagnetic particle detectors (EDs) and 1188 muon detectors (MDs). The data recorded by the EDs and MDs are used to reconstruct primary information of cosmic ray and gamma-ray showers. This information is used for physical analysis in gamma-ray astronomy and cosmic ray physics. To ensure the reliability of the LHAASO-KM2A data, a three-level quality control system has been established. It is used to monitor the status of detector units, stability of reconstructed parameters and the performance of the array based on observations of the Crab Nebula and Moon shadow. This paper will introduce the control system and its application on the LHAASO-KM2A data collected from August 2021 to July 2023. During this period, the pointing and angular resolution of the array were stable. From the observations of the Moon shadow and Crab Nebula, the results achieved using the two methods are consistent with each other. According to the observation of the Crab Nebula at energies from 25 TeV to 100 TeV, the time averaged pointing errors are estimated to be $-0.003^{\circ} \pm 0.005^{\circ}$ and $0.001^{\circ} \pm 0.006^{\circ}$ in the R.A. and Dec directions, respectively.
△ Less
Submitted 13 June, 2024; v1 submitted 20 May, 2024;
originally announced May 2024.
-
Knowledge Graph Pruning for Recommendation
Authors:
Fake Lin,
Xi Zhu,
Ziwei Zhao,
Deqiang Huang,
Yu Yu,
Xueying Li,
Zhi Zheng,
Tong Xu,
Enhong Chen
Abstract:
Recent years have witnessed the prosperity of knowledge graph based recommendation system (KGRS), which enriches the representation of users, items, and entities by structural knowledge with striking improvement. Nevertheless, its unaffordable computational cost still limits researchers from exploring more sophisticated models. We observe that the bottleneck for training efficiency arises from the…
▽ More
Recent years have witnessed the prosperity of knowledge graph based recommendation system (KGRS), which enriches the representation of users, items, and entities by structural knowledge with striking improvement. Nevertheless, its unaffordable computational cost still limits researchers from exploring more sophisticated models. We observe that the bottleneck for training efficiency arises from the knowledge graph, which is plagued by the well-known issue of knowledge explosion. Recently, some works have attempted to slim the inflated KG via summarization techniques. However, these summarized nodes may ignore the collaborative signals and deviate from the facts that nodes in knowledge graph represent symbolic abstractions of entities from the real-world. To this end, in this paper, we propose a novel approach called KGTrimmer for knowledge graph pruning tailored for recommendation, to remove the unessential nodes while minimizing performance degradation. Specifically, we design an importance evaluator from a dual-view perspective. For the collective view, we embrace the idea of collective intelligence by extracting community consensus based on abundant collaborative signals, i.e. nodes are considered important if they attract attention of numerous users. For the holistic view, we learn a global mask to identify the valueless nodes from their inherent properties or overall popularity. Next, we build an end-to-end importance-aware graph neural network, which injects filtered knowledge to enhance the distillation of valuable user-item collaborative signals. Ultimately, we generate a pruned knowledge graph with lightweight, stable, and robust properties to facilitate the following-up recommendation task. Extensive experiments are conducted on three publicly available datasets to prove the effectiveness and generalization ability of KGTrimmer.
△ Less
Submitted 9 July, 2024; v1 submitted 19 May, 2024;
originally announced May 2024.
-
Cross-Domain Knowledge Distillation for Low-Resolution Human Pose Estimation
Authors:
Zejun Gu,
Zhong-Qiu Zhao,
Henghui Ding,
Hao Shen,
Zhao Zhang,
De-Shuang Huang
Abstract:
In practical applications of human pose estimation, low-resolution inputs frequently occur, and existing state-of-the-art models perform poorly with low-resolution images. This work focuses on boosting the performance of low-resolution models by distilling knowledge from a high-resolution model. However, we face the challenge of feature size mismatch and class number mismatch when applying knowled…
▽ More
In practical applications of human pose estimation, low-resolution inputs frequently occur, and existing state-of-the-art models perform poorly with low-resolution images. This work focuses on boosting the performance of low-resolution models by distilling knowledge from a high-resolution model. However, we face the challenge of feature size mismatch and class number mismatch when applying knowledge distillation to networks with different input resolutions. To address this issue, we propose a novel cross-domain knowledge distillation (CDKD) framework. In this framework, we construct a scale-adaptive projector ensemble (SAPE) module to spatially align feature maps between models of varying input resolutions. It adopts a projector ensemble to map low-resolution features into multiple common spaces and adaptively merges them based on multi-scale information to match high-resolution features. Additionally, we construct a cross-class alignment (CCA) module to solve the problem of the mismatch of class numbers. By combining an easy-to-hard training (ETHT) strategy, the CCA module further enhances the distillation performance. The effectiveness and efficiency of our approach are demonstrated by extensive experiments on two common benchmark datasets: MPII and COCO. The code is made available in supplementary material.
△ Less
Submitted 19 May, 2024;
originally announced May 2024.
-
Know in AdVance: Linear-Complexity Forecasting of Ad Campaign Performance with Evolving User Interest
Authors:
XiaoYu Wang,
YongHui Guo,
Hui Sheng,
Peili Lv,
Chi Zhou,
Wei Huang,
ShiQin Ta,
Dongbo Huang,
XiuJin Yang,
Lan Xu,
Hao Zhou,
Yusheng Ji
Abstract:
Real-time Bidding (RTB) advertisers wish to \textit{know in advance} the expected cost and yield of ad campaigns to avoid trial-and-error expenses. However, Campaign Performance Forecasting (CPF), a sequence modeling task involving tens of thousands of ad auctions, poses challenges of evolving user interest, auction representation, and long context, making coarse-grained and static-modeling method…
▽ More
Real-time Bidding (RTB) advertisers wish to \textit{know in advance} the expected cost and yield of ad campaigns to avoid trial-and-error expenses. However, Campaign Performance Forecasting (CPF), a sequence modeling task involving tens of thousands of ad auctions, poses challenges of evolving user interest, auction representation, and long context, making coarse-grained and static-modeling methods sub-optimal. We propose \textit{AdVance}, a time-aware framework that integrates local auction-level and global campaign-level modeling. User preference and fatigue are disentangled using a time-positioned sequence of clicked items and a concise vector of all displayed items. Cross-attention, conditioned on the fatigue vector, captures the dynamics of user interest toward each candidate ad. Bidders compete with each other, presenting a complete graph similar to the self-attention mechanism. Hence, we employ a Transformer Encoder to compress each auction into embedding by solving auxiliary tasks. These sequential embeddings are then summarized by a conditional state space model (SSM) to comprehend long-range dependencies while maintaining global linear complexity. Considering the irregular time intervals between auctions, we make SSM's parameters dependent on the current auction embedding and the time interval. We further condition SSM's global predictions on the accumulation of local results. Extensive evaluations and ablation studies demonstrate its superiority over state-of-the-art methods. AdVance has been deployed on the Tencent Advertising platform, and A/B tests show a remarkable 4.5\% uplift in Average Revenue per User (ARPU).
△ Less
Submitted 17 May, 2024;
originally announced May 2024.
-
Learning Coarse-Grained Dynamics on Graph
Authors:
Yin Yu,
John Harlim,
Daning Huang,
Yan Li
Abstract:
We consider a Graph Neural Network (GNN) non-Markovian modeling framework to identify coarse-grained dynamical systems on graphs. Our main idea is to systematically determine the GNN architecture by inspecting how the leading term of the Mori-Zwanzig memory term depends on the coarse-grained interaction coefficients that encode the graph topology. Based on this analysis, we found that the appropri…
▽ More
We consider a Graph Neural Network (GNN) non-Markovian modeling framework to identify coarse-grained dynamical systems on graphs. Our main idea is to systematically determine the GNN architecture by inspecting how the leading term of the Mori-Zwanzig memory term depends on the coarse-grained interaction coefficients that encode the graph topology. Based on this analysis, we found that the appropriate GNN architecture that will account for $K$-hop dynamical interactions has to employ a Message Passing (MP) mechanism with at least $2K$ steps. We also deduce that the memory length required for an accurate closure model decreases as a function of the interaction strength under the assumption that the interaction strength exhibits a power law that decays as a function of the hop distance. Supporting numerical demonstrations on two examples, a heterogeneous Kuramoto oscillator model and a power system, suggest that the proposed GNN architecture can predict the coarse-grained dynamics under fixed and time-varying graph topologies.
△ Less
Submitted 15 May, 2024;
originally announced May 2024.
-
Themis: Automatic and Efficient Deep Learning System Testing with Strong Fault Detection Capability
Authors:
Dong Huang,
Xiaofei Xie,
Heming Cui
Abstract:
Deep Learning Systems (DLSs) have been widely applied in safety-critical tasks such as autopilot. However, when a perturbed input is fed into a DLS for inference, the DLS often has incorrect outputs (i.e., faults). DLS testing techniques (e.g., DeepXplore) detect such faults by generating perturbed inputs to explore data flows that induce faults. Since a DLS often has infinitely many data flows, e…
▽ More
Deep Learning Systems (DLSs) have been widely applied in safety-critical tasks such as autopilot. However, when a perturbed input is fed into a DLS for inference, the DLS often has incorrect outputs (i.e., faults). DLS testing techniques (e.g., DeepXplore) detect such faults by generating perturbed inputs to explore data flows that induce faults. Since a DLS often has infinitely many data flows, existing techniques require developers to manually specify a set of activation values in a DLS's neurons for exploring fault-inducing data flows. Unfortunately, recent studies show that such manual effort is tedious and can detect only a tiny proportion of fault-inducing data flows.
In this paper, we present Themis, the first automatic DLS testing system, which attains strong fault detection capability by ensuring a full coverage of fault-inducing data flows at a high probability. Themis carries a new workflow for automatically and systematically revealing data flows whose internal neurons' outputs vary substantially when the inputs are slightly perturbed, as these data flows are likely fault-inducing. We evaluated Themis on ten different DLSs and found that on average the number of faults detected by Themis was 3.78X more than four notable DLS testing techniques. By retraining all evaluated DLSs with the detected faults, Themis also increased (regained) these DLSs' accuracies on average 14.7X higher than all baselines.
△ Less
Submitted 24 May, 2024; v1 submitted 15 May, 2024;
originally announced May 2024.
-
Discovery of Very-high-energy Gamma-ray Emissions from the Low Luminosity AGN NGC 4278 by LHAASO
Authors:
Zhen Cao,
F. Aharonian,
Q. An,
Axikegu,
Y. X. Bai,
Y. W. Bao,
D. Bastieri,
X. J. Bi,
Y. J. Bi,
J. T. Cai,
Q. Cao,
W. Y. Cao,
Zhe Cao,
J. Chang,
J. F. Chang,
A. M. Chen,
E. S. Chen,
Liang Chen,
Lin Chen,
Long Chen,
M. J. Chen,
M. L. Chen,
Q. H. Chen,
S. H. Chen,
S. Z. Chen
, et al. (255 additional authors not shown)
Abstract:
The first source catalog of Large High Altitude Air Shower Observatory reported the detection of a very-high-energy gamma ray source, 1LHAASO J1219+2915. In this paper a further detailed study of the spectral and temporal behavior of this point-like source have been carried. The best-fit position of the TeV source ($\rm{RA}=185.05^{\circ}\pm0.04^{\circ}$, $\rm{Dec}=29.25^{\circ}\pm0.03^{\circ}$) i…
▽ More
The first source catalog of Large High Altitude Air Shower Observatory reported the detection of a very-high-energy gamma ray source, 1LHAASO J1219+2915. In this paper a further detailed study of the spectral and temporal behavior of this point-like source have been carried. The best-fit position of the TeV source ($\rm{RA}=185.05^{\circ}\pm0.04^{\circ}$, $\rm{Dec}=29.25^{\circ}\pm0.03^{\circ}$) is compatible with NGC 4278 within $\sim0.03$ degree. Variation analysis shows an indication of the variability at a few months level in the TeV band, which is consistent with low frequency observations. Based on these observations, we report the detection of TeV $γ$-ray emissions from this low-luminosity AGN NGC 4278. The observations by LHAASO-WCDA during active period has a significance level of 8.8\,$σ$ with best-fit photon spectral index $\varGamma=2.56\pm0.14$ and a flux $f_{1-10\,\rm{TeV}}=(7.0\pm1.1_{\rm{sta}}\pm0.35_{\rm{syst}})\times10^{-13}\,\rm{photons\,cm^{-2}\,s^{-1}}$, or approximately $5\%$ of the Crab Nebula. The discovery of VHE from NGC 4278 indicates that the compact, weak radio jet can efficiently accelerate particles and emit TeV photons.
△ Less
Submitted 13 May, 2024;
originally announced May 2024.