Showing 51–100 of 5,715 results for author: Kim, H

Contextual Sprint Classification in Soccer Based on Deep Learning

Authors: Hyunsung Kim, Gun-Hee Joe, Jinsung Yoon, Sang-Ki Ko

Abstract: The analysis of high-intensity runs (or sprints) in soccer has long been a topic of interest for sports science researchers and practitioners. In particular, recent studies suggested contextualizing sprints based on their tactical purposes to better understand the physical-tactical requirements of modern match-play. However, they have a limitation in scalability, as human experts have to manually… ▽ More The analysis of high-intensity runs (or sprints) in soccer has long been a topic of interest for sports science researchers and practitioners. In particular, recent studies suggested contextualizing sprints based on their tactical purposes to better understand the physical-tactical requirements of modern match-play. However, they have a limitation in scalability, as human experts have to manually classify hundreds of sprints for every match. To address this challenge, this paper proposes a deep learning framework for automatically classifying sprints in soccer into contextual categories. The proposed model covers the permutation-invariant and sequential nature of multi-agent trajectories in soccer by deploying Set Transformers and a bidirectional GRU. We train the model with category labels made through the collaboration of human annotators and a rule-based classifier. Experimental results show that our model classifies sprints in the test dataset into 15 categories with the accuracy of 77.65%, implying the potential of the proposed framework for facilitating the integrated analysis of soccer sprints at scale. △ Less

Submitted 21 June, 2024; originally announced June 2024.

Comments: Accepted at IJCAI 2024 Workshop on Intelligent Technologies for Precision Sports Science (IT4PSS 2024)

PreSto: An In-Storage Data Preprocessing System for Training Recommendation Models

Authors: Yunjae Lee, Hyeseong Kim, Minsoo Rhu

Abstract: Training recommendation systems (RecSys) faces several challenges as it requires the "data preprocessing" stage to preprocess an ample amount of raw data and feed them to the GPU for training in a seamless manner. To sustain high training throughput, state-of-the-art solutions reserve a large fleet of CPU servers for preprocessing which incurs substantial deployment cost and power consumption. Our… ▽ More Training recommendation systems (RecSys) faces several challenges as it requires the "data preprocessing" stage to preprocess an ample amount of raw data and feed them to the GPU for training in a seamless manner. To sustain high training throughput, state-of-the-art solutions reserve a large fleet of CPU servers for preprocessing which incurs substantial deployment cost and power consumption. Our characterization reveals that prior CPU-centric preprocessing is bottlenecked on feature generation and feature normalization operations as it fails to reap out the abundant inter-/intra-feature parallelism in RecSys preprocessing. PreSto is a storage-centric preprocessing system leveraging In-Storage Processing (ISP), which offloads the bottlenecked preprocessing operations to our ISP units. We show that PreSto outperforms the baseline CPU-centric system with a $9.6\times$ speedup in end-to-end preprocessing time, $4.3\times$ enhancement in cost-efficiency, and $11.3\times$ improvement in energyefficiency on average for production-scale RecSys preprocessing. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Journal ref: Published at 51th IEEE/ACM International Symposium on Computer Architecture (ISCA-51), 2024

CONMOD: Controllable Neural Frame-based Modulation Effects

Authors: Gyubin Lee, Hounsu Kim, Junwon Lee, Juhan Nam

Abstract: Deep learning models have seen widespread use in modelling LFO-driven audio effects, such as phaser and flanger. Although existing neural architectures exhibit high-quality emulation of individual effects, they do not possess the capability to manipulate the output via control parameters. To address this issue, we introduce Controllable Neural Frame-based Modulation Effects (CONMOD), a single blac… ▽ More Deep learning models have seen widespread use in modelling LFO-driven audio effects, such as phaser and flanger. Although existing neural architectures exhibit high-quality emulation of individual effects, they do not possess the capability to manipulate the output via control parameters. To address this issue, we introduce Controllable Neural Frame-based Modulation Effects (CONMOD), a single black-box model which emulates various LFO-driven effects in a frame-wise manner, offering control over LFO frequency and feedback parameters. Additionally, the model is capable of learning the continuous embedding space of two distinct phaser effects, enabling us to steer between effects and achieve creative outputs. Our model outperforms previous work while possessing both controllability and universality, presenting opportunities to enhance creativity in modern LFO-driven audio effects. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Attention-aware Post-training Quantization without Backpropagation

Authors: Junhan Kim, Ho-young Kim, Eulrang Cho, Chungman Lee, Joonyoung Kim, Yongkweon Jeon

Abstract: Quantization is a promising solution for deploying large-scale language models (LLMs) on resource-constrained devices. Existing quantization approaches, however, rely on gradient-based optimization, regardless of it being post-training quantization (PTQ) or quantization-aware training (QAT), which becomes problematic for hyper-scale LLMs with billions of parameters. This overhead can be alleviated… ▽ More Quantization is a promising solution for deploying large-scale language models (LLMs) on resource-constrained devices. Existing quantization approaches, however, rely on gradient-based optimization, regardless of it being post-training quantization (PTQ) or quantization-aware training (QAT), which becomes problematic for hyper-scale LLMs with billions of parameters. This overhead can be alleviated via recently proposed backpropagation-free PTQ methods; however, their performance is somewhat limited by their lack of consideration of inter-layer dependencies. In this paper, we thus propose a novel PTQ algorithm that considers inter-layer dependencies without relying on backpropagation. The fundamental concept involved is the development of attention-aware Hessian matrices, which facilitates the consideration of inter-layer dependencies within the attention module. Extensive experiments demonstrate that the proposed algorithm significantly outperforms conventional PTQ methods, particularly for low bit-widths. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Comments: 20 pages, under review

Sound event detection based on auxiliary decoder and maximum probability aggregation for DCASE Challenge 2024 Task 4

Authors: Sang Won Son, Jongyeon Park, Hong Kook Kim, Sulaiman Vesal, Jeong Eun Lim

Abstract: In this report, we propose three novel methods for developing a sound event detection (SED) model for the DCASE 2024 Challenge Task 4. First, we propose an auxiliary decoder attached to the final convolutional block to improve feature extraction capabilities while reducing dependency on embeddings from pre-trained large models. The proposed auxiliary decoder operates independently from the main de… ▽ More In this report, we propose three novel methods for developing a sound event detection (SED) model for the DCASE 2024 Challenge Task 4. First, we propose an auxiliary decoder attached to the final convolutional block to improve feature extraction capabilities while reducing dependency on embeddings from pre-trained large models. The proposed auxiliary decoder operates independently from the main decoder, enhancing performance of the convolutional block during the initial training stages by assigning a different weight strategy between main and auxiliary decoder losses. Next, to address the time interval issue between the DESED and MAESTRO datasets, we propose maximum probability aggregation (MPA) during the training step. The proposed MPA method enables the model's output to be aligned with soft labels of 1 s in the MAESTRO dataset. Finally, we propose a multi-channel input feature that employs various versions of logmel and MFCC features to generate time-frequency pattern. The experimental results demonstrate the efficacy of these proposed methods in a view of improving SED performance by achieving a balanced enhancement across different datasets and label types. Ultimately, this approach presents a significant step forward in developing more robust and flexible SED models △ Less

Submitted 24 June, 2024; v1 submitted 17 June, 2024; originally announced June 2024.

Comments: DCASE 2024 challenge Task4, 4 pages

Advancing Cross-Domain Generalizability in Face Anti-Spoofing: Insights, Design, and Metrics

Authors: Hyojin Kim, Jiyoon Lee, Yonghyun Jeong, Haneol Jang, YoungJoon Yoo

Abstract: This paper presents a novel perspective for enhancing anti-spoofing performance in zero-shot data domain generalization. Unlike traditional image classification tasks, face anti-spoofing datasets display unique generalization characteristics, necessitating novel zero-shot data domain generalization. One step forward to the previous frame-wise spoofing prediction, we introduce a nuanced metric calc… ▽ More This paper presents a novel perspective for enhancing anti-spoofing performance in zero-shot data domain generalization. Unlike traditional image classification tasks, face anti-spoofing datasets display unique generalization characteristics, necessitating novel zero-shot data domain generalization. One step forward to the previous frame-wise spoofing prediction, we introduce a nuanced metric calculation that aggregates frame-level probabilities for a video-wise prediction, to tackle the gap between the reported frame-wise accuracy and instability in real-world use-case. This approach enables the quantification of bias and variance in model predictions, offering a more refined analysis of model generalization. Our investigation reveals that simply scaling up the backbone of models does not inherently improve the mentioned instability, leading us to propose an ensembled backbone method from a Bayesian perspective. The probabilistically ensembled backbone both improves model robustness measured from the proposed metric and spoofing accuracy, and also leverages the advantages of measuring uncertainty, allowing for enhanced sampling during training that contributes to model generalization across new datasets. We evaluate the proposed method from the benchmark OMIC dataset and also the public CelebA-Spoof and SiW-Mv2. Our final model outperforms existing state-of-the-art methods across the datasets, showcasing advancements in Bias, Variance, HTER, and AUC metrics. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Comments: 10 pages with 4 figures, Accepted by CVPRW 2024

Elaborating Higgs to dimuon decay from gluon fusion by decorrelation and jet substructure

Authors: Subin Han, Hyung Do Kim

Abstract: Discovery of the Higgs boson decay to dimuon is anticipated soon based on the current evidence. Precise categorization of the events without affecting the invariant mass shape is crucial in the analysis. Decorrelation of the invariant mass and the output of discriminators (the score of discriminators) is essential for consistent and precise analysis. In this paper we use distance correlation as th… ▽ More Discovery of the Higgs boson decay to dimuon is anticipated soon based on the current evidence. Precise categorization of the events without affecting the invariant mass shape is crucial in the analysis. Decorrelation of the invariant mass and the output of discriminators (the score of discriminators) is essential for consistent and precise analysis. In this paper we use distance correlation as the additional loss function to achieve the decorrelation for discriminators and examine various analysis methods. The analyses with and without jet substructure variables are presented. Adding jet substructure variables considerably improves the significance of the Higgs to dimuon signal from gluon fusion. △ Less

Submitted 17 June, 2024; originally announced June 2024.

Comments: 25 pages, 7 figures, 7 tables

Non-unitary Coupled Cluster Enabled by Mid-circuit Measurements on Quantum Computers

Authors: Alexandre Fleury, James Brown, Erika Lloyd, Maritza Hernandez, Isaac H. Kim

Abstract: Many quantum algorithms rely on a quality initial state for optimal performance. Preparing an initial state for specific applications can considerably reduce the cost of probabilistic algorithms such as the well studied quantum phase estimation (QPE). Fortunately, in the application space of quantum chemistry, generating approximate wave functions for molecular systems is well studied, and quantum… ▽ More Many quantum algorithms rely on a quality initial state for optimal performance. Preparing an initial state for specific applications can considerably reduce the cost of probabilistic algorithms such as the well studied quantum phase estimation (QPE). Fortunately, in the application space of quantum chemistry, generating approximate wave functions for molecular systems is well studied, and quantum computing algorithms stand to benefit from importing these classical methods directly into a quantum circuit. In this work, we propose a state preparation method based on coupled cluster (CC) theory, which is a pillar of quantum chemistry on classical computers, by incorporating mid-circuit measurements into the circuit construction. Currently, the most well studied state preparation method for quantum chemistry on quantum computers is the variational quantum eigensolver (VQE) with a unitary-CC with single- and double-electron excitation terms (UCCSD) ansatz whose operations are limited to unitary gates. We verify the accuracy of our state preparation protocol using mid-circuit measurements by performing energy evaluation and state overlap computation for a set of small chemical systems. We further demonstrate that our approach leads to a reduction of the classical computation overhead, and the number of CNOT and T gates by 28% and 57% on average when compared against the standard VQE-UCCSD protocol. △ Less

Submitted 28 June, 2024; v1 submitted 17 June, 2024; originally announced June 2024.

Comments: 26 pages, 6 figures; title changed, references added

Non-freeness of parabolic two-generator groups

Authors: Philip Choi, Kyeonghee Jo, Hyuk Kim, Junho Lee

Abstract: A complex number $λ$ is said to be non-free if the subgroup of $SL(2,\bc)$ generated by $$X=\begin{pmatrix} 1& 1\\ 0 & 1 \end{pmatrix} \,\, \text{and}\,\,\,Y_λ=\begin{pmatrix} 1& 0\\ λ& 1 \end{pmatrix}$$ is not a free group of rank 2. In this case the number $λ$ is called a relation number, and it has been a long standing problem to determine the relation numbers. In this paper, we characteriz… ▽ More A complex number $λ$ is said to be non-free if the subgroup of $SL(2,\bc)$ generated by $$X=\begin{pmatrix} 1& 1\\ 0 & 1 \end{pmatrix} \,\, \text{and}\,\,\,Y_λ=\begin{pmatrix} 1& 0\\ λ& 1 \end{pmatrix}$$ is not a free group of rank 2. In this case the number $λ$ is called a relation number, and it has been a long standing problem to determine the relation numbers. In this paper, we characterize the relation numbers by establishing the equivalence between $λ$ being a relation number and $u:=\sqrt{- λ}$ being a root of a `generalized Chebyshev polynomial'. The generalized Chebyshev polynomials of degree $k$ are given by a sequence of $k$ integers $(n_1, n_2,\cdots, n_k)$ using the usual recursive formula, and thereby can be studied systematically using continuants and continued fractions. Such formulation, then, enables us to prove that, the question whether a given number $λ$ is a relation number of $u$-degree $k$ can be answered by checking only finitely many generalized Chebyshev polynomials. Based on these theorems, we design an algorithm deciding any given number is a relation number with minimal degree $k$. With its computer implementation we provide a few sample examples, with a particular emphasis on the well known conjecture that every rational number in the interval $(-4, 4)$ is a relation number. △ Less

Submitted 17 June, 2024; originally announced June 2024.

Comments: 43 pages, 2 figures

MSC Class: 20E05; 11B39; 11J70; 30F35; 30F40

Semi-Supervised Domain Adaptation Using Target-Oriented Domain Augmentation for 3D Object Detection

Authors: Yecheol Kim, Junho Lee, Changsoo Park, Hyoung won Kim, Inho Lim, Christopher Chang, Jun Won Choi

Abstract: 3D object detection is crucial for applications like autonomous driving and robotics. However, in real-world environments, variations in sensor data distribution due to sensor upgrades, weather changes, and geographic differences can adversely affect detection performance. Semi-Supervised Domain Adaptation (SSDA) aims to mitigate these challenges by transferring knowledge from a source domain, abu… ▽ More 3D object detection is crucial for applications like autonomous driving and robotics. However, in real-world environments, variations in sensor data distribution due to sensor upgrades, weather changes, and geographic differences can adversely affect detection performance. Semi-Supervised Domain Adaptation (SSDA) aims to mitigate these challenges by transferring knowledge from a source domain, abundant in labeled data, to a target domain where labels are scarce. This paper presents a new SSDA method referred to as Target-Oriented Domain Augmentation (TODA) specifically tailored for LiDAR-based 3D object detection. TODA efficiently utilizes all available data, including labeled data in the source domain, and both labeled data and unlabeled data in the target domain to enhance domain adaptation performance. TODA consists of two stages: TargetMix and AdvMix. TargetMix employs mixing augmentation accounting for LiDAR sensor characteristics to facilitate feature alignment between the source-domain and target-domain. AdvMix applies point-wise adversarial augmentation with mixing augmentation, which perturbs the unlabeled data to align the features within both labeled and unlabeled data in the target domain. Our experiments conducted on the challenging domain adaptation tasks demonstrate that TODA outperforms existing domain adaptation techniques designed for 3D object detection by significant margins. The code is available at: https://github.com/rasd3/TODA. △ Less

Submitted 17 June, 2024; originally announced June 2024.

Comments: Accepted to IEEE Transactions on Intelligent Vehicles (T-IV). The code is available at: https://github.com/rasd3/TODA

Performance Improvement of Language-Queried Audio Source Separation Based on Caption Augmentation From Large Language Models for DCASE Challenge 2024 Task 9

Authors: Do Hyun Lee, Yoonah Song, Hong Kook Kim

Abstract: We present a prompt-engineering-based text-augmentation approach applied to a language-queried audio source separation (LASS) task. To enhance the performance of LASS, the proposed approach utilizes large language models (LLMs) to generate multiple captions corresponding to each sentence of the training dataset. To this end, we first perform experiments to identify the most effective prompts for c… ▽ More We present a prompt-engineering-based text-augmentation approach applied to a language-queried audio source separation (LASS) task. To enhance the performance of LASS, the proposed approach utilizes large language models (LLMs) to generate multiple captions corresponding to each sentence of the training dataset. To this end, we first perform experiments to identify the most effective prompts for caption augmentation with a smaller number of captions. A LASS model trained with these augmented captions demonstrates improved performance on the DCASE 2024 Task 9 validation set compared to that trained without augmentation. This study highlights the effectiveness of LLM-based caption augmentation in advancing language-queried audio source separation. △ Less

Submitted 17 June, 2024; originally announced June 2024.

Comments: DCASE 2024 Challenge Task 9, 4 pages

SpoT-Mamba: Learning Long-Range Dependency on Spatio-Temporal Graphs with Selective State Spaces

Authors: Jinhyeok Choi, Heehyeon Kim, Minhyeong An, Joyce Jiyoung Whang

Abstract: Spatio-temporal graph (STG) forecasting is a critical task with extensive applications in the real world, including traffic and weather forecasting. Although several recent methods have been proposed to model complex dynamics in STGs, addressing long-range spatio-temporal dependencies remains a significant challenge, leading to limited performance gains. Inspired by a recently proposed state space… ▽ More Spatio-temporal graph (STG) forecasting is a critical task with extensive applications in the real world, including traffic and weather forecasting. Although several recent methods have been proposed to model complex dynamics in STGs, addressing long-range spatio-temporal dependencies remains a significant challenge, leading to limited performance gains. Inspired by a recently proposed state space model named Mamba, which has shown remarkable capability of capturing long-range dependency, we propose a new STG forecasting framework named SpoT-Mamba. SpoT-Mamba generates node embeddings by scanning various node-specific walk sequences. Based on the node embeddings, it conducts temporal scans to capture long-range spatio-temporal dependencies. Experimental results on the real-world traffic forecasting dataset demonstrate the effectiveness of SpoT-Mamba. △ Less

Submitted 17 June, 2024; originally announced June 2024.

Comments: 6 pages, 2 figures, 3 tables. Spatio-Temporal Reasoning and Learning (STRL) Workshop at the 33rd International Joint Conference on Artificial Intelligence (IJCAI 2024)

THEANINE: Revisiting Memory Management in Long-term Conversations with Timeline-augmented Response Generation

Authors: Seo Hyun Kim, Kai Tzu-iunn Ong, Taeyoon Kwon, Namyoung Kim, Keummin Ka, SeongHyeon Bae, Yohan Jo, Seung-won Hwang, Dongha Lee, Jinyoung Yeo

Abstract: Large language models (LLMs) are capable of processing lengthy dialogue histories during prolonged interaction with users without additional memory modules; however, their responses tend to overlook or incorrectly recall information from the past. In this paper, we revisit memory-augmented response generation in the era of LLMs. While prior work focuses on getting rid of outdated memories, we argu… ▽ More Large language models (LLMs) are capable of processing lengthy dialogue histories during prolonged interaction with users without additional memory modules; however, their responses tend to overlook or incorrectly recall information from the past. In this paper, we revisit memory-augmented response generation in the era of LLMs. While prior work focuses on getting rid of outdated memories, we argue that such memories can provide contextual cues that help dialogue systems understand the development of past events and, therefore, benefit response generation. We present Theanine, a framework that augments LLMs' response generation with memory timelines -- series of memories that demonstrate the development and causality of relevant past events. Along with Theanine, we introduce TeaFarm, a counterfactual-driven question-answering pipeline addressing the limitation of G-Eval in long-term conversations. Supplementary videos of our methods and the TeaBag dataset for TeaFarm evaluation are in https://theanine-693b0.web.app/. △ Less

Submitted 16 June, 2024; originally announced June 2024.

Comments: Under Review

Augmenting Biomedical Named Entity Recognition with General-domain Resources

Authors: Yu Yin, Hyunjae Kim, Xiao Xiao, Chih Hsuan Wei, Jaewoo Kang, Zhiyong Lu, Hua Xu, Meng Fang, Qingyu Chen

Abstract: Training a neural network-based biomedical named entity recognition (BioNER) model usually requires extensive and costly human annotations. While several studies have employed multi-task learning with multiple BioNER datasets to reduce human effort, this approach does not consistently yield performance improvements and may introduce label ambiguity in different biomedical corpora. We aim to tackle… ▽ More Training a neural network-based biomedical named entity recognition (BioNER) model usually requires extensive and costly human annotations. While several studies have employed multi-task learning with multiple BioNER datasets to reduce human effort, this approach does not consistently yield performance improvements and may introduce label ambiguity in different biomedical corpora. We aim to tackle those challenges through transfer learning from easily accessible resources with fewer concept overlaps with biomedical datasets. In this paper, we proposed GERBERA, a simple-yet-effective method that utilized a general-domain NER dataset for training. Specifically, we performed multi-task learning to train a pre-trained biomedical language model with both the target BioNER dataset and the general-domain dataset. Subsequently, we fine-tuned the models specifically for the BioNER dataset. We systematically evaluated GERBERA on five datasets of eight entity types, collectively consisting of 81,410 instances. Despite using fewer biomedical resources, our models demonstrated superior performance compared to baseline models trained with multiple additional BioNER datasets. Specifically, our models consistently outperformed the baselines in six out of eight entity types, achieving an average improvement of 0.9% over the best baseline performance across eight biomedical entity types sourced from five different corpora. Our method was especially effective in amplifying performance on BioNER datasets characterized by limited data, with a 4.7% improvement in F1 scores on the JNLPBA-RNA dataset. △ Less

Submitted 18 June, 2024; v1 submitted 15 June, 2024; originally announced June 2024.

Comments: We make data, codes, and models publicly available via https://github.com/qingyu-qc/bioner_gerbera

Lightweight Audio Segmentation for Long-form Speech Translation

Authors: Jaesong Lee, Soyoon Kim, Hanbyul Kim, Joon Son Chung

Abstract: Speech segmentation is an essential part of speech translation (ST) systems in real-world scenarios. Since most ST models are designed to process speech segments, long-form audio must be partitioned into shorter segments before translation. Recently, data-driven approaches for the speech segmentation task have been developed. Although the approaches improve overall translation quality, a performan… ▽ More Speech segmentation is an essential part of speech translation (ST) systems in real-world scenarios. Since most ST models are designed to process speech segments, long-form audio must be partitioned into shorter segments before translation. Recently, data-driven approaches for the speech segmentation task have been developed. Although the approaches improve overall translation quality, a performance gap exists due to a mismatch between the models and ST systems. In addition, the prior works require large self-supervised speech models, which consume significant computational resources. In this work, we propose a segmentation model that achieves better speech translation quality with a small model size. We propose an ASR-with-punctuation task as an effective pre-training strategy for the segmentation model. We also show that proper integration of the speech segmentation model into the underlying ST system is critical to improve overall translation quality at inference time. △ Less

Submitted 15 June, 2024; originally announced June 2024.

Comments: Accepted to Interspeech 2024

Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wild

Authors: Lingni Ma, Yuting Ye, Fangzhou Hong, Vladimir Guzov, Yifeng Jiang, Rowan Postyeni, Luis Pesqueira, Alexander Gamino, Vijay Baiyya, Hyo Jin Kim, Kevin Bailey, David Soriano Fosas, C. Karen Liu, Ziwei Liu, Jakob Engel, Renzo De Nardi, Richard Newcombe

Abstract: We introduce Nymeria - a large-scale, diverse, richly annotated human motion dataset collected in the wild with multiple multimodal egocentric devices. The dataset comes with a) full-body 3D motion ground truth; b) egocentric multimodal recordings from Project Aria devices with RGB, grayscale, eye-tracking cameras, IMUs, magnetometer, barometer, and microphones; and c) an additional "observer" dev… ▽ More We introduce Nymeria - a large-scale, diverse, richly annotated human motion dataset collected in the wild with multiple multimodal egocentric devices. The dataset comes with a) full-body 3D motion ground truth; b) egocentric multimodal recordings from Project Aria devices with RGB, grayscale, eye-tracking cameras, IMUs, magnetometer, barometer, and microphones; and c) an additional "observer" device providing a third-person viewpoint. We compute world-aligned 6DoF transformations for all sensors, across devices and capture sessions. The dataset also provides 3D scene point clouds and calibrated gaze estimation. We derive a protocol to annotate hierarchical language descriptions of in-context human motion, from fine-grain pose narrations, to atomic actions and activity summarization. To the best of our knowledge, the Nymeria dataset is the world largest in-the-wild collection of human motion with natural and diverse activities; first of its kind to provide synchronized and localized multi-device multimodal egocentric data; and the world largest dataset with motion-language descriptions. It contains 1200 recordings of 300 hours of daily activities from 264 participants across 50 locations, travelling a total of 399Km. The motion-language descriptions provide 310.5K sentences in 8.64M words from a vocabulary size of 6545. To demonstrate the potential of the dataset we define key research tasks for egocentric body tracking, motion synthesis, and action recognition and evaluate several state-of-the-art baseline algorithms. Data and code will be open-sourced. △ Less

Submitted 14 June, 2024; originally announced June 2024.

Projected background and sensitivity of AMoRE-II

Authors: A. Agrawal, V. V. Alenkov, P. Aryal, J. Beyer, B. Bhandari, R. S. Boiko, K. Boonin, O. Buzanov, C. R. Byeon, N. Chanthima, M. K. Cheoun, J. S. Choe, Seonho Choi, S. Choudhury, J. S. Chung, F. A. Danevich, M. Djamal, D. Drung, C. Enss, A. Fleischmann, A. M. Gangapshev, L. Gastaldo, Y. M. Gavrilyuk, A. M. Gezhaev, O. Gileva , et al. (81 additional authors not shown)

Abstract: AMoRE-II aims to search for neutrinoless double beta decay with an array of 423 Li$_2$$^{100}$MoO$_4$ crystals operating in the cryogenic system as the main phase of the Advanced Molybdenum-based Rare process Experiment (AMoRE). AMoRE has been planned to operate in three phases: AMoRE-pilot, AMoRE-I, and AMoRE-II. AMoRE-II is currently being installed at the Yemi Underground Laboratory, located ap… ▽ More AMoRE-II aims to search for neutrinoless double beta decay with an array of 423 Li$_2$$^{100}$MoO$_4$ crystals operating in the cryogenic system as the main phase of the Advanced Molybdenum-based Rare process Experiment (AMoRE). AMoRE has been planned to operate in three phases: AMoRE-pilot, AMoRE-I, and AMoRE-II. AMoRE-II is currently being installed at the Yemi Underground Laboratory, located approximately 1000 meters deep in Jeongseon, Korea. The goal of AMoRE-II is to reach up to $T^{0νββ}_{1/2}$ $\sim$ 6 $\times$ 10$^{26}$ years, corresponding to an effective Majorana mass of 15 - 29 meV, covering all the inverted mass hierarchy regions. To achieve this, the background level of the experimental configurations and possible background sources of gamma and beta events should be well understood. We have intensively performed Monte Carlo simulations using the GEANT4 toolkit in all the experimental configurations with potential sources. We report the estimated background level that meets the 10$^{-4}$counts/(keV$\cdot$kg$\cdot$yr) requirement for AMoRE-II in the region of interest (ROI) and show the projected half-life sensitivity based on the simulation study. △ Less

Submitted 13 June, 2024; originally announced June 2024.

Observation of Declination Dependence in the Cosmic Ray Energy Spectrum

Authors: The Telescope Array Collaboration, R. U. Abbasi, T. Abu-Zayyad, M. Allen, J. W. Belz, D. R. Bergman, I. Buckland, W. Campbell, B. G. Cheon, K. Endo, A. Fedynitch, T. Fujii, K. Fujisue, K. Fujita, M. Fukushima, G. Furlich, Z. Gerber, N. Globus, W. Hanlon, N. Hayashida, H. He, K. Hibino, R. Higuchi, D. Ikeda, T. Ishii , et al. (101 additional authors not shown)

Abstract: We report on an observation of the difference between northern and southern skies of the ultrahigh energy cosmic ray energy spectrum with a significance of ${\sim}8σ$. We use measurements from the two largest experiments$\unicode{x2014}$the Telescope Array observing the northern hemisphere and the Pierre Auger Observatory viewing the southern hemisphere. Since the comparison of two measurements fr… ▽ More We report on an observation of the difference between northern and southern skies of the ultrahigh energy cosmic ray energy spectrum with a significance of ${\sim}8σ$. We use measurements from the two largest experiments$\unicode{x2014}$the Telescope Array observing the northern hemisphere and the Pierre Auger Observatory viewing the southern hemisphere. Since the comparison of two measurements from different observatories introduces the issue of possible systematic differences between detectors and analyses, we validate the methodology of the comparison by examining the region of the sky where the apertures of the two observatories overlap. Although the spectra differ in this region, we find that there is only a $1.8σ$ difference between the spectrum measurements when anisotropic regions are removed and a fiducial cut in the aperture is applied. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: 8 pages, 6 figures

Jet modification via $π^0$-hadron correlations in Au$+$Au collisions at $\sqrt{s_{_{NN}}}=200$ GeV

Authors: PHENIX Collaboration, N. J. Abdulameer, U. Acharya, A. Adare, S. Afanasiev, C. Aidala, N. N. Ajitanand, Y. Akiba, H. Al-Bataineh, J. Alexander, M. Alfred, K. Aoki, N. Apadula, L. Aphecetche, J. Asai, H. Asano, E. T. Atomssa, R. Averbeck, T. C. Awes, B. Azmoun, V. Babintsev, M. Bai, G. Baksay, L. Baksay, A. Baldisseri , et al. (510 additional authors not shown)

Abstract: High-momentum two-particle correlations are a useful tool for studying jet-quenching effects in the quark-gluon plasma. Angular correlations between neutral-pion triggers and charged hadrons with transverse momenta in the range 4--12~GeV/$c$ and 0.5--7~GeV/$c$, respectively, have been measured by the PHENIX experiment in 2014 for Au$+$Au collisions at $\sqrt{s_{_{NN}}}=200$~GeV. Suppression is obs… ▽ More High-momentum two-particle correlations are a useful tool for studying jet-quenching effects in the quark-gluon plasma. Angular correlations between neutral-pion triggers and charged hadrons with transverse momenta in the range 4--12~GeV/$c$ and 0.5--7~GeV/$c$, respectively, have been measured by the PHENIX experiment in 2014 for Au$+$Au collisions at $\sqrt{s_{_{NN}}}=200$~GeV. Suppression is observed in the yield of high-momentum jet fragments opposite the trigger particle, which indicates jet suppression stemming from in-medium partonic energy loss, while enhancement is observed for low-momentum particles. The ratio and differences between the yield in Au$+$Au collisions and $p$$+$$p$ collisions, $I_{AA}$ and $Δ_{AA}$, as a function of the trigger-hadron azimuthal separation, $Δφ$, are measured for the first time at the Relativistic Heavy Ion Collider. These results better quantify how the yield of low-$p_T$ associated hadrons is enhanced at wide angle, which is crucial for studying energy loss as well as medium-response effects. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: 534 authors from 83 institutions, 12 pages, 7 figures. v1 is version submitted to Physical Review C. HEPdata tables for the points plotted in figures for this and previous PHENIX publications are (or will be) publicly available at http://www.phenix.bnl.gov/papers.html

Category-level Neural Field for Reconstruction of Partially Observed Objects in Indoor Environment

Authors: Taekbeom Lee, Youngseok Jang, H. Jin Kim

Abstract: Neural implicit representation has attracted attention in 3D reconstruction through various success cases. For further applications such as scene understanding or editing, several works have shown progress towards object compositional reconstruction. Despite their superior performance in observed regions, their performance is still limited in reconstructing objects that are partially observed. To… ▽ More Neural implicit representation has attracted attention in 3D reconstruction through various success cases. For further applications such as scene understanding or editing, several works have shown progress towards object compositional reconstruction. Despite their superior performance in observed regions, their performance is still limited in reconstructing objects that are partially observed. To better treat this problem, we introduce category-level neural fields that learn meaningful common 3D information among objects belonging to the same category present in the scene. Our key idea is to subcategorize objects based on their observed shape for better training of the category-level model. Then we take advantage of the neural field to conduct the challenging task of registering partially observed objects by selecting and aligning against representative objects selected by ray-based uncertainty. Experiments on both simulation and real-world datasets demonstrate that our method improves the reconstruction of unobserved parts for several categories. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: RA-L. 8 pages, 8 figures, 4 tables

Functional voxel hierarchy and afferent capacity revealed mental state transition on dynamic correlation resting-state fMRI

Authors: Dong Soo Lee, Hyun Joo Kim, Youngmin Huh, Yeon Koo Kang, Wonseok Whi, Hyekyoung Lee, Hyejin Kang

Abstract: Voxel hierarchy on dynamic brain graphs is produced by k core percolation on functional dynamic amplitude correlation of resting-state fMRI. Directed graphs and their afferent/efferent capacities are produced by Markov modeling of the universal cover of undirected graphs simultaneously with the calculation of volume entropy. Positive and unsigned negative brain graphs were analyzed separately on s… ▽ More Voxel hierarchy on dynamic brain graphs is produced by k core percolation on functional dynamic amplitude correlation of resting-state fMRI. Directed graphs and their afferent/efferent capacities are produced by Markov modeling of the universal cover of undirected graphs simultaneously with the calculation of volume entropy. Positive and unsigned negative brain graphs were analyzed separately on sliding-window representation to underpin the visualization and quantitation of mental dynamic states with their transitions. Voxel hierarchy animation maps of positive graphs revealed abrupt changes in coreness k and kmaxcore, which we called mental state transitions. Afferent voxel capacities of the positive graphs also revealed transient modules composed of dominating voxels/independent components and their exchanges representing mental state transitions. Animation and quantification plots of voxel hierarchy and afferent capacity corroborated each other in underpinning mental state transitions and afferent module exchange on the positive directed functional connectivity graphs. We propose the use of spatiotemporal trajectories of voxels on positive dynamic graphs to construct hierarchical structures by k core percolation and quantified in- and out-flows of information of voxels by volume entropy/directed graphs to subserve diverse resting mental state transitions on resting-state fMRI graphs in normal human individuals. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Guiding Frame-Level CTC Alignments Using Self-knowledge Distillation

Authors: Eungbeom Kim, Hantae Kim, Kyogu Lee

Abstract: Transformer encoder with connectionist temporal classification (CTC) framework is widely used for automatic speech recognition (ASR). However, knowledge distillation (KD) for ASR displays a problem of disagreement between teacher-student models in frame-level alignment which ultimately hinders it from improving the student model's performance. In order to resolve this problem, this paper introduce… ▽ More Transformer encoder with connectionist temporal classification (CTC) framework is widely used for automatic speech recognition (ASR). However, knowledge distillation (KD) for ASR displays a problem of disagreement between teacher-student models in frame-level alignment which ultimately hinders it from improving the student model's performance. In order to resolve this problem, this paper introduces a self-knowledge distillation (SKD) method that guides the frame-level alignment during the training time. In contrast to the conventional method using separate teacher and student models, this study introduces a simple and effective method sharing encoder layers and applying the sub-model as the student model. Overall, our approach is effective in improving both the resource efficiency as well as performance. We also conducted an experimental analysis of the spike timings to illustrate that the proposed method improves performance by reducing the alignment disagreement. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: Accepted by Interspeech 2024

Assessing the Impact of Alpha Particles on Thermal Confinement in JET D-T Plasmas through Global GENE-Tango Simulations

Authors: A. Di Siena, J. Garcia, R. Bilato, K. Kirov, J. Varela A. Banon Navarro, Hyun-Tae Kim, C. Challis, J. Hobirk, A. Kappatou, E. Lerche, D. Spong, C. Angioni, T. Gorler, E. Poli, M. Bergmann, F. Jenko, JET contributors

Abstract: The capability of the global, electromagnetic gyrokinetic GENE code interfaced with the transport Tango solver is exploited to address the impact of fusion alpha particles (in their dual role of fast particles and heating source) on plasma profiles and performance at JET in the discharges with the highest quasi-stationary peak fusion power during the DTE2 experimental campaigns. Employing radially… ▽ More The capability of the global, electromagnetic gyrokinetic GENE code interfaced with the transport Tango solver is exploited to address the impact of fusion alpha particles (in their dual role of fast particles and heating source) on plasma profiles and performance at JET in the discharges with the highest quasi-stationary peak fusion power during the DTE2 experimental campaigns. Employing radially global nonlinear electromagnetic GENE-Tango simulations, we compare results with/without alpha particles and alpha heating. Our findings reveal that alpha particles have a negligible impact on turbulent transport, with GENE-Tango converging to similar plasma profiles regardless of their inclusion as a kinetic species in GENE. On the other hand, alpha heating is found to contribute to the peaking of the electron temperature profiles, leading to a 1keV drop on the on-axis electron temperature when alpha heating is neglected in Tango. The minimal impact of alpha particles on turbulent transport in this JET discharge - despite this being the shot with the highest fusion output - is attributed to the low content of fusion alpha in this discharge. To assess the potential impact of alpha particles on turbulent transport in regimes with higher alpha particle density, as expected in ITER and fusion reactors, we artificially increased the alpha particle concentration to levels expected for ITER. By performing global nonlinear GENE standalone simulations, we found that increasing the alpha particle density beyond five times the nominal value lead to significant overall turbulence destabilization. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Discrete Dictionary-based Decomposition Layer for Structured Representation Learning

Authors: Taewon Park, Hyun-Chul Kim, Minho Lee

Abstract: Neuro-symbolic neural networks have been extensively studied to integrate symbolic operations with neural networks, thereby improving systematic generalization. Specifically, Tensor Product Representation (TPR) framework enables neural networks to perform differentiable symbolic operations by encoding the symbolic structure of data within vector spaces. However, TPR-based neural networks often str… ▽ More Neuro-symbolic neural networks have been extensively studied to integrate symbolic operations with neural networks, thereby improving systematic generalization. Specifically, Tensor Product Representation (TPR) framework enables neural networks to perform differentiable symbolic operations by encoding the symbolic structure of data within vector spaces. However, TPR-based neural networks often struggle to decompose unseen data into structured TPR representations, undermining their symbolic operations. To address this decomposition problem, we propose a Discrete Dictionary-based Decomposition (D3) layer designed to enhance the decomposition capabilities of TPR-based models. D3 employs discrete, learnable key-value dictionaries trained to capture symbolic features essential for decomposition operations. It leverages the prior knowledge acquired during training to generate structured TPR representations by mapping input data to pre-learned symbolic features within these dictionaries. D3 is a straightforward drop-in layer that can be seamlessly integrated into any TPR-based model without modifications. Our experimental results demonstrate that D3 significantly improves the systematic generalization of various TPR-based models while requiring fewer additional parameters. Notably, D3 outperforms baseline models on the synthetic task that demands the systematic decomposition of unseen combinatorial data. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Frustrated phonon with charge density wave in vanadium Kagome metal

Authors: Seung-Phil Heo, Choongjae Won, Heemin Lee, Hanbyul Kim, Eunyoung Park, Sung Yun Lee, Junha Hwang, Hyeongi Choi, Sang-Youn Park, Byungjune Lee, Woo-Suk Noh, Hoyoung Jang, Jae-Hoon Park, Dongbin Shin, Changyong Song

Abstract: Crystals with unique ionic arrangements and strong electronic correlations serve as a fertile ground for the emergence of exotic phases, as evidenced by the coexistence of charge density wave (CDW) and superconductivity in vanadium Kagome metals, specifically AV3Sb5 (where A represents K, Rb, or Cs). The formation of a star of David CDW superstructure, resulting from the coordinated displacements… ▽ More Crystals with unique ionic arrangements and strong electronic correlations serve as a fertile ground for the emergence of exotic phases, as evidenced by the coexistence of charge density wave (CDW) and superconductivity in vanadium Kagome metals, specifically AV3Sb5 (where A represents K, Rb, or Cs). The formation of a star of David CDW superstructure, resulting from the coordinated displacements of vanadium ions on a corner sharing triangular lattice, has garnered significant attention in efforts to comprehend the influence of electron phonon interaction within this geometrically intricate lattice. However, understanding of the underlying mechanism behind CDW formation, coupled with symmetry protected lattice vibrations, remains elusive. In this study, we employed time resolved X ray scattering experiments utilising an X ray free electron laser. Our findings reveal that the phonon mode associated with the out of plane motion of Cs ions becomes frustrated in the CDW phase. Furthermore, we observed the photoinduced emergence of a metastable CDW phase, facilitated by the alleviation of frustration through nonadiabatic changes in free energy. By elucidating the longstanding puzzle surrounding the intervention of phonons in CDW ordering, this research offers fresh insights into the competition between phonons and periodic lattice distortions, a phenomenon widespread in other correlated quantum materials including layered high Tc superconductors. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Comments: Manuscript: 20 pages, 4 figures, SI: 14 pages, 8 figures

Exploring new constraints on Kahler moduli space of 6d N = 1 Supergravity

Authors: Hee-Cheol Kim, Cumrun Vafa

Abstract: We propose new constraints for 6d (1, 0) supergravity theories based on consistency conditions on the Kahler moduli spaces of their 5d reductions. The requirement that both the metric and the BPS string tensions in the Kahler moduli space are positive imposes specific restrictions on the Chern-Simons coefficients in the 5d effective Lagrangians that are derived from the Kaluza-Klein reductions of… ▽ More We propose new constraints for 6d (1, 0) supergravity theories based on consistency conditions on the Kahler moduli spaces of their 5d reductions. The requirement that both the metric and the BPS string tensions in the Kahler moduli space are positive imposes specific restrictions on the Chern-Simons coefficients in the 5d effective Lagrangians that are derived from the Kaluza-Klein reductions of 6d theories. Moreover, the emergence of local interacting 5d CFTs when the moduli space metric degenerates introduces additional constraints coming from the analysis of 5d SCFTs. Focusing on the moduli spaces of 6d supergravity theories without a tensor multiplet and their Higgsings, we show that these constraints require the presence of certain primary states in the 2d worldvolume CFTs on 1/2 BPS strings. We specifically analyze a class of SU(2) models and infinite families of U(1) models using these constraints, and demonstrate that the theories featuring a 1-form symmetry in their massless spectra, unless the 1-form symmetry is gauged, fail to satisfy the constraints and therefore belong to the Swampland. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Comments: 32 pages

Shear thickening in suspensions of particles with dynamic brush layers

Authors: Hojin Kim, Michael van der Naald, Finn A. Braaten, Thomas A. Witten, Stuart J. Rowan, Heinrich M. Jaeger

Abstract: Control of frictional interactions among liquid-suspended particles has led to tunable, strikingly non-Newtonian rheology via the formation of strong flow constraints as particles come into close proximity under shear. Typically, these frictional interactions have been in the form of physical contact, controllable via particle shape and surface roughness. We investigate a different route, where mo… ▽ More Control of frictional interactions among liquid-suspended particles has led to tunable, strikingly non-Newtonian rheology via the formation of strong flow constraints as particles come into close proximity under shear. Typically, these frictional interactions have been in the form of physical contact, controllable via particle shape and surface roughness. We investigate a different route, where molecular bridging between nearby particle surfaces generates a controllable "sticky" friction. This is achieved with surface-functionalized colloidal particles capable of forming dynamic covalent bonds with telechelic polymers that comprise the suspending fluid. At low shear stress this results in particles coated with a uniform polymer brush layer. Beyond an onset stress the telechelic polymers become capable of bridging and generate shear thickening. Over the size range investigated, we find that the dynamic brush layer leads to dependence of the onset stress on particle diameter that closely follows a power law with exponent -1.76. In the shear thickening regime, we observe an enhanced dilation in measurements of the first normal stress difference and reduction in the extrapolated volume fraction required for jamming, both consistent with an effective particle friction that increases with decreasing particle diameter. These results are discussed in light of predictions for suspensions of hard spheres and of polymer-grafted particles. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Decoupled Marked Temporal Point Process using Neural Ordinary Differential Equations

Authors: Yujee Song, Donghyun Lee, Rui Meng, Won Hwa Kim

Abstract: A Marked Temporal Point Process (MTPP) is a stochastic process whose realization is a set of event-time data. MTPP is often used to understand complex dynamics of asynchronous temporal events such as money transaction, social media, healthcare, etc. Recent studies have utilized deep neural networks to capture complex temporal dependencies of events and generate embedding that aptly represent the o… ▽ More A Marked Temporal Point Process (MTPP) is a stochastic process whose realization is a set of event-time data. MTPP is often used to understand complex dynamics of asynchronous temporal events such as money transaction, social media, healthcare, etc. Recent studies have utilized deep neural networks to capture complex temporal dependencies of events and generate embedding that aptly represent the observed events. While most previous studies focus on the inter-event dependencies and their representations, how individual events influence the overall dynamics over time has been under-explored. In this regime, we propose a Decoupled MTPP framework that disentangles characterization of a stochastic process into a set of evolving influences from different events. Our approach employs Neural Ordinary Differential Equations (Neural ODEs) to learn flexible continuous dynamics of these influences while simultaneously addressing multiple inference problems, such as density estimation and survival rate computation. We emphasize the significance of disentangling the influences by comparing our framework with state-of-the-art methods on real-life datasets, and provide analysis on the model behavior for potential applications. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Comments: 18 pages, 8 figures, The Twelfth International Conference on Learning Representations (ICLR 2024)

Exclusion of the Cosmological Triangle in Reactor-Based Search for Axion-Like Particles

Authors: Byung Ju Park, Jae Jin Choi, Eunju Jeon, Jinyu Kim, Kyungwon Kim, Sung Hyun Kim, Sun Kee Kim, Yeongduk Kim, Young Ju Ko, Byoung-Cheol Koh, Chang Hyon Ha, Seo Hyun Lee, In Soo Lee, Hyunseok Lee, Hyun Su Lee, Jaison Lee, Yoomin Oh, Doojin Kim

Abstract: We report new constraints on axion-like particle (ALP) using data corresponding to a sodium iodine target exposure of 3063 kg$\cdot$days from the neutrino elastic scattering observation with NaI (NEON) experiment. A 16.7 kg of thallium-doped sodium iodide target was located 23.7 meters from a 2.8 GW thermal power nuclear reactor. We searched for ALPs produced by high-flux photons by comparing the… ▽ More We report new constraints on axion-like particle (ALP) using data corresponding to a sodium iodine target exposure of 3063 kg$\cdot$days from the neutrino elastic scattering observation with NaI (NEON) experiment. A 16.7 kg of thallium-doped sodium iodide target was located 23.7 meters from a 2.8 GW thermal power nuclear reactor. We searched for ALPs produced by high-flux photons by comparing the energy spectra of data collected during reactor-on (1596 kg$\cdot$days exposure) and reactor-off (1467 kg$\cdot$days exposure) periods. No signal consistent with ALP interaction was identified, allowing us to set exclusion limits at the 95% confidence level. Our limits cover previously unexplored regions for both photon couplings (${g_{aγ}}$) and electron couplings (${g_{ae}}$) for axion masses around 1 MeV/c$^2$. Notably, the NEON data excludes the unconstrained region identified by laboratory-based searches for photon couplings within the "cosmological triangle" for the first time. The observed 95\% confidence level limits reach as low as ${g_{aγ}}$ of 4.33$\times$ 10$^{-8}$ GeV$^{-1}$ and ${g_{ae}}$ of 1.10$\times$ 10$^{-9}$ for axion masses of 1.7 MeV/c$^2$ and 1.0 MeV/c$^2$, respectively. △ Less

Submitted 11 June, 2024; v1 submitted 10 June, 2024; originally announced June 2024.

Adapting Pretrained ViTs with Convolution Injector for Visuo-Motor Control

Authors: Dongyoon Hwang, Byungkun Lee, Hojoon Lee, Hyunseung Kim, Jaegul Choo

Abstract: Vision Transformers (ViT), when paired with large-scale pretraining, have shown remarkable performance across various computer vision tasks, primarily due to their weak inductive bias. However, while such weak inductive bias aids in pretraining scalability, this may hinder the effective adaptation of ViTs for visuo-motor control tasks as a result of the absence of control-centric inductive biases.… ▽ More Vision Transformers (ViT), when paired with large-scale pretraining, have shown remarkable performance across various computer vision tasks, primarily due to their weak inductive bias. However, while such weak inductive bias aids in pretraining scalability, this may hinder the effective adaptation of ViTs for visuo-motor control tasks as a result of the absence of control-centric inductive biases. Such absent inductive biases include spatial locality and translation equivariance bias which convolutions naturally offer. To this end, we introduce Convolution Injector (CoIn), an add-on module that injects convolutions which are rich in locality and equivariance biases into a pretrained ViT for effective adaptation in visuo-motor control. We evaluate CoIn with three distinct types of pretrained ViTs (CLIP, MVP, VC-1) across 12 varied control tasks within three separate domains (Adroit, MetaWorld, DMC), and demonstrate that CoIn consistently enhances control task performance across all experimented environments and models, validating the effectiveness of providing pretrained ViTs with control-centric biases. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Comments: accepted to ICML 2024

GCAPS: GPU Context-Aware Preemptive Priority-based Scheduling for Real-Time Tasks

Authors: Yidi Wang, Cong Liu, Daniel Wong, Hyoseung Kim

Abstract: Scheduling real-time tasks that utilize GPUs with analyzable guarantees poses a significant challenge due to the intricate interaction between CPU and GPU resources, as well as the complex GPU hardware and software stack. While much research has been conducted in the real-time research community, several limitations persist, including the absence or limited availability of GPU-level preemption, ex… ▽ More Scheduling real-time tasks that utilize GPUs with analyzable guarantees poses a significant challenge due to the intricate interaction between CPU and GPU resources, as well as the complex GPU hardware and software stack. While much research has been conducted in the real-time research community, several limitations persist, including the absence or limited availability of GPU-level preemption, extended blocking times, and/or the need for extensive modifications to program code. In this paper, we propose GCAPS, a GPU Context-Aware Preemptive Scheduling approach for real-time GPU tasks. Our approach exerts control over GPU context scheduling at the device driver level and enables preemption of GPU execution based on task priorities by simply adding one-line macros to GPU segment boundaries. In addition, we provide a comprehensive response time analysis of GPU-using tasks for both our proposed approach as well as the default Nvidia GPU driver scheduling that follows a work-conserving round-robin policy. Through empirical evaluations and case studies, we demonstrate the effectiveness of the proposed approaches in improving taskset schedulability and response time. The results highlight significant improvements over prior work as well as the default scheduling approach, with up to 40% higher schedulability, while also achieving predictable worst-case behavior on Nvidia Jetson embedded platforms. △ Less

Submitted 7 June, 2024; originally announced June 2024.

Comments: Accepted by ECRTS 2024. arXiv admin note: substantial text overlap with arXiv:2401.16529

Astrometric Search for Ultralight Dark Matter

Authors: Hyungjin Kim

Abstract: Precision astrometry offers a way to probe new physics. By measuring the angular position of light sources at unprecedented precision, astrometry could probe minuscule fluctuations of underlying spacetime. This work explores the possibility of probing ultralight dark matter candidates using precision astrometry. Through the coherent and stochastic density fluctuations over the scale of its wavelen… ▽ More Precision astrometry offers a way to probe new physics. By measuring the angular position of light sources at unprecedented precision, astrometry could probe minuscule fluctuations of underlying spacetime. This work explores the possibility of probing ultralight dark matter candidates using precision astrometry. Through the coherent and stochastic density fluctuations over the scale of its wavelength, ultralight dark matter perturbs the propagation of light and the geodesics of the observer and source, leading to unique time-dependent signatures in the angular position of background light sources. With detector specifications similar to the current and future astrometry observations, such as Gaia and Roman Space Telescope, it is shown that the ultralight scalar dark matter of mass $10^{-18}\,{\rm eV} \, \textrm{--} \, 10^{-16} \,{\rm eV}$ could be probed when its density near the solar system is about a few thousand times larger than the nominal dark matter density measured on a much larger kpc-scale. This sensitivity is comparable to current pulsar timing array observations at a similar mass range. Explicit expressions for the angular deflection induced by most generic metric perturbations are derived and its gauge invariance is explicitly checked at the linear order. △ Less

Submitted 5 June, 2024; originally announced June 2024.

Comments: 20 pages, 4 figures

Report number: DESY-24-063

Local to Global: Learning Dynamics and Effect of Initialization for Transformers

Authors: Ashok Vardhan Makkuva, Marco Bondaschi, Chanakya Ekbote, Adway Girish, Alliot Nagle, Hyeji Kim, Michael Gastpar

Abstract: In recent years, transformer-based models have revolutionized deep learning, particularly in sequence modeling. To better understand this phenomenon, there is a growing interest in using Markov input processes to study transformers. However, our current understanding in this regard remains limited with many fundamental questions about how transformers learn Markov chains still unanswered. In this… ▽ More In recent years, transformer-based models have revolutionized deep learning, particularly in sequence modeling. To better understand this phenomenon, there is a growing interest in using Markov input processes to study transformers. However, our current understanding in this regard remains limited with many fundamental questions about how transformers learn Markov chains still unanswered. In this paper, we address this by focusing on first-order Markov chains and single-layer transformers, providing a comprehensive characterization of the learning dynamics in this context. Specifically, we prove that transformer parameters trained on next-token prediction loss can either converge to global or local minima, contingent on the initialization and the Markovian data properties, and we characterize the precise conditions under which this occurs. To the best of our knowledge, this is the first result of its kind highlighting the role of initialization. We further demonstrate that our theoretical findings are corroborated by empirical evidence. Based on these insights, we provide guidelines for the initialization of transformer parameters and demonstrate their effectiveness. Finally, we outline several open problems in this arena. Code is available at: https://github.com/Bond1995/Markov. △ Less

Submitted 27 June, 2024; v1 submitted 5 June, 2024; originally announced June 2024.

Language Model Can Do Knowledge Tracing: Simple but Effective Method to Integrate Language Model and Knowledge Tracing Task

Authors: Unggi Lee, Jiyeong Bae, Dohee Kim, Sookbun Lee, Jaekwon Park, Taekyung Ahn, Gunho Lee, Damji Stratton, Hyeoncheol Kim

Abstract: Knowledge Tracing (KT) is a critical task in online learning for modeling student knowledge over time. Despite the success of deep learning-based KT models, which rely on sequences of numbers as data, most existing approaches fail to leverage the rich semantic information in the text of questions and concepts. This paper proposes Language model-based Knowledge Tracing (LKT), a novel framework that… ▽ More Knowledge Tracing (KT) is a critical task in online learning for modeling student knowledge over time. Despite the success of deep learning-based KT models, which rely on sequences of numbers as data, most existing approaches fail to leverage the rich semantic information in the text of questions and concepts. This paper proposes Language model-based Knowledge Tracing (LKT), a novel framework that integrates pre-trained language models (PLMs) with KT methods. By leveraging the power of language models to capture semantic representations, LKT effectively incorporates textual information and significantly outperforms previous KT models on large benchmark datasets. Moreover, we demonstrate that LKT can effectively address the cold-start problem in KT by leveraging the semantic knowledge captured by PLMs. Interpretability of LKT is enhanced compared to traditional KT models due to its use of text-rich data. We conducted the local interpretable model-agnostic explanation technique and analysis of attention scores to interpret the model performance further. Our work highlights the potential of integrating PLMs with KT and paves the way for future research in KT domain. △ Less

Submitted 9 June, 2024; v1 submitted 4 June, 2024; originally announced June 2024.

Comments: 11 pages, 5 figures, 3 tables

Slow and Steady Wins the Race: Maintaining Plasticity with Hare and Tortoise Networks

Authors: Hojoon Lee, Hyeonseo Cho, Hyunseung Kim, Donghu Kim, Dugki Min, Jaegul Choo, Clare Lyle

Abstract: This study investigates the loss of generalization ability in neural networks, revisiting warm-starting experiments from Ash & Adams. Our empirical analysis reveals that common methods designed to enhance plasticity by maintaining trainability provide limited benefits to generalization. While reinitializing the network can be effective, it also risks losing valuable prior knowledge. To this end, w… ▽ More This study investigates the loss of generalization ability in neural networks, revisiting warm-starting experiments from Ash & Adams. Our empirical analysis reveals that common methods designed to enhance plasticity by maintaining trainability provide limited benefits to generalization. While reinitializing the network can be effective, it also risks losing valuable prior knowledge. To this end, we introduce the Hare & Tortoise, inspired by the brain's complementary learning system. Hare & Tortoise consists of two components: the Hare network, which rapidly adapts to new information analogously to the hippocampus, and the Tortoise network, which gradually integrates knowledge akin to the neocortex. By periodically reinitializing the Hare network to the Tortoise's weights, our method preserves plasticity while retaining general knowledge. Hare & Tortoise can effectively maintain the network's ability to generalize, which improves advanced reinforcement learning algorithms on the Atari-100k benchmark. The code is available at https://github.com/dojeon-ai/hare-tortoise. △ Less

Submitted 1 June, 2024; originally announced June 2024.

Comments: accepted to ICML 2024

Applying Fine-Tuned LLMs for Reducing Data Needs in Load Profile Analysis

Authors: Yi Hu, Hyeonjin Kim, Kai Ye, Ning Lu

Abstract: This paper presents a novel method for utilizing fine-tuned Large Language Models (LLMs) to minimize data requirements in load profile analysis, demonstrated through the restoration of missing data in power system load profiles. A two-stage fine-tuning strategy is proposed to adapt a pre-trained LLMs, i.e., GPT-3.5, for missing data restoration tasks. Through empirical evaluation, we demonstrate t… ▽ More This paper presents a novel method for utilizing fine-tuned Large Language Models (LLMs) to minimize data requirements in load profile analysis, demonstrated through the restoration of missing data in power system load profiles. A two-stage fine-tuning strategy is proposed to adapt a pre-trained LLMs, i.e., GPT-3.5, for missing data restoration tasks. Through empirical evaluation, we demonstrate the effectiveness of the fine-tuned model in accurately restoring missing data, achieving comparable performance to state-of-the-art specifically designed models such as BERT-PIN. Key findings include the importance of prompt engineering and the optimal utilization of fine-tuning samples, highlighting the efficiency of few-shot learning in transferring knowledge from general user cases to specific target users. Furthermore, the proposed approach demonstrates notable cost-effectiveness and time efficiency compared to training models from scratch, making it a practical solution for scenarios with limited data availability and computing resources. This research has significant potential for application to other power system load profile analysis tasks. Consequently, it advances the use of LLMs in power system analytics, offering promising implications for enhancing the resilience and efficiency of power distribution systems. △ Less

Submitted 2 June, 2024; originally announced June 2024.

Identifying Sample Size and Accuracy and Precision of the Estimators in Case-Crossover Designs with Distributed Lags of Heteroskedastic Time-Varying Continuous Exposures Measured with Simple or Complex Error

Authors: Honghyok Kim

Abstract: Understanding of sample size, statistical power, and the accuracy and precision of the estimator in epidemiological research can facilitate power and bias analyses. However, such understanding can become complicated for several reasons. First, exposures varying spatiotemporally may be heteroskedastic. Second, distributed lags of exposures may be used to identify critical exposure time-windows. Thi… ▽ More Understanding of sample size, statistical power, and the accuracy and precision of the estimator in epidemiological research can facilitate power and bias analyses. However, such understanding can become complicated for several reasons. First, exposures varying spatiotemporally may be heteroskedastic. Second, distributed lags of exposures may be used to identify critical exposure time-windows. Third, exposure measurement error may exist, impacting the accuracy and/or precision of the estimator that consequently affects sample size and statistical power. Fourth, research may rely on different study designs, so understanding may differ. For example, case-crossover designs as matched case-control designs, are used to estimate health effects of short-term exposures. To address these gaps, I developed approximation equations for sample size, estimates of the estimators and standard errors, including polynomials for non-linear effect estimation. With air pollution exposure estimates, I examined approximations using statistical simulations. Overall, sample size, the accuracy and precision of the estimators can be approximated based on external information about validation, without validation data in hand. For distributed lags, approximations may perform well if residual confounding due to covariate measurement errors is not severe. This condition may be difficult to identify without validation data, so validation research is recommended in identifying critical exposure time-windows. △ Less

Submitted 18 June, 2024; v1 submitted 4 June, 2024; originally announced June 2024.

Comments: Submitted for peer-reviewed publication

Measure-Observe-Remeasure: An Interactive Paradigm for Differentially-Private Exploratory Analysis

Authors: Priyanka Nanayakkara, Hyeok Kim, Yifan Wu, Ali Sarvghad, Narges Mahyar, Gerome Miklau, Jessica Hullman

Abstract: Differential privacy (DP) has the potential to enable privacy-preserving analysis on sensitive data, but requires analysts to judiciously spend a limited ``privacy loss budget'' $ε$ across queries. Analysts conducting exploratory analyses do not, however, know all queries in advance and seldom have DP expertise. Thus, they are limited in their ability to specify $ε$ allotments across queries prior… ▽ More Differential privacy (DP) has the potential to enable privacy-preserving analysis on sensitive data, but requires analysts to judiciously spend a limited ``privacy loss budget'' $ε$ across queries. Analysts conducting exploratory analyses do not, however, know all queries in advance and seldom have DP expertise. Thus, they are limited in their ability to specify $ε$ allotments across queries prior to an analysis. To support analysts in spending $ε$ efficiently, we propose a new interactive analysis paradigm, Measure-Observe-Remeasure, where analysts ``measure'' the database with a limited amount of $ε$, observe estimates and their errors, and remeasure with more $ε$ as needed. We instantiate the paradigm in an interactive visualization interface which allows analysts to spend increasing amounts of $ε$ under a total budget. To observe how analysts interact with the Measure-Observe-Remeasure paradigm via the interface, we conduct a user study that compares the utility of $ε$ allocations and findings from sensitive data participants make to the allocations and findings expected of a rational agent who faces the same decision task. We find that participants are able to use the workflow relatively successfully, including using budget allocation strategies that maximize over half of the available utility stemming from $ε$ allocation. Their loss in performance relative to a rational agent appears to be driven more by their inability to access information and report it than to allocate $ε$. △ Less

Submitted 4 June, 2024; originally announced June 2024.

Comments: Published in IEEE Symposium on Security and Privacy (SP) 2024

Journal ref: in 2024 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA, 2024 pp. 231-231

CODE: Contrasting Self-generated Description to Combat Hallucination in Large Multi-modal Models

Authors: Junho Kim, Hyunjun Kim, Yeonju Kim, Yong Man Ro

Abstract: Large Multi-modal Models (LMMs) have recently demonstrated remarkable abilities in visual context understanding and coherent response generation. However, alongside these advancements, the issue of hallucinations has emerged as a significant challenge, producing erroneous responses that are unrelated to the visual contents. In this paper, we introduce a novel contrastive-based decoding method, COu… ▽ More Large Multi-modal Models (LMMs) have recently demonstrated remarkable abilities in visual context understanding and coherent response generation. However, alongside these advancements, the issue of hallucinations has emerged as a significant challenge, producing erroneous responses that are unrelated to the visual contents. In this paper, we introduce a novel contrastive-based decoding method, COuntering DEscription Contrastive Decoding (CODE), which leverages self-generated descriptions as contrasting references during the decoding phase of LMMs to address hallucination issues. CODE utilizes the comprehensive descriptions from model itself as visual counterpart to correct and improve response alignment with actual visual content. By dynamically adjusting the information flow and distribution of next-token predictions in the LMM's vocabulary, CODE enhances the coherence and informativeness of generated responses. Extensive experiments demonstrate that our method significantly reduces hallucinations and improves cross-modal consistency across various benchmarks and cutting-edge LMMs. Our method provides a simple yet effective decoding strategy that can be integrated to existing LMM frameworks without additional training. △ Less

Submitted 3 June, 2024; originally announced June 2024.

Comments: Project page: https://ivy-lvlm.github.io/CODE/

General relativistic self-gravitating equilibrium disks around rotating neutron stars

Authors: Yoonsoo Kim, Jinho Kim, Hee Il Kim, Hyung Mok Lee

Abstract: In modeling a relativistic disk around a compact object, the self-gravity of the disk is often neglected while it needs to be incorporated for more accurate descriptions in several circumstances. Extending the Komatsu-Eriguchi-Hachisu self-consistent field method, we present numerical models of a rapidly rotating neutron star with a self-gravitating disk in stationary equilibrium. In particular, o… ▽ More In modeling a relativistic disk around a compact object, the self-gravity of the disk is often neglected while it needs to be incorporated for more accurate descriptions in several circumstances. Extending the Komatsu-Eriguchi-Hachisu self-consistent field method, we present numerical models of a rapidly rotating neutron star with a self-gravitating disk in stationary equilibrium. In particular, our approach allows us to obtain numerical solutions involving a massive disk with the rest mass $O(10^{-1})-O(10^0) M_\odot$ closely attached to a rotating neutron star. We also assess the impact of self-gravity on the internal structure of the disk and the neutron star. These axisymmetric, stationary solutions can be employed for simulations involving the neutron star-disk system in the context of high-energy transients and gravitational wave emissions. △ Less

Submitted 2 June, 2024; originally announced June 2024.

Comments: 15 pages, 12 figures

Expanding the Attack Scenarios of SAE J1939: A Comprehensive Analysis of Established and Novel Vulnerabilities in Transport Protocol

Authors: Hwejae Lee, Hyosun Lee, Saehee Jun, Huy Kang Kim

Abstract: Following the enactment of the UN Regulation, substantial efforts have been directed toward implementing intrusion detection and prevention systems (IDPSs) and vulnerability analysis in Controller Area Network (CAN). However, Society of Automotive Engineers (SAE) J1939 protocol, despite its extensive application in camping cars and commercial vehicles, has seen limited vulnerability identification… ▽ More Following the enactment of the UN Regulation, substantial efforts have been directed toward implementing intrusion detection and prevention systems (IDPSs) and vulnerability analysis in Controller Area Network (CAN). However, Society of Automotive Engineers (SAE) J1939 protocol, despite its extensive application in camping cars and commercial vehicles, has seen limited vulnerability identification, which raises significant safety concerns in the event of security breaches. In this research, we explore and demonstrate attack techniques specific to SAE J1939 communication protocol. We introduce 14 attack scenarios, enhancing the discourse with seven scenarios recognized in the previous research and unveiling seven novel scenarios through our elaborate study. To verify the feasibility of these scenarios, we leverage a sophisticated testbed that facilitates real-time communication and the simulation of attacks. Our testing confirms the successful execution of 11 scenarios, underscoring their imminent threat to commercial vehicle operations. Some attacks will be difficult to detect because they only inject a single message. These results highlight unique vulnerabilities within SAE J1939 protocol, indicating the automotive cybersecurity community needs to address the identified risks. △ Less

Submitted 2 June, 2024; originally announced June 2024.

Comments: 18 pages, 7 figures, 5 tables; This is the accepted version of ESCAR USA 2024

MSC Class: 68M25 ACM Class: K.6.5

A Blueprint Architecture of Compound AI Systems for Enterprise

Authors: Eser Kandogan, Sajjadur Rahman, Nikita Bhutani, Dan Zhang, Rafael Li Chen, Kushan Mitra, Sairam Gurajada, Pouya Pezeshkpour, Hayate Iso, Yanlin Feng, Hannah Kim, Chen Shen, Jin Wang, Estevam Hruschka

Abstract: Large Language Models (LLMs) have showcased remarkable capabilities surpassing conventional NLP challenges, creating opportunities for use in production use cases. Towards this goal, there is a notable shift to building compound AI systems, wherein LLMs are integrated into an expansive software infrastructure with many components like models, retrievers, databases and tools. In this paper, we intr… ▽ More Large Language Models (LLMs) have showcased remarkable capabilities surpassing conventional NLP challenges, creating opportunities for use in production use cases. Towards this goal, there is a notable shift to building compound AI systems, wherein LLMs are integrated into an expansive software infrastructure with many components like models, retrievers, databases and tools. In this paper, we introduce a blueprint architecture for compound AI systems to operate in enterprise settings cost-effectively and feasibly. Our proposed architecture aims for seamless integration with existing compute and data infrastructure, with ``stream'' serving as the key orchestration concept to coordinate data and instructions among agents and other components. Task and data planners, respectively, break down, map, and optimize tasks and data to available agents and data sources defined in respective registries, given production constraints such as accuracy and latency. △ Less

Submitted 1 June, 2024; originally announced June 2024.

Comments: Compound AI Systems Workshop at the Data+AI Summit 2024

Do's and Don'ts: Learning Desirable Skills with Instruction Videos

Authors: Hyunseung Kim, Byungkun Lee, Hojoon Lee, Dongyoon Hwang, Donghu Kim, Jaegul Choo

Abstract: Unsupervised skill discovery is a learning paradigm that aims to acquire diverse behaviors without explicit rewards. However, it faces challenges in learning complex behaviors and often leads to learning unsafe or undesirable behaviors. For instance, in various continuous control tasks, current unsupervised skill discovery methods succeed in learning basic locomotions like standing but struggle wi… ▽ More Unsupervised skill discovery is a learning paradigm that aims to acquire diverse behaviors without explicit rewards. However, it faces challenges in learning complex behaviors and often leads to learning unsafe or undesirable behaviors. For instance, in various continuous control tasks, current unsupervised skill discovery methods succeed in learning basic locomotions like standing but struggle with learning more complex movements such as walking and running. Moreover, they may acquire unsafe behaviors like tripping and rolling or navigate to undesirable locations such as pitfalls or hazardous areas. In response, we present DoDont (Do's and Don'ts), an instruction-based skill discovery algorithm composed of two stages. First, in an instruction learning stage, DoDont leverages action-free instruction videos to train an instruction network to distinguish desirable transitions from undesirable ones. Then, in the skill learning stage, the instruction network adjusts the reward function of the skill discovery algorithm to weight the desired behaviors. Specifically, we integrate the instruction network into a distance-maximizing skill discovery algorithm, where the instruction network serves as the distance function. Empirically, with less than 8 instruction videos, DoDont effectively learns desirable behaviors and avoids undesirable ones across complex continuous control tasks. Code and videos are available at https://mynsng.github.io/dodont/ △ Less

Submitted 1 June, 2024; originally announced June 2024.

Mirror Symmetry and Level-rank Duality for 3d $\mathcal{N} = 4$ Rank 0 SCFTs

Authors: Thomas Creutzig, Niklas Garner, Heeyeon Kim

Abstract: We introduce a family of 3d $\mathcal{N} = 4$ superconformal field theories that have zero-dimensional Coulomb and Higgs branches and propose that the rational vertex operator algebras $W^{\text{min}}_{k - \scriptstyle{\frac{1}{2}}}(\mathfrak{sp}_{2N})$ and $L_{k}(\mathfrak{osp}_{1|2N})$ model the modular tensor categories of line operators in their topological $A$ and $B$ twists, respectively. Ou… ▽ More We introduce a family of 3d $\mathcal{N} = 4$ superconformal field theories that have zero-dimensional Coulomb and Higgs branches and propose that the rational vertex operator algebras $W^{\text{min}}_{k - \scriptstyle{\frac{1}{2}}}(\mathfrak{sp}_{2N})$ and $L_{k}(\mathfrak{osp}_{1|2N})$ model the modular tensor categories of line operators in their topological $A$ and $B$ twists, respectively. Our analysis indicates that the action of 3d mirror symmetry on this family of theories is related to a novel level-rank duality and leads to several conjectural $q$-series identities of independent interest. △ Less

Submitted 31 May, 2024; originally announced June 2024.

Comments: 29 pages, 1 figure; comments welcome!

KU-DMIS at EHRSQL 2024:Generating SQL query via question templatization in EHR

Authors: Hajung Kim, Chanhwi Kim, Hoonick Lee, Kyochul Jang, Jiwoo Lee, Kyungjae Lee, Gangwoo Kim, Jaewoo Kang

Abstract: Transforming natural language questions into SQL queries is crucial for precise data retrieval from electronic health record (EHR) databases. A significant challenge in this process is detecting and rejecting unanswerable questions that request information beyond the database's scope or exceed the system's capabilities. In this paper, we introduce a novel text-to-SQL framework that robustly handle… ▽ More Transforming natural language questions into SQL queries is crucial for precise data retrieval from electronic health record (EHR) databases. A significant challenge in this process is detecting and rejecting unanswerable questions that request information beyond the database's scope or exceed the system's capabilities. In this paper, we introduce a novel text-to-SQL framework that robustly handles out-of-domain questions and verifies the generated queries with query execution.Our framework begins by standardizing the structure of questions into a templated format. We use a powerful large language model (LLM), fine-tuned GPT-3.5 with detailed prompts involving the table schemas of the EHR database system. Our experimental results demonstrate the effectiveness of our framework on the EHRSQL-2024 benchmark benchmark, a shared task in the ClinicalNLP workshop. Although a straightforward fine-tuning of GPT shows promising results on the development set, it struggled with the out-of-domain questions in the test set. With our framework, we improve our system's adaptability and achieve competitive performances in the official leaderboard of the EHRSQL-2024 challenge. △ Less

Submitted 19 June, 2024; v1 submitted 21 May, 2024; originally announced June 2024.

Comments: Published at ClinicalNLP workshop @ NAACL 2024

Open Ko-LLM Leaderboard: Evaluating Large Language Models in Korean with Ko-H5 Benchmark

Authors: Chanjun Park, Hyeonwoo Kim, Dahyun Kim, Seonghwan Cho, Sanghoon Kim, Sukyung Lee, Yungi Kim, Hwalsuk Lee

Abstract: This paper introduces the Open Ko-LLM Leaderboard and the Ko-H5 Benchmark as vital tools for evaluating Large Language Models (LLMs) in Korean. Incorporating private test sets while mirroring the English Open LLM Leaderboard, we establish a robust evaluation framework that has been well integrated in the Korean LLM community. We perform data leakage analysis that shows the benefit of private test… ▽ More This paper introduces the Open Ko-LLM Leaderboard and the Ko-H5 Benchmark as vital tools for evaluating Large Language Models (LLMs) in Korean. Incorporating private test sets while mirroring the English Open LLM Leaderboard, we establish a robust evaluation framework that has been well integrated in the Korean LLM community. We perform data leakage analysis that shows the benefit of private test sets along with a correlation study within the Ko-H5 benchmark and temporal analyses of the Ko-H5 score. Moreover, we present empirical support for the need to expand beyond set benchmarks. We hope the Open Ko-LLM Leaderboard sets precedent for expanding LLM evaluation to foster more linguistic diversity. △ Less

Submitted 30 May, 2024; originally announced May 2024.

Comments: Accepted at ACL 2024 Main

Knockout: A simple way to handle missing inputs

Authors: Minh Nguyen, Batuhan K. Karaman, Heejong Kim, Alan Q. Wang, Fengbei Liu, Mert R. Sabuncu

Abstract: Deep learning models can extract predictive and actionable information from complex inputs. The richer the inputs, the better these models usually perform. However, models that leverage rich inputs (e.g., multi-modality) can be difficult to deploy widely, because some inputs may be missing at inference. Current popular solutions to this problem include marginalization, imputation, and training mul… ▽ More Deep learning models can extract predictive and actionable information from complex inputs. The richer the inputs, the better these models usually perform. However, models that leverage rich inputs (e.g., multi-modality) can be difficult to deploy widely, because some inputs may be missing at inference. Current popular solutions to this problem include marginalization, imputation, and training multiple models. Marginalization can obtain calibrated predictions but it is computationally costly and therefore only feasible for low dimensional inputs. Imputation may result in inaccurate predictions because it employs point estimates for missing variables and does not work well for high dimensional inputs (e.g., images). Training multiple models whereby each model takes different subsets of inputs can work well but requires knowing missing input patterns in advance. Furthermore, training and retaining multiple models can be costly. We propose an efficient way to learn both the conditional distribution using full inputs and the marginal distributions. Our method, Knockout, randomly replaces input features with appropriate placeholder values during training. We provide a theoretical justification of Knockout and show that it can be viewed as an implicit marginalization strategy. We evaluate Knockout in a wide range of simulations and real-world datasets and show that it can offer strong empirical performance. △ Less

Submitted 3 June, 2024; v1 submitted 30 May, 2024; originally announced May 2024.

Evaluating the Effectiveness and Robustness of Visual Similarity-based Phishing Detection Models

Authors: Fujiao Ji, Kiho Lee, Hyungjoon Koo, Wenhao You, Euijin Choo, Hyoungshick Kim, Doowon Kim

Abstract: Phishing attacks pose a significant threat to Internet users, with cybercriminals elaborately replicating the visual appearance of legitimate websites to deceive victims. Visual similarity-based detection systems have emerged as an effective countermeasure, but their effectiveness and robustness in real-world scenarios have been unexplored. In this paper, we comprehensively scrutinize and evaluate… ▽ More Phishing attacks pose a significant threat to Internet users, with cybercriminals elaborately replicating the visual appearance of legitimate websites to deceive victims. Visual similarity-based detection systems have emerged as an effective countermeasure, but their effectiveness and robustness in real-world scenarios have been unexplored. In this paper, we comprehensively scrutinize and evaluate state-of-the-art visual similarity-based anti-phishing models using a large-scale dataset of 450K real-world phishing websites. Our analysis reveals that while certain models maintain high accuracy, others exhibit notably lower performance than results on curated datasets, highlighting the importance of real-world evaluation. In addition, we observe the real-world tactic of manipulating visual components that phishing attackers employ to circumvent the detection systems. To assess the resilience of existing models against adversarial attacks and robustness, we apply visible and perturbation-based manipulations to website logos, which adversaries typically target. We then evaluate the models' robustness in handling these adversarial samples. Our findings reveal vulnerabilities in several models, emphasizing the need for more robust visual similarity techniques capable of withstanding sophisticated evasion attempts. We provide actionable insights for enhancing the security of phishing defense systems, encouraging proactive actions. To the best of our knowledge, this work represents the first large-scale, systematic evaluation of visual similarity-based models for phishing detection in real-world settings, necessitating the development of more effective and robust defenses. △ Less

Submitted 29 May, 2024; originally announced May 2024.

Comments: 12 pages

Robust Optimization in Protein Fitness Landscapes Using Reinforcement Learning in Latent Space

Authors: Minji Lee, Luiz Felipe Vecchietti, Hyunkyu Jung, Hyun Joo Ro, Meeyoung Cha, Ho Min Kim

Abstract: Proteins are complex molecules responsible for different functions in nature. Enhancing the functionality of proteins and cellular fitness can significantly impact various industries. However, protein optimization using computational methods remains challenging, especially when starting from low-fitness sequences. We propose LatProtRL, an optimization method to efficiently traverse a latent space… ▽ More Proteins are complex molecules responsible for different functions in nature. Enhancing the functionality of proteins and cellular fitness can significantly impact various industries. However, protein optimization using computational methods remains challenging, especially when starting from low-fitness sequences. We propose LatProtRL, an optimization method to efficiently traverse a latent space learned by an encoder-decoder leveraging a large protein language model. To escape local optima, our optimization is modeled as a Markov decision process using reinforcement learning acting directly in latent space. We evaluate our approach on two important fitness optimization tasks, demonstrating its ability to achieve comparable or superior fitness over baseline methods. Our findings and in vitro evaluation show that the generated sequences can reach high-fitness regions, suggesting a substantial potential of LatProtRL in lab-in-the-loop scenarios. △ Less

Submitted 29 May, 2024; originally announced May 2024.

Comments: ICML 2024

I See You: Teacher Analytics with GPT-4 Vision-Powered Observational Assessment

Authors: Unggi Lee, Yeil Jeong, Junbo Koh, Gyuri Byun, Yunseo Lee, Hyunwoong Lee, Seunmin Eun, Jewoong Moon, Cheolil Lim, Hyeoncheol Kim

Abstract: This preliminary study explores the integration of GPT-4 Vision (GPT-4V) technology into teacher analytics, focusing on its applicability in observational assessment to enhance reflective teaching practice. This research is grounded in developing a Video-based Automatic Assessment System (VidAAS) empowered by GPT-4V. Our approach aims to revolutionize teachers' assessment of students' practices by… ▽ More This preliminary study explores the integration of GPT-4 Vision (GPT-4V) technology into teacher analytics, focusing on its applicability in observational assessment to enhance reflective teaching practice. This research is grounded in developing a Video-based Automatic Assessment System (VidAAS) empowered by GPT-4V. Our approach aims to revolutionize teachers' assessment of students' practices by leveraging Generative Artificial Intelligence (GenAI) to offer detailed insights into classroom dynamics. Our research methodology encompasses a comprehensive literature review, prototype development of the VidAAS, and usability testing with in-service teachers. The study findings provide future research avenues for VidAAS design, implementation, and integration in teacher analytics, underscoring the potential of GPT-4V to provide real-time, scalable feedback and a deeper understanding of the classroom. △ Less

Submitted 30 May, 2024; v1 submitted 28 May, 2024; originally announced May 2024.

Comments: 27 pages, 5 figures, 4 tables