Showing 1–25 of 25 results for author: Ju, H

BrainFounder: Towards Brain Foundation Models for Neuroimage Analysis

Authors: Joseph Cox, Peng Liu, Skylar E. Stolte, Yunchao Yang, Kang Liu, Kyle B. See, Huiwen Ju, Ruogu Fang

Abstract: The burgeoning field of brain health research increasingly leverages artificial intelligence (AI) to interpret and analyze neurological data. This study introduces a novel approach towards the creation of medical foundation models by integrating a large-scale multi-modal magnetic resonance imaging (MRI) dataset derived from 41,400 participants in its own. Our method involves a novel two-stage pret… ▽ More The burgeoning field of brain health research increasingly leverages artificial intelligence (AI) to interpret and analyze neurological data. This study introduces a novel approach towards the creation of medical foundation models by integrating a large-scale multi-modal magnetic resonance imaging (MRI) dataset derived from 41,400 participants in its own. Our method involves a novel two-stage pretraining approach using vision transformers. The first stage is dedicated to encoding anatomical structures in generally healthy brains, identifying key features such as shapes and sizes of different brain regions. The second stage concentrates on spatial information, encompassing aspects like location and the relative positioning of brain structures. We rigorously evaluate our model, BrainFounder, using the Brain Tumor Segmentation (BraTS) challenge and Anatomical Tracings of Lesions After Stroke v2.0 (ATLAS v2.0) datasets. BrainFounder demonstrates a significant performance gain, surpassing the achievements of the previous winning solutions using fully supervised learning. Our findings underscore the impact of scaling up both the complexity of the model and the volume of unlabeled training data derived from generally healthy brains, which enhances the accuracy and predictive capabilities of the model in complex neuroimaging tasks with MRI. The implications of this research provide transformative insights and practical applications in healthcare and make substantial steps towards the creation of foundation models for Medical AI. Our pretrained models and training code can be found at https://github.com/lab-smile/GatorBrain. △ Less

Submitted 14 June, 2024; originally announced June 2024.

Comments: 17 pages, 5 figures, to be published in Medical Image Analysis

Multi-Domain Recommendation to Attract Users via Domain Preference Modeling

Authors: Hyunjun Ju, SeongKu Kang, Dongha Lee, Junyoung Hwang, Sanghwan Jang, Hwanjo Yu

Abstract: Recently, web platforms have been operating various service domains simultaneously. Targeting a platform that operates multiple service domains, we introduce a new task, Multi-Domain Recommendation to Attract Users (MDRAU), which recommends items from multiple ``unseen'' domains with which each user has not interacted yet, by using knowledge from the user's ``seen'' domains. In this paper, we poin… ▽ More Recently, web platforms have been operating various service domains simultaneously. Targeting a platform that operates multiple service domains, we introduce a new task, Multi-Domain Recommendation to Attract Users (MDRAU), which recommends items from multiple ``unseen'' domains with which each user has not interacted yet, by using knowledge from the user's ``seen'' domains. In this paper, we point out two challenges of MDRAU task. First, there are numerous possible combinations of mappings from seen to unseen domains because users have usually interacted with a different subset of service domains. Second, a user might have different preferences for each of the target unseen domains, which requires that recommendations reflect the user's preferences on domains as well as items. To tackle these challenges, we propose DRIP framework that models users' preferences at two levels (i.e., domain and item) and learns various seen-unseen domain mappings in a unified way with masked domain modeling. Our extensive experiments demonstrate the effectiveness of DRIP in MDRAU task and its ability to capture users' domain-level preferences. △ Less

Submitted 26 March, 2024; originally announced March 2024.

Comments: Accepted to AAAI'24

A Semantic Search Engine for Mathlib4

Authors: Guoxiong Gao, Haocheng Ju, Jiedong Jiang, Zihan Qin, Bin Dong

Abstract: The interactive theorem prover, Lean, enables the verification of formal mathematical proofs and is backed by an expanding community. Central to this ecosystem is its mathematical library, mathlib4, which lays the groundwork for the formalization of an expanding range of mathematical theories. However, searching for theorems in mathlib4 can be challenging. To successfully search in mathlib4, users… ▽ More The interactive theorem prover, Lean, enables the verification of formal mathematical proofs and is backed by an expanding community. Central to this ecosystem is its mathematical library, mathlib4, which lays the groundwork for the formalization of an expanding range of mathematical theories. However, searching for theorems in mathlib4 can be challenging. To successfully search in mathlib4, users often need to be familiar with its naming conventions or documentation strings. Therefore, creating a semantic search engine that can be used easily by individuals with varying familiarity with mathlib4 is very important. In this paper, we present a semantic search engine for mathlib4 that accepts informal queries and finds the relevant theorems. We also establish a benchmark for assessing the performance of various search engines for mathlib4. △ Less

Submitted 20 March, 2024; originally announced March 2024.

NeuJeans: Private Neural Network Inference with Joint Optimization of Convolution and Bootstrapping

Authors: Jae Hyung Ju, Jaiyoung Park, Jongmin Kim, Donghwan Kim, Jung Ho Ahn

Abstract: Fully homomorphic encryption (FHE) is a promising cryptographic primitive for realizing private neural network inference (PI) services by allowing a client to fully offload the inference task to a cloud server while keeping the client data oblivious to the server. This work proposes NeuJeans, an FHE-based solution for the PI of deep convolutional neural networks (CNNs). NeuJeans tackles the critic… ▽ More Fully homomorphic encryption (FHE) is a promising cryptographic primitive for realizing private neural network inference (PI) services by allowing a client to fully offload the inference task to a cloud server while keeping the client data oblivious to the server. This work proposes NeuJeans, an FHE-based solution for the PI of deep convolutional neural networks (CNNs). NeuJeans tackles the critical problem of the enormous computational cost for the FHE evaluation of convolutional layers (conv2d), mainly due to the high cost of data reordering and bootstrapping. We first propose an encoding method introducing nested structures inside encoded vectors for FHE, which enables us to develop efficient conv2d algorithms with reduced data reordering costs. However, the new encoding method also introduces additional computations for conversion between encoding methods, which could negate its advantages. We discover that fusing conv2d with bootstrapping eliminates such computations while reducing the cost of bootstrapping. Then, we devise optimized execution flows for various types of conv2d and apply them to end-to-end implementation of CNNs. NeuJeans accelerates the performance of conv2d by up to 5.68 times compared to state-of-the-art FHE-based PI work and performs the PI of a CNN at the scale of ImageNet (ResNet18) within a mere few seconds △ Less

Submitted 7 December, 2023; originally announced December 2023.

Comments: 16 pages, 9 figures

Graph Neural Networks for Road Safety Modeling: Datasets and Evaluations for Accident Analysis

Authors: Abhinav Nippani, Dongyue Li, Haotian Ju, Haris N. Koutsopoulos, Hongyang R. Zhang

Abstract: We consider the problem of traffic accident analysis on a road network based on road network connections and traffic volume. Previous works have designed various deep-learning methods using historical records to predict traffic accident occurrences. However, there is a lack of consensus on how accurate existing methods are, and a fundamental issue is the lack of public accident datasets for compre… ▽ More We consider the problem of traffic accident analysis on a road network based on road network connections and traffic volume. Previous works have designed various deep-learning methods using historical records to predict traffic accident occurrences. However, there is a lack of consensus on how accurate existing methods are, and a fundamental issue is the lack of public accident datasets for comprehensive evaluations. This paper constructs a large-scale, unified dataset of traffic accident records from official reports of various states in the US, totaling 9 million records, accompanied by road networks and traffic volume reports. Using this new dataset, we evaluate existing deep-learning methods for predicting the occurrence of accidents on road networks. Our main finding is that graph neural networks such as GraphSAGE can accurately predict the number of accidents on roads with less than 22% mean absolute error (relative to the actual count) and whether an accident will occur or not with over 87% AUROC, averaged over states. We achieve these results by using multitask learning to account for cross-state variabilities (e.g., availability of accident labels) and transfer learning to combine traffic volume with accident prediction. Ablation studies highlight the importance of road graph-structural features, amongst other features. Lastly, we discuss the implications of the analysis and develop a package for easily using our new dataset. △ Less

Submitted 12 February, 2024; v1 submitted 31 October, 2023; originally announced November 2023.

Comments: 24 pages. Appeared in NeurIPS 2023 Datasets Track

Boosting Multitask Learning on Graphs through Higher-Order Task Affinities

Authors: Dongyue Li, Haotian Ju, Aneesh Sharma, Hongyang R. Zhang

Abstract: Predicting node labels on a given graph is a widely studied problem with many applications, including community detection and molecular graph prediction. This paper considers predicting multiple node labeling functions on graphs simultaneously and revisits this problem from a multitask learning perspective. For a concrete example, consider overlapping community detection: each community membership… ▽ More Predicting node labels on a given graph is a widely studied problem with many applications, including community detection and molecular graph prediction. This paper considers predicting multiple node labeling functions on graphs simultaneously and revisits this problem from a multitask learning perspective. For a concrete example, consider overlapping community detection: each community membership is a binary node classification task. Due to complex overlapping patterns, we find that negative transfer is prevalent when we apply naive multitask learning to multiple community detection, as task relationships are highly nonlinear across different node labeling. To address the challenge, we develop an algorithm to cluster tasks into groups based on a higher-order task affinity measure. We then fit a multitask model on each task group, resulting in a boosting procedure on top of the baseline model. We estimate the higher-order task affinity measure between two tasks as the prediction loss of one task in the presence of another task and a random subset of other tasks. Then, we use spectral clustering on the affinity score matrix to identify task grouping. We design several speedup techniques to compute the higher-order affinity scores efficiently and show that they can predict negative transfers more accurately than pairwise task affinities. We validate our procedure using various community detection and molecular graph prediction data sets, showing favorable results compared with existing methods. Lastly, we provide a theoretical analysis to show that under a planted block model of tasks on graphs, our affinity scores can provably separate tasks into groups. △ Less

Submitted 14 March, 2024; v1 submitted 24 June, 2023; originally announced June 2023.

Comments: 16 pages. Appeared in KDD 2023

Noise Stability Optimization for Flat Minima with Tight Rates

Authors: Haotian Ju, Dongyue Li, Hongyang R. Zhang

Abstract: We consider minimizing a perturbed function $F(W) = \mathbb{E}_{U}[f(W + U)]$, given a function $f: \mathbb{R}^d \rightarrow \mathbb{R}$ and a random sample $U$ from a distribution $\mathcal{P}$ with mean zero. When $\mathcal{P}$ is the isotropic Gaussian, $F(W)$ is roughly equal to $f(W)$ plus a penalty on the trace of $\nabla^2 f(W)$, scaled by the variance of $\mathcal{P}$. This penalty on the… ▽ More We consider minimizing a perturbed function $F(W) = \mathbb{E}_{U}[f(W + U)]$, given a function $f: \mathbb{R}^d \rightarrow \mathbb{R}$ and a random sample $U$ from a distribution $\mathcal{P}$ with mean zero. When $\mathcal{P}$ is the isotropic Gaussian, $F(W)$ is roughly equal to $f(W)$ plus a penalty on the trace of $\nabla^2 f(W)$, scaled by the variance of $\mathcal{P}$. This penalty on the Hessian has the benefit of improving generalization, through PAC-Bayes analysis. It is useful in low-sample regimes, for instance, when a (large) pre-trained model is fine-tuned on a small data set. One way to minimize $F$ is by adding $U$ to $W$, and then run SGD. We observe, empirically, that this noise injection does not provide significant gains over SGD, in our experiments of conducting fine-tuning on three image classification data sets. We design a simple, practical algorithm that adds noise along both $U$ and $-U$, with the option of adding several perturbations and taking their average. We analyze the convergence of this algorithm, showing tight rates on the norm of the output's gradient. We provide a comprehensive empirical analysis of our algorithm, by first showing that in an over-parameterized matrix sensing problem, it can find solutions with lower test loss than naive noise injection. Then, we compare our algorithm with four sharpness-reducing training methods (such as the Sharpness-Aware Minimization (Foret et al., 2021)). We find that our algorithm can outperform them by up to 1.8% test accuracy, for fine-tuning ResNet on six image classification data sets. It leads to a 17.7% (and 12.8%) reduction in the trace (and largest eigenvalue) of the Hessian matrix of the loss surface. This form of regularization on the Hessian is compatible with $\ell_2$ weight decay (and data augmentation), in the sense that combining both can lead to improved empirical performance. △ Less

Submitted 18 April, 2024; v1 submitted 14 June, 2023; originally announced June 2023.

Comments: 36 pages, 3 tables

CoTran: An LLM-based Code Translator using Reinforcement Learning with Feedback from Compiler and Symbolic Execution

Authors: Prithwish Jana, Piyush Jha, Haoyang Ju, Gautham Kishore, Aryan Mahajan, Vijay Ganesh

Abstract: In this paper, we present an LLM-based code translation method and an associated tool called CoTran, that translates whole-programs from one high-level programming language to another. Current LLM-based code translation methods lack a training approach to ensure that the translated code reliably compiles or bears substantial functional equivalence to the input code. In our work, we train an LLM vi… ▽ More In this paper, we present an LLM-based code translation method and an associated tool called CoTran, that translates whole-programs from one high-level programming language to another. Current LLM-based code translation methods lack a training approach to ensure that the translated code reliably compiles or bears substantial functional equivalence to the input code. In our work, we train an LLM via reinforcement learning, by modifying the fine-tuning process to incorporate compiler feedback and symbolic execution (symexec)-based equivalence testing feedback that checks for functional equivalence between the input and output programs. The idea is to guide an LLM-in-training, via compiler and symexec-based testing feedback, by letting it know how far it is from producing perfect translations. We report on extensive experiments comparing CoTran with 14 other code translation tools that include human-written transpilers, LLM-based translation tools, and ChatGPT over a benchmark of more than 57,000 Java-Python equivalent pairs, and we show that CoTran outperforms them on relevant metrics such as compilation accuracy (CompAcc) and functional equivalence accuracy (FEqAcc). For example, our tool achieves 48.68% FEqAcc, 76.98% CompAcc for Python-to-Java translation, whereas the nearest competing tool (PLBART-base) only gets 38.26% and 75.77% resp. Also, built upon CodeT5, CoTran achieves +11.23%, +14.89% improvement on FEqAcc and +4.07%, +8.14% on CompAcc for Java-to-Python and Python-to-Java translation resp. △ Less

Submitted 16 January, 2024; v1 submitted 11 June, 2023; originally announced June 2023.

ACM Class: I.2.7; I.2.5; D.2

A Comparative Study of Deep Learning and Iterative Algorithms for Joint Channel Estimation and Signal Detection in OFDM Systems

Authors: Haocheng Ju, Haimiao Zhang, Lin Li, Xiao Li, Bin Dong

Abstract: Joint channel estimation and signal detection (JCESD) is crucial in orthogonal frequency division multiplexing (OFDM) systems, but traditional algorithms perform poorly in low signal-to-noise ratio (SNR) scenarios. Deep learning (DL) methods have been investigated, but concerns regarding computational expense and lack of validation in low-SNR settings remain. Hence, the development of a robust and… ▽ More Joint channel estimation and signal detection (JCESD) is crucial in orthogonal frequency division multiplexing (OFDM) systems, but traditional algorithms perform poorly in low signal-to-noise ratio (SNR) scenarios. Deep learning (DL) methods have been investigated, but concerns regarding computational expense and lack of validation in low-SNR settings remain. Hence, the development of a robust and low-complexity model that can deliver excellent performance across a wide range of SNRs is highly desirable. In this paper, we aim to establish a benchmark where traditional algorithms and DL methods are validated on different channel models, Doppler, and SNR settings, particularly focusing on the semi-blind setting. In particular, we propose a new DL model where the backbone network is formed by unrolling the iterative algorithm, and the hyperparameters are estimated by hypernetworks. Additionally, we adapt a lightweight DenseNet to the task of JCESD for comparison. We evaluate different methods in three aspects: generalization in terms of bit error rate (BER), robustness, and complexity. Our results indicate that DL approaches outperform traditional algorithms in the challenging low-SNR setting, while the iterative algorithm performs better in high-SNR settings. Furthermore, the iterative algorithm is more robust in the presence of carrier frequency offset, whereas DL methods excel when signals are corrupted by asymmetric Gaussian noise. △ Less

Submitted 20 June, 2024; v1 submitted 7 March, 2023; originally announced March 2023.

Comments: Code is available at https://github.com/j991222/MIMO_JCESD

Journal ref: Signal Processing 223 (2024), 109554

Generalization in Graph Neural Networks: Improved PAC-Bayesian Bounds on Graph Diffusion

Authors: Haotian Ju, Dongyue Li, Aneesh Sharma, Hongyang R. Zhang

Abstract: Graph neural networks are widely used tools for graph prediction tasks. Motivated by their empirical performance, prior works have developed generalization bounds for graph neural networks, which scale with graph structures in terms of the maximum degree. In this paper, we present generalization bounds that instead scale with the largest singular value of the graph neural network's feature diffusi… ▽ More Graph neural networks are widely used tools for graph prediction tasks. Motivated by their empirical performance, prior works have developed generalization bounds for graph neural networks, which scale with graph structures in terms of the maximum degree. In this paper, we present generalization bounds that instead scale with the largest singular value of the graph neural network's feature diffusion matrix. These bounds are numerically much smaller than prior bounds for real-world graphs. We also construct a lower bound of the generalization gap that matches our upper bound asymptotically. To achieve these results, we analyze a unified model that includes prior works' settings (i.e., convolutional and message-passing networks) and new settings (i.e., graph isomorphism networks). Our key idea is to measure the stability of graph neural networks against noise perturbations using Hessians. Empirically, we find that Hessian-based measurements correlate with the observed generalization gaps of graph neural networks accurately. Optimizing noise stability properties for fine-tuning pretrained graph neural networks also improves test performance on several graph-level classification tasks. △ Less

Submitted 23 October, 2023; v1 submitted 9 February, 2023; originally announced February 2023.

Comments: 36 pages. Appeared in AISTATS 2023

Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guarantees

Authors: Haotian Ju, Dongyue Li, Hongyang R. Zhang

Abstract: We consider fine-tuning a pretrained deep neural network on a target task. We study the generalization properties of fine-tuning to understand the problem of overfitting, which has often been observed (e.g., when the target dataset is small or when the training labels are noisy). Existing generalization measures for deep networks depend on notions such as distance from the initialization (i.e., th… ▽ More We consider fine-tuning a pretrained deep neural network on a target task. We study the generalization properties of fine-tuning to understand the problem of overfitting, which has often been observed (e.g., when the target dataset is small or when the training labels are noisy). Existing generalization measures for deep networks depend on notions such as distance from the initialization (i.e., the pretrained network) of the fine-tuned model and noise stability properties of deep networks. This paper identifies a Hessian-based distance measure through PAC-Bayesian analysis, which is shown to correlate well with observed generalization gaps of fine-tuned models. Theoretically, we prove Hessian distance-based generalization bounds for fine-tuned models. We also describe an extended study of fine-tuning against label noise, where overfitting remains a critical problem. We present an algorithm and a generalization error guarantee for this algorithm under a class conditional independent noise model. Empirically, we observe that the Hessian-based distance measure can match the scale of the observed generalization gap of fine-tuned models in practice. We also test our algorithm on several image classification tasks with noisy training labels, showing gains over prior methods and decreases in the Hessian distance measure of the fine-tuned model. △ Less

Submitted 22 December, 2023; v1 submitted 6 June, 2022; originally announced June 2022.

Comments: 38 pages. Appeared in ICML 2022

Energy-Efficient Ultra-Dense Network with Deep Reinforcement Learning

Authors: Hyungyu Ju, Seungnyun Kim, Youngjoon Kim, Byonghyo Shim

Abstract: With the explosive growth in mobile data traffic, ultra-dense network (UDN) where a large number of small cells are densely deployed on top of macro cells has received a great deal of attention in recent years. While UDN offers a number of benefits, an upsurge of energy consumption in UDN due to the intensive deployment of small cells has now become a major bottleneck in achieving the primary goal… ▽ More With the explosive growth in mobile data traffic, ultra-dense network (UDN) where a large number of small cells are densely deployed on top of macro cells has received a great deal of attention in recent years. While UDN offers a number of benefits, an upsurge of energy consumption in UDN due to the intensive deployment of small cells has now become a major bottleneck in achieving the primary goals viz., 100-fold increase in the throughput in 5G+ and 6G. In recent years, an approach to reduce the energy consumption of base stations (BSs) by selectively turning off the lightly-loaded BSs, referred to as the sleep mode technique, has been suggested. However, determining the appropriate active/sleep modes of BSs is a difficult task due to the huge computational overhead and inefficiency caused by the frequent BS mode conversion. An aim of this paper is to propose a deep reinforcement learning (DRL)-based approach to achieve a reduction of energy consumption in UDN. Key ingredient of the proposed scheme is to use decision selection network to reduce the size of action space. Numerical results show that the proposed scheme can significantly reduce the energy consumption of UDN while ensuring the rate requirement of network. △ Less

Submitted 25 December, 2021; originally announced December 2021.

Comments: Submitted to IEEE Transactions on Wireless Communications. Copyright 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses

Weakly Supervised Temporal Anomaly Segmentation with Dynamic Time Warping

Authors: Dongha Lee, Sehun Yu, Hyunjun Ju, Hwanjo Yu

Abstract: Most recent studies on detecting and localizing temporal anomalies have mainly employed deep neural networks to learn the normal patterns of temporal data in an unsupervised manner. Unlike them, the goal of our work is to fully utilize instance-level (or weak) anomaly labels, which only indicate whether any anomalous events occurred or not in each instance of temporal data. In this paper, we prese… ▽ More Most recent studies on detecting and localizing temporal anomalies have mainly employed deep neural networks to learn the normal patterns of temporal data in an unsupervised manner. Unlike them, the goal of our work is to fully utilize instance-level (or weak) anomaly labels, which only indicate whether any anomalous events occurred or not in each instance of temporal data. In this paper, we present WETAS, a novel framework that effectively identifies anomalous temporal segments (i.e., consecutive time points) in an input instance. WETAS learns discriminative features from the instance-level labels so that it infers the sequential order of normal and anomalous segments within each instance, which can be used as a rough segmentation mask. Based on the dynamic time warping (DTW) alignment between the input instance and its segmentation mask, WETAS obtains the result of temporal segmentation, and simultaneously, it further enhances itself by using the mask as additional supervision. Our experiments show that WETAS considerably outperforms other baselines in terms of the localization of temporal anomalies, and also it provides more informative results than point-level detection methods. △ Less

Submitted 15 August, 2021; originally announced August 2021.

Comments: ICCV 2021. 8 pages, References (2 pages), Appendix (3 pages), 6 figures

Bootstrapping User and Item Representations for One-Class Collaborative Filtering

Authors: Dongha Lee, SeongKu Kang, Hyunjun Ju, Chanyoung Park, Hwanjo Yu

Abstract: The goal of one-class collaborative filtering (OCCF) is to identify the user-item pairs that are positively-related but have not been interacted yet, where only a small portion of positive user-item interactions (e.g., users' implicit feedback) are observed. For discriminative modeling between positive and negative interactions, most previous work relied on negative sampling to some extent, which… ▽ More The goal of one-class collaborative filtering (OCCF) is to identify the user-item pairs that are positively-related but have not been interacted yet, where only a small portion of positive user-item interactions (e.g., users' implicit feedback) are observed. For discriminative modeling between positive and negative interactions, most previous work relied on negative sampling to some extent, which refers to considering unobserved user-item pairs as negative, as actual negative ones are unknown. However, the negative sampling scheme has critical limitations because it may choose "positive but unobserved" pairs as negative. This paper proposes a novel OCCF framework, named as BUIR, which does not require negative sampling. To make the representations of positively-related users and items similar to each other while avoiding a collapsed solution, BUIR adopts two distinct encoder networks that learn from each other; the first encoder is trained to predict the output of the second encoder as its target, while the second encoder provides the consistent targets by slowly approximating the first encoder. In addition, BUIR effectively alleviates the data sparsity issue of OCCF, by applying stochastic data augmentation to encoder inputs. Based on the neighborhood information of users and items, BUIR randomly generates the augmented views of each positive interaction each time it encodes, then further trains the model by this self-supervision. Our extensive experiments demonstrate that BUIR consistently and significantly outperforms all baseline methods by a large margin especially for much sparse datasets in which any assumptions about negative interactions are less valid. △ Less

Submitted 13 May, 2021; originally announced May 2021.

Comments: SIGIR 2021. 9 pages + references (1 page). 6 figures

Highly-scalable stochastic neuron based on Ovonic Threshold Switch (OTS) and its applications in Restricted Boltzmann Machine (RBM)

Authors: Seong-il Im, Hyejin Lee, Jaesang Lee, Jae-Seung Jeong, Joon Young Kwak, Keunsu Kim, Jeong Ho Cho, Hyunsu Ju, Suyoun Lee

Abstract: Interest in Restricted Boltzmann Machine (RBM) is growing as a generative stochastic artificial neural network to implement a novel energy-efficient machine-learning (ML) technique. For a hardware implementation of the RBM, an essential building block is a reliable stochastic binary neuron device that generates random spikes following the Boltzmann distribution. Here, we propose a highly-scalable… ▽ More Interest in Restricted Boltzmann Machine (RBM) is growing as a generative stochastic artificial neural network to implement a novel energy-efficient machine-learning (ML) technique. For a hardware implementation of the RBM, an essential building block is a reliable stochastic binary neuron device that generates random spikes following the Boltzmann distribution. Here, we propose a highly-scalable stochastic neuron device based on Ovonic Threshold Switch (OTS) which utilizes the random emission and capture process of traps as the source of stochasticity. The switching probability is well described by the Boltzmann distribution, which can be controlled by operating parameters. As a candidate for a true random number generator (TRNG), it passes 15 among the 16 tests of the National Institute of Standards and Technology (NIST) Statistical Test Suite (Special Publication 800-22). In addition, the recognition task of handwritten digits (MNIST) is demonstrated using a simulated RBM network consisting of the proposed device with a maximum recognition accuracy of 86.07 %. Furthermore, reconstruction of images is successfully demonstrated using images contaminated with noises, resulting in images with the noise removed. These results show the promising properties of OTS-based stochastic neuron devices for applications in RBM systems. △ Less

Submitted 21 October, 2020; originally announced October 2020.

The network structure of scientific revolutions

Authors: Harang Ju, Dale Zhou, Ann S. Blevins, David M. Lydon-Staley, Judith Kaplan, Julio R. Tuma, Danielle S. Bassett

Abstract: Philosophers of science have long postulated how collective scientific knowledge grows. Empirical validation has been challenging due to limitations in collecting and systematizing large historical records. Here, we capitalize on the largest online encyclopedia to formulate knowledge as growing networks of articles and their hyperlinked inter-relations. We demonstrate that concept networks grow no… ▽ More Philosophers of science have long postulated how collective scientific knowledge grows. Empirical validation has been challenging due to limitations in collecting and systematizing large historical records. Here, we capitalize on the largest online encyclopedia to formulate knowledge as growing networks of articles and their hyperlinked inter-relations. We demonstrate that concept networks grow not by expanding from their core but rather by creating and filling knowledge gaps, a process which produces discoveries that are more frequently awarded Nobel prizes than others. Moreover, we operationalize paradigms as network modules to reveal a temporal signature in structural stability across scientific subjects. In a network formulation of scientific discovery, data-driven conditions underlying breakthroughs depend just as much on identifying uncharted gaps as on advancing solutions within scientific communities. △ Less

Submitted 10 December, 2020; v1 submitted 16 October, 2020; originally announced October 2020.

An Efficient Planar Bundle Adjustment Algorithm

Authors: Lipu Zhou, Daniel Koppel, Hui Ju, Frank Steinbruecker, Michael Kaess

Abstract: This paper presents an efficient algorithm for the least-squares problem using the point-to-plane cost, which aims to jointly optimize depth sensor poses and plane parameters for 3D reconstruction. We call this least-squares problem \textbf{Planar Bundle Adjustment} (PBA), due to the similarity between this problem and the original Bundle Adjustment (BA) in visual reconstruction. As planes ubiquit… ▽ More This paper presents an efficient algorithm for the least-squares problem using the point-to-plane cost, which aims to jointly optimize depth sensor poses and plane parameters for 3D reconstruction. We call this least-squares problem \textbf{Planar Bundle Adjustment} (PBA), due to the similarity between this problem and the original Bundle Adjustment (BA) in visual reconstruction. As planes ubiquitously exist in the man-made environment, they are generally used as landmarks in SLAM algorithms for various depth sensors. PBA is important to reduce drift and improve the quality of the map. However, directly adopting the well-established BA framework in visual reconstruction will result in a very inefficient solution for PBA. This is because a 3D point only has one observation at a camera pose. In contrast, a depth sensor can record hundreds of points in a plane at a time, which results in a very large nonlinear least-squares problem even for a small-scale space. Fortunately, we find that there exist a special structure of the PBA problem. We introduce a reduced Jacobian matrix and a reduced residual vector, and prove that they can replace the original Jacobian matrix and residual vector in the generally adopted Levenberg-Marquardt (LM) algorithm. This significantly reduces the computational cost. Besides, when planes are combined with other features for 3D reconstruction, the reduced Jacobian matrix and residual vector can also replace the corresponding parts derived from planes. Our experimental results verify that our algorithm can significantly reduce the computational time compared to the solution using the traditional BA framework. Besides, our algorithm is faster, more accuracy, and more robust to initialization errors compared to the start-of-the-art solution using the plane-to-plane cost △ Less

Submitted 16 August, 2020; v1 submitted 30 May, 2020; originally announced June 2020.

Magnetic skyrmion artificial synapse for neuromorphic computing

Authors: Kyung Mee Song, Jae-Seung Jeong, Biao Pan, Xichao Zhang, Jing Xia, Sun Kyung Cha, Tae-Eon Park, Kwangsu Kim, Simone Finizio, Joerg Raabe, Joonyeon Chang, Yan Zhou, Weisheng Zhao, Wang Kang, Hyunsu Ju, Seonghoon Woo

Abstract: Since the experimental discovery of magnetic skyrmions achieved one decade ago, there have been significant efforts to bring the virtual particles into all-electrical fully functional devices, inspired by their fascinating physical and topological properties suitable for future low-power electronics. Here, we experimentally demonstrate such a device: electrically-operating skyrmion-based artificia… ▽ More Since the experimental discovery of magnetic skyrmions achieved one decade ago, there have been significant efforts to bring the virtual particles into all-electrical fully functional devices, inspired by their fascinating physical and topological properties suitable for future low-power electronics. Here, we experimentally demonstrate such a device: electrically-operating skyrmion-based artificial synaptic device designed for neuromorphic computing. We present that controlled current-induced creation, motion, detection and deletion of skyrmions in ferrimagnetic multilayers can be harnessed in a single device at room temperature to imitate the behaviors of biological synapses. Using simulations, we demonstrate that such skyrmion-based synapses could be used to perform neuromorphic pattern-recognition computing using handwritten recognition data set, reaching to the accuracy of ~89 percents, comparable to the software-based training accuracy of ~94 percents. Chip-level simulation then highlights the potential of skyrmion synapse compared to existing technologies. Our findings experimentally illustrate the basic concepts of skyrmion-based fully functional electronic devices while providing a new building block in the emerging field of spintronics-based bio-inspired computing. △ Less

Submitted 30 September, 2019; v1 submitted 1 July, 2019; originally announced July 2019.

Comments: 11 pages, 4 figures

Journal ref: Nature Electronics 3, 148 (2020)

CURE: Curvature Regularization For Missing Data Recovery

Authors: Bin Dong, Haocheng Ju, Yiping Lu, Zuoqiang Shi

Abstract: Missing data recovery is an important and yet challenging problem in imaging and data science. Successful models often adopt certain carefully chosen regularization. Recently, the low dimension manifold model (LDMM) was introduced by S.Osher et al. and shown effective in image inpainting. They observed that enforcing low dimensionality on image patch manifold serves as a good image regularizer. In… ▽ More Missing data recovery is an important and yet challenging problem in imaging and data science. Successful models often adopt certain carefully chosen regularization. Recently, the low dimension manifold model (LDMM) was introduced by S.Osher et al. and shown effective in image inpainting. They observed that enforcing low dimensionality on image patch manifold serves as a good image regularizer. In this paper, we observe that having only the low dimension manifold regularization is not enough sometimes, and we need smoothness as well. For that, we introduce a new regularization by combining the low dimension manifold regularization with a higher order Curvature Regularization, and we call this new regularization CURE for short. The key step of solving CURE is to solve a biharmonic equation on a manifold. We further introduce a weighted version of CURE, called WeCURE, in a similar manner as the weighted nonlocal Laplacian (WNLL) method. Numerical experiments for image inpainting and semi-supervised learning show that the proposed CURE and WeCURE significantly outperform LDMM and WNLL respectively. △ Less

Submitted 25 November, 2019; v1 submitted 28 January, 2019; originally announced January 2019.

Comments: 17 pages, 7 figures, 4 tables

A highly scalable and energy-efficient artificial neuron using an Ovonic Threshold Switch (OTS) featuring the spike-frequency adaptation and chaotic activity

Authors: Milim Lee, Youngjo Kim, Seong Won Cho, Joon Young Kwak, Hyunsu Ju, Yeonjin Yi, Byung-ki Cheong, Suyoun Lee

Abstract: As an essential building block for developing a large-scale brain-inspired computing system, we present a highly scalable and energy-efficient artificial neuron device composed of an Ovonic Threshold Switch (OTS) and a few passive electrical components. It shows not only the basic integrate-and-fire (I&F) function and the rate coding ability, but also the spike-frequency adaptation (SFA) property… ▽ More As an essential building block for developing a large-scale brain-inspired computing system, we present a highly scalable and energy-efficient artificial neuron device composed of an Ovonic Threshold Switch (OTS) and a few passive electrical components. It shows not only the basic integrate-and-fire (I&F) function and the rate coding ability, but also the spike-frequency adaptation (SFA) property and the chaotic activity. The latter two, being the most common features found in the mammalian cortex, are particularly essential for the realization of the energy-efficient signal processing, learning, and adaptation to environments1-3, but have been hard to achieve up to now. Furthermore, with our OTS-based neuron device employing the reservoir computing technique combined with delayed feedback dynamics, spoken-digit recognition task has been performed with a considerable degree of recognition accuracy. From a comparison with a Mott memristor-based artificial neuron device, it is shown that the OTS-based artificial neuron is much more energy-efficient by about 100 times. These results show that our OTS-based artificial neuron device is promising for the application in the development of a large-scale brain-inspired computing system. △ Less

Submitted 19 October, 2018; originally announced October 2018.

Journal ref: Phys. Rev. Applied 13, 064056 (2020)

Full-Duplex Operations in Wireless Powered Communication Networks

Authors: Hyungsik Ju, Yuro Lee, Tae-Joong Kim

Abstract: In this paper, a wireless powered communication network (WPCN) consisting of a hybrid access point (H-AP) and multiple user equipment (UEs), all of which operate in full-duplex (FD), is described. We first propose a transceiver structure that enables FD operation of each UE to simultaneously receive energy in the downlink (DL) and transmit information in the uplink (UL). We then provide an energy… ▽ More In this paper, a wireless powered communication network (WPCN) consisting of a hybrid access point (H-AP) and multiple user equipment (UEs), all of which operate in full-duplex (FD), is described. We first propose a transceiver structure that enables FD operation of each UE to simultaneously receive energy in the downlink (DL) and transmit information in the uplink (UL). We then provide an energy usage model in the proposed UE transceiver that accounts for the energy leakage from the transmit chain to the receive chain. It is shown that the throughput of an FD WPCN using the proposed FD UEs can be maximized by optimally allocating the UL transmission time to the UEs by solving a convex optimization problem. Simulation results reveal that the use of the proposed FD UEs efficiently improves the throughput of a WPCN with practical self-interference cancellation (SIC) capability at the H-AP. With current SIC technologies reducing the power of the residual self-interference to the level of background noise, the proposed FD WPCN using FD UEs achieves 18% and 25 % of throughput gain as compared to the conventional FD WPCN using HD UEs and HD WPCN, respectively. △ Less

Submitted 26 June, 2017; v1 submitted 3 March, 2017; originally announced March 2017.

Comments: 7 pages, 7 figures

User Cooperation in Wireless Powered Communication Networks

Authors: Hyungsik Ju, Rui Zhang

Abstract: This paper studies user cooperation in the emerging wireless powered communication network (WPCN) for throughput optimization. For the purpose of exposition, we consider a two-user WPCN, in which one hybrid access point (H-AP) broadcasts wireless energy to two distributed users in the downlink (DL) and the users transmit their independent information using their individually harvested energy to th… ▽ More This paper studies user cooperation in the emerging wireless powered communication network (WPCN) for throughput optimization. For the purpose of exposition, we consider a two-user WPCN, in which one hybrid access point (H-AP) broadcasts wireless energy to two distributed users in the downlink (DL) and the users transmit their independent information using their individually harvested energy to the H-AP in the uplink (UL) through time-division-multiple-access (TDMA). We propose user cooperation in the WPCN where the user which is nearer to the H-AP and has a better channel for DL energy harvesting and UL information transmission uses part of its allocated UL time and DL harvested energy to help to relay the far user's information to the H-AP, in order to achieve more balanced throughput optimization. We maximize the weighted sum-rate (WSR) of the two users by jointly optimizing the time and power allocations in the network for both wireless energy transfer in the DL and wireless information transmission and relaying in the UL. Simulation results show that the proposed user cooperation scheme can effectively improve the achievable throughput in the WPCN with desired user fairness. △ Less

Submitted 7 April, 2014; v1 submitted 27 March, 2014; originally announced March 2014.

Comments: 7 figures

Optimal Resource Allocation in Full-Duplex Wireless-Powered Communication Network

Authors: Hyungsik Ju, Rui Zhang

Abstract: This paper studies optimal resource allocation in the wireless-powered communication network (WPCN), where one hybrid access-point (H-AP) operating in full-duplex (FD) broadcasts wireless energy to a set of distributed users in the downlink (DL) and at the same time receives independent information from the users via time-division-multiple-access (TDMA) in the uplink (UL). We design an efficient p… ▽ More This paper studies optimal resource allocation in the wireless-powered communication network (WPCN), where one hybrid access-point (H-AP) operating in full-duplex (FD) broadcasts wireless energy to a set of distributed users in the downlink (DL) and at the same time receives independent information from the users via time-division-multiple-access (TDMA) in the uplink (UL). We design an efficient protocol to support simultaneous wireless energy transfer (WET) in the DL and wireless information transmission (WIT) in the UL for the proposed FD-WPCN. We jointly optimize the time allocations to the H-AP for DL WET and different users for UL WIT as well as the transmit power allocations over time at the H-AP to maximize the users' weighted sum-rate of UL information transmission with harvested energy. We consider both the cases with perfect and imperfect self-interference cancellation (SIC) at the H-AP, for which we obtain optimal and suboptimal time and power allocation solutions, respectively. Furthermore, we consider the half-duplex (HD) WPCN as a baseline scheme and derive its optimal resource allocation solution. Simulation results show that the FD-WPCN outperforms HD-WPCN when effective SIC can be implemented and more stringent peak power constraint is applied at the H-AP. △ Less

Submitted 15 September, 2014; v1 submitted 11 March, 2014; originally announced March 2014.

Comments: 31 pages, 10 figures, 2 tables

Throughput Maximization in Wireless Powered Communication Networks

Authors: Hyunsgsik Ju, Rui Zhang

Abstract: This paper studies the newly emerging wireless powered communication network (WPCN) in which one hybrid access point (H-AP) with constant power supply coordinates the wireless energy/information transmissions to/from distributed users that do not have energy sources. A "harvest-then-transmit" protocol is proposed where all users first harvest the wireless energy broadcast by the H-AP in the downli… ▽ More This paper studies the newly emerging wireless powered communication network (WPCN) in which one hybrid access point (H-AP) with constant power supply coordinates the wireless energy/information transmissions to/from distributed users that do not have energy sources. A "harvest-then-transmit" protocol is proposed where all users first harvest the wireless energy broadcast by the H-AP in the downlink (DL) and then send their independent information to the H-AP in the uplink (UL) by time-division-multiple-access (TDMA). First, we study the sum-throughput maximization of all users by jointly optimizing the time allocation for the DL wireless power transfer versus the users' UL information transmissions given a total time constraint based on the users' DL and UL channels as well as their average harvested energy values. By applying convex optimization techniques, we obtain the closed-form expressions for the optimal time allocations to maximize the sum-throughput. Our solution reveals "doubly near-far" phenomenon due to both the DL and UL distance-dependent signal attenuation, where a far user from the H-AP, which receives less wireless energy than a nearer user in the DL, has to transmit with more power in the UL for reliable information transmission. Consequently, the maximum sum-throughput is achieved by allocating substantially more time to the near users than the far users, thus resulting in unfair rate allocation among different users. To overcome this problem, we furthermore propose a new performance metric so-called common-throughput with the additional constraint that all users should be allocated with an equal rate regardless of their distances to the H-AP. We present an efficient algorithm to solve the common-throughput maximization problem. Simulation results demonstrate the effectiveness of the common-throughput approach for solving the new doubly near-far problem in WPCNs. △ Less

Submitted 21 July, 2014; v1 submitted 30 April, 2013; originally announced April 2013.

Comments: 26 pages, 12 figures

Journal ref: IEEE Transactions on Wireless Communications, vol. 13, no. 1, pp. 418-428, Jan. 2014

A Novel Mode Switching Scheme Utilizing Random Beamforming for Opportunistic Energy Harvesting

Authors: Hyungsik Ju, Rui Zhang

Abstract: Since radio signals carry both energy and information at the same time, a unified study on simultaneous wireless information and power transfer (SWIPT) has recently drawn a significant attention for achieving wireless powered communication networks. In this paper, we study a multiple-input single-output (MISO) multicast SWIPT network with one multi-antenna transmitter sending common information to… ▽ More Since radio signals carry both energy and information at the same time, a unified study on simultaneous wireless information and power transfer (SWIPT) has recently drawn a significant attention for achieving wireless powered communication networks. In this paper, we study a multiple-input single-output (MISO) multicast SWIPT network with one multi-antenna transmitter sending common information to multiple single-antenna receivers simultaneously along with opportunistic wireless energy harvesting at each receiver. From the practical consideration, we assume that the channel state information (CSI) is only known at each respective receiver but is unavailable at the transmitter. We propose a novel receiver mode switching scheme for SWIPT based on a new application of the conventional random beamforming technique at the multi-antenna transmitter, which generates artificial channel fading to enable more efficient energy harvesting at each receiver when the received power exceeds a certain threshold. For the proposed scheme, we investigate the achievable information rate, harvested average power and/or power outage probability, as well as their various trade-offs in both AWGN and quasi-static fading channels. Compared to a reference scheme of periodic receiver mode switching without random transmit beamforming, the proposed scheme is shown to be able to achieve better rate-energy trade-offs when the harvested energy target is sufficiently large. Particularly, it is revealed that employing one single random beam for the proposed scheme is asymptotically optimal as the transmit power increases to infinity, and also performs the best with finite transmit power for the high harvested energy regime of most practical interests, thus leading to an appealing low-complexity implementation. Finally, we compare the rate-energy performances of the proposed scheme with different random beam designs. △ Less

Submitted 21 July, 2014; v1 submitted 21 January, 2013; originally announced January 2013.

Comments: 36 pages, 17 figures

Journal ref: IEEE Transactions on Wireless Communications, vol. 13, no. 4, pp. 2150-2162, April 2014