Showing 1–50 of 62 results for author: Jung, T

PristiQ: A Co-Design Framework for Preserving Data Security of Quantum Learning in the Cloud

Authors: Zhepeng Wang, Yi Sheng, Nirajan Koirala, Kanad Basu, Taeho Jung, Cheng-Chang Lu, Weiwen Jiang

Abstract: Benefiting from cloud computing, today's early-stage quantum computers can be remotely accessed via the cloud services, known as Quantum-as-a-Service (QaaS). However, it poses a high risk of data leakage in quantum machine learning (QML). To run a QML model with QaaS, users need to locally compile their quantum circuits including the subcircuit of data encoding first and then send the compiled cir… ▽ More Benefiting from cloud computing, today's early-stage quantum computers can be remotely accessed via the cloud services, known as Quantum-as-a-Service (QaaS). However, it poses a high risk of data leakage in quantum machine learning (QML). To run a QML model with QaaS, users need to locally compile their quantum circuits including the subcircuit of data encoding first and then send the compiled circuit to the QaaS provider for execution. If the QaaS provider is untrustworthy, the subcircuit to encode the raw data can be easily stolen. Therefore, we propose a co-design framework for preserving the data security of QML with the QaaS paradigm, namely PristiQ. By introducing an encryption subcircuit with extra secure qubits associated with a user-defined security key, the security of data can be greatly enhanced. And an automatic search algorithm is proposed to optimize the model to maintain its performance on the encrypted quantum data. Experimental results on simulation and the actual IBM quantum computer both prove the ability of PristiQ to provide high security for the quantum data while maintaining the model performance in QML. △ Less

Submitted 20 April, 2024; originally announced April 2024.

Chain-of-Instructions: Compositional Instruction Tuning on Large Language Models

Authors: Shirley Anugrah Hayati, Taehee Jung, Tristan Bodding-Long, Sudipta Kar, Abhinav Sethy, Joo-Kyung Kim, Dongyeop Kang

Abstract: Fine-tuning large language models (LLMs) with a collection of large and diverse instructions has improved the model's generalization to different tasks, even for unseen tasks. However, most existing instruction datasets include only single instructions, and they struggle to follow complex instructions composed of multiple subtasks. In this work, we propose a novel concept of compositional instruct… ▽ More Fine-tuning large language models (LLMs) with a collection of large and diverse instructions has improved the model's generalization to different tasks, even for unseen tasks. However, most existing instruction datasets include only single instructions, and they struggle to follow complex instructions composed of multiple subtasks. In this work, we propose a novel concept of compositional instructions called chain-of-instructions (CoI), where the output of one instruction becomes an input for the next like a chain. Unlike the conventional practice of solving single instruction tasks, our proposed method encourages a model to solve each subtask step by step until the final answer is reached. CoI-tuning (i.e., fine-tuning with CoI instructions) improves the model's ability to handle instructions composed of multiple subtasks as well as unseen composite tasks such as multilingual summarization. Overall, our study find that simple CoI tuning of existing instruction data can provide consistent generalization to solve more complex, unseen, and longer chains of instructions. △ Less

Submitted 24 June, 2024; v1 submitted 18 February, 2024; originally announced February 2024.

From Word Embedding to Reading Embedding Using Large Language Model, EEG and Eye-tracking

Authors: Yuhong Zhang, Shilai Yang, Gert Cauwenberghs, Tzyy-Ping Jung

Abstract: Reading comprehension, a fundamental cognitive ability essential for knowledge acquisition, is a complex skill, with a notable number of learners lacking proficiency in this domain. This study introduces innovative tasks for Brain-Computer Interface (BCI), predicting the relevance of words or tokens read by individuals to the target inference words. We use state-of-the-art Large Language Models (L… ▽ More Reading comprehension, a fundamental cognitive ability essential for knowledge acquisition, is a complex skill, with a notable number of learners lacking proficiency in this domain. This study introduces innovative tasks for Brain-Computer Interface (BCI), predicting the relevance of words or tokens read by individuals to the target inference words. We use state-of-the-art Large Language Models (LLMs) to guide a new reading embedding representation in training. This representation, integrating EEG and eye-tracking biomarkers through an attention-based transformer encoder, achieved a mean 5-fold cross-validation accuracy of 68.7% across nine subjects using a balanced sample, with the highest single-subject accuracy reaching 71.2%. This study pioneers the integration of LLMs, EEG, and eye-tracking for predicting human reading comprehension at the word level. We fine-tune the pre-trained Bidirectional Encoder Representations from Transformers (BERT) model for word embedding, devoid of information about the reading tasks. Despite this absence of task-specific details, the model effortlessly attains an accuracy of 92.7%, thereby validating our findings from LLMs. This work represents a preliminary step toward developing tools to assist reading. △ Less

Submitted 28 January, 2024; originally announced January 2024.

Self supervised convolutional kernel based handcrafted feature harmonization: Enhanced left ventricle hypertension disease phenotyping on echocardiography

Authors: Jina Lee, Youngtaek Hong, Dawun Jeong, Yeonggul Jang, Jaeik Jeon, Sihyeon Jeong, Taekgeun Jung, Yeonyee E. Yoon, Inki Moon, Seung-Ah Lee, Hyuk-Jae Chang

Abstract: Radiomics, a medical imaging technique, extracts quantitative handcrafted features from images to predict diseases. Harmonization in those features ensures consistent feature extraction across various imaging devices and protocols. Methods for harmonization include standardized imaging protocols, statistical adjustments, and evaluating feature robustness. Myocardial diseases such as Left Ventricul… ▽ More Radiomics, a medical imaging technique, extracts quantitative handcrafted features from images to predict diseases. Harmonization in those features ensures consistent feature extraction across various imaging devices and protocols. Methods for harmonization include standardized imaging protocols, statistical adjustments, and evaluating feature robustness. Myocardial diseases such as Left Ventricular Hypertrophy (LVH) and Hypertensive Heart Disease (HHD) are diagnosed via echocardiography, but variable imaging settings pose challenges. Harmonization techniques are crucial for applying handcrafted features in disease diagnosis in such scenario. Self-supervised learning (SSL) enhances data understanding within limited datasets and adapts to diverse data settings. ConvNeXt-V2 integrates convolutional layers into SSL, displaying superior performance in various tasks. This study focuses on convolutional filters within SSL, using them as preprocessing to convert images into feature maps for handcrafted feature harmonization. Our proposed method excelled in harmonization evaluation and exhibited superior LVH classification performance compared to existing methods. △ Less

Submitted 22 November, 2023; v1 submitted 13 October, 2023; originally announced October 2023.

Comments: 11 pages, 7 figures

Integrating LLM, EEG, and Eye-Tracking Biomarker Analysis for Word-Level Neural State Classification in Semantic Inference Reading Comprehension

Authors: Yuhong Zhang, Qin Li, Sujal Nahata, Tasnia Jamal, Shih-kuen Cheng, Gert Cauwenberghs, Tzyy-Ping Jung

Abstract: With the recent proliferation of large language models (LLMs), such as Generative Pre-trained Transformers (GPT), there has been a significant shift in exploring human and machine comprehension of semantic language meaning. This shift calls for interdisciplinary research that bridges cognitive science and natural language processing (NLP). This pilot study aims to provide insights into individuals… ▽ More With the recent proliferation of large language models (LLMs), such as Generative Pre-trained Transformers (GPT), there has been a significant shift in exploring human and machine comprehension of semantic language meaning. This shift calls for interdisciplinary research that bridges cognitive science and natural language processing (NLP). This pilot study aims to provide insights into individuals' neural states during a semantic relation reading-comprehension task. We propose jointly analyzing LLMs, eye-gaze, and electroencephalographic (EEG) data to study how the brain processes words with varying degrees of relevance to a keyword during reading. We also use a feature engineering approach to improve the fixation-related EEG data classification while participants read words with high versus low relevance to the keyword. The best validation accuracy in this word-level classification is over 60\% across 12 subjects. Words of high relevance to the inference keyword had significantly more eye fixations per word: 1.0584 compared to 0.6576 when excluding no-fixation words, and 1.5126 compared to 1.4026 when including them. This study represents the first attempt to classify brain states at a word level using LLM knowledge. It provides valuable insights into human cognitive abilities and the realm of Artificial General Intelligence (AGI), and offers guidance for developing potential reading-assisted technologies. △ Less

Submitted 17 October, 2023; v1 submitted 27 September, 2023; originally announced September 2023.

Privacy Preserving In-memory Computing Engine

Authors: Haoran Geng, Jianqiao Mo, Dayane Reis, Jonathan Takeshita, Taeho Jung, Brandon Reagen, Michael Niemier, Xiaobo Sharon Hu

Abstract: Privacy has rapidly become a major concern/design consideration. Homomorphic Encryption (HE) and Garbled Circuits (GC) are privacy-preserving techniques that support computations on encrypted data. HE and GC can complement each other, as HE is more efficient for linear operations, while GC is more effective for non-linear operations. Together, they enable complex computing tasks, such as machine l… ▽ More Privacy has rapidly become a major concern/design consideration. Homomorphic Encryption (HE) and Garbled Circuits (GC) are privacy-preserving techniques that support computations on encrypted data. HE and GC can complement each other, as HE is more efficient for linear operations, while GC is more effective for non-linear operations. Together, they enable complex computing tasks, such as machine learning, to be performed exactly on ciphertexts. However, HE and GC introduce two major bottlenecks: an elevated computational overhead and high data transfer costs. This paper presents PPIMCE, an in-memory computing (IMC) fabric designed to mitigate both computational overhead and data transfer issues. Through the use of multiple IMC cores for high parallelism, and by leveraging in-SRAM IMC for data management, PPIMCE offers a compact, energy-efficient solution for accelerating HE and GC. PPIMCE achieves a 107X speedup against a CPU implementation of GC. Additionally, PPIMCE achieves a 1,500X and 800X speedup compared to CPU and GPU implementations of CKKS-based HE multiplications. For privacy-preserving machine learning inference, PPIMCE attains a 1,000X speedup compared to CPU and a 12X speedup against CraterLake, the state-of-art privacy preserving computation accelerator. △ Less

Submitted 10 August, 2023; v1 submitted 4 August, 2023; originally announced August 2023.

Proof-of-Federated-Learning-Subchain: Free Partner Selection Subchain Based on Federated Learning

Authors: Boyang Li, Bingyu Shen, Qing Lu, Taeho Jung, Yiyu Shi

Abstract: The continuous thriving of the Blockchain society motivates research in novel designs of schemes supporting cryptocurrencies. Previously multiple Proof-of-Deep-Learning(PoDL) consensuses have been proposed to replace hashing with useful work such as deep learning model training tasks. The energy will be more efficiently used while maintaining the ledger. However deep learning models are problem-sp… ▽ More The continuous thriving of the Blockchain society motivates research in novel designs of schemes supporting cryptocurrencies. Previously multiple Proof-of-Deep-Learning(PoDL) consensuses have been proposed to replace hashing with useful work such as deep learning model training tasks. The energy will be more efficiently used while maintaining the ledger. However deep learning models are problem-specific and can be extremely complex. Current PoDL consensuses still require much work to realize in the real world. In this paper, we proposed a novel consensus named Proof-of-Federated-Learning-Subchain(PoFLSC) to fill the gap. We applied a subchain to record the training, challenging, and auditing activities and emphasized the importance of valuable datasets in partner selection. We simulated 20 miners in the subchain to demonstrate the effectiveness of PoFLSC. When we reduce the pool size concerning the reservation priority order, the drop rate difference in the performance in different scenarios further exhibits that the miner with a higher Shapley Value (SV) will gain a better opportunity to be selected when the size of the subchain pool is limited. In the conducted experiments, the PoFLSC consensus supported the subchain manager to be aware of reservation priority and the core partition of contributors to establish and maintain a competitive subchain. △ Less

Submitted 30 July, 2023; originally announced July 2023.

Comments: 7 pages, 7 figures

Using EEG Signals to Assess Workload during Memory Retrieval in a Real-world Scenario

Authors: Kuan-Jung Chiang, Steven Dong, Chung-Kuan Cheng, Tzyy-Ping Jung

Abstract: Objective: The Electroencephalogram (EEG) is gaining popularity as a physiological measure for neuroergonomics in human factor studies because it is objective, less prone to bias, and capable of assessing the dynamics of cognitive states. This study investigated the associations between memory workload and EEG during participants' typical office tasks on a single-monitor and dual-monitor arrangeme… ▽ More Objective: The Electroencephalogram (EEG) is gaining popularity as a physiological measure for neuroergonomics in human factor studies because it is objective, less prone to bias, and capable of assessing the dynamics of cognitive states. This study investigated the associations between memory workload and EEG during participants' typical office tasks on a single-monitor and dual-monitor arrangement. We expect a higher memory workload for the single-monitor arrangement. Approach: We designed an experiment that mimics the scenario of a subject performing some office work and examined whether the subjects experienced various levels of memory workload in two different office setups: 1) a single-monitor setup and 2) a dual-monitor setup. We used EEG band power, mutual information, and coherence as features to train machine learning models to classify high versus low memory workload states. Main results: The study results showed that these characteristics exhibited significant differences that were consistent across all participants. We also verified the robustness and consistency of these EEG signatures in a different data set collected during a Sternberg task in a prior study. Significance: The study found the EEG correlates of memory workload across individuals, demonstrating the effectiveness of using EEG analysis in conducting real-world neuroergonomic studies. △ Less

Submitted 13 May, 2023; originally announced May 2023.

Comments: This paper is published in Journal of Neural Engineering (2023). 19 pages, 9 figures

Cluster-Guided Label Generation in Extreme Multi-Label Classification

Authors: Taehee Jung, Joo-Kyung Kim, Sungjin Lee, Dongyeop Kang

Abstract: For extreme multi-label classification (XMC), existing classification-based models poorly perform for tail labels and often ignore the semantic relations among labels, like treating "Wikipedia" and "Wiki" as independent and separate labels. In this paper, we cast XMC as a generation task (XLGen), where we benefit from pre-trained text-to-text models. However, generating labels from the extremely l… ▽ More For extreme multi-label classification (XMC), existing classification-based models poorly perform for tail labels and often ignore the semantic relations among labels, like treating "Wikipedia" and "Wiki" as independent and separate labels. In this paper, we cast XMC as a generation task (XLGen), where we benefit from pre-trained text-to-text models. However, generating labels from the extremely large label space is challenging without any constraints or guidance. We, therefore, propose to guide label generation using label cluster information to hierarchically generate lower-level labels. We also find that frequency-based label ordering and using decoding ensemble methods are critical factors for the improvements in XLGen. XLGen with cluster guidance significantly outperforms the classification and generation baselines on tail labels, and also generally improves the overall performance in four popular XMC benchmarks. In human evaluation, we also find XLGen generates unseen but plausible labels. Our code is now available at https://github.com/alexa/xlgen-eacl-2023. △ Less

Submitted 17 February, 2023; originally announced February 2023.

Journal ref: EACL 2023

A Collaboration Strategy in the Mining Pool for Proof-of-Neural-Architecture Consensus

Authors: Boyang Li, Qing Lu, Weiwen Jiang, Taeho Jung, Yiyu Shi

Abstract: In most popular public accessible cryptocurrency systems, the mining pool plays a key role because mining cryptocurrency with the mining pool turns the non-profitable situation into profitable for individual miners. In many recent novel blockchain consensuses, the deep learning training procedure becomes the task for miners to prove their workload, thus the computation power of miners will not pur… ▽ More In most popular public accessible cryptocurrency systems, the mining pool plays a key role because mining cryptocurrency with the mining pool turns the non-profitable situation into profitable for individual miners. In many recent novel blockchain consensuses, the deep learning training procedure becomes the task for miners to prove their workload, thus the computation power of miners will not purely be spent on the hash puzzle. In this way, the hardware and energy will support the blockchain service and deep learning training simultaneously. While the incentive of miners is to earn tokens, individual miners are motivated to join mining pools to become more competitive. In this paper, we are the first to demonstrate a mining pool solution for novel consensuses based on deep learning. The mining pool manager partitions the full searching space into subspaces and all miners are scheduled to collaborate on the Neural Architecture Search (NAS) tasks in the assigned subspace. Experiments demonstrate that the performance of this type of mining pool is more competitive than an individual miner. Due to the uncertainty of miners' behaviors, the mining pool manager checks the standard deviation of the performance of high reward miners and prepares backup miners to ensure the completion of the tasks of high reward miners. △ Less

Submitted 5 May, 2022; originally announced June 2022.

Demo: Real-Time Semantic Communications with a Vision Transformer

Authors: Hanju Yoo, Taehun Jung, Linglong Dai, Songkuk Kim, Chan-Byoung Chae

Abstract: Semantic communications are expected to enable the more effective delivery of meaning rather than a precise transfer of symbols. In this paper, we propose an end-to-end deep neural network-based architecture for image transmission and demonstrate its feasibility in a real-time wireless channel by implementing a prototype based on a field-programmable gate array (FPGA). We demonstrate that this sys… ▽ More Semantic communications are expected to enable the more effective delivery of meaning rather than a precise transfer of symbols. In this paper, we propose an end-to-end deep neural network-based architecture for image transmission and demonstrate its feasibility in a real-time wireless channel by implementing a prototype based on a field-programmable gate array (FPGA). We demonstrate that this system outperforms the traditional 256-quadrature amplitude modulation system in the low signal-to-noise ratio regime with the popular CIFAR-10 dataset. To the best of our knowledge, this is the first work that implements and investigates real-time semantic communications with a vision transformer. △ Less

Submitted 8 May, 2022; originally announced May 2022.

IC-U-Net: A U-Net-based Denoising Autoencoder Using Mixtures of Independent Components for Automatic EEG Artifact Removal

Authors: Chun-Hsiang Chuang, Kong-Yi Chang, Chih-Sheng Huang, Tzyy-Ping Jung

Abstract: Electroencephalography (EEG) signals are often contaminated with artifacts. It is imperative to develop a practical and reliable artifact removal method to prevent misinterpretations of neural signals and underperformance of brain-computer interfaces. This study developed a new artifact removal method, IC-U-Net, which is based on the U-Net architecture for removing pervasive EEG artifacts and reco… ▽ More Electroencephalography (EEG) signals are often contaminated with artifacts. It is imperative to develop a practical and reliable artifact removal method to prevent misinterpretations of neural signals and underperformance of brain-computer interfaces. This study developed a new artifact removal method, IC-U-Net, which is based on the U-Net architecture for removing pervasive EEG artifacts and reconstructing brain sources. The IC-U-Net was trained using mixtures of brain and non-brain sources decomposed by independent component analysis and employed an ensemble of loss functions to model complex signal fluctuations in EEG recordings. The effectiveness of the proposed method in recovering brain sources and removing various artifacts (e.g., eye blinks/movements, muscle activities, and line/channel noises) was demonstrated in a simulation study and three real-world EEG datasets collected at rest and while driving and walking. IC-U-Net is user-friendly and publicly available, does not require parameter tuning or artifact type designations, and has no limitations on channel numbers. Given the increasing need to image natural brain dynamics in a mobile setting, IC-U-Net offers a promising end-to-end solution for automatically removing artifacts from EEG recordings. △ Less

Submitted 22 November, 2021; v1 submitted 18 November, 2021; originally announced November 2021.

Enhancing an Intelligent Digital Twin with a Self-organized Reconfiguration Management based on Adaptive Process Models

Authors: Timo Müller, Benjamin Lindemann, Tobias Jung, Nasser Jazdi, Michael Weyrich

Abstract: Shorter product life cycles and increasing individualization of production leads to an increased reconfiguration demand in the domain of industrial automation systems, which will be dominated by cyber-physical production systems in the future. In constantly changing systems, however, not all configuration alternatives of the almost infinite state space are fully understood. Thus, certain configura… ▽ More Shorter product life cycles and increasing individualization of production leads to an increased reconfiguration demand in the domain of industrial automation systems, which will be dominated by cyber-physical production systems in the future. In constantly changing systems, however, not all configuration alternatives of the almost infinite state space are fully understood. Thus, certain configurations can lead to process instability, a reduction in quality or machine failures. Therefore, this paper presents an approach that enhances an intelligent Digital Twin with a self-organized reconfiguration management based on adaptive process models in order to find optimized configurations more comprehensively. △ Less

Submitted 7 July, 2021; originally announced July 2021.

Comments: 6 pages, 2 figures. Submitted to 54th CIRP Conference on Manufacturing Systems 2021

CommitBERT: Commit Message Generation Using Pre-Trained Programming Language Model

Authors: Tae-Hwan Jung

Abstract: Commit message is a document that summarizes source code changes in natural language. A good commit message clearly shows the source code changes, so this enhances collaboration between developers. Therefore, our work is to develop a model that automatically writes the commit message. To this end, we release 345K datasets consisting of code modification and commit messages in six programming lan… ▽ More Commit message is a document that summarizes source code changes in natural language. A good commit message clearly shows the source code changes, so this enhances collaboration between developers. Therefore, our work is to develop a model that automatically writes the commit message. To this end, we release 345K datasets consisting of code modification and commit messages in six programming languages (Python, PHP, Go, Java, JavaScript, and Ruby). Similar to the neural machine translation (NMT) model, using our dataset, we feed the code modification to the encoder input and the commit message to the decoder input and measure the result of the generated commit message with BLEU-4. Also, we propose the following two training methods to improve the result of generating the commit message: (1) A method of preprocessing the input to feed the code modification to the encoder input. (2) A method that uses an initial weight suitable for the code domain to reduce the gap in contextual representation between programming language (PL) and natural language (NL). Training code, dataset, and pre-trained weights are available at https://github.com/graykode/commit-autosuggestions △ Less

Submitted 29 May, 2021; originally announced May 2021.

Comments: 8 pages, 3 figures, 4 Tables

Genomic Encryption of Biometric Information for Privacy-Preserving Forensics

Authors: Taeho Jung, Ryan Karl, Geoffrey H. Siwo

Abstract: DNA fingerprinting is a cornerstone for human identification in forensics, where the sequence of highly polymorphic short tandem repeats (STRs) from an individual is compared against a DNA database. This presents significant privacy risks to individuals with DNA profiles in the database due to hacking by malicious attackers who may access the data and misuse it for secondary purposes. In this pape… ▽ More DNA fingerprinting is a cornerstone for human identification in forensics, where the sequence of highly polymorphic short tandem repeats (STRs) from an individual is compared against a DNA database. This presents significant privacy risks to individuals with DNA profiles in the database due to hacking by malicious attackers who may access the data and misuse it for secondary purposes. In this paper, we propose a novel cryptographic framework for jointly encrypting DNA-based fingerprints (STRs) with other biometric data, for example, facial images, such that the STRs and biometrics information of an individual are revealed only when a positive match is found, i.e. the STRs act as decryption keys. Specifically, when a search is performed on the encrypted database using STR sequences of an individual in the database, a perfect match generates the facial image and/ or other biometrics of the individual while the lack of a match returns a null result. By jointly encrypting DNA fingerprints and other biometrics using the unique STRs generated keys, our approach ensures perfect privacy of the encrypted information with decryption of only the record with STRs matching the query. This safeguards the information of other individuals in the same database. The proposed approach can also be used to securely authenticate the identity of individuals or biological material in scenarios beyond forensics including tracking the identity of samples for clinical genetics and cell therapies. △ Less

Submitted 26 March, 2021; originally announced March 2021.

Comments: 6 pages, 2 figures

Boosting Template-based SSVEP Decoding by Cross-domain Transfer Learning

Authors: Kuan-Jung Chiang, Chun-Shu Wei, Masaki Nakanishi, Tzyy-Ping Jung

Abstract: Objective: This study aims to establish a generalized transfer-learning framework for boosting the performance of steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) by leveraging cross-domain data transferring. Approach: We enhanced the state-of-the-art template-based SSVEP decoding through incorporating a least-squares transformation (LST)-based transfer learning… ▽ More Objective: This study aims to establish a generalized transfer-learning framework for boosting the performance of steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) by leveraging cross-domain data transferring. Approach: We enhanced the state-of-the-art template-based SSVEP decoding through incorporating a least-squares transformation (LST)-based transfer learning to leverage calibration data across multiple domains (sessions, subjects, and EEG montages). Main results: Study results verified the efficacy of LST in obviating the variability of SSVEPs when transferring existing data across domains. Furthermore, the LST-based method achieved significantly higher SSVEP-decoding accuracy than the standard task-related component analysis (TRCA)-based method and the non-LST naive transfer-learning method. Significance: This study demonstrated the capability of the LST-based transfer learning to leverage existing data across subjects and/or devices with an in-depth investigation of its rationale and behavior in various circumstances. The proposed framework significantly improved the SSVEP decoding accuracy over the standard TRCA approach when calibration data are limited. Its performance in calibration reduction could facilitate plug-and-play SSVEP-based BCIs and further practical applications. △ Less

Submitted 9 February, 2021; originally announced February 2021.

Comments: Mirror version of the manuscript in the Journal of Neural Engineering on IOP Science (https://iopscience.iop.org/article/10.1088/1741-2552/abcb6e/meta), Journal of Neural Engineering (2020)

EEG-Based Brain-Computer Interfaces Are Vulnerable to Backdoor Attacks

Authors: Lubin Meng, Jian Huang, Zhigang Zeng, Xue Jiang, Shan Yu, Tzyy-Ping Jung, Chin-Teng Lin, Ricardo Chavarriaga, Dongrui Wu

Abstract: Research and development of electroencephalogram (EEG) based brain-computer interfaces (BCIs) have advanced rapidly, partly due to deeper understanding of the brain and wide adoption of sophisticated machine learning approaches for decoding the EEG signals. However, recent studies have shown that machine learning algorithms are vulnerable to adversarial attacks. This article proposes to use narrow… ▽ More Research and development of electroencephalogram (EEG) based brain-computer interfaces (BCIs) have advanced rapidly, partly due to deeper understanding of the brain and wide adoption of sophisticated machine learning approaches for decoding the EEG signals. However, recent studies have shown that machine learning algorithms are vulnerable to adversarial attacks. This article proposes to use narrow period pulse for poisoning attack of EEG-based BCIs, which is implementable in practice and has never been considered before. One can create dangerous backdoors in the machine learning model by injecting poisoning samples into the training set. Test samples with the backdoor key will then be classified into the target class specified by the attacker. What most distinguishes our approach from previous ones is that the backdoor key does not need to be synchronized with the EEG trials, making it very easy to implement. The effectiveness and robustness of the backdoor attack approach is demonstrated, highlighting a critical security concern for EEG-based BCIs and calling for urgent attention to address it. △ Less

Submitted 2 January, 2021; v1 submitted 30 October, 2020; originally announced November 2020.

Journal ref: IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2023

Large Product Key Memory for Pretrained Language Models

Authors: Gyuwan Kim, Tae-Hwan Jung

Abstract: Product key memory (PKM) proposed by Lample et al. (2019) enables to improve prediction accuracy by increasing model capacity efficiently with insignificant computational overhead. However, their empirical application is only limited to causal language modeling. Motivated by the recent success of pretrained language models (PLMs), we investigate how to incorporate large PKM into PLMs that can be f… ▽ More Product key memory (PKM) proposed by Lample et al. (2019) enables to improve prediction accuracy by increasing model capacity efficiently with insignificant computational overhead. However, their empirical application is only limited to causal language modeling. Motivated by the recent success of pretrained language models (PLMs), we investigate how to incorporate large PKM into PLMs that can be finetuned for a wide variety of downstream NLP tasks. We define a new memory usage metric, and careful observation using this metric reveals that most memory slots remain outdated during the training of PKM-augmented models. To train better PLMs by tackling this issue, we propose simple but effective solutions: (1) initialization from the model weights pretrained without memory and (2) augmenting PKM by addition rather than replacing a feed-forward network. We verify that both of them are crucial for the pretraining of PKM-augmented PLMs, enhancing memory utilization and downstream performance. Code and pretrained weights are available at https://github.com/clovaai/pkm-transformers. △ Less

Submitted 8 October, 2020; originally announced October 2020.

Comments: Accepted to Findings of EMNLP 2020; 10 pages, 4 figures

A Virtual Reality Game as a Tool to Assess Physiological Correlations of Stress

Authors: Daniel H. Lee, Tzyy-Ping Jung

Abstract: The objective of this study is to develop and use a virtual reality game as a tool to assess the effects of realistic stress on the behavioral and physiological responses of participants. The game is based on a popular Steam game called Keep Talking Nobody Explodes, where the player collaborates with another person to defuse a bomb. Varying levels of difficulties in solving a puzzle and time press… ▽ More The objective of this study is to develop and use a virtual reality game as a tool to assess the effects of realistic stress on the behavioral and physiological responses of participants. The game is based on a popular Steam game called Keep Talking Nobody Explodes, where the player collaborates with another person to defuse a bomb. Varying levels of difficulties in solving a puzzle and time pressures will result in different stress levels that can be measured in terms of errors, response time lengths, and other physiological measurements. The game was developed using 3D programming tools including Blender and virtual reality development kit (VRTK). To measure response times accurately, we added LSL (Lab Stream Layer) Markers to collect and synchronize physiological signals, behavioral data, and the timing of game events. We recorded Electrocardiogram (ECG) data during gameplay to assess heart rate and heart-rate variability (HRV) that have been shown as reliable indicators of stress. Our empirical results showed that heart rate increased significantly while HRV reduced significantly when the participants under high stress, which are consistent with the prior mainstream stress research. We further experimented with other tools to enhance communication between two players under adverse conditions and found that an automatic speech recognition software effectively enhanced the communication between the players by displaying keywords into the player's headset that lead to the facilitation of finding the solution of the puzzles or modules. This VR game framework is publicly available in Github and allows researchers to measure and synchronize other physiological signals such as electroencephalogram, electromyogram, and pupillometry. △ Less

Submitted 30 September, 2020; originally announced September 2020.

Comments: 13 pages, 10 figures

ACM Class: H.5.0

Federated Dynamic GNN with Secure Aggregation

Authors: Meng Jiang, Taeho Jung, Ryan Karl, Tong Zhao

Abstract: Given video data from multiple personal devices or street cameras, can we exploit the structural and dynamic information to learn dynamic representation of objects for applications such as distributed surveillance, without storing data at a central server that leads to a violation of user privacy? In this work, we introduce Federated Dynamic Graph Neural Network (Feddy), a distributed and secured… ▽ More Given video data from multiple personal devices or street cameras, can we exploit the structural and dynamic information to learn dynamic representation of objects for applications such as distributed surveillance, without storing data at a central server that leads to a violation of user privacy? In this work, we introduce Federated Dynamic Graph Neural Network (Feddy), a distributed and secured framework to learn the object representations from multi-user graph sequences: i) It aggregates structural information from nearby objects in the current graph as well as dynamic information from those in the previous graph. It uses a self-supervised loss of predicting the trajectories of objects. ii) It is trained in a federated learning manner. The centrally located server sends the model to user devices. Local models on the respective user devices learn and periodically send their learning to the central server without ever exposing the user's data to server. iii) Studies showed that the aggregated parameters could be inspected though decrypted when broadcast to clients for model synchronizing, after the server performed a weighted average. We design an appropriate aggregation mechanism of secure aggregation primitives that can protect the security and privacy in federated learning with scalability. Experiments on four video camera datasets (in four different scenes) as well as simulation demonstrate that Feddy achieves great effectiveness and security. △ Less

Submitted 15 September, 2020; originally announced September 2020.

Comments: 16 pages, 2 figures

Opacity of Discrete Event Systems with Active Intruder

Authors: Alireza Partovi, Taeho Jung, Lin Hai

Abstract: Opacity is a security property formalizing the information leakage of a system to an external observer, namely intruder. The conventional opacity that has been studied in the Discrete Event System (DES) literature usually assumes passive intruders, who only observe the behavior of the system. However, in many cybersecurity concerns, such as web service, active intruders, who are capable of influen… ▽ More Opacity is a security property formalizing the information leakage of a system to an external observer, namely intruder. The conventional opacity that has been studied in the Discrete Event System (DES) literature usually assumes passive intruders, who only observe the behavior of the system. However, in many cybersecurity concerns, such as web service, active intruders, who are capable of influencing the system's behavior beyond passive observations, need to be considered and defended against. We are therefore motivated to extend the opacity notions to handle active intruders. For this, we model the system as a non-deterministic finite-state transducer. It is assumed that the intruder has a full knowledge of the system structure and is capable of interacting with the system by injecting different inputs and observing its responses. In this setup, we first introduce reactive current-state opacity (RCSO) notion characterizing a property that the system does not leak its secret state regardless of how the intruder manipulates the system behavior. We furthermore extend this notion to language-based and initial-state reactive opacity notions, and study the relationship among them. It turns out that all the proposed reactive opacity notions are equivalent to RCSO. We therefore focus on RCSO and study its verification problem. It is shown that the RCSO can be verified by constructing an observer automaton. △ Less

Submitted 24 July, 2020; originally announced July 2020.

Computing-in-Memory for Performance and Energy Efficient Homomorphic Encryption

Authors: Dayane Reis, Jonathan Takeshita, Taeho Jung, Michael Niemier, Xiaobo Sharon Hu

Abstract: Homomorphic encryption (HE) allows direct computations on encrypted data. Despite numerous research efforts, the practicality of HE schemes remains to be demonstrated. In this regard, the enormous size of ciphertexts involved in HE computations degrades computational efficiency. Near-memory Processing (NMP) and Computing-in-memory (CiM) - paradigms where computation is done within the memory bound… ▽ More Homomorphic encryption (HE) allows direct computations on encrypted data. Despite numerous research efforts, the practicality of HE schemes remains to be demonstrated. In this regard, the enormous size of ciphertexts involved in HE computations degrades computational efficiency. Near-memory Processing (NMP) and Computing-in-memory (CiM) - paradigms where computation is done within the memory boundaries - represent architectural solutions for reducing latency and energy associated with data transfers in data-intensive applications such as HE. This paper introduces CiM-HE, a Computing-in-memory (CiM) architecture that can support operations for the B/FV scheme, a somewhat homomorphic encryption scheme for general computation. CiM-HE hardware consists of customized peripherals such as sense amplifiers, adders, bit-shifters, and sequencing circuits. The peripherals are based on CMOS technology, and could support computations with memory cells of different technologies. Circuit-level simulations are used to evaluate our CiM-HE framework assuming a 6T-SRAM memory. We compare our CiM-HE implementation against (i) two optimized CPU HE implementations, and (ii) an FPGA-based HE accelerator implementation. When compared to a CPU solution, CiM-HE obtains speedups between 4.6x and 9.1x, and energy savings between 266.4x and 532.8x for homomorphic multiplications (the most expensive HE operation). Also, a set of four end-to-end tasks, i.e., mean, variance, linear regression, and inference are up to 1.1x, 7.7x, 7.1x, and 7.5x faster (and 301.1x, 404.6x, 532.3x, and 532.8x more energy efficient). Compared to CPU-based HE in a previous work, CiM-HE obtain 14.3x speed-up and >2600x energy savings. Finally, our design offers 2.2x speed-up with 88.1x energy savings compared to a state-of-the-art FPGA-based accelerator. △ Less

Submitted 19 August, 2020; v1 submitted 5 May, 2020; originally announced May 2020.

Comments: 14 pages

Journal ref: IEEE Transactions on Very Large Scale Integration (VLSI) Systems ( Volume: 28, Issue: 11, Nov. 2020)

Secure Single-Server Nearly-Identical Image Deduplication

Authors: Jonathan Takeshita, Ryan Karl, Taeho Jung

Abstract: Cloud computing is often utilized for file storage. Clients of cloud storage services want to ensure the privacy of their data, and both clients and servers want to use as little storage as possible. Cross-user deduplication is one method to reduce the amount of storage a server uses. Deduplication and privacy are naturally conflicting goals, especially for nearly-identical (``fuzzy'') deduplicati… ▽ More Cloud computing is often utilized for file storage. Clients of cloud storage services want to ensure the privacy of their data, and both clients and servers want to use as little storage as possible. Cross-user deduplication is one method to reduce the amount of storage a server uses. Deduplication and privacy are naturally conflicting goals, especially for nearly-identical (``fuzzy'') deduplication, as some information about the data must be used to perform deduplication. Prior solutions thus utilize multiple servers, or only function for exact deduplication. In this paper, we present a single-server protocol for cross-user nearly-identical deduplication based on secure locality-sensitive hashing (SLSH). We formally define our ideal security, and rigorously prove our protocol secure against fully malicious, colluding adversaries with a proof by simulation. We show experimentally that the individual parts of the protocol are computationally feasible, and further discuss practical issues of security and efficiency. △ Less

Submitted 5 May, 2020; originally announced May 2020.

Posterior Calibrated Training on Sentence Classification Tasks

Authors: Taehee Jung, Dongyeop Kang, Hua Cheng, Lucas Mentch, Thomas Schaaf

Abstract: Most classification models work by first predicting a posterior probability distribution over all classes and then selecting that class with the largest estimated probability. In many settings however, the quality of posterior probability itself (e.g., 65% chance having diabetes), gives more reliable information than the final predicted class alone. When these methods are shown to be poorly calibr… ▽ More Most classification models work by first predicting a posterior probability distribution over all classes and then selecting that class with the largest estimated probability. In many settings however, the quality of posterior probability itself (e.g., 65% chance having diabetes), gives more reliable information than the final predicted class alone. When these methods are shown to be poorly calibrated, most fixes to date have relied on posterior calibration, which rescales the predicted probabilities but often has little impact on final classifications. Here we propose an end-to-end training procedure called posterior calibrated (PosCal) training that directly optimizes the objective while minimizing the difference between the predicted and empirical posterior probabilities.We show that PosCal not only helps reduce the calibration error but also improve task performance by penalizing drops in performance of both objectives. Our PosCal achieves about 2.5% of task performance gain and 16.1% of calibration error reduction on GLUE (Wang et al., 2018) compared to the baseline. We achieved the comparable task performance with 13.2% calibration error reduction on xSLUE (Kang and Hovy, 2019), but not outperforming the two-stage calibration baseline. PosCal training can be easily extendable to any types of classification tasks as a form of regularization term. Also, PosCal has the advantage that it incrementally tracks needed statistics for the calibration objective during the training process, making efficient use of large training sets. △ Less

Submitted 1 May, 2020; v1 submitted 29 April, 2020; originally announced April 2020.

Comments: Accepted at ACL 2020

Wireless VR/Haptic Open Platform for Multimodal Teleoperation

Authors: Tae Hun Jung, Hanju Yoo, Yuna Jin, Chae Eun Rhee, Chan-Byoung Chae

Abstract: With emerging trends in the fifth generation and robotics, the Internet of Skills will enable us to deliver skills or expertise anywhere over the Internet. In this paper, we propose a wireless connected virtual reality and haptic communication open platform to show the proof of concept for multimodal teleoperation systems in real-time. We focus on a practical implementation with commercial product… ▽ More With emerging trends in the fifth generation and robotics, the Internet of Skills will enable us to deliver skills or expertise anywhere over the Internet. In this paper, we propose a wireless connected virtual reality and haptic communication open platform to show the proof of concept for multimodal teleoperation systems in real-time. We focus on a practical implementation with commercial products to facilitate the access and modification of the system. The performance of the system is measured in terms of system latency and user-centric metrics. △ Less

Submitted 26 April, 2020; originally announced April 2020.

Ensuring Privacy in Location-Based Services: A Model-based Approach

Authors: Alireza Partovi, Wei Zheng, Taeho Jung, Hai Lin

Abstract: In recent years, the widespread of mobile devices equipped with GPS and communication chips has led to the growing use of location-based services (LBS) in which a user receives a service based on his current location. The disclosure of user's location, however, can raise serious concerns about user privacy in general, and location privacy in particular which led to the development of various locat… ▽ More In recent years, the widespread of mobile devices equipped with GPS and communication chips has led to the growing use of location-based services (LBS) in which a user receives a service based on his current location. The disclosure of user's location, however, can raise serious concerns about user privacy in general, and location privacy in particular which led to the development of various location privacy-preserving mechanisms aiming to enhance the location privacy while using LBS applications. In this paper, we propose to model the user mobility pattern and utility of the LBS as a Markov decision process (MDP), and inspired by probabilistic current state opacity notation, we introduce a new location privacy metric, namely $ε-$privacy, that quantifies the adversary belief over the user's current location. We exploit this dynamic model to design a LPPM that while it ensures the utility of service is being fully utilized, independent of the adversary prior knowledge about the user, it can guarantee a user-specified privacy level can be achieved for an infinite time horizon. The overall privacy-preserving framework, including the construction of the user mobility model as a MDP, and design of the proposed LPPM, are demonstrated and validated with real-world experimental data. △ Less

Submitted 23 February, 2020; originally announced February 2020.

Tiny noise, big mistakes: Adversarial perturbations induce errors in Brain-Computer Interface spellers

Authors: Xiao Zhang, Dongrui Wu, Lieyun Ding, Hanbin Luo, Chin-Teng Lin, Tzyy-Ping Jung, Ricardo Chavarriaga

Abstract: An electroencephalogram (EEG) based brain-computer interface (BCI) speller allows a user to input text to a computer by thought. It is particularly useful to severely disabled individuals, e.g., amyotrophic lateral sclerosis patients, who have no other effective means of communication with another person or a computer. Most studies so far focused on making EEG-based BCI spellers faster and more re… ▽ More An electroencephalogram (EEG) based brain-computer interface (BCI) speller allows a user to input text to a computer by thought. It is particularly useful to severely disabled individuals, e.g., amyotrophic lateral sclerosis patients, who have no other effective means of communication with another person or a computer. Most studies so far focused on making EEG-based BCI spellers faster and more reliable; however, few have considered their security. This study, for the first time, shows that P300 and steady-state visual evoked potential BCI spellers are very vulnerable, i.e., they can be severely attacked by adversarial perturbations, which are too tiny to be noticed when added to EEG signals, but can mislead the spellers to spell anything the attacker wants. The consequence could range from merely user frustration to severe misdiagnosis in clinical applications. We hope our research can attract more attention to the security of EEG-based BCI spellers, and more broadly, EEG-based BCIs, which has received little attention before. △ Less

Submitted 16 July, 2020; v1 submitted 30 January, 2020; originally announced January 2020.

Journal ref: National Science Review, 8(4), 2021

EEG-based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and their Applications

Authors: Xiaotong Gu, Zehong Cao, Alireza Jolfaei, Peng Xu, Dongrui Wu, Tzyy-Ping Jung, Chin-Teng Lin

Abstract: Brain-Computer Interface (BCI) is a powerful communication tool between users and systems, which enhances the capability of the human brain in communicating and interacting with the environment directly. Advances in neuroscience and computer science in the past decades have led to exciting developments in BCI, thereby making BCI a top interdisciplinary research area in computational neuroscience a… ▽ More Brain-Computer Interface (BCI) is a powerful communication tool between users and systems, which enhances the capability of the human brain in communicating and interacting with the environment directly. Advances in neuroscience and computer science in the past decades have led to exciting developments in BCI, thereby making BCI a top interdisciplinary research area in computational neuroscience and intelligence. Recent technological advances such as wearable sensing devices, real-time data streaming, machine learning, and deep learning approaches have increased interest in electroencephalographic (EEG) based BCI for translational and healthcare applications. Many people benefit from EEG-based BCIs, which facilitate continuous monitoring of fluctuations in cognitive states under monotonous tasks in the workplace or at home. In this study, we survey the recent literature of EEG signal sensing technologies and computational intelligence approaches in BCI applications, compensated for the gaps in the systematic summary of the past five years (2015-2019). In specific, we first review the current status of BCI and its significant obstacles. Then, we present advanced signal sensing and enhancement technologies to collect and clean EEG signals, respectively. Furthermore, we demonstrate state-of-art computational intelligence techniques, including interpretable fuzzy models, transfer learning, deep learning, and combinations, to monitor, maintain, or track human cognitive states and operating performance in prevalent applications. Finally, we deliver a couple of innovative BCI-inspired healthcare applications and discuss some future research directions in EEG-based BCIs. △ Less

Submitted 28 January, 2020; originally announced January 2020.

Comments: Submitting to IEEE/ACM Transactions on Computational Biology and Bioinformatics

Multi-lane Detection Using Instance Segmentation and Attentive Voting

Authors: Donghoon Chang, Vinjohn Chirakkal, Shubham Goswami, Munawar Hasan, Taekwon Jung, Jinkeon Kang, Seok-Cheol Kee, Dongkyu Lee, Ajit Pratap Singh

Abstract: Autonomous driving is becoming one of the leading industrial research areas. Therefore many automobile companies are coming up with semi to fully autonomous driving solutions. Among these solutions, lane detection is one of the vital driver-assist features that play a crucial role in the decision-making process of the autonomous vehicle. A variety of solutions have been proposed to detect lanes on… ▽ More Autonomous driving is becoming one of the leading industrial research areas. Therefore many automobile companies are coming up with semi to fully autonomous driving solutions. Among these solutions, lane detection is one of the vital driver-assist features that play a crucial role in the decision-making process of the autonomous vehicle. A variety of solutions have been proposed to detect lanes on the road, which ranges from using hand-crafted features to the state-of-the-art end-to-end trainable deep learning architectures. Most of these architectures are trained in a traffic constrained environment. In this paper, we propose a novel solution to multi-lane detection, which outperforms state of the art methods in terms of both accuracy and speed. To achieve this, we also offer a dataset with a more intuitive labeling scheme as compared to other benchmark datasets. Using our approach, we are able to obtain a lane segmentation accuracy of 99.87% running at 54.53 fps (average). △ Less

Submitted 1 January, 2020; originally announced January 2020.

Comments: Accepted in ICCAS 2019 - The 19th International Conference on Control, Automation and Systems, Corresponding Author: Shubham Goswami

White-Box Target Attack for EEG-Based BCI Regression Problems

Authors: Lubin Meng, Chin-Teng Lin, Tzyy-Ring Jung, Dongrui Wu

Abstract: Machine learning has achieved great success in many applications, including electroencephalogram (EEG) based brain-computer interfaces (BCIs). Unfortunately, many machine learning models are vulnerable to adversarial examples, which are crafted by adding deliberately designed perturbations to the original inputs. Many adversarial attack approaches for classification problems have been proposed, bu… ▽ More Machine learning has achieved great success in many applications, including electroencephalogram (EEG) based brain-computer interfaces (BCIs). Unfortunately, many machine learning models are vulnerable to adversarial examples, which are crafted by adding deliberately designed perturbations to the original inputs. Many adversarial attack approaches for classification problems have been proposed, but few have considered target adversarial attacks for regression problems. This paper proposes two such approaches. More specifically, we consider white-box target attacks for regression problems, where we know all information about the regression model to be attacked, and want to design small perturbations to change the regression output by a pre-determined amount. Experiments on two BCI regression problems verified that both approaches are effective. Moreover, adversarial examples generated from both approaches are also transferable, which means that we can use adversarial examples generated from one known regression model to attack an unknown regression model, i.e., to perform black-box attacks. To our knowledge, this is the first study on adversarial attacks for EEG-based BCI regression problems, which calls for more attention on the security of BCI systems. △ Less

Submitted 7 November, 2019; originally announced November 2019.

Modeling EEG data distribution with a Wasserstein Generative Adversarial Network to predict RSVP Events

Authors: Sharaj Panwar, Paul Rad, Tzyy-Ping Jung, Yufei Huang

Abstract: Electroencephalography (EEG) data are difficult to obtain due to complex experimental setups and reduced comfort with prolonged wearing. This poses challenges to train powerful deep learning model with the limited EEG data. Being able to generate EEG data computationally could address this limitation. We propose a novel Wasserstein Generative Adversarial Network with gradient penalty (WGAN-GP) to… ▽ More Electroencephalography (EEG) data are difficult to obtain due to complex experimental setups and reduced comfort with prolonged wearing. This poses challenges to train powerful deep learning model with the limited EEG data. Being able to generate EEG data computationally could address this limitation. We propose a novel Wasserstein Generative Adversarial Network with gradient penalty (WGAN-GP) to synthesize EEG data. This network addresses several modeling challenges of simulating time-series EEG data including frequency artifacts and training instability. We further extended this network to a class-conditioned variant that also includes a classification branch to perform event-related classification. We trained the proposed networks to generate one and 64-channel data resembling EEG signals routinely seen in a rapid serial visual presentation (RSVP) experiment and demonstrated the validity of the generated samples. We also tested intra-subject cross-session classification performance for classifying the RSVP target events and showed that class-conditioned WGAN-GP can achieve improved event-classification performance over EEGNet. △ Less

Submitted 1 July, 2020; v1 submitted 11 November, 2019; originally announced November 2019.

Earlier Isn't Always Better: Sub-aspect Analysis on Corpus and System Biases in Summarization

Authors: Taehee Jung, Dongyeop Kang, Lucas Mentch, Eduard Hovy

Abstract: Despite the recent developments on neural summarization systems, the underlying logic behind the improvements from the systems and its corpus-dependency remains largely unexplored. Position of sentences in the original text, for example, is a well known bias for news summarization. Following in the spirit of the claim that summarization is a combination of sub-functions, we define three sub-aspect… ▽ More Despite the recent developments on neural summarization systems, the underlying logic behind the improvements from the systems and its corpus-dependency remains largely unexplored. Position of sentences in the original text, for example, is a well known bias for news summarization. Following in the spirit of the claim that summarization is a combination of sub-functions, we define three sub-aspects of summarization: position, importance, and diversity and conduct an extensive analysis of the biases of each sub-aspect with respect to the domain of nine different summarization corpora (e.g., news, academic papers, meeting minutes, movie script, books, posts). We find that while position exhibits substantial bias in news articles, this is not the case, for example, with academic papers and meeting minutes. Furthermore, our empirical study shows that different types of summarization systems (e.g., neural-based) are composed of different degrees of the sub-aspects. Our study provides useful lessons regarding consideration of underlying sub-aspects when collecting a new summarization dataset or developing a new system. △ Less

Submitted 30 August, 2019; originally announced August 2019.

Comments: EMNLP 2019

Utilizing Deep Learning Towards Multi-modal Bio-sensing and Vision-based Affective Computing

Authors: Siddharth Siddharth, Tzyy-Ping Jung, Terrence J. Sejnowski

Abstract: In recent years, the use of bio-sensing signals such as electroencephalogram (EEG), electrocardiogram (ECG), etc. have garnered interest towards applications in affective computing. The parallel trend of deep-learning has led to a huge leap in performance towards solving various vision-based research problems such as object detection. Yet, these advances in deep-learning have not adequately transl… ▽ More In recent years, the use of bio-sensing signals such as electroencephalogram (EEG), electrocardiogram (ECG), etc. have garnered interest towards applications in affective computing. The parallel trend of deep-learning has led to a huge leap in performance towards solving various vision-based research problems such as object detection. Yet, these advances in deep-learning have not adequately translated into bio-sensing research. This work applies novel deep-learning-based methods to various bio-sensing and video data of four publicly available multi-modal emotion datasets. For each dataset, we first individually evaluate the emotion-classification performance obtained by each modality. We then evaluate the performance obtained by fusing the features from these modalities. We show that our algorithms outperform the results reported by other studies for emotion/valence/arousal/liking classification on DEAP and MAHNOB-HCI datasets and set up benchmarks for the newer AMIGOS and DREAMER datasets. We also evaluate the performance of our algorithms by combining the datasets and by using transfer learning to show that the proposed method overcomes the inconsistencies between the datasets. Hence, we do a thorough analysis of multi-modal affective data from more than 120 subjects and 2,800 trials. Finally, utilizing a convolution-deconvolution network, we propose a new technique towards identifying salient brain regions corresponding to various affective states. △ Less

Submitted 16 May, 2019; originally announced May 2019.

Comments: Accepted for publication in IEEE Transactions on Affective Computing. This version on the arXiv is the updated version of the same manuscript

DLBC: A Deep Learning-Based Consensus in Blockchains for Deep Learning Services

Authors: Boyang Li, Changhao Chenli, Xiaowei Xu, Yiyu Shi, Taeho Jung

Abstract: With the increasing artificial intelligence application, deep neural network (DNN) has become an emerging task. However, to train a good deep learning model will suffer from enormous computation cost and energy consumption. Recently, blockchain has been widely used, and during its operation, a huge amount of computation resources are wasted for the Proof of Work (PoW) consensus. In this paper, we… ▽ More With the increasing artificial intelligence application, deep neural network (DNN) has become an emerging task. However, to train a good deep learning model will suffer from enormous computation cost and energy consumption. Recently, blockchain has been widely used, and during its operation, a huge amount of computation resources are wasted for the Proof of Work (PoW) consensus. In this paper, we propose DLBC to exploit the computation power of miners for deep learning training as proof of useful work instead of calculating hash values. it distinguishes itself from recent proof of useful work mechanisms by addressing various limitations of them. Specifically, DLBC handles multiple tasks, larger model and training datasets, and introduces a comprehensive ranking mechanism that considers tasks difficulty(e.g., model complexity, network burden, data size, queue length). We also applied DNN-watermark [1] to improve the robustness. In Section V, the average overhead of digital signature is 1.25, 0.001, 0.002 and 0.98 seconds, respectively, and the average overhead of network is 3.77, 3.01, 0.37 and 0.41 seconds, respectively. Embedding a watermark takes 3 epochs and removing a watermark takes 30 epochs. This penalty of removing watermark will prevent attackers from stealing, improving, and resubmitting DL models from honest miners. △ Less

Submitted 30 January, 2020; v1 submitted 15 April, 2019; originally announced April 2019.

Energy-recycling Blockchain with Proof-of-Deep-Learning

Authors: Changhao Chenli, Boyang Li, Yiyu Shi, Taeho Jung

Abstract: An enormous amount of energy is wasted in Proofof-Work (PoW) mechanisms adopted by popular blockchain applications (e.g., PoW-based cryptocurrencies), because miners must conduct a large amount of computation. Owing to this, one serious rising concern is that the energy waste not only dilutes the value of the blockchain but also hinders its further application. In this paper, we propose a novel bl… ▽ More An enormous amount of energy is wasted in Proofof-Work (PoW) mechanisms adopted by popular blockchain applications (e.g., PoW-based cryptocurrencies), because miners must conduct a large amount of computation. Owing to this, one serious rising concern is that the energy waste not only dilutes the value of the blockchain but also hinders its further application. In this paper, we propose a novel blockchain design that fully recycles the energy required for facilitating and maintaining it, which is re-invested to the computation of deep learning. We realize this by proposing Proof-of-Deep-Learning (PoDL) such that a valid proof for a new block can be generated if and only if a proper deep learning model is produced. We present a proof-of-concept design of PoDL that is compatible with the majority of the cryptocurrencies that are based on hash-based PoW mechanisms. Our benchmark and simulation results show that the proposed design is feasible for various popular cryptocurrencies such as Bitcoin, Bitcoin Cash, and Litecoin. △ Less

Submitted 11 February, 2019; originally announced February 2019.

Comments: 5 pages

Cross-Subject Transfer Learning Improves the Practicality of Real-World Applications of Brain-Computer Interfaces

Authors: Kuan-Jung Chiang, Chun-Shu Wei, Masaki Nakanishi, Tzyy-Ping Jung

Abstract: Steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) have shown its robustness in facilitating high-efficiency communication. State-of-the-art training-based SSVEP decoding methods such as extended Canonical Correlation Analysis (CCA) and Task-Related Component Analysis (TRCA) are the major players that elevate the efficiency of the SSVEP-based BCIs through a calibra… ▽ More Steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) have shown its robustness in facilitating high-efficiency communication. State-of-the-art training-based SSVEP decoding methods such as extended Canonical Correlation Analysis (CCA) and Task-Related Component Analysis (TRCA) are the major players that elevate the efficiency of the SSVEP-based BCIs through a calibration process. However, due to notable human variability across individuals and within individuals over time, calibration (training) data collection is non-negligible and often laborious and time-consuming, deteriorating the practicality of SSVEP BCIs in a real-world context. This study aims to develop a cross-subject transferring approach to reduce the need for collecting training data from a test user with a newly proposed least-squares transformation (LST) method. Study results show the capability of the LST in reducing the number of training templates required for a 40-class SSVEP BCI. The LST method may lead to numerous real-world applications using near-zero-training/plug-and-play high-speed SSVEP BCIs. △ Less

Submitted 13 March, 2019; v1 submitted 5 October, 2018; originally announced October 2018.

Comments: 4 pages, 3 figures, 1 table. For NER'19

Multi-modal Approach for Affective Computing

Authors: Siddharth Siddharth, Tzyy-Ping Jung, Terrence J. Sejnowski

Abstract: Throughout the past decade, many studies have classified human emotions using only a single sensing modality such as face video, electroencephalogram (EEG), electrocardiogram (ECG), galvanic skin response (GSR), etc. The results of these studies are constrained by the limitations of these modalities such as the absence of physiological biomarkers in the face-video analysis, poor spatial resolution… ▽ More Throughout the past decade, many studies have classified human emotions using only a single sensing modality such as face video, electroencephalogram (EEG), electrocardiogram (ECG), galvanic skin response (GSR), etc. The results of these studies are constrained by the limitations of these modalities such as the absence of physiological biomarkers in the face-video analysis, poor spatial resolution in EEG, poor temporal resolution of the GSR etc. Scant research has been conducted to compare the merits of these modalities and understand how to best use them individually and jointly. Using multi-modal AMIGOS dataset, this study compares the performance of human emotion classification using multiple computational approaches applied to face videos and various bio-sensing modalities. Using a novel method for compensating physiological baseline we show an increase in the classification accuracy of various approaches that we use. Finally, we present a multi-modal emotion-classification approach in the domain of affective computing research. △ Less

Submitted 20 June, 2018; v1 submitted 25 April, 2018; originally announced April 2018.

Comments: Published in IEEE 40th International Engineering in Medicine and Biology Conference (EMBC) 2018

An Affordable Bio-Sensing and Activity Tagging Platform for HCI Research

Authors: Siddharth, Aashish Patel, Tzyy-Ping Jung, Terrence J. Sejnowski

Abstract: We present a novel multi-modal bio-sensing platform capable of integrating multiple data streams for use in real-time applications. The system is composed of a central compute module and a companion headset. The compute node collects, time-stamps and transmits the data while also providing an interface for a wide range of sensors including electroencephalogram, photoplethysmogram, electrocardiogra… ▽ More We present a novel multi-modal bio-sensing platform capable of integrating multiple data streams for use in real-time applications. The system is composed of a central compute module and a companion headset. The compute node collects, time-stamps and transmits the data while also providing an interface for a wide range of sensors including electroencephalogram, photoplethysmogram, electrocardiogram, and eye gaze among others. The companion headset contains the gaze tracking cameras. By integrating many of the measurements systems into an accessible package, we are able to explore previously unanswerable questions ranging from open-environment interactions to emotional response studies. Though some of the integrated sensors are designed from the ground-up to fit into a compact form factor, we validate the accuracy of the sensors and find that they perform similarly to, and in some cases better than, alternatives. △ Less

Submitted 21 February, 2018; originally announced February 2018.

VoiceMask: Anonymize and Sanitize Voice Input on Mobile Devices

Authors: Jianwei Qian, Haohua Du, Jiahui Hou, Linlin Chen, Taeho Jung, Xiang-Yang Li, Yu Wang, Yanbo Deng

Abstract: Voice input has been tremendously improving the user experience of mobile devices by freeing our hands from typing on the small screen. Speech recognition is the key technology that powers voice input, and it is usually outsourced to the cloud for the best performance. However, the cloud might compromise users' privacy by identifying their identities by voice, learning their sensitive input conten… ▽ More Voice input has been tremendously improving the user experience of mobile devices by freeing our hands from typing on the small screen. Speech recognition is the key technology that powers voice input, and it is usually outsourced to the cloud for the best performance. However, the cloud might compromise users' privacy by identifying their identities by voice, learning their sensitive input content via speech recognition, and then profiling the mobile users based on the content. In this paper, we design an intermediate between users and the cloud, named VoiceMask, to sanitize users' voice data before sending it to the cloud for speech recognition. We analyze the potential privacy risks and aim to protect users' identities and sensitive input content from being disclosed to the cloud. VoiceMask adopts a carefully designed voice conversion mechanism that is resistant to several attacks. Meanwhile, it utilizes an evolution-based keyword substitution technique to sanitize the voice input content. The two sanitization phases are all performed in the resource-limited mobile device while still maintaining the usability and accuracy of the cloud-supported speech recognition service. We implement the voice sanitizer on Android systems and present extensive experimental results that validate the effectiveness and efficiency of our app. It is demonstrated that we are able to reduce the chance of a user's voice being identified from 50 people by 84% while keeping the drop of speech recognition accuracy within 14.2%. △ Less

Submitted 30 November, 2017; originally announced November 2017.

Social Network De-anonymization: More Adversarial Knowledge, More Users Re-Identified?

Authors: Jianwei Qian, Xiang-Yang Li, Yu Wang, Shaojie Tang, Taeho Jung, Yang Fan

Abstract: Following the trend of data trading and data publishing, many online social networks have enabled potentially sensitive data to be exchanged or shared on the web. As a result, users' privacy could be exposed to malicious third parties since they are extremely vulnerable to de-anonymization attacks, i.e., the attacker links the anonymous nodes in the social network to their real identities with the… ▽ More Following the trend of data trading and data publishing, many online social networks have enabled potentially sensitive data to be exchanged or shared on the web. As a result, users' privacy could be exposed to malicious third parties since they are extremely vulnerable to de-anonymization attacks, i.e., the attacker links the anonymous nodes in the social network to their real identities with the help of background knowledge. Previous work in social network de-anonymization mostly focuses on designing accurate and efficient de-anonymization methods. We study this topic from a different perspective and attempt to investigate the intrinsic relation between the attacker's knowledge and the expected de-anonymization gain. One common intuition is that the more auxiliary information the attacker has, the more accurate de-anonymization becomes. However, their relation is much more sophisticated than that. To simplify the problem, we attempt to quantify background knowledge and de-anonymization gain under several assumptions. Our theoretical analysis and simulations on synthetic and real network data show that more background knowledge may not necessarily lead to more de-anonymization gain in certain cases. Though our analysis is based on a few assumptions, the findings still leave intriguing implications for the attacker to make better use of the background knowledge when performing de-anonymization, and for the data owners to better measure the privacy risk when releasing their data to third parties. △ Less

Submitted 30 October, 2017; originally announced October 2017.

EEG-Based User Reaction Time Estimation Using Riemannian Geometry Features

Authors: Dongrui Wu, Brent J. Lance, Vernon J. Lawhern, Stephen Gordon, Tzyy-Ping Jung, Chin-Teng Lin

Abstract: Riemannian geometry has been successfully used in many brain-computer interface (BCI) classification problems and demonstrated superior performance. In this paper, for the first time, it is applied to BCI regression problems, an important category of BCI applications. More specifically, we propose a new feature extraction approach for Electroencephalogram (EEG) based BCI regression problems: a spa… ▽ More Riemannian geometry has been successfully used in many brain-computer interface (BCI) classification problems and demonstrated superior performance. In this paper, for the first time, it is applied to BCI regression problems, an important category of BCI applications. More specifically, we propose a new feature extraction approach for Electroencephalogram (EEG) based BCI regression problems: a spatial filter is first used to increase the signal quality of the EEG trials and also to reduce the dimensionality of the covariance matrices, and then Riemannian tangent space features are extracted. We validate the performance of the proposed approach in reaction time estimation from EEG signals measured in a large-scale sustained-attention psychomotor vigilance task, and show that compared with the traditional powerband features, the tangent space features can reduce the root mean square estimation error by 4.30-8.30%, and increase the estimation correlation coefficient by 6.59-11.13%. △ Less

Submitted 27 April, 2017; originally announced April 2017.

Comments: arXiv admin note: text overlap with arXiv:1702.02914

Journal ref: IEEE Trans. on Neural Systems and Rehabilitation Engineering, 25(11), pp. 2157-2168, 2017

Spatial Filtering for EEG-Based Regression Problems in Brain-Computer Interface (BCI)

Authors: Dongrui Wu, Jung-Tai King, Chun-Hsiang Chuang, Chin-Teng Lin, Tzyy-Ping Jung

Abstract: Electroencephalogram (EEG) signals are frequently used in brain-computer interfaces (BCIs), but they are easily contaminated by artifacts and noises, so preprocessing must be done before they are fed into a machine learning algorithm for classification or regression. Spatial filters have been widely used to increase the signal-to-noise ratio of EEG for BCI classification problems, but their applic… ▽ More Electroencephalogram (EEG) signals are frequently used in brain-computer interfaces (BCIs), but they are easily contaminated by artifacts and noises, so preprocessing must be done before they are fed into a machine learning algorithm for classification or regression. Spatial filters have been widely used to increase the signal-to-noise ratio of EEG for BCI classification problems, but their applications in BCI regression problems have been very limited. This paper proposes two common spatial pattern (CSP) filters for EEG-based regression problems in BCI, which are extended from the CSP filter for classification, by making use of fuzzy sets. Experimental results on EEG-based response speed estimation from a large-scale study, which collected 143 sessions of sustained-attention psychomotor vigilance task data from 17 subjects during a 5-month period, demonstrate that the two proposed spatial filters can significantly increase the EEG signal quality. When used in LASSO and k-nearest neighbors regression for user response speed estimation, the spatial filters can reduce the root mean square estimation error by 10.02-19.77%, and at the same time increase the correlation to the true response speed by 19.39-86.47%. △ Less

Submitted 9 February, 2017; originally announced February 2017.

Journal ref: IEEE Trans. on Fuzzy Systems, 26(2), pp. 771-781, 2018

Cloud-based Privacy Preserving Image Storage, Sharing and Search

Authors: Lan Zhang, Taeho Jung, Puchun Feng, Xiang-Yang Li, Yunhao Liu

Abstract: High-resolution cameras produce huge volume of high quality images everyday. It is extremely challenging to store, share and especially search those huge images, for which increasing number of cloud services are presented to support such functionalities. However, images tend to contain rich sensitive information (\eg, people, location and event), and people's privacy concerns hinder their readily… ▽ More High-resolution cameras produce huge volume of high quality images everyday. It is extremely challenging to store, share and especially search those huge images, for which increasing number of cloud services are presented to support such functionalities. However, images tend to contain rich sensitive information (\eg, people, location and event), and people's privacy concerns hinder their readily participation into the services provided by untrusted third parties. In this work, we introduce PIC: a Privacy-preserving large-scale Image search system on Cloud. Our system enables efficient yet secure content-based image search with fine-grained access control, and it also provides privacy-preserving image storage and sharing among users. Users can specify who can/cannot search on their images when using the system, and they can search on others' images if they satisfy the condition specified by the image owners. Majority of the computationally intensive jobs are outsourced to the cloud side, and users only need to submit the query and receive the result throughout the entire image search. Specially, to deal with massive images, we design our system suitable for distributed and parallel computation and introduce several optimizations to further expedite the search process. We implement a prototype of PIC including both cloud side and client side. The cloud side is a cluster of computers with distributed file system (Hadoop HDFS) and MapReduce architecture (Hadoop MapReduce). The client side is built for both Windows OS laptops and Android phones. We evaluate the prototype system with large sets of real-life photos. Our security analysis and evaluation results show that PIC successfully protect the image privacy at a low cost of computation and communication. △ Less

Submitted 24 October, 2014; originally announced October 2014.

Comments: 15 pages, 12 figures

Outsource Photo Sharing and Searching for Mobile Devices With Privacy Protection

Authors: Lan Zhang, Taeho Jung, Cihang Liu, Xuan Ding, Xiang-Yang Li, Yunhao Liu

Abstract: With the proliferation of mobile devices, cloud-based photo sharing and searching services are becoming common due to the mobile devices' resource constrains. Meanwhile, there is also increasing concern about privacy in photos. In this work, we present a framework \ourprotocolNSP, which enables cloud servers to provide privacy-preserving photo sharing and search as a service to mobile device users… ▽ More With the proliferation of mobile devices, cloud-based photo sharing and searching services are becoming common due to the mobile devices' resource constrains. Meanwhile, there is also increasing concern about privacy in photos. In this work, we present a framework \ourprotocolNSP, which enables cloud servers to provide privacy-preserving photo sharing and search as a service to mobile device users. Privacy-seeking users can share their photos via our framework to allow only their authorized friends to browse and search their photos using resource-bounded mobile devices. This is achieved by our carefully designed architecture and novel outsourced privacy-preserving computation protocols, through which no information about the outsourced photos or even the search contents (including the results) would be revealed to the cloud servers. Our framework is compatible with most of the existing image search technologies, and it requires few changes to the existing cloud systems. The evaluation of our prototype system with 31,772 real-life images shows the communication and computation efficiency of our system. △ Less

Submitted 24 October, 2014; originally announced October 2014.

Comments: 10 pages, 13 figures

Spatiotemporal Sparse Bayesian Learning with Applications to Compressed Sensing of Multichannel Physiological Signals

Authors: Zhilin Zhang, Tzyy-Ping Jung, Scott Makeig, Zhouyue Pi, Bhaskar D. Rao

Abstract: Energy consumption is an important issue in continuous wireless telemonitoring of physiological signals. Compressed sensing (CS) is a promising framework to address it, due to its energy-efficient data compression procedure. However, most CS algorithms have difficulty in data recovery due to non-sparsity characteristic of many physiological signals. Block sparse Bayesian learning (BSBL) is an effe… ▽ More Energy consumption is an important issue in continuous wireless telemonitoring of physiological signals. Compressed sensing (CS) is a promising framework to address it, due to its energy-efficient data compression procedure. However, most CS algorithms have difficulty in data recovery due to non-sparsity characteristic of many physiological signals. Block sparse Bayesian learning (BSBL) is an effective approach to recover such signals with satisfactory recovery quality. However, it is time-consuming in recovering multichannel signals, since its computational load almost linearly increases with the number of channels. This work proposes a spatiotemporal sparse Bayesian learning algorithm to recover multichannel signals simultaneously. It not only exploits temporal correlation within each channel signal, but also exploits inter-channel correlation among different channel signals. Furthermore, its computational load is not significantly affected by the number of channels. The proposed algorithm was applied to brain computer interface (BCI) and EEG-based driver's drowsiness estimation. Results showed that the algorithm had both better recovery performance and much higher speed than BSBL. Particularly, the proposed algorithm ensured that the BCI classification and the drowsiness estimation had little degradation even when data were compressed by 80%, making it very suitable for continuous wireless telemonitoring of multichannel signals. △ Less

Submitted 14 November, 2014; v1 submitted 21 April, 2014; originally announced April 2014.

Comments: Codes are available at: https://sites.google.com/site/researchbyzhang/stsbl

Journal ref: IEEE Transactions On Neural Systems And Rehabilitation Engineering, Vol. 22, No. 6, pp. 1186-1197, November 2014

Compressed Sensing for Energy-Efficient Wireless Telemonitoring: Challenges and Opportunities

Authors: Zhilin Zhang, Bhaskar D. Rao, Tzyy-Ping Jung

Abstract: As a lossy compression framework, compressed sensing has drawn much attention in wireless telemonitoring of biosignals due to its ability to reduce energy consumption and make possible the design of low-power devices. However, the non-sparseness of biosignals presents a major challenge to compressed sensing. This study proposes and evaluates a spatio-temporal sparse Bayesian learning algorithm, wh… ▽ More As a lossy compression framework, compressed sensing has drawn much attention in wireless telemonitoring of biosignals due to its ability to reduce energy consumption and make possible the design of low-power devices. However, the non-sparseness of biosignals presents a major challenge to compressed sensing. This study proposes and evaluates a spatio-temporal sparse Bayesian learning algorithm, which has the desired ability to recover such non-sparse biosignals. It exploits both temporal correlation in each individual biosignal and inter-channel correlation among biosignals from different channels. The proposed algorithm was used for compressed sensing of multichannel electroencephalographic (EEG) signals for estimating vehicle drivers' drowsiness. Results showed that the drowsiness estimation was almost unaffected even if raw EEG signals (containing various artifacts) were compressed by 90%. △ Less

Submitted 21 April, 2014; v1 submitted 15 November, 2013; originally announced November 2013.

Comments: Invited paper for 2013 Asilomar Conference on Signals, Systems & Computers (Asilomar 2013)

SmartLoc: Sensing Landmarks Silently for Smartphone Based Metropolitan Localization

Authors: Cheng Bo, Xiang-Yang Li, Taeho Jung, Xufei Mao

Abstract: We present \emph{SmartLoc}, a localization system to estimate the location and the traveling distance by leveraging the lower-power inertial sensors embedded in smartphones as a supplementary to GPS. To minimize the negative impact of sensor noises, \emph{SmartLoc} exploits the intermittent strong GPS signals and uses the linear regression to build a prediction model which is based on the trace es… ▽ More We present \emph{SmartLoc}, a localization system to estimate the location and the traveling distance by leveraging the lower-power inertial sensors embedded in smartphones as a supplementary to GPS. To minimize the negative impact of sensor noises, \emph{SmartLoc} exploits the intermittent strong GPS signals and uses the linear regression to build a prediction model which is based on the trace estimated from inertial sensors and the one computed from the GPS. Furthermore, we utilize landmarks (e.g., bridge, traffic lights) detected automatically and special driving patterns (e.g., turning, uphill, and downhill) from inertial sensory data to improve the localization accuracy when the GPS signal is weak. Our evaluations of \emph{SmartLoc} in the city demonstrates its technique viability and significant localization accuracy improvement compared with GPS and other approaches: the error is approximately 20m for 90% of time while the known mean error of GPS is 42.22m. △ Less

Submitted 31 August, 2013; originally announced October 2013.

Enabling Privacy-preserving Auctions in Big Data

Authors: Taeho Jung, Xiang-Yang Li

Abstract: We study how to enable auctions in the big data context to solve many upcoming data-based decision problems in the near future. We consider the characteristics of the big data including, but not limited to, velocity, volume, variety, and veracity, and we believe any auction mechanism design in the future should take the following factors into consideration: 1) generality (variety); 2) efficiency a… ▽ More We study how to enable auctions in the big data context to solve many upcoming data-based decision problems in the near future. We consider the characteristics of the big data including, but not limited to, velocity, volume, variety, and veracity, and we believe any auction mechanism design in the future should take the following factors into consideration: 1) generality (variety); 2) efficiency and scalability (velocity and volume); 3) truthfulness and verifiability (veracity). In this paper, we propose a privacy-preserving construction for auction mechanism design in the big data, which prevents adversaries from learning unnecessary information except those implied in the valid output of the auction. More specifically, we considered one of the most general form of the auction (to deal with the variety), and greatly improved the the efficiency and scalability by approximating the NP-hard problems and avoiding the design based on garbled circuits (to deal with velocity and volume), and finally prevented stakeholders from lying to each other for their own benefit (to deal with the veracity). We achieve these by introducing a novel privacy-preserving winner determination algorithm and a novel payment mechanism. Additionally, we further employ a blind signature scheme as a building block to let bidders verify the authenticity of their payment reported by the auctioneer. The comparison with peer work shows that we improve the asymptotic performance of peer works' overhead from the exponential growth to a linear growth and from linear growth to a logarithmic growth, which greatly improves the scalability. △ Less

Submitted 21 January, 2015; v1 submitted 28 August, 2013; originally announced August 2013.

Journal ref: 2015 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), pp 173-178

PDA: Semantically Secure Time-Series Data Analytics with Dynamic Subgroups

Authors: Taeho Jung, Junze Han, Xiang-Yang Li

Abstract: Third-party analysis on private records is becoming increasingly important due to the widespread data collection for various analysis purposes. However, the data in its original form often contains sensitive information about individuals, and its publication will severely breach their privacy. In this paper, we present a novel Privacy-preserving Data Analytics framework PDA, which allows a third-p… ▽ More Third-party analysis on private records is becoming increasingly important due to the widespread data collection for various analysis purposes. However, the data in its original form often contains sensitive information about individuals, and its publication will severely breach their privacy. In this paper, we present a novel Privacy-preserving Data Analytics framework PDA, which allows a third-party aggregator to obliviously conduct many different types of polynomial-based analysis on private data records provided by a dynamic sub-group of users. Notably, every user needs to keep only O(n) keys to join data analysis among O(2^n) different groups of users, and any data analysis that is represented by polynomials is supported by our framework. Besides, a real implementation shows the performance of our framework is comparable to the peer works who present ad-hoc solutions for specific data analysis applications. Despite such nice properties of PDA, it is provably secure against a very powerful attacker (chosen-plaintext attack) even in the Dolev-Yao network model where all communication channels are insecure. △ Less

Submitted 25 September, 2015; v1 submitted 28 August, 2013; originally announced August 2013.

A General Framework for Privacy-Preserving Distributed Greedy Algorithm

Authors: Taeho Jung, Xiang-Yang Li, Lan Zhang

Abstract: Increasingly more attention is paid to the privacy in online applications due to the widespread data collection for various analysis purposes. Sensitive information might be mined from the raw data during the analysis, and this led to a great privacy concern among people (data providers) these days. To deal with this privacy concerns, multitudes of privacy-preserving computation schemes are propos… ▽ More Increasingly more attention is paid to the privacy in online applications due to the widespread data collection for various analysis purposes. Sensitive information might be mined from the raw data during the analysis, and this led to a great privacy concern among people (data providers) these days. To deal with this privacy concerns, multitudes of privacy-preserving computation schemes are proposed to address various computation problems, and we have found many of them fall into a class of problems which can be solved by greedy algorithms. In this paper, we propose a framework for distributed greedy algorithms in which instances in the feasible set come from different parties. By our framework, most generic distributed greedy algorithms can be converted to a privacy preserving one which achieves the same result as the original greedy algorithm while the private information associated with the instances is still protected. △ Less

Submitted 25 July, 2014; v1 submitted 8 July, 2013; originally announced July 2013.

Comments: This paper has been withdrawn due to personal reasons

Journal ref: BigCom 2015, LNCS 9196, pp. 88-102, 2015