Showing 1–50 of 62 results for author: Su, P

Forward and Backward State Abstractions for Off-policy Evaluation

Authors: Meiling Hao, Pingfan Su, Liyuan Hu, Zoltan Szabo, Qingyuan Zhao, Chengchun Shi

Abstract: Off-policy evaluation (OPE) is crucial for evaluating a target policy's impact offline before its deployment. However, achieving accurate OPE in large state spaces remains challenging.This paper studies state abstractions-originally designed for policy learning-in the context of OPE. Our contributions are three-fold: (i) We define a set of irrelevance conditions central to learning state abstracti… ▽ More Off-policy evaluation (OPE) is crucial for evaluating a target policy's impact offline before its deployment. However, achieving accurate OPE in large state spaces remains challenging.This paper studies state abstractions-originally designed for policy learning-in the context of OPE. Our contributions are three-fold: (i) We define a set of irrelevance conditions central to learning state abstractions for OPE. (ii) We derive sufficient conditions for achieving irrelevance in Q-functions and marginalized importance sampling ratios, the latter obtained by constructing a time-reversed Markov decision process (MDP) based on the observed MDP. (iii) We propose a novel two-step procedure that sequentially projects the original state space into a smaller space, which substantially simplify the sample complexity of OPE arising from high cardinality. △ Less

Submitted 27 June, 2024; originally announced June 2024.

Comments: 42 pages, 5 figures

ACM Class: G.3; I.2.6; G.1.2

JNI Global References Are Still Vulnerable: Attacks and Defenses

Authors: Yi He, Yuan Zhou, Yacong Gu, Purui Su, Qi Li, Yajin Zhou, Yong Jiang

Abstract: System services and resources in Android are accessed through IPC based mechanisms. Previous research has demonstrated that they are vulnerable to the denial-of-service attack (DoS attack). For instance, the JNI global reference (JGR), which is widely used by system services, can be exhausted to cause the system reboot (hence the name JGRE attack). Even though the Android team tries to fix the pro… ▽ More System services and resources in Android are accessed through IPC based mechanisms. Previous research has demonstrated that they are vulnerable to the denial-of-service attack (DoS attack). For instance, the JNI global reference (JGR), which is widely used by system services, can be exhausted to cause the system reboot (hence the name JGRE attack). Even though the Android team tries to fix the problem by enforcing security checks, we find that it is still possible to construct a JGR exhaustion DoS attack in the latest Android system. In this paper, we propose a new JGR exhaustion DoS attack, which is effective in different Android versions, including the latest one (i.e., Android 10). Specifically, we developed JGREAnalyzer, a tool that can systematically detect JGR vulnerable services APIs via a call graph analysis and a forwarding reachability analysis. We applied this tool to different Android versions and found multiple vulnerabilities. In particular, among 148 system services in Android 10, 12 of them have 21 vulnerabilities. Among them, 9 can be successfully exploited without any permissions. We further analyze the root cause of the vulnerabilities and propose a new defense to mitigate the JGRE attack by restricting resource consumption via global reference counting. △ Less

Submitted 1 May, 2024; originally announced May 2024.

Incentive-Compatible Vertiport Reservation in Advanced Air Mobility: An Auction-Based Approach

Authors: Pan-Yang Su, Chinmay Maheshwari, Victoria Tuck, Shankar Sastry

Abstract: The rise of advanced air mobility (AAM) is expected to become a multibillion-dollar industry in the near future. Market-based mechanisms are touted to be an integral part of AAM operations, which comprise heterogeneous operators with private valuations. In this work, we study the problem of designing a mechanism to coordinate the movement of electric vertical take-off and landing (eVTOL) aircraft,… ▽ More The rise of advanced air mobility (AAM) is expected to become a multibillion-dollar industry in the near future. Market-based mechanisms are touted to be an integral part of AAM operations, which comprise heterogeneous operators with private valuations. In this work, we study the problem of designing a mechanism to coordinate the movement of electric vertical take-off and landing (eVTOL) aircraft, operated by multiple operators each having heterogeneous valuations associated with their fleet, between vertiports, while enforcing the arrival, departure, and parking constraints at vertiports. Particularly, we propose an incentive-compatible and individually rational vertiport reservation mechanism that maximizes a social welfare metric, which encapsulates the objective of maximizing the overall valuations of all operators while minimizing the congestion at vertiports. Additionally, we improve the computational tractability of designing the reservation mechanism by proposing a mixed binary linear programming approach that is based on constructing network flow graph corresponding to the underlying problem. △ Less

Submitted 7 July, 2024; v1 submitted 26 March, 2024; originally announced March 2024.

Comments: 26 pages, 2 figures, 1 table

MSC Class: 91B03; 91A68; 90B06; 90C27

Bodioid: philosophical reflections on the hybrid of bodies and artefacts towards post-human

Authors: Jiang Xu, Gang Sun, Jingyu Xu, Pujie Su

Abstract: The advent of the post-human era has blurred the boundary between the body and artefacts. Further, external materials and information are more deeply integrated into the body, making emerging technology a key driving force for shaping post-human existence and promoting bodily evolution. Based on this, this study analyses the transformation process of three technological forms, namely tools, machin… ▽ More The advent of the post-human era has blurred the boundary between the body and artefacts. Further, external materials and information are more deeply integrated into the body, making emerging technology a key driving force for shaping post-human existence and promoting bodily evolution. Based on this, this study analyses the transformation process of three technological forms, namely tools, machines, and cyborgs, and reveals the construction of bodies and artefacts. From the phenomenological perspective, the essences of body and artefact existences are reflected upon, and the 'existence is construction' viewpoint is proposed. Furthermore, a technological design concept, 'bodioid', is proposed to meticulously depict the characteristics of integrating similarities and differences towards unity between the body and artefacts, based on the theoretical foundation of technology mediation and the materialization of morality. Finally, through analogizing the organizational form of language, the two key forms and specific mechanisms of bodioid construction, namely extension and mirroring, are indicated. With this in mind, the post-human existence landscape is discussed with the objective of providing theoretical insights into the study of the underlying philosophical principles of technological design. △ Less

Submitted 5 March, 2024; originally announced March 2024.

Single-pixel 3D imaging based on fusion temporal data of single photon detector and millimeter-wave radar

Authors: Tingqin Lai, Xiaolin Liang, Yi Zhu, Xinyi Wu, Lianye Liao, Xuelin Yuan, Ping Su, Shihai Sun

Abstract: Recently, there has been increased attention towards 3D imaging using single-pixel single-photon detection (also known as temporal data) due to its potential advantages in terms of cost and power efficiency. However, to eliminate the symmetry blur in the reconstructed images, a fixed background is required. This paper proposes a fusion-data-based 3D imaging method that utilizes a single-pixel sing… ▽ More Recently, there has been increased attention towards 3D imaging using single-pixel single-photon detection (also known as temporal data) due to its potential advantages in terms of cost and power efficiency. However, to eliminate the symmetry blur in the reconstructed images, a fixed background is required. This paper proposes a fusion-data-based 3D imaging method that utilizes a single-pixel single-photon detector and a millimeter-wave radar to capture temporal histograms of a scene from multiple perspectives. Subsequently, the 3D information can be reconstructed from the one-dimensional fusion temporal data by using Artificial Neural Network (ANN). Both the simulation and experimental results demonstrate that our fusion method effectively eliminates symmetry blur and improves the quality of the reconstructed images. △ Less

Submitted 20 October, 2023; originally announced December 2023.

Comments: Accepted by Chinese Optics Letters, and comments are welcome

Journal ref: Chinese Optics Letters, Vol.2, No.2, 2024

Reveal the Mathematical Structures of Honeyword Security Metrics

Authors: Pengcheng Su, Haibo Cheng, Wenting Li, Ping Wang

Abstract: Honeyword is a representative ``honey" technique to detect intruders by luring them with decoy data. This kind of honey technique blends a primary object (from distribution $P$) with decoy samples (from distribution $Q$). In this research, we focus on two key Honeyword security metrics: the flatness function and the success-number function. Previous researchers are engaged in designing experimenta… ▽ More Honeyword is a representative ``honey" technique to detect intruders by luring them with decoy data. This kind of honey technique blends a primary object (from distribution $P$) with decoy samples (from distribution $Q$). In this research, we focus on two key Honeyword security metrics: the flatness function and the success-number function. Previous researchers are engaged in designing experimental methods to estimate their values. We've derived theoretical formulas on both metrics of the strongest $\mathcal{A}$ using the optimal guessing strategy, marking a first in the field. The mathematical structures of these metrics are intriguing: the flatness function has an expression as $ε(i)=\sum_{j=1}^{i}\int_{0}^{+\infty}\tbinom{k-1}{j-1} f(x)G^{k-j}(x)(1-G(x))^{j-1}dx$. In particular, the most important one, $ε(1)$ is $\frac{1}{k}(M-\int_{0}^{M}G^k(x)dx)+b$, where $M=\max_{x: Q(x)\neq 0}\frac{P(x)}{Q(x)}$, $b=\sum_{x: Q(x)=0}P(x)$, and $G$ is a cumulative distribution function derived from $P$ and $Q$. This formula provides a criterion to compare different honey distributions: the one with smaller $M$ and $b$ is more satisfactory. The mathematical structure of the success-number function is a series of convolutions with beta distribution kernels: $λ_U(i)=U\sum_{j=1}^{i}\int_{\frac{1}{k}}^{1} \frac{φ(x)}{1-φ(x)} \tbinom{U-1}{j-1} x^{U-j}(1-x)^{j-1}dx$, where $U$ is the number of users in the system and $φ(x)$ is a monotonically increasing function. For further elaboration, we made some representative calculations. Our findings offer insights into security assessments for Honeyword and similar honey techniques, contributing to enhanced security measures in these systems. △ Less

Submitted 17 November, 2023; originally announced November 2023.

A Brief Survey of Open Radio Access Network (O-RAN) Security

Authors: Yi-Zih Chen, Terrance Yu-Hao Chen, Po-Jung Su, Chi-Ting Liu

Abstract: Open Radio Access Network (O-RAN), a novel architecture that separates the traditional radio access network (RAN) into multiple disaggregated components, leads a revolution in the telecommunication ecosystems. Compared to the traditional RAN, the proposed O-RAN paradigm is more flexible and more cost-effective for the operators, vendors, and the public. The key design considerations of O-RAN inclu… ▽ More Open Radio Access Network (O-RAN), a novel architecture that separates the traditional radio access network (RAN) into multiple disaggregated components, leads a revolution in the telecommunication ecosystems. Compared to the traditional RAN, the proposed O-RAN paradigm is more flexible and more cost-effective for the operators, vendors, and the public. The key design considerations of O-RAN include virtualization and intelligent capabilities in order to meet the new requirements of 5G. However, because of the open nature and the newly imported techniques in O-RAN architecture, the assessment of the security in O-RAN architecture during its early development stage is crucial. This project aims to present an investigation of the current ORAN architecture from several attack surfaces, including (1) Architectural openness, (2) Cloud and Virtualization, (3) Network slicing, and (4) Machine Learning. The existing attack surfaces and corresponding mitigation methods of these attacks are also surveyed and provided in this report, serving as a guiding principle and valuable recommendation for the O-RAN implementers and framework designers. △ Less

Submitted 3 November, 2023; originally announced November 2023.

Technical Note: Feasibility of translating 3.0T-trained Deep-Learning Segmentation Models Out-of-the-Box on Low-Field MRI 0.55T Knee-MRI of Healthy Controls

Authors: Rupsa Bhattacharjee, Zehra Akkaya, Johanna Luitjens, Pan Su, Yang Yang, Valentina Pedoia, Sharmila Majumdar

Abstract: In the current study, our purpose is to evaluate the feasibility of applying deep learning (DL) enabled algorithms to quantify bilateral knee biomarkers in healthy controls scanned at 0.55T and compared with 3.0T. The current study assesses the performance of standard in-practice bone, and cartilage segmentation algorithms at 0.55T, both qualitatively and quantitatively, in terms of comparing segm… ▽ More In the current study, our purpose is to evaluate the feasibility of applying deep learning (DL) enabled algorithms to quantify bilateral knee biomarkers in healthy controls scanned at 0.55T and compared with 3.0T. The current study assesses the performance of standard in-practice bone, and cartilage segmentation algorithms at 0.55T, both qualitatively and quantitatively, in terms of comparing segmentation performance, areas of improvement, and compartment-wise cartilage thickness values between 0.55T vs. 3.0T. Initial results demonstrate a usable to good technical feasibility of translating existing quantitative deep-learning-based image segmentation techniques, trained on 3.0T, out of 0.55T for knee MRI, in a multi-vendor acquisition environment. Especially in terms of segmenting cartilage compartments, the models perform almost equivalent to 3.0T in terms of Likert ranking. The 0.55T low-field sustainable and easy-to-install MRI, as demonstrated, thus, can be utilized for evaluating knee cartilage thickness and bone segmentations aided by established DL algorithms trained at higher-field strengths out-of-the-box initially. This could be utilized at the far-spread point-of-care locations with a lack of radiologists available to manually segment low-field images, at least till a decent base of low-field data pool is collated. With further fine-tuning with manual labeling of low-field data or utilizing synthesized higher SNR images from low-field images, OA biomarker quantification performance is potentially guaranteed to be further improved. △ Less

Submitted 26 October, 2023; originally announced October 2023.

Comments: 11 Pages, 3 Figures, 2 Tables

Recovering from Privacy-Preserving Masking with Large Language Models

Authors: Arpita Vats, Zhe Liu, Peng Su, Debjyoti Paul, Yingyi Ma, Yutong Pang, Zeeshan Ahmed, Ozlem Kalinli

Abstract: Model adaptation is crucial to handle the discrepancy between proxy training data and actual users data received. To effectively perform adaptation, textual data of users is typically stored on servers or their local devices, where downstream natural language processing (NLP) models can be directly trained using such in-domain data. However, this might raise privacy and security concerns due to th… ▽ More Model adaptation is crucial to handle the discrepancy between proxy training data and actual users data received. To effectively perform adaptation, textual data of users is typically stored on servers or their local devices, where downstream natural language processing (NLP) models can be directly trained using such in-domain data. However, this might raise privacy and security concerns due to the extra risks of exposing user information to adversaries. Replacing identifying information in textual data with a generic marker has been recently explored. In this work, we leverage large language models (LLMs) to suggest substitutes of masked tokens and have their effectiveness evaluated on downstream language modeling tasks. Specifically, we propose multiple pre-trained and fine-tuned LLM-based approaches and perform empirical studies on various datasets for the comparison of these methods. Experimental results show that models trained on the obfuscation corpora are able to achieve comparable performance with the ones trained on the original data without privacy-preserving token masking. △ Less

Submitted 13 December, 2023; v1 submitted 12 September, 2023; originally announced September 2023.

Comments: Accepted to ICASSP

3D Semantic Subspace Traverser: Empowering 3D Generative Model with Shape Editing Capability

Authors: Ruowei Wang, Yu Liu, Pei Su, Jianwei Zhang, Qijun Zhao

Abstract: Shape generation is the practice of producing 3D shapes as various representations for 3D content creation. Previous studies on 3D shape generation have focused on shape quality and structure, without or less considering the importance of semantic information. Consequently, such generative models often fail to preserve the semantic consistency of shape structure or enable manipulation of the seman… ▽ More Shape generation is the practice of producing 3D shapes as various representations for 3D content creation. Previous studies on 3D shape generation have focused on shape quality and structure, without or less considering the importance of semantic information. Consequently, such generative models often fail to preserve the semantic consistency of shape structure or enable manipulation of the semantic attributes of shapes during generation. In this paper, we proposed a novel semantic generative model named 3D Semantic Subspace Traverser that utilizes semantic attributes for category-specific 3D shape generation and editing. Our method utilizes implicit functions as the 3D shape representation and combines a novel latent-space GAN with a linear subspace model to discover semantic dimensions in the local latent space of 3D shapes. Each dimension of the subspace corresponds to a particular semantic attribute, and we can edit the attributes of generated shapes by traversing the coefficients of those dimensions. Experimental results demonstrate that our method can produce plausible shapes with complex structures and enable the editing of semantic attributes. The code and trained models are available at https://github.com/TrepangCat/3D_Semantic_Subspace_Traverser △ Less

Submitted 15 August, 2023; v1 submitted 26 July, 2023; originally announced July 2023.

Comments: Published in ICCV 2023. Code: https://github.com/TrepangCat/3D_Semantic_Subspace_Traverser

Discovery of Optimal Quantum Error Correcting Codes via Reinforcement Learning

Authors: Vincent Paul Su, ChunJun Cao, Hong-Ye Hu, Yariv Yanay, Charles Tahan, Brian Swingle

Abstract: The recently introduced Quantum Lego framework provides a powerful method for generating complex quantum error correcting codes (QECCs) out of simple ones. We gamify this process and unlock a new avenue for code design and discovery using reinforcement learning (RL). One benefit of RL is that we can specify \textit{arbitrary} properties of the code to be optimized. We train on two such properties,… ▽ More The recently introduced Quantum Lego framework provides a powerful method for generating complex quantum error correcting codes (QECCs) out of simple ones. We gamify this process and unlock a new avenue for code design and discovery using reinforcement learning (RL). One benefit of RL is that we can specify \textit{arbitrary} properties of the code to be optimized. We train on two such properties, maximizing the code distance, and minimizing the probability of logical error under biased Pauli noise. For the first, we show that the trained agent identifies ways to increase code distance beyond naive concatenation, saturating the linear programming bound for CSS codes on 13 qubits. With a learning objective to minimize the logical error probability under biased Pauli noise, we find the best known CSS code at this task for $\lesssim 20$ qubits. Compared to other (locally deformed) CSS codes, including Surface, XZZX, and 2D Color codes, our $[[17,1,3]]$ code construction actually has \textit{lower} adversarial distance, yet better protects the logical information, highlighting the importance of QECC desiderata. Lastly, we comment on how this RL framework can be used in conjunction with physical quantum devices to tailor a code without explicit characterization of the noise model. △ Less

Submitted 12 June, 2023; v1 submitted 10 May, 2023; originally announced May 2023.

Comments: 10 pages + appendices; v2 figure updated and note added

Language Agnostic Data-Driven Inverse Text Normalization

Authors: Szu-Jui Chen, Debjyoti Paul, Yutong Pang, Peng Su, Xuedong Zhang

Abstract: With the emergence of automatic speech recognition (ASR) models, converting the spoken form text (from ASR) to the written form is in urgent need. This inverse text normalization (ITN) problem attracts the attention of researchers from various fields. Recently, several works show that data-driven ITN methods can output high-quality written form text. Due to the scarcity of labeled spoken-written d… ▽ More With the emergence of automatic speech recognition (ASR) models, converting the spoken form text (from ASR) to the written form is in urgent need. This inverse text normalization (ITN) problem attracts the attention of researchers from various fields. Recently, several works show that data-driven ITN methods can output high-quality written form text. Due to the scarcity of labeled spoken-written datasets, the studies on non-English data-driven ITN are quite limited. In this work, we propose a language-agnostic data-driven ITN framework to fill this gap. Specifically, we leverage the data augmentation in conjunction with neural machine translated data for low resource languages. Moreover, we design an evaluation method for language agnostic ITN model when only English data is available. Our empirical evaluation shows this language agnostic modeling approach is effective for low resource languages while preserving the performance for high resource languages. △ Less

Submitted 23 January, 2023; v1 submitted 20 January, 2023; originally announced January 2023.

Towards Quantum Gravity in the Lab on Quantum Processors

Authors: Illya Shapoval, Vincent Paul Su, Wibe de Jong, Miro Urbanek, Brian Swingle

Abstract: The holographic principle and its realization in the AdS/CFT correspondence led to unexpected connections between general relativity and quantum information. This set the stage for studying aspects of quantum gravity models, which are otherwise difficult to access, in table-top quantum-computational experiments. Recent works have designed a special teleportation protocol that realizes a surprising… ▽ More The holographic principle and its realization in the AdS/CFT correspondence led to unexpected connections between general relativity and quantum information. This set the stage for studying aspects of quantum gravity models, which are otherwise difficult to access, in table-top quantum-computational experiments. Recent works have designed a special teleportation protocol that realizes a surprising communication phenomenon most naturally explained by the physics of a traversable wormhole. In this work, we have carried out quantum experiments based on this protocol on state-of-the-art quantum computers. The target quantum processing units (QPUs) included the Quantinuum's trapped-ion System Model H1-1 and five IBM superconducting QPUs of various architectures, with public and premium user access. We report the observed teleportation signals from these QPUs with the best one reaching 80% of theoretical predictions. We outline the experimental challenges we have faced in the course of implementation, as well as the new theoretical insights into quantum dynamics the work has led to. We also developed QGLab -- an open-source end-to-end software solution that facilitates conducting the wormhole-inspired teleportation experiments on state-of-the-art and emergent generations of QPUs supported by the Qiskit and tket SDKs. We consider our study and deliverables as an early practical step towards the realization of more complex experiments for the indirect probing of quantum gravity in the lab. △ Less

Submitted 11 October, 2023; v1 submitted 27 May, 2022; originally announced May 2022.

Comments: 21 pages, 6 figures, 2 tables, 1 listing; updated to match journal

Journal ref: Quantum 7, 1138 (2023)

OJXPerf: Featherlight Object Replica Detection for Java Programs

Authors: Bolun Li, Hao Xu, Qidong Zhao, Pengfei Su, Milind Chabbi, Shuyin Jiao, Xu Liu

Abstract: Memory bloat is an important source of inefficiency in complex production software, especially in software written in managed languages such as Java. Prior approaches to this problem have focused on identifying objects that outlive their life span. Few studies have, however, looked into whether and to what extent myriad objects of the same type are identical. A quantitative assessment of identical… ▽ More Memory bloat is an important source of inefficiency in complex production software, especially in software written in managed languages such as Java. Prior approaches to this problem have focused on identifying objects that outlive their life span. Few studies have, however, looked into whether and to what extent myriad objects of the same type are identical. A quantitative assessment of identical objects with code-level attribution can assist developers in refactoring code to eliminate object bloat, and favor reuse of existing object(s). The result is reduced memory pressure, reduced allocation and garbage collection, enhanced data locality, and reduced re-computation, all of which result in superior performance. We develop OJXPerf, a lightweight sampling-based profiler, which probabilistically identifies identical objects. OJXPerf employs hardware performance monitoring units (PMU) in conjunction with hardware debug registers to sample and compare field values of different objects of the same type allocated at the same calling context but potentially accessed at different program points. The result is a lightweight measurement, a combination of object allocation contexts and usage contexts ordered by duplication frequency. This class of duplicated objects is relatively easier to optimize. OJXPerf incurs 9% runtime and 6% memory overheads on average. We empirically show the benefit of OJXPerf by using its profiles to instruct us to optimize a number of Java programs, including well-known benchmarks and real-world applications. The results show a noticeable reduction in memory usage (up to 11%) and a significant speedup (up to 25%). △ Less

Submitted 23 March, 2022; originally announced March 2022.

Journal ref: 44th International Conference on Software Engineering (ICSE 2022)

A Systematic Study of Android Non-SDK (Hidden) Service API Security

Authors: Yi He, Yacong Gu, Purui Su, Kun Sun, Yajin Zhou, Zhi Wang, Qi Li

Abstract: Android allows apps to communicate with its system services via system service helpers so that these apps can use various functions provided by the system services. Meanwhile, the system services rely on their service helpers to enforce security checks for protection. Unfortunately, the security checks in the service helpers may be bypassed via directly exploiting the non-SDK (hidden) APIs, degrad… ▽ More Android allows apps to communicate with its system services via system service helpers so that these apps can use various functions provided by the system services. Meanwhile, the system services rely on their service helpers to enforce security checks for protection. Unfortunately, the security checks in the service helpers may be bypassed via directly exploiting the non-SDK (hidden) APIs, degrading the stability and posing severe security threats such as privilege escalation, automatic function execution without users' interactions, crashes, and DoS attacks. Google has proposed various approaches to address this problem, e.g., case-by-case fixing the bugs or even proposing a blacklist to block all the non-SDK APIs. However, the developers can still figure out new ways of exploiting these hidden APIs to evade the non-SDKs restrictions. In this paper, we systematically study the vulnerabilities due to the hidden API exploitation and analyze the effectiveness of Google's countermeasures. We aim to answer if there are still vulnerable hidden APIs that can be exploited in the newest Android 12. We develop a static analysis tool called ServiceAudit to automatically mine the inconsistent security enforcement between service helper classes and the hidden service APIs. We apply ServiceAudit to Android 6~12. Our tool discovers 112 vulnerabilities in Android 6 with higher precision than existing approaches. Moreover, in Android 11 and 12, we identify more than 25 hidden APIs with inconsistent protections; however, only one of the vulnerable APIs can lead to severe security problems in Android 11, and none of them work on Android 12. △ Less

Submitted 17 March, 2022; originally announced March 2022.

Journal ref: 10.1109/TDSC.2022.3160872

Motion Prediction via Joint Dependency Modeling in Phase Space

Authors: Pengxiang Su, Zhenguang Liu, Shuang Wu, Lei Zhu, Yifang Yin, Xuanjing Shen

Abstract: Motion prediction is a classic problem in computer vision, which aims at forecasting future motion given the observed pose sequence. Various deep learning models have been proposed, achieving state-of-the-art performance on motion prediction. However, existing methods typically focus on modeling temporal dynamics in the pose space. Unfortunately, the complicated and high dimensionality nature of h… ▽ More Motion prediction is a classic problem in computer vision, which aims at forecasting future motion given the observed pose sequence. Various deep learning models have been proposed, achieving state-of-the-art performance on motion prediction. However, existing methods typically focus on modeling temporal dynamics in the pose space. Unfortunately, the complicated and high dimensionality nature of human motion brings inherent challenges for dynamic context capturing. Therefore, we move away from the conventional pose based representation and present a novel approach employing a phase space trajectory representation of individual joints. Moreover, current methods tend to only consider the dependencies between physically connected joints. In this paper, we introduce a novel convolutional neural model to effectively leverage explicit prior knowledge of motion anatomy, and simultaneously capture both spatial and temporal information of joint trajectory dynamics. We then propose a global optimization module that learns the implicit relationships between individual joint features. Empirically, our method is evaluated on large-scale 3D human motion benchmark datasets (i.e., Human3.6M, CMU MoCap). These results demonstrate that our method sets the new state-of-the-art on the benchmark datasets. Our code will be available at https://github.com/Pose-Group/TEID. △ Less

Submitted 7 January, 2022; originally announced January 2022.

Measuring Outcomes in Healthcare Economics using Artificial Intelligence: with Application to Resource Management

Authors: Chih-Hao Huang, Feras A. Batarseh, Adel Boueiz, Ajay Kulkarni, Po-Hsuan Su, Jahan Aman

Abstract: The quality of service in healthcare is constantly challenged by outlier events such as pandemics (i.e. Covid-19) and natural disasters (such as hurricanes and earthquakes). In most cases, such events lead to critical uncertainties in decision making, as well as in multiple medical and economic aspects at a hospital. External (geographic) or internal factors (medical and managerial), lead to shift… ▽ More The quality of service in healthcare is constantly challenged by outlier events such as pandemics (i.e. Covid-19) and natural disasters (such as hurricanes and earthquakes). In most cases, such events lead to critical uncertainties in decision making, as well as in multiple medical and economic aspects at a hospital. External (geographic) or internal factors (medical and managerial), lead to shifts in planning and budgeting, but most importantly, reduces confidence in conventional processes. In some cases, support from other hospitals proves necessary, which exacerbates the planning aspect. This manuscript presents three data-driven methods that provide data-driven indicators to help healthcare managers organize their economics and identify the most optimum plan for resources allocation and sharing. Conventional decision-making methods fall short in recommending validated policies for managers. Using reinforcement learning, genetic algorithms, traveling salesman, and clustering, we experimented with different healthcare variables and presented tools and outcomes that could be applied at health institutes. Experiments are performed; the results are recorded, evaluated, and presented. △ Less

Submitted 14 November, 2021; originally announced November 2021.

Comments: This paper is published at Cambridge University Press Journal of Data & Policy

Journal ref: Data & Policy, 3, E30

ConvFiT: Conversational Fine-Tuning of Pretrained Language Models

Authors: Ivan Vulić, Pei-Hao Su, Sam Coope, Daniela Gerz, Paweł Budzianowski, Iñigo Casanueva, Nikola Mrkšić, Tsung-Hsien Wen

Abstract: Transformer-based language models (LMs) pretrained on large text collections are proven to store a wealth of semantic knowledge. However, 1) they are not effective as sentence encoders when used off-the-shelf, and 2) thus typically lag behind conversationally pretrained (e.g., via response selection) encoders on conversational tasks such as intent detection (ID). In this work, we propose ConvFiT,… ▽ More Transformer-based language models (LMs) pretrained on large text collections are proven to store a wealth of semantic knowledge. However, 1) they are not effective as sentence encoders when used off-the-shelf, and 2) thus typically lag behind conversationally pretrained (e.g., via response selection) encoders on conversational tasks such as intent detection (ID). In this work, we propose ConvFiT, a simple and efficient two-stage procedure which turns any pretrained LM into a universal conversational encoder (after Stage 1 ConvFiT-ing) and task-specialised sentence encoder (after Stage 2). We demonstrate that 1) full-blown conversational pretraining is not required, and that LMs can be quickly transformed into effective conversational encoders with much smaller amounts of unannotated data; 2) pretrained LMs can be fine-tuned into task-specialised sentence encoders, optimised for the fine-grained semantics of a particular task. Consequently, such specialised sentence encoders allow for treating ID as a simple semantic similarity task based on interpretable nearest neighbours retrieval. We validate the robustness and versatility of the ConvFiT framework with such similarity-based inference on the standard ID evaluation sets: ConvFiT-ed LMs achieve state-of-the-art ID performance across the board, with particular gains in the most challenging, few-shot setups. △ Less

Submitted 21 September, 2021; originally announced September 2021.

Comments: EMNLP 2021 (long paper)

Quantum-Inspired Keyword Search on Multi-Model Databases

Authors: Gongsheng Yuan, Jiaheng Lu, Peifeng Su

Abstract: With the rising applications implemented in different domains, it is inevitable to require databases to adopt corresponding appropriate data models to store and exchange data derived from various sources. To handle these data models in a single platform, the community of databases introduces a multi-model database. And many vendors are improving their products from supporting a single data model t… ▽ More With the rising applications implemented in different domains, it is inevitable to require databases to adopt corresponding appropriate data models to store and exchange data derived from various sources. To handle these data models in a single platform, the community of databases introduces a multi-model database. And many vendors are improving their products from supporting a single data model to being multi-model databases. Although this brings benefits, spending lots of enthusiasm to master one of the multi-model query languages for exploring a database is unfriendly to most users. Therefore, we study using keyword searches as an alternative way to explore and query multi-model databases. In this paper, we attempt to utilize quantum physics's probabilistic formalism to bring the problem into vector spaces and represent events (e.g., words) as subspaces. Then we employ a density matrix to encapsulate all the information over these subspaces and use density matrices to measure the divergence between query and candidate answers for finding top-\textit{k} the most relevant results. In this process, we propose using pattern mining to identify compounds for improving accuracy and using dimensionality reduction for reducing complexity. Finally, empirical experiments demonstrate the performance superiority of our approaches over the state-of-the-art approaches. △ Less

Submitted 31 August, 2021; originally announced September 2021.

Comments: 16 pages, 5 figures, Dasfaa

Improving BERT Model Using Contrastive Learning for Biomedical Relation Extraction

Authors: Peng Su, Yifan Peng, K. Vijay-Shanker

Abstract: Contrastive learning has been used to learn a high-quality representation of the image in computer vision. However, contrastive learning is not widely utilized in natural language processing due to the lack of a general method of data augmentation for text data. In this work, we explore the method of employing contrastive learning to improve the text representation from the BERT model for relation… ▽ More Contrastive learning has been used to learn a high-quality representation of the image in computer vision. However, contrastive learning is not widely utilized in natural language processing due to the lack of a general method of data augmentation for text data. In this work, we explore the method of employing contrastive learning to improve the text representation from the BERT model for relation extraction. The key knob of our framework is a unique contrastive pre-training step tailored for the relation extraction tasks by seamlessly integrating linguistic knowledge into the data augmentation. Furthermore, we investigate how large-scale data constructed from the external knowledge bases can enhance the generality of contrastive pre-training of BERT. The experimental results on three relation extraction benchmark datasets demonstrate that our method can improve the BERT model representation and achieve state-of-the-art performance. In addition, we explore the interpretability of models by showing that BERT with contrastive pre-training relies more on rationales for prediction. Our code and data are publicly available at: https://github.com/udel-biotm-lab/BERT-CLRE. △ Less

Submitted 28 April, 2021; originally announced April 2021.

Comments: Accepted by BioNLP 2021

Multilingual and Cross-Lingual Intent Detection from Spoken Data

Authors: Daniela Gerz, Pei-Hao Su, Razvan Kusztos, Avishek Mondal, Michał Lis, Eshan Singhal, Nikola Mrkšić, Tsung-Hsien Wen, Ivan Vulić

Abstract: We present a systematic study on multilingual and cross-lingual intent detection from spoken data. The study leverages a new resource put forth in this work, termed MInDS-14, a first training and evaluation resource for the intent detection task with spoken data. It covers 14 intents extracted from a commercial system in the e-banking domain, associated with spoken examples in 14 diverse language… ▽ More We present a systematic study on multilingual and cross-lingual intent detection from spoken data. The study leverages a new resource put forth in this work, termed MInDS-14, a first training and evaluation resource for the intent detection task with spoken data. It covers 14 intents extracted from a commercial system in the e-banking domain, associated with spoken examples in 14 diverse language varieties. Our key results indicate that combining machine translation models with state-of-the-art multilingual sentence encoders (e.g., LaBSE) can yield strong intent detectors in the majority of target languages covered in MInDS-14, and offer comparative analyses across different axes: e.g., zero-shot versus few-shot learning, translation direction, and impact of speech recognition. We see this work as an important step towards more inclusive development and evaluation of multilingual intent detectors from spoken data, in a much wider spectrum of languages compared to prior work. △ Less

Submitted 17 April, 2021; originally announced April 2021.

DJXPerf: Identifying Memory Inefficiencies via Object-centric Profiling for Java

Authors: Bolun Li, Pengfei Su, Milind Chabbi, Shuyin Jiao, Xu Liu

Abstract: Java is the "go-to" programming language choice for developing scalable enterprise cloud applications. In such systems, even a few percent CPU time savings can offer a significant competitive advantage and cost saving. Although performance tools abound in Java, those that focus on the data locality in the memory hierarchy are rare. In this paper, we present DJXPerf, a lightweight, object-centric… ▽ More Java is the "go-to" programming language choice for developing scalable enterprise cloud applications. In such systems, even a few percent CPU time savings can offer a significant competitive advantage and cost saving. Although performance tools abound in Java, those that focus on the data locality in the memory hierarchy are rare. In this paper, we present DJXPerf, a lightweight, object-centric memory profiler for Java, which associates memory-hierarchy performance metrics (e.g., cache/TLB misses) with Java objects. DJXPerf uses statistical sampling of hardware performance monitoring counters to attribute metrics to not only source code locations but also Java objects. DJXPerf presents Java object allocation contexts combined with their usage contexts and presents them ordered by the poor locality behaviors. DJXPerf's performance measurement, object attribution, and presentation techniques guide optimizing object allocation, layout, and access patterns. DJXPerf incurs only ~8% runtime overhead and ~5% memory overhead on average, requiring no modifications to hardware, OS, Java virtual machine, or application source code, which makes it attractive to use in production. Guided by DJXPerf, we study and optimize a number of Java and Scala programs, including well-known benchmarks and real-world applications, and demonstrate significant speedups. △ Less

Submitted 7 April, 2021; originally announced April 2021.

Comments: 13 pages (including 2-page reference), 5 figures, 2 tables

Gradient Regularized Contrastive Learning for Continual Domain Adaptation

Authors: Shixiang Tang, Peng Su, Dapeng Chen, Wanli Ouyang

Abstract: Human beings can quickly adapt to environmental changes by leveraging learning experience. However, adapting deep neural networks to dynamic environments by machine learning algorithms remains a challenge. To better understand this issue, we study the problem of continual domain adaptation, where the model is presented with a labelled source domain and a sequence of unlabelled target domains. The… ▽ More Human beings can quickly adapt to environmental changes by leveraging learning experience. However, adapting deep neural networks to dynamic environments by machine learning algorithms remains a challenge. To better understand this issue, we study the problem of continual domain adaptation, where the model is presented with a labelled source domain and a sequence of unlabelled target domains. The obstacles in this problem are both domain shift and catastrophic forgetting. We propose Gradient Regularized Contrastive Learning (GRCL) to solve the obstacles. At the core of our method, gradient regularization plays two key roles: (1) enforcing the gradient not to harm the discriminative ability of source features which can, in turn, benefit the adaptation ability of the model to target domains; (2) constraining the gradient not to increase the classification loss on old target domains, which enables the model to preserve the performance on old target domains when adapting to an in-coming target domain. Experiments on Digits, DomainNet and Office-Caltech benchmarks demonstrate the strong performance of our approach when compared to the other state-of-the-art methods. △ Less

Submitted 23 March, 2021; originally announced March 2021.

Comments: Accepted by AAAI2021 (poster). arXiv admin note: text overlap with arXiv:2007.12942

An O(n) time algorithm for finding Hamilton cycles with high probability

Authors: Rajko Nenadov, Angelika Steger, Pascal Su

Abstract: We design a randomized algorithm that finds a Hamilton cycle in $\mathcal{O}(n)$ time with high probability in a random graph $G_{n,p}$ with edge probability $p\ge C \log n / n$. This closes a gap left open in a seminal paper by Angluin and Valiant from 1979. We design a randomized algorithm that finds a Hamilton cycle in $\mathcal{O}(n)$ time with high probability in a random graph $G_{n,p}$ with edge probability $p\ge C \log n / n$. This closes a gap left open in a seminal paper by Angluin and Valiant from 1979. △ Less

Submitted 4 December, 2020; originally announced December 2020.

Mastermind with a Linear Number of Queries

Authors: Anders Martinsson, Pascal Su

Abstract: Since the 1960s Mastermind has been studied for the combinatorial and information theoretical interest the game has to offer. Many results have been discovered starting with Erdős and Rényi determining the optimal number of queries needed for two colors. For $k$ colors and $n$ positions, Chvátal found asymptotically optimal bounds when $k \le n^{1-ε}$. Following a sequence of gradual improvements… ▽ More Since the 1960s Mastermind has been studied for the combinatorial and information theoretical interest the game has to offer. Many results have been discovered starting with Erdős and Rényi determining the optimal number of queries needed for two colors. For $k$ colors and $n$ positions, Chvátal found asymptotically optimal bounds when $k \le n^{1-ε}$. Following a sequence of gradual improvements for $k \geq n$ colors, the central open question is to resolve the gap between $Ω(n)$ and $\mathcal{O}(n\log \log n)$ for $k=n$. In this paper, we resolve this gap by presenting the first algorithm for solving $k=n$ Mastermind with a linear number of queries. As a consequence, we are able to determine the query complexity of Mastermind for any parameters $k$ and $n$. △ Less

Submitted 19 September, 2023; v1 submitted 11 November, 2020; originally announced November 2020.

Investigation of BERT Model on Biomedical Relation Extraction Based on Revised Fine-tuning Mechanism

Authors: Peng Su, K. Vijay-Shanker

Abstract: With the explosive growth of biomedical literature, designing automatic tools to extract information from the literature has great significance in biomedical research. Recently, transformer-based BERT models adapted to the biomedical domain have produced leading results. However, all the existing BERT models for relation classification only utilize partial knowledge from the last layer. In this pa… ▽ More With the explosive growth of biomedical literature, designing automatic tools to extract information from the literature has great significance in biomedical research. Recently, transformer-based BERT models adapted to the biomedical domain have produced leading results. However, all the existing BERT models for relation classification only utilize partial knowledge from the last layer. In this paper, we will investigate the method of utilizing the entire layer in the fine-tuning process of BERT model. To the best of our knowledge, we are the first to explore this method. The experimental results illustrate that our method improves the BERT model performance and outperforms the state-of-the-art methods on three benchmark datasets for different relation extraction tasks. In addition, further analysis indicates that the key knowledge about the relations can be learned from the last layer of BERT model. △ Less

Submitted 31 October, 2020; originally announced November 2020.

CS2-Net: Deep Learning Segmentation of Curvilinear Structures in Medical Imaging

Authors: Lei Mou, Yitian Zhao, Huazhu Fu, Yonghuai Liu, Jun Cheng, Yalin Zheng, Pan Su, Jianlong Yang, Li Chen, Alejandro F Frang, Masahiro Akiba, Jiang Liu

Abstract: Automated detection of curvilinear structures, e.g., blood vessels or nerve fibres, from medical and biomedical images is a crucial early step in automatic image interpretation associated to the management of many diseases. Precise measurement of the morphological changes of these curvilinear organ structures informs clinicians for understanding the mechanism, diagnosis, and treatment of e.g. card… ▽ More Automated detection of curvilinear structures, e.g., blood vessels or nerve fibres, from medical and biomedical images is a crucial early step in automatic image interpretation associated to the management of many diseases. Precise measurement of the morphological changes of these curvilinear organ structures informs clinicians for understanding the mechanism, diagnosis, and treatment of e.g. cardiovascular, kidney, eye, lung, and neurological conditions. In this work, we propose a generic and unified convolution neural network for the segmentation of curvilinear structures and illustrate in several 2D/3D medical imaging modalities. We introduce a new curvilinear structure segmentation network (CS2-Net), which includes a self-attention mechanism in the encoder and decoder to learn rich hierarchical representations of curvilinear structures. Two types of attention modules - spatial attention and channel attention - are utilized to enhance the inter-class discrimination and intra-class responsiveness, to further integrate local features with their global dependencies and normalization, adaptively. Furthermore, to facilitate the segmentation of curvilinear structures in medical images, we employ a 1x3 and a 3x1 convolutional kernel to capture boundary features. ... △ Less

Submitted 19 October, 2020; v1 submitted 14 October, 2020; originally announced October 2020.

Variational Preparation of the Sachdev-Ye-Kitaev Thermofield Double

Authors: Vincent Paul Su

Abstract: We provide an algorithm for preparing the thermofield double (TFD) state of the Sachdev-Ye-Kitaev model without the need for an auxiliary bath. Following previous work, the TFD can be cast as the approximate ground state of a Hamiltonian, $H_{\text{TFD}}$. Using variational quantum circuits, we propose and implement a gradient-based algorithm for learning parameters that find this ground state, an… ▽ More We provide an algorithm for preparing the thermofield double (TFD) state of the Sachdev-Ye-Kitaev model without the need for an auxiliary bath. Following previous work, the TFD can be cast as the approximate ground state of a Hamiltonian, $H_{\text{TFD}}$. Using variational quantum circuits, we propose and implement a gradient-based algorithm for learning parameters that find this ground state, an application of the variational quantum eigensolver. Concretely, we find quantum circuits that prepare the ground state of $H_{\text{TFD}}$ for the $q=4$ SYK model up to $N=12$. △ Less

Submitted 10 December, 2020; v1 submitted 9 September, 2020; originally announced September 2020.

Comments: 20 pages, 8 figures; v2 references added

Journal ref: Phys. Rev. A 104, 012427 (2021)

Map-Adaptive Goal-Based Trajectory Prediction

Authors: Lingyao Zhang, Po-Hsun Su, Jerrick Hoang, Galen Clark Haynes, Micol Marchetti-Bowick

Abstract: We present a new method for multi-modal, long-term vehicle trajectory prediction. Our approach relies on using lane centerlines captured in rich maps of the environment to generate a set of proposed goal paths for each vehicle. Using these paths -- which are generated at run time and therefore dynamically adapt to the scene -- as spatial anchors, we predict a set of goal-based trajectories along w… ▽ More We present a new method for multi-modal, long-term vehicle trajectory prediction. Our approach relies on using lane centerlines captured in rich maps of the environment to generate a set of proposed goal paths for each vehicle. Using these paths -- which are generated at run time and therefore dynamically adapt to the scene -- as spatial anchors, we predict a set of goal-based trajectories along with a categorical distribution over the goals. This approach allows us to directly model the goal-directed behavior of traffic actors, which unlocks the potential for more accurate long-term prediction. Our experimental results on both a large-scale internal driving dataset and on the public nuScenes dataset show that our model outperforms state-of-the-art approaches for vehicle trajectory prediction over a 6-second horizon. We also empirically demonstrate that our model is better able to generalize to road scenes from a completely new city than existing methods. △ Less

Submitted 13 November, 2020; v1 submitted 9 September, 2020; originally announced September 2020.

Comments: Published at CoRL 2020

Journal ref: Conference on Robot Learning (CoRL) 2020

Contrastive Visual-Linguistic Pretraining

Authors: Lei Shi, Kai Shuang, Shijie Geng, Peng Su, Zhengkai Jiang, Peng Gao, Zuohui Fu, Gerard de Melo, Sen Su

Abstract: Several multi-modality representation learning approaches such as LXMERT and ViLBERT have been proposed recently. Such approaches can achieve superior performance due to the high-level semantic information captured during large-scale multimodal pretraining. However, as ViLBERT and LXMERT adopt visual region regression and classification loss, they often suffer from domain gap and noisy label probl… ▽ More Several multi-modality representation learning approaches such as LXMERT and ViLBERT have been proposed recently. Such approaches can achieve superior performance due to the high-level semantic information captured during large-scale multimodal pretraining. However, as ViLBERT and LXMERT adopt visual region regression and classification loss, they often suffer from domain gap and noisy label problems, based on the visual features having been pretrained on the Visual Genome dataset. To overcome these issues, we propose unbiased Contrastive Visual-Linguistic Pretraining (CVLP), which constructs a visual self-supervised loss built upon contrastive learning. We evaluate CVLP on several down-stream tasks, including VQA, GQA and NLVR2 to validate the superiority of contrastive learning on multi-modality representation learning. Our code is available at: https://github.com/ArcherYunDong/CVLP-. △ Less

Submitted 26 July, 2020; originally announced July 2020.

Gradient Regularized Contrastive Learning for Continual Domain Adaptation

Authors: Peng Su, Shixiang Tang, Peng Gao, Di Qiu, Ni Zhao, Xiaogang Wang

Abstract: Human beings can quickly adapt to environmental changes by leveraging learning experience. However, the poor ability of adapting to dynamic environments remains a major challenge for AI models. To better understand this issue, we study the problem of continual domain adaptation, where the model is presented with a labeled source domain and a sequence of unlabeled target domains. There are two majo… ▽ More Human beings can quickly adapt to environmental changes by leveraging learning experience. However, the poor ability of adapting to dynamic environments remains a major challenge for AI models. To better understand this issue, we study the problem of continual domain adaptation, where the model is presented with a labeled source domain and a sequence of unlabeled target domains. There are two major obstacles in this problem: domain shifts and catastrophic forgetting. In this work, we propose Gradient Regularized Contrastive Learning to solve the above obstacles. At the core of our method, gradient regularization plays two key roles: (1) enforces the gradient of contrastive loss not to increase the supervised training loss on the source domain, which maintains the discriminative power of learned features; (2) regularizes the gradient update on the new domain not to increase the classification loss on the old target domains, which enables the model to adapt to an in-coming target domain while preserving the performance of previously observed domains. Hence our method can jointly learn both semantically discriminative and domain-invariant features with labeled source domain and unlabeled target domains. The experiments on Digits, DomainNet and Office-Caltech benchmarks demonstrate the strong performance of our approach when compared to the state-of-the-art. △ Less

Submitted 25 July, 2020; originally announced July 2020.

Adversarial Learning for Supervised and Semi-supervised Relation Extraction in Biomedical Literature

Authors: Peng Su, K. Vijay-Shanker

Abstract: Adversarial training is a technique of improving model performance by involving adversarial examples in the training process. In this paper, we investigate adversarial training with multiple adversarial examples to benefit the relation extraction task. We also apply adversarial training technique in semi-supervised scenarios to utilize unlabeled data. The evaluation results on protein-protein inte… ▽ More Adversarial training is a technique of improving model performance by involving adversarial examples in the training process. In this paper, we investigate adversarial training with multiple adversarial examples to benefit the relation extraction task. We also apply adversarial training technique in semi-supervised scenarios to utilize unlabeled data. The evaluation results on protein-protein interaction and protein subcellular localization task illustrate adversarial training provides improvement on the supervised model, and is also effective on involving unlabeled data in the semi-supervised training case. In addition, our method achieves state-of-the-art performance on two benchmarking datasets. △ Less

Submitted 25 September, 2020; v1 submitted 8 May, 2020; originally announced May 2020.

Adapting Object Detectors with Conditional Domain Normalization

Authors: Peng Su, Kun Wang, Xingyu Zeng, Shixiang Tang, Dapeng Chen, Di Qiu, Xiaogang Wang

Abstract: Real-world object detectors are often challenged by the domain gaps between different datasets. In this work, we present the Conditional Domain Normalization (CDN) to bridge the domain gap. CDN is designed to encode different domain inputs into a shared latent space, where the features from different domains carry the same domain attribute. To achieve this, we first disentangle the domain-specific… ▽ More Real-world object detectors are often challenged by the domain gaps between different datasets. In this work, we present the Conditional Domain Normalization (CDN) to bridge the domain gap. CDN is designed to encode different domain inputs into a shared latent space, where the features from different domains carry the same domain attribute. To achieve this, we first disentangle the domain-specific attribute out of the semantic features from one domain via a domain embedding module, which learns a domain-vector to characterize the corresponding domain attribute information. Then this domain-vector is used to encode the features from another domain through a conditional normalization, resulting in different domains' features carrying the same domain attribute. We incorporate CDN into various convolution stages of an object detector to adaptively address the domain shifts of different level's representation. In contrast to existing adaptation works that conduct domain confusion learning on semantic features to remove domain-specific factors, CDN aligns different domain distributions by modulating the semantic features of one domain conditioned on the learned domain-vector of another domain. Extensive experiments show that CDN outperforms existing methods remarkably on both real-to-real and synthetic-to-real adaptation benchmarks, including 2D image detection and 3D point cloud detection. △ Less

Submitted 22 July, 2020; v1 submitted 16 March, 2020; originally announced March 2020.

Comments: Accepted at ECCV 2020

The Quantum Entropy Cone of Hypergraphs

Authors: Ning Bao, Newton Cheng, Sergio Hernández-Cuenca, Vincent P. Su

Abstract: In this work, we generalize the graph-theoretic techniques used for the holographic entropy cone to study hypergraphs and their analogously-defined entropy cone. This allows us to develop a framework to efficiently compute entropies and prove inequalities satisfied by hypergraphs. In doing so, we discover a class of quantum entropy vectors which reach beyond those of holographic states and obey co… ▽ More In this work, we generalize the graph-theoretic techniques used for the holographic entropy cone to study hypergraphs and their analogously-defined entropy cone. This allows us to develop a framework to efficiently compute entropies and prove inequalities satisfied by hypergraphs. In doing so, we discover a class of quantum entropy vectors which reach beyond those of holographic states and obey constraints intimately related to the ones obeyed by stabilizer states and linear ranks. We show that, at least up to 4 parties, the hypergraph cone is identical to the stabilizer entropy cone, thus demonstrating that the hypergraph framework is broadly applicable to the study of entanglement entropy. We conjecture that this equality continues to hold for higher party numbers and report on partial progress on this direction. To physically motivate this conjectured equivalence, we also propose a plausible method inspired by tensor networks to construct a quantum state from a given hypergraph such that their entropy vectors match. △ Less

Submitted 12 February, 2020; originally announced February 2020.

Comments: 40+6 pages, 7 figures

Journal ref: SciPost Phys. 9 (2020) 5, 067

ConveRT: Efficient and Accurate Conversational Representations from Transformers

Authors: Matthew Henderson, Iñigo Casanueva, Nikola Mrkšić, Pei-Hao Su, Tsung-Hsien Wen, Ivan Vulić

Abstract: General-purpose pretrained sentence encoders such as BERT are not ideal for real-world conversational AI applications; they are computationally heavy, slow, and expensive to train. We propose ConveRT (Conversational Representations from Transformers), a pretraining framework for conversational tasks satisfying all the following requirements: it is effective, affordable, and quick to train. We pret… ▽ More General-purpose pretrained sentence encoders such as BERT are not ideal for real-world conversational AI applications; they are computationally heavy, slow, and expensive to train. We propose ConveRT (Conversational Representations from Transformers), a pretraining framework for conversational tasks satisfying all the following requirements: it is effective, affordable, and quick to train. We pretrain using a retrieval-based response selection task, effectively leveraging quantization and subword-level parameterization in the dual encoder to build a lightweight memory- and energy-efficient model. We show that ConveRT achieves state-of-the-art performance across widely established response selection tasks. We also demonstrate that the use of extended dialog history as context yields further performance gains. Finally, we show that pretrained representations from the proposed encoder can be transferred to the intent classification task, yielding strong results across three diverse data sets. ConveRT trains substantially faster than standard sentence encoders or previous state-of-the-art dual encoders. With its reduced size and superior performance, we believe this model promises wider portability and scalability for Conversational AI applications. △ Less

Submitted 29 April, 2020; v1 submitted 9 November, 2019; originally announced November 2019.

PolyResponse: A Rank-based Approach to Task-Oriented Dialogue with Application in Restaurant Search and Booking

Authors: Matthew Henderson, Ivan Vulić, Iñigo Casanueva, Paweł Budzianowski, Daniela Gerz, Sam Coope, Georgios Spithourakis, Tsung-Hsien Wen, Nikola Mrkšić, Pei-Hao Su

Abstract: We present PolyResponse, a conversational search engine that supports task-oriented dialogue. It is a retrieval-based approach that bypasses the complex multi-component design of traditional task-oriented dialogue systems and the use of explicit semantics in the form of task-specific ontologies. The PolyResponse engine is trained on hundreds of millions of examples extracted from real conversation… ▽ More We present PolyResponse, a conversational search engine that supports task-oriented dialogue. It is a retrieval-based approach that bypasses the complex multi-component design of traditional task-oriented dialogue systems and the use of explicit semantics in the form of task-specific ontologies. The PolyResponse engine is trained on hundreds of millions of examples extracted from real conversations: it learns what responses are appropriate in different conversational contexts. It then ranks a large index of text and visual responses according to their similarity to the given context, and narrows down the list of relevant entities during the multi-turn conversation. We introduce a restaurant search and booking system powered by the PolyResponse engine, currently available in 8 different languages. △ Less

Submitted 3 September, 2019; originally announced September 2019.

Comments: EMNLP 2019 (Demo paper)

Pinpointing Performance Inefficiencies in Java

Authors: Pengfei Su, Qingsen Wang, Milind Chabbi, Xu Liu

Abstract: Many performance inefficiencies such as inappropriate choice of algorithms or data structures, developers' inattention to performance, and missed compiler optimizations show up as wasteful memory operations. Wasteful memory operations are those that produce/consume data to/from memory that may have been avoided. We present, JXPerf, a lightweight performance analysis tool for pinpointing wasteful m… ▽ More Many performance inefficiencies such as inappropriate choice of algorithms or data structures, developers' inattention to performance, and missed compiler optimizations show up as wasteful memory operations. Wasteful memory operations are those that produce/consume data to/from memory that may have been avoided. We present, JXPerf, a lightweight performance analysis tool for pinpointing wasteful memory operations in Java programs. Traditional byte-code instrumentation for such analysis (1) introduces prohibitive overheads and (2) misses inefficiencies in machine code generation. JXPerf overcomes both of these problems. JXPerf uses hardware performance monitoring units to sample memory locations accessed by a program and uses hardware debug registers to monitor subsequent accesses to the same memory. The result is a lightweight measurement at machine-code level with attribution of inefficiencies to their provenance: machine and source code within full calling contexts. JXPerf introduces only 7% runtime overhead and 7% memory overhead making it useful in production. Guided by JXPerf, we optimize several Java applications by improving code generation and choosing superior data structures and algorithms, which yield significant speedups. △ Less

Submitted 28 June, 2019; originally announced June 2019.

Comments: This is a full-version of our ESEC/FSE'2019 paper

Training Neural Response Selection for Task-Oriented Dialogue Systems

Authors: Matthew Henderson, Ivan Vulić, Daniela Gerz, Iñigo Casanueva, Paweł Budzianowski, Sam Coope, Georgios Spithourakis, Tsung-Hsien Wen, Nikola Mrkšić, Pei-Hao Su

Abstract: Despite their popularity in the chatbot literature, retrieval-based models have had modest impact on task-oriented dialogue systems, with the main obstacle to their application being the low-data regime of most task-oriented dialogue tasks. Inspired by the recent success of pretraining in language modelling, we propose an effective method for deploying response selection in task-oriented dialogue.… ▽ More Despite their popularity in the chatbot literature, retrieval-based models have had modest impact on task-oriented dialogue systems, with the main obstacle to their application being the low-data regime of most task-oriented dialogue tasks. Inspired by the recent success of pretraining in language modelling, we propose an effective method for deploying response selection in task-oriented dialogue. To train response selection models for task-oriented dialogue tasks, we propose a novel method which: 1) pretrains the response selection model on large general-domain conversational corpora; and then 2) fine-tunes the pretrained model for the target dialogue domain, relying only on the small in-domain dataset to capture the nuances of the given dialogue domain. Our evaluation on six diverse application domains, ranging from e-commerce to banking, demonstrates the effectiveness of the proposed training method. △ Less

Submitted 7 June, 2019; v1 submitted 4 June, 2019; originally announced June 2019.

Comments: ACL 2019 long paper

Defocused images removal of axial overlapping scattering particles by using three-dimensional nonlinear diffusion based on digital holography

Authors: Wei-Na Li, Zhengyun Zhang, Jianshe Ma, Xiaohao Wang, Ping Su

Abstract: We propose a three-dimensional nonlinear diffusion method to implement the similar autofocusing function of multiple micro-objects and simultaneously remove the defocused images, which can distinguish the locations of certain sized scattering particles that are overlapping along z-axis. It is applied to all of the reconstruction slices that are generated from the captured hologram after each back… ▽ More We propose a three-dimensional nonlinear diffusion method to implement the similar autofocusing function of multiple micro-objects and simultaneously remove the defocused images, which can distinguish the locations of certain sized scattering particles that are overlapping along z-axis. It is applied to all of the reconstruction slices that are generated from the captured hologram after each back propagation. For certain small sized particles, the maxima of maximum gradient magnitude of each reconstruction slice appears at the ground truth z position after applying the proposed scheme when the reconstruction range along z-axis is sufficiently long and the reconstruction depth spacing is sufficiently fine. Therefore, the reconstructed image at ground truth z position is remained, while the defocused images are diffused out. The results demonstrated that the proposed scheme can diffuse out the defocused images which are 20 um away from the ground truth z position in spite of that several scattering particles with different diameters are completely overlapping along z-axis with a distance of 800 um when the hologram pixel pitch is 2 um. It also demonstrated that the sparsity distribution of the ground truth z slice cannot be affected by the sparsity distribution of corresponding defocused images when the diameter of the particle is not more than 35um and the reconstruction depth spacing is not less than 20 um. △ Less

Submitted 14 August, 2019; v1 submitted 23 April, 2019; originally announced April 2019.

Comments: no

A Repository of Conversational Datasets

Authors: Matthew Henderson, Paweł Budzianowski, Iñigo Casanueva, Sam Coope, Daniela Gerz, Girish Kumar, Nikola Mrkšić, Georgios Spithourakis, Pei-Hao Su, Ivan Vulić, Tsung-Hsien Wen

Abstract: Progress in Machine Learning is often driven by the availability of large datasets, and consistent evaluation metrics for comparing modeling approaches. To this end, we present a repository of conversational datasets consisting of hundreds of millions of examples, and a standardised evaluation procedure for conversational response selection models using '1-of-100 accuracy'. The repository contains… ▽ More Progress in Machine Learning is often driven by the availability of large datasets, and consistent evaluation metrics for comparing modeling approaches. To this end, we present a repository of conversational datasets consisting of hundreds of millions of examples, and a standardised evaluation procedure for conversational response selection models using '1-of-100 accuracy'. The repository contains scripts that allow researchers to reproduce the standard datasets, or to adapt the pre-processing and data filtering steps to their needs. We introduce and evaluate several competitive baselines for conversational response selection, whose implementations are shared in the repository, as well as a neural encoder model that is trained on the entire training set. △ Less

Submitted 28 May, 2019; v1 submitted 12 April, 2019; originally announced April 2019.

Journal ref: Proceedings of the Workshop on NLP for Conversational AI (2019)

Redundant Loads: A Software Inefficiency Indicator

Authors: Pengfei Su, Shasha Wen, Hailong Yang, Milind Chabbi, Xu Liu

Abstract: Modern software packages have become increasingly complex with millions of lines of code and references to many external libraries. Redundant operations are a common performance limiter in these code bases. Missed compiler optimization opportunities, inappropriate data structure and algorithm choices, and developers' inattention to performance are some common reasons for the existence of redundant… ▽ More Modern software packages have become increasingly complex with millions of lines of code and references to many external libraries. Redundant operations are a common performance limiter in these code bases. Missed compiler optimization opportunities, inappropriate data structure and algorithm choices, and developers' inattention to performance are some common reasons for the existence of redundant operations. Developers mainly depend on compilers to eliminate redundant operations. However, compilers' static analysis often misses optimization opportunities due to ambiguities and limited analysis scope; automatic optimizations to algorithmic and data structural problems are out of scope. We develop LoadSpy, a whole-program profiler to pinpoint redundant memory load operations, which are often a symptom of many redundant operations. The strength of LoadSpy exists in identifying and quantifying redundant load operations in programs and associating the redundancies with program execution contexts and scopes to focus developers' attention on problematic code. LoadSpy works on fully optimized binaries, adopts various optimization techniques to reduce its overhead, and provides a rich graphic user interface, which make it a complete developer tool. Applying LoadSpy showed that a large fraction of redundant loads is common in modern software packages despite highest levels of automatic compiler optimizations. Guided by LoadSpy, we optimize several well-known benchmarks and real-world applications, yielding significant speedups. △ Less

Submitted 14 February, 2019; originally announced February 2019.

Comments: This paper is a full-version of our ICSE paper

Feudal Reinforcement Learning for Dialogue Management in Large Domains

Authors: Iñigo Casanueva, Paweł Budzianowski, Pei-Hao Su, Stefan Ultes, Lina Rojas-Barahona, Bo-Hsiang Tseng, Milica Gašić

Abstract: Reinforcement learning (RL) is a promising approach to solve dialogue policy optimisation. Traditional RL algorithms, however, fail to scale to large domains due to the curse of dimensionality. We propose a novel Dialogue Management architecture, based on Feudal RL, which decomposes the decision into two steps; a first step where a master policy selects a subset of primitive actions, and a second… ▽ More Reinforcement learning (RL) is a promising approach to solve dialogue policy optimisation. Traditional RL algorithms, however, fail to scale to large domains due to the curse of dimensionality. We propose a novel Dialogue Management architecture, based on Feudal RL, which decomposes the decision into two steps; a first step where a master policy selects a subset of primitive actions, and a second step where a primitive action is chosen from the selected subset. The structural information included in the domain ontology is used to abstract the dialogue state space, taking the decisions at each step using different parts of the abstracted state. This, combined with an information sharing mechanism between slots, increases the scalability to large domains. We show that an implementation of this approach, based on Deep-Q Networks, significantly outperforms previous state of the art in several dialogue domains and environments, without the need of any additional reward signal. △ Less

Submitted 8 March, 2018; originally announced March 2018.

Comments: Accepted as a short paper in NAACL 2018

Sample Efficient Deep Reinforcement Learning for Dialogue Systems with Large Action Spaces

Authors: Gellért Weisz, Paweł Budzianowski, Pei-Hao Su, Milica Gašić

Abstract: In spoken dialogue systems, we aim to deploy artificial intelligence to build automated dialogue agents that can converse with humans. A part of this effort is the policy optimisation task, which attempts to find a policy describing how to respond to humans, in the form of a function taking the current state of the dialogue and returning the response of the system. In this paper, we investigate de… ▽ More In spoken dialogue systems, we aim to deploy artificial intelligence to build automated dialogue agents that can converse with humans. A part of this effort is the policy optimisation task, which attempts to find a policy describing how to respond to humans, in the form of a function taking the current state of the dialogue and returning the response of the system. In this paper, we investigate deep reinforcement learning approaches to solve this problem. Particular attention is given to actor-critic methods, off-policy reinforcement learning with experience replay, and various methods aimed at reducing the bias and variance of estimators. When combined, these methods result in the previously proposed ACER algorithm that gave competitive results in gaming environments. These environments however are fully observable and have a relatively small action set so in this paper we examine the application of ACER to dialogue policy optimisation. We show that this method beats the current state-of-the-art in deep learning approaches for spoken dialogue systems. This not only leads to a more sample efficient algorithm that can train faster, but also allows us to apply the algorithm in more difficult environments than before. We thus experiment with learning in a very large action space, which has two orders of magnitude more actions than previously considered. We find that ACER trains significantly faster than the current state-of-the-art. △ Less

Submitted 11 February, 2018; originally announced February 2018.

A Benchmarking Environment for Reinforcement Learning Based Task Oriented Dialogue Management

Authors: Iñigo Casanueva, Paweł Budzianowski, Pei-Hao Su, Nikola Mrkšić, Tsung-Hsien Wen, Stefan Ultes, Lina Rojas-Barahona, Steve Young, Milica Gašić

Abstract: Dialogue assistants are rapidly becoming an indispensable daily aid. To avoid the significant effort needed to hand-craft the required dialogue flow, the Dialogue Management (DM) module can be cast as a continuous Markov Decision Process (MDP) and trained through Reinforcement Learning (RL). Several RL models have been investigated over recent years. However, the lack of a common benchmarking fram… ▽ More Dialogue assistants are rapidly becoming an indispensable daily aid. To avoid the significant effort needed to hand-craft the required dialogue flow, the Dialogue Management (DM) module can be cast as a continuous Markov Decision Process (MDP) and trained through Reinforcement Learning (RL). Several RL models have been investigated over recent years. However, the lack of a common benchmarking framework makes it difficult to perform a fair comparison between different models and their capability to generalise to different environments. Therefore, this paper proposes a set of challenging simulated environments for dialogue model development and evaluation. To provide some baselines, we investigate a number of representative parametric algorithms, namely deep reinforcement learning algorithms - DQN, A2C and Natural Actor-Critic and compare them to a non-parametric model, GP-SARSA. Both the environments and policy models are implemented using the publicly available PyDial toolkit and released on-line, in order to establish a testbed framework for further experiments and to facilitate experimental reproducibility. △ Less

Submitted 6 April, 2018; v1 submitted 29 November, 2017; originally announced November 2017.

Comments: Accepted at the Deep Reinforcement Learning Symposium, 31st Conference on Neural Information Processing Systems (NIPS 2017) Paper updated with minor changes

Nearest Common Ancestors: Universal Trees and Improved Labeling Schemes

Authors: Fabian Kuhn, Konstantinos Panagiotou, Pascal Su

Abstract: We investigate the nearest common ancestor (NCA) function in rooted trees. As the main conceptual contribution, the paper introduces universal trees for the NCA function: For a given family of rooted trees, an NCA-universal tree $S$ is a rooted tree such that any tree $T$ of the family can be embedded into $S$ such that the embedding of the NCA in $T$ of two nodes of $T$ is equal to the NCA in… ▽ More We investigate the nearest common ancestor (NCA) function in rooted trees. As the main conceptual contribution, the paper introduces universal trees for the NCA function: For a given family of rooted trees, an NCA-universal tree $S$ is a rooted tree such that any tree $T$ of the family can be embedded into $S$ such that the embedding of the NCA in $T$ of two nodes of $T$ is equal to the NCA in $S$ of the embeddings of the two nodes. As the main technical result we give explicit constructions of NCA-universal trees of size $n^{2.318}$ for the family of rooted $n$-vertex trees and of size $n^{1.894}$ for the family of rooted binary $n$-vertex trees. A direct consequence is the explicit construction of NCA-labeling schemes with labels of size $2.318\log_2 n$ and $1.894\log_2 n$ for the two families of rooted trees. This improves on the best known such labeling schemes established by Alstrup, Halvorsen and Larsen [SODA 2014]. △ Less

Submitted 27 July, 2017; originally announced July 2017.

Reward-Balancing for Statistical Spoken Dialogue Systems using Multi-objective Reinforcement Learning

Authors: Stefan Ultes, Paweł Budzianowski, Iñigo Casanueva, Nikola Mrkšić, Lina Rojas-Barahona, Pei-Hao Su, Tsung-Hsien Wen, Milica Gašić, Steve Young

Abstract: Reinforcement learning is widely used for dialogue policy optimization where the reward function often consists of more than one component, e.g., the dialogue success and the dialogue length. In this work, we propose a structured method for finding a good balance between these components by searching for the optimal reward component weighting. To render this search feasible, we use multi-objective… ▽ More Reinforcement learning is widely used for dialogue policy optimization where the reward function often consists of more than one component, e.g., the dialogue success and the dialogue length. In this work, we propose a structured method for finding a good balance between these components by searching for the optimal reward component weighting. To render this search feasible, we use multi-objective reinforcement learning to significantly reduce the number of training dialogues required. We apply our proposed method to find optimized component weights for six domains and compare them to a default baseline. △ Less

Submitted 19 July, 2017; originally announced July 2017.

Comments: Accepted at SIGDial 2017

Sample-efficient Actor-Critic Reinforcement Learning with Supervised Data for Dialogue Management

Authors: Pei-Hao Su, Pawel Budzianowski, Stefan Ultes, Milica Gasic, Steve Young

Abstract: Deep reinforcement learning (RL) methods have significant potential for dialogue policy optimisation. However, they suffer from a poor performance in the early stages of learning. This is especially problematic for on-line learning with real users. Two approaches are introduced to tackle this problem. Firstly, to speed up the learning process, two sample-efficient neural networks algorithms: trust… ▽ More Deep reinforcement learning (RL) methods have significant potential for dialogue policy optimisation. However, they suffer from a poor performance in the early stages of learning. This is especially problematic for on-line learning with real users. Two approaches are introduced to tackle this problem. Firstly, to speed up the learning process, two sample-efficient neural networks algorithms: trust region actor-critic with experience replay (TRACER) and episodic natural actor-critic with experience replay (eNACER) are presented. For TRACER, the trust region helps to control the learning step size and avoid catastrophic model changes. For eNACER, the natural gradient identifies the steepest ascent direction in policy space to speed up the convergence. Both models employ off-policy learning with experience replay to improve sample-efficiency. Secondly, to mitigate the cold start issue, a corpus of demonstration data is utilised to pre-train the models prior to on-line reinforcement learning. Combining these two approaches, we demonstrate a practical approach to learn deep RL-based dialogue policies and demonstrate their effectiveness in a task-oriented information seeking domain. △ Less

Submitted 5 July, 2017; v1 submitted 1 July, 2017; originally announced July 2017.

Comments: Accepted as a long paper in SigDial 2017

Sub-domain Modelling for Dialogue Management with Hierarchical Reinforcement Learning

Authors: Paweł Budzianowski, Stefan Ultes, Pei-Hao Su, Nikola Mrkšić, Tsung-Hsien Wen, Iñigo Casanueva, Lina Rojas-Barahona, Milica Gašić

Abstract: Human conversation is inherently complex, often spanning many different topics/domains. This makes policy learning for dialogue systems very challenging. Standard flat reinforcement learning methods do not provide an efficient framework for modelling such dialogues. In this paper, we focus on the under-explored problem of multi-domain dialogue management. First, we propose a new method for hierarc… ▽ More Human conversation is inherently complex, often spanning many different topics/domains. This makes policy learning for dialogue systems very challenging. Standard flat reinforcement learning methods do not provide an efficient framework for modelling such dialogues. In this paper, we focus on the under-explored problem of multi-domain dialogue management. First, we propose a new method for hierarchical reinforcement learning using the option framework. Next, we show that the proposed architecture learns faster and arrives at a better policy than the existing flat ones do. Moreover, we show how pretrained policies can be adapted to more complex systems with an additional set of new actions. In doing that, we show that our approach has the potential to facilitate policy optimisation for more sophisticated multi-domain dialogue systems. △ Less

Submitted 17 July, 2017; v1 submitted 19 June, 2017; originally announced June 2017.

Comments: Update of the section 4 and the bibliography

Long-term Blood Pressure Prediction with Deep Recurrent Neural Networks

Authors: Peng Su, Xiao-Rong Ding, Yuan-Ting Zhang, Jing Liu, Fen Miao, Ni Zhao

Abstract: Existing methods for arterial blood pressure (BP) estimation directly map the input physiological signals to output BP values without explicitly modeling the underlying temporal dependencies in BP dynamics. As a result, these models suffer from accuracy decay over a long time and thus require frequent calibration. In this work, we address this issue by formulating BP estimation as a sequence predi… ▽ More Existing methods for arterial blood pressure (BP) estimation directly map the input physiological signals to output BP values without explicitly modeling the underlying temporal dependencies in BP dynamics. As a result, these models suffer from accuracy decay over a long time and thus require frequent calibration. In this work, we address this issue by formulating BP estimation as a sequence prediction problem in which both the input and target are temporal sequences. We propose a novel deep recurrent neural network (RNN) consisting of multilayered Long Short-Term Memory (LSTM) networks, which are incorporated with (1) a bidirectional structure to access larger-scale context information of input sequence, and (2) residual connections to allow gradients in deep RNN to propagate more effectively. The proposed deep RNN model was tested on a static BP dataset, and it achieved root mean square error (RMSE) of 3.90 and 2.66 mmHg for systolic BP (SBP) and diastolic BP (DBP) prediction respectively, surpassing the accuracy of traditional BP prediction models. On a multi-day BP dataset, the deep RNN achieved RMSE of 3.84, 5.25, 5.80 and 5.81 mmHg for the 1st day, 2nd day, 4th day and 6th month after the 1st day SBP prediction, and 1.80, 4.78, 5.0, 5.21 mmHg for corresponding DBP prediction, respectively, which outperforms all previous models with notable improvement. The experimental results suggest that modeling the temporal dependencies in BP dynamics significantly improves the long-term BP prediction accuracy. △ Less

Submitted 14 January, 2018; v1 submitted 12 May, 2017; originally announced May 2017.

Comments: To appear in IEEE BHI 2018

Exploiting Sentence and Context Representations in Deep Neural Models for Spoken Language Understanding

Authors: Lina M. Rojas Barahona, Milica Gasic, Nikola Mrkšić, Pei-Hao Su, Stefan Ultes, Tsung-Hsien Wen, Steve Young

Abstract: This paper presents a deep learning architecture for the semantic decoder component of a Statistical Spoken Dialogue System. In a slot-filling dialogue, the semantic decoder predicts the dialogue act and a set of slot-value pairs from a set of n-best hypotheses returned by the Automatic Speech Recognition. Most current models for spoken language understanding assume (i) word-aligned semantic annot… ▽ More This paper presents a deep learning architecture for the semantic decoder component of a Statistical Spoken Dialogue System. In a slot-filling dialogue, the semantic decoder predicts the dialogue act and a set of slot-value pairs from a set of n-best hypotheses returned by the Automatic Speech Recognition. Most current models for spoken language understanding assume (i) word-aligned semantic annotations as in sequence taggers and (ii) delexicalisation, or a mapping of input words to domain-specific concepts using heuristics that try to capture morphological variation but that do not scale to other domains nor to language variation (e.g., morphology, synonyms, paraphrasing ). In this work the semantic decoder is trained using unaligned semantic annotations and it uses distributed semantic representation learning to overcome the limitations of explicit delexicalisation. The proposed architecture uses a convolutional neural network for the sentence representation and a long-short term memory network for the context representation. Results are presented for the publicly available DSTC2 corpus and an In-car corpus which is similar to DSTC2 but has a significantly higher word error rate (WER). △ Less

Submitted 13 October, 2016; originally announced October 2016.