Showing 1–10 of 10 results for author: Halder, K

Characterizing and Measuring Linguistic Dataset Drift

Authors: Tyler A. Chang, Kishaloy Halder, Neha Anna John, Yogarshi Vyas, Yassine Benajiba, Miguel Ballesteros, Dan Roth

Abstract: NLP models often degrade in performance when real world data distributions differ markedly from training data. However, existing dataset drift metrics in NLP have generally not considered specific dimensions of linguistic drift that affect model performance, and they have not been validated in their ability to predict model performance at the individual example level, where such metrics are often… ▽ More NLP models often degrade in performance when real world data distributions differ markedly from training data. However, existing dataset drift metrics in NLP have generally not considered specific dimensions of linguistic drift that affect model performance, and they have not been validated in their ability to predict model performance at the individual example level, where such metrics are often used in practice. In this paper, we propose three dimensions of linguistic dataset drift: vocabulary, structural, and semantic drift. These dimensions correspond to content word frequency divergences, syntactic divergences, and meaning changes not captured by word frequencies (e.g. lexical semantic change). We propose interpretable metrics for all three drift dimensions, and we modify past performance prediction methods to predict model performance at both the example and dataset level for English sentiment classification and natural language inference. We find that our drift metrics are more effective than previous metrics at predicting out-of-domain model accuracies (mean 16.8% root mean square error decrease), particularly when compared to popular fine-tuned embedding distances (mean 47.7% error decrease). Fine-tuned embedding distances are much more effective at ranking individual examples by expected performance, but decomposing into vocabulary, structural, and semantic drift produces the best example rankings of all considered model-agnostic drift metrics (mean 6.7% ROC AUC increase). △ Less

Submitted 26 May, 2023; originally announced May 2023.

Comments: Accepted to ACL 2023

A Weak Supervision Approach for Few-Shot Aspect Based Sentiment

Authors: Robert Vacareanu, Siddharth Varia, Kishaloy Halder, Shuai Wang, Giovanni Paolini, Neha Anna John, Miguel Ballesteros, Smaranda Muresan

Abstract: We explore how weak supervision on abundant unlabeled data can be leveraged to improve few-shot performance in aspect-based sentiment analysis (ABSA) tasks. We propose a pipeline approach to construct a noisy ABSA dataset, and we use it to adapt a pre-trained sequence-to-sequence model to the ABSA tasks. We test the resulting model on three widely used ABSA datasets, before and after fine-tuning.… ▽ More We explore how weak supervision on abundant unlabeled data can be leveraged to improve few-shot performance in aspect-based sentiment analysis (ABSA) tasks. We propose a pipeline approach to construct a noisy ABSA dataset, and we use it to adapt a pre-trained sequence-to-sequence model to the ABSA tasks. We test the resulting model on three widely used ABSA datasets, before and after fine-tuning. Our proposed method preserves the full fine-tuning performance while showing significant improvements (15.84% absolute F1) in the few-shot learning scenario for the harder tasks. In zero-shot (i.e., without fine-tuning), our method outperforms the previous state of the art on the aspect extraction sentiment classification (AESC) task and is, additionally, capable of performing the harder aspect sentiment triplet extraction (ASTE) task. △ Less

Submitted 19 May, 2023; originally announced May 2023.

Task-Specific Embeddings for Ante-Hoc Explainable Text Classification

Authors: Kishaloy Halder, Josip Krapac, Alan Akbik, Anthony Brew, Matti Lyra

Abstract: Current state-of-the-art approaches to text classification typically leverage BERT-style Transformer models with a softmax classifier, jointly fine-tuned to predict class labels of a target task. In this paper, we instead propose an alternative training objective in which we learn task-specific embeddings of text: our proposed objective learns embeddings such that all texts that share the same tar… ▽ More Current state-of-the-art approaches to text classification typically leverage BERT-style Transformer models with a softmax classifier, jointly fine-tuned to predict class labels of a target task. In this paper, we instead propose an alternative training objective in which we learn task-specific embeddings of text: our proposed objective learns embeddings such that all texts that share the same target class label should be close together in the embedding space, while all others should be far apart. This allows us to replace the softmax classifier with a more interpretable k-nearest-neighbor classification approach. In a series of experiments, we show that this yields a number of interesting benefits: (1) The resulting order induced by distances in the embedding space can be used to directly explain classification decisions. (2) This facilitates qualitative inspection of the training data, helping us to better understand the problem space and identify labelling quality issues. (3) The learned distances to some degree generalize to unseen classes, allowing us to incrementally add new classes without retraining the model. We present extensive experiments which show that the benefits of ante-hoc explainability and incremental learning come at no cost in overall classification accuracy, thus pointing to practical applicability of our proposed approach. △ Less

Submitted 30 November, 2022; originally announced December 2022.

Enhancing Product Safety in E-Commerce with NLP

Authors: Kishaloy Halder, Josip Krapac, Dmitry Goryunov, Anthony Brew, Matti Lyra, Alsida Dizdari, William Gillett, Adrien Renahy, Sinan Tang

Abstract: Ensuring safety of the products offered to the customers is of paramount importance to any e- commerce platform. Despite stringent quality and safety checking of products listed on these platforms, occasionally customers might receive a product that can pose a safety issue arising out of its use. In this paper, we present an innovative mechanism of how a large scale multinational e-commerce platfo… ▽ More Ensuring safety of the products offered to the customers is of paramount importance to any e- commerce platform. Despite stringent quality and safety checking of products listed on these platforms, occasionally customers might receive a product that can pose a safety issue arising out of its use. In this paper, we present an innovative mechanism of how a large scale multinational e-commerce platform, Zalando, uses Natural Language Processing techniques to assist timely investigation of the potentially unsafe products mined directly from customer written claims in unstructured plain text. We systematically describe the types of safety issues that concern Zalando customers. We demonstrate how we map this core business problem into a supervised text classification problem with highly imbalanced, noisy, multilingual data in a AI-in-the-loop setup with a focus on Key Performance Indicator (KPI) driven evaluation. Finally, we present detailed ablation studies to show a comprehensive comparison between different classification techniques. We conclude the work with how this NLP model was deployed. △ Less

Submitted 25 October, 2022; originally announced October 2022.

Instruction Tuning for Few-Shot Aspect-Based Sentiment Analysis

Authors: Siddharth Varia, Shuai Wang, Kishaloy Halder, Robert Vacareanu, Miguel Ballesteros, Yassine Benajiba, Neha Anna John, Rishita Anubhai, Smaranda Muresan, Dan Roth

Abstract: Aspect-based Sentiment Analysis (ABSA) is a fine-grained sentiment analysis task which involves four elements from user-generated texts: aspect term, aspect category, opinion term, and sentiment polarity. Most computational approaches focus on some of the ABSA sub-tasks such as tuple (aspect term, sentiment polarity) or triplet (aspect term, opinion term, sentiment polarity) extraction using eithe… ▽ More Aspect-based Sentiment Analysis (ABSA) is a fine-grained sentiment analysis task which involves four elements from user-generated texts: aspect term, aspect category, opinion term, and sentiment polarity. Most computational approaches focus on some of the ABSA sub-tasks such as tuple (aspect term, sentiment polarity) or triplet (aspect term, opinion term, sentiment polarity) extraction using either pipeline or joint modeling approaches. Recently, generative approaches have been proposed to extract all four elements as (one or more) quadruplets from text as a single task. In this work, we take a step further and propose a unified framework for solving ABSA, and the associated sub-tasks to improve the performance in few-shot scenarios. To this end, we fine-tune a T5 model with instructional prompts in a multi-task learning fashion covering all the sub-tasks, as well as the entire quadruple prediction task. In experiments with multiple benchmark datasets, we show that the proposed multi-task prompting approach brings performance boost (by absolute 8.29 F1) in the few-shot learning setting. △ Less

Submitted 11 June, 2023; v1 submitted 12 October, 2022; originally announced October 2022.

Comments: Camera ready copy for WASSA at ACL 2023

Computational modeling of Human-nCoV protein-protein interaction network

Authors: Sovan Saha, Anup Kumar Halder, Soumyendu Sekhar Bandyopadhyay, Piyali Chatterjee, Mita Nasipuri, Subhadip Basu

Abstract: COVID-19 has created a global pandemic with high morbidity and mortality in 2020. Novel coronavirus (nCoV), also known as Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV2), is responsible for this deadly disease. International Committee on Taxonomy of Viruses (ICTV) has declared that nCoV is highly genetically similar to SARS-CoV epidemic in 2003 (89% similarity). Limited number of clini… ▽ More COVID-19 has created a global pandemic with high morbidity and mortality in 2020. Novel coronavirus (nCoV), also known as Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV2), is responsible for this deadly disease. International Committee on Taxonomy of Viruses (ICTV) has declared that nCoV is highly genetically similar to SARS-CoV epidemic in 2003 (89% similarity). Limited number of clinically validated Human-nCoV protein interaction data is available in the literature. With this hypothesis, the present work focuses on developing a computational model for nCoV-Human protein interaction network, using the experimentally validated SARS-CoV-Human protein interactions. Initially, level-1 and level-2 human spreader proteins are identified in SARS-CoV-Human interaction network, using Susceptible-Infected-Susceptible (SIS) model. These proteins are considered as potential human targets for nCoV bait proteins. A gene-ontology based fuzzy affinity function has been used to construct the nCoV-Human protein interaction network at 99.98% specificity threshold. This also identifies the level-1 human spreaders for COVID-19 in human protein-interaction network. Level-2 human spreaders are subsequently identified using the SIS model. The derived host-pathogen interaction network is finally validated using 7 potential FDA listed drugs for COVID-19 with significant overlap between the known drug target proteins and the identified spreader proteins. △ Less

Submitted 5 May, 2020; originally announced May 2020.

Comments: Total number of pages is 12. Total number of figures is 7. Total number of tables is 1

Modeling Information Need of Users in Search Sessions

Authors: Kishaloy Halder, Heng-Tze Cheng, Ellie Ka In Chio, Georgios Roumpos, Tao Wu, Ritesh Agarwal

Abstract: Users issue queries to Search Engines, and try to find the desired information in the results produced. They repeat this process if their information need is not met at the first place. It is crucial to identify the important words in a query that depict the actual information need of the user and will determine the course of a search session. To this end, we propose a sequence-to-sequence based n… ▽ More Users issue queries to Search Engines, and try to find the desired information in the results produced. They repeat this process if their information need is not met at the first place. It is crucial to identify the important words in a query that depict the actual information need of the user and will determine the course of a search session. To this end, we propose a sequence-to-sequence based neural architecture that leverages the set of past queries issued by users, and results that were explored by them. Firstly, we employ our model for predicting the words in the current query that are important and would be retained in the next query. Additionally, as a downstream application of our model, we evaluate it on the widely popular task of next query suggestion. We show that our intuitive strategy of capturing information need can yield superior performance at these tasks on two large real-world search log datasets. △ Less

Submitted 3 January, 2020; originally announced January 2020.

Performance Analysis of Robust Stable PID Controllers Using Dominant Pole Placement for SOPTD Process Models

Authors: Saptarshi Das, Kaushik Halder, Amitava Gupta

Abstract: This paper derives new formulations for designing dominant pole placement based proportional-integral-derivative (PID) controllers to handle second order processes with time delays (SOPTD). Previously, similar attempts have been made for pole placement in delay-free systems. The presence of the time delay term manifests itself as a higher order system with variable number of interlaced poles and z… ▽ More This paper derives new formulations for designing dominant pole placement based proportional-integral-derivative (PID) controllers to handle second order processes with time delays (SOPTD). Previously, similar attempts have been made for pole placement in delay-free systems. The presence of the time delay term manifests itself as a higher order system with variable number of interlaced poles and zeros upon Pade approximation, which makes it difficult to achieve precise pole placement control. We here report the analytical expressions to constrain the closed loop dominant and non-dominant poles at the desired locations in the complex s-plane, using a third order Pade approximation for the delay term. However, invariance of the closed loop performance with different time delay approximation has also been verified using increasing order of Pade, representing a closed to reality higher order delay dynamics. The choice of the nature of non-dominant poles e.g. all being complex, real or a combination of them modifies the characteristic equation and influences the achievable stability regions. The effect of different types of non-dominant poles and the corresponding stability regions are obtained for nine test-bench processes indicating different levels of open-loop damping and lag to delay ratio. Next, we investigate which expression yields a wider stability region in the design parameter space by using Monte Carlo simulations while uniformly sampling a chosen design parameter space. Various time and frequency domain control performance parameters are investigated next, as well as their deviations with uncertain process parameters, using thousands of Monte Carlo simulations, around the robust stable solution for each of the nine test-bench processes. △ Less

Submitted 28 January, 2018; originally announced January 2018.

Comments: 50 pages, 42 figures, Knowledge-Based Systems, 2018

Sentiment Analysis for Twitter : Going Beyond Tweet Text

Authors: Lahari Poddar, Kishaloy Halder, Xianyan Jia

Abstract: Analysing sentiment of tweets is important as it helps to determine the users' opinion. Knowing people's opinion is crucial for several purposes starting from gathering knowledge about customer base, e-governance, campaigning and many more. In this report, we aim to develop a system to detect the sentiment from tweets. We employ several linguistic features along with some other external sources of… ▽ More Analysing sentiment of tweets is important as it helps to determine the users' opinion. Knowing people's opinion is crucial for several purposes starting from gathering knowledge about customer base, e-governance, campaigning and many more. In this report, we aim to develop a system to detect the sentiment from tweets. We employ several linguistic features along with some other external sources of information to detect the sentiment of a tweet. We show that augmenting the 140 character-long tweet with information harvested from external urls shared in the tweet as well as Social Media features enhances the sentiment prediction accuracy significantly. △ Less

Submitted 28 November, 2016; originally announced November 2016.

Properties of Farey Sequence and their Applications to Digital Image Processing

Authors: Soham Das, Kishaloy Halder, Sanjoy Pratihar, Partha Bhowmick

Abstract: Farey sequence has been a topic of interest to the mathematicians since the very beginning of last century. With the emergence of various algorithms involving the digital plane in recent times, several interesting works related with the Farey sequence have come up. Our work is related with the problem of searching an arbitrary fraction in a Farey sequence and its relevance to image processing. Giv… ▽ More Farey sequence has been a topic of interest to the mathematicians since the very beginning of last century. With the emergence of various algorithms involving the digital plane in recent times, several interesting works related with the Farey sequence have come up. Our work is related with the problem of searching an arbitrary fraction in a Farey sequence and its relevance to image processing. Given an arbitrary fraction p/q (0 < p < q) and a Farey sequence Fn of order n, we propose a novel algorithm using the Regula Falsi method and the concept of Farey table to efficiently find the fraction of Fn closest to p/q. All computations are in the integer domain only, which is its added benefit. Some contemporary applications of image processing have also been shown where such concepts can be incorporated. Experimental results have been furnished to demonstrate its efficiency and elegance. △ Less

Submitted 25 September, 2015; originally announced September 2015.