-
Leveraging Uncertainty Estimates To Improve Classifier Performance
Authors:
Gundeep Arora,
Srujana Merugu,
Anoop Saladi,
Rajeev Rastogi
Abstract:
Binary classification involves predicting the label of an instance based on whether the model score for the positive class exceeds a threshold chosen based on the application requirements (e.g., maximizing recall for a precision bound). However, model scores are often not aligned with the true positivity rate. This is especially true when the training involves a differential sampling across classe…
▽ More
Binary classification involves predicting the label of an instance based on whether the model score for the positive class exceeds a threshold chosen based on the application requirements (e.g., maximizing recall for a precision bound). However, model scores are often not aligned with the true positivity rate. This is especially true when the training involves a differential sampling across classes or there is distributional drift between train and test settings. In this paper, we provide theoretical analysis and empirical evidence of the dependence of model score estimation bias on both uncertainty and score itself. Further, we formulate the decision boundary selection in terms of both model score and uncertainty, prove that it is NP-hard, and present algorithms based on dynamic programming and isotonic regression. Evaluation of the proposed algorithms on three real-world datasets yield 25%-40% gain in recall at high precision bounds over the traditional approach of using model score alone, highlighting the benefits of leveraging uncertainty.
△ Less
Submitted 20 November, 2023;
originally announced November 2023.
-
Fairness in Ranking under Disparate Uncertainty
Authors:
Richa Rastogi,
Thorsten Joachims
Abstract:
Ranking is a ubiquitous method for focusing the attention of human evaluators on a manageable subset of options. Its use as part of human decision-making processes ranges from surfacing potentially relevant products on an e-commerce site to prioritizing college applications for human review. While ranking can make human evaluation more effective by focusing attention on the most promising options,…
▽ More
Ranking is a ubiquitous method for focusing the attention of human evaluators on a manageable subset of options. Its use as part of human decision-making processes ranges from surfacing potentially relevant products on an e-commerce site to prioritizing college applications for human review. While ranking can make human evaluation more effective by focusing attention on the most promising options, we argue that it can introduce unfairness if the uncertainty of the underlying relevance model differs between groups of options. Unfortunately, such disparity in uncertainty appears widespread, often to the detriment of minority groups for which relevance estimates can have higher uncertainty due to a lack of data or appropriate features. To address this fairness issue, we propose Equal-Opportunity Ranking (EOR) as a new fairness criterion for ranking and show that it corresponds to a group-wise fair lottery among the relevant options even in the presence of disparate uncertainty. EOR optimizes for an even cost burden on all groups, unlike the conventional Probability Ranking Principle, and is fundamentally different from existing notions of fairness in rankings, such as demographic parity and proportional Rooney rule constraints that are motivated by proportional representation relative to group size. To make EOR ranking practical, we present an efficient algorithm for computing it in time $O(n \log(n))$ and prove its close approximation guarantee to the globally optimal solution. In a comprehensive empirical evaluation on synthetic data, a US Census dataset, and a real-world audit of Amazon search queries, we find that the algorithm reliably guarantees EOR fairness while providing effective rankings.
△ Less
Submitted 11 July, 2024; v1 submitted 4 September, 2023;
originally announced September 2023.
-
Enhancing Pattern Classification in Support Vector Machines through Matrix Formulation
Authors:
Sambhav Jain Reshma Rastogi
Abstract:
Support Vector Machines (SVM) have gathered significant acclaim as classifiers due to their successful implementation of Statistical Learning Theory. However, in the context of multiclass and multilabel settings, the reliance on vector-based formulations in existing SVM-based models poses limitations regarding flexibility and ease of incorporating additional terms to handle specific challenges. To…
▽ More
Support Vector Machines (SVM) have gathered significant acclaim as classifiers due to their successful implementation of Statistical Learning Theory. However, in the context of multiclass and multilabel settings, the reliance on vector-based formulations in existing SVM-based models poses limitations regarding flexibility and ease of incorporating additional terms to handle specific challenges. To overcome these limitations, our research paper focuses on introducing a matrix formulation for SVM that effectively addresses these constraints. By employing the Accelerated Gradient Descent method in the dual, we notably enhance the efficiency of solving the Matrix-SVM problem. Experimental evaluations on multilabel and multiclass datasets demonstrate that Matrix SVM achieves superior time efficacy while delivering similar results to Binary Relevance SVM.
Moreover, our matrix formulation unveils crucial insights and advantages that may not be readily apparent in traditional vector-based notations. We emphasize that numerous multilabel models can be viewed as extensions of SVM, with customised modifications to meet specific requirements. The matrix formulation presented in this paper establishes a solid foundation for developing more sophisticated models capable of effectively addressing the distinctive challenges encountered in multilabel learning.
△ Less
Submitted 18 July, 2023;
originally announced July 2023.
-
Semi-Parametric Inducing Point Networks and Neural Processes
Authors:
Richa Rastogi,
Yair Schiff,
Alon Hacohen,
Zhaozhi Li,
Ian Lee,
Yuntian Deng,
Mert R. Sabuncu,
Volodymyr Kuleshov
Abstract:
We introduce semi-parametric inducing point networks (SPIN), a general-purpose architecture that can query the training set at inference time in a compute-efficient manner. Semi-parametric architectures are typically more compact than parametric models, but their computational complexity is often quadratic. In contrast, SPIN attains linear complexity via a cross-attention mechanism between datapoi…
▽ More
We introduce semi-parametric inducing point networks (SPIN), a general-purpose architecture that can query the training set at inference time in a compute-efficient manner. Semi-parametric architectures are typically more compact than parametric models, but their computational complexity is often quadratic. In contrast, SPIN attains linear complexity via a cross-attention mechanism between datapoints inspired by inducing point methods. Querying large training sets can be particularly useful in meta-learning, as it unlocks additional training signal, but often exceeds the scaling limits of existing models. We use SPIN as the basis of the Inducing Point Neural Process, a probabilistic model which supports large contexts in meta-learning and achieves high accuracy where existing models fail. In our experiments, SPIN reduces memory requirements, improves accuracy across a range of meta-learning tasks, and improves state-of-the-art performance on an important practical problem, genotype imputation.
△ Less
Submitted 30 March, 2023; v1 submitted 23 May, 2022;
originally announced May 2022.
-
Efficient Learning of Pinball TWSVM using Privileged Information and its applications
Authors:
Reshma Rastogi,
Aman Pal
Abstract:
In any learning framework, an expert knowledge always plays a crucial role. But, in the field of machine learning, the knowledge offered by an expert is rarely used. Moreover, machine learning algorithms (SVM based) generally use hinge loss function which is sensitive towards the noise. Thus, in order to get the advantage from an expert knowledge and to reduce the sensitivity towards the noise, in…
▽ More
In any learning framework, an expert knowledge always plays a crucial role. But, in the field of machine learning, the knowledge offered by an expert is rarely used. Moreover, machine learning algorithms (SVM based) generally use hinge loss function which is sensitive towards the noise. Thus, in order to get the advantage from an expert knowledge and to reduce the sensitivity towards the noise, in this paper, we propose privileged information based Twin Pinball Support Vector Machine classifier (Pin-TWSVMPI) where expert's knowledge is in the form of privileged information. The proposed Pin-TWSVMPI incorporates privileged information by using correcting function so as to obtain two nonparallel decision hyperplanes. Further, in order to make computations more efficient and fast, we use Sequential Minimal Optimization (SMO) technique for obtaining the classifier and have also shown its application for Pedestrian detection and Handwritten digit recognition. Further, for UCI datasets, we first implement a procedure which extracts privileged information from the features of the dataset which are then further utilized by Pin-TWSVMPI that leads to enhancement in classification accuracy with comparatively lesser computational time.
△ Less
Submitted 14 July, 2021;
originally announced July 2021.
-
Improvement over Pinball Loss Support Vector Machine
Authors:
Pritam Anand,
Reshma Rastogi,
Suresh Chandra
Abstract:
Recently, there have been several papers that discuss the extension of the Pinball loss Support Vector Machine (Pin-SVM) model, originally proposed by Huang et al.,[1][2]. Pin-SVM classifier deals with the pinball loss function, which has been defined in terms of the parameter $τ$. The parameter $τ$ can take values in $[ -1,1]$. The existing Pin-SVM model requires to solve the same optimization pr…
▽ More
Recently, there have been several papers that discuss the extension of the Pinball loss Support Vector Machine (Pin-SVM) model, originally proposed by Huang et al.,[1][2]. Pin-SVM classifier deals with the pinball loss function, which has been defined in terms of the parameter $τ$. The parameter $τ$ can take values in $[ -1,1]$. The existing Pin-SVM model requires to solve the same optimization problem for all values of $τ$ in $[ -1,1]$. In this paper, we improve the existing Pin-SVM model for the binary classification task. At first, we note that there is major difficulty in Pin-SVM model (Huang et al. [1]) for $ -1 \leq τ< 0$. Specifically, we show that the Pin-SVM model requires the solution of different optimization problem for $ -1 \leq τ< 0$. We further propose a unified model termed as Unified Pin-SVM which results in a QPP valid for all $-1\leq τ\leq 1$ and hence more convenient to use. The proposed Unified Pin-SVM model can obtain a significant improvement in accuracy over the existing Pin-SVM model which has also been empirically justified by extensive numerical experiments with real-world datasets.
△ Less
Submitted 2 June, 2021;
originally announced June 2021.
-
Comprehensive Review On Twin Support Vector Machines
Authors:
M. Tanveer,
T. Rajani,
R. Rastogi,
Y. H. Shao,
M. A. Ganaie
Abstract:
Twin support vector machine (TWSVM) and twin support vector regression (TSVR) are newly emerging efficient machine learning techniques which offer promising solutions for classification and regression challenges respectively. TWSVM is based upon the idea to identify two nonparallel hyperplanes which classify the data points to their respective classes. It requires to solve two small sized quadrati…
▽ More
Twin support vector machine (TWSVM) and twin support vector regression (TSVR) are newly emerging efficient machine learning techniques which offer promising solutions for classification and regression challenges respectively. TWSVM is based upon the idea to identify two nonparallel hyperplanes which classify the data points to their respective classes. It requires to solve two small sized quadratic programming problems (QPPs) in lieu of solving single large size QPP in support vector machine (SVM) while TSVR is formulated on the lines of TWSVM and requires to solve two SVM kind problems. Although there has been good research progress on these techniques; there is limited literature on the comparison of different variants of TSVR. Thus, this review presents a rigorous analysis of recent research in TWSVM and TSVR simultaneously mentioning their limitations and advantages. To begin with we first introduce the basic theory of support vector machine, TWSVM and then focus on the various improvements and applications of TWSVM, and then we introduce TSVR and its various enhancements. Finally, we suggest future research and development prospects.
△ Less
Submitted 18 March, 2022; v1 submitted 1 May, 2021;
originally announced May 2021.
-
CRISP: A Probabilistic Model for Individual-Level COVID-19 Infection Risk Estimation Based on Contact Data
Authors:
Ralf Herbrich,
Rajeev Rastogi,
Roland Vollgraf
Abstract:
We present CRISP (COVID-19 Risk Score Prediction), a probabilistic graphical model for COVID-19 infection spread through a population based on the SEIR model where we assume access to (1) mutual contacts between pairs of individuals across time across various channels (e.g., Bluetooth contact traces), as well as (2) test outcomes at given times for infection, exposure and immunity tests. Our micro…
▽ More
We present CRISP (COVID-19 Risk Score Prediction), a probabilistic graphical model for COVID-19 infection spread through a population based on the SEIR model where we assume access to (1) mutual contacts between pairs of individuals across time across various channels (e.g., Bluetooth contact traces), as well as (2) test outcomes at given times for infection, exposure and immunity tests. Our micro-level model keeps track of the infection state for each individual at every point in time, ranging from susceptible, exposed, infectious to recovered. We develop both a Monte Carlo EM as well as a message passing algorithm to infer contact-channel specific infection transmission probabilities. Our Monte Carlo algorithm uses Gibbs sampling to draw samples of the latent infection status of each individual over the entire time period of analysis, given the latent infection status of all contacts and test outcome data. Experimental results with simulated data demonstrate our CRISP model can be parametrized by the reproduction factor $R_0$ and exhibits population-level infectiousness and recovery time series similar to those of the classical SEIR model. However, due to the individual contact data, this model allows fine grained control and inference for a wide range of COVID-19 mitigation and suppression policy measures. Moreover, the block-Gibbs sampling algorithm is able to support efficient testing in a test-trace-isolate approach to contain COVID-19 infection spread. To the best of our knowledge, this is the first model with efficient inference for COVID-19 infection spread based on individual-level contact data; most epidemic models are macro-level models that reason over entire populations. The implementation of CRISP is available in Python and C++ at https://github.com/zalandoresearch/CRISP.
△ Less
Submitted 30 June, 2022; v1 submitted 9 June, 2020;
originally announced June 2020.
-
A $ν$- support vector quantile regression model with automatic accuracy control
Authors:
Pritam Anand,
Reshma Rastogi,
Suresh Chandra
Abstract:
This paper proposes a novel '$ν$-support vector quantile regression' ($ν$-SVQR) model for the quantile estimation. It can facilitate the automatic control over accuracy by creating a suitable asymmetric $ε$-insensitive zone according to the variance present in data. The proposed $ν$-SVQR model uses the $ν$ fraction of training data points for the estimation of the quantiles. In the $ν$-SVQR model,…
▽ More
This paper proposes a novel '$ν$-support vector quantile regression' ($ν$-SVQR) model for the quantile estimation. It can facilitate the automatic control over accuracy by creating a suitable asymmetric $ε$-insensitive zone according to the variance present in data. The proposed $ν$-SVQR model uses the $ν$ fraction of training data points for the estimation of the quantiles. In the $ν$-SVQR model, training points asymptotically appear above and below of the asymmetric $ε$-insensitive tube in the ratio of $1-τ$ and $τ$. Further, there are other interesting properties of the proposed $ν$-SVQR model, which we have briefly described in this paper. These properties have been empirically verified using the artificial and real world dataset also.
△ Less
Submitted 21 October, 2019;
originally announced October 2019.
-
A new asymmetric $ε$-insensitive pinball loss function based support vector quantile regression model
Authors:
Pritam Anand,
Reshma Rastogi,
Suresh Chandra
Abstract:
In this paper, we propose a novel asymmetric $ε$-insensitive pinball loss function for quantile estimation. There exists some pinball loss functions which attempt to incorporate the $ε$-insensitive zone approach in it but, they fail to extend the $ε$-insensitive approach for quantile estimation in true sense. The proposed asymmetric $ε$-insensitive pinball loss function can make an asymmetric $ε$-…
▽ More
In this paper, we propose a novel asymmetric $ε$-insensitive pinball loss function for quantile estimation. There exists some pinball loss functions which attempt to incorporate the $ε$-insensitive zone approach in it but, they fail to extend the $ε$-insensitive approach for quantile estimation in true sense. The proposed asymmetric $ε$-insensitive pinball loss function can make an asymmetric $ε$- insensitive zone of fixed width around the data and divide it using $τ$ value for the estimation of the $τ$th quantile. The use of the proposed asymmetric $ε$-insensitive pinball loss function in Support Vector Quantile Regression (SVQR) model improves its prediction ability significantly. It also brings the sparsity back in SVQR model. Further, the numerical results obtained by several experiments carried on artificial and real world datasets empirically show the efficacy of the proposed `$ε$-Support Vector Quantile Regression' ($ε$-SVQR) model over other existing SVQR models.
△ Less
Submitted 19 August, 2019;
originally announced August 2019.
-
Support Vector Regression via a Combined Reward Cum Penalty Loss Function
Authors:
Pritam Anand,
Reshma Rastogi,
Suresh Chandra
Abstract:
In this paper, we introduce a novel combined reward cum penalty loss function to handle the regression problem. The proposed combined reward cum penalty loss function penalizes the data points which lie outside the $ε$-tube of the regressor and also assigns reward for the data points which lie inside of the $ε$-tube of the regressor. The combined reward cum penalty loss function based regression (…
▽ More
In this paper, we introduce a novel combined reward cum penalty loss function to handle the regression problem. The proposed combined reward cum penalty loss function penalizes the data points which lie outside the $ε$-tube of the regressor and also assigns reward for the data points which lie inside of the $ε$-tube of the regressor. The combined reward cum penalty loss function based regression (RP-$ε$-SVR) model has several interesting properties which are investigated in this paper and are also supported with the experimental results.
△ Less
Submitted 3 May, 2020; v1 submitted 28 April, 2019;
originally announced April 2019.
-
On Stability Problems of Omega and 3-Disjoint Paths Omega Multi-stage Interconnection Networks
Authors:
Ravi Rastogi,
Nitin,
Durg Singh Chauhan,
Mahesh Chandra Govil
Abstract:
The research paper emphasizes that the Stable Matching problems are the same as the problems of stable configurations of Multi-stage Interconnection Networks (MIN). We have discusses the Stability Problems of Existing Regular Omega Multi-stage Interconnection Network (OMIN) and Proposed 3-Disjoint Paths Omega Multi-stage Interconnection Network (3DON) using the approaches and solutions provided by…
▽ More
The research paper emphasizes that the Stable Matching problems are the same as the problems of stable configurations of Multi-stage Interconnection Networks (MIN). We have discusses the Stability Problems of Existing Regular Omega Multi-stage Interconnection Network (OMIN) and Proposed 3-Disjoint Paths Omega Multi-stage Interconnection Network (3DON) using the approaches and solutions provided by the Stable Matching Problem. Specifically, Stable Marriage Problem is used as an example of Stable Matching. On application of the concept of the Stable Marriage over the MINs states that OMIN is highly stable in comparison to 3DON.
△ Less
Submitted 6 February, 2012;
originally announced February 2012.
-
Case Tool: Fast Interconnections with New 3-Disjoint Paths MIN Simulation Module
Authors:
Ravi Rastogi,
Amit Singh,
Nikhil Singhal,
Nitin,
Durg Singh Chauhan
Abstract:
Multi-stage interconnection networks (MIN) can be designed to achieve fault tolerance and collision solving by providing a set of disjoint paths. In this paper, we are discussing the new simulator added to the tool designed for developing fault tolerant MINs. The designed tool is one of its own kind and will help the user in developing 2 and 3-disjoint path networks. The java technology has been u…
▽ More
Multi-stage interconnection networks (MIN) can be designed to achieve fault tolerance and collision solving by providing a set of disjoint paths. In this paper, we are discussing the new simulator added to the tool designed for developing fault tolerant MINs. The designed tool is one of its own kind and will help the user in developing 2 and 3-disjoint path networks. The java technology has been used to design the tool and have been tested on different software platform.
△ Less
Submitted 3 February, 2012;
originally announced February 2012.
-
Disjoint Paths Multi-stage Interconnection Networks Stability Problem
Authors:
Ravi Rastogi,
Nitin,
Durg Singh Chauhan,
Mahesh Chandra Govil
Abstract:
This research paper emphasizes that the Stable Matching problems are the same as the problems of stable configurations of Multi-stage Interconnection Networks (MIN). The authors have solved the Stability Problem of Existing Regular Gamma Multi-stage Interconnection Network (GMIN), 3-Disjoint Gamma Multi-stage Interconnection Network (3DGMIN) and 3-Disjoint Path Cyclic Gamma Multi-stage Interconnec…
▽ More
This research paper emphasizes that the Stable Matching problems are the same as the problems of stable configurations of Multi-stage Interconnection Networks (MIN). The authors have solved the Stability Problem of Existing Regular Gamma Multi-stage Interconnection Network (GMIN), 3-Disjoint Gamma Multi-stage Interconnection Network (3DGMIN) and 3-Disjoint Path Cyclic Gamma Multi-stage Interconnection Network (3DCGMIN) using the approaches and solutions provided by the Stable Matching Problem. Specifically Stable Marriage Problem is used as an example of Stable Matching. For MINs to prove Stable two existing algorithms are used:-the first algorithm generates the MINs Preferences List in time and second algorithm produces a set of most Optimal Pairs of the Switching Elements (SEs) (derived from the MINs Preferences List) in time. Moreover, the paper also solves the problem of Ties that occurs between the Optimal Pairs. The results are promising as the comparison of the MINs based on their stability shows that the ASEN, ABN, CLN, GMIN, 3DCGMIN are highly stable in comparison to HZTN, QTN, DGMIN. However, on comparing the irregular and regular MINs in totality upon their stability the regular MINs comes out to be more stable than the irregular MINs.
△ Less
Submitted 3 February, 2012;
originally announced February 2012.
