Showing 1–50 of 81 results for author: Singh, B

InVi: Object Insertion In Videos Using Off-the-Shelf Diffusion Models

Authors: Nirat Saini, Navaneeth Bodla, Ashish Shrivastava, Avinash Ravichandran, Xiao Zhang, Abhinav Shrivastava, Bharat Singh

Abstract: We introduce InVi, an approach for inserting or replacing objects within videos (referred to as inpainting) using off-the-shelf, text-to-image latent diffusion models. InVi targets controlled manipulation of objects and blending them seamlessly into a background video unlike existing video editing methods that focus on comprehensive re-styling or entire scene alterations. To achieve this goal, we… ▽ More We introduce InVi, an approach for inserting or replacing objects within videos (referred to as inpainting) using off-the-shelf, text-to-image latent diffusion models. InVi targets controlled manipulation of objects and blending them seamlessly into a background video unlike existing video editing methods that focus on comprehensive re-styling or entire scene alterations. To achieve this goal, we tackle two key challenges. Firstly, for high quality control and blending, we employ a two-step process involving inpainting and matching. This process begins with inserting the object into a single frame using a ControlNet-based inpainting diffusion model, and then generating subsequent frames conditioned on features from an inpainted frame as an anchor to minimize the domain gap between the background and the object. Secondly, to ensure temporal coherence, we replace the diffusion model's self-attention layers with extended-attention layers. The anchor frame features serve as the keys and values for these layers, enhancing consistency across frames. Our approach removes the need for video-specific fine-tuning, presenting an efficient and adaptable solution. Experimental results demonstrate that InVi achieves realistic object insertion with consistent blending and coherence across frames, outperforming existing methods. △ Less

Submitted 15 July, 2024; originally announced July 2024.

GenMM: Geometrically and Temporally Consistent Multimodal Data Generation for Video and LiDAR

Authors: Bharat Singh, Viveka Kulharia, Luyu Yang, Avinash Ravichandran, Ambrish Tyagi, Ashish Shrivastava

Abstract: Multimodal synthetic data generation is crucial in domains such as autonomous driving, robotics, augmented/virtual reality, and retail. We propose a novel approach, GenMM, for jointly editing RGB videos and LiDAR scans by inserting temporally and geometrically consistent 3D objects. Our method uses a reference image and 3D bounding boxes to seamlessly insert and blend new objects into target video… ▽ More Multimodal synthetic data generation is crucial in domains such as autonomous driving, robotics, augmented/virtual reality, and retail. We propose a novel approach, GenMM, for jointly editing RGB videos and LiDAR scans by inserting temporally and geometrically consistent 3D objects. Our method uses a reference image and 3D bounding boxes to seamlessly insert and blend new objects into target videos. We inpaint the 2D Regions of Interest (consistent with 3D boxes) using a diffusion-based video inpainting model. We then compute semantic boundaries of the object and estimate it's surface depth using state-of-the-art semantic segmentation and monocular depth estimation techniques. Subsequently, we employ a geometry-based optimization algorithm to recover the 3D shape of the object's surface, ensuring it fits precisely within the 3D bounding box. Finally, LiDAR rays intersecting with the new object surface are updated to reflect consistent depths with its geometry. Our experiments demonstrate the effectiveness of GenMM in inserting various 3D objects across video and LiDAR modalities. △ Less

Submitted 15 June, 2024; originally announced June 2024.

Streamflow Prediction with Uncertainty Quantification for Water Management: A Constrained Reasoning and Learning Approach

Authors: Mohammed Amine Gharsallaoui, Bhupinderjeet Singh, Supriya Savalkar, Aryan Deshwal, Yan Yan, Ananth Kalyanaraman, Kirti Rajagopalan, Janardhan Rao Doppa

Abstract: Predicting the spatiotemporal variation in streamflow along with uncertainty quantification enables decision-making for sustainable management of scarce water resources. Process-based hydrological models (aka physics-based models) are based on physical laws, but using simplifying assumptions which can lead to poor accuracy. Data-driven approaches offer a powerful alternative, but they require larg… ▽ More Predicting the spatiotemporal variation in streamflow along with uncertainty quantification enables decision-making for sustainable management of scarce water resources. Process-based hydrological models (aka physics-based models) are based on physical laws, but using simplifying assumptions which can lead to poor accuracy. Data-driven approaches offer a powerful alternative, but they require large amount of training data and tend to produce predictions that are inconsistent with physical laws. This paper studies a constrained reasoning and learning (CRL) approach where physical laws represented as logical constraints are integrated as a layer in the deep neural network. To address small data setting, we develop a theoretically-grounded training approach to improve the generalization accuracy of deep models. For uncertainty quantification, we combine the synergistic strengths of Gaussian processes (GPs) and deep temporal models (i.e., deep models for time-series forecasting) by passing the learned latent representation as input to a standard distance-based kernel. Experiments on multiple real-world datasets demonstrate the effectiveness of both CRL and GP with deep kernel approaches over strong baseline methods. △ Less

Submitted 31 May, 2024; originally announced June 2024.

Co-Occurring of Object Detection and Identification towards unlabeled object discovery

Authors: Binay Kumar Singh, Niels Da Vitoria Lobo

Abstract: In this paper, we propose a novel deep learning based approach for identifying co-occurring objects in conjunction with base objects in multilabel object categories. Nowadays, with the advancement in computer vision based techniques we need to know about co-occurring objects with respect to base object for various purposes. The pipeline of the proposed work is composed of two stages: in the first… ▽ More In this paper, we propose a novel deep learning based approach for identifying co-occurring objects in conjunction with base objects in multilabel object categories. Nowadays, with the advancement in computer vision based techniques we need to know about co-occurring objects with respect to base object for various purposes. The pipeline of the proposed work is composed of two stages: in the first stage of the proposed model we detect all the bounding boxes present in the image and their corresponding labels, then in the second stage we perform co-occurrence matrix analysis. In co-occurrence matrix analysis, we set base classes based on the maximum occurrences of the labels and build association rules and generate frequent patterns. These frequent patterns will show base classes and their corresponding co-occurring classes. We performed our experiments on two publicly available datasets: Pascal VOC and MS-COCO. The experimental results on public benchmark dataset is reported in Sec 4. Further we extend this work by considering all frequently objects as unlabeled and what if they are occluded as well. △ Less

Submitted 25 March, 2024; originally announced March 2024.

Comments: 6 pages, 2 figures,

STREAMLINE: An Automated Machine Learning Pipeline for Biomedicine Applied to Examine the Utility of Photography-Based Phenotypes for OSA Prediction Across International Sleep Centers

Authors: Ryan J. Urbanowicz, Harsh Bandhey, Brendan T. Keenan, Greg Maislin, Sy Hwang, Danielle L. Mowery, Shannon M. Lynch, Diego R. Mazzotti, Fang Han, Qing Yun Li, Thomas Penzel, Sergio Tufik, Lia Bittencourt, Thorarinn Gislason, Philip de Chazal, Bhajan Singh, Nigel McArdle, Ning-Hung Chen, Allan Pack, Richard J. Schwab, Peter A. Cistulli, Ulysses J. Magalang

Abstract: While machine learning (ML) includes a valuable array of tools for analyzing biomedical data, significant time and expertise is required to assemble effective, rigorous, and unbiased pipelines. Automated ML (AutoML) tools seek to facilitate ML application by automating a subset of analysis pipeline elements. In this study we develop and validate a Simple, Transparent, End-to-end Automated Machine… ▽ More While machine learning (ML) includes a valuable array of tools for analyzing biomedical data, significant time and expertise is required to assemble effective, rigorous, and unbiased pipelines. Automated ML (AutoML) tools seek to facilitate ML application by automating a subset of analysis pipeline elements. In this study we develop and validate a Simple, Transparent, End-to-end Automated Machine Learning Pipeline (STREAMLINE) and apply it to investigate the added utility of photography-based phenotypes for predicting obstructive sleep apnea (OSA); a common and underdiagnosed condition associated with a variety of health, economic, and safety consequences. STREAMLINE is designed to tackle biomedical binary classification tasks while adhering to best practices and accommodating complexity, scalability, reproducibility, customization, and model interpretation. Benchmarking analyses validated the efficacy of STREAMLINE across data simulations with increasingly complex patterns of association. Then we applied STREAMLINE to evaluate the utility of demographics (DEM), self-reported comorbidities (DX), symptoms (SYM), and photography-based craniofacial (CF) and intraoral (IO) anatomy measures in predicting any OSA or moderate/severe OSA using 3,111 participants from Sleep Apnea Global Interdisciplinary Consortium (SAGIC). OSA analyses identified a significant increase in ROC-AUC when adding CF to DEM+DX+SYM to predict moderate/severe OSA. A consistent but non-significant increase in PRC-AUC was observed with the addition of each subsequent feature set to predict any OSA, with CF and IO yielding minimal improvements. Application of STREAMLINE to OSA data suggests that CF features provide additional value in predicting moderate/severe OSA, but neither CF nor IO features meaningfully improved the prediction of any OSA beyond established demographics, comorbidity and symptom characteristics. △ Less

Submitted 8 December, 2023; originally announced December 2023.

Comments: 23 pages, 7 figures, 1 table, 1 supplemental information document (77 pages), and 7 ancillary files

Attention-based Models for Snow-Water Equivalent Prediction

Authors: Krishu K. Thapa, Bhupinderjeet Singh, Supriya Savalkar, Alan Fern, Kirti Rajagopalan, Ananth Kalyanaraman

Abstract: Snow Water-Equivalent (SWE) -- the amount of water available if snowpack is melted -- is a key decision variable used by water management agencies to make irrigation, flood control, power generation and drought management decisions. SWE values vary spatiotemporally -- affected by weather, topography and other environmental factors. While daily SWE can be measured by Snow Telemetry (SNOTEL) station… ▽ More Snow Water-Equivalent (SWE) -- the amount of water available if snowpack is melted -- is a key decision variable used by water management agencies to make irrigation, flood control, power generation and drought management decisions. SWE values vary spatiotemporally -- affected by weather, topography and other environmental factors. While daily SWE can be measured by Snow Telemetry (SNOTEL) stations with requisite instrumentation, such stations are spatially sparse requiring interpolation techniques to create spatiotemporally complete data. While recent efforts have explored machine learning (ML) for SWE prediction, a number of recent ML advances have yet to be considered. The main contribution of this paper is to explore one such ML advance, attention mechanisms, for SWE prediction. Our hypothesis is that attention has a unique ability to capture and exploit correlations that may exist across locations or the temporal spectrum (or both). We present a generic attention-based modeling framework for SWE prediction and adapt it to capture spatial attention and temporal attention. Our experimental results on 323 SNOTEL stations in the Western U.S. demonstrate that our attention-based models outperform other machine learning approaches. We also provide key results highlighting the differences between spatial and temporal attention in this context and a roadmap toward deployment for generating spatially-complete SWE maps. △ Less

Submitted 3 November, 2023; originally announced November 2023.

Comments: 7 pages, To be published in Proceedings of The Thirty-Sixth Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-24)

ACM Class: I.2

Crop Disease Classification using Support Vector Machines with Green Chromatic Coordinate (GCC) and Attention based feature extraction for IoT based Smart Agricultural Applications

Authors: Shashwat Jha, Vishvaditya Luhach, Gauri Shanker Gupta, Beependra Singh

Abstract: Crops hold paramount significance as they serve as the primary provider of energy, nutrition, and medicinal benefits for the human population. Plant diseases, however, can negatively affect leaves during agricultural cultivation, resulting in significant losses in crop output and economic value. Therefore, it is crucial for farmers to identify crop diseases. However, this method frequently necessi… ▽ More Crops hold paramount significance as they serve as the primary provider of energy, nutrition, and medicinal benefits for the human population. Plant diseases, however, can negatively affect leaves during agricultural cultivation, resulting in significant losses in crop output and economic value. Therefore, it is crucial for farmers to identify crop diseases. However, this method frequently necessitates hard work, a lot of planning, and in-depth familiarity with plant pathogens. Given these numerous obstacles, it is essential to provide solutions that can easily interface with mobile and IoT devices so that our farmers can guarantee the best possible crop development. Various machine learning (ML) as well as deep learning (DL) algorithms have been created & studied for the identification of plant disease detection, yielding substantial and promising results. This article presents a novel classification method that builds on prior work by utilising attention-based feature extraction, RGB channel-based chromatic analysis, Support Vector Machines (SVM) for improved performance, and the ability to integrate with mobile applications and IoT devices after quantization of information. Several disease classification algorithms were compared with the suggested model, and it was discovered that, in terms of accuracy, Vision Transformer-based feature extraction and additional Green Chromatic Coordinate feature with SVM classification achieved an accuracy of (GCCViT-SVM) - 99.69%, whereas after quantization for IoT device integration achieved an accuracy of - 97.41% while almost reducing 4x in size. Our findings have profound implications because they have the potential to transform how farmers identify crop illnesses with precise and fast information, thereby preserving agricultural output and ensuring food security. △ Less

Submitted 6 November, 2023; v1 submitted 1 November, 2023; originally announced November 2023.

QBSD: Quartile-Based Seasonality Decomposition for Cost-Effective Time Series Forecasting

Authors: Ebenezer RHP Isaac, Bulbul Singh

Abstract: In the telecom domain, precise forecasting of time series patterns, such as cell key performance indicators (KPIs), plays a pivotal role in enhancing service quality and operational efficiency. State-of-the-art forecasting approaches prioritize forecasting accuracy at the expense of computational performance, rendering them less suitable for data-intensive applications encompassing systems with a… ▽ More In the telecom domain, precise forecasting of time series patterns, such as cell key performance indicators (KPIs), plays a pivotal role in enhancing service quality and operational efficiency. State-of-the-art forecasting approaches prioritize forecasting accuracy at the expense of computational performance, rendering them less suitable for data-intensive applications encompassing systems with a multitude of time series variables. To address this issue, we introduce QBSD, a live forecasting approach tailored to optimize the trade-off between accuracy and computational complexity. We have evaluated the performance of QBSD against state-of-the-art forecasting approaches on publicly available datasets. We have also extended this investigation to our curated network KPI dataset, now publicly accessible, to showcase the effect of dynamic operating ranges that varies with time. The results demonstrate that the proposed method excels in runtime efficiency compared to the leading algorithms available while maintaining competitive forecast accuracy. △ Less

Submitted 16 August, 2023; v1 submitted 9 June, 2023; originally announced June 2023.

LD-ZNet: A Latent Diffusion Approach for Text-Based Image Segmentation

Authors: Koutilya Pnvr, Bharat Singh, Pallabi Ghosh, Behjat Siddiquie, David Jacobs

Abstract: Large-scale pre-training tasks like image classification, captioning, or self-supervised techniques do not incentivize learning the semantic boundaries of objects. However, recent generative foundation models built using text-based latent diffusion techniques may learn semantic boundaries. This is because they have to synthesize intricate details about all objects in an image based on a text descr… ▽ More Large-scale pre-training tasks like image classification, captioning, or self-supervised techniques do not incentivize learning the semantic boundaries of objects. However, recent generative foundation models built using text-based latent diffusion techniques may learn semantic boundaries. This is because they have to synthesize intricate details about all objects in an image based on a text description. Therefore, we present a technique for segmenting real and AI-generated images using latent diffusion models (LDMs) trained on internet-scale datasets. First, we show that the latent space of LDMs (z-space) is a better input representation compared to other feature representations like RGB images or CLIP encodings for text-based image segmentation. By training the segmentation models on the latent z-space, which creates a compressed representation across several domains like different forms of art, cartoons, illustrations, and photographs, we are also able to bridge the domain gap between real and AI-generated images. We show that the internal features of LDMs contain rich semantic information and present a technique in the form of LD-ZNet to further boost the performance of text-based segmentation. Overall, we show up to 6% improvement over standard baselines for text-to-image segmentation on natural images. For AI-generated imagery, we show close to 20% improvement compared to state-of-the-art techniques. The project is available at https://koutilya-pnvr.github.io/LD-ZNet/. △ Less

Submitted 23 August, 2023; v1 submitted 22 March, 2023; originally announced March 2023.

Comments: Supplementary material is included in the paper following the references section

Massively Multilingual Language Models for Cross Lingual Fact Extraction from Low Resource Indian Languages

Authors: Bhavyajeet Singh, Pavan Kandru, Anubhav Sharma, Vasudeva Varma

Abstract: Massive knowledge graphs like Wikidata attempt to capture world knowledge about multiple entities. Recent approaches concentrate on automatically enriching these KGs from text. However a lot of information present in the form of natural text in low resource languages is often missed out. Cross Lingual Information Extraction aims at extracting factual information in the form of English triples from… ▽ More Massive knowledge graphs like Wikidata attempt to capture world knowledge about multiple entities. Recent approaches concentrate on automatically enriching these KGs from text. However a lot of information present in the form of natural text in low resource languages is often missed out. Cross Lingual Information Extraction aims at extracting factual information in the form of English triples from low resource Indian Language text. Despite its massive potential, progress made on this task is lagging when compared to Monolingual Information Extraction. In this paper, we propose the task of Cross Lingual Fact Extraction(CLFE) from text and devise an end-to-end generative approach for the same which achieves an overall F1 score of 77.46. △ Less

Submitted 9 February, 2023; originally announced February 2023.

Comments: 5 pages, 2 page Apendix, 3 figures, accepted at 19th International Conference on Natural Language Processing

Two-step hyperparameter optimization method: Accelerating hyperparameter search by using a fraction of a training dataset

Authors: Sungduk Yu, Mike Pritchard, Po-Lun Ma, Balwinder Singh, Sam Silva

Abstract: Hyperparameter optimization (HPO) is an important step in machine learning (ML) model development, but common practices are archaic -- primarily relying on manual or grid searches. This is partly because adopting advanced HPO algorithms introduces added complexity to the workflow, leading to longer computation times. This poses a notable challenge to ML applications, as suboptimal hyperparameter s… ▽ More Hyperparameter optimization (HPO) is an important step in machine learning (ML) model development, but common practices are archaic -- primarily relying on manual or grid searches. This is partly because adopting advanced HPO algorithms introduces added complexity to the workflow, leading to longer computation times. This poses a notable challenge to ML applications, as suboptimal hyperparameter selections curtail the potential of ML model performance, ultimately obstructing the full exploitation of ML techniques. In this article, we present a two-step HPO method as a strategic solution to curbing computational demands and wait times, gleaned from practical experiences in applied ML parameterization work. The initial phase involves a preliminary evaluation of hyperparameters on a small subset of the training dataset, followed by a re-evaluation of the top-performing candidate models post-retraining with the entire training dataset. This two-step HPO method is universally applicable across HPO search algorithms, and we argue it has attractive efficiency gains. As a case study, we present our recent application of the two-step HPO method to the development of neural network emulators for aerosol activation. Although our primary use case is a data-rich limit with many millions of samples, we also find that using up to 0.0025% of the data (a few thousand samples) in the initial step is sufficient to find optimal hyperparameter configurations from much more extensive sampling, achieving up to 135-times speedup. The benefits of this method materialize through an assessment of hyperparameters and model performance, revealing the minimal model complexity required to achieve the best performance. The assortment of top-performing models harvested from the HPO process allows us to choose a high-performing model with a low inference cost for efficient use in global climate models (GCMs). △ Less

Submitted 7 September, 2023; v1 submitted 7 February, 2023; originally announced February 2023.

Journal ref: Artificial Intelligence for the Earth Systems, 3(1), 2024, e230013

Metaheuristic for Hub-Spoke Facility Location Problem: Application to Indian E-commerce Industry

Authors: Aakash Sachdeva, Bhupinder Singh, Rahul Prasad, Nakshatra Goel, Ronit Mondal, Jatin Munjal, Abhishek Bhatnagar, Manjeet Dahiya

Abstract: Indian e-commerce industry has evolved over the last decade and is expected to grow over the next few years. The focus has now shifted to turnaround time (TAT) due to the emergence of many third-party logistics providers and higher customer expectations. The key consideration for delivery providers is to balance their overall operating costs while meeting the promised TAT to their customers. E-com… ▽ More Indian e-commerce industry has evolved over the last decade and is expected to grow over the next few years. The focus has now shifted to turnaround time (TAT) due to the emergence of many third-party logistics providers and higher customer expectations. The key consideration for delivery providers is to balance their overall operating costs while meeting the promised TAT to their customers. E-commerce delivery partners operate through a network of facilities whose strategic locations help to run the operations efficiently. In this work, we identify the locations of hubs throughout the country and their corresponding mapping with the distribution centers. The objective is to minimize the total network costs with TAT adherence. We use Genetic Algorithm and leverage business constraints to reduce the solution search space and hence the solution time. The results indicate an improvement of 9.73% in TAT compliance compared with the current scenario. △ Less

Submitted 16 December, 2022; originally announced December 2022.

Tree DNN: A Deep Container Network

Authors: Brijraj Singh, Swati Gupta, Mayukh Das, Praveen Doreswamy Naidu, Sharan Kumar Allur

Abstract: Multi-Task Learning (MTL) has shown its importance at user products for fast training, data efficiency, reduced overfitting etc. MTL achieves it by sharing the network parameters and training a network for multiple tasks simultaneously. However, MTL does not provide the solution, if each task needs training from a different dataset. In order to solve the stated problem, we have proposed an archite… ▽ More Multi-Task Learning (MTL) has shown its importance at user products for fast training, data efficiency, reduced overfitting etc. MTL achieves it by sharing the network parameters and training a network for multiple tasks simultaneously. However, MTL does not provide the solution, if each task needs training from a different dataset. In order to solve the stated problem, we have proposed an architecture named TreeDNN along with it's training methodology. TreeDNN helps in training the model with multiple datasets simultaneously, where each branch of the tree may need a different training dataset. We have shown in the results that TreeDNN provides competitive performance with the advantage of reduced ROM requirement for parameter storage and increased responsiveness of the system by loading only specific branch at inference time. △ Less

Submitted 7 December, 2022; originally announced December 2022.

A CNN-LSTM Combination Network for Cataract Detection using Eye Fundus Images

Authors: Dishant Padalia, Abhishek Mazumdar, Bharati Singh

Abstract: According to multiple authoritative authorities, including the World Health Organization, vision-related impairments and disorders are becoming a significant issue. According to a recent report, one of the leading causes of irreversible blindness in persons over the age of 50 is delayed cataract treatment. A cataract is a cloudy spot in the eye's lens that causes visual loss. Cataracts often devel… ▽ More According to multiple authoritative authorities, including the World Health Organization, vision-related impairments and disorders are becoming a significant issue. According to a recent report, one of the leading causes of irreversible blindness in persons over the age of 50 is delayed cataract treatment. A cataract is a cloudy spot in the eye's lens that causes visual loss. Cataracts often develop slowly and consequently result in difficulty in driving, reading, and even recognizing faces. This necessitates the development of rapid and dependable diagnosis and treatment solutions for ocular illnesses. Previously, such visual illness diagnosis were done manually, which was time-consuming and prone to human mistake. However, as technology advances, automated, computer-based methods that decrease both time and human labor while producing trustworthy results are now accessible. In this study, we developed a CNN-LSTM-based model architecture with the goal of creating a low-cost diagnostic system that can classify normal and cataractous cases of ocular disease from fundus images. The proposed model was trained on the publicly available ODIR dataset, which included fundus images of patients' left and right eyes. The suggested architecture outperformed previous systems with a state-of-the-art 97.53% accuracy. △ Less

Submitted 28 October, 2022; originally announced October 2022.

Comments: 8 pages, 3 figures

Configured Grant for Ultra-Reliable and Low-Latency Communications: Standardization and Beyond

Authors: Majid Gerami, Bikramjit Singh

Abstract: Uplink configured Grant allocation has been introduced in 3rd Generation Partnership Project New Radio Release 15. This is beneficial in supporting Ultra-Reliable and Low Latency Communication for industrial communication, a key Fifth Generation mobile communication usage scenario. This scheduling mechanism enables a user with periodic traffic to transmits its data readily and bypasses the control… ▽ More Uplink configured Grant allocation has been introduced in 3rd Generation Partnership Project New Radio Release 15. This is beneficial in supporting Ultra-Reliable and Low Latency Communication for industrial communication, a key Fifth Generation mobile communication usage scenario. This scheduling mechanism enables a user with periodic traffic to transmits its data readily and bypasses the control signaling entailed to scheduling requests and scheduling grants and provides low latency access. To facilitate ultra-reliable communication, the scheduling mechanism can allow users to transmit consecutive redundant transmissions in a pre-defined period. However, if the traffic is semi-deterministic, the current standardized configured grant allocation is not equipped to emulate the traffic as the configured grant's period is pre-configured and fixed. This article describes the recent advancements in the standardization process in Release 15 and 16 for configured grant allocation and the prospective solutions to accommodate semi-deterministic traffic behavior for configured grant allocations. △ Less

Submitted 17 October, 2022; originally announced October 2022.

Comments: Accepted in IEEE Communications Standard Magazine 2021, 5 figures

XF2T: Cross-lingual Fact-to-Text Generation for Low-Resource Languages

Authors: Shivprasad Sagare, Tushar Abhishek, Bhavyajeet Singh, Anubhav Sharma, Manish Gupta, Vasudeva Varma

Abstract: Multiple business scenarios require an automated generation of descriptive human-readable text from structured input data. Hence, fact-to-text generation systems have been developed for various downstream tasks like generating soccer reports, weather and financial reports, medical reports, person biographies, etc. Unfortunately, previous work on fact-to-text (F2T) generation has focused primarily… ▽ More Multiple business scenarios require an automated generation of descriptive human-readable text from structured input data. Hence, fact-to-text generation systems have been developed for various downstream tasks like generating soccer reports, weather and financial reports, medical reports, person biographies, etc. Unfortunately, previous work on fact-to-text (F2T) generation has focused primarily on English mainly due to the high availability of relevant datasets. Only recently, the problem of cross-lingual fact-to-text (XF2T) was proposed for generation across multiple languages alongwith a dataset, XALIGN for eight languages. However, there has been no rigorous work on the actual XF2T generation problem. We extend XALIGN dataset with annotated data for four more languages: Punjabi, Malayalam, Assamese and Oriya. We conduct an extensive study using popular Transformer-based text generation models on our extended multi-lingual dataset, which we call XALIGNV2. Further, we investigate the performance of different text generation strategies: multiple variations of pretraining, fact-aware embeddings and structure-aware input encoding. Our extensive experiments show that a multi-lingual mT5 model which uses fact-aware embeddings with structure-aware input encoding leads to best results on average across the twelve languages. We make our code, dataset and model publicly available, and hope that this will help advance further research in this critical area. △ Less

Submitted 22 September, 2022; originally announced September 2022.

Topological Optimized Convolutional Visual Recurrent Network for Brain Tumor Segmentation and Classification

Authors: Dhananjay Joshi, Bhupesh Kumar Singh, Kapil Kumar Nagwanshi, Nitin S. Choubey

Abstract: In today's world of health care, brain tumor detection has become common. However, the manual brain tumor classification approach is time-consuming. So Deep Convolutional Neural Network (DCNN) is used by many researchers in the medical field for making accurate diagnoses and aiding in the patient's treatment. The traditional techniques have problems such as overfitting and the inability to extract… ▽ More In today's world of health care, brain tumor detection has become common. However, the manual brain tumor classification approach is time-consuming. So Deep Convolutional Neural Network (DCNN) is used by many researchers in the medical field for making accurate diagnoses and aiding in the patient's treatment. The traditional techniques have problems such as overfitting and the inability to extract necessary features. To overcome these problems, we developed the Topological Data Analysis based Improved Persistent Homology (TDA-IPH) and Convolutional Transfer learning and Visual Recurrent learning with Elephant Herding Optimization hyper-parameter tuning (CTVR-EHO) models for brain tumor segmentation and classification. Initially, the Topological Data Analysis based Improved Persistent Homology is designed to segment the brain tumor image. Then, from the segmented image, features are extracted using TL via the AlexNet model and Bidirectional Visual Long Short-Term Memory (Bi-VLSTM). Next, elephant Herding Optimization (EHO) is used to tune the hyperparameters of both networks to get an optimal result. Finally, extracted features are concatenated and classified using the softmax activation layer. The simulation result of this proposed CTVR-EHO and TDA-IPH method is analyzed based on precision, accuracy, recall, loss, and F score metrics. When compared to other existing brain tumor segmentation and classification models, the proposed CTVR-EHO and TDA-IPH approaches show high accuracy (99.8%), high recall (99.23%), high precision (99.67%), and high F score (99.59%). △ Less

Submitted 14 July, 2024; v1 submitted 6 June, 2022; originally announced July 2022.

MSC Class: 68U10 ACM Class: I.4

BPFISH: Blockchain and Privacy-preserving FL Inspired Smart Healthcare

Authors: Moirangthem Biken Singh, Ajay Pratap

Abstract: This paper proposes Federated Learning (FL) based smart healthcare system where Medical Centers (MCs) train the local model using the data collected from patients and send the model weights to the miners in a blockchain-based robust framework without sharing raw data, keeping privacy preservation into deliberation. We formulate an optimization problem by maximizing the utility and minimizing the l… ▽ More This paper proposes Federated Learning (FL) based smart healthcare system where Medical Centers (MCs) train the local model using the data collected from patients and send the model weights to the miners in a blockchain-based robust framework without sharing raw data, keeping privacy preservation into deliberation. We formulate an optimization problem by maximizing the utility and minimizing the loss function considering energy consumption and FL process delay of MCs for learning effective models on distributed healthcare data underlying a blockchain-based framework. We propose a solution in two stages: first, offer a stable matching-based association algorithm to maximize the utility of both miners and MCs and then solve loss minimization using Stochastic Gradient Descent (SGD) algorithm employing FL under Differential Privacy (DP) and blockchain technology. Moreover, we incorporate blockchain technology to provide tempered resistant and decentralized model weight sharing in the proposed FL-based framework. The effectiveness of the proposed model is shown through simulation on real-world healthcare data comparing other state-of-the-art techniques. △ Less

Submitted 27 July, 2022; v1 submitted 24 July, 2022; originally announced July 2022.

On the modern deep learning approaches for precipitation downscaling

Authors: Bipin Kumar, Kaustubh Atey, Bhupendra Bahadur Singh, Rajib Chattopadhyay, Nachiket Acharya, Manmeet Singh, Ravi S. Nanjundiah, Suryachandra A. Rao

Abstract: Deep Learning (DL) based downscaling has become a popular tool in earth sciences recently. Increasingly, different DL approaches are being adopted to downscale coarser precipitation data and generate more accurate and reliable estimates at local (~few km or even smaller) scales. Despite several studies adopting dynamical or statistical downscaling of precipitation, the accuracy is limited by the a… ▽ More Deep Learning (DL) based downscaling has become a popular tool in earth sciences recently. Increasingly, different DL approaches are being adopted to downscale coarser precipitation data and generate more accurate and reliable estimates at local (~few km or even smaller) scales. Despite several studies adopting dynamical or statistical downscaling of precipitation, the accuracy is limited by the availability of ground truth. A key challenge to gauge the accuracy of such methods is to compare the downscaled data to point-scale observations which are often unavailable at such small scales. In this work, we carry out the DL-based downscaling to estimate the local precipitation data from the India Meteorological Department (IMD), which was created by approximating the value from station location to a grid point. To test the efficacy of different DL approaches, we apply four different methods of downscaling and evaluate their performance. The considered approaches are (i) Deep Statistical Downscaling (DeepSD), augmented Convolutional Long Short Term Memory (ConvLSTM), fully convolutional network (U-NET), and Super-Resolution Generative Adversarial Network (SR-GAN). A custom VGG network, used in the SR-GAN, is developed in this work using precipitation data. The results indicate that SR-GAN is the best method for precipitation data downscaling. The downscaled data is validated with precipitation values at IMD station. This DL method offers a promising alternative to statistical downscaling. △ Less

Submitted 2 July, 2022; originally announced July 2022.

Report number: https://link.springer.com/epdf/10.1007/s12145-023-00970-4?sharing_token=M2LtWJ53pv-LFqv8L_7A9_e4RwlQNchNByi7wbcMAY6_OzYwFMXUC7cAoOcWE6w2ZfADFLxgA09ceiotTMNU3MFJkk4Uz7yDh2Sm_5GVwT31ims1NgmcJlE9PNP5VLG9KcfXtKgbCDXMyShcFb_r1fWMORXAH5iwFTYmyJReRXs%3D

Journal ref: Earth Science Informatics, 2023

AutoCoMet: Smart Neural Architecture Search via Co-Regulated Shaping Reinforcement

Authors: Mayukh Das, Brijraj Singh, Harsh Kanti Chheda, Pawan Sharma, Pradeep NS

Abstract: Designing suitable deep model architectures, for AI-driven on-device apps and features, at par with rapidly evolving mobile hardware and increasingly complex target scenarios is a difficult task. Though Neural Architecture Search (NAS/AutoML) has made this easier by shifting paradigm from extensive manual effort to automated architecture learning from data, yet it has major limitations, leading to… ▽ More Designing suitable deep model architectures, for AI-driven on-device apps and features, at par with rapidly evolving mobile hardware and increasingly complex target scenarios is a difficult task. Though Neural Architecture Search (NAS/AutoML) has made this easier by shifting paradigm from extensive manual effort to automated architecture learning from data, yet it has major limitations, leading to critical bottlenecks in the context of mobile devices, including model-hardware fidelity, prohibitive search times and deviation from primary target objective(s). Thus, we propose AutoCoMet that can learn the most suitable DNN architecture optimized for varied types of device hardware and task contexts, ~ 3x faster. Our novel co-regulated shaping reinforcement controller together with the high fidelity hardware meta-behavior predictor produces a smart, fast NAS framework that adapts to context via a generalized formalism for any kind of multi-criteria optimization. △ Less

Submitted 29 March, 2022; originally announced March 2022.

Comments: ICPR 2022

XAlign: Cross-lingual Fact-to-Text Alignment and Generation for Low-Resource Languages

Authors: Tushar Abhishek, Shivprasad Sagare, Bhavyajeet Singh, Anubhav Sharma, Manish Gupta, Vasudeva Varma

Abstract: Multiple critical scenarios (like Wikipedia text generation given English Infoboxes) need automated generation of descriptive text in low resource (LR) languages from English fact triples. Previous work has focused on English fact-to-text (F2T) generation. To the best of our knowledge, there has been no previous attempt on cross-lingual alignment or generation for LR languages. Building an effecti… ▽ More Multiple critical scenarios (like Wikipedia text generation given English Infoboxes) need automated generation of descriptive text in low resource (LR) languages from English fact triples. Previous work has focused on English fact-to-text (F2T) generation. To the best of our knowledge, there has been no previous attempt on cross-lingual alignment or generation for LR languages. Building an effective cross-lingual F2T (XF2T) system requires alignment between English structured facts and LR sentences. We propose two unsupervised methods for cross-lingual alignment. We contribute XALIGN, an XF2T dataset with 0.45M pairs across 8 languages, of which 5402 pairs have been manually annotated. We also train strong baseline XF2T generation models on the XAlign dataset. △ Less

Submitted 24 April, 2022; v1 submitted 1 February, 2022; originally announced February 2022.

Comments: Update the code repository and acknowledgement

Semantic Annotation and Querying Framework based on Semi-structured Ayurvedic Text

Authors: Hrishikesh Terdalkar, Arnab Bhattacharya, Madhulika Dubey, Ramamurthy S, Bhavna Naneria Singh

Abstract: Knowledge bases (KB) are an important resource in a number of natural language processing (NLP) and information retrieval (IR) tasks, such as semantic search, automated question-answering etc. They are also useful for researchers trying to gain information from a text. Unfortunately, however, the state-of-the-art in Sanskrit NLP does not yet allow automated construction of knowledge bases due to u… ▽ More Knowledge bases (KB) are an important resource in a number of natural language processing (NLP) and information retrieval (IR) tasks, such as semantic search, automated question-answering etc. They are also useful for researchers trying to gain information from a text. Unfortunately, however, the state-of-the-art in Sanskrit NLP does not yet allow automated construction of knowledge bases due to unavailability or lack of sufficient accuracy of tools and methods. Thus, in this work, we describe our efforts on manual annotation of Sanskrit text for the purpose of knowledge graph (KG) creation. We choose the chapter Dhanyavarga from Bhavaprakashanighantu of the Ayurvedic text Bhavaprakasha for annotation. The constructed knowledge graph contains 410 entities and 764 relationships. Since Bhavaprakashanighantu is a technical glossary text that describes various properties of different substances, we develop an elaborate ontology to capture the semantics of the entity and relationship types present in the text. To query the knowledge graph, we design 31 query templates that cover most of the common question patterns. For both manual annotation and querying, we customize the Sangrahaka framework previously developed by us. The entire system including the dataset is available from https://sanskrit.iitk.ac.in/ayurveda/ . We hope that the knowledge graph that we have created through manual annotation and subsequent curation will help in development and testing of NLP tools in future as well as studying of the Bhavaprakasanighantu text. △ Less

Submitted 31 January, 2022; originally announced February 2022.

Comments: 19 pages including appendix

Journal ref: n Proceedings of the Computational Sanskrit & Digital Humanities: Selected papers presented at the 18th World Sanskrit Conference, 2023, pages 155--173, Canberra, Australia (Online mode). Association for Computational Linguistics

Sentiment Analysis of Microblogging dataset on Coronavirus Pandemic

Authors: Nosin Ibna Mahbub, Md Rakibul Islam, Md Al Amin, Md Khairul Islam, Bikash Chandra Singh, Md Imran Hossain Showrov, Anirudda Sarkar

Abstract: Sentiment analysis can largely influence the people to get the update of the current situation. Coronavirus (COVID-19) is a contagious illness caused by the coronavirus 2 that causes severe respiratory symptoms. The lives of millions have continued to be affected by this pandemic, several countries have resorted to a full lockdown. During this lockdown, people have taken social networks to express… ▽ More Sentiment analysis can largely influence the people to get the update of the current situation. Coronavirus (COVID-19) is a contagious illness caused by the coronavirus 2 that causes severe respiratory symptoms. The lives of millions have continued to be affected by this pandemic, several countries have resorted to a full lockdown. During this lockdown, people have taken social networks to express their emotions to find a way to calm themselves down. People are spreading their sentiments through microblogging websites as one of the most preventive steps of this disease is the socialization to gain people's awareness to stay home and keep their distance when they are outside home. Twitter is a popular online social media platform for exchanging ideas. People can post their different sentiments, which can be used to aware people. But, some people want to spread fake news to frighten the people. So, it is necessary to identify the positive, negative, and neutral thoughts so that the positive opinions can be delivered to the mass people for spreading awareness to the people. Moreover, a huge volume of data is floating on Twitter. So, it is also important to identify the context of the dataset. In this paper, we have analyzed the Twitter dataset for evaluating the sentiment using several machine learning algorithms. Later, we have found out the context learning of the dataset based on the sentiments. △ Less

Submitted 17 November, 2021; originally announced November 2021.

Comments: 7 pages, 5 figures, 5th IEEE International Conference on Electrical Information and Communication Technology (EICT)

MSC Class: 68Uxx ACM Class: I.7

Journal ref: 2021 5th International Conference on Electrical Information and Communication Technology (EICT)

Improved Robustness and Safety for Pre-Adaptation of Meta Reinforcement Learning with Prior Regularization

Authors: Lu Wen, Songan Zhang, H. Eric Tseng, Baljeet Singh, Dimitar Filev, Huei Peng

Abstract: Meta Reinforcement Learning (Meta-RL) has seen substantial advancements recently. In particular, off-policy methods were developed to improve the data efficiency of Meta-RL techniques. \textit{Probabilistic embeddings for actor-critic RL} (PEARL) is a leading approach for multi-MDP adaptation problems. A major drawback of many existing Meta-RL methods, including PEARL, is that they do not explicit… ▽ More Meta Reinforcement Learning (Meta-RL) has seen substantial advancements recently. In particular, off-policy methods were developed to improve the data efficiency of Meta-RL techniques. \textit{Probabilistic embeddings for actor-critic RL} (PEARL) is a leading approach for multi-MDP adaptation problems. A major drawback of many existing Meta-RL methods, including PEARL, is that they do not explicitly consider the safety of the prior policy when it is exposed to a new task for the first time. Safety is essential for many real-world applications, including field robots and Autonomous Vehicles (AVs). In this paper, we develop the PEARL PLUS (PEARL$^+$) algorithm, which optimizes the policy for both prior (pre-adaptation) safety and posterior (after-adaptation) performance. Building on top of PEARL, our proposed PEARL$^+$ algorithm introduces a prior regularization term in the reward function and a new Q-network for recovering the state-action value under prior context assumptions, to improve the robustness to task distribution shift and safety of the trained network exposed to a new task for the first time. The performance of PEARL$^+$ is validated by solving three safety-critical problems related to robots and AVs, including two MuJoCo benchmark problems. From the simulation experiments, we show that safety of the prior policy is significantly improved and more robust to task distribution shift compared to PEARL. △ Less

Submitted 9 February, 2023; v1 submitted 18 August, 2021; originally announced August 2021.

Multi-Contextual Design of Convolutional Neural Network for Steganalysis

Authors: Brijesh Singh, Arijit Sur, Pinaki Mitra

Abstract: In recent times, deep learning-based steganalysis classifiers became popular due to their state-of-the-art performance. Most deep steganalysis classifiers usually extract noise residuals using high-pass filters as preprocessing steps and feed them to their deep model for classification. It is observed that recent steganographic embedding does not always restrict their embedding in the high-frequen… ▽ More In recent times, deep learning-based steganalysis classifiers became popular due to their state-of-the-art performance. Most deep steganalysis classifiers usually extract noise residuals using high-pass filters as preprocessing steps and feed them to their deep model for classification. It is observed that recent steganographic embedding does not always restrict their embedding in the high-frequency zone; instead, they distribute it as per embedding policy. Therefore, besides noise residual, learning the embedding zone is another challenging task. In this work, unlike the conventional approaches, the proposed model first extracts the noise residual using learned denoising kernels to boost the signal-to-noise ratio. After preprocessing, the sparse noise residuals are fed to a novel Multi-Contextual Convolutional Neural Network (M-CNET) that uses heterogeneous context size to learn the sparse and low-amplitude representation of noise residuals. The model performance is further improved by incorporating the Self-Attention module to focus on the areas prone to steganalytic embedding. A set of comprehensive experiments is performed to show the proposed scheme's efficacy over the prior arts. Besides, an ablation study is given to justify the contribution of various modules of the proposed architecture. △ Less

Submitted 4 November, 2021; v1 submitted 19 June, 2021; originally announced June 2021.

Comments: Under Review

Criticality and Utility-aware Fog Computing System for Remote Health Monitoring

Authors: Moirangthem Biken Singh, Navneet Taunk, Naveen Kumar Mall, Ajay Pratap

Abstract: Growing remote health monitoring system allows constant monitoring of the patient's condition and performance of preventive and control check-ups outside medical facilities. However, the real-time smart-healthcare application poses a delay constraint that has to be solved efficiently. Fog computing is emerging as an efficient solution for such real-time applications. Moreover, different medical ce… ▽ More Growing remote health monitoring system allows constant monitoring of the patient's condition and performance of preventive and control check-ups outside medical facilities. However, the real-time smart-healthcare application poses a delay constraint that has to be solved efficiently. Fog computing is emerging as an efficient solution for such real-time applications. Moreover, different medical centers are getting attracted to the growing IoT-based remote healthcare system in order to make a profit by hiring Fog computing resources. However, there is a need for an efficient algorithmic model for allocation of limited fog computing resources in the criticality-aware smart-healthcare system considering the profit of medical centers. Thus, the objective of this work is to maximize the system utility calculated as a linear combination of the profit of the medical center and the loss of patients. To measure profit, we propose a flat-pricing-based model. Further, we propose a swapping-based heuristic to maximize the system utility. The proposed heuristic is tested on various parameters and shown to perform close to the optimal with criticality-awareness in its core. Through extensive simulations, we show that the proposed heuristic achieves an average utility of $96\%$ of the optimal, in polynomial time complexity. △ Less

Submitted 2 April, 2022; v1 submitted 24 May, 2021; originally announced May 2021.

Scale Normalized Image Pyramids with AutoFocus for Object Detection

Authors: Bharat Singh, Mahyar Najibi, Abhishek Sharma, Larry S. Davis

Abstract: We present an efficient foveal framework to perform object detection. A scale normalized image pyramid (SNIP) is generated that, like human vision, only attends to objects within a fixed size range at different scales. Such a restriction of objects' size during training affords better learning of object-sensitive filters, and therefore, results in better accuracy. However, the use of an image pyra… ▽ More We present an efficient foveal framework to perform object detection. A scale normalized image pyramid (SNIP) is generated that, like human vision, only attends to objects within a fixed size range at different scales. Such a restriction of objects' size during training affords better learning of object-sensitive filters, and therefore, results in better accuracy. However, the use of an image pyramid increases the computational cost. Hence, we propose an efficient spatial sub-sampling scheme which only operates on fixed-size sub-regions likely to contain objects (as object locations are known during training). The resulting approach, referred to as Scale Normalized Image Pyramid with Efficient Resampling or SNIPER, yields up to 3 times speed-up during training. Unfortunately, as object locations are unknown during inference, the entire image pyramid still needs processing. To this end, we adopt a coarse-to-fine approach, and predict the locations and extent of object-like regions which will be processed in successive scales of the image pyramid. Intuitively, it's akin to our active human-vision that first skims over the field-of-view to spot interesting regions for further processing and only recognizes objects at the right resolution. The resulting algorithm is referred to as AutoFocus and results in a 2.5-5 times speed-up during inference when used with SNIP. △ Less

Submitted 10 February, 2021; originally announced February 2021.

Comments: Accepted in T-PAMI 2021

Configured Grant for Semi-Deterministic Traffic for Ultra-Reliable and Low Latency Communications

Authors: Bikramjit Singh, Majid Gerami

Abstract: Configured Grant-based allocation has been adopted in New Radio 3rd Generation Partnership Project Release 16. This scheme is beneficial in supporting Ultra-Reliable and Low Latency Communication for industrial communication, a key Fifth Generation mobile communication usage scenario. This scheduling mechanism enables a user with a periodic traffic to transmit its data readily and bypasses the sig… ▽ More Configured Grant-based allocation has been adopted in New Radio 3rd Generation Partnership Project Release 16. This scheme is beneficial in supporting Ultra-Reliable and Low Latency Communication for industrial communication, a key Fifth Generation mobile communication usage scenario. This scheduling mechanism enables a user with a periodic traffic to transmit its data readily and bypasses the signaling entailed to scheduling requests and scheduling grants; and provides low latency access. To facilitate ultra-reliable communication, the scheduling mechanism can allow the user to transmit redundant transmissions at consecutive repetition occasions in a pre-defined period. However, for the user with semi-deterministic traffic, the reliability and latency performance with Configured Grant-based allocation deteriorates. This can be due to, e.g., late data arrival in the buffer, and the user unable to transmit its repetitions, which leads to reliability degradation. To improve the Configured Grant reliability performance with semi-deterministic traffic, we consider various allocation designs utilizing, e.g., additional unlicensed spectrum, or flexible transmission in a Configured Grant period, or allowing time-gaps between the repetitions, etc. These enhancements could be a stepping-stone for Sixth Generation Configured Grant models. △ Less

Submitted 9 September, 2020; originally announced September 2020.

Comments: 6G Wireless Summit 2020, poster paper, 2 pages

COVID-19 Pandemic Outbreak in the Subcontinent: A data-driven analysis

Authors: Bikash Chandra Singh, Zulfikar Alom, Mohammad Muntasir Rahman, Mrinal Kanti Baowaly, Mohammad Abdul Azim

Abstract: Human civilization is experiencing a critical situation that presents itself for a new coronavirus disease 2019 (COVID-19). This virus emerged in late December 2019 in Wuhan city, Hubei, China. The grim fact of COVID-19 is, it is highly contagious in nature, therefore, spreads rapidly all over the world and causes severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Responding to the seve… ▽ More Human civilization is experiencing a critical situation that presents itself for a new coronavirus disease 2019 (COVID-19). This virus emerged in late December 2019 in Wuhan city, Hubei, China. The grim fact of COVID-19 is, it is highly contagious in nature, therefore, spreads rapidly all over the world and causes severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Responding to the severity of COVID-19 research community directs the attention to the analysis of COVID-19, to diminish its antagonistic impact towards society. Numerous studies claim that the subcontinent, i.e., Bangladesh, India, and Pakistan, could remain in the worst affected region by the COVID-19. In order to prevent the spread of COVID-19, it is important to predict the trend of COVID-19 beforehand the planning of effective control strategies. Fundamentally, the idea is to dependably estimate the reproduction number to judge the spread rate of COVID-19 in a particular region. Consequently, this paper uses publicly available epidemiological data of Bangladesh, India, and Pakistan to estimate the reproduction numbers. More specifically, we use various models (for example, susceptible infection recovery (SIR), exponential growth (EG), sequential Bayesian (SB), maximum likelihood (ML) and time dependent (TD)) to estimate the reproduction numbers and observe the model fitness in the corresponding data set. Experimental results show that the reproduction numbers produced by these models are greater than 1.2 (approximately) indicates that COVID-19 is gradually spreading in the subcontinent. △ Less

Submitted 22 August, 2020; originally announced August 2020.

Comments: 11 pages, 7 figures, Submitted to: Travel Medicine and Infectious Disease

ASAP-NMS: Accelerating Non-Maximum Suppression Using Spatially Aware Priors

Authors: Rohun Tripathi, Vasu Singla, Mahyar Najibi, Bharat Singh, Abhishek Sharma, Larry Davis

Abstract: The widely adopted sequential variant of Non Maximum Suppression (or Greedy-NMS) is a crucial module for object-detection pipelines. Unfortunately, for the region proposal stage of two/multi-stage detectors, NMS is turning out to be a latency bottleneck due to its sequential nature. In this article, we carefully profile Greedy-NMS iterations to find that a major chunk of computation is wasted in c… ▽ More The widely adopted sequential variant of Non Maximum Suppression (or Greedy-NMS) is a crucial module for object-detection pipelines. Unfortunately, for the region proposal stage of two/multi-stage detectors, NMS is turning out to be a latency bottleneck due to its sequential nature. In this article, we carefully profile Greedy-NMS iterations to find that a major chunk of computation is wasted in comparing proposals that are already far-away and have a small chance of suppressing each other. We address this issue by comparing only those proposals that are generated from nearby anchors. The translation-invariant property of the anchor lattice affords generation of a lookup table, which provides an efficient access to nearby proposals, during NMS. This leads to an Accelerated NMS algorithm which leverages Spatially Aware Priors, or ASAP-NMS, and improves the latency of the NMS step from 13.6ms to 1.2 ms on a CPU without sacrificing the accuracy of a state-of-the-art two-stage detector on COCO and VOC datasets. Importantly, ASAP-NMS is agnostic to image resolution and can be used as a simple drop-in module during inference. Using ASAP-NMS at run-time only, we obtain an mAP of 44.2\%@25Hz on the COCO dataset with a V100 GPU. △ Less

Submitted 21 August, 2020; v1 submitted 19 July, 2020; originally announced July 2020.

Comments: Under Review at CVIU

MOSQUITO-NET: A deep learning based CADx system for malaria diagnosis along with model interpretation using GradCam and class activation maps

Authors: Aayush Kumar, Sanat B Singh, Suresh Chandra Satapathy, Minakhi Rout

Abstract: Malaria is considered one of the deadliest diseases in today world which causes thousands of deaths per year. The parasites responsible for malaria are scientifically known as Plasmodium which infects the red blood cells in human beings. The parasites are transmitted by a female class of mosquitos known as Anopheles. The diagnosis of malaria requires identification and manual counting of parasitiz… ▽ More Malaria is considered one of the deadliest diseases in today world which causes thousands of deaths per year. The parasites responsible for malaria are scientifically known as Plasmodium which infects the red blood cells in human beings. The parasites are transmitted by a female class of mosquitos known as Anopheles. The diagnosis of malaria requires identification and manual counting of parasitized cells by medical practitioners in microscopic blood smears. Due to the unavailability of resources, its diagnostic accuracy is largely affected by large scale screening. State of the art Computer-aided diagnostic techniques based on deep learning algorithms such as CNNs, with end to end feature extraction and classification, have widely contributed to various image recognition tasks. In this paper, we evaluate the performance of custom made convnet Mosquito-Net, to classify the infected and uninfected cells for malaria diagnosis which could be deployed on the edge and mobile devices owing to its fewer parameters and less computation power. Therefore, it can be wildly preferred for diagnosis in remote and countryside areas where there is a lack of medical facilities. △ Less

Submitted 19 June, 2020; v1 submitted 17 June, 2020; originally announced June 2020.

Comments: arXiv admin note: text overlap with arXiv:2003.09871 by other authors

RSO: A Gradient Free Sampling Based Approach For Training Deep Neural Networks

Authors: Rohun Tripathi, Bharat Singh

Abstract: We propose RSO (random search optimization), a gradient free Markov Chain Monte Carlo search based approach for training deep neural networks. To this end, RSO adds a perturbation to a weight in a deep neural network and tests if it reduces the loss on a mini-batch. If this reduces the loss, the weight is updated, otherwise the existing weight is retained. Surprisingly, we find that repeating this… ▽ More We propose RSO (random search optimization), a gradient free Markov Chain Monte Carlo search based approach for training deep neural networks. To this end, RSO adds a perturbation to a weight in a deep neural network and tests if it reduces the loss on a mini-batch. If this reduces the loss, the weight is updated, otherwise the existing weight is retained. Surprisingly, we find that repeating this process a few times for each weight is sufficient to train a deep neural network. The number of weight updates for RSO is an order of magnitude lesser when compared to backpropagation with SGD. RSO can make aggressive weight updates in each step as there is no concept of learning rate. The weight update step for individual layers is also not coupled with the magnitude of the loss. RSO is evaluated on classification tasks on MNIST and CIFAR-10 datasets with deep neural networks of 6 to 10 layers where it achieves an accuracy of 99.1% and 81.8% respectively. We also find that after updating the weights just 5 times, the algorithm obtains a classification accuracy of 98% on MNIST. △ Less

Submitted 12 May, 2020; originally announced May 2020.

Comments: Technical Report

Multi-label natural language processing to identify diagnosis and procedure codes from MIMIC-III inpatient notes

Authors: A. K. Bhavani Singh, Mounika Guntu, Ananth Reddy Bhimireddy, Judy W. Gichoya, Saptarshi Purkayastha

Abstract: In the United States, 25% or greater than 200 billion dollars of hospital spending accounts for administrative costs that involve services for medical coding and billing. With the increasing number of patient records, manual assignment of the codes performed is overwhelming, time-consuming and error-prone, causing billing errors. Natural language processing can automate the extraction of codes/lab… ▽ More In the United States, 25% or greater than 200 billion dollars of hospital spending accounts for administrative costs that involve services for medical coding and billing. With the increasing number of patient records, manual assignment of the codes performed is overwhelming, time-consuming and error-prone, causing billing errors. Natural language processing can automate the extraction of codes/labels from unstructured clinical notes, which can aid human coders to save time, increase productivity, and verify medical coding errors. Our objective is to identify appropriate diagnosis and procedure codes from clinical notes by performing multi-label classification. We used de-identified data of critical care patients from the MIMIC-III database and subset the data to select the ten (top-10) and fifty (top-50) most common diagnoses and procedures, which covers 47.45% and 74.12% of all admissions respectively. We implemented state-of-the-art Bidirectional Encoder Representations from Transformers (BERT) to fine-tune the language model on 80% of the data and validated on the remaining 20%. The model achieved an overall accuracy of 87.08%, an F1 score of 85.82%, and an AUC of 91.76% for top-10 codes. For the top-50 codes, our model achieved an overall accuracy of 93.76%, an F1 score of 92.24%, and AUC of 91%. When compared to previously published research, our model outperforms in predicting codes from the clinical text. We discuss approaches to generalize the knowledge discovery process of our MIMIC-BERT to other clinical notes. This can help human coders to save time, prevent backlogs, and additional costs due to coding errors. △ Less

Submitted 16 March, 2020; originally announced March 2020.

Comments: This is a shortened version of the Capstone Project that was accepted by the Faculty of Indiana University, in partial fulfillment of the requirements for the degree of Master of Science in Health Informatics

Recognizing Instagram Filtered Images with Feature De-stylization

Authors: Zhe Wu, Zuxuan Wu, Bharat Singh, Larry S. Davis

Abstract: Deep neural networks have been shown to suffer from poor generalization when small perturbations are added (like Gaussian noise), yet little work has been done to evaluate their robustness to more natural image transformations like photo filters. This paper presents a study on how popular pretrained models are affected by commonly used Instagram filters. To this end, we introduce ImageNet-Instagra… ▽ More Deep neural networks have been shown to suffer from poor generalization when small perturbations are added (like Gaussian noise), yet little work has been done to evaluate their robustness to more natural image transformations like photo filters. This paper presents a study on how popular pretrained models are affected by commonly used Instagram filters. To this end, we introduce ImageNet-Instagram, a filtered version of ImageNet, where 20 popular Instagram filters are applied to each image in ImageNet. Our analysis suggests that simple structure preserving filters which only alter the global appearance of an image can lead to large differences in the convolutional feature space. To improve generalization, we introduce a lightweight de-stylization module that predicts parameters used for scaling and shifting feature maps to "undo" the changes incurred by filters, inverting the process of style transfer tasks. We further demonstrate the module can be readily plugged into modern CNN architectures together with skip connections. We conduct extensive studies on ImageNet-Instagram, and show quantitatively and qualitatively, that the proposed module, among other things, can effectively improve generalization by simply learning normalization parameters without retraining the entire network, thus recovering the alterations in the feature space caused by the filters. △ Less

Submitted 30 December, 2019; originally announced December 2019.

Comments: Accepted in AAAI 2020 as an oral presentation paper

FastV2C-HandNet: Fast Voxel to Coordinate Hand Pose Estimation with 3D Convolutional Neural Networks

Authors: Rohan Lekhwani, Bhupendra Singh

Abstract: Hand pose estimation from monocular depth images has been an important and challenging problem in the Computer Vision community. In this paper, we present a novel approach to estimate 3D hand joint locations from 2D depth images. Unlike most of the previous methods, our model captures the 3D spatial information from a depth image thereby giving it a greater understanding of the input. We voxelize… ▽ More Hand pose estimation from monocular depth images has been an important and challenging problem in the Computer Vision community. In this paper, we present a novel approach to estimate 3D hand joint locations from 2D depth images. Unlike most of the previous methods, our model captures the 3D spatial information from a depth image thereby giving it a greater understanding of the input. We voxelize the input depth map to capture the 3D features of the input and perform 3D data augmentations to make our network robust to real-world images. Our network is trained in an end-to-end manner which reduces time and space complexity significantly when compared to other methods. Through extensive experiments, we show that our model outperforms state-of-the-art methods with respect to the time it takes to train and predict 3D hand joint locations. This makes our method more suitable for real-world hand pose estimation scenarios. △ Less

Submitted 20 February, 2020; v1 submitted 15 July, 2019; originally announced July 2019.

Comments: 13 pages, 5 figures, 2 tables

Deep Learning in the Automotive Industry: Recent Advances and Application Examples

Authors: Kanwar Bharat Singh, Mustafa Ali Arat

Abstract: One of the most exciting technology breakthroughs in the last few years has been the rise of deep learning. State-of-the-art deep learning models are being widely deployed in academia and industry, across a variety of areas, from image analysis to natural language processing. These models have grown from fledgling research subjects to mature techniques in real-world use. The increasing scale of da… ▽ More One of the most exciting technology breakthroughs in the last few years has been the rise of deep learning. State-of-the-art deep learning models are being widely deployed in academia and industry, across a variety of areas, from image analysis to natural language processing. These models have grown from fledgling research subjects to mature techniques in real-world use. The increasing scale of data, computational power and the associated algorithmic innovations are the main drivers for the progress we see in this field. These developments also have a huge potential for the automotive industry and therefore the interest in deep learning-based technology is growing. A lot of the product innovations, such as self-driving cars, parking and lane-change assist or safety functions, such as autonomous emergency braking, are powered by deep learning algorithms. Deep learning is poised to offer gains in performance and functionality for most ADAS (Advanced Driver Assistance System) solutions. Virtual sensing for vehicle dynamics application, vehicle inspection/heath monitoring, automated driving and data-driven product development are key areas that are expected to get the most attention. This article provides an overview of the recent advances and some associated challenges in deep learning techniques in the context of automotive applications. △ Less

Submitted 24 June, 2019; v1 submitted 20 June, 2019; originally announced June 2019.

Automatic Long-Term Deception Detection in Group Interaction Videos

Authors: Chongyang Bai, Maksim Bolonkin, Judee Burgoon, Chao Chen, Norah Dunbar, Bharat Singh, V. S. Subrahmanian, Zhe Wu

Abstract: Most work on automated deception detection (ADD) in video has two restrictions: (i) it focuses on a video of one person, and (ii) it focuses on a single act of deception in a one or two minute video. In this paper, we propose a new ADD framework which captures long term deception in a group setting. We study deception in the well-known Resistance game (like Mafia and Werewolf) which consists of 5-… ▽ More Most work on automated deception detection (ADD) in video has two restrictions: (i) it focuses on a video of one person, and (ii) it focuses on a single act of deception in a one or two minute video. In this paper, we propose a new ADD framework which captures long term deception in a group setting. We study deception in the well-known Resistance game (like Mafia and Werewolf) which consists of 5-8 players of whom 2-3 are spies. Spies are deceptive throughout the game (typically 30-65 minutes) to keep their identity hidden. We develop an ensemble predictive model to identify spies in Resistance videos. We show that features from low-level and high-level video analysis are insufficient, but when combined with a new class of features that we call LiarRank, produce the best results. We achieve AUCs of over 0.70 in a fully automated setting. Our demo can be found at http://home.cs.dartmouth.edu/~mbolonkin/scan/demo/ △ Less

Submitted 15 June, 2019; v1 submitted 15 May, 2019; originally announced May 2019.

Comments: ICME 2019

Multi-resolution Networks For Flexible Irregular Time Series Modeling (Multi-FIT)

Authors: Bhanu Pratap Singh, Iman Deznabi, Bharath Narasimhan, Bryon Kucharski, Rheeya Uppaal, Akhila Josyula, Madalina Fiterau

Abstract: Missing values, irregularly collected samples, and multi-resolution signals commonly occur in multivariate time series data, making predictive tasks difficult. These challenges are especially prevalent in the healthcare domain, where patients' vital signs and electronic records are collected at different frequencies and have occasionally missing information due to the imperfections in equipment or… ▽ More Missing values, irregularly collected samples, and multi-resolution signals commonly occur in multivariate time series data, making predictive tasks difficult. These challenges are especially prevalent in the healthcare domain, where patients' vital signs and electronic records are collected at different frequencies and have occasionally missing information due to the imperfections in equipment or patient circumstances. Researchers have handled each of these issues differently, often handling missing data through mean value imputation and then using sequence models over the multivariate signals while ignoring the different resolution of signals. We propose a unified model named Multi-resolution Flexible Irregular Time series Network (Multi-FIT). The building block for Multi-FIT is the FIT network. The FIT network creates an informative dense representation at each time step using signal information such as last observed value, time difference since the last observed time stamp and overall mean for the signal. Vertical FIT (FIT-V) is a variant of FIT which also models the relationship between different temporal signals while creating the informative dense representations for the signal. The multi-FIT model uses multiple FIT networks for sets of signals with different resolutions, further facilitating the construction of flexible representations. Our model has three main contributions: a.) it does not impute values but rather creates informative representations to provide flexibility to the model for creating task-specific representations b.) it models the relationship between different signals in the form of support signals c.) it models different resolutions in parallel before merging them for the final prediction task. The FIT, FIT-V and Multi-FIT networks improve upon the state-of-the-art models for three predictive tasks, including the forecasting of patient survival. △ Less

Submitted 30 April, 2019; originally announced May 2019.

An Analysis of Pre-Training on Object Detection

Authors: Hengduo Li, Bharat Singh, Mahyar Najibi, Zuxuan Wu, Larry S. Davis

Abstract: We provide a detailed analysis of convolutional neural networks which are pre-trained on the task of object detection. To this end, we train detectors on large datasets like OpenImagesV4, ImageNet Localization and COCO. We analyze how well their features generalize to tasks like image classification, semantic segmentation and object detection on small datasets like PASCAL-VOC, Caltech-256, SUN-397… ▽ More We provide a detailed analysis of convolutional neural networks which are pre-trained on the task of object detection. To this end, we train detectors on large datasets like OpenImagesV4, ImageNet Localization and COCO. We analyze how well their features generalize to tasks like image classification, semantic segmentation and object detection on small datasets like PASCAL-VOC, Caltech-256, SUN-397, Flowers-102 etc. Some important conclusions from our analysis are --- 1) Pre-training on large detection datasets is crucial for fine-tuning on small detection datasets, especially when precise localization is needed. For example, we obtain 81.1% mAP on the PASCAL-VOC dataset at 0.7 IoU after pre-training on OpenImagesV4, which is 7.6% better than the recently proposed DeformableConvNetsV2 which uses ImageNet pre-training. 2) Detection pre-training also benefits other localization tasks like semantic segmentation but adversely affects image classification. 3) Features for images (like avg. pooled Conv5) which are similar in the object detection feature space are likely to be similar in the image classification feature space but the converse is not true. 4) Visualization of features reveals that detection neurons have activations over an entire object, while activations for classification networks typically focus on parts. Therefore, detection networks are poor at classification when multiple instances are present in an image or when an instance only covers a small fraction of an image. △ Less

Submitted 11 April, 2019; originally announced April 2019.

EvalAI: Towards Better Evaluation Systems for AI Agents

Authors: Deshraj Yadav, Rishabh Jain, Harsh Agrawal, Prithvijit Chattopadhyay, Taranjeet Singh, Akash Jain, Shiv Baran Singh, Stefan Lee, Dhruv Batra

Abstract: We introduce EvalAI, an open source platform for evaluating and comparing machine learning (ML) and artificial intelligence algorithms (AI) at scale. EvalAI is built to provide a scalable solution to the research community to fulfill the critical need of evaluating machine learning models and agents acting in an environment against annotations or with a human-in-the-loop. This will help researcher… ▽ More We introduce EvalAI, an open source platform for evaluating and comparing machine learning (ML) and artificial intelligence algorithms (AI) at scale. EvalAI is built to provide a scalable solution to the research community to fulfill the critical need of evaluating machine learning models and agents acting in an environment against annotations or with a human-in-the-loop. This will help researchers, students, and data scientists to create, collaborate, and participate in AI challenges organized around the globe. By simplifying and standardizing the process of benchmarking these models, EvalAI seeks to lower the barrier to entry for participating in the global scientific effort to push the frontiers of machine learning and artificial intelligence, thereby increasing the rate of measurable progress in this domain. △ Less

Submitted 10 February, 2019; originally announced February 2019.

TAN: Temporal Aggregation Network for Dense Multi-label Action Recognition

Authors: Xiyang Dai, Bharat Singh, Joe Yue-Hei Ng, Larry S. Davis

Abstract: We present Temporal Aggregation Network (TAN) which decomposes 3D convolutions into spatial and temporal aggregation blocks. By stacking spatial and temporal convolutions repeatedly, TAN forms a deep hierarchical representation for capturing spatio-temporal information in videos. Since we do not apply 3D convolutions in each layer but only apply temporal aggregation blocks once after each spatial… ▽ More We present Temporal Aggregation Network (TAN) which decomposes 3D convolutions into spatial and temporal aggregation blocks. By stacking spatial and temporal convolutions repeatedly, TAN forms a deep hierarchical representation for capturing spatio-temporal information in videos. Since we do not apply 3D convolutions in each layer but only apply temporal aggregation blocks once after each spatial downsampling layer in the network, we significantly reduce the model complexity. The use of dilated convolutions at different resolutions of the network helps in aggregating multi-scale spatio-temporal information efficiently. Experiments show that our model is well suited for dense multi-label action recognition, which is a challenging sub-topic of action recognition that requires predicting multiple action labels in each frame. We outperform state-of-the-art methods by 5% and 3% on the Charades and Multi-THUMOS dataset respectively. △ Less

Submitted 14 December, 2018; originally announced December 2018.

Comments: WACV 2019

FA-RPN: Floating Region Proposals for Face Detection

Authors: Mahyar Najibi, Bharat Singh, Larry S. Davis

Abstract: We propose a novel approach for generating region proposals for performing face-detection. Instead of classifying anchor boxes using features from a pixel in the convolutional feature map, we adopt a pooling-based approach for generating region proposals. However, pooling hundreds of thousands of anchors which are evaluated for generating proposals becomes a computational bottleneck during inferen… ▽ More We propose a novel approach for generating region proposals for performing face-detection. Instead of classifying anchor boxes using features from a pixel in the convolutional feature map, we adopt a pooling-based approach for generating region proposals. However, pooling hundreds of thousands of anchors which are evaluated for generating proposals becomes a computational bottleneck during inference. To this end, an efficient anchor placement strategy for reducing the number of anchor-boxes is proposed. We then show that proposals generated by our network (Floating Anchor Region Proposal Network, FA-RPN) are better than RPN for generating region proposals for face detection. We discuss several beneficial features of FA-RPN proposals like iterative refinement, placement of fractional anchors and changing anchors which can be enabled without making any changes to the trained model. Our face detector based on FA-RPN obtains 89.4% mAP with a ResNet-50 backbone on the WIDER dataset. △ Less

Submitted 13 December, 2018; originally announced December 2018.

AutoFocus: Efficient Multi-Scale Inference

Authors: Mahyar Najibi, Bharat Singh, Larry S. Davis

Abstract: This paper describes AutoFocus, an efficient multi-scale inference algorithm for deep-learning based object detectors. Instead of processing an entire image pyramid, AutoFocus adopts a coarse to fine approach and only processes regions which are likely to contain small objects at finer scales. This is achieved by predicting category agnostic segmentation maps for small objects at coarser scales, c… ▽ More This paper describes AutoFocus, an efficient multi-scale inference algorithm for deep-learning based object detectors. Instead of processing an entire image pyramid, AutoFocus adopts a coarse to fine approach and only processes regions which are likely to contain small objects at finer scales. This is achieved by predicting category agnostic segmentation maps for small objects at coarser scales, called FocusPixels. FocusPixels can be predicted with high recall, and in many cases, they only cover a small fraction of the entire image. To make efficient use of FocusPixels, an algorithm is proposed which generates compact rectangular FocusChips which enclose FocusPixels. The detector is only applied inside FocusChips, which reduces computation while processing finer scales. Different types of error can arise when detections from FocusChips of multiple scales are combined, hence techniques to correct them are proposed. AutoFocus obtains an mAP of 47.9% (68.3% at 50% overlap) on the COCO test-dev set while processing 6.4 images per second on a Titan X (Pascal) GPU. This is 2.5X faster than our multi-scale baseline detector and matches its mAP. The number of pixels processed in the pyramid can be reduced by 5X with a 1% drop in mAP. AutoFocus obtains more than 10% mAP gain compared to RetinaNet but runs at the same speed with the same ResNet-101 backbone. △ Less

Submitted 1 August, 2019; v1 submitted 4 December, 2018; originally announced December 2018.

Comments: To appear in Proceedings of International Conference on Computer Vision (ICCV), 2019

Open Vocabulary Learning on Source Code with a Graph-Structured Cache

Authors: Milan Cvitkovic, Badal Singh, Anima Anandkumar

Abstract: Machine learning models that take computer program source code as input typically use Natural Language Processing (NLP) techniques. However, a major challenge is that code is written using an open, rapidly changing vocabulary due to, e.g., the coinage of new variable and method names. Reasoning over such a vocabulary is not something for which most NLP methods are designed. We introduce a Graph-St… ▽ More Machine learning models that take computer program source code as input typically use Natural Language Processing (NLP) techniques. However, a major challenge is that code is written using an open, rapidly changing vocabulary due to, e.g., the coinage of new variable and method names. Reasoning over such a vocabulary is not something for which most NLP methods are designed. We introduce a Graph-Structured Cache to address this problem; this cache contains a node for each new word the model encounters with edges connecting each word to its occurrences in the code. We find that combining this graph-structured cache strategy with recent Graph-Neural-Network-based models for supervised learning on code improves the models' performance on a code completion task and a variable naming task --- with over $100\%$ relative improvement on the latter --- at the cost of a moderate increase in computation time. △ Less

Submitted 19 May, 2019; v1 submitted 18 October, 2018; originally announced October 2018.

Comments: Published in the International Conference on Machine Learning (ICML 2019), 13 pages

A Datamining Approach for Emotions Extraction and Discovering Cricketers performance from Stadium to Sensex

Authors: Amit Agarwal, Brijraj Singh, Jatin Bedi, Durga Toshniwal

Abstract: Microblogging sites are the direct platform for the users to express their views. It has been observed from previous studies that people are viable to flaunt their emotions for events (eg. natural catastrophes, sports, academics etc.), for persons (actor/actress, sports person, scientist) and for the places they visit. In this study we focused on a sport event, particularly the cricket tournament… ▽ More Microblogging sites are the direct platform for the users to express their views. It has been observed from previous studies that people are viable to flaunt their emotions for events (eg. natural catastrophes, sports, academics etc.), for persons (actor/actress, sports person, scientist) and for the places they visit. In this study we focused on a sport event, particularly the cricket tournament and collected the emotions of the fans for their favorite players using their tweets. Further, we acquired the stock market performance of the brands which are either endorsing the players or sponsoring the match in the tournament. It has been observed that performance of the player triggers the users to flourish their emotions over social media therefore, we observed correlation between players performance and fans' emotions. Therefore, we found the direct connection between player's performance with brand's behavior on stock market. △ Less

Submitted 2 September, 2018; originally announced September 2018.

Comments: Accepted as a Workshop paper at WSDM 2018

Reduction of Redundant Rules in Association Rule Mining-Based Bug Assignment

Authors: Meera Sharma, Abhishek Tandon, Madhu Kumari, V B Singh

Abstract: Bug triaging is a process to decide what to do with newly coming bug reports. In this paper, we have mined association rules for the prediction of bug assignee of a newly reported bug using different bug attributes, namely, severity, priority, component and operating system. To deal with the problem of large data sets, we have taken subsets of data set by dividing the large data set using K-means… ▽ More Bug triaging is a process to decide what to do with newly coming bug reports. In this paper, we have mined association rules for the prediction of bug assignee of a newly reported bug using different bug attributes, namely, severity, priority, component and operating system. To deal with the problem of large data sets, we have taken subsets of data set by dividing the large data set using K-means clustering algorithm. We have used an Apriori algorithm in MATLAB to generate association rules. We have extracted the association rules for top 5 assignees in each cluster.The proposed method has been empirically validated on 14696 bug reports of Mozilla open source software project, namely, Seamonkey, Firefox and Bugzilla. The proposed method provides an improvement over the existing techniques for bug assignment problem. △ Less

Submitted 23 July, 2018; originally announced July 2018.

Comments: 14 pages

Journal ref: International Journal of Reliability, Quality and Safety Engineering Vol. 24, No. 6 (2017) 1740005 (14 pages) World Scientific Publishing Company

Soft Sampling for Robust Object Detection

Authors: Zhe Wu, Navaneeth Bodla, Bharat Singh, Mahyar Najibi, Rama Chellappa, Larry S. Davis

Abstract: We study the robustness of object detection under the presence of missing annotations. In this setting, the unlabeled object instances will be treated as background, which will generate an incorrect training signal for the detector. Interestingly, we observe that after dropping 30% of the annotations (and labeling them as background), the performance of CNN-based object detectors like Faster-RCNN… ▽ More We study the robustness of object detection under the presence of missing annotations. In this setting, the unlabeled object instances will be treated as background, which will generate an incorrect training signal for the detector. Interestingly, we observe that after dropping 30% of the annotations (and labeling them as background), the performance of CNN-based object detectors like Faster-RCNN only drops by 5% on the PASCAL VOC dataset. We provide a detailed explanation for this result. To further bridge the performance gap, we propose a simple yet effective solution, called Soft Sampling. Soft Sampling re-weights the gradients of RoIs as a function of overlap with positive instances. This ensures that the uncertain background regions are given a smaller weight compared to the hardnegatives. Extensive experiments on curated PASCAL VOC datasets demonstrate the effectiveness of the proposed Soft Sampling method at different annotation drop rates. Finally, we show that on OpenImagesV3, which is a real-world dataset with missing annotations, Soft Sampling outperforms standard detection baselines by over 3%. △ Less

Submitted 21 July, 2019; v1 submitted 18 June, 2018; originally announced June 2018.

Comments: Accepted in BMVC 2019

SNIPER: Efficient Multi-Scale Training

Authors: Bharat Singh, Mahyar Najibi, Larry S. Davis

Abstract: We present SNIPER, an algorithm for performing efficient multi-scale training in instance level visual recognition tasks. Instead of processing every pixel in an image pyramid, SNIPER processes context regions around ground-truth instances (referred to as chips) at the appropriate scale. For background sampling, these context-regions are generated using proposals extracted from a region proposal n… ▽ More We present SNIPER, an algorithm for performing efficient multi-scale training in instance level visual recognition tasks. Instead of processing every pixel in an image pyramid, SNIPER processes context regions around ground-truth instances (referred to as chips) at the appropriate scale. For background sampling, these context-regions are generated using proposals extracted from a region proposal network trained with a short learning schedule. Hence, the number of chips generated per image during training adaptively changes based on the scene complexity. SNIPER only processes 30% more pixels compared to the commonly used single scale training at 800x1333 pixels on the COCO dataset. But, it also observes samples from extreme resolutions of the image pyramid, like 1400x2000 pixels. As SNIPER operates on resampled low resolution chips (512x512 pixels), it can have a batch size as large as 20 on a single GPU even with a ResNet-101 backbone. Therefore it can benefit from batch-normalization during training without the need for synchronizing batch-normalization statistics across GPUs. SNIPER brings training of instance level recognition tasks like object detection closer to the protocol for image classification and suggests that the commonly accepted guideline that it is important to train on high resolution images for instance level visual recognition tasks might not be correct. Our implementation based on Faster-RCNN with a ResNet-101 backbone obtains an mAP of 47.6% on the COCO dataset for bounding box detection and can process 5 images per second during inference with a single GPU. Code is available at https://github.com/MahyarNajibi/SNIPER/. △ Less

Submitted 13 December, 2018; v1 submitted 23 May, 2018; originally announced May 2018.

Comments: Presented at the NIPS 2018 conference

Deception Detection in Videos

Authors: Zhe Wu, Bharat Singh, Larry S. Davis, V. S. Subrahmanian

Abstract: We present a system for covert automated deception detection in real-life courtroom trial videos. We study the importance of different modalities like vision, audio and text for this task. On the vision side, our system uses classifiers trained on low level video features which predict human micro-expressions. We show that predictions of high-level micro-expressions can be used as features for dec… ▽ More We present a system for covert automated deception detection in real-life courtroom trial videos. We study the importance of different modalities like vision, audio and text for this task. On the vision side, our system uses classifiers trained on low level video features which predict human micro-expressions. We show that predictions of high-level micro-expressions can be used as features for deception prediction. Surprisingly, IDT (Improved Dense Trajectory) features which have been widely used for action recognition, are also very good at predicting deception in videos. We fuse the score of classifiers trained on IDT features and high-level micro-expressions to improve performance. MFCC (Mel-frequency Cepstral Coefficients) features from the audio domain also provide a significant boost in performance, while information from transcripts is not very beneficial for our system. Using various classifiers, our automated system obtains an AUC of 0.877 (10-fold cross-validation) when evaluated on subjects which were not part of the training set. Even though state-of-the-art methods use human annotations of micro-expressions for deception detection, our fully automated approach outperforms them by 5%. When combined with human annotations of micro-expressions, our AUC improves to 0.922. We also present results of a user-study to analyze how well do average humans perform on this task, what modalities they use for deception detection and how they perform if only one modality is accessible. Our project page can be found at \url{https://doubaibai.github.io/DARE/}. △ Less

Submitted 12 December, 2017; originally announced December 2017.

Comments: AAAI 2018, project page: https://doubaibai.github.io/DARE/

R-FCN-3000 at 30fps: Decoupling Detection and Classification

Authors: Bharat Singh, Hengduo Li, Abhishek Sharma, Larry S. Davis

Abstract: We present R-FCN-3000, a large-scale real-time object detector in which objectness detection and classification are decoupled. To obtain the detection score for an RoI, we multiply the objectness score with the fine-grained classification score. Our approach is a modification of the R-FCN architecture in which position-sensitive filters are shared across different object classes for performing loc… ▽ More We present R-FCN-3000, a large-scale real-time object detector in which objectness detection and classification are decoupled. To obtain the detection score for an RoI, we multiply the objectness score with the fine-grained classification score. Our approach is a modification of the R-FCN architecture in which position-sensitive filters are shared across different object classes for performing localization. For fine-grained classification, these position-sensitive filters are not needed. R-FCN-3000 obtains an mAP of 34.9% on the ImageNet detection dataset and outperforms YOLO-9000 by 18% while processing 30 images per second. We also show that the objectness learned by R-FCN-3000 generalizes to novel classes and the performance increases with the number of training object classes - supporting the hypothesis that it is possible to learn a universal objectness detector. Code will be made available. △ Less

Submitted 5 December, 2017; originally announced December 2017.

Comments: CVPR 2018 submission