-
Multi-hypothesis 3D human pose estimation metrics favor miscalibrated distributions
Authors:
Paweł A. Pierzchlewicz,
R. James Cotton,
Mohammad Bashiri,
Fabian H. Sinz
Abstract:
Due to depth ambiguities and occlusions, lifting 2D poses to 3D is a highly ill-posed problem. Well-calibrated distributions of possible poses can make these ambiguities explicit and preserve the resulting uncertainty for downstream tasks. This study shows that previous attempts, which account for these ambiguities via multiple hypotheses generation, produce miscalibrated distributions. We identif…
▽ More
Due to depth ambiguities and occlusions, lifting 2D poses to 3D is a highly ill-posed problem. Well-calibrated distributions of possible poses can make these ambiguities explicit and preserve the resulting uncertainty for downstream tasks. This study shows that previous attempts, which account for these ambiguities via multiple hypotheses generation, produce miscalibrated distributions. We identify that miscalibration can be attributed to the use of sample-based metrics such as minMPJPE. In a series of simulations, we show that minimizing minMPJPE, as commonly done, should converge to the correct mean prediction. However, it fails to correctly capture the uncertainty, thus resulting in a miscalibrated distribution. To mitigate this problem, we propose an accurate and well-calibrated model called Conditional Graph Normalizing Flow (cGNFs). Our model is structured such that a single cGNF can estimate both conditional and marginal densities within the same model - effectively solving a zero-shot density estimation problem. We evaluate cGNF on the Human~3.6M dataset and show that cGNF provides a well-calibrated distribution estimate while being close to state-of-the-art in terms of overall minMPJPE. Furthermore, cGNF outperforms previous methods on occluded joints while it remains well-calibrated.
△ Less
Submitted 20 October, 2022;
originally announced October 2022.
-
The Sensorium competition on predicting large-scale mouse primary visual cortex activity
Authors:
Konstantin F. Willeke,
Paul G. Fahey,
Mohammad Bashiri,
Laura Pede,
Max F. Burg,
Christoph Blessing,
Santiago A. Cadena,
Zhiwei Ding,
Konstantin-Klemens Lurz,
Kayla Ponder,
Taliah Muhammad,
Saumil S. Patel,
Alexander S. Ecker,
Andreas S. Tolias,
Fabian H. Sinz
Abstract:
The neural underpinning of the biological visual system is challenging to study experimentally, in particular as the neuronal activity becomes increasingly nonlinear with respect to visual input. Artificial neural networks (ANNs) can serve a variety of goals for improving our understanding of this complex system, not only serving as predictive digital twins of sensory cortex for novel hypothesis g…
▽ More
The neural underpinning of the biological visual system is challenging to study experimentally, in particular as the neuronal activity becomes increasingly nonlinear with respect to visual input. Artificial neural networks (ANNs) can serve a variety of goals for improving our understanding of this complex system, not only serving as predictive digital twins of sensory cortex for novel hypothesis generation in silico, but also incorporating bio-inspired architectural motifs to progressively bridge the gap between biological and machine vision. The mouse has recently emerged as a popular model system to study visual information processing, but no standardized large-scale benchmark to identify state-of-the-art models of the mouse visual system has been established. To fill this gap, we propose the Sensorium benchmark competition. We collected a large-scale dataset from mouse primary visual cortex containing the responses of more than 28,000 neurons across seven mice stimulated with thousands of natural images, together with simultaneous behavioral measurements that include running speed, pupil dilation, and eye movements. The benchmark challenge will rank models based on predictive performance for neuronal responses on a held-out test set, and includes two tracks for model input limited to either stimulus only (Sensorium) or stimulus plus behavior (Sensorium+). We provide a starting kit to lower the barrier for entry, including tutorials, pre-trained baseline models, and APIs with one line commands for data loading and submission. We would like to see this as a starting point for regular challenges and data releases, and as a standard tool for measuring progress in large-scale neural system identification models of the mouse visual system and beyond.
△ Less
Submitted 17 June, 2022;
originally announced June 2022.
-
Toward a Knowledge Discovery Framework for Data Science Job Market in the United States
Authors:
Mojtaba Heidarysafa,
Kamran Kowsari,
Masoud Bashiri,
Donald E. Brown
Abstract:
The growth of the data science field requires better tools to understand such a fast-paced growing domain. Moreover, individuals from different backgrounds became interested in following a career as data scientists. Therefore, providing a quantitative guide for individuals and organizations to understand the skills required in the job market would be crucial. This paper introduces a framework to a…
▽ More
The growth of the data science field requires better tools to understand such a fast-paced growing domain. Moreover, individuals from different backgrounds became interested in following a career as data scientists. Therefore, providing a quantitative guide for individuals and organizations to understand the skills required in the job market would be crucial. This paper introduces a framework to analyze the job market for data science-related jobs within the US while providing an interface to access insights in this market. The proposed framework includes three sub-modules allowing continuous data collection, information extraction, and a web-based dashboard visualization to investigate the spatial and temporal distribution of data science-related jobs and skills. The result of this work shows important skills for the main branches of data science jobs and attempts to provide a skill-based definition of these data science branches. The current version of this application is deployed on the web and allows individuals and institutes to investigate skills required for data science positions through the industry lens.
△ Less
Submitted 20 July, 2021; v1 submitted 14 June, 2021;
originally announced June 2021.
-
Data-Driven Intersection Management Solutions for Mixed Traffic of Human-Driven and Connected and Automated Vehicles
Authors:
Masoud Bashiri
Abstract:
This dissertation proposes two solutions for urban traffic control in the presence of connected and automated vehicles. First a centralized platoon-based controller is proposed for the cooperative intersection management problem that takes advantage of the platooning systems and V2I communication to generate fast and smooth traffic flow at a single intersection.
Second, a data-driven approach is…
▽ More
This dissertation proposes two solutions for urban traffic control in the presence of connected and automated vehicles. First a centralized platoon-based controller is proposed for the cooperative intersection management problem that takes advantage of the platooning systems and V2I communication to generate fast and smooth traffic flow at a single intersection.
Second, a data-driven approach is proposed for adaptive signal control in the presence of connected vehicles. The proposed system relies on a data-driven method for optimal signal timing and a data-driven heuristic method for estimating routing decisions. It requires no additional sensors to be installed at the intersection, reducing the installation costs compared to typical settings of state-of-the-practice adaptive signal controllers.
The proposed traffic controller contains an optimal signal timing module and a traffic state estimator. The signal timing module is a neural network model trained on microscopic simulation data to achieve optimal results according to a given performance metric such as vehicular delay or average queue length. The traffic state estimator relies on connected vehicles' information to estimate the traffic's routing decisions. A heuristic method is proposed to minimize the estimation error. With sufficient parameter tuning, the estimation error decreases as the market penetration rate (MPR) of connected vehicles grows. Estimation error is below 30% for an MPR of 10% and it shrinks below 20% when MPR grows larger than 30%.
Simulations showed that the proposed traffic controller outperforms Highway Capacity Manual's methodology and given proper offline parameter tuning, it can decrease average vehicular delay by up to 25%.
△ Less
Submitted 9 December, 2020;
originally announced December 2020.
-
Consistent Robust Adversarial Prediction for General Multiclass Classification
Authors:
Rizal Fathony,
Kaiser Asif,
Anqi Liu,
Mohammad Ali Bashiri,
Wei Xing,
Sima Behpour,
Xinhua Zhang,
Brian D. Ziebart
Abstract:
We propose a robust adversarial prediction framework for general multiclass classification. Our method seeks predictive distributions that robustly optimize non-convex and non-continuous multiclass loss metrics against the worst-case conditional label distributions (the adversarial distributions) that (approximately) match the statistics of the training data. Although the optimized loss metrics ar…
▽ More
We propose a robust adversarial prediction framework for general multiclass classification. Our method seeks predictive distributions that robustly optimize non-convex and non-continuous multiclass loss metrics against the worst-case conditional label distributions (the adversarial distributions) that (approximately) match the statistics of the training data. Although the optimized loss metrics are non-convex and non-continuous, the dual formulation of the framework is a convex optimization problem that can be recast as a risk minimization model with a prescribed convex surrogate loss we call the adversarial surrogate loss. We show that the adversarial surrogate losses fill an existing gap in surrogate loss construction for general multiclass classification problems, by simultaneously aligning better with the original multiclass loss, guaranteeing Fisher consistency, enabling a way to incorporate rich feature spaces via the kernel trick, and providing competitive performance in practice.
△ Less
Submitted 20 November, 2019; v1 submitted 18 December, 2018;
originally announced December 2018.
-
Distributionally Robust Graphical Models
Authors:
Rizal Fathony,
Ashkan Rezaei,
Mohammad Ali Bashiri,
Xinhua Zhang,
Brian D. Ziebart
Abstract:
In many structured prediction problems, complex relationships between variables are compactly defined using graphical structures. The most prevalent graphical prediction methods---probabilistic graphical models and large margin methods---have their own distinct strengths but also possess significant drawbacks. Conditional random fields (CRFs) are Fisher consistent, but they do not permit integrati…
▽ More
In many structured prediction problems, complex relationships between variables are compactly defined using graphical structures. The most prevalent graphical prediction methods---probabilistic graphical models and large margin methods---have their own distinct strengths but also possess significant drawbacks. Conditional random fields (CRFs) are Fisher consistent, but they do not permit integration of customized loss metrics into their learning process. Large-margin models, such as structured support vector machines (SSVMs), have the flexibility to incorporate customized loss metrics, but lack Fisher consistency guarantees. We present adversarial graphical models (AGM), a distributionally robust approach for constructing a predictor that performs robustly for a class of data distributions defined using a graphical structure. Our approach enjoys both the flexibility of incorporating customized loss metrics into its design as well as the statistical guarantee of Fisher consistency. We present exact learning and prediction algorithms for AGM with time complexity similar to existing graphical models and show the practical benefits of our approach with experiments.
△ Less
Submitted 6 November, 2018;
originally announced November 2018.
-
PAIM: Platoon-based Autonomous Intersection Management
Authors:
Masoud Bashiri,
Hassan Jafarzadeh,
Cody Fleming
Abstract:
With the emergence of autonomous ground vehicles and the recent advancements in Intelligent Transportation Systems, Autonomous Traffic Management has garnered more and more attention. Autonomous Intersection Management (AIM), also known as Cooperative Intersection Management (CIM) is among the more challenging traffic problems that poses important questions related to safety and optimization in te…
▽ More
With the emergence of autonomous ground vehicles and the recent advancements in Intelligent Transportation Systems, Autonomous Traffic Management has garnered more and more attention. Autonomous Intersection Management (AIM), also known as Cooperative Intersection Management (CIM) is among the more challenging traffic problems that poses important questions related to safety and optimization in terms of delays, fuel consumption, emissions and reliability. Previously we introduced two stop-sign based policies for autonomous intersection management that were compatible with platoons of autonomous vehicles. These policies outperformed regular stop-sign policy both in terms of average delay per vehicle and variance in delay. This paper introduces a reservation-based policy that utilizes the cost functions from our previous work to derive optimal schedules for platoons of vehicles. The proposed policy guarantees safety by not allowing vehicles with conflicting turning movement to be in the conflict zone at the same time. Moreover, a greedy algorithm is designed to search through all possible schedules to pick the best that minimizes a cost function based on a trade-off between total delay and variance in delay. A simulator software is designed to compare the results of the proposed policy in terms of average delay per vehicle and variance in delay with that of a 4-phase traffic light.
△ Less
Submitted 18 September, 2018;
originally announced September 2018.
