Showing 1–8 of 8 results for author: Edwards, R

Estimating Classification Confidence Using Kernel Densities

Authors: Peter Salamon, David Salamon, V. Adrian Cantu, Michelle An, Tyler Perry, Robert A. Edwards, Anca M. Segall

Abstract: This paper investigates the post-hoc calibration of confidence for "exploratory" machine learning classification problems. The difficulty in these problems stems from the continuing desire to push the boundaries of which categories have enough examples to generalize from when curating datasets, and confusion regarding the validity of those categories. We argue that for such problems the "one-versu… ▽ More This paper investigates the post-hoc calibration of confidence for "exploratory" machine learning classification problems. The difficulty in these problems stems from the continuing desire to push the boundaries of which categories have enough examples to generalize from when curating datasets, and confusion regarding the validity of those categories. We argue that for such problems the "one-versus-all" approach (top-label calibration) must be used rather than the "calibrate-the-full-response-matrix" approach advocated elsewhere in the literature. We introduce and test four new algorithms designed to handle the idiosyncrasies of category-specific confidence estimation. Chief among these methods is the use of kernel density ratios for confidence calibration including a novel, bulletproof algorithm for choosing the bandwidth. We test our claims and explore the limits of calibration on a bioinformatics application (PhANNs) as well as the classic MNIST benchmark. Finally, our analysis argues that post-hoc calibration should always be performed, should be based only on the test dataset, and should be sanity-checked visually. △ Less

Submitted 14 September, 2022; v1 submitted 13 July, 2022; originally announced July 2022.

The Rational Selection of Goal Operations and the Integration ofSearch Strategies with Goal-Driven Autonomy

Authors: Sravya Kondrakunta, Venkatsampath Raja Gogineni, Michael T. Cox, Demetris Coleman, Xiaobao Tan, Tony Lin, Mengxue Hou, Fumin Zhang, Frank McQuarrie, Catherine R. Edwards

Abstract: Intelligent physical systems as embodied cognitive systems must perform high-level reasoning while concurrently managing an underlying control architecture. The link between cognition and control must manage the problem of converting continuous values from the real world to symbolic representations (and back). To generate effective behaviors, reasoning must include a capacity to replan, acquire an… ▽ More Intelligent physical systems as embodied cognitive systems must perform high-level reasoning while concurrently managing an underlying control architecture. The link between cognition and control must manage the problem of converting continuous values from the real world to symbolic representations (and back). To generate effective behaviors, reasoning must include a capacity to replan, acquire and update new information, detect and respond to anomalies, and perform various operations on system goals. But, these processes are not independent and need further exploration. This paper examines an agent's choices when multiple goal operations co-occur and interact, and it establishes a method of choosing between them. We demonstrate the benefits and discuss the trade offs involved with this and show positive results in a dynamic marine search task. △ Less

Submitted 21 January, 2022; originally announced January 2022.

Comments: Presented at The Ninth Advances in Cognitive Systems (ACS) Conference 2021 (arXiv:2201.06134)

Report number: Report-no: ACS2021/08

Recommendations to clarify NASA open source requirements

Authors: John D. Haiducek, Thom R. Edwards, Wade Duvall, Sarah R. Cannon, Kai Germaschewski, Jason E. Kooi

Abstract: The software community has specific definitions for terms such as "open source software," "free software," and "permissive license," but scientists proposing software development efforts to NASA are not always knowledgeable about these definitions. Misunderstandings about the meaning of these terms can result in problems of fairness with solicitations, because scientists who interpret the terms di… ▽ More The software community has specific definitions for terms such as "open source software," "free software," and "permissive license," but scientists proposing software development efforts to NASA are not always knowledgeable about these definitions. Misunderstandings about the meaning of these terms can result in problems of fairness with solicitations, because scientists who interpret the terms differently than NASA intends may either needlessly limit the scope of their proposed work, or unwittingly propose work that does not comply with software licensing requirements. It is therefore recommended that NASA adopt definitions of the above terms that are in line with software community usage, that these definitions be communicated as part of solicitations to ensure a common understanding, and that proposals be required to identify what software licenses the proposers expect to use. △ Less

Submitted 3 November, 2021; originally announced November 2021.

Alexa, Play Fetch! A Review of Alexa Skills for Pets

Authors: Justin Edwards, Orla Cooney, Rachel Edwards

Abstract: Alexa Skills are used for a variety of daily routines and purposes, but little research has focused on a key part of many people's daily lives: their pets. We present a systematic review categorizing the purposes of 88 Alexa Skills aimed at pets and pet owners and introduce a veterinary perspective to assess their benefits and risks. We present 8 themes of the purposes for Skills aimed at pets and… ▽ More Alexa Skills are used for a variety of daily routines and purposes, but little research has focused on a key part of many people's daily lives: their pets. We present a systematic review categorizing the purposes of 88 Alexa Skills aimed at pets and pet owners and introduce a veterinary perspective to assess their benefits and risks. We present 8 themes of the purposes for Skills aimed at pets and their owners: Calming, Animal Audience, Smart Device, Tracking, Training and Health, Translator, Entertainment/Trivia, and Other - Human Audience. Broadly, we find that these purposes mirror the purposes people have for using Alexa overall, and they largely are supported by veterinary evidence, though caution must be used when Skills relate to animal health. More collaboration between Conversational Agent researchers and animal scientists is called for to better understand the efficacy of using Alexa with pets. △ Less

Submitted 31 October, 2021; originally announced November 2021.

Comments: Accepted, Animal Computer Interaction 2021 (ACI '21)

The application of adaptive minimum match k-nearest neighbors to identify at-risk students in health professions education

Authors: Anshul Kumar, Taylor DiJohnson, Roger Edwards, Lisa Walker

Abstract: Purpose: When a learner fails to reach a milestone, educators often wonder if there had been any warning signs that could have allowed them to intervene sooner. Machine learning can predict which students are at risk of failing a high-stakes certification exam. If predictions can be made well in advance of the exam, then educators can meaningfully intervene before students take the exam to reduce… ▽ More Purpose: When a learner fails to reach a milestone, educators often wonder if there had been any warning signs that could have allowed them to intervene sooner. Machine learning can predict which students are at risk of failing a high-stakes certification exam. If predictions can be made well in advance of the exam, then educators can meaningfully intervene before students take the exam to reduce the chances of a failing score. Methods: Using already-collected, first-year student assessment data from five cohorts in a Master of Physician Assistant Studies program, the authors implement an "adaptive minimum match" version of the k-nearest neighbors algorithm (AMMKNN), using changing numbers of neighbors to predict each student's future exam scores on the Physician Assistant National Certifying Examination (PANCE). Validation occurred in two ways: Leave-one-out cross-validation (LOOCV) and evaluating the predictions in a new cohort. Results: AMMKNN achieved an accuracy of 93% in LOOCV. AMMKNN generates a predicted PANCE score for each student, one year before they are scheduled to take the exam. Students can then be classified into extra support, optional extra support, or no extra support groups. The educator then has one year to provide the appropriate customized support to each category of student. Conclusions: Predictive analytics can identify at-risk students, so they can receive additional support or remediation when preparing for high-stakes certification exams. Educators can use the included methods and code to generate predicted test outcomes for students. The authors recommend that educators use this or similar predictive methods responsibly and transparently, as one of many tools used to support students. △ Less

Submitted 9 August, 2022; v1 submitted 4 August, 2021; originally announced August 2021.

Journal ref: JPAE 34(3):p 171-177, September 2023

92c/MFlops/s, Ultra-Large-Scale Neural-Network Training on a PIII Cluster

Authors: Douglas Aberdeen, Jonathan Baxter, Robert Edwards

Abstract: Artificial neural networks with millions of adjustable parameters and a similar number of training examples are a potential solution for difficult, large-scale pattern recognition problems in areas such as speech and face recognition, classification of large volumes of web data, and finance. The bottleneck is that neural network training involves iterative gradient descent and is extremely computa… ▽ More Artificial neural networks with millions of adjustable parameters and a similar number of training examples are a potential solution for difficult, large-scale pattern recognition problems in areas such as speech and face recognition, classification of large volumes of web data, and finance. The bottleneck is that neural network training involves iterative gradient descent and is extremely computationally intensive. In this paper we present a technique for distributed training of Ultra Large Scale Neural Networks (ULSNN) on Bunyip, a Linux-based cluster of 196 Pentium III processors. To illustrate ULSNN training we describe an experiment in which a neural network with 1.73 million adjustable parameters was trained to recognize machine-printed Japanese characters from a database containing 9 million training patterns. The training runs with a average performance of 163.3 GFlops/s (single precision). With a machine cost of \$150,913, this yields a price/performance ratio of 92.4c/MFlops/s (single precision). For comparison purposes, training using double precision and the ATLAS DGEMM produces a sustained performance of 70 MFlops/s or \$2.16 / MFlop/s (double precision). △ Less

Submitted 12 November, 2019; originally announced November 2019.

Comments: SC '00: Proceedings of the 2000 ACM/IEEE Conference on Supercomputing

Journal ref: ACM/IEEE SC 2000 Conference (SC00)

A Framework for Lattice QCD Calculations on GPUs

Authors: F. T. Winter, M. A. Clark, R. G. Edwards, B. Joó

Abstract: Computing platforms equipped with accelerators like GPUs have proven to provide great computational power. However, exploiting such platforms for existing scientific applications is not a trivial task. Current GPU programming frameworks such as CUDA C/C++ require low-level programming from the developer in order to achieve high performance code. As a result porting of applications to GPUs is typic… ▽ More Computing platforms equipped with accelerators like GPUs have proven to provide great computational power. However, exploiting such platforms for existing scientific applications is not a trivial task. Current GPU programming frameworks such as CUDA C/C++ require low-level programming from the developer in order to achieve high performance code. As a result porting of applications to GPUs is typically limited to time-dominant algorithms and routines, leaving the remainder not accelerated which can open a serious Amdahl's law issue. The lattice QCD application Chroma allows to explore a different porting strategy. The layered structure of the software architecture logically separates the data-parallel from the application layer. The QCD Data-Parallel software layer provides data types and expressions with stencil-like operations suitable for lattice field theory and Chroma implements algorithms in terms of this high-level interface. Thus by porting the low-level layer one can effectively move the whole application in one swing to a different platform. The QDP-JIT/PTX library, the reimplementation of the low-level layer, provides a framework for lattice QCD calculations for the CUDA architecture. The complete software interface is supported and thus applications can be run unaltered on GPU-based parallel computers. This reimplementation was possible due to the availability of a JIT compiler (part of the NVIDIA Linux kernel driver) which translates an assembly-like language (PTX) to GPU code. The expression template technique is used to build PTX code generators and a software cache manages the GPU memory. This reimplementation allows us to deploy an efficient implementation of the full gauge-generation program with dynamical fermions on large-scale GPU-based machines such as Titan and Blue Waters which accelerates the algorithm by more than an order of magnitude. △ Less

Submitted 25 August, 2014; originally announced August 2014.

Comments: 10 pages, 6 figures, as published in the proceedings of IPDPS '14

Export Behaviour Modeling Using EvoNF Approach

Authors: Ron Edwards, Ajith Abraham, Sonja Petrovic-Lazarevic

Abstract: The academic literature suggests that the extent of exporting by multinational corporation subsidiaries (MCS) depends on their product manufactured, resources, tax protection, customers and markets, involvement strategy, financial independence and suppliers' relationship with a multinational corporation (MNC). The aim of this paper is to model the complex export pattern behaviour using a Takagi-… ▽ More The academic literature suggests that the extent of exporting by multinational corporation subsidiaries (MCS) depends on their product manufactured, resources, tax protection, customers and markets, involvement strategy, financial independence and suppliers' relationship with a multinational corporation (MNC). The aim of this paper is to model the complex export pattern behaviour using a Takagi-Sugeno fuzzy inference system in order to determine the actual volume of MCS export output (sales exported). The proposed fuzzy inference system is optimised by using neural network learning and evolutionary computation. Empirical results clearly show that the proposed approach could model the export behaviour reasonable well compared to a direct neural network approach. △ Less

Submitted 15 May, 2004; originally announced May 2004.

ACM Class: I.2.0

Journal ref: The International Conference on Computational Science 2003 (ICCS 2003), Springer Verlag, Lecture Notes in Computer Science Volume 2660, Sloot P.M.A. et al (Eds.), pp. 169-178, 2003