-
Specialty-Oriented Generalist Medical AI for Chest CT Screening
Authors:
Chuang Niu,
Qing Lyu,
Christopher D. Carothers,
Parisa Kaviani,
Josh Tan,
Pingkun Yan,
Mannudeep K. Kalra,
Christopher T. Whitlow,
Ge Wang
Abstract:
Modern medical records include a vast amount of multimodal free text clinical data and imaging data from radiology, cardiology, and digital pathology. Fully mining such big data requires multitasking; otherwise, occult but important aspects may be overlooked, adversely affecting clinical management and population healthcare. Despite remarkable successes of AI in individual tasks with single-modal…
▽ More
Modern medical records include a vast amount of multimodal free text clinical data and imaging data from radiology, cardiology, and digital pathology. Fully mining such big data requires multitasking; otherwise, occult but important aspects may be overlooked, adversely affecting clinical management and population healthcare. Despite remarkable successes of AI in individual tasks with single-modal data, the progress in developing generalist medical AI remains relatively slow to combine multimodal data for multitasks because of the dual challenges of data curation and model architecture. The data challenge involves querying and curating multimodal structured and unstructured text, alphanumeric, and especially 3D tomographic scans on an individual patient level for real-time decisions and on a scale to estimate population health statistics. The model challenge demands a scalable and adaptable network architecture to integrate multimodal datasets for diverse clinical tasks. Here we propose the first-of-its-kind medical multimodal-multitask foundation model (M3FM) with application in lung cancer screening and related tasks. After we curated a comprehensive multimodal multitask dataset consisting of 49 clinical data types including 163,725 chest CT series and 17 medical tasks involved in LCS, we develop a multimodal question-answering framework as a unified training and inference strategy to synergize multimodal information and perform multiple tasks via free-text prompting. M3FM consistently outperforms the state-of-the-art single-modal task-specific models, identifies multimodal data elements informative for clinical tasks and flexibly adapts to new tasks with a small out-of-distribution dataset. As a specialty-oriented generalist medical AI model, M3FM paves the way for similar breakthroughs in other areas of medicine, closing the gap between specialists and the generalist.
△ Less
Submitted 24 April, 2024; v1 submitted 3 April, 2023;
originally announced April 2023.
-
Unbiased Deterministic Total Ordering of Parallel Simulations with Simultaneous Events
Authors:
Neil McGlohon,
Christopher D. Carothers
Abstract:
In the area of discrete event simulation (DES), event simultaneity occurs when any two events are scheduled to happen at the same point in simulated time. Simulation determinism is the expectation that the same semantically configured simulation will be guaranteed to repeatedly reproduce identical results. Since events in DES are the sole mechanism for state change, ensuring consistent real-time e…
▽ More
In the area of discrete event simulation (DES), event simultaneity occurs when any two events are scheduled to happen at the same point in simulated time. Simulation determinism is the expectation that the same semantically configured simulation will be guaranteed to repeatedly reproduce identical results. Since events in DES are the sole mechanism for state change, ensuring consistent real-time event processing order is crucial to maintaining determinism. This is synonymous with finding a consistent total ordering of events.
In this work, we extend the concept of virtual time to utilize an arbitrary-length series of tie-breaking values to preserve determinism in parallel, optimistically executed simulations without imposing additional bias influencing the ordering of otherwise incomparable events. Furthermore, by changing the core pseudo-random number generator seed at initialization, different orderings of events incomparable by standard virtual time can be observed, allowing for fair probing of other potential simulation outcomes. We implement and evaluate this extended definition of virtual time in the Rensselaer Optimistic Simulation System (ROSS) with three simulation models and discuss the importance of deterministic event ordering given the existence of event ties.
△ Less
Submitted 30 April, 2021;
originally announced May 2021.
-
Training Deep Neural Networks with Constrained Learning Parameters
Authors:
Prasanna Date,
Christopher D. Carothers,
John E. Mitchell,
James A. Hendler,
Malik Magdon-Ismail
Abstract:
Today's deep learning models are primarily trained on CPUs and GPUs. Although these models tend to have low error, they consume high power and utilize large amount of memory owing to double precision floating point learning parameters. Beyond the Moore's law, a significant portion of deep learning tasks would run on edge computing systems, which will form an indispensable part of the entire comput…
▽ More
Today's deep learning models are primarily trained on CPUs and GPUs. Although these models tend to have low error, they consume high power and utilize large amount of memory owing to double precision floating point learning parameters. Beyond the Moore's law, a significant portion of deep learning tasks would run on edge computing systems, which will form an indispensable part of the entire computation fabric. Subsequently, training deep learning models for such systems will have to be tailored and adopted to generate models that have the following desirable characteristics: low error, low memory, and low power. We believe that deep neural networks (DNNs), where learning parameters are constrained to have a set of finite discrete values, running on neuromorphic computing systems would be instrumental for intelligent edge computing systems having these desirable characteristics. To this extent, we propose the Combinatorial Neural Network Training Algorithm (CoNNTrA), that leverages a coordinate gradient descent-based approach for training deep learning models with finite discrete learning parameters. Next, we elaborate on the theoretical underpinnings and evaluate the computational complexity of CoNNTrA. As a proof of concept, we use CoNNTrA to train deep learning models with ternary learning parameters on the MNIST, Iris and ImageNet data sets and compare their performance to the same models trained using Backpropagation. We use following performance metrics for the comparison: (i) Training error; (ii) Validation error; (iii) Memory usage; and (iv) Training time. Our results indicate that CoNNTrA models use 32x less memory and have errors at par with the Backpropagation models.
△ Less
Submitted 1 September, 2020;
originally announced September 2020.
-
A Visual Analytics Framework for Reviewing Streaming Performance Data
Authors:
Suraj P. Kesavan,
Takanori Fujiwara,
Jianping Kelvin Li,
Caitlin Ross,
Misbah Mubarak,
Christopher D. Carothers,
Robert B. Ross,
Kwan-Liu Ma
Abstract:
Understanding and tuning the performance of extreme-scale parallel computing systems demands a streaming approach due to the computational cost of applying offline algorithms to vast amounts of performance log data. Analyzing large streaming data is challenging because the rate of receiving data and limited time to comprehend data make it difficult for the analysts to sufficiently examine the data…
▽ More
Understanding and tuning the performance of extreme-scale parallel computing systems demands a streaming approach due to the computational cost of applying offline algorithms to vast amounts of performance log data. Analyzing large streaming data is challenging because the rate of receiving data and limited time to comprehend data make it difficult for the analysts to sufficiently examine the data without missing important changes or patterns. To support streaming data analysis, we introduce a visual analytic framework comprising of three modules: data management, analysis, and interactive visualization. The data management module collects various computing and communication performance metrics from the monitored system using streaming data processing techniques and feeds the data to the other two modules. The analysis module automatically identifies important changes and patterns at the required latency. In particular, we introduce a set of online and progressive analysis methods for not only controlling the computational costs but also helping analysts better follow the critical aspects of the analysis results. Finally, the interactive visualization module provides the analysts with a coherent view of the changes and patterns in the continuously captured performance data. Through a multi-faceted case study on performance analysis of parallel discrete-event simulation, we demonstrate the effectiveness of our framework for identifying bottlenecks and locating outliers.
△ Less
Submitted 25 January, 2020;
originally announced January 2020.
