Showing 1–27 of 27 results for author: Suarez, J

PufferLib: Making Reinforcement Learning Libraries and Environments Play Nice

Authors: Joseph Suarez

Abstract: You have an environment, a model, and a reinforcement learning library that are designed to work together but don't. PufferLib makes them play nice. The library provides one-line environment wrappers that eliminate common compatibility problems and fast vectorization to accelerate training. With PufferLib, you can use familiar libraries like CleanRL and SB3 to scale from classic benchmarks like At… ▽ More You have an environment, a model, and a reinforcement learning library that are designed to work together but don't. PufferLib makes them play nice. The library provides one-line environment wrappers that eliminate common compatibility problems and fast vectorization to accelerate training. With PufferLib, you can use familiar libraries like CleanRL and SB3 to scale from classic benchmarks like Atari and Procgen to complex simulators like NetHack and Neural MMO. We release pip packages and prebuilt images with dependencies for dozens of environments. All of our code is free and open-source software under the MIT license, complete with baselines, documentation, and support at pufferai.github.io. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Massively Multiagent Minigames for Training Generalist Agents

Authors: Kyoung Whan Choe, Ryan Sullivan, Joseph Suárez

Abstract: We present Meta MMO, a collection of many-agent minigames for use as a reinforcement learning benchmark. Meta MMO is built on top of Neural MMO, a massively multiagent environment that has been the subject of two previous NeurIPS competitions. Our work expands Neural MMO with several computationally efficient minigames. We explore generalization across Meta MMO by learning to play several minigame… ▽ More We present Meta MMO, a collection of many-agent minigames for use as a reinforcement learning benchmark. Meta MMO is built on top of Neural MMO, a massively multiagent environment that has been the subject of two previous NeurIPS competitions. Our work expands Neural MMO with several computationally efficient minigames. We explore generalization across Meta MMO by learning to play several minigames with a single set of weights. We release the environment, baselines, and training code under the MIT license. We hope that Meta MMO will spur additional progress on Neural MMO and, more generally, will serve as a useful benchmark for many-agent generalization. △ Less

Submitted 7 June, 2024; originally announced June 2024.

Effects and Coeffects in Call-By-Push-Value (Extended Version)

Authors: Cassia Torczon, Emmanuel Suárez Acevedo, Shubh Agrawal, Joey Velez-Ginorio, Stephanie Weirich

Abstract: Effect and coeffect tracking are a useful way to integrate many types of compile-time analysis, such as cost, liveness or dataflow, into a language's type system. However, their interactions with call-by-push-value (CBPV), a computational model useful in compilation for its isolation of effects and its ability to encompass both call-by-name and call-by-value computations, are still poorly understo… ▽ More Effect and coeffect tracking are a useful way to integrate many types of compile-time analysis, such as cost, liveness or dataflow, into a language's type system. However, their interactions with call-by-push-value (CBPV), a computational model useful in compilation for its isolation of effects and its ability to encompass both call-by-name and call-by-value computations, are still poorly understood. We present fundamental results about those interactions, in particular effect and coeffect soundness. The former asserts that our CBPV-with-effects system accurately predicts the effects that the program may trigger during execution, and the latter asserts that our CBPV-with-coeffects system accurately tracks the demands a program makes on its environment. We prove our results for a core CBPV calculus and maintain generality across a broad range of effects and coeffects, laying the groundwork for future investigation. △ Less

Submitted 20 November, 2023; originally announced November 2023.

Neural MMO 2.0: A Massively Multi-task Addition to Massively Multi-agent Learning

Authors: Joseph Suárez, Phillip Isola, Kyoung Whan Choe, David Bloomin, Hao Xiang Li, Nikhil Pinnaparaju, Nishaanth Kanna, Daniel Scott, Ryan Sullivan, Rose S. Shuman, Lucas de Alcântara, Herbie Bradley, Louis Castricato, Kirsty You, Yuhao Jiang, Qimai Li, Jiaxin Chen, Xiaolong Zhu

Abstract: Neural MMO 2.0 is a massively multi-agent environment for reinforcement learning research. The key feature of this new version is a flexible task system that allows users to define a broad range of objectives and reward signals. We challenge researchers to train agents capable of generalizing to tasks, maps, and opponents never seen during training. Neural MMO features procedurally generated maps… ▽ More Neural MMO 2.0 is a massively multi-agent environment for reinforcement learning research. The key feature of this new version is a flexible task system that allows users to define a broad range of objectives and reward signals. We challenge researchers to train agents capable of generalizing to tasks, maps, and opponents never seen during training. Neural MMO features procedurally generated maps with 128 agents in the standard setting and support for up to. Version 2.0 is a complete rewrite of its predecessor with three-fold improved performance and compatibility with CleanRL. We release the platform as free and open-source software with comprehensive documentation available at neuralmmo.github.io and an active community Discord. To spark initial research on this new platform, we are concurrently running a competition at NeurIPS 2023. △ Less

Submitted 7 November, 2023; originally announced November 2023.

The NeurIPS 2022 Neural MMO Challenge: A Massively Multiagent Competition with Specialization and Trade

Authors: Enhong Liu, Joseph Suarez, Chenhui You, Bo Wu, Bingcheng Chen, Jun Hu, Jiaxin Chen, Xiaolong Zhu, Clare Zhu, Julian Togelius, Sharada Mohanty, Weijun Hong, Rui Du, Yibing Zhang, Qinwen Wang, Xinhang Li, Zheng Yuan, Xiang Li, Yuejia Huang, Kun Zhang, Hanhui Yang, Shiqi Tang, Phillip Isola

Abstract: In this paper, we present the results of the NeurIPS-2022 Neural MMO Challenge, which attracted 500 participants and received over 1,600 submissions. Like the previous IJCAI-2022 Neural MMO Challenge, it involved agents from 16 populations surviving in procedurally generated worlds by collecting resources and defeating opponents. This year's competition runs on the latest v1.6 Neural MMO, which in… ▽ More In this paper, we present the results of the NeurIPS-2022 Neural MMO Challenge, which attracted 500 participants and received over 1,600 submissions. Like the previous IJCAI-2022 Neural MMO Challenge, it involved agents from 16 populations surviving in procedurally generated worlds by collecting resources and defeating opponents. This year's competition runs on the latest v1.6 Neural MMO, which introduces new equipment, combat, trading, and a better scoring system. These elements combine to pose additional robustness and generalization challenges not present in previous competitions. This paper summarizes the design and results of the challenge, explores the potential of this environment as a benchmark for learning methods, and presents some practical reinforcement learning training approaches for complex tasks with sparse rewards. Additionally, we have open-sourced our baselines, including environment wrappers, benchmarks, and visualization tools for future research. △ Less

Submitted 6 November, 2023; originally announced November 2023.

Reward Scale Robustness for Proximal Policy Optimization via DreamerV3 Tricks

Authors: Ryan Sullivan, Akarsh Kumar, Shengyi Huang, John P. Dickerson, Joseph Suarez

Abstract: Most reinforcement learning methods rely heavily on dense, well-normalized environment rewards. DreamerV3 recently introduced a model-based method with a number of tricks that mitigate these limitations, achieving state-of-the-art on a wide range of benchmarks with a single set of hyperparameters. This result sparked discussion about the generality of the tricks, since they appear to be applicable… ▽ More Most reinforcement learning methods rely heavily on dense, well-normalized environment rewards. DreamerV3 recently introduced a model-based method with a number of tricks that mitigate these limitations, achieving state-of-the-art on a wide range of benchmarks with a single set of hyperparameters. This result sparked discussion about the generality of the tricks, since they appear to be applicable to other reinforcement learning algorithms. Our work applies DreamerV3's tricks to PPO and is the first such empirical study outside of the original work. Surprisingly, we find that the tricks presented do not transfer as general improvements to PPO. We use a high quality PPO reference implementation and present extensive ablation studies totaling over 10,000 A100 hours on the Arcade Learning Environment and the DeepMind Control Suite. Though our experiments demonstrate that these tricks do not generally outperform PPO, we identify cases where they succeed and offer insight into the relationship between the implementation tricks. In particular, PPO with these tricks performs comparably to PPO on Atari games with reward clipping and significantly outperforms PPO without reward clipping. △ Less

Submitted 26 October, 2023; originally announced October 2023.

Comments: Accepted to NeurIPS 2023

Making Logical Relations More Relatable (Proof Pearl)

Authors: Emmanuel Suárez Acevedo, Stephanie Weirich

Abstract: Mechanical proofs by logical relations often involve tedious reasoning about substitution. In this paper, we show that this is not necessarily the case, by developing, in Agda, a proof that all simply typed lambda calculus expressions evaluate to values. A formalization of the proof is remarkably short (~40 lines of code), making for an excellent introduction to the technique of proofs by logical… ▽ More Mechanical proofs by logical relations often involve tedious reasoning about substitution. In this paper, we show that this is not necessarily the case, by developing, in Agda, a proof that all simply typed lambda calculus expressions evaluate to values. A formalization of the proof is remarkably short (~40 lines of code), making for an excellent introduction to the technique of proofs by logical relations not only on paper but also in a mechanized setting. We then show that this process extends to more sophisticated reasoning by also proving the totality of normalization by evaluation. Although these proofs are not new, we believe presenting them will empower both new and experienced programming language theorists in their use of logical relations. △ Less

Submitted 27 September, 2023; originally announced September 2023.

Comments: Submitted for publication

Benchmarking Robustness and Generalization in Multi-Agent Systems: A Case Study on Neural MMO

Authors: Yangkun Chen, Joseph Suarez, Junjie Zhang, Chenghui Yu, Bo Wu, Hanmo Chen, Hengman Zhu, Rui Du, Shanliang Qian, Shuai Liu, Weijun Hong, Jinke He, Yibing Zhang, Liang Zhao, Clare Zhu, Julian Togelius, Sharada Mohanty, Jiaxin Chen, Xiu Li, Xiaolong Zhu, Phillip Isola

Abstract: We present the results of the second Neural MMO challenge, hosted at IJCAI 2022, which received 1600+ submissions. This competition targets robustness and generalization in multi-agent systems: participants train teams of agents to complete a multi-task objective against opponents not seen during training. The competition combines relatively complex environment design with large numbers of agents… ▽ More We present the results of the second Neural MMO challenge, hosted at IJCAI 2022, which received 1600+ submissions. This competition targets robustness and generalization in multi-agent systems: participants train teams of agents to complete a multi-task objective against opponents not seen during training. The competition combines relatively complex environment design with large numbers of agents in the environment. The top submissions demonstrate strong success on this task using mostly standard reinforcement learning (RL) methods combined with domain-specific engineering. We summarize the competition design and results and suggest that, as an academic community, competitions may be a powerful approach to solving hard problems and establishing a solid benchmark for algorithms. We will open-source our benchmark including the environment wrapper, baselines, a visualization tool, and selected policies for further research. △ Less

Submitted 30 August, 2023; originally announced August 2023.

Towards Model-Size Agnostic, Compute-Free, Memorization-based Inference of Deep Learning

Authors: Davide Giacomini, Maeesha Binte Hashem, Jeremiah Suarez, Swarup Bhunia, Amit Ranjan Trivedi

Abstract: The rapid advancement of deep neural networks has significantly improved various tasks, such as image and speech recognition. However, as the complexity of these models increases, so does the computational cost and the number of parameters, making it difficult to deploy them on resource-constrained devices. This paper proposes a novel memorization-based inference (MBI) that is compute free and onl… ▽ More The rapid advancement of deep neural networks has significantly improved various tasks, such as image and speech recognition. However, as the complexity of these models increases, so does the computational cost and the number of parameters, making it difficult to deploy them on resource-constrained devices. This paper proposes a novel memorization-based inference (MBI) that is compute free and only requires lookups. Specifically, our work capitalizes on the inference mechanism of the recurrent attention model (RAM), where only a small window of input domain (glimpse) is processed in a one time step, and the outputs from multiple glimpses are combined through a hidden vector to determine the overall classification output of the problem. By leveraging the low-dimensionality of glimpse, our inference procedure stores key value pairs comprising of glimpse location, patch vector, etc. in a table. The computations are obviated during inference by utilizing the table to read out key-value pairs and performing compute-free inference by memorization. By exploiting Bayesian optimization and clustering, the necessary lookups are reduced, and accuracy is improved. We also present in-memory computing circuits to quickly look up the matching key vector to an input query. Compared to competitive compute-in-memory (CIM) approaches, MBI improves energy efficiency by almost 2.7 times than multilayer perceptions (MLP)-CIM and by almost 83 times than ResNet20-CIM for MNIST character recognition. △ Less

Submitted 14 July, 2023; originally announced July 2023.

Semi-supervised Clustering with Two Types of Background Knowledge: Fusing Pairwise Constraints and Monotonicity Constraints

Authors: Germán González-Almagro, Juan Luis Suárez, Pablo Sánchez-Bermejo, José-Ramón Cano, Salvador García

Abstract: This study addresses the problem of performing clustering in the presence of two types of background knowledge: pairwise constraints and monotonicity constraints. To achieve this, the formal framework to perform clustering under monotonicity constraints is, firstly, defined, resulting in a specific distance measure. Pairwise constraints are integrated afterwards by designing an objective function… ▽ More This study addresses the problem of performing clustering in the presence of two types of background knowledge: pairwise constraints and monotonicity constraints. To achieve this, the formal framework to perform clustering under monotonicity constraints is, firstly, defined, resulting in a specific distance measure. Pairwise constraints are integrated afterwards by designing an objective function which combines the proposed distance measure and a pairwise constraint-based penalty term, in order to fuse both types of information. This objective function can be optimized with an EM optimization scheme. The proposed method serves as the first approach to the problem it addresses, as it is the first method designed to work with the two types of background knowledge mentioned above. Our proposal is tested in a variety of benchmark datasets and in a real-world case of study. △ Less

Submitted 25 February, 2023; originally announced February 2023.

Some Optimization Solutions for Relief Distribution

Authors: Jhoirene Clemente, Jessie James Suarez, Olivia Demetria, Perry Go, Dylan Salcedo

Abstract: Humanitarian logistics remain a challenging area of application for operations research. In relief distribution, the main goal is to deliver all the supplies to those that are in need in the fastest way possible. In this paper, we present different optimization solutions for relief distribution. We present a formalization of the three main problems in the humanitarian logistics aspect of relief di… ▽ More Humanitarian logistics remain a challenging area of application for operations research. In relief distribution, the main goal is to deliver all the supplies to those that are in need in the fastest way possible. In this paper, we present different optimization solutions for relief distribution. We present a formalization of the three main problems in the humanitarian logistics aspect of relief distribution. We identify the optimal location of the distribution centers. We match the number of supplies to the number of demands for each distribution center based on the distribution of demands. We provide the assignment of tasks to delivery fleet according to the location and the road network of the region. For each delivery truck, we provide an optimal sequence of visits to pre-assigned distribution centers. △ Less

Submitted 22 April, 2022; originally announced April 2022.

ACM Class: J.7

Journal ref: In Proceedings of the Philippine Computing Science Congress 2021

Client Monitoring Software: A Monitoring Tool for Greatleaf Land Inc

Authors: Carlo H. Godoy Jr., Jerico C. Torayno, Audrey Rose Abbey C. Magtarayo, Mark Wilson J. Suarez, Armando Embile, Daven Christian O. Estopia

Abstract: Monitoring typically supports greater analysis and allows for a lot deeper data collection on a Web browser level. Analysts may usually see the use of web-based monitoring software within an entire client context when it comes to client-side monitoring, on the other hand. In the case of Greatleaf Land Inc., their monitoring is somehow traditional. Traditional means that the paper method of monitor… ▽ More Monitoring typically supports greater analysis and allows for a lot deeper data collection on a Web browser level. Analysts may usually see the use of web-based monitoring software within an entire client context when it comes to client-side monitoring, on the other hand. In the case of Greatleaf Land Inc., their monitoring is somehow traditional. Traditional means that the paper method of monitoring is being utilized. When arranging piles of paper and afterwards forgetting about it, users lose track of where the information is situated. Another notable issue is that some information situating on a written on paper isn't easily cant be detected beyond skimming, which is a process normally being used to fast track records on a flipping pages, and sometimes there's also the fact that written records situating on papers can only situate in one location at a certain time unless the custodian has given up and made copies for everything. This study will check the feasibility of a web-based client monitoring software for Greatleaf Land Inc. By using ISO 25010, the study will determine if it is beneficial for Greatleaf Land Inc. to have an online monitoring software rather than settling to the traditional method that is currently being used by their company. △ Less

Submitted 19 January, 2022; originally announced February 2022.

Comments: 8 pages, 8 Figures, 4 Tables

MSC Class: C.3 ACM Class: C.3

The Neural MMO Platform for Massively Multiagent Research

Authors: Joseph Suarez, Yilun Du, Clare Zhu, Igor Mordatch, Phillip Isola

Abstract: Neural MMO is a computationally accessible research platform that combines large agent populations, long time horizons, open-ended tasks, and modular game systems. Existing environments feature subsets of these properties, but Neural MMO is the first to combine them all. We present Neural MMO as free and open source software with active support, ongoing development, documentation, and additional t… ▽ More Neural MMO is a computationally accessible research platform that combines large agent populations, long time horizons, open-ended tasks, and modular game systems. Existing environments feature subsets of these properties, but Neural MMO is the first to combine them all. We present Neural MMO as free and open source software with active support, ongoing development, documentation, and additional training, logging, and visualization tools to help users adapt to this new setting. Initial baselines on the platform demonstrate that agents trained in large populations explore more and learn a progression of skills. We raise other more difficult problems such as many-team cooperation as open research questions which Neural MMO is well-suited to answer. Finally, we discuss current limitations of the platform, potential mitigations, and plans for continued development. △ Less

Submitted 14 October, 2021; originally announced October 2021.

A Survey on Deep Learning Techniques for Video Anomaly Detection

Authors: Jessie James P. Suarez, Prospero C. Naval Jr

Abstract: Anomaly detection in videos is a problem that has been studied for more than a decade. This area has piqued the interest of researchers due to its wide applicability. Because of this, there has been a wide array of approaches that have been proposed throughout the years and these approaches range from statistical-based approaches to machine learning-based approaches. Numerous surveys have already… ▽ More Anomaly detection in videos is a problem that has been studied for more than a decade. This area has piqued the interest of researchers due to its wide applicability. Because of this, there has been a wide array of approaches that have been proposed throughout the years and these approaches range from statistical-based approaches to machine learning-based approaches. Numerous surveys have already been conducted on this area but this paper focuses on providing an overview on the recent advances in the field of anomaly detection using Deep Learning. Deep Learning has been applied successfully in many fields of artificial intelligence such as computer vision, natural language processing and more. This survey, however, focuses on how Deep Learning has improved and provided more insights to the area of video anomaly detection. This paper provides a categorization of the different Deep Learning approaches with respect to their objectives. Additionally, it also discusses the commonly used datasets along with the common evaluation metrics. Afterwards, a discussion synthesizing all of the recent approaches is made to provide direction and possible areas for future research. △ Less

Submitted 29 September, 2020; originally announced September 2020.

A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages

Authors: Pedro Javier Ortiz Suárez, Laurent Romary, Benoît Sagot

Abstract: We use the multilingual OSCAR corpus, extracted from Common Crawl via language classification, filtering and cleaning, to train monolingual contextualized word embeddings (ELMo) for five mid-resource languages. We then compare the performance of OSCAR-based and Wikipedia-based ELMo embeddings for these languages on the part-of-speech tagging and parsing tasks. We show that, despite the noise in th… ▽ More We use the multilingual OSCAR corpus, extracted from Common Crawl via language classification, filtering and cleaning, to train monolingual contextualized word embeddings (ELMo) for five mid-resource languages. We then compare the performance of OSCAR-based and Wikipedia-based ELMo embeddings for these languages on the part-of-speech tagging and parsing tasks. We show that, despite the noise in the Common-Crawl-based OSCAR data, embeddings trained on OSCAR perform much better than monolingual embeddings trained on Wikipedia. They actually equal or improve the current state of the art in tagging and parsing for all five languages. In particular, they also improve over multilingual Wikipedia-based contextual embeddings (multilingual BERT), which almost always constitutes the previous state of the art, thereby showing that the benefit of a larger, more diverse corpus surpasses the cross-lingual benefit of multilingual embedding architectures. △ Less

Submitted 18 June, 2020; v1 submitted 11 June, 2020; originally announced June 2020.

Journal ref: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, July 2020, Online

COVIDGR dataset and COVID-SDNet methodology for predicting COVID-19 based on Chest X-Ray images

Authors: S. Tabik, A. Gómez-Ríos, J. L. Martín-Rodríguez, I. Sevillano-García, M. Rey-Area, D. Charte, E. Guirado, J. L. Suárez, J. Luengo, M. A. Valero-González, P. García-Villanova, E. Olmedo-Sánchez, F. Herrera

Abstract: Currently, Coronavirus disease (COVID-19), one of the most infectious diseases in the 21st century, is diagnosed using RT-PCR testing, CT scans and/or Chest X-Ray (CXR) images. CT (Computed Tomography) scanners and RT-PCR testing are not available in most medical centers and hence in many cases CXR images become the most time/cost effective tool for assisting clinicians in making decisions. Deep l… ▽ More Currently, Coronavirus disease (COVID-19), one of the most infectious diseases in the 21st century, is diagnosed using RT-PCR testing, CT scans and/or Chest X-Ray (CXR) images. CT (Computed Tomography) scanners and RT-PCR testing are not available in most medical centers and hence in many cases CXR images become the most time/cost effective tool for assisting clinicians in making decisions. Deep learning neural networks have a great potential for building COVID-19 triage systems and detecting COVID-19 patients, especially patients with low severity. Unfortunately, current databases do not allow building such systems as they are highly heterogeneous and biased towards severe cases. This paper is three-fold: (i) we demystify the high sensitivities achieved by most recent COVID-19 classification models, (ii) under a close collaboration with Hospital Universitario Clínico San Cecilio, Granada, Spain, we built COVIDGR-1.0, a homogeneous and balanced database that includes all levels of severity, from normal with Positive RT-PCR, Mild, Moderate to Severe. COVIDGR-1.0 contains 426 positive and 426 negative PA (PosteroAnterior) CXR views and (iii) we propose COVID Smart Data based Network (COVID-SDNet) methodology for improving the generalization capacity of COVID-classification models. Our approach reaches good and stable results with an accuracy of $97.72\% \pm 0.95 \%$, $86.90\% \pm 3.20\%$, $61.80\% \pm 5.49\%$ in severe, moderate and mild COVID-19 severity levels (Paper accepted for publication in Journal of Biomedical and Health Informatics). Our approach could help in the early detection of COVID-19. COVIDGR-1.0 along with the severity level labels are available to the scientific community through this link https://dasci.es/es/transferencia/open-data/covidgr/. △ Less

Submitted 11 November, 2020; v1 submitted 2 June, 2020; originally announced June 2020.

Comments: Paper accepted in Journal of Biomedical And Health Informatics

Establishing a New State-of-the-Art for French Named Entity Recognition

Authors: Pedro Javier Ortiz Suárez, Yoann Dupont, Benjamin Muller, Laurent Romary, Benoît Sagot

Abstract: The French TreeBank developed at the University Paris 7 is the main source of morphosyntactic and syntactic annotations for French. However, it does not include explicit information related to named entities, which are among the most useful information for several natural language processing tasks and applications. Moreover, no large-scale French corpus with named entity annotations contain refere… ▽ More The French TreeBank developed at the University Paris 7 is the main source of morphosyntactic and syntactic annotations for French. However, it does not include explicit information related to named entities, which are among the most useful information for several natural language processing tasks and applications. Moreover, no large-scale French corpus with named entity annotations contain referential information, which complement the type and the span of each mention with an indication of the entity it refers to. We have manually annotated the French TreeBank with such information, after an automatic pre-annotation step. We sketch the underlying annotation guidelines and we provide a few figures about the resulting annotations. △ Less

Submitted 27 May, 2020; originally announced May 2020.

Journal ref: LREC 2020 - 12th Language Resources and Evaluation Conference, May 2020, Marseille, France

A Physiology-Driven Computational Model for Post-Cardiac Arrest Outcome Prediction

Authors: Han B. Kim, Hieu Nguyen, Qingchu Jin, Sharmila Tamby, Tatiana Gelaf Romer, Eric Sung, Ran Liu, Joseph Greenstein, Jose I. Suarez, Christian Storm, Raimond Winslow, Robert D. Stevens

Abstract: Patients resuscitated from cardiac arrest (CA) face a high risk of neurological disability and death, however pragmatic methods are lacking for accurate and reliable prognostication. The aim of this study was to build computational models to predict post-CA outcome by leveraging high-dimensional patient data available early after admission to the intensive care unit (ICU). We hypothesized that mod… ▽ More Patients resuscitated from cardiac arrest (CA) face a high risk of neurological disability and death, however pragmatic methods are lacking for accurate and reliable prognostication. The aim of this study was to build computational models to predict post-CA outcome by leveraging high-dimensional patient data available early after admission to the intensive care unit (ICU). We hypothesized that model performance could be enhanced by integrating physiological time series (PTS) data and by training machine learning (ML) classifiers. We compared three models integrating features extracted from the electronic health records (EHR) alone, features derived from PTS collected in the first 24hrs after ICU admission (PTS24), and models integrating PTS24 and EHR. Outcomes of interest were survival and neurological outcome at ICU discharge. Combined EHR-PTS24 models had higher discrimination (area under the receiver operating characteristic curve [AUC]) than models which used either EHR or PTS24 alone, for the prediction of survival (AUC 0.85, 0.80 and 0.68 respectively) and neurological outcome (0.87, 0.83 and 0.78). The best ML classifier achieved higher discrimination than the reference logistic regression model (APACHE III) for survival (AUC 0.85 vs 0.70) and neurological outcome prediction (AUC 0.87 vs 0.75). Feature analysis revealed previously unknown factors to be associated with post-CA recovery. Results attest to the effectiveness of ML models for post-CA predictive modeling and suggest that PTS recorded in very early phase after resuscitation encode short-term outcome probabilities. △ Less

Submitted 11 February, 2020; v1 submitted 9 February, 2020; originally announced February 2020.

Comments: 51 pages, 7 figures, 4 supplementary figures

ACM Class: J.3; I.2.1; I.6.4; G.3

Journal ref: Anaesthesia Critical Care & Pain Medicine 41.1 (2022): 101015

Neural MMO v1.3: A Massively Multiagent Game Environment for Training and Evaluating Neural Networks

Authors: Joseph Suarez, Yilun Du, Igor Mordatch, Phillip Isola

Abstract: Progress in multiagent intelligence research is fundamentally limited by the number and quality of environments available for study. In recent years, simulated games have become a dominant research platform within reinforcement learning, in part due to their accessibility and interpretability. Previous works have targeted and demonstrated success on arcade, first person shooter (FPS), real-time st… ▽ More Progress in multiagent intelligence research is fundamentally limited by the number and quality of environments available for study. In recent years, simulated games have become a dominant research platform within reinforcement learning, in part due to their accessibility and interpretability. Previous works have targeted and demonstrated success on arcade, first person shooter (FPS), real-time strategy (RTS), and massive online battle arena (MOBA) games. Our work considers massively multiplayer online role-playing games (MMORPGs or MMOs), which capture several complexities of real-world learning that are not well modeled by any other game genre. We present Neural MMO, a massively multiagent game environment inspired by MMOs and discuss our progress on two more general challenges in multiagent systems engineering for AI research: distributed infrastructure and game IO. We further demonstrate that standard policy gradient methods and simple baseline models can learn interesting emergent exploration and specialization behaviors in this setting. △ Less

Submitted 16 April, 2020; v1 submitted 31 January, 2020; originally announced January 2020.

CamemBERT: a Tasty French Language Model

Authors: Louis Martin, Benjamin Muller, Pedro Javier Ortiz Suárez, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah, Benoît Sagot

Abstract: Pretrained language models are now ubiquitous in Natural Language Processing. Despite their success, most available models have either been trained on English data or on the concatenation of data in multiple languages. This makes practical use of such models --in all languages except English-- very limited. In this paper, we investigate the feasibility of training monolingual Transformer-based lan… ▽ More Pretrained language models are now ubiquitous in Natural Language Processing. Despite their success, most available models have either been trained on English data or on the concatenation of data in multiple languages. This makes practical use of such models --in all languages except English-- very limited. In this paper, we investigate the feasibility of training monolingual Transformer-based language models for other languages, taking French as an example and evaluating our language models on part-of-speech tagging, dependency parsing, named entity recognition and natural language inference tasks. We show that the use of web crawled data is preferable to the use of Wikipedia data. More surprisingly, we show that a relatively small web crawled dataset (4GB) leads to results that are as good as those obtained using larger datasets (130+GB). Our best performing model CamemBERT reaches or improves the state of the art in all four downstream tasks. △ Less

Submitted 21 May, 2020; v1 submitted 10 November, 2019; originally announced November 2019.

Comments: ACL 2020 long paper. Web site: https://camembert-model.fr

Journal ref: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, July 2020, Online

GAN You Do the GAN GAN?

Authors: Joseph Suarez

Abstract: Generative Adversarial Networks (GANs) have become a dominant class of generative models. In recent years, GAN variants have yielded especially impressive results in the synthesis of a variety of forms of data. Examples include compelling natural and artistic images, textures, musical sequences, and 3D object files. However, one obvious synthesis candidate is missing. In this work, we answer one o… ▽ More Generative Adversarial Networks (GANs) have become a dominant class of generative models. In recent years, GAN variants have yielded especially impressive results in the synthesis of a variety of forms of data. Examples include compelling natural and artistic images, textures, musical sequences, and 3D object files. However, one obvious synthesis candidate is missing. In this work, we answer one of deep learning's most pressing questions: GAN you do the GAN GAN? That is, is it possible to train a GAN to model a distribution of GANs? We release the full source code for this project under the MIT license. △ Less

Submitted 1 April, 2019; originally announced April 2019.

Comments: 3 pages

Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents

Authors: Joseph Suarez, Yilun Du, Phillip Isola, Igor Mordatch

Abstract: The emergence of complex life on Earth is often attributed to the arms race that ensued from a huge number of organisms all competing for finite resources. We present an artificial intelligence research environment, inspired by the human game genre of MMORPGs (Massively Multiplayer Online Role-Playing Games, a.k.a. MMOs), that aims to simulate this setting in microcosm. As with MMORPGs and the rea… ▽ More The emergence of complex life on Earth is often attributed to the arms race that ensued from a huge number of organisms all competing for finite resources. We present an artificial intelligence research environment, inspired by the human game genre of MMORPGs (Massively Multiplayer Online Role-Playing Games, a.k.a. MMOs), that aims to simulate this setting in microcosm. As with MMORPGs and the real world alike, our environment is persistent and supports a large and variable number of agents. Our environment is well suited to the study of large-scale multiagent interaction: it requires that agents learn robust combat and navigation policies in the presence of large populations attempting to do the same. Baseline experiments reveal that population size magnifies and incentivizes the development of skillful behaviors and results in agents that outcompete agents trained in smaller populations. We further show that the policies of agents with unshared weights naturally diverge to fill different niches in order to avoid competition. △ Less

Submitted 2 March, 2019; originally announced March 2019.

DDRprog: A CLEVR Differentiable Dynamic Reasoning Programmer

Authors: Joseph Suarez, Justin Johnson, Fei-Fei Li

Abstract: We present a novel Dynamic Differentiable Reasoning (DDR) framework for jointly learning branching programs and the functions composing them; this resolves a significant nondifferentiability inhibiting recent dynamic architectures. We apply our framework to two settings in two highly compact and data efficient architectures: DDRprog for CLEVR Visual Question Answering and DDRstack for reverse Poli… ▽ More We present a novel Dynamic Differentiable Reasoning (DDR) framework for jointly learning branching programs and the functions composing them; this resolves a significant nondifferentiability inhibiting recent dynamic architectures. We apply our framework to two settings in two highly compact and data efficient architectures: DDRprog for CLEVR Visual Question Answering and DDRstack for reverse Polish notation expression evaluation. DDRprog uses a recurrent controller to jointly predict and execute modular neural programs that directly correspond to the underlying question logic; it explicitly forks subprocesses to handle logical branching. By effectively leveraging additional structural supervision, we achieve a large improvement over previous approaches in subtask consistency and a small improvement in overall accuracy. We further demonstrate the benefits of structural supervision in the RPN setting: the inclusion of a stack assumption in DDRstack allows our approach to generalize to long expressions where an LSTM fails the task. △ Less

Submitted 30 March, 2018; originally announced March 2018.

Effective Approaches to Batch Parallelization for Dynamic Neural Network Architectures

Authors: Joseph Suarez, Clare Zhu

Abstract: We present a simple dynamic batching approach applicable to a large class of dynamic architectures that consistently yields speedups of over 10x. We provide performance bounds when the architecture is not known a priori and a stronger bound in the special case where the architecture is a predetermined balanced tree. We evaluate our approach on Johnson et al.'s recent visual question answering (VQA… ▽ More We present a simple dynamic batching approach applicable to a large class of dynamic architectures that consistently yields speedups of over 10x. We provide performance bounds when the architecture is not known a priori and a stronger bound in the special case where the architecture is a predetermined balanced tree. We evaluate our approach on Johnson et al.'s recent visual question answering (VQA) result of his CLEVR dataset by Inferring and Executing Programs (IEP). We also evaluate on sparsely gated mixture of experts layers and achieve speedups of up to 1000x over the naive implementation. △ Less

Submitted 8 July, 2017; originally announced July 2017.

Comments: Code at https://github.com/jsuarez5341/Efficient-Dynamic-Batching

The Life of Lazarillo de Tormes and of His Machine Learning Adversities

Authors: Javier de la Rosa, Juan-Luis Suárez

Abstract: Summit work of the Spanish Golden Age and forefather of the so-called picaresque novel, The Life of Lazarillo de Tormes and of His Fortunes and Adversities still remains an anonymous text. Although distinguished scholars have tried to attribute it to different authors based on a variety of criteria, a consensus has yet to be reached. The list of candidates is long and not all of them enjoy the sam… ▽ More Summit work of the Spanish Golden Age and forefather of the so-called picaresque novel, The Life of Lazarillo de Tormes and of His Fortunes and Adversities still remains an anonymous text. Although distinguished scholars have tried to attribute it to different authors based on a variety of criteria, a consensus has yet to be reached. The list of candidates is long and not all of them enjoy the same support within the scholarly community. Analyzing their works from a data-driven perspective and applying machine learning techniques for style and text fingerprinting, we shed light on the authorship of the Lazarillo. As in a state-of-the-art survey, we discuss the methods used and how they perform in our specific case. According to our methodology, the most likely author seems to be Juan Arce de Otálora, closely followed by Alfonso de Valdés. The method states that not certain attribution can be made with the given corpus. △ Less

Submitted 16 November, 2016; originally announced November 2016.

Comments: 66 pages, 11 figures

Journal ref: Lemir: Revista de Literatura Española Medieval y del Renacimiento, 20 (2016)

Exploring the Use of RPAs as 5G Points of Presence

Authors: Javier Suarez, Ivan Vidal, Jaime Garcia-Reinoso, Francisco Valera, Arturo Azcorra

Abstract: This paper presents an early exploration and preliminary results on the use of Remotely Piloted Aircrafts (RPA) as 5G points of presence. The use of RPAs in the 5G arena would enable a cost-effective deployment of functions over mobile nodes that could be integrated on demand into the programmable and unified 5G infrastructure, enhancing the capacity of the network to flexibly adapt to the particu… ▽ More This paper presents an early exploration and preliminary results on the use of Remotely Piloted Aircrafts (RPA) as 5G points of presence. The use of RPAs in the 5G arena would enable a cost-effective deployment of functions over mobile nodes that could be integrated on demand into the programmable and unified 5G infrastructure, enhancing the capacity of the network to flexibly adapt to the particular service requirements in a geographical area. As a first step, we evaluate the feasibility and the cost, in terms of energy consumption, of using virtualisation techniques over resource-constrained aerial vehicle platforms, as a fundamental software technology in the evolution towards 5G. We complement this evaluation presenting a proof-of-concept that considers the use of these platforms to enable real-time 5G communications in emergency cases. △ Less

Submitted 24 March, 2016; originally announced March 2016.

Comments: Under review, EuCNC 2016

ICE: Enabling Non-Experts to Build Models Interactively for Large-Scale Lopsided Problems

Authors: Patrice Simard, David Chickering, Aparna Lakshmiratan, Denis Charles, Leon Bottou, Carlos Garcia Jurado Suarez, David Grangier, Saleema Amershi, Johan Verwey, Jina Suh

Abstract: Quick interaction between a human teacher and a learning machine presents numerous benefits and challenges when working with web-scale data. The human teacher guides the machine towards accomplishing the task of interest. The learning machine leverages big data to find examples that maximize the training value of its interaction with the teacher. When the teacher is restricted to labeling examples… ▽ More Quick interaction between a human teacher and a learning machine presents numerous benefits and challenges when working with web-scale data. The human teacher guides the machine towards accomplishing the task of interest. The learning machine leverages big data to find examples that maximize the training value of its interaction with the teacher. When the teacher is restricted to labeling examples selected by the machine, this problem is an instance of active learning. When the teacher can provide additional information to the machine (e.g., suggestions on what examples or predictive features should be used) as the learning task progresses, then the problem becomes one of interactive learning. To accommodate the two-way communication channel needed for efficient interactive learning, the teacher and the machine need an environment that supports an interaction language. The machine can access, process, and summarize more examples than the teacher can see in a lifetime. Based on the machine's output, the teacher can revise the definition of the task or make it more precise. Both the teacher and the machine continuously learn and benefit from the interaction. We have built a platform to (1) produce valuable and deployable models and (2) support research on both the machine learning and user interface challenges of the interactive learning problem. The platform relies on a dedicated, low-latency, distributed, in-memory architecture that allows us to construct web-scale learning machines with quick interaction speed. The purpose of this paper is to describe this architecture and demonstrate how it supports our research efforts. Preliminary results are presented as illustrations of the architecture but are not the primary focus of the paper. △ Less

Submitted 16 September, 2014; originally announced September 2014.