-
BlackJAX: Composable Bayesian inference in JAX
Authors:
Alberto Cabezas,
Adrien Corenflos,
Junpeng Lao,
Rémi Louf,
Antoine Carnec,
Kaustubh Chaudhari,
Reuben Cohn-Gordon,
Jeremie Coullon,
Wei Deng,
Sam Duffield,
Gerardo Durán-Martín,
Marcin Elantkowski,
Dan Foreman-Mackey,
Michele Gregori,
Carlos Iguaran,
Ravin Kumar,
Martin Lysy,
Kevin Murphy,
Juan Camilo Orduz,
Karm Patel,
Xi Wang,
Rob Zinkov
Abstract:
BlackJAX is a library implementing sampling and variational inference algorithms commonly used in Bayesian computation. It is designed for ease of use, speed, and modularity by taking a functional approach to the algorithms' implementation. BlackJAX is written in Python, using JAX to compile and run NumpPy-like samplers and variational methods on CPUs, GPUs, and TPUs. The library integrates well w…
▽ More
BlackJAX is a library implementing sampling and variational inference algorithms commonly used in Bayesian computation. It is designed for ease of use, speed, and modularity by taking a functional approach to the algorithms' implementation. BlackJAX is written in Python, using JAX to compile and run NumpPy-like samplers and variational methods on CPUs, GPUs, and TPUs. The library integrates well with probabilistic programming languages by working directly with the (un-normalized) target log density function. BlackJAX is intended as a collection of low-level, composable implementations of basic statistical 'atoms' that can be combined to perform well-defined Bayesian inference, but also provides high-level routines for ease of use. It is designed for users who need cutting-edge methods, researchers who want to create complex sampling methods, and people who want to learn how these work.
△ Less
Submitted 22 February, 2024; v1 submitted 16 February, 2024;
originally announced February 2024.
-
Efficient Guided Generation for Large Language Models
Authors:
Brandon T. Willard,
Rémi Louf
Abstract:
In this article we show how the problem of neural text generation can be constructively reformulated in terms of transitions between the states of a finite-state machine. This framework leads to an efficient approach to guiding text generation with regular expressions and context-free grammars by allowing the construction of an index over a language model's vocabulary. The approach is model agnost…
▽ More
In this article we show how the problem of neural text generation can be constructively reformulated in terms of transitions between the states of a finite-state machine. This framework leads to an efficient approach to guiding text generation with regular expressions and context-free grammars by allowing the construction of an index over a language model's vocabulary. The approach is model agnostic, allows one to enforce domain-specific knowledge and constraints, and enables the construction of reliable interfaces by guaranteeing the structure of the generated text. It adds little overhead to the token sequence generation process and significantly outperforms existing solutions. An implementation is provided in the open source Python library Outlines
△ Less
Submitted 19 August, 2023; v1 submitted 18 July, 2023;
originally announced July 2023.
-
HuggingFace's Transformers: State-of-the-art Natural Language Processing
Authors:
Thomas Wolf,
Lysandre Debut,
Victor Sanh,
Julien Chaumond,
Clement Delangue,
Anthony Moi,
Pierric Cistac,
Tim Rault,
Rémi Louf,
Morgan Funtowicz,
Joe Davison,
Sam Shleifer,
Patrick von Platen,
Clara Ma,
Yacine Jernite,
Julien Plu,
Canwen Xu,
Teven Le Scao,
Sylvain Gugger,
Mariama Drame,
Quentin Lhoest,
Alexander M. Rush
Abstract:
Recent progress in natural language processing has been driven by advances in both model architecture and model pretraining. Transformer architectures have facilitated building higher-capacity models and pretraining has made it possible to effectively utilize this capacity for a wide variety of tasks. \textit{Transformers} is an open-source library with the goal of opening up these advances to the…
▽ More
Recent progress in natural language processing has been driven by advances in both model architecture and model pretraining. Transformer architectures have facilitated building higher-capacity models and pretraining has made it possible to effectively utilize this capacity for a wide variety of tasks. \textit{Transformers} is an open-source library with the goal of opening up these advances to the wider machine learning community. The library consists of carefully engineered state-of-the art Transformer architectures under a unified API. Backing this library is a curated collection of pretrained models made by and available for the community. \textit{Transformers} is designed to be extensible by researchers, simple for practitioners, and fast and robust in industrial deployments. The library is available at \url{https://github.com/huggingface/transformers}.
△ Less
Submitted 13 July, 2020; v1 submitted 8 October, 2019;
originally announced October 2019.
-
Emergence of hierarchy in cost driven growth of spatial networks
Authors:
Rémi Louf,
Pablo Jensen,
Marc Barthelemy
Abstract:
One of the most important features of spatial networks such as transportation networks, power grids, Internet, neural networks, is the existence of a cost associated with the length of links. Such a cost has a profound influence on the global structure of these networks which usually display a hierarchical spatial organization. The link between local constraints and large-scale structure is howeve…
▽ More
One of the most important features of spatial networks such as transportation networks, power grids, Internet, neural networks, is the existence of a cost associated with the length of links. Such a cost has a profound influence on the global structure of these networks which usually display a hierarchical spatial organization. The link between local constraints and large-scale structure is however not elucidated and we introduce here a generic model for the growth of spatial networks based on the general concept of cost benefit analysis. This model depends essentially on one single scale and produces a family of networks which range from the star-graph to the minimum spanning tree and which are characterised by a continuously varying exponent. We show that spatial hierarchy emerges naturally, with structures composed of various hubs controlling geographically separated service areas, and appears as a large-scale consequence of local cost-benefit considerations. Our model thus provides the first building blocks for a better understanding of the evolution of spatial networks and their properties. We also find that, surprisingly, the average detour is minimal in the intermediate regime, as a result of a large diversity in link lengths. Finally, we estimate the important parameters for various world railway networks and find that --remarkably-- they all fall in this intermediate regime, suggesting that spatial hierarchy is a crucial feature for these systems and probably possesses an important evolutionary advantage.
△ Less
Submitted 14 May, 2013;
originally announced May 2013.
