-
A Scalable Approach to Modeling on Accelerated Neuromorphic Hardware
Authors:
Eric Müller,
Elias Arnold,
Oliver Breitwieser,
Milena Czierlinski,
Arne Emmel,
Jakob Kaiser,
Christian Mauch,
Sebastian Schmitt,
Philipp Spilger,
Raphael Stock,
Yannik Stradmann,
Johannes Weis,
Andreas Baumbach,
Sebastian Billaudelle,
Benjamin Cramer,
Falk Ebert,
Julian Göltz,
Joscha Ilmberger,
Vitali Karasenko,
Mitja Kleider,
Aron Leibfried,
Christian Pehle,
Johannes Schemmel
Abstract:
Neuromorphic systems open up opportunities to enlarge the explorative space for computational research. However, it is often challenging to unite efficiency and usability. This work presents the software aspects of this endeavor for the BrainScaleS-2 system, a hybrid accelerated neuromorphic hardware architecture based on physical modeling. We introduce key aspects of the BrainScaleS-2 Operating S…
▽ More
Neuromorphic systems open up opportunities to enlarge the explorative space for computational research. However, it is often challenging to unite efficiency and usability. This work presents the software aspects of this endeavor for the BrainScaleS-2 system, a hybrid accelerated neuromorphic hardware architecture based on physical modeling. We introduce key aspects of the BrainScaleS-2 Operating System: experiment workflow, API layering, software design, and platform operation. We present use cases to discuss and derive requirements for the software and showcase the implementation. The focus lies on novel system and software features such as multi-compartmental neurons, fast re-configuration for hardware-in-the-loop training, applications for the embedded processors, the non-spiking operation mode, interactive platform access, and sustainable hardware/software co-development. Finally, we discuss further developments in terms of hardware scale-up, system usability and efficiency.
△ Less
Submitted 21 March, 2022;
originally announced March 2022.
-
Inference with Artificial Neural Networks on Analog Neuromorphic Hardware
Authors:
Johannes Weis,
Philipp Spilger,
Sebastian Billaudelle,
Yannik Stradmann,
Arne Emmel,
Eric Müller,
Oliver Breitwieser,
Andreas Grübl,
Joscha Ilmberger,
Vitali Karasenko,
Mitja Kleider,
Christian Mauch,
Korbinian Schreiber,
Johannes Schemmel
Abstract:
The neuromorphic BrainScaleS-2 ASIC comprises mixed-signal neurons and synapse circuits as well as two versatile digital microprocessors. Primarily designed to emulate spiking neural networks, the system can also operate in a vector-matrix multiplication and accumulation mode for artificial neural networks. Analog multiplication is carried out in the synapse circuits, while the results are accumul…
▽ More
The neuromorphic BrainScaleS-2 ASIC comprises mixed-signal neurons and synapse circuits as well as two versatile digital microprocessors. Primarily designed to emulate spiking neural networks, the system can also operate in a vector-matrix multiplication and accumulation mode for artificial neural networks. Analog multiplication is carried out in the synapse circuits, while the results are accumulated on the neurons' membrane capacitors. Designed as an analog, in-memory computing device, it promises high energy efficiency. Fixed-pattern noise and trial-to-trial variations, however, require the implemented networks to cope with a certain level of perturbations. Further limitations are imposed by the digital resolution of the input values (5 bit), matrix weights (6 bit) and resulting neuron activations (8 bit). In this paper, we discuss BrainScaleS-2 as an analog inference accelerator and present calibration as well as optimization strategies, highlighting the advantages of training with hardware in the loop. Among other benchmarks, we classify the MNIST handwritten digits dataset using a two-dimensional convolution and two dense layers. We reach 98.0% test accuracy, closely matching the performance of the same network evaluated in software.
△ Less
Submitted 1 July, 2020; v1 submitted 23 June, 2020;
originally announced June 2020.
-
The Operating System of the Neuromorphic BrainScaleS-1 System
Authors:
Eric Müller,
Sebastian Schmitt,
Christian Mauch,
Sebastian Billaudelle,
Andreas Grübl,
Maurice Güttler,
Dan Husmann,
Joscha Ilmberger,
Sebastian Jeltsch,
Jakob Kaiser,
Johann Klähn,
Mitja Kleider,
Christoph Koke,
José Montes,
Paul Müller,
Johannes Partzsch,
Felix Passenberg,
Hartmut Schmidt,
Bernhard Vogginger,
Jonas Weidner,
Christian Mayr,
Johannes Schemmel
Abstract:
BrainScaleS-1 is a wafer-scale mixed-signal accelerated neuromorphic system targeted for research in the fields of computational neuroscience and beyond-von-Neumann computing. The BrainScaleS Operating System (BrainScaleS OS) is a software stack giving users the possibility to emulate networks described in the high-level network description language PyNN with minimal knowledge of the system. At th…
▽ More
BrainScaleS-1 is a wafer-scale mixed-signal accelerated neuromorphic system targeted for research in the fields of computational neuroscience and beyond-von-Neumann computing. The BrainScaleS Operating System (BrainScaleS OS) is a software stack giving users the possibility to emulate networks described in the high-level network description language PyNN with minimal knowledge of the system. At the same time, expert usage is facilitated by allowing to hook into the system at any depth of the stack. We present operation and development methodologies implemented for the BrainScaleS-1 neuromorphic architecture and walk through the individual components of BrainScaleS OS constituting the software stack for BrainScaleS-1 platform operation.
△ Less
Submitted 2 February, 2022; v1 submitted 30 March, 2020;
originally announced March 2020.
-
Versatile emulation of spiking neural networks on an accelerated neuromorphic substrate
Authors:
Sebastian Billaudelle,
Yannik Stradmann,
Korbinian Schreiber,
Benjamin Cramer,
Andreas Baumbach,
Dominik Dold,
Julian Göltz,
Akos F. Kungl,
Timo C. Wunderlich,
Andreas Hartel,
Eric Müller,
Oliver Breitwieser,
Christian Mauch,
Mitja Kleider,
Andreas Grübl,
David Stöckel,
Christian Pehle,
Arthur Heimbrecht,
Philipp Spilger,
Gerd Kiene,
Vitali Karasenko,
Walter Senn,
Mihai A. Petrovici,
Johannes Schemmel,
Karlheinz Meier
Abstract:
We present first experimental results on the novel BrainScaleS-2 neuromorphic architecture based on an analog neuro-synaptic core and augmented by embedded microprocessors for complex plasticity and experiment control. The high acceleration factor of 1000 compared to biological dynamics enables the execution of computationally expensive tasks, by allowing the fast emulation of long-duration experi…
▽ More
We present first experimental results on the novel BrainScaleS-2 neuromorphic architecture based on an analog neuro-synaptic core and augmented by embedded microprocessors for complex plasticity and experiment control. The high acceleration factor of 1000 compared to biological dynamics enables the execution of computationally expensive tasks, by allowing the fast emulation of long-duration experiments or rapid iteration over many consecutive trials. The flexibility of our architecture is demonstrated in a suite of five distinct experiments, which emphasize different aspects of the BrainScaleS-2 system.
△ Less
Submitted 9 May, 2022; v1 submitted 30 December, 2019;
originally announced December 2019.
-
Accelerated physical emulation of Bayesian inference in spiking neural networks
Authors:
Akos F. Kungl,
Sebastian Schmitt,
Johann Klähn,
Paul Müller,
Andreas Baumbach,
Dominik Dold,
Alexander Kugele,
Nico Gürtler,
Luziwei Leng,
Eric Müller,
Christoph Koke,
Mitja Kleider,
Christian Mauch,
Oliver Breitwieser,
Maurice Güttler,
Dan Husmann,
Kai Husmann,
Joscha Ilmberger,
Andreas Hartel,
Vitali Karasenko,
Andreas Grübl,
Johannes Schemmel,
Karlheinz Meier,
Mihai A. Petrovici
Abstract:
The massively parallel nature of biological information processing plays an important role for its superiority to human-engineered computing devices. In particular, it may hold the key to overcoming the von Neumann bottleneck that limits contemporary computer architectures. Physical-model neuromorphic devices seek to replicate not only this inherent parallelism, but also aspects of its microscopic…
▽ More
The massively parallel nature of biological information processing plays an important role for its superiority to human-engineered computing devices. In particular, it may hold the key to overcoming the von Neumann bottleneck that limits contemporary computer architectures. Physical-model neuromorphic devices seek to replicate not only this inherent parallelism, but also aspects of its microscopic dynamics in analog circuits emulating neurons and synapses. However, these machines require network models that are not only adept at solving particular tasks, but that can also cope with the inherent imperfections of analog substrates. We present a spiking network model that performs Bayesian inference through sampling on the BrainScaleS neuromorphic platform, where we use it for generative and discriminative computations on visual data. By illustrating its functionality on this platform, we implicitly demonstrate its robustness to various substrate-specific distortive effects, as well as its accelerated capability for computation. These results showcase the advantages of brain-inspired physical computation and provide important building blocks for large-scale neuromorphic applications.
△ Less
Submitted 1 April, 2020; v1 submitted 6 July, 2018;
originally announced July 2018.
-
Pattern representation and recognition with accelerated analog neuromorphic systems
Authors:
Mihai A. Petrovici,
Sebastian Schmitt,
Johann Klähn,
David Stöckel,
Anna Schroeder,
Guillaume Bellec,
Johannes Bill,
Oliver Breitwieser,
Ilja Bytschok,
Andreas Grübl,
Maurice Güttler,
Andreas Hartel,
Stephan Hartmann,
Dan Husmann,
Kai Husmann,
Sebastian Jeltsch,
Vitali Karasenko,
Mitja Kleider,
Christoph Koke,
Alexander Kononov,
Christian Mauch,
Eric Müller,
Paul Müller,
Johannes Partzsch,
Thomas Pfeil
, et al. (11 additional authors not shown)
Abstract:
Despite being originally inspired by the central nervous system, artificial neural networks have diverged from their biological archetypes as they have been remodeled to fit particular tasks. In this paper, we review several possibilites to reverse map these architectures to biologically more realistic spiking networks with the aim of emulating them on fast, low-power neuromorphic hardware. Since…
▽ More
Despite being originally inspired by the central nervous system, artificial neural networks have diverged from their biological archetypes as they have been remodeled to fit particular tasks. In this paper, we review several possibilites to reverse map these architectures to biologically more realistic spiking networks with the aim of emulating them on fast, low-power neuromorphic hardware. Since many of these devices employ analog components, which cannot be perfectly controlled, finding ways to compensate for the resulting effects represents a key challenge. Here, we discuss three different strategies to address this problem: the addition of auxiliary network components for stabilizing activity, the utilization of inherently robust architectures and a training method for hardware-emulated networks that functions without perfect knowledge of the system's dynamics and parameters. For all three scenarios, we corroborate our theoretical considerations with experimental results on accelerated analog neuromorphic platforms.
△ Less
Submitted 3 July, 2017; v1 submitted 17 March, 2017;
originally announced March 2017.
-
Neuromorphic Hardware In The Loop: Training a Deep Spiking Network on the BrainScaleS Wafer-Scale System
Authors:
Sebastian Schmitt,
Johann Klaehn,
Guillaume Bellec,
Andreas Gruebl,
Maurice Guettler,
Andreas Hartel,
Stephan Hartmann,
Dan Husmann,
Kai Husmann,
Vitali Karasenko,
Mitja Kleider,
Christoph Koke,
Christian Mauch,
Eric Mueller,
Paul Mueller,
Johannes Partzsch,
Mihai A. Petrovici,
Stefan Schiefer,
Stefan Scholze,
Bernhard Vogginger,
Robert Legenstein,
Wolfgang Maass,
Christian Mayr,
Johannes Schemmel,
Karlheinz Meier
Abstract:
Emulating spiking neural networks on analog neuromorphic hardware offers several advantages over simulating them on conventional computers, particularly in terms of speed and energy consumption. However, this usually comes at the cost of reduced control over the dynamics of the emulated networks. In this paper, we demonstrate how iterative training of a hardware-emulated network can compensate for…
▽ More
Emulating spiking neural networks on analog neuromorphic hardware offers several advantages over simulating them on conventional computers, particularly in terms of speed and energy consumption. However, this usually comes at the cost of reduced control over the dynamics of the emulated networks. In this paper, we demonstrate how iterative training of a hardware-emulated network can compensate for anomalies induced by the analog substrate. We first convert a deep neural network trained in software to a spiking network on the BrainScaleS wafer-scale neuromorphic system, thereby enabling an acceleration factor of 10 000 compared to the biological time domain. This mapping is followed by the in-the-loop training, where in each training step, the network activity is first recorded in hardware and then used to compute the parameter updates in software via backpropagation. An essential finding is that the parameter updates do not have to be precise, but only need to approximately follow the correct gradient, which simplifies the computation of updates. Using this approach, after only several tens of iterations, the spiking network shows an accuracy close to the ideal software-emulated prototype. The presented techniques show that deep spiking networks emulated on analog neuromorphic devices can attain good computational performance despite the inherent variations of the analog substrate.
△ Less
Submitted 6 March, 2017;
originally announced March 2017.