Showing 1–7 of 7 results for author: Kudo, S

Flexible Variational Information Bottleneck: Achieving Diverse Compression with a Single Training

Authors: Sota Kudo, Naoaki Ono, Shigehiko Kanaya, Ming Huang

Abstract: Information Bottleneck (IB) is a widely used framework that enables the extraction of information related to a target random variable from a source random variable. In the objective function, IB controls the trade-off between data compression and predictiveness through the Lagrange multiplier $β$. Traditionally, to find the trade-off to be learned, IB requires a search for $β$ through multiple tra… ▽ More Information Bottleneck (IB) is a widely used framework that enables the extraction of information related to a target random variable from a source random variable. In the objective function, IB controls the trade-off between data compression and predictiveness through the Lagrange multiplier $β$. Traditionally, to find the trade-off to be learned, IB requires a search for $β$ through multiple training cycles, which is computationally expensive. In this study, we introduce Flexible Variational Information Bottleneck (FVIB), an innovative framework for classification task that can obtain optimal models for all values of $β$ with single, computationally efficient training. We theoretically demonstrate that across all values of reasonable $β$, FVIB can simultaneously maximize an approximation of the objective function for Variational Information Bottleneck (VIB), the conventional IB method. Then we empirically show that FVIB can learn the VIB objective as effectively as VIB. Furthermore, in terms of calibration performance, FVIB outperforms other IB and calibration methods by enabling continuous optimization of $β$. Our codes are available at https://github.com/sotakudo/fvib. △ Less

Submitted 2 February, 2024; originally announced February 2024.

MLPerf HPC: A Holistic Benchmark Suite for Scientific Machine Learning on HPC Systems

Authors: Steven Farrell, Murali Emani, Jacob Balma, Lukas Drescher, Aleksandr Drozd, Andreas Fink, Geoffrey Fox, David Kanter, Thorsten Kurth, Peter Mattson, Dawei Mu, Amit Ruhela, Kento Sato, Koichi Shirahata, Tsuguchika Tabaru, Aristeidis Tsaris, Jan Balewski, Ben Cumming, Takumi Danjo, Jens Domke, Takaaki Fukai, Naoto Fukumoto, Tatsuya Fukushi, Balazs Gerofi, Takumi Honda , et al. (18 additional authors not shown)

Abstract: Scientific communities are increasingly adopting machine learning and deep learning models in their applications to accelerate scientific insights. High performance computing systems are pushing the frontiers of performance with a rich diversity of hardware resources and massive scale-out capabilities. There is a critical need to understand fair and effective benchmarking of machine learning appli… ▽ More Scientific communities are increasingly adopting machine learning and deep learning models in their applications to accelerate scientific insights. High performance computing systems are pushing the frontiers of performance with a rich diversity of hardware resources and massive scale-out capabilities. There is a critical need to understand fair and effective benchmarking of machine learning applications that are representative of real-world scientific use cases. MLPerf is a community-driven standard to benchmark machine learning workloads, focusing on end-to-end performance metrics. In this paper, we introduce MLPerf HPC, a benchmark suite of large-scale scientific machine learning training applications driven by the MLCommons Association. We present the results from the first submission round, including a diverse set of some of the world's largest HPC systems. We develop a systematic framework for their joint analysis and compare them in terms of data staging, algorithmic convergence, and compute performance. As a result, we gain a quantitative understanding of optimizations on different subsystems such as staging and on-node loading of data, compute-unit utilization, and communication scheduling, enabling overall $>10 \times$ (end-to-end) performance improvements through system scaling. Notably, our analysis shows a scale-dependent interplay between the dataset size, a system's memory hierarchy, and training convergence that underlines the importance of near-compute storage. To overcome the data-parallel scalability challenge at large batch sizes, we discuss specific learning techniques and hybrid data-and-model parallelism that are effective on large systems. We conclude by characterizing each benchmark with respect to low-level memory, I/O, and network behavior to parameterize extended roofline performance models in future rounds. △ Less

Submitted 26 October, 2021; v1 submitted 21 October, 2021; originally announced October 2021.

Seating preference analysis for hybrid workplaces

Authors: Mohammad Saiedur Rahaman, Shaw Kudo, Tim Rawling, Yongli Ren, Flora D. Salim

Abstract: Due to the increasing nature of flexible work and the recent requirements from COVID-19 restrictions, workplaces are becoming more hybrid (i.e. allowing workers to work between traditional office spaces and elsewhere including from home). Since workplaces are different in design, layout and available facilities, many workers find it difficult to adjust accordingly. Eventually, this impacts negativ… ▽ More Due to the increasing nature of flexible work and the recent requirements from COVID-19 restrictions, workplaces are becoming more hybrid (i.e. allowing workers to work between traditional office spaces and elsewhere including from home). Since workplaces are different in design, layout and available facilities, many workers find it difficult to adjust accordingly. Eventually, this impacts negatively towards work productivity and other related parameters including concentration, stress, and mood while at work. One of the key factors that causes this negative work experience is directly linked to the available seating arrangements. In this paper, we conduct an analysis to understand various seating preferences of 37 workers with varying demographics, using the data collected pre-COVID-19, and analyse the findings in the context of hybrid workplace settings. We also discuss a list of implications illustrating how our findings can be adapted across wider hybrid work settings. △ Less

Submitted 30 July, 2020; originally announced July 2020.

Comments: 4 pages, 3 figures

Journal ref: NFW '20: Symposium on New Future of Work, August 03--05, 2020

An Ambient-Physical System to Infer Concentration in Open-plan Workplace

Authors: Mohammad Saiedur Rahaman, Jonathan Liono, Yongli Ren, Jeffrey Chan, Shaw Kudo, Tim Rawling, Flora D. Salim

Abstract: One of the core challenges in open-plan workspaces is to ensure a good level of concentration for the workers while performing their tasks. Hence, being able to infer concentration levels of workers will allow building designers, managers, and workers to estimate what effect different open-plan layouts will have and to find an optimal one. In this research, we present an ambient-physical system to… ▽ More One of the core challenges in open-plan workspaces is to ensure a good level of concentration for the workers while performing their tasks. Hence, being able to infer concentration levels of workers will allow building designers, managers, and workers to estimate what effect different open-plan layouts will have and to find an optimal one. In this research, we present an ambient-physical system to investigate the concentration inference problem. Specifically, we deploy a series of pervasive sensors to capture various ambient and physical signals related to perceived concentration at work. The practicality of our system has been tested on two large open-plan workplaces with different designs and layouts. The empirical results highlight promising applications of pervasive sensing in occupational concentration inference, which can be adopted to enhance the capabilities of modern workplaces. △ Less

Submitted 26 May, 2020; originally announced May 2020.

Comments: 12 pages, 14 figures

White Paper from Workshop on Large-scale Parallel Numerical Computing Technology (LSPANC 2020): HPC and Computer Arithmetic toward Minimal-Precision Computing

Authors: Roman Iakymchuk, Daichi Mukunoki, Artur Podobas, Fabienne Jézéquel, Toshiyuki Imamura, Norihisa Fujita, Jens Huthmann, Shuhei Kudo, Yiyu Tan, Jens Domke, Kai Torben Ohlhus, Takeshi Fukaya, Takeo Hoshi, Yuki Murakami, Maho Nakata, Takeshi Ogita, Kentaro Sano, Taisuke Boku

Abstract: In numerical computations, precision of floating-point computations is a key factor to determine the performance (speed and energy-efficiency) as well as the reliability (accuracy and reproducibility). However, precision generally plays a contrary role for both. Therefore, the ultimate concept for maximizing both at the same time is the minimal-precision computing through precision-tuning, which a… ▽ More In numerical computations, precision of floating-point computations is a key factor to determine the performance (speed and energy-efficiency) as well as the reliability (accuracy and reproducibility). However, precision generally plays a contrary role for both. Therefore, the ultimate concept for maximizing both at the same time is the minimal-precision computing through precision-tuning, which adjusts the optimal precision for each operation and data. Several studies have been already conducted for it so far (e.g. Precimoniuos and Verrou), but the scope of those studies is limited to the precision-tuning alone. Hence, we aim to propose a broader concept of the minimal-precision computing system with precision-tuning, involving both hardware and software stack. In 2019, we have started the Minimal-Precision Computing project to propose a more broad concept of the minimal-precision computing system with precision-tuning, involving both hardware and software stack. Specifically, our system combines (1) a precision-tuning method based on Discrete Stochastic Arithmetic (DSA), (2) arbitrary-precision arithmetic libraries, (3) fast and accurate numerical libraries, and (4) Field-Programmable Gate Array (FPGA) with High-Level Synthesis (HLS). In this white paper, we aim to provide an overview of various technologies related to minimal- and mixed-precision, to outline the future direction of the project, as well as to discuss current challenges together with our project members and guest speakers at the LSPANC 2020 workshop; https://www.r-ccs.riken.jp/labs/lpnctrt/lspanc2020jan/. △ Less

Submitted 11 April, 2020; v1 submitted 9 April, 2020; originally announced April 2020.

Report number: hal-02536316

A Parallelizable Energy-Preserving Integrator MB4 and Its Application to Quantum-Mechanical Wavepacket Dynamics

Authors: Tsubasa Sakai, Shuhei Kudo, Hiroto Imachi, Yuto Miyatake, Takeo Hoshi, Yusaku Yamamoto

Abstract: In simulating physical systems, conservation of the total energy is often essential, especially when energy conversion between different forms of energy occurs frequently. Recently, a new fourth order energy-preserving integrator named MB4 was proposed based on the so-called continuous stage Runge--Kutta methods (Y.~Miyatake and J.~C.~Butcher, SIAM J.~Numer.~Anal., 54(3), 1993-2013). A salient fea… ▽ More In simulating physical systems, conservation of the total energy is often essential, especially when energy conversion between different forms of energy occurs frequently. Recently, a new fourth order energy-preserving integrator named MB4 was proposed based on the so-called continuous stage Runge--Kutta methods (Y.~Miyatake and J.~C.~Butcher, SIAM J.~Numer.~Anal., 54(3), 1993-2013). A salient feature of this method is that it is parallelizable, which makes its computational time for one time step comparable to that of second order methods. In this paper, we illustrate how to apply the MB4 method to a concrete ordinary differential equation using the nonlinear Schrödinger-type equation on a two-dimensional grid as an example. This system is a prototypical model of two-dimensional disordered organic material and is difficult to solve with standard methods like the classical Runge--Kutta methods due to the nonlinearity and the $δ$-function like potential coming from defects. Numerical tests show that the method can solve the equation stably and preserves the total energy to 16-digit accuracy throughout the simulation. It is also shown that parallelization of the method yields up to 2.8 times speedup using 3 computational nodes. △ Less

Submitted 9 March, 2020; originally announced March 2020.

Multiscale Universal Interface: A Concurrent Framework for Coupling Heterogeneous Solvers

Authors: Yu-Hang Tang, Shuhei Kudo, Xin Bian, Zhen Li, George E. Karniadakis

Abstract: Concurrently coupled numerical simulations using heterogeneous solvers are powerful tools for modeling multiscale phenomena. However, major modifications to existing codes are often required to enable such simulations, posing significant difficulties in practice. In this paper we present a C++ library, i.e. the Multiscale Universal Interface (MUI), which is capable of facilitating the coupling eff… ▽ More Concurrently coupled numerical simulations using heterogeneous solvers are powerful tools for modeling multiscale phenomena. However, major modifications to existing codes are often required to enable such simulations, posing significant difficulties in practice. In this paper we present a C++ library, i.e. the Multiscale Universal Interface (MUI), which is capable of facilitating the coupling effort for a wide range of multiscale simulations. The library adopts a header-only form with minimal external dependency and hence can be easily dropped into existing codes. A data sampler concept is introduced, combined with a hybrid dynamic/static typing mechanism, to create an easily customizable framework for solver-independent data interpretation. The library integrates MPI MPMD support and an asynchronous communication protocol to handle inter-solver information exchange irrespective of the solvers' own MPI awareness. Template metaprogramming is heavily employed to simultaneously improve runtime performance and code flexibility. We validated the library by solving three different multiscale problems, which also serve to demonstrate the flexibility of the framework in handling heterogeneous models and solvers. In the first example, a Couette flow was simulated using two concurrently coupled Smoothed Particle Hydrodynamics (SPH) simulations of different spatial resolutions. In the second example, we coupled the deterministic SPH method with the stochastic Dissipative Particle Dynamics (DPD) method to study the effect of surface grafting on the hydrodynamics properties on the surface. In the third example, we consider conjugate heat transfer between a solid domain and a fluid domain by coupling the particle-based energy-conserving DPD (eDPD) method with the Finite Element Method (FEM). △ Less

Submitted 7 March, 2015; v1 submitted 5 November, 2014; originally announced November 2014.

Comments: The library source code is freely available under the GPLv3 license at http://www.cfm.brown.edu/repo/release/MUI/