Showing 1–27 of 27 results for author: Nahrstedt, K

TraceNet: Segment one thing efficiently

Authors: Mingyuan Wu, Zichuan Liu, Haozhen Zheng, Hongpeng Guo, Bo Chen, Xin Lu, Klara Nahrstedt

Abstract: Efficient single instance segmentation is essential for unlocking features in the mobile imaging applications, such as capture or editing. Existing on-the-fly mobile imaging applications scope the segmentation task to portraits or the salient subject due to the computational constraints. Instance segmentation, despite its recent developments towards efficient networks, is still heavy due to the co… ▽ More Efficient single instance segmentation is essential for unlocking features in the mobile imaging applications, such as capture or editing. Existing on-the-fly mobile imaging applications scope the segmentation task to portraits or the salient subject due to the computational constraints. Instance segmentation, despite its recent developments towards efficient networks, is still heavy due to the cost of computation on the entire image to identify all instances. To address this, we propose and formulate a one tap driven single instance segmentation task that segments a single instance selected by a user via a positive tap. This task, in contrast to the broader task of segmenting anything as suggested in the Segment Anything Model \cite{sam}, focuses on efficient segmentation of a single instance specified by the user. To solve this problem, we present TraceNet, which explicitly locates the selected instance by way of receptive field tracing. TraceNet identifies image regions that are related to the user tap and heavy computations are only performed on selected regions of the image. Therefore overall computation cost and memory consumption are reduced during inference. We evaluate the performance of TraceNet on instance IoU average over taps and the proportion of the region that a user tap can fall into for a high-quality single-instance mask. Experimental results on MS-COCO and LVIS demonstrate the effectiveness and efficiency of the proposed approach. TraceNet can jointly achieve the efficiency and interactivity, filling in the gap between needs for efficient mobile inference and recent research trend towards multimodal and interactive segmentation models. △ Less

Submitted 21 June, 2024; originally announced June 2024.

Federated Transfer Learning with Task Personalization for Condition Monitoring in Ultrasonic Metal Welding

Authors: Ahmadreza Eslaminia, Yuquan Meng, Klara Nahrstedt, Chenhui Shao

Abstract: Ultrasonic metal welding (UMW) is a key joining technology with widespread industrial applications. Condition monitoring (CM) capabilities are critically needed in UMW applications because process anomalies significantly deteriorate the joining quality. Recently, machine learning models emerged as a promising tool for CM in many manufacturing applications due to their ability to learn complex patt… ▽ More Ultrasonic metal welding (UMW) is a key joining technology with widespread industrial applications. Condition monitoring (CM) capabilities are critically needed in UMW applications because process anomalies significantly deteriorate the joining quality. Recently, machine learning models emerged as a promising tool for CM in many manufacturing applications due to their ability to learn complex patterns. Yet, the successful deployment of these models requires substantial training data that may be expensive and time-consuming to collect. Additionally, many existing machine learning models lack generalizability and cannot be directly applied to new process configurations (i.e., domains). Such issues may be potentially alleviated by pooling data across manufacturers, but data sharing raises critical data privacy concerns. To address these challenges, this paper presents a Federated Transfer Learning with Task Personalization (FTL-TP) framework that provides domain generalization capabilities in distributed learning while ensuring data privacy. By effectively learning a unified representation from feature space, FTL-TP can adapt CM models for clients working on similar tasks, thereby enhancing their overall adaptability and performance jointly. To demonstrate the effectiveness of FTL-TP, we investigate two distinct UMW CM tasks, tool condition monitoring and workpiece surface condition classification. Compared with state-of-the-art FL algorithms, FTL-TP achieves a 5.35%--8.08% improvement of accuracy in CM in new target domains. FTL-TP is also shown to perform excellently in challenging scenarios involving unbalanced data distributions and limited client fractions. Furthermore, by implementing the FTL-TP method on an edge-cloud architecture, we show that this method is both viable and efficient in practice. The FTL-TP framework is readily extensible to various other manufacturing applications. △ Less

Submitted 20 April, 2024; originally announced April 2024.

Comments: 37 pages, 8 figures

FedCore: Straggler-Free Federated Learning with Distributed Coresets

Authors: Hongpeng Guo, Haotian Gu, Xiaoyang Wang, Bo Chen, Eun Kyung Lee, Tamar Eilam, Deming Chen, Klara Nahrstedt

Abstract: Federated learning (FL) is a machine learning paradigm that allows multiple clients to collaboratively train a shared model while keeping their data on-premise. However, the straggler issue, due to slow clients, often hinders the efficiency and scalability of FL. This paper presents FedCore, an algorithm that innovatively tackles the straggler problem via the decentralized selection of coresets, r… ▽ More Federated learning (FL) is a machine learning paradigm that allows multiple clients to collaboratively train a shared model while keeping their data on-premise. However, the straggler issue, due to slow clients, often hinders the efficiency and scalability of FL. This paper presents FedCore, an algorithm that innovatively tackles the straggler problem via the decentralized selection of coresets, representative subsets of a dataset. Contrary to existing centralized coreset methods, FedCore creates coresets directly on each client in a distributed manner, ensuring privacy preservation in FL. FedCore translates the coreset optimization problem into a more tractable k-medoids clustering problem and operates distributedly on each client. Theoretical analysis confirms FedCore's convergence, and practical evaluations demonstrate an 8x reduction in FL training time, without compromising model accuracy. Our extensive evaluations also show that FedCore generalizes well to existing FL frameworks. △ Less

Submitted 31 January, 2024; originally announced February 2024.

WeldMon: A Cost-effective Ultrasonic Welding Machine Condition Monitoring System

Authors: Beitong Tian, Kuan-Chieh Lu, Ahmadreza Eslaminia, Yaohui Wang, Chenhui Shao, Klara Nahrstedt

Abstract: Ultrasonic welding machines play a critical role in the lithium battery industry, facilitating the bonding of batteries with conductors. Ensuring high-quality welding is vital, making tool condition monitoring systems essential for early-stage quality control. However, existing monitoring methods face challenges in cost, downtime, and adaptability. In this paper, we present WeldMon, an affordable… ▽ More Ultrasonic welding machines play a critical role in the lithium battery industry, facilitating the bonding of batteries with conductors. Ensuring high-quality welding is vital, making tool condition monitoring systems essential for early-stage quality control. However, existing monitoring methods face challenges in cost, downtime, and adaptability. In this paper, we present WeldMon, an affordable ultrasonic welding machine condition monitoring system that utilizes a custom data acquisition system and a data analysis pipeline designed for real-time analysis. Our classification algorithm combines auto-generated features and hand-crafted features, achieving superior cross-validation accuracy (95.8% on average over all testing tasks) compared to the state-of-the-art method (92.5%) in condition classification tasks. Our data augmentation approach alleviates the concept drift problem, enhancing tool condition classification accuracy by 8.3%. All algorithms run locally, requiring only 385 milliseconds to process data for each welding cycle. We deploy WeldMon and a commercial system on an actual ultrasonic welding machine, performing a comprehensive comparison. Our findings highlight the potential for developing cost-effective, high-performance, and reliable tool condition monitoring systems. △ Less

Submitted 4 August, 2023; originally announced August 2023.

Comments: 9 pages, 5 figures

DeepStream: Bandwidth Efficient Multi-Camera Video Streaming for Deep Learning Analytics

Authors: Hongpeng Guo, Beitong Tian, Zhe Yang, Bo Chen, Qian Zhou, Shengzhong Liu, Klara Nahrstedt, Claudiu Danilov

Abstract: Deep learning video analytic systems process live video feeds from multiple cameras with computer vision models deployed on edge or cloud. To optimize utility for these systems, which usually corresponds to query accuracy, efficient bandwidth management for the cameras competing for the fluctuating network resources is crucial. We propose DeepStream, a bandwidth efficient multi-camera video stream… ▽ More Deep learning video analytic systems process live video feeds from multiple cameras with computer vision models deployed on edge or cloud. To optimize utility for these systems, which usually corresponds to query accuracy, efficient bandwidth management for the cameras competing for the fluctuating network resources is crucial. We propose DeepStream, a bandwidth efficient multi-camera video streaming system for deep learning video analytics. DeepStream addresses the challenge of limited and fluctuating bandwidth resources by offering several tailored solutions. We design a novel Regions of Interest detection (ROIDet) algorithm which can run in real time on resource constraint devices, such as Raspberry Pis, to remove spatial redundancy in video frames and reduce the amount of data to be transmitted. We also propose a content-aware bandwidth optimization framework and an Elastic Transmission Mechanism that exploits correlations among video contents. We implement DeepStream on Raspberry Pis and a desktop computer. Evaluations on real-world datasets show that DeepStream's ROIDet algorithm saves up to 54\% bandwidth with less than 1\% accuracy drop. Additionally,DeepStream improves utility by up to 23\% compared to baselines under the same bandwidth conditions. △ Less

Submitted 26 June, 2023; originally announced June 2023.

360TripleView: 360-Degree Video View Management System Driven by Convergence Value of Viewing Preferences

Authors: Qian Zhou, Michael Zink, Ramesh Sitaraman, Klara Nahrstedt

Abstract: 360-degree video has become increasingly popular in content consumption. However, finding the viewing direction for important content within each frame poses a significant challenge. Existing approaches rely on either viewer input or algorithmic determination to select the viewing direction, but neither mode consistently outperforms the other in terms of content-importance. In this paper, we propo… ▽ More 360-degree video has become increasingly popular in content consumption. However, finding the viewing direction for important content within each frame poses a significant challenge. Existing approaches rely on either viewer input or algorithmic determination to select the viewing direction, but neither mode consistently outperforms the other in terms of content-importance. In this paper, we propose 360TripleView, the first view management system for 360-degree video that automatically infers and utilizes the better view mode for each frame, ultimately providing viewers with higher content-importance views. Through extensive experiments and a user study, we demonstrate that 360TripleView achieves over 90\% accuracy in inferring the better mode and significantly enhances content-importance compared to existing methods. △ Less

Submitted 3 December, 2023; v1 submitted 13 June, 2023; originally announced June 2023.

Coordinated Science Laboratory 70th Anniversary Symposium: The Future of Computing

Authors: Klara Nahrstedt, Naresh Shanbhag, Vikram Adve, Nancy Amato, Romit Roy Choudhury, Carl Gunter, Nam Sung Kim, Olgica Milenkovic, Sayan Mitra, Lav Varshney, Yurii Vlasov, Sarita Adve, Rashid Bashir, Andreas Cangellaris, James DiCarlo, Katie Driggs-Campbell, Nick Feamster, Mattia Gazzola, Karrie Karahalios, Sanmi Koyejo, Paul Kwiat, Bo Li, Negar Mehr, Ravish Mehra, Andrew Miller , et al. (3 additional authors not shown)

Abstract: In 2021, the Coordinated Science Laboratory CSL, an Interdisciplinary Research Unit at the University of Illinois Urbana-Champaign, hosted the Future of Computing Symposium to celebrate its 70th anniversary. CSL's research covers the full computing stack, computing's impact on society and the resulting need for social responsibility. In this white paper, we summarize the major technological points… ▽ More In 2021, the Coordinated Science Laboratory CSL, an Interdisciplinary Research Unit at the University of Illinois Urbana-Champaign, hosted the Future of Computing Symposium to celebrate its 70th anniversary. CSL's research covers the full computing stack, computing's impact on society and the resulting need for social responsibility. In this white paper, we summarize the major technological points, insights, and directions that speakers brought forward during the Future of Computing Symposium. Participants discussed topics related to new computing paradigms, technologies, algorithms, behaviors, and research challenges to be expected in the future. The symposium focused on new computing paradigms that are going beyond traditional computing and the research needed to support their realization. These needs included stressing security and privacy, the end to end human cyber physical systems and with them the analysis of the end to end artificial intelligence needs. Furthermore, advances that enable immersive environments for users, the boundaries between humans and machines will blur and become seamless. Particular integration challenges were made clear in the final discussion on the integration of autonomous driving, robo taxis, pedestrians, and future cities. Innovative approaches were outlined to motivate the next generation of researchers to work on these challenges. The discussion brought out the importance of considering not just individual research areas, but innovations at the intersections between computing research efforts and relevant application domains, such as health care, transportation, energy systems, and manufacturing. △ Less

Submitted 4 October, 2022; originally announced October 2022.

Hierarchical Semi-Supervised Contrastive Learning for Contamination-Resistant Anomaly Detection

Authors: Gaoang Wang, Yibing Zhan, Xinchao Wang, Mingli Song, Klara Nahrstedt

Abstract: Anomaly detection aims at identifying deviant samples from the normal data distribution. Contrastive learning has provided a successful way to sample representation that enables effective discrimination on anomalies. However, when contaminated with unlabeled abnormal samples in training set under semi-supervised settings, current contrastive-based methods generally 1) ignore the comprehensive rela… ▽ More Anomaly detection aims at identifying deviant samples from the normal data distribution. Contrastive learning has provided a successful way to sample representation that enables effective discrimination on anomalies. However, when contaminated with unlabeled abnormal samples in training set under semi-supervised settings, current contrastive-based methods generally 1) ignore the comprehensive relation between training data, leading to suboptimal performance, and 2) require fine-tuning, resulting in low efficiency. To address the above two issues, in this paper, we propose a novel hierarchical semi-supervised contrastive learning (HSCL) framework, for contamination-resistant anomaly detection. Specifically, HSCL hierarchically regulates three complementary relations: sample-to-sample, sample-to-prototype, and normal-to-abnormal relations, enlarging the discrimination between normal and abnormal samples with a comprehensive exploration of the contaminated data. Besides, HSCL is an end-to-end learning approach that can efficiently learn discriminative representations without fine-tuning. HSCL achieves state-of-the-art performance in multiple scenarios, such as one-class classification and cross-dataset detection. Extensive ablation studies further verify the effectiveness of each considered relation. The code is available at https://github.com/GaoangW/HSCL. △ Less

Submitted 24 July, 2022; originally announced July 2022.

ProvLet: A Provenance Management Service for Long Tail Microscopy Data

Authors: Hessam Moeini, Todd Nicholson, Klara Nahrstedt, Gianni Pezzarossi

Abstract: Provenance management must be present to enhance the overall security and reliability of long-tail microscopy (LTM) data management systems. However, there are challenges in provenance for domains with LTM data. The provenance data need to be collected more frequently, which increases system overheads (in terms of computation and storage) and results in scalability issues. Moreover, in most scient… ▽ More Provenance management must be present to enhance the overall security and reliability of long-tail microscopy (LTM) data management systems. However, there are challenges in provenance for domains with LTM data. The provenance data need to be collected more frequently, which increases system overheads (in terms of computation and storage) and results in scalability issues. Moreover, in most scientific application domains a provenance solution must consider network-related events as well. Therefore, provenance data in LTM data management systems are highly diverse and must be organized and processed carefully. In this paper, we introduce a novel provenance service, called ProvLet, to collect, distribute, analyze, and visualize provenance data in LTM data management systems. This means (1) we address how to filter and store the desired transactions on disk; (2) we consider a data organization model at higher level data abstractions, suitable for step-by-step scientific experiments, such as datasets and collections, and develop provenance algorithms over these data abstractions, rather than solutions considering low-level abstractions such as files and folders. (3) We utilize ProvLet's log files and visualize provenance information for further forensics explorations. The validation of ProvLet with actual long tail microscopy data, collected over a period of six years, shows a provenance service that yields a low system overhead and enables scalability. △ Less

Submitted 22 September, 2021; originally announced September 2021.

Comments: 5 pages, 5 figures

CrossRoI: Cross-camera Region of Interest Optimization for Efficient Real Time Video Analytics at Scale

Authors: Hongpeng Guo, Shuochao Yao, Zhe Yang, Qian Zhou, Klara Nahrstedt

Abstract: Video cameras are pervasively deployed in city scale for public good or community safety (i.e. traffic monitoring or suspected person tracking). However, analyzing large scale video feeds in real time is data intensive and poses severe challenges to network and computation systems today. We present CrossRoI, a resource-efficient system that enables real time video analytics at scale via harnessing… ▽ More Video cameras are pervasively deployed in city scale for public good or community safety (i.e. traffic monitoring or suspected person tracking). However, analyzing large scale video feeds in real time is data intensive and poses severe challenges to network and computation systems today. We present CrossRoI, a resource-efficient system that enables real time video analytics at scale via harnessing the videos content associations and redundancy across a fleet of cameras. CrossRoI exploits the intrinsic physical correlations of cross-camera viewing fields to drastically reduce the communication and computation costs. CrossRoI removes the repentant appearances of same objects in multiple cameras without harming comprehensive coverage of the scene. CrossRoI operates in two phases - an offline phase to establish cross-camera correlations, and an efficient online phase for real time video inference. Experiments on real-world video feeds show that CrossRoI achieves 42% - 65% reduction for network overhead and 25% - 34% reduction for response delay in real time video analytics applications with more than 99% query accuracy, when compared to baseline methods. If integrated with SotA frame filtering systems, the performance gains of CrossRoI reach 50% - 80% (network overhead) and 33% - 61% (end-to-end delay). △ Less

Submitted 13 May, 2021; originally announced May 2021.

Comments: accepted in 12th ACM Multimedia Systems Conference (MMsys 21')

DeepRT: A Soft Real Time Scheduler for Computer Vision Applications on the Edge

Authors: Zhe Yang, Klara Nahrstedt, Hongpeng Guo, Qian Zhou

Abstract: The ubiquity of smartphone cameras and IoT cameras, together with the recent boom of deep learning and deep neural networks, proliferate various computer vision driven mobile and IoT applications deployed on the edge. This paper focuses on applications which make soft real time requests to perform inference on their data - they desire prompt responses within designated deadlines, but occasional de… ▽ More The ubiquity of smartphone cameras and IoT cameras, together with the recent boom of deep learning and deep neural networks, proliferate various computer vision driven mobile and IoT applications deployed on the edge. This paper focuses on applications which make soft real time requests to perform inference on their data - they desire prompt responses within designated deadlines, but occasional deadline misses are acceptable. Supporting soft real time applications on a multi-tenant edge server is not easy, since the requests sharing the limited GPU computing resources of an edge server interfere with each other. In order to tackle this problem, we comprehensively evaluate how latency and throughput respond to different GPU execution plans. Based on this analysis, we propose a GPU scheduler, DeepRT, which provides latency guarantee to the requests while maintaining high overall system throughput. The key component of DeepRT, DisBatcher, batches data from different requests as much as possible while it is proven to provide latency guarantee for requests admitted by an Admission Control Module. DeepRT also includes an Adaptation Module which tackles overruns. Our evaluation results show that DeepRT outperforms state-of-the-art works in terms of the number of deadline misses and throughput. △ Less

Submitted 4 May, 2021; originally announced May 2021.

Comments: Accepted by the Sixth ACM/IEEE Symposium on Edge Computing, 2021

ACM Class: C.2.4; I.4.0

Robusta: Robust AutoML for Feature Selection via Reinforcement Learning

Authors: Xiaoyang Wang, Bo Li, Yibo Zhang, Bhavya Kailkhura, Klara Nahrstedt

Abstract: Several AutoML approaches have been proposed to automate the machine learning (ML) process, such as searching for the ML model architectures and hyper-parameters. However, these AutoML pipelines only focus on improving the learning accuracy of benign samples while ignoring the ML model robustness under adversarial attacks. As ML systems are increasingly being used in a variety of mission-critical… ▽ More Several AutoML approaches have been proposed to automate the machine learning (ML) process, such as searching for the ML model architectures and hyper-parameters. However, these AutoML pipelines only focus on improving the learning accuracy of benign samples while ignoring the ML model robustness under adversarial attacks. As ML systems are increasingly being used in a variety of mission-critical applications, improving the robustness of ML systems has become of utmost importance. In this paper, we propose the first robust AutoML framework, Robusta--based on reinforcement learning (RL)--to perform feature selection, aiming to select features that lead to both accurate and robust ML systems. We show that a variation of the 0-1 robust loss can be directly optimized via an RL-based combinatorial search in the feature selection scenario. In addition, we employ heuristics to accelerate the search procedure based on feature scoring metrics, which are mutual information scores, tree-based classifiers feature importance scores, F scores, and Integrated Gradient (IG) scores, as well as their combinations. We conduct extensive experiments and show that the proposed framework is able to improve the model robustness by up to 22% while maintaining competitive accuracy on benign samples compared with other feature selection methods. △ Less

Submitted 14 January, 2021; originally announced January 2021.

Safety, Security, and Privacy Threats Posed by Accelerating Trends in the Internet of Things

Authors: Kevin Fu, Tadayoshi Kohno, Daniel Lopresti, Elizabeth Mynatt, Klara Nahrstedt, Shwetak Patel, Debra Richardson, Ben Zorn

Abstract: The Internet of Things (IoT) is already transforming industries, cities, and homes. The economic value of this transformation across all industries is estimated to be trillions of dollars and the societal impact on energy efficiency, health, and productivity are enormous. Alongside potential benefits of interconnected smart devices comes increased risk and potential for abuse when embedding sensin… ▽ More The Internet of Things (IoT) is already transforming industries, cities, and homes. The economic value of this transformation across all industries is estimated to be trillions of dollars and the societal impact on energy efficiency, health, and productivity are enormous. Alongside potential benefits of interconnected smart devices comes increased risk and potential for abuse when embedding sensing and intelligence into every device. One of the core problems with the increasing number of IoT devices is the increased complexity that is required to operate them safely and securely. This increased complexity creates new safety, security, privacy, and usability challenges far beyond the difficult challenges individuals face just securing a single device. We highlight some of the negative trends that smart devices and collections of devices cause and we argue that issues related to security, physical safety, privacy, and usability are tightly interconnected and solutions that address all four simultaneously are needed. Tight safety and security standards for individual devices based on existing technology are needed. Likewise research that determines the best way for individuals to confidently manage collections of devices must guide the future deployments of such systems. △ Less

Submitted 31 July, 2020; originally announced August 2020.

Comments: A Computing Community Consortium (CCC) white paper, 9 pages

SiEVE: Semantically Encoded Video Analytics on Edge and Cloud

Authors: Tarek Elgamal, Shu Shi, Varun Gupta, Rittwik Jana, Klara Nahrstedt

Abstract: Recent advances in computer vision and neural networks have made it possible for more surveillance videos to be automatically searched and analyzed by algorithms rather than humans. This happened in parallel with advances in edge computing where videos are analyzed over hierarchical clusters that contain edge devices, close to the video source. However, the current video analysis pipeline has seve… ▽ More Recent advances in computer vision and neural networks have made it possible for more surveillance videos to be automatically searched and analyzed by algorithms rather than humans. This happened in parallel with advances in edge computing where videos are analyzed over hierarchical clusters that contain edge devices, close to the video source. However, the current video analysis pipeline has several disadvantages when dealing with such advances. For example, video encoders have been designed for a long time to please human viewers and be agnostic of the downstream analysis task (e.g., object detection). Moreover, most of the video analytics systems leverage 2-tier architecture where the encoded video is sent to either a remote cloud or a private edge server but does not efficiently leverage both of them. In response to these advances, we present SIEVE, a 3-tier video analytics system to reduce the latency and increase the throughput of analytics over video streams. In SIEVE, we present a novel technique to detect objects in compressed video streams. We refer to this technique as semantic video encoding because it allows video encoders to be aware of the semantics of the downstream task (e.g., object detection). Our results show that by leveraging semantic video encoding, we achieve close to 100% object detection accuracy with decompressing only 3.5% of the video frames which results in more than 100x speedup compared to classical approaches that decompress every video frame. △ Less

Submitted 1 June, 2020; originally announced June 2020.

Serdab: An IoT Framework for Partitioning Neural Networks Computation across Multiple Enclaves

Authors: Tarek Elgamal, Klara Nahrstedt

Abstract: Recent advances in Deep Neural Networks (DNN) and Edge Computing have made it possible to automatically analyze streams of videos from home/security cameras over hierarchical clusters that include edge devices, close to the video source, as well as remote cloud compute resources. However, preserving the privacy and confidentiality of users' sensitive data as it passes through different devices rem… ▽ More Recent advances in Deep Neural Networks (DNN) and Edge Computing have made it possible to automatically analyze streams of videos from home/security cameras over hierarchical clusters that include edge devices, close to the video source, as well as remote cloud compute resources. However, preserving the privacy and confidentiality of users' sensitive data as it passes through different devices remains a concern to most users. Private user data is subject to attacks by malicious attackers or misuse by internal administrators who may use the data in activities that are not explicitly approved by the user. To address this challenge, we present Serdab, a distributed orchestration framework for deploying deep neural network computation across multiple secure enclaves (e.g., Intel SGX). Secure enclaves provide a guarantee on the privacy of the data/code deployed inside it. However, their limited hardware resources make them inefficient when solely running an entire deep neural network. To bridge this gap, Serdab presents a DNN partitioning strategy to distribute the layers of the neural network across multiple enclave devices or across an enclave device and other hardware accelerators. Our partitioning strategy achieves up to 4.7x speedup compared to executing the entire neural network in one enclave. △ Less

Submitted 12 May, 2020; originally announced May 2020.

Nanotechnology-inspired Information Processing Systems of the Future

Authors: Randy Bryant, Mark Hill, Tom Kazior, Daniel Lee, Jie Liu, Klara Nahrstedt, Vijay Narayanan, Jan Rabaey, Hava Siegelmann, Naresh Shanbhag, Naveen Verma, H. -S. Philip Wong

Abstract: Nanoscale semiconductor technology has been a key enabler of the computing revolution. It has done so via advances in new materials and manufacturing processes that resulted in the size of the basic building block of computing systems - the logic switch and memory devices - being reduced into the nanoscale regime. Nanotechnology has provided increased computing functionality per unit volume, energ… ▽ More Nanoscale semiconductor technology has been a key enabler of the computing revolution. It has done so via advances in new materials and manufacturing processes that resulted in the size of the basic building block of computing systems - the logic switch and memory devices - being reduced into the nanoscale regime. Nanotechnology has provided increased computing functionality per unit volume, energy, and cost. In order for computing systems to continue to deliver substantial benefits for the foreseeable future to society at large, it is critical that the very notion of computing be examined in the light of nanoscale realities. In particular, one needs to ask what it means to compute when the very building block - the logic switch - no longer exhibits the level of determinism required by the von Neumann architecture. There needs to be a sustained and heavy investment in a nation-wide Vertically Integrated Semiconductor Ecosystem (VISE). VISE is a program in which research and development is conducted seamlessly across the entire compute stack - from applications, systems and algorithms, architectures, circuits and nanodevices, and materials. A nation-wide VISE provides clear strategic advantages in ensuring the US's global superiority in semiconductors. First, a VISE provides the highest quality seed-corn for nurturing transformative ideas that are critically needed today in order for nanotechnology-inspired computing to flourish. It does so by dramatically opening up new areas of semiconductor research that are inspired and driven by new application needs. Second, a VISE creates a very high barrier to entry from foreign competitors because it is extremely hard to establish, and even harder to duplicate. △ Less

Submitted 5 May, 2020; originally announced May 2020.

Comments: A Computing Community Consortium (CCC) workshop report, 18 pages

Report number: ccc2016report_3

Report of 2017 NSF Workshop on Multimedia Challenges, Opportunities and Research Roadmaps

Authors: Shih-Fu Chang, Alex Hauptmann, Louis-Philippe Morency, Sameer Antani, Dick Bulterman, Carlos Busso, Joyce Chai, Julia Hirschberg, Ramesh Jain, Ketan Mayer-Patel, Reuven Meth, Raymond Mooney, Klara Nahrstedt, Shri Narayanan, Prem Natarajan, Sharon Oviatt, Balakrishnan Prabhakaran, Arnold Smeulders, Hari Sundaram, Zhengyou Zhang, Michelle Zhou

Abstract: With the transformative technologies and the rapidly changing global R&D landscape, the multimedia and multimodal community is now faced with many new opportunities and uncertainties. With the open source dissemination platform and pervasive computing resources, new research results are being discovered at an unprecedented pace. In addition, the rapid exchange and influence of ideas across traditi… ▽ More With the transformative technologies and the rapidly changing global R&D landscape, the multimedia and multimodal community is now faced with many new opportunities and uncertainties. With the open source dissemination platform and pervasive computing resources, new research results are being discovered at an unprecedented pace. In addition, the rapid exchange and influence of ideas across traditional discipline boundaries have made the emphasis on multimedia multimodal research even more important than before. To seize these opportunities and respond to the challenges, we have organized a workshop to specifically address and brainstorm the challenges, opportunities, and research roadmaps for MM research. The two-day workshop, held on March 30 and 31, 2017 in Washington DC, was sponsored by the Information and Intelligent Systems Division of the National Science Foundation of the United States. Twenty-three (23) invited participants were asked to review and identify research areas in the MM field that are most important over the next 10-15 year timeframe. Important topics were selected through discussion and consensus, and then discussed in depth in breakout groups. Breakout groups reported initial discussion results to the whole group, who continued with further extensive deliberation. For each identified topic, a summary was produced after the workshop to describe the main findings, including the state of the art, challenges, and research roadmaps planned for the next 5, 10, and 15 years in the identified area. △ Less

Submitted 6 August, 2019; originally announced August 2019.

Comments: Long Report of NSF Workshop on Multimedia Challenges, Opportunities and Research Roadmaps, held in March 2017, Washington DC. Short report available separately

Costless: Optimizing Cost of Serverless Computing through Function Fusion and Placement

Authors: Tarek Elgamal, Atul Sandur, Klara Nahrstedt, Gul Agha

Abstract: Serverless computing has recently experienced significant adoption by several applications, especially Internet of Things (IoT) applications. In serverless computing, rather than deploying and managing dedicated virtual machines, users are able to deploy individual functions, and pay only for the time that their code is actually executing. However, since serverless platforms are relatively new, th… ▽ More Serverless computing has recently experienced significant adoption by several applications, especially Internet of Things (IoT) applications. In serverless computing, rather than deploying and managing dedicated virtual machines, users are able to deploy individual functions, and pay only for the time that their code is actually executing. However, since serverless platforms are relatively new, they have a completely different pricing model that depends on the memory, duration, and the number of executions of a sequence/workflow of functions. In this paper we present an algorithm that optimizes the price of serverless applications in AWS Lambda. We first describe the factors affecting price of serverless applications which include: (1) fusing a sequence of functions, (2) splitting functions across edge and cloud resources, and (3) allocating the memory for each function. We then present an efficient algorithm to explore different function fusion-placement solutions and find the solution that optimizes the application's price while keeping the latency under a certain threshold. Our results on image processing workflows show that the algorithm can find solutions optimizing the price by more than 35%-57% with only 5%-15% increase in latency. We also show that our algorithm can find non-trivial memory configurations that reduce both latency and price. △ Less

Submitted 23 November, 2018; originally announced November 2018.

Advanced Cyberinfrastructure for Science, Engineering, and Public Policy

Authors: Vasant G. Honavar, Katherine Yelick, Klara Nahrstedt, Holly Rushmeier, Jennifer Rexford, Mark D. Hill, Elizabeth Bradley, Elizabeth Mynatt

Abstract: Progress in many domains increasingly benefits from our ability to view the systems through a computational lens, i.e., using computational abstractions of the domains; and our ability to acquire, share, integrate, and analyze disparate types of data. These advances would not be possible without the advanced data and computational cyberinfrastructure and tools for data capture, integration, analys… ▽ More Progress in many domains increasingly benefits from our ability to view the systems through a computational lens, i.e., using computational abstractions of the domains; and our ability to acquire, share, integrate, and analyze disparate types of data. These advances would not be possible without the advanced data and computational cyberinfrastructure and tools for data capture, integration, analysis, modeling, and simulation. However, despite, and perhaps because of, advances in "big data" technologies for data acquisition, management and analytics, the other largely manual, and labor-intensive aspects of the decision making process, e.g., formulating questions, designing studies, organizing, curating, connecting, correlating and integrating crossdomain data, drawing inferences and interpreting results, have become the rate-limiting steps to progress. Advancing the capability and capacity for evidence-based improvements in science, engineering, and public policy requires support for (1) computational abstractions of the relevant domains coupled with computational methods and tools for their analysis, synthesis, simulation, visualization, sharing, and integration; (2) cognitive tools that leverage and extend the reach of human intellect, and partner with humans on all aspects of the activity; (3) nimble and trustworthy data cyber-infrastructures that connect, manage a variety of instruments, multiple interrelated data types and associated metadata, data representations, processes, protocols and workflows; and enforce applicable security and data access and use policies; and (4) organizational and social structures and processes for collaborative and coordinated activity across disciplinary and institutional boundaries. △ Less

Submitted 30 June, 2017; originally announced July 2017.

Comments: A Computing Community Consortium (CCC) white paper, 9 pages. arXiv admin note: text overlap with arXiv:1604.02006

Theseus: Incentivizing Truth Discovery in Mobile Crowd Sensing Systems

Authors: Haiming Jin, Lu Su, Klara Nahrstedt

Abstract: The recent proliferation of human-carried mobile devices has given rise to mobile crowd sensing (MCS) systems that outsource sensory data collection to the public crowd. In order to identify truthful values from (crowd) workers' noisy or even conflicting sensory data, truth discovery algorithms, which jointly estimate workers' data quality and the underlying truths through quality-aware data aggre… ▽ More The recent proliferation of human-carried mobile devices has given rise to mobile crowd sensing (MCS) systems that outsource sensory data collection to the public crowd. In order to identify truthful values from (crowd) workers' noisy or even conflicting sensory data, truth discovery algorithms, which jointly estimate workers' data quality and the underlying truths through quality-aware data aggregation, have drawn significant attention. However, the power of these algorithms could not be fully unleashed in MCS systems, unless workers' strategic reduction of their sensing effort is properly tackled. To address this issue, in this paper, we propose a payment mechanism, named Theseus, that deals with workers' such strategic behavior, and incentivizes high-effort sensing from workers. We ensure that, at the Bayesian Nash Equilibrium of the non-cooperative game induced by Theseus, all participating workers will spend their maximum possible effort on sensing, which improves their data quality. As a result, the aggregated results calculated subsequently by truth discovery algorithms based on workers' data will be highly accurate. Additionally, Theseus bears other desirable properties, including individual rationality and budget feasibility. We validate the desirable properties of Theseus through theoretical analysis, as well as extensive simulations. △ Less

Submitted 11 May, 2017; originally announced May 2017.

A Rural Lens on a Research Agenda for Intelligent Infrastructure

Authors: Ellen Zegura, Beki Grinter, Elizabeth Belding, Klara Nahrstedt

Abstract: A National Agenda for Intelligent Infrastructure is not complete without explicit consideration of the needs of rural communities. While the American population has urbanized, the United States depends on rural communities for agriculture, fishing, forestry, manufacturing and mining. Approximately 20% of the US population lives in rural areas with a skew towards aging adults. Further, nearly 25% o… ▽ More A National Agenda for Intelligent Infrastructure is not complete without explicit consideration of the needs of rural communities. While the American population has urbanized, the United States depends on rural communities for agriculture, fishing, forestry, manufacturing and mining. Approximately 20% of the US population lives in rural areas with a skew towards aging adults. Further, nearly 25% of Veterans live in rural America. And yet, when intelligent infrastructure is imagined, it is often done so with implicit or explicit bias towards cities. In this brief we describe the unique opportunities for rural communities and offer an inclusive vision of intelligent infrastructure research. In this paper, we argue for a set of coordinated actions to ensure that rural Americans are not left behind in this digital revolution. These technological platforms and applications, supported by appropriate policy, will address key issues in transportation, energy, agriculture, public safety and health. We believe that rather than being a set of needs, the rural United States presents a set of exciting possibilities for novel innovation benefiting not just those living there, but the American economy more broadly △ Less

Submitted 4 May, 2017; originally announced May 2017.

Comments: A Computing Community Consortium (CCC) white paper, 6 pages

City-Scale Intelligent Systems and Platforms

Authors: Klara Nahrstedt, Christos G. Cassandras, Charlie Catlett

Abstract: As of 2014, 54% of the earth's population resides in urban areas, and it is steadily increasing, expecting to reach 66% by 2050. Urban areas range from small cities with tens of thousands of people to megacities with greater than 10 million people. Roughly 12% of the global population today lives in 28 megacities, and at least 40 are projected by 2030. At these scales, the urban infrastructure suc… ▽ More As of 2014, 54% of the earth's population resides in urban areas, and it is steadily increasing, expecting to reach 66% by 2050. Urban areas range from small cities with tens of thousands of people to megacities with greater than 10 million people. Roughly 12% of the global population today lives in 28 megacities, and at least 40 are projected by 2030. At these scales, the urban infrastructure such as roads, buildings, and utility networks will cover areas as large as New England. This steady urbanization and the resulting expansion of infrastructure, combined with renewal of aging urban infrastructure, represent tens of trillion of dollars in new urban infrastructure investment over the coming decades. These investments must balance factors including impact on clean air and water, energy and maintenance costs, and the productivity and health of city dwellers. Moreover, cost-effective management and sustainability of these growing urban areas will be one of the most critical challenges to our society, motivating the concept of science- and data-driven urban design, retrofit, and operation-that is, "Smart Cities". △ Less

Submitted 4 May, 2017; originally announced May 2017.

Comments: A Computing Community Consortium (CCC) white paper, 8 pages

A National Research Agenda for Intelligent Infrastructure

Authors: Elizabeth Mynatt, Jennifer Clark, Greg Hager, Dan Lopresti, Greg Morrisett, Klara Nahrstedt, George Pappas, Shwetak Patel, Jennifer Rexford, Helen Wright, Ben Zorn

Abstract: Our infrastructure touches the day-to-day life of each of our fellow citizens, and its capabilities, integrity and sustainability are crucial to the overall competitiveness and prosperity of our country. Unfortunately, the current state of U.S. infrastructure is not good: the American Society of Civil Engineers' latest report on America's infrastructure ranked it at a D+ -- in need of $3.9 trillio… ▽ More Our infrastructure touches the day-to-day life of each of our fellow citizens, and its capabilities, integrity and sustainability are crucial to the overall competitiveness and prosperity of our country. Unfortunately, the current state of U.S. infrastructure is not good: the American Society of Civil Engineers' latest report on America's infrastructure ranked it at a D+ -- in need of $3.9 trillion in new investments. This dire situation constrains the growth of our economy, threatens our quality of life, and puts our global leadership at risk. The ASCE report called out three actions that need to be taken to address our infrastructure problem: 1) investment and planning in the system; 2) bold leadership by elected officials at the local and federal state; and 3) planning sustainability and resiliency in our infrastructure. While our immediate infrastructure needs are critical, it would be shortsighted to simply replicate more of what we have today. By doing so, we miss the opportunity to create Intelligent Infrastructure that will provide the foundation for increased safety and resilience, improved efficiencies and civic services, and broader economic opportunities and job growth. Indeed, our challenge is to proactively engage the declining, incumbent national infrastructure system and not merely repair it, but to enhance it; to create an internationally competitive cyber-physical system that provides an immediate opportunity for better services for citizens and that acts as a platform for a 21st century, high-tech economy and beyond. △ Less

Submitted 4 May, 2017; originally announced May 2017.

Comments: A Computing Community Consortium (CCC) white paper, 7 pages

Crowdsensing in Opportunistic Mobile Social Networks: A Context-aware and Human-centric Approach

Authors: Phuong Nguyen, Klara Nahrstedt

Abstract: In recent years, there have been efforts to collect human contact traces during social events (e.g., conferences) using Bluetooth devices (e.g., mobile phones, iMotes). The results of these studies have enabled the ability to do the crowd-sourcing task from within the crowd, in order to answer questions, such as: what is the current density of the crowd, or how many people are attending the event?… ▽ More In recent years, there have been efforts to collect human contact traces during social events (e.g., conferences) using Bluetooth devices (e.g., mobile phones, iMotes). The results of these studies have enabled the ability to do the crowd-sourcing task from within the crowd, in order to answer questions, such as: what is the current density of the crowd, or how many people are attending the event? However, in those studies, the sensing devices are usually distributed and configured in a certain manner. For example, the number of devices is fixed, people register for the devices on a volunteering basis. In this paper, we treat the above problem as an optimization problem and draw the connection to the vertex cover problem in graph theory. Since finding the optimal solution for minimum vertex cover problem is NP-complete, approximation algorithms have to be used. However, we will show that the well-known approximation algorithms do not perform well with the crowd-sensing task. In this paper, we propose the notions of node observability and coverage utility score and design a new context-aware approximation algorithm to find vertex cover that is tailored for crowd-sensing task. In addition, we design human-centric bootstrapping strategies to make initial assignment of sensing devices based on meta information about the participants (e.g., interests, friendship). The motivation is to assign the sensing task to a more "socialized" device to obtain better sensing coverage. We perform comprehensive experiments on real-world data traces obtained from previous experimental studies in conference and academic social context. The results show that our proposed approach significantly outperforms the baseline approximation algorithms in terms of sensing coverage. △ Less

Submitted 27 April, 2017; originally announced April 2017.

Comments: Long version of the IEEE MASS 2015 poster abstract titled "Context-aware Crowd-sensing in Opportunistic Mobile Social Network"

CENTURION: Incentivizing Multi-Requester Mobile Crowd Sensing

Authors: Haiming Jin, Lu Su, Klara Nahrstedt

Abstract: The recent proliferation of increasingly capable mobile devices has given rise to mobile crowd sensing (MCS) systems that outsource the collection of sensory data to a crowd of participating workers that carry various mobile devices. Aware of the paramount importance of effectively incentivizing participation in such systems, the research community has proposed a wide variety of incentive mechanis… ▽ More The recent proliferation of increasingly capable mobile devices has given rise to mobile crowd sensing (MCS) systems that outsource the collection of sensory data to a crowd of participating workers that carry various mobile devices. Aware of the paramount importance of effectively incentivizing participation in such systems, the research community has proposed a wide variety of incentive mechanisms. However, different from most of these existing mechanisms which assume the existence of only one data requester, we consider MCS systems with multiple data requesters, which are actually more common in practice. Specifically, our incentive mechanism is based on double auction, and is able to stimulate the participation of both data requesters and workers. In real practice, the incentive mechanism is typically not an isolated module, but interacts with the data aggregation mechanism that aggregates workers' data. For this reason, we propose CENTURION, a novel integrated framework for multi-requester MCS systems, consisting of the aforementioned incentive and data aggregation mechanism. CENTURION's incentive mechanism satisfies truthfulness, individual rationality, computational efficiency, as well as guaranteeing non-negative social welfare, and its data aggregation mechanism generates highly accurate aggregated results. The desirable properties of CENTURION are validated through both theoretical analysis and extensive simulations. △ Less

Submitted 5 January, 2017; originally announced January 2017.

Systems Computing Challenges in the Internet of Things

Authors: Rajeev Alur, Emery Berger, Ann W. Drobnis, Limor Fix, Kevin Fu, Gregory D. Hager, Daniel Lopresti, Klara Nahrstedt, Elizabeth Mynatt, Shwetak Patel, Jennifer Rexford, John A. Stankovic, Benjamin Zorn

Abstract: A recent McKinsey report estimates the economic impact of the Internet of Things (IoT) to be between $3.9 to $11 trillion dollars by 20251 . IoT has the potential to have a profound impact on our daily lives, including technologies for the home, for health, for transportation, and for managing our natural resources. The Internet was largely driven by information and ideas generated by people, but… ▽ More A recent McKinsey report estimates the economic impact of the Internet of Things (IoT) to be between $3.9 to $11 trillion dollars by 20251 . IoT has the potential to have a profound impact on our daily lives, including technologies for the home, for health, for transportation, and for managing our natural resources. The Internet was largely driven by information and ideas generated by people, but advances in sensing and hardware have enabled computers to more easily observe the physical world. Coupling this additional layer of information with advances in machine learning brings dramatic new capabilities including the ability to capture and process tremendous amounts of data; to predict behaviors, activities, and the future in uncanny ways; and to manipulate the physical world in response. This trend will fundamentally change how people interact with physical objects and the environment. Success in developing value-added capabilities around IoT requires a broad approach that includes expertise in sensing and hardware, machine learning, networked systems, human-computer interaction, security, and privacy. Strategies for making IoT practical and spurring its ultimate adoption also require a multifaceted approach that often transcends technology, such as with concerns over data security, privacy, public policy, and regulatory issues. In this paper we argue that existing best practices in building robust and secure systems are insufficient to address the new challenges that IoT systems will present. We provide recommendations regarding investments in research areas that will help address inadequacies in existing systems, practices, tools, and policies. △ Less

Submitted 11 April, 2016; originally announced April 2016.

Comments: A Computing Community Consortium (CCC) white paper, 15 pages

Smart Communities Internet of Things

Authors: Klara Nahrstedt, Daniel Lopresti, Ben Zorn, Ann W. Drobnis, Beth Mynatt, Shwetak Patel, Helen V. Wright

Abstract: Today's cities face many challenges due to population growth, aging population, pedestrian and vehicular traffic congestion, water usage increase, increased electricity demands, crumbling physical infrastructure of buildings, roads, water sewage, power grid, and declining health care services. Moreover, major trends indicate the global urbanization of society, and the associated pressures it bring… ▽ More Today's cities face many challenges due to population growth, aging population, pedestrian and vehicular traffic congestion, water usage increase, increased electricity demands, crumbling physical infrastructure of buildings, roads, water sewage, power grid, and declining health care services. Moreover, major trends indicate the global urbanization of society, and the associated pressures it brings, will continue to accelerate. One of the approaches to assist in solving some of the challenges is to deploy extensive IT technology. It has been recognized that cyber-technology plays a key role in improving quality of people's lives, strengthening business and helping government agencies serve citizens better. In this white paper, we discuss the benefits and challenges of cyber-technologies within "Smart Cities", especially the IoT (Internet of Things) for smart communities, which means considering the benefits and challenges of IoT cyber-technologies on smart cities physical infrastructures and their human stakeholders. To point out the IoT challenges, we will first present the framework within which IoT lives, and then proceed with the challenges, conclusions and recommendations. △ Less

Submitted 7 April, 2016; originally announced April 2016.

Comments: A Computing Community Consortium (CCC) white paper, 9 pages