-
Fast and Accurate Relative Motion Tracking for Two Industrial Robots
Authors:
Honglu He,
Chen-lung Lu,
Glenn Saunders,
Pinghai Yang,
Jeffrey Schoonover,
John Wason,
Santiago Paternain,
Agung Julius,
John T. Wen
Abstract:
Industrial robotic applications such as spraying, welding, and additive manufacturing frequently require fast, accurate, and uniform motion along a 3D spatial curve. To increase process throughput, some manufacturers propose a dual-robot setup to overcome the speed limitation of a single robot. Industrial robot motion is programmed through waypoints connected by motion primitives (Cartesian linear…
▽ More
Industrial robotic applications such as spraying, welding, and additive manufacturing frequently require fast, accurate, and uniform motion along a 3D spatial curve. To increase process throughput, some manufacturers propose a dual-robot setup to overcome the speed limitation of a single robot. Industrial robot motion is programmed through waypoints connected by motion primitives (Cartesian linear and circular paths and linear joint paths at constant Cartesian speed). The actual robot motion is affected by the blending between these motion primitives and the pose of the robot (an outstretched/close to singularity pose tends to have larger path-tracking errors). Choosing the waypoints and the speed along each motion segment to achieve the performance requirement is challenging. At present, there is no automated solution, and laborious manual tuning by robot experts is needed to approach the desired performance. In this paper, we present a systematic three-step approach to designing and programming a dual-robot system to optimize system performance. The first step is to select the relative placement between the two robots based on the specified relative motion path. The second step is to select the relative waypoints and the motion primitives. The final step is to update the waypoints iteratively based on the actual relative motion. Waypoint iteration is first executed in simulation and then completed using the actual robots. For performance measures, we use the mean path speed subject to the relative position and orientation constraints and the path speed uniformity constraint. We have demonstrated the effectiveness of this method with ABB and FANUC robots on two challenging test curves. The performance improvement over the current industrial practice baseline is over 300%. Compared to the optimized single-arm case that we have previously reported, the improvement is over 14%.
△ Less
Submitted 9 April, 2024;
originally announced April 2024.
-
High-Speed High-Accuracy Spatial Curve Tracking Using Motion Primitives in Industrial Robots
Authors:
Honglu He,
Chen-lung Lu,
Yunshi Wen,
Glenn Saunders,
Pinghai Yang,
Jeffrey Schoonover,
Agung Julius,
John T. Wen
Abstract:
Industrial robots are increasingly deployed in applications requiring an end effector tool to closely track a specified path, such as in spraying and welding. Performance and productivity present possibly conflicting objectives: tracking accuracy, path speed, and motion uniformity. Industrial robots are programmed through motion primitives consisting of waypoints connected by pre-defined motion se…
▽ More
Industrial robots are increasingly deployed in applications requiring an end effector tool to closely track a specified path, such as in spraying and welding. Performance and productivity present possibly conflicting objectives: tracking accuracy, path speed, and motion uniformity. Industrial robots are programmed through motion primitives consisting of waypoints connected by pre-defined motion segments, with specified parameters such as path speed and blending zone. The actual executed robot motion depends on the robot joint servo controller and joint motion constraints (velocity, acceleration, etc.) which are largely unknown to the users. Programming a robot to achieve the desired performance today is time-consuming and mostly manual, requiring tuning a large number of coupled parameters in the motion primitives. The performance also depends on the choice of additional parameters: possible redundant degrees of freedom, location of the target curve, and the robot configuration. This paper presents a systematic approach to optimize the robot motion primitives for performance. The approach first selects the static parameters, then the motion primitives, and finally iteratively update the waypoints to minimize the tracking error. The ultimate performance objective is to maximize the path speed subject to the tracking accuracy and speed uniformity constraints over the entire path. We have demonstrated the effectiveness of this approach in simulation for ABB and FANUC robots for two challenging example curves, and experimentally for an ABB robot. Comparing with the baseline using the current industry practice, the optimized performance shows over 200% performance improvement.
△ Less
Submitted 5 January, 2023;
originally announced January 2023.
-
Neuro-symbolic Models for Interpretable Time Series Classification using Temporal Logic Description
Authors:
Ruixuan Yan,
Tengfei Ma,
Achille Fokoue,
Maria Chang,
Agung Julius
Abstract:
Most existing Time series classification (TSC) models lack interpretability and are difficult to inspect. Interpretable machine learning models can aid in discovering patterns in data as well as give easy-to-understand insights to domain specialists. In this study, we present Neuro-Symbolic Time Series Classification (NSTSC), a neuro-symbolic model that leverages signal temporal logic (STL) and ne…
▽ More
Most existing Time series classification (TSC) models lack interpretability and are difficult to inspect. Interpretable machine learning models can aid in discovering patterns in data as well as give easy-to-understand insights to domain specialists. In this study, we present Neuro-Symbolic Time Series Classification (NSTSC), a neuro-symbolic model that leverages signal temporal logic (STL) and neural network (NN) to accomplish TSC tasks using multi-view data representation and expresses the model as a human-readable, interpretable formula. In NSTSC, each neuron is linked to a symbolic expression, i.e., an STL (sub)formula. The output of NSTSC is thus interpretable as an STL formula akin to natural language, describing temporal and logical relations hidden in the data. We propose an NSTSC-based classifier that adopts a decision-tree approach to learn formula structures and accomplish a multiclass TSC task. The proposed smooth activation functions for wSTL allow the model to be learned in an end-to-end fashion. We test NSTSC on a real-world wound healing dataset from mice and benchmark datasets from the UCR time-series repository, demonstrating that NSTSC achieves comparable performance with the state-of-the-art models. Furthermore, NSTSC can generate interpretable formulas that match with domain knowledge.
△ Less
Submitted 15 September, 2022;
originally announced September 2022.
-
Weighted Graph-Based Signal Temporal Logic Inference Using Neural Networks
Authors:
Nasim Baharisangari,
Kazuma Hirota,
Ruixuan Yan,
Agung Julius,
Zhe Xu
Abstract:
Extracting spatial-temporal knowledge from data is useful in many applications. It is important that the obtained knowledge is human-interpretable and amenable to formal analysis. In this paper, we propose a method that trains neural networks to learn spatial-temporal properties in the form of weighted graph-based signal temporal logic (wGSTL) formulas. For learning wGSTL formulas, we introduce a…
▽ More
Extracting spatial-temporal knowledge from data is useful in many applications. It is important that the obtained knowledge is human-interpretable and amenable to formal analysis. In this paper, we propose a method that trains neural networks to learn spatial-temporal properties in the form of weighted graph-based signal temporal logic (wGSTL) formulas. For learning wGSTL formulas, we introduce a flexible wGSTL formula structure in which the user's preference can be applied in the inferred wGSTL formulas. In the proposed framework, each neuron of the neural networks corresponds to a subformula in a flexible wGSTL formula structure. We initially train a neural network to learn the wGSTL operators and then train a second neural network to learn the parameters in a flexible wGSTL formula structure. We use a COVID-19 dataset and a rain prediction dataset to evaluate the performance of the proposed framework and algorithms. We compare the performance of the proposed framework with three baseline classification methods including K-nearest neighbors, decision trees, support vector machine, and artificial neural networks. The classification accuracy obtained by the proposed framework is comparable with the baseline classification methods.
△ Less
Submitted 6 January, 2022; v1 submitted 16 September, 2021;
originally announced September 2021.
-
Neuradicon: operational representation learning of neuroimaging reports
Authors:
Henry Watkins,
Robert Gray,
Adam Julius,
Yee-Haur Mah,
Walter H. L. Pinaya,
Paul Wright,
Ashwani Jha,
Holger Engleitner,
Jorge Cardoso,
Sebastien Ourselin,
Geraint Rees,
Rolf Jaeger,
Parashkev Nachev
Abstract:
Radiological reports typically summarize the content and interpretation of imaging studies in unstructured form that precludes quantitative analysis. This limits the monitoring of radiological services to throughput undifferentiated by content, impeding specific, targeted operational optimization. Here we present Neuradicon, a natural language processing (NLP) framework for quantitative analysis o…
▽ More
Radiological reports typically summarize the content and interpretation of imaging studies in unstructured form that precludes quantitative analysis. This limits the monitoring of radiological services to throughput undifferentiated by content, impeding specific, targeted operational optimization. Here we present Neuradicon, a natural language processing (NLP) framework for quantitative analysis of neuroradiological reports. Our framework is a hybrid of rule-based and artificial intelligence models to represent neurological reports in succinct, quantitative form optimally suited to operational guidance. We demonstrate the application of Neuradicon to operational phenotyping of a corpus of 336,569 reports, and report excellent generalizability across time and two independent healthcare institutions.
△ Less
Submitted 27 November, 2023; v1 submitted 21 July, 2021;
originally announced July 2021.
-
Neural Network for Weighted Signal Temporal Logic
Authors:
Ruixuan Yan,
Agung Julius
Abstract:
In this paper, we propose a neuro-symbolic framework called weighted Signal Temporal Logic Neural Network (wSTL-NN) that combines the characteristics of neural networks and temporal logics. Weighted Signal Temporal Logic (wSTL) formulas are recursively composed of subformulas that are combined using logical and temporal operators. The quantitative semantics of wSTL is defined such that the quantit…
▽ More
In this paper, we propose a neuro-symbolic framework called weighted Signal Temporal Logic Neural Network (wSTL-NN) that combines the characteristics of neural networks and temporal logics. Weighted Signal Temporal Logic (wSTL) formulas are recursively composed of subformulas that are combined using logical and temporal operators. The quantitative semantics of wSTL is defined such that the quantitative satisfaction of subformulas with higher weights has more influence on the quantitative satisfaction of the overall wSTL formula. In the wSTL-NN, each neuron corresponds to a wSTL subformula, and its output corresponds to the quantitative satisfaction of the formula. We use wSTL-NN to represent wSTL formulas as features to classify time series data. STL features are more explainable than those used in classical methods. The wSTL-NN is end-to-end differentiable, which allows learning of wSTL formulas to be done using back-propagation. To reduce the number of weights, we introduce two techniques to sparsify the wSTL-NN.We apply our framework to an occupancy detection time-series dataset to learn a classifier that predicts the occupancy status of an office room.
△ Less
Submitted 8 April, 2021;
originally announced April 2021.
-
Temporal Logic Inference for Hybrid System Observation with Spatial and Temporal Uncertainties
Authors:
Zhe Xu,
Yi Deng,
Agung Julius
Abstract:
In this paper, we present a mechanism for building hybrid system observers to differentiate between specific positions of the hybrid system. The mechanism is designed through inferring metric temporal logic (MTL) formulae from simulated trajectories from the hybrid system. We first approximate the system behavior by simulating finitely many trajectories with timerobust tube segments around them. T…
▽ More
In this paper, we present a mechanism for building hybrid system observers to differentiate between specific positions of the hybrid system. The mechanism is designed through inferring metric temporal logic (MTL) formulae from simulated trajectories from the hybrid system. We first approximate the system behavior by simulating finitely many trajectories with timerobust tube segments around them. These time-robust tube segments account for both spatial and temporal uncertainties that exist in the hybrid system with initial state variations. The inferred MTL formulae classify different time-robust tube segments and thus can be used for classifying the hybrid system behaviors in a provably correct fashion. We implement our approach on a model of a smart building testbed to distinguish two cases of room occupancy.
△ Less
Submitted 4 April, 2020;
originally announced April 2020.
-
Controller Synthesis of Wind Turbine Generator and Energy Storage System with Stochastic Wind Variations under Temporal Logic Specifications
Authors:
Zhe Xu,
Agung Julius,
Ufuk Topcu,
Joe H. Chow
Abstract:
In this paper, we present a controller synthesis approach for wind turbine generators (WTG) and energy storage systems with metric temporal logic (MTL) specifications, with provable probabilistic guarantees in the stochastic environment of wind power generation. The MTL specifications are requirements for the grid frequency deviations, WTG rotor speed variations and the power flow constraints at d…
▽ More
In this paper, we present a controller synthesis approach for wind turbine generators (WTG) and energy storage systems with metric temporal logic (MTL) specifications, with provable probabilistic guarantees in the stochastic environment of wind power generation. The MTL specifications are requirements for the grid frequency deviations, WTG rotor speed variations and the power flow constraints at different lines. We present the stochastic control bisimulation function, which bounds the divergence of the trajectories of a switched stochastic control system and the switched nominal control system in a probabilistic fashion.We first design a feedforward controller by solving an optimization problem for the nominal trajectory of the deterministic control system with robustness against initial state variations and stochastic uncertainties. Then we generate a feedback control law from the data of the simulated trajectories. We implement our control method on both a four-bus system and a nine-bus system, and test the effectiveness of the method with a generation loss disturbance. We also test the advantage of the feedback controller over the feedforward controller when unexpected disturbance occurs.
△ Less
Submitted 26 November, 2019;
originally announced November 2019.
-
Graph Temporal Logic Inference for Classification and Identification
Authors:
Zhe Xu,
Alexander J Nettekoven,
A. Agung Julius,
Ufuk Topcu
Abstract:
Inferring spatial-temporal properties from data is important for many complex systems, such as additive manufacturing systems, swarm robotic systems and biological networks. Such systems can often be modeled as a labeled graph where labels on the nodes and edges represent relevant measurements such as temperatures and distances. We introduce graph temporal logic (GTL) which can express properties…
▽ More
Inferring spatial-temporal properties from data is important for many complex systems, such as additive manufacturing systems, swarm robotic systems and biological networks. Such systems can often be modeled as a labeled graph where labels on the nodes and edges represent relevant measurements such as temperatures and distances. We introduce graph temporal logic (GTL) which can express properties such as "whenever a node's label is above 10, for the next 3 time units there are always at least two neighboring nodes with an edge label of at most 2 where the node labels are above 5". This paper is a first attempt to infer spatial (graph) temporal logic formulas from data for classification and identification. For classification, we infer a GTL formula that classifies two sets of graph temporal trajectories with minimal misclassification rate. For identification, we infer a GTL formula that is informative and is satisfied by the graph temporal trajectories in the dataset with high probability. The informativeness of a GTL formula is measured by the information gain with respect to given prior knowledge represented by a prior probability distribution. We implement the proposed approach to classify the graph patterns of tensile specimens built from selective laser sintering (SLS) process with varying strengths, and to identify informative spatial-temporal patterns from experimental data of the SLS cooldown process and simulation data of a swarm of robots.
△ Less
Submitted 22 March, 2019;
originally announced March 2019.
-
Information-Guided Temporal Logic Inference with Prior Knowledge
Authors:
Zhe Xu,
Melkior Ornik,
A. Agung Julius,
Ufuk Topcu
Abstract:
This paper investigates the problem of inferring knowledge from data so that the inferred knowledge is interpretable and informative to humans who have prior knowledge. Given a dataset as a collection of system trajectories, we infer parametric linear temporal logic (pLTL) formulas that are informative and satisfied by the trajectories in the dataset with high probability. The informativeness of t…
▽ More
This paper investigates the problem of inferring knowledge from data so that the inferred knowledge is interpretable and informative to humans who have prior knowledge. Given a dataset as a collection of system trajectories, we infer parametric linear temporal logic (pLTL) formulas that are informative and satisfied by the trajectories in the dataset with high probability. The informativeness of the inferred formula is measured by the information gain with respect to given prior knowledge represented by a prior probability distribution. We first present two algorithms to compute the information gain with a focus on two types of prior probability distributions: stationary probability distributions and probability distributions expressed by discrete time Markov chains. Then we provide a method to solve the inference problem for a subset of pLTL formulas with polynomial time complexity with respect to the number of Boolean connectives in the formula. We provide implementations of the proposed approach on explaining anomalous patterns, patterns changes and explaining the policies of Markov decision processes.
△ Less
Submitted 21 November, 2018;
originally announced November 2018.
-
Census Signal Temporal Logic Inference for Multi-Agent Group Behavior Analysis
Authors:
Zhe Xu,
Agung Julius
Abstract:
In this paper, we define a novel census signal temporal logic (CensusSTL) that focuses on the number of agents in different subsets of a group that complete a certain task specified by the signal temporal logic (STL). CensusSTL consists of an "inner logic" STL formula and an "outer logic" STL formula. We present a new inference algorithm to infer CensusSTL formulae from the trajectory data of a gr…
▽ More
In this paper, we define a novel census signal temporal logic (CensusSTL) that focuses on the number of agents in different subsets of a group that complete a certain task specified by the signal temporal logic (STL). CensusSTL consists of an "inner logic" STL formula and an "outer logic" STL formula. We present a new inference algorithm to infer CensusSTL formulae from the trajectory data of a group of agents. We first identify the "inner logic" STL formula and then infer the subgroups based on whether the agents' behaviors satisfy the "inner logic" formula at each time point. We use two different approaches to infer the subgroups based on similarity and complementarity, respectively. The "outer logic" CensusSTL formula is inferred from the census trajectories of different subgroups. We apply the algorithm in analyzing data from a soccer match by inferring the CensusSTL formula for different subgroups of a soccer team.
△ Less
Submitted 5 October, 2016;
originally announced October 2016.
-
An MILP Approach for Real-time Optimal Controller Synthesis with Metric Temporal Logic Specifications
Authors:
Sayan Saha,
A. Agung Julius
Abstract:
The fundamental idea of this work is to synthesize reactive controllers such that closed-loop execution trajectories of the system satisfy desired specifications that ensure correct system behaviors, while optimizing a desired performance criteria. In our approach, the correctness of a system's behavior can be defined according to the system's relation to the environment, for example, the output t…
▽ More
The fundamental idea of this work is to synthesize reactive controllers such that closed-loop execution trajectories of the system satisfy desired specifications that ensure correct system behaviors, while optimizing a desired performance criteria. In our approach, the correctness of a system's behavior can be defined according to the system's relation to the environment, for example, the output trajectories of the system terminate in a goal set without entering an unsafe set. Using Metric Temporal Logic (MTL) specifications we can further capture complex system behaviors and timing requirements, such as the output trajectories must pass through a number of way-points within a certain time frame before terminating in the goal set. Given a Mixed Logical Dynamical (MLD) system and system specifications in terms of MTL formula or simpler reach-avoid specifications, our goal is to find a closed-loop trajectory that satisfies the specifications, in non-deterministic environments. Using an MILP framework we search over the space of input signals to obtain such valid trajectories of the system, by adding constraints to satisfy the MTL formula only when necessary, to avoid the exponential complexity of solving MILP problems. We also present experimental results for planning a path for a mobile robot through a dynamically changing environment with a desired task specification.
△ Less
Submitted 8 March, 2016;
originally announced March 2016.
-
Tracking Tetrahymena Pyriformis Cells using Decision Trees
Authors:
Quan Wang,
Yan Ou,
A. Agung Julius,
Kim L. Boyer,
Min Jun Kim
Abstract:
Matching cells over time has long been the most difficult step in cell tracking. In this paper, we approach this problem by recasting it as a classification problem. We construct a feature set for each cell, and compute a feature difference vector between a cell in the current frame and a cell in a previous frame. Then we determine whether the two cells represent the same cell over time by trainin…
▽ More
Matching cells over time has long been the most difficult step in cell tracking. In this paper, we approach this problem by recasting it as a classification problem. We construct a feature set for each cell, and compute a feature difference vector between a cell in the current frame and a cell in a previous frame. Then we determine whether the two cells represent the same cell over time by training decision trees as our binary classifiers. With the output of decision trees, we are able to formulate an assignment problem for our cell association task and solve it using a modified version of the Hungarian algorithm.
△ Less
Submitted 12 July, 2012;
originally announced July 2012.
