-
Architecting Intermediate Layers for Efficient Composition of Data Management and Machine Learning Systems
Authors:
Supun Abeysinghe,
Fei Wang,
Gregory Essertel,
Tiark Rompf
Abstract:
Modern data analytics workloads combine relational data processing with machine learning (ML). Most DBMS handle these workloads by offloading these ML operations to external specialized ML systems. While both DBMS and ML systems go to great lengths to optimize performance for their specific workloads, significant performance is lost when used in combination, due to data movement across system boun…
▽ More
Modern data analytics workloads combine relational data processing with machine learning (ML). Most DBMS handle these workloads by offloading these ML operations to external specialized ML systems. While both DBMS and ML systems go to great lengths to optimize performance for their specific workloads, significant performance is lost when used in combination, due to data movement across system boundaries, conversions between incompatible internal data formats, and the lack of cross system optimizations.
A key idea to remove these bottlenecks is to integrate existing data manipulation systems with ML systems by building a common intermediate layer (IR). Although this idea has been explored before (Weld, Delite), previous such attempts require significant re-engineering of prior systems and still fall short in achieving best-of-breed performance for individual tasks (e.g., SQL, Deep Learning). Specifically, they rely on re-implementing existing systems using a generic set of operators and fail to match best-of-breed individual performance due to the inability to recover high-level optimizations from this generic IR through compiler analysis.
We present Flern, the first intermediate-layer integration between DB and ML systems that are best-of-breed individually, competitive with the best compiled query engines such as HyPer on comprehensive relational benchmarks (TPC-H) and competitive with TensorFlow and PyTorch in state-of-the-art ML models (e.g., DeepSpeech, SqueezeNet, Transformers) and also represents a new state-of-the-art for integration. A key realization is to architect intermediate layers based on generative programming capabilities, which preserves high-level contextual information for cross optimizations and enables the construction of a variety of complex structures and cross system optimizations with minimal effort.
△ Less
Submitted 5 November, 2023;
originally announced November 2023.
-
Graph IRs for Impure Higher-Order Languages (Technical Report)
Authors:
Oliver Bračevac,
Guannan Wei,
Songlin Jia,
Supun Abeysinghe,
Yuxuan Jiang,
Yuyan Bao,
Tiark Rompf
Abstract:
This is a companion report for the OOPSLA 2023 paper of the same title, presenting a detailed end-to-end account of the $λ^*_{\mathsf{G}}$ graph IR, at a level of detail beyond a regular conference paper. Our first concern is adequacy and soundness of $λ^*_{\mathsf{G}}$, which we derive from a direct-style imperative functional language (a variant of Bao et al.'s $λ^*$-calculus with reachability t…
▽ More
This is a companion report for the OOPSLA 2023 paper of the same title, presenting a detailed end-to-end account of the $λ^*_{\mathsf{G}}$ graph IR, at a level of detail beyond a regular conference paper. Our first concern is adequacy and soundness of $λ^*_{\mathsf{G}}$, which we derive from a direct-style imperative functional language (a variant of Bao et al.'s $λ^*$-calculus with reachability types and a simple effect system) by a series of type-preserving translations into a calculus in monadic normalform (MNF). Static reachability types and effects entirely inform $λ^*_{\mathsf{G}}$'s dependency synthesis. We argue for its adequacy by proving its functional properties along with dependency safety via progress and preservation lemmas with respect to a notion of call-by-value (CBV) reduction that checks the observed order of effects.
Our second concern is establishing the correctness of $λ^*_{\mathsf{G}}$'s equational rules that drive compiler optimizations (e.g., DCE, $λ$-hoisting, etc.), by proving contextual equivalence using logical relations. A key insight is that the functional properties of dependency synthesis permit a logical relation on $λ^*_{\mathsf{G}}$ in MNF in terms of previously developed logical relations for the direct-style $λ^*$-calculus.
Finally, we also include a longer version of the conference paper's section on code generation and code motion for $λ^*_{\mathsf{G}}$ as implemented in Scala~LMS.
△ Less
Submitted 14 September, 2023;
originally announced September 2023.
-
Gated Recurrent Neural Network Approach for Multilabel Emotion Detection in Microblogs
Authors:
Prabod Rathnayaka,
Supun Abeysinghe,
Chamod Samarajeewa,
Isura Manchanayake,
Malaka J. Walpola,
Rashmika Nawaratne,
Tharindu Bandaragoda,
Damminda Alahakoon
Abstract:
People express their opinions and emotions freely in social media posts and online reviews that contain valuable feedback for multiple stakeholders such as businesses and political campaigns. Manually extracting opinions and emotions from large volumes of such posts is an impossible task. Therefore, automated processing of these posts to extract opinions and emotions is an important research probl…
▽ More
People express their opinions and emotions freely in social media posts and online reviews that contain valuable feedback for multiple stakeholders such as businesses and political campaigns. Manually extracting opinions and emotions from large volumes of such posts is an impossible task. Therefore, automated processing of these posts to extract opinions and emotions is an important research problem. However, human emotion detection is a challenging task due to the complexity and nuanced nature. To overcome these barriers, researchers have extensively used techniques such as deep learning, distant supervision, and transfer learning. In this paper, we propose a novel Pyramid Attention Network (PAN) based model for emotion detection in microblogs. The main advantage of our approach is that PAN has the capability to evaluate sentences in different perspectives to capture multiple emotions existing in a single text. The proposed model was evaluated on a recently released dataset and the results achieved the state-of-the-art accuracy of 58.9%.
△ Less
Submitted 17 July, 2019;
originally announced July 2019.
-
Enhancing Decision Making Capacity in Tourism Domain Using Social Media Analytics
Authors:
Supun Abeysinghe,
Isura Manchanayake,
Chamod Samarajeewa,
Prabod Rathnayaka,
Malaka J. Walpola,
Rashmika Nawaratne,
Tharindu Bandaragoda,
Damminda Alahakoon
Abstract:
Social media has gained an immense popularity over the last decade. People tend to express opinions about their daily encounters on social media freely. These daily encounters include the places they traveled, hotels or restaurants they have tried and aspects related to tourism in general. Since people usually express their true experiences on social media, the expressed opinions contain valuable…
▽ More
Social media has gained an immense popularity over the last decade. People tend to express opinions about their daily encounters on social media freely. These daily encounters include the places they traveled, hotels or restaurants they have tried and aspects related to tourism in general. Since people usually express their true experiences on social media, the expressed opinions contain valuable information that can be used to generate business value and aid in decision-making processes. Due to the large volume of data, it is not a feasible task to manually go through each and every item and extract the information. Hence, we propose a social media analytics platform which has the capability to identify discussion pathways and aspects with their corresponding sentiment and deeper emotions using machine learning techniques and a visualization tool which shows the extracted insights in a comprehensible and concise manner. Identified topic pathways and aspects will give a decision maker some insight into what are the most discussed topics about the entity whereas associated sentiments and emotions will help to identify the feedback.
△ Less
Submitted 19 December, 2018;
originally announced December 2018.
-
Sentylic at IEST 2018: Gated Recurrent Neural Network and Capsule Network Based Approach for Implicit Emotion Detection
Authors:
Prabod Rathnayaka,
Supun Abeysinghe,
Chamod Samarajeewa,
Isura Manchanayake,
Malaka Walpola
Abstract:
In this paper, we present the system we have used for the Implicit WASSA 2018 Implicit Emotion Shared Task. The task is to predict the emotion of a tweet of which the explicit mentions of emotion terms have been removed. The idea is to come up with a model which has the ability to implicitly identify the emotion expressed given the context words. We have used a Gated Recurrent Neural Network (GRU)…
▽ More
In this paper, we present the system we have used for the Implicit WASSA 2018 Implicit Emotion Shared Task. The task is to predict the emotion of a tweet of which the explicit mentions of emotion terms have been removed. The idea is to come up with a model which has the ability to implicitly identify the emotion expressed given the context words. We have used a Gated Recurrent Neural Network (GRU) and a Capsule Network based model for the task. Pre-trained word embeddings have been utilized to incorporate contextual knowledge about words into the model. GRU layer learns latent representations using the input word embeddings. Subsequent Capsule Network layer learns high-level features from that hidden representation. The proposed model managed to achieve a macro-F1 score of 0.692.
△ Less
Submitted 5 September, 2018;
originally announced September 2018.
