-
Fisher-Weighted Merge of Contrastive Learning Models in Sequential Recommendation
Authors:
Jung Hyun Ryu,
Jaeheyoung Jeon,
Jewoong Cho,
Myungjoo Kang 1
Abstract:
Along with the exponential growth of online platforms and services, recommendation systems have become essential for identifying relevant items based on user preferences. The domain of sequential recommendation aims to capture evolving user preferences over time. To address dynamic preference, various contrastive learning methods have been proposed to target data sparsity, a challenge in recommend…
▽ More
Along with the exponential growth of online platforms and services, recommendation systems have become essential for identifying relevant items based on user preferences. The domain of sequential recommendation aims to capture evolving user preferences over time. To address dynamic preference, various contrastive learning methods have been proposed to target data sparsity, a challenge in recommendation systems due to the limited user-item interactions. In this paper, we are the first to apply the Fisher-Merging method to Sequential Recommendation, addressing and resolving practical challenges associated with it. This approach ensures robust fine-tuning by merging the parameters of multiple models, resulting in improved overall performance. Through extensive experiments, we demonstrate the effectiveness of our proposed methods, highlighting their potential to advance the state-of-the-art in sequential learning and recommendation systems.
△ Less
Submitted 5 July, 2023;
originally announced July 2023.
-
AdaptiveRec: Adaptively Construct Pairs for Contrastive Learning in Sequential Recommendation
Authors:
Jaeheyoung Jeon,
Jung Hyun Ryu,
Jewoong Cho,
Myungjoo Kang
Abstract:
This paper presents a solution to the challenges faced by contrastive learning in sequential recommendation systems. In particular, it addresses the issue of false negative, which limits the effectiveness of recommendation algorithms. By introducing an advanced approach to contrastive learning, the proposed method improves the quality of item embeddings and mitigates the problem of falsely categor…
▽ More
This paper presents a solution to the challenges faced by contrastive learning in sequential recommendation systems. In particular, it addresses the issue of false negative, which limits the effectiveness of recommendation algorithms. By introducing an advanced approach to contrastive learning, the proposed method improves the quality of item embeddings and mitigates the problem of falsely categorizing similar instances as dissimilar. Experimental results demonstrate performance enhancements compared to existing systems. The flexibility and applicability of the proposed approach across various recommendation scenarios further highlight its value in enhancing sequential recommendation systems.
△ Less
Submitted 7 July, 2023;
originally announced July 2023.
-
AnomalyBERT: Self-Supervised Transformer for Time Series Anomaly Detection using Data Degradation Scheme
Authors:
Yungi Jeong,
Eunseok Yang,
Jung Hyun Ryu,
Imseong Park,
Myungjoo Kang
Abstract:
Mechanical defects in real situations affect observation values and cause abnormalities in multivariate time series, such as sensor values or network data. To perceive abnormalities in such data, it is crucial to understand the temporal context and interrelation between variables simultaneously. The anomaly detection task for time series, especially for unlabeled data, has been a challenging probl…
▽ More
Mechanical defects in real situations affect observation values and cause abnormalities in multivariate time series, such as sensor values or network data. To perceive abnormalities in such data, it is crucial to understand the temporal context and interrelation between variables simultaneously. The anomaly detection task for time series, especially for unlabeled data, has been a challenging problem, and we address it by applying a suitable data degradation scheme to self-supervised model training. We define four types of synthetic outliers and propose the degradation scheme in which a portion of input data is replaced with one of the synthetic outliers. Inspired by the self-attention mechanism, we design a Transformer-based architecture to recognize the temporal context and detect unnatural sequences with high efficiency. Our model converts multivariate data points into temporal representations with relative position bias and yields anomaly scores from these representations. Our method, AnomalyBERT, shows a great capability of detecting anomalies contained in complex time series and surpasses previous state-of-the-art methods on five real-world benchmarks. Our code is available at https://github.com/Jhryu30/AnomalyBERT.
△ Less
Submitted 8 May, 2023;
originally announced May 2023.
-
Meta-Learning with Adaptive Weighted Loss for Imbalanced Cold-Start Recommendation
Authors:
Minchang Kim,
Yongjin Yang,
Jung Hyun Ryu,
Taesup Kim
Abstract:
Sequential recommenders have made great strides in capturing a user's preferences. Nevertheless, the cold-start recommendation remains a fundamental challenge as they typically involve limited user-item interactions for personalization. Recently, gradient-based meta-learning approaches have emerged in the sequential recommendation field due to their fast adaptation and easy-to-integrate abilities.…
▽ More
Sequential recommenders have made great strides in capturing a user's preferences. Nevertheless, the cold-start recommendation remains a fundamental challenge as they typically involve limited user-item interactions for personalization. Recently, gradient-based meta-learning approaches have emerged in the sequential recommendation field due to their fast adaptation and easy-to-integrate abilities. The meta-learning algorithms formulate the cold-start recommendation as a few-shot learning problem, where each user is represented as a task to be adapted. While meta-learning algorithms generally assume that task-wise samples are evenly distributed over classes or values, user-item interactions in real-world applications do not conform to such a distribution (e.g., watching favorite videos multiple times, leaving only positive ratings without any negative ones). Consequently, imbalanced user feedback, which accounts for the majority of task training data, may dominate the user adaptation process and prevent meta-learning algorithms from learning meaningful meta-knowledge for personalized recommendations. To alleviate this limitation, we propose a novel sequential recommendation framework based on gradient-based meta-learning that captures the imbalanced rating distribution of each user and computes adaptive loss for user-specific learning. Our work is the first to tackle the impact of imbalanced ratings in cold-start sequential recommendation scenarios. Through extensive experiments conducted on real-world datasets, we demonstrate the effectiveness of our framework.
△ Less
Submitted 21 August, 2023; v1 submitted 28 February, 2023;
originally announced February 2023.
-
Channel Reconstruction-Based Hybrid Precoding for Millimeter Wave Multi-User MIMO Systems
Authors:
Miguel R. Castellanos,
Vasanthan Raghavan,
Jung H. Ryu,
Ozge H. Koymen,
Junyi Li,
David J. Love,
Borja Peleato
Abstract:
The focus of this paper is on multi-user MIMO transmissions for millimeter wave systems with a hybrid precoding architecture at the base-station. To enable multi-user transmissions, the base-station uses a cell-specific codebook of beamforming vectors over an initial beam alignment phase. Each user uses a user-specific codebook of beamforming vectors to learn the top-P (where P >= 1) beam pairs in…
▽ More
The focus of this paper is on multi-user MIMO transmissions for millimeter wave systems with a hybrid precoding architecture at the base-station. To enable multi-user transmissions, the base-station uses a cell-specific codebook of beamforming vectors over an initial beam alignment phase. Each user uses a user-specific codebook of beamforming vectors to learn the top-P (where P >= 1) beam pairs in terms of the observed SNR in a single-user setting. The top-P beam indices along with their SNRs are fed back from each user and the base-station leverages this information to generate beam weights for simultaneous transmissions. A typical method to generate the beam weights is to use only the best beam for each user and either steer energy along this beam, or to utilize this information to reduce multi-user interference. The other beams are used as fall back options to address blockage or mobility. Such an approach completely discards information learned about the channel condition(s) even though each user feeds back this information. With this background, this work develops an advanced directional precoding structure for simultaneous transmissions at the cost of an additional marginal feedback overhead. This construction relies on three main innovations: 1) Additional feedback to allow the base-station to reconstruct a rank-P approximation of the channel matrix between it and each user, 2) A zeroforcing structure that leverages this information to combat multi-user interference by remaining agnostic of the receiver beam knowledge in the precoder design, and 3) A hybrid precoding architecture that allows both amplitude and phase control at low-complexity and cost to allow the implementation of the zeroforcing structure. Numerical studies show that the proposed scheme results in a significant sum rate performance improvement over naive schemes even with a coarse initial beam alignment codebook.
△ Less
Submitted 14 February, 2018;
originally announced February 2018.
