-
Unsupervised Few-Shot Continual Learning for Remote Sensing Image Scene Classification
Authors:
Muhammad Anwar Ma'sum,
Mahardhika Pratama,
Ramasamy Savitha,
Lin Liu,
Habibullah,
Ryszard Kowalczyk
Abstract:
A continual learning (CL) model is desired for remote sensing image analysis because of varying camera parameters, spectral ranges, resolutions, etc. There exist some recent initiatives to develop CL techniques in this domain but they still depend on massive labelled samples which do not fully fit remote sensing applications because ground truths are often obtained via field-based surveys. This pa…
▽ More
A continual learning (CL) model is desired for remote sensing image analysis because of varying camera parameters, spectral ranges, resolutions, etc. There exist some recent initiatives to develop CL techniques in this domain but they still depend on massive labelled samples which do not fully fit remote sensing applications because ground truths are often obtained via field-based surveys. This paper addresses this problem with a proposal of unsupervised flat-wide learning approach (UNISA) for unsupervised few-shot continual learning approaches of remote sensing image scene classifications which do not depend on any labelled samples for its model updates. UNISA is developed from the idea of prototype scattering and positive sampling for learning representations while the catastrophic forgetting problem is tackled with the flat-wide learning approach combined with a ball generator to address the data scarcity problem. Our numerical study with remote sensing image scene datasets and a hyperspectral dataset confirms the advantages of our solution. Source codes of UNISA are shared publicly in \url{https://github.com/anwarmaxsum/UNISA} to allow convenient future studies and reproductions of our numerical results.
△ Less
Submitted 3 June, 2024;
originally announced June 2024.
-
Few-Shot Class Incremental Learning via Robust Transformer Approach
Authors:
Naeem Paeedeh,
Mahardhika Pratama,
Sunu Wibirama,
Wolfgang Mayer,
Zehong Cao,
Ryszard Kowalczyk
Abstract:
Few-Shot Class-Incremental Learning presents an extension of the Class Incremental Learning problem where a model is faced with the problem of data scarcity while addressing the catastrophic forgetting problem. This problem remains an open problem because all recent works are built upon the convolutional neural networks performing sub-optimally compared to the transformer approaches. Our paper pre…
▽ More
Few-Shot Class-Incremental Learning presents an extension of the Class Incremental Learning problem where a model is faced with the problem of data scarcity while addressing the catastrophic forgetting problem. This problem remains an open problem because all recent works are built upon the convolutional neural networks performing sub-optimally compared to the transformer approaches. Our paper presents Robust Transformer Approach built upon the Compact Convolution Transformer. The issue of overfitting due to few samples is overcome with the notion of the stochastic classifier, where the classifier's weights are sampled from a distribution with mean and variance vectors, thus increasing the likelihood of correct classifications, and the batch-norm layer to stabilize the training process. The issue of CF is dealt with the idea of delta parameters, small task-specific trainable parameters while keeping the backbone networks frozen. A non-parametric approach is developed to infer the delta parameters for the model's predictions. The prototype rectification approach is applied to avoid biased prototype calculations due to the issue of data scarcity. The advantage of ROBUSTA is demonstrated through a series of experiments in the benchmark problems where it is capable of outperforming prior arts with big margins without any data augmentation protocols.
△ Less
Submitted 7 May, 2024;
originally announced May 2024.
-
Dynamic Long-Term Time-Series Forecasting via Meta Transformer Networks
Authors:
Muhammad Anwar Ma'sum,
MD Rasel Sarkar,
Mahardhika Pratama,
Savitha Ramasamy,
Sreenatha Anavatti,
Lin Liu,
Habibullah,
Ryszard Kowalczyk
Abstract:
A reliable long-term time-series forecaster is highly demanded in practice but comes across many challenges such as low computational and memory footprints as well as robustness against dynamic learning environments. This paper proposes Meta-Transformer Networks (MANTRA) to deal with the dynamic long-term time-series forecasting tasks. MANTRA relies on the concept of fast and slow learners where a…
▽ More
A reliable long-term time-series forecaster is highly demanded in practice but comes across many challenges such as low computational and memory footprints as well as robustness against dynamic learning environments. This paper proposes Meta-Transformer Networks (MANTRA) to deal with the dynamic long-term time-series forecasting tasks. MANTRA relies on the concept of fast and slow learners where a collection of fast learners learns different aspects of data distributions while adapting quickly to changes. A slow learner tailors suitable representations to fast learners. Fast adaptations to dynamic environments are achieved using the universal representation transformer layers producing task-adapted representations with a small number of parameters. Our experiments using four datasets with different prediction lengths demonstrate the advantage of our approach with at least $3\%$ improvements over the baseline algorithms for both multivariate and univariate settings. Source codes of MANTRA are publicly available in \url{https://github.com/anwarmaxsum/MANTRA}.
△ Less
Submitted 25 January, 2024;
originally announced January 2024.
-
Few-Shot Continual Learning via Flat-to-Wide Approaches
Authors:
Muhammad Anwar Ma'sum,
Mahardhika Pratama,
Edwin Lughofer,
Lin Liu,
Habibullah,
Ryszard Kowalczyk
Abstract:
Existing approaches on continual learning call for a lot of samples in their training processes. Such approaches are impractical for many real-world problems having limited samples because of the overfitting problem. This paper proposes a few-shot continual learning approach, termed FLat-tO-WidE AppRoach (FLOWER), where a flat-to-wide learning process finding the flat-wide minima is proposed to ad…
▽ More
Existing approaches on continual learning call for a lot of samples in their training processes. Such approaches are impractical for many real-world problems having limited samples because of the overfitting problem. This paper proposes a few-shot continual learning approach, termed FLat-tO-WidE AppRoach (FLOWER), where a flat-to-wide learning process finding the flat-wide minima is proposed to address the catastrophic forgetting problem. The issue of data scarcity is overcome with a data augmentation approach making use of a ball generator concept to restrict the sampling space into the smallest enclosing ball. Our numerical studies demonstrate the advantage of FLOWER achieving significantly improved performances over prior arts notably in the small base tasks. For further study, source codes of FLOWER, competitor algorithms and experimental logs are shared publicly in \url{https://github.com/anwarmaxsum/FLOWER}.
△ Less
Submitted 13 July, 2023; v1 submitted 25 June, 2023;
originally announced June 2023.
-
Truthful Market-based Trading of Cloud Resources with Reservation Price
Authors:
Sergei Chichin,
Quoc Bao Vo,
Ryszard Kowalczyk
Abstract:
With the rapidly growing demand for the cloud services, a need for efficient methods to trade computing resources increases. Commonly used fixed-price model is not always the best approach for trading cloud resources, because of its inflexible and static nature. Dynamic trading systems, which make use of market mechanisms, show promise for more efficient resource allocation and pricing in the clou…
▽ More
With the rapidly growing demand for the cloud services, a need for efficient methods to trade computing resources increases. Commonly used fixed-price model is not always the best approach for trading cloud resources, because of its inflexible and static nature. Dynamic trading systems, which make use of market mechanisms, show promise for more efficient resource allocation and pricing in the cloud. However, most of the existing mechanisms ignore the seller's costs of providing the resources. In order to address it, we propose a single-sided market mechanism for trading virtual machine instances in the cloud, where the cloud provider can express the reservation prices for traded cloud services. We investigate the theoretical properties of the proposed mechanism and prove that it is truthful, i.e. the buyers do not have an incentive to lie about their true valuation of the resources. We perform extensive experiments in order to investigate the impact of the reserve price on the market outcome. Our experiments show that the proposed mechanism yields near optimal allocations and has a low execution time.
△ Less
Submitted 30 January, 2014;
originally announced January 2014.
-
On Quantified Linguistic Approximation
Authors:
Ryszard Kowalczyk
Abstract:
Most fuzzy systems including fuzzy decision support and fuzzy control systems provide out-puts in the form of fuzzy sets that represent the inferred conclusions. Linguistic interpretation of such outputs often involves the use of linguistic approximation that assigns a linguistic label to a fuzzy set based on the predefined primary terms, linguistic modifiers and linguistic connectives. More gener…
▽ More
Most fuzzy systems including fuzzy decision support and fuzzy control systems provide out-puts in the form of fuzzy sets that represent the inferred conclusions. Linguistic interpretation of such outputs often involves the use of linguistic approximation that assigns a linguistic label to a fuzzy set based on the predefined primary terms, linguistic modifiers and linguistic connectives. More generally, linguistic approximation can be formalized in the terms of the re-translation rules that correspond to the translation rules in ex-plicitation (e.g. simple, modifier, composite, quantification and qualification rules) in com-puting with words [Zadeh 1996]. However most existing methods of linguistic approximation use the simple, modifier and composite re-translation rules only. Although these methods can provide a sufficient approximation of simple fuzzy sets the approximation of more complex ones that are typical in many practical applications of fuzzy systems may be less satisfactory. Therefore the question arises why not use in linguistic ap-proximation also other re-translation rules corre-sponding to the translation rules in explicitation to advantage. In particular linguistic quantifica-tion may be desirable in situations where the conclusions interpreted as quantified linguistic propositions can be more informative and natu-ral. This paper presents some aspects of linguis-tic approximation in the context of the re-translation rules and proposes an approach to linguistic approximation with the use of quantifi-cation rules, i.e. quantified linguistic approxima-tion. Two methods of the quantified linguistic approximation are considered with the use of lin-guistic quantifiers based on the concepts of the non-fuzzy and fuzzy cardinalities of fuzzy sets. A number of examples are provided to illustrate the proposed approach.
△ Less
Submitted 23 January, 2013;
originally announced January 2013.
-
Efficient algorithm for estimation of qualitative expected utility in possibilistic case-based reasoning
Authors:
Jakub Brzostowski,
Ryszard Kowalczyk
Abstract:
We propose an efficient algorithm for estimation of possibility based qualitative expected utility. It is useful for decision making mechanisms where each possible decision is assigned a multi-attribute possibility distribution. The computational complexity of ordinary methods calculating the expected utility based on discretization is growing exponentially with the number of attributes, and may b…
▽ More
We propose an efficient algorithm for estimation of possibility based qualitative expected utility. It is useful for decision making mechanisms where each possible decision is assigned a multi-attribute possibility distribution. The computational complexity of ordinary methods calculating the expected utility based on discretization is growing exponentially with the number of attributes, and may become infeasible with a high number of these attributes. We present series of theorems and lemmas proving the correctness of our algorithm that exibits a linear computational complexity. Our algorithm has been applied in the context of selecting the most prospective partners in multi-party multi-attribute negotiation, and can also be used in making decisions about potential offers during the negotiation as other similar problems.
△ Less
Submitted 4 July, 2012;
originally announced July 2012.
-
An Efficient Protocol for Negotiation over Combinatorial Domains with Incomplete Information
Authors:
Minyi Li,
Quoc Bao Vo,
Ryszard Kowalczyk
Abstract:
We study the problem of agent-based negotiation in combinatorial domains. It is difficult to reach optimal agreements in bilateral or multi-lateral negotiations when the agents' preferences for the possible alternatives are not common knowledge. Self-interested agents often end up negotiating inefficient agreements in such situations. In this paper, we present a protocol for negotiation in combina…
▽ More
We study the problem of agent-based negotiation in combinatorial domains. It is difficult to reach optimal agreements in bilateral or multi-lateral negotiations when the agents' preferences for the possible alternatives are not common knowledge. Self-interested agents often end up negotiating inefficient agreements in such situations. In this paper, we present a protocol for negotiation in combinatorial domains which can lead rational agents to reach optimal agreements under incomplete information setting. Our proposed protocol enables the negotiating agents to identify efficient solutions using distributed search that visits only a small subspace of the whole outcome space. Moreover, the proposed protocol is sufficiently general that it is applicable to most preference representation models in combinatorial domains. We also present results of experiments that demonstrate the feasibility and computational efficiency of our approach.
△ Less
Submitted 14 February, 2012;
originally announced February 2012.
