-
Robust Semi-supervised Learning via $f$-Divergence and $α$-Rényi Divergence
Authors:
Gholamali Aminian,
Amirhossien Bagheri,
Mahyar JafariNodeh,
Radmehr Karimian,
Mohammad-Hossein Yassaee
Abstract:
This paper investigates a range of empirical risk functions and regularization methods suitable for self-training methods in semi-supervised learning. These approaches draw inspiration from various divergence measures, such as $f$-divergences and $α$-Rényi divergences. Inspired by the theoretical foundations rooted in divergences, i.e., $f$-divergences and $α$-Rényi divergence, we also provide val…
▽ More
This paper investigates a range of empirical risk functions and regularization methods suitable for self-training methods in semi-supervised learning. These approaches draw inspiration from various divergence measures, such as $f$-divergences and $α$-Rényi divergences. Inspired by the theoretical foundations rooted in divergences, i.e., $f$-divergences and $α$-Rényi divergence, we also provide valuable insights to enhance the understanding of our empirical risk functions and regularization techniques. In the pseudo-labeling and entropy minimization techniques as self-training methods for effective semi-supervised learning, the self-training process has some inherent mismatch between the true label and pseudo-label (noisy pseudo-labels) and some of our empirical risk functions are robust, concerning noisy pseudo-labels. Under some conditions, our empirical risk functions demonstrate better performance when compared to traditional self-training methods.
△ Less
Submitted 1 May, 2024;
originally announced May 2024.
-
Generalization Error of Graph Neural Networks in the Mean-field Regime
Authors:
Gholamali Aminian,
Yixuan He,
Gesine Reinert,
Łukasz Szpruch,
Samuel N. Cohen
Abstract:
This work provides a theoretical framework for assessing the generalization error of graph neural networks in the over-parameterized regime, where the number of parameters surpasses the quantity of data points. We explore two widely utilized types of graph neural networks: graph convolutional neural networks and message passing graph neural networks. Prior to this study, existing bounds on the gen…
▽ More
This work provides a theoretical framework for assessing the generalization error of graph neural networks in the over-parameterized regime, where the number of parameters surpasses the quantity of data points. We explore two widely utilized types of graph neural networks: graph convolutional neural networks and message passing graph neural networks. Prior to this study, existing bounds on the generalization error in the over-parametrized regime were uninformative, limiting our understanding of over-parameterized network performance. Our novel approach involves deriving upper bounds within the mean-field regime for evaluating the generalization error of these graph neural networks. We establish upper bounds with a convergence rate of $O(1/n)$, where $n$ is the number of graph samples. These upper bounds offer a theoretical assurance of the networks' performance on unseen data in the challenging over-parameterized regime and overall contribute to our understanding of their performance.
△ Less
Submitted 1 July, 2024; v1 submitted 10 February, 2024;
originally announced February 2024.
-
Mean-field Analysis of Generalization Errors
Authors:
Gholamali Aminian,
Samuel N. Cohen,
Łukasz Szpruch
Abstract:
We propose a novel framework for exploring weak and $L_2$ generalization errors of algorithms through the lens of differential calculus on the space of probability measures. Specifically, we consider the KL-regularized empirical risk minimization problem and establish generic conditions under which the generalization error convergence rate, when training on a sample of size $n$, is…
▽ More
We propose a novel framework for exploring weak and $L_2$ generalization errors of algorithms through the lens of differential calculus on the space of probability measures. Specifically, we consider the KL-regularized empirical risk minimization problem and establish generic conditions under which the generalization error convergence rate, when training on a sample of size $n$, is $\mathcal{O}(1/n)$. In the context of supervised learning with a one-hidden layer neural network in the mean-field regime, these conditions are reflected in suitable integrability and regularity assumptions on the loss and activation functions.
△ Less
Submitted 20 June, 2023;
originally announced June 2023.
-
On the Generalization Error of Meta Learning for the Gibbs Algorithm
Authors:
Yuheng Bu,
Harsha Vardhan Tetali,
Gholamali Aminian,
Miguel Rodrigues,
Gregory Wornell
Abstract:
We analyze the generalization ability of joint-training meta learning algorithms via the Gibbs algorithm. Our exact characterization of the expected meta generalization error for the meta Gibbs algorithm is based on symmetrized KL information, which measures the dependence between all meta-training datasets and the output parameters, including task-specific and meta parameters. Additionally, we de…
▽ More
We analyze the generalization ability of joint-training meta learning algorithms via the Gibbs algorithm. Our exact characterization of the expected meta generalization error for the meta Gibbs algorithm is based on symmetrized KL information, which measures the dependence between all meta-training datasets and the output parameters, including task-specific and meta parameters. Additionally, we derive an exact characterization of the meta generalization error for the super-task Gibbs algorithm, in terms of conditional symmetrized KL information within the super-sample and super-task framework introduced in Steinke and Zakynthinou (2020) and Hellstrom and Durisi (2022) respectively. Our results also enable us to provide novel distribution-free generalization error upper bounds for these Gibbs algorithms applicable to meta learning.
△ Less
Submitted 27 April, 2023;
originally announced April 2023.
-
Information-theoretic Characterizations of Generalization Error for the Gibbs Algorithm
Authors:
Gholamali Aminian,
Yuheng Bu,
Laura Toni,
Miguel R. D. Rodrigues,
Gregory W. Wornell
Abstract:
Various approaches have been developed to upper bound the generalization error of a supervised learning algorithm. However, existing bounds are often loose and even vacuous when evaluated in practice. As a result, they may fail to characterize the exact generalization ability of a learning algorithm. Our main contributions are exact characterizations of the expected generalization error of the wel…
▽ More
Various approaches have been developed to upper bound the generalization error of a supervised learning algorithm. However, existing bounds are often loose and even vacuous when evaluated in practice. As a result, they may fail to characterize the exact generalization ability of a learning algorithm. Our main contributions are exact characterizations of the expected generalization error of the well-known Gibbs algorithm (a.k.a. Gibbs posterior) using different information measures, in particular, the symmetrized KL information between the input training samples and the output hypothesis. Our result can be applied to tighten existing expected generalization error and PAC-Bayesian bounds. Our information-theoretic approach is versatile, as it also characterizes the generalization error of the Gibbs algorithm with a data-dependent regularizer and that of the Gibbs algorithm in the asymptotic regime, where it converges to the standard empirical risk minimization algorithm. Of particular relevance, our results highlight the role the symmetrized KL information plays in controlling the generalization error of the Gibbs algorithm.
△ Less
Submitted 18 October, 2022;
originally announced October 2022.
-
How Does Pseudo-Labeling Affect the Generalization Error of the Semi-Supervised Gibbs Algorithm?
Authors:
Haiyun He,
Gholamali Aminian,
Yuheng Bu,
Miguel Rodrigues,
Vincent Y. F. Tan
Abstract:
We provide an exact characterization of the expected generalization error (gen-error) for semi-supervised learning (SSL) with pseudo-labeling via the Gibbs algorithm. The gen-error is expressed in terms of the symmetrized KL information between the output hypothesis, the pseudo-labeled dataset, and the labeled dataset. Distribution-free upper and lower bounds on the gen-error can also be obtained.…
▽ More
We provide an exact characterization of the expected generalization error (gen-error) for semi-supervised learning (SSL) with pseudo-labeling via the Gibbs algorithm. The gen-error is expressed in terms of the symmetrized KL information between the output hypothesis, the pseudo-labeled dataset, and the labeled dataset. Distribution-free upper and lower bounds on the gen-error can also be obtained. Our findings offer new insights that the generalization performance of SSL with pseudo-labeling is affected not only by the information between the output hypothesis and input training data but also by the information {\em shared} between the {\em labeled} and {\em pseudo-labeled} data samples. This serves as a guideline to choose an appropriate pseudo-labeling method from a given family of methods. To deepen our understanding, we further explore two examples -- mean estimation and logistic regression. In particular, we analyze how the ratio of the number of unlabeled to labeled data $λ$ affects the gen-error under both scenarios. As $λ$ increases, the gen-error for mean estimation decreases and then saturates at a value larger than when all the samples are labeled, and the gap can be quantified {\em exactly} with our analysis, and is dependent on the \emph{cross-covariance} between the labeled and pseudo-labeled data samples. For logistic regression, the gen-error and the variance component of the excess risk also decrease as $λ$ increases.
△ Less
Submitted 15 June, 2023; v1 submitted 15 October, 2022;
originally announced October 2022.
-
Learning Algorithm Generalization Error Bounds via Auxiliary Distributions
Authors:
Gholamali Aminian,
Saeed Masiha,
Laura Toni,
Miguel R. D. Rodrigues
Abstract:
Generalization error bounds are essential for comprehending how well machine learning models work. In this work, we suggest a novel method, i.e., the Auxiliary Distribution Method, that leads to new upper bounds on expected generalization errors that are appropriate for supervised learning scenarios. We show that our general upper bounds can be specialized under some conditions to new bounds invol…
▽ More
Generalization error bounds are essential for comprehending how well machine learning models work. In this work, we suggest a novel method, i.e., the Auxiliary Distribution Method, that leads to new upper bounds on expected generalization errors that are appropriate for supervised learning scenarios. We show that our general upper bounds can be specialized under some conditions to new bounds involving the $α$-Jensen-Shannon, $α$-Rényi ($0< α< 1$) information between a random variable modeling the set of training samples and another random variable modeling the set of hypotheses. Our upper bounds based on $α$-Jensen-Shannon information are also finite. Additionally, we demonstrate how our auxiliary distribution method can be used to derive the upper bounds on excess risk of some learning algorithms in the supervised learning context {\blue and the generalization error under the distribution mismatch scenario in supervised learning algorithms, where the distribution mismatch is modeled as $α$-Jensen-Shannon or $α$-Rényi divergence between the distribution of test and training data samples distributions.} We also outline the conditions for which our proposed upper bounds might be tighter than other earlier upper bounds.
△ Less
Submitted 16 April, 2024; v1 submitted 2 October, 2022;
originally announced October 2022.
-
Semi-supervised Batch Learning From Logged Data
Authors:
Gholamali Aminian,
Armin Behnamnia,
Roberto Vega,
Laura Toni,
Chengchun Shi,
Hamid R. Rabiee,
Omar Rivasplata,
Miguel R. D. Rodrigues
Abstract:
Off-policy learning methods are intended to learn a policy from logged data, which includes context, action, and feedback (cost or reward) for each sample point. In this work, we build on the counterfactual risk minimization framework, which also assumes access to propensity scores. We propose learning methods for problems where feedback is missing for some samples, so there are samples with feedb…
▽ More
Off-policy learning methods are intended to learn a policy from logged data, which includes context, action, and feedback (cost or reward) for each sample point. In this work, we build on the counterfactual risk minimization framework, which also assumes access to propensity scores. We propose learning methods for problems where feedback is missing for some samples, so there are samples with feedback and samples missing-feedback in the logged data. We refer to this type of learning as semi-supervised batch learning from logged data, which arises in a wide range of application domains. We derive a novel upper bound for the true risk under the inverse propensity score estimator to address this kind of learning problem. Using this bound, we propose a regularized semi-supervised batch learning method with logged data where the regularization term is feedback-independent and, as a result, can be evaluated using the logged missing-feedback data. Consequently, even though feedback is only present for some samples, a learning policy can be learned by leveraging the missing-feedback samples. The results of experiments derived from benchmark datasets indicate that these algorithms achieve policies with better performance in comparison with logging policies.
△ Less
Submitted 18 February, 2024; v1 submitted 15 September, 2022;
originally announced September 2022.
-
Tighter Expected Generalization Error Bounds via Convexity of Information Measures
Authors:
Gholamali Aminian,
Yuheng Bu,
Gregory Wornell,
Miguel Rodrigues
Abstract:
Generalization error bounds are essential to understanding machine learning algorithms. This paper presents novel expected generalization error upper bounds based on the average joint distribution between the output hypothesis and each input training sample. Multiple generalization error upper bounds based on different information measures are provided, including Wasserstein distance, total variat…
▽ More
Generalization error bounds are essential to understanding machine learning algorithms. This paper presents novel expected generalization error upper bounds based on the average joint distribution between the output hypothesis and each input training sample. Multiple generalization error upper bounds based on different information measures are provided, including Wasserstein distance, total variation distance, KL divergence, and Jensen-Shannon divergence. Due to the convexity of the information measures, the proposed bounds in terms of Wasserstein distance and total variation distance are shown to be tighter than their counterparts based on individual samples in the literature. An example is provided to demonstrate the tightness of the proposed generalization error bounds.
△ Less
Submitted 24 February, 2022;
originally announced February 2022.
-
An Information-theoretical Approach to Semi-supervised Learning under Covariate-shift
Authors:
Gholamali Aminian,
Mahed Abroshan,
Mohammad Mahdi Khalili,
Laura Toni,
Miguel R. D. Rodrigues
Abstract:
A common assumption in semi-supervised learning is that the labeled, unlabeled, and test data are drawn from the same distribution. However, this assumption is not satisfied in many applications. In many scenarios, the data is collected sequentially (e.g., healthcare) and the distribution of the data may change over time often exhibiting so-called covariate shifts. In this paper, we propose an app…
▽ More
A common assumption in semi-supervised learning is that the labeled, unlabeled, and test data are drawn from the same distribution. However, this assumption is not satisfied in many applications. In many scenarios, the data is collected sequentially (e.g., healthcare) and the distribution of the data may change over time often exhibiting so-called covariate shifts. In this paper, we propose an approach for semi-supervised learning algorithms that is capable of addressing this issue. Our framework also recovers some popular methods, including entropy minimization and pseudo-labeling. We provide new information-theoretical based generalization error upper bounds inspired by our novel framework. Our bounds are applicable to both general semi-supervised learning and the covariate-shift scenario. Finally, we show numerically that our method outperforms previous approaches proposed for semi-supervised learning under the covariate shift.
△ Less
Submitted 24 February, 2022;
originally announced February 2022.
-
Characterizing and Understanding the Generalization Error of Transfer Learning with Gibbs Algorithm
Authors:
Yuheng Bu,
Gholamali Aminian,
Laura Toni,
Miguel Rodrigues,
Gregory Wornell
Abstract:
We provide an information-theoretic analysis of the generalization ability of Gibbs-based transfer learning algorithms by focusing on two popular transfer learning approaches, $α$-weighted-ERM and two-stage-ERM. Our key result is an exact characterization of the generalization behaviour using the conditional symmetrized KL information between the output hypothesis and the target training samples g…
▽ More
We provide an information-theoretic analysis of the generalization ability of Gibbs-based transfer learning algorithms by focusing on two popular transfer learning approaches, $α$-weighted-ERM and two-stage-ERM. Our key result is an exact characterization of the generalization behaviour using the conditional symmetrized KL information between the output hypothesis and the target training samples given the source samples. Our results can also be applied to provide novel distribution-free generalization error upper bounds on these two aforementioned Gibbs algorithms. Our approach is versatile, as it also characterizes the generalization errors and excess risks of these two Gibbs algorithms in the asymptotic regime, where they converge to the $α$-weighted-ERM and two-stage-ERM, respectively. Based on our theoretical results, we show that the benefits of transfer learning can be viewed as a bias-variance trade-off, with the bias induced by the source distribution and the variance induced by the lack of target samples. We believe this viewpoint can guide the choice of transfer learning algorithms in practice.
△ Less
Submitted 2 November, 2021;
originally announced November 2021.
-
Characterizing the Generalization Error of Gibbs Algorithm with Symmetrized KL information
Authors:
Gholamali Aminian,
Yuheng Bu,
Laura Toni,
Miguel R. D. Rodrigues,
Gregory Wornell
Abstract:
Bounding the generalization error of a supervised learning algorithm is one of the most important problems in learning theory, and various approaches have been developed. However, existing bounds are often loose and lack of guarantees. As a result, they may fail to characterize the exact generalization ability of a learning algorithm. Our main contribution is an exact characterization of the expec…
▽ More
Bounding the generalization error of a supervised learning algorithm is one of the most important problems in learning theory, and various approaches have been developed. However, existing bounds are often loose and lack of guarantees. As a result, they may fail to characterize the exact generalization ability of a learning algorithm. Our main contribution is an exact characterization of the expected generalization error of the well-known Gibbs algorithm in terms of symmetrized KL information between the input training samples and the output hypothesis. Such a result can be applied to tighten existing expected generalization error bound. Our analysis provides more insight on the fundamental role the symmetrized KL information plays in controlling the generalization error of the Gibbs algorithm.
△ Less
Submitted 28 July, 2021;
originally announced July 2021.
-
Information-Theoretic Bounds on the Moments of the Generalization Error of Learning Algorithms
Authors:
Gholamali Aminian,
Laura Toni,
Miguel R. D. Rodrigues
Abstract:
Generalization error bounds are critical to understanding the performance of machine learning models. In this work, building upon a new bound of the expected value of an arbitrary function of the population and empirical risk of a learning algorithm, we offer a more refined analysis of the generalization behaviour of a machine learning models based on a characterization of (bounds) to their genera…
▽ More
Generalization error bounds are critical to understanding the performance of machine learning models. In this work, building upon a new bound of the expected value of an arbitrary function of the population and empirical risk of a learning algorithm, we offer a more refined analysis of the generalization behaviour of a machine learning models based on a characterization of (bounds) to their generalization error moments. We discuss how the proposed bounds -- which also encompass new bounds to the expected generalization error -- relate to existing bounds in the literature. We also discuss how the proposed generalization error moment bounds can be used to construct new generalization error high-probability bounds.
△ Less
Submitted 5 May, 2021; v1 submitted 3 February, 2021;
originally announced February 2021.
-
Jensen-Shannon Information Based Characterization of the Generalization Error of Learning Algorithms
Authors:
Gholamali Aminian,
Laura Toni,
Miguel R. D. Rodrigues
Abstract:
Generalization error bounds are critical to understanding the performance of machine learning models. In this work, we propose a new information-theoretic based generalization error upper bound applicable to supervised learning scenarios. We show that our general bound can specialize in various previous bounds. We also show that our general bound can be specialized under some conditions to a new b…
▽ More
Generalization error bounds are critical to understanding the performance of machine learning models. In this work, we propose a new information-theoretic based generalization error upper bound applicable to supervised learning scenarios. We show that our general bound can specialize in various previous bounds. We also show that our general bound can be specialized under some conditions to a new bound involving the Jensen-Shannon information between a random variable modelling the set of training samples and another random variable modelling the hypothesis. We also prove that our bound can be tighter than mutual information-based bounds under some conditions.
△ Less
Submitted 8 January, 2021; v1 submitted 23 October, 2020;
originally announced October 2020.
-
On the Capacity of a Class of Signal-Dependent Noise Channels
Authors:
Hamid Ghourchian,
Gholamali Aminian,
Amin Gohari,
Mahtab Mirmohseni,
Masoumeh Nasiri-Kenari
Abstract:
In some applications, the variance of additive measurement noise depends on the signal that we aim to measure. For instance, additive Gaussian signal-dependent noise (AGSDN) channel models are used in molecular and optical communication. Herein we provide lower and upper bounds on the capacity of additive signal-dependent noise (ASDN) channels. The idea of the first lower bound is the extension of…
▽ More
In some applications, the variance of additive measurement noise depends on the signal that we aim to measure. For instance, additive Gaussian signal-dependent noise (AGSDN) channel models are used in molecular and optical communication. Herein we provide lower and upper bounds on the capacity of additive signal-dependent noise (ASDN) channels. The idea of the first lower bound is the extension of the majorization inequality, and for the second one, it uses some calculations based on the fact that $h(Y) > h (Y|Z)$. Both of them are valid for all additive signal-dependent noise (ASDN) channels defined in the paper. The upper bound is based on a previous idea of the authors ("symmetric relative entropy") and is used for the additive Gaussian signal-dependent noise (AGSDN) channels. These bounds indicate that in ASDN channels (unlike the classical AWGN channels), the capacity does not necessarily become larger by making the variance function of the noise smaller. We also provide sufficient conditions under which the capacity becomes infinity. This is complemented by a number of conditions that imply capacity is finite and a unique capacity achieving measure exists (in the sense of the output measure).
△ Less
Submitted 2 June, 2017; v1 submitted 12 February, 2017;
originally announced February 2017.
-
On Medium Chemical Reaction in Diffusion-Based Molecular Communication: a Two-Way Relaying Example
Authors:
Maryam Farahnak-Ghazani,
Gholamali Aminian,
Mahtab Mirmohseni,
Amin Gohari,
Masoumeh Nasiri-Kenari
Abstract:
Chemical reactions are a prominent feature of molecular communication (MC) systems, with no direct parallels in wireless communications. While chemical reactions may be used inside the transmitter nodes, receiver nodes or the communication medium, we focus on its utility in the medium in this paper. Such chemical reactions can be used to perform computation over the medium as molecules diffuse and…
▽ More
Chemical reactions are a prominent feature of molecular communication (MC) systems, with no direct parallels in wireless communications. While chemical reactions may be used inside the transmitter nodes, receiver nodes or the communication medium, we focus on its utility in the medium in this paper. Such chemical reactions can be used to perform computation over the medium as molecules diffuse and react with each other (physical-layer computation). We propose the use of chemical reactions for the following purposes: (i) to reduce signal-dependent observation noise of receivers by reducing the signal density, (ii) to realize molecular physical-layer network coding (molecular PNC) by performing the natural XOR operation inside the medium, and (iii) to reduce the inter-symbol interference (ISI) of other transmitters by canceling out the remaining molecules from previous transmissions. To make the ideas formal, we consider an explicit two-way relaying example with a transparent receiver (which has a signal-dependent noise). The proposed ideas are used to define a modulation scheme (which we call the PNC scheme). We compare the PNC with a previously proposed scheme for this problem where the XOR operation is performed at the relay node (using a molecular logic gate). We call the latter, the straightforward network coding (SNC). It is observed that in addition to the simplicity of the proposed PNC scheme, it outperforms the SNC scheme especially when we consider ISI.
△ Less
Submitted 24 April, 2018; v1 submitted 19 April, 2016;
originally announced April 2016.
-
On the Capacity of Point-to-Point and Multiple-Access Molecular Communications with Ligand-Receptors
Authors:
Gholamali Aminian,
Maryam Farahnak Ghazani,
Mahtab Mirmohseni,
Masoumeh Nasiri Kenari,
Faramarz Fekri
Abstract:
In this paper, we consider the bacterial point-to-point and multiple-access molecular communications with ligand-receptors. For the point-to-point communication, we investigate common signaling methods, namely the Level Scenario (LS), which uses one type of a molecule with different concentration levels, and the Type Scenario (TS), which employs multiple types of molecules with a single concentrat…
▽ More
In this paper, we consider the bacterial point-to-point and multiple-access molecular communications with ligand-receptors. For the point-to-point communication, we investigate common signaling methods, namely the Level Scenario (LS), which uses one type of a molecule with different concentration levels, and the Type Scenario (TS), which employs multiple types of molecules with a single concentration level. We investigate the trade-offs between the two scenarios from the capacity point of view. We derive an upper bound on the capacity using a Binomial Channel (BIC) model and the symmetrized Kullback-Leibler (KL) divergence. A lower bound is also derived when the environment noise is negligible. For the TS, we also consider the effect of blocking of a receptor by a different molecule type. Then, we consider multiple-access communications, for which we investigate three scenarios based on molecule and receptor types, i.e., same types of molecules with Different Labeling and Same types of Receptors (DLSR), Different types of Molecules and Receptors (DMDR), and Same types of Molecules and Receptors (SMSR). We investigate the trade-offs among the three scenarios from the total capacity point of view. We derive some inner bounds on the capacity region of these scenarios when the environment noise is negligible.
△ Less
Submitted 19 September, 2015;
originally announced September 2015.
-
On the Capacity of Level and Type Modulation in Molecular Communication with Ligand Receptors
Authors:
Gholamali Aminian,
Mahtab Mirmohseni,
Masoumeh Nasiri Kenari,
Faramarz Fekri
Abstract:
In this paper, we consider the bacterial point-to-point communication problem with one transmitter and one receiver by considering the ligand receptor binding process. The most commonly investigated signalling model, referred to as the Level Scenario (LS), uses one type of a molecule with different concentration levels for signaling. An alternative approach is to employ multiple types of molecules…
▽ More
In this paper, we consider the bacterial point-to-point communication problem with one transmitter and one receiver by considering the ligand receptor binding process. The most commonly investigated signalling model, referred to as the Level Scenario (LS), uses one type of a molecule with different concentration levels for signaling. An alternative approach is to employ multiple types of molecules with a single concentration level, referred to as the Type Scenario (TS). We investigate the trade-offs between the two scenarios for the ligand receptor from the capacity point of view. For this purpose, we evaluate the capacity using numerical algorithms. Moreover, we derive an upper bound on the capacity of the ligand receptor using a Binomial Channel (BIC) model using symmetrized Kullback-Leibler (KL) divergence. A lower bound is also derived when the environment noise is negligible. Finally, we analyse the effect of blocking of a receptor by a molecule of a different type, by proposing a new Markov model in the multiple-type signalling.
△ Less
Submitted 16 April, 2015;
originally announced April 2015.
-
Capacity of Diffusion based Molecular Communication Networks over LTI-Poisson Channels
Authors:
Hamidreza Arjmandi,
Gholamali Aminian,
Amin Gohari,
Masoumeh Nasiri Kenari,
Urbashi Mitra
Abstract:
In this paper, the capacity of a diffusion based molecular communication network under the model of a Linear Time Invarient-Poisson (LTI-Poisson) channel is studied. Introduced in the context of molecular communication, the LTI-Poisson model is a natural extension of the conventional memoryless Poisson channel to include memory. Exploiting prior art on linear ISI channels, a computable finite-lett…
▽ More
In this paper, the capacity of a diffusion based molecular communication network under the model of a Linear Time Invarient-Poisson (LTI-Poisson) channel is studied. Introduced in the context of molecular communication, the LTI-Poisson model is a natural extension of the conventional memoryless Poisson channel to include memory. Exploiting prior art on linear ISI channels, a computable finite-letter characterization of the capacity of single-hop LTI-Poisson networks is provided. Then, the problem of finding more explicit bounds on the capacity is examined, where lower and upper bounds for the point to point case are provided. Furthermore, an approach for bounding mutual information in the low SNR regime using the symmetrized KL divergence is introduced and its applicability to Poisson channels is shown. To best of our knowledge, the first non-trivial upper bound on the capacity of Poisson channel with a maximum transmission constraint in the low SNR regime is found. Numerical results show that the proposed upper bound is of the same order as the capacity in the low SNR regime.
△ Less
Submitted 16 October, 2014; v1 submitted 15 October, 2014;
originally announced October 2014.
