Showing 1–32 of 32 results for author: Kaya, H

Audio-Visual Compound Expression Recognition Method based on Late Modality Fusion and Rule-based Decision

Authors: Elena Ryumina, Maxim Markitantov, Dmitry Ryumin, Heysem Kaya, Alexey Karpov

Abstract: This paper presents the results of the SUN team for the Compound Expressions Recognition Challenge of the 6th ABAW Competition. We propose a novel audio-visual method for compound expression recognition. Our method relies on emotion recognition models that fuse modalities at the emotion probability level, while decisions regarding the prediction of compound expressions are based on predefined rule… ▽ More This paper presents the results of the SUN team for the Compound Expressions Recognition Challenge of the 6th ABAW Competition. We propose a novel audio-visual method for compound expression recognition. Our method relies on emotion recognition models that fuse modalities at the emotion probability level, while decisions regarding the prediction of compound expressions are based on predefined rules. Notably, our method does not use any training data specific to the target task. Thus, the problem is a zero-shot classification task. The method is evaluated in multi-corpus training and cross-corpus validation setups. Using our proposed method is achieved an F1-score value equals to 22.01% on the C-EXPR-DB test subset. Our findings from the challenge demonstrate that the proposed method can potentially form a basis for developing intelligent tools for annotating audio-visual data in the context of human's basic and compound emotions. △ Less

Submitted 29 March, 2024; v1 submitted 19 March, 2024; originally announced March 2024.

Comments: 7 pages, 3 figures

SUN Team's Contribution to ABAW 2024 Competition: Audio-visual Valence-Arousal Estimation and Expression Recognition

Authors: Denis Dresvyanskiy, Maxim Markitantov, Jiawei Yu, Peitong Li, Heysem Kaya, Alexey Karpov

Abstract: As emotions play a central role in human communication, automatic emotion recognition has attracted increasing attention in the last two decades. While multimodal systems enjoy high performances on lab-controlled data, they are still far from providing ecological validity on non-lab-controlled, namely 'in-the-wild' data. This work investigates audiovisual deep learning approaches for emotion recog… ▽ More As emotions play a central role in human communication, automatic emotion recognition has attracted increasing attention in the last two decades. While multimodal systems enjoy high performances on lab-controlled data, they are still far from providing ecological validity on non-lab-controlled, namely 'in-the-wild' data. This work investigates audiovisual deep learning approaches for emotion recognition in-the-wild problem. We particularly explore the effectiveness of architectures based on fine-tuned Convolutional Neural Networks (CNN) and Public Dimensional Emotion Model (PDEM), for video and audio modality, respectively. We compare alternative temporal modeling and fusion strategies using the embeddings from these multi-stage trained modality-specific Deep Neural Networks (DNN). We report results on the AffWild2 dataset under Affective Behavior Analysis in-the-Wild 2024 (ABAW'24) challenge protocol. △ Less

Submitted 19 March, 2024; originally announced March 2024.

Comments: 9 pages,

ACM Class: I.2.1; I.2.6; I.2.10

Privacy Constrained Fairness Estimation for Decision Trees

Authors: Florian van der Steen, Fré Vink, Heysem Kaya

Abstract: The protection of sensitive data becomes more vital, as data increases in value and potency. Furthermore, the pressure increases from regulators and society on model developers to make their Artificial Intelligence (AI) models non-discriminatory. To boot, there is a need for interpretable, transparent AI models for high-stakes tasks. In general, measuring the fairness of any AI model requires the… ▽ More The protection of sensitive data becomes more vital, as data increases in value and potency. Furthermore, the pressure increases from regulators and society on model developers to make their Artificial Intelligence (AI) models non-discriminatory. To boot, there is a need for interpretable, transparent AI models for high-stakes tasks. In general, measuring the fairness of any AI model requires the sensitive attributes of the individuals in the dataset, thus raising privacy concerns. In this work, the trade-offs between fairness, privacy and interpretability are further explored. We specifically examine the Statistical Parity (SP) of Decision Trees (DTs) with Differential Privacy (DP), that are each popular methods in their respective subfield. We propose a novel method, dubbed Privacy-Aware Fairness Estimation of Rules (PAFER), that can estimate SP in a DP-aware manner for DTs. DP, making use of a third-party legal entity that securely holds this sensitive data, guarantees privacy by adding noise to the sensitive data. We experimentally compare several DP mechanisms. We show that using the Laplacian mechanism, the method is able to estimate SP with low error while guaranteeing the privacy of the individuals in the dataset with high certainty. We further show experimentally and theoretically that the method performs better for DTs that humans generally find easier to interpret. △ Less

Submitted 13 December, 2023; originally announced December 2023.

Comments: 52 pages, under review in Applied Intelligence journal

Bit-Interleaved Multiple Access: Improved Fairness, Reliability, and Latency for Massive IoT Networks

Authors: Ferdi Kara, Hakan Kaya, Halim Yanikomeroglu, Chan-Tong Lam, Ben K. Ng

Abstract: In this paper, we propose bit-interleaved multiple access (BIMA) to enable Internet-of-Things (IoT) networks where a massive connection is required with limited resource blocks. First, by providing a true power allocation (PA) constraint for conventional NOMA with practical constraints, we demonstrate that it cannot support massive connections. To this end, we propose BIMA where there are no stric… ▽ More In this paper, we propose bit-interleaved multiple access (BIMA) to enable Internet-of-Things (IoT) networks where a massive connection is required with limited resource blocks. First, by providing a true power allocation (PA) constraint for conventional NOMA with practical constraints, we demonstrate that it cannot support massive connections. To this end, we propose BIMA where there are no strict PA constraints, unlike conventional NOMA, thus allowing a high number of devices. We provide a comprehensive analytical framework for BIMA for all key performance indicators (KPIs) (i.e., ergodic capacity [EC], outage probability [OP], and bit error rate [BER]). We evaluate Jain's fairness index and proportional fairness index in terms of all KPIs. Based on the extensive computer simulations, we reveal that BIMA outperforms conventional NOMA significantly, with a performance gain of up to 20-30dB. This performance gain becomes greater when more devices are supported. BIMA provides a full diversity order and enables the implementation of an arbitrary number of devices and modulation orders, which is crucial for IoT networks in dense areas. BIMA guarantees a fairness system where none of the devices gets a severe performance and the sum-rate is shared in a fair manner among devices by guarantying QoS satisfaction. Finally, we provide an intense complexity and latency analysis and demonstrate that BIMA provides lower latency compared to conventional NOMA since it allows parallel computing at the receivers and no iterative operations are required. We show that BIMA reduces latency by up to 350\% for specific devices and 170\% on average. △ Less

Submitted 12 April, 2023; originally announced April 2023.

Comments: accepted in IEEE

The effects of gender bias in word embeddings on depression prediction

Authors: Gizem Sogancioglu, Heysem Kaya

Abstract: Word embeddings are extensively used in various NLP problems as a state-of-the-art semantic feature vector representation. Despite their success on various tasks and domains, they might exhibit an undesired bias for stereotypical categories due to statistical and societal biases that exist in the dataset they are trained on. In this study, we analyze the gender bias in four different pre-trained w… ▽ More Word embeddings are extensively used in various NLP problems as a state-of-the-art semantic feature vector representation. Despite their success on various tasks and domains, they might exhibit an undesired bias for stereotypical categories due to statistical and societal biases that exist in the dataset they are trained on. In this study, we analyze the gender bias in four different pre-trained word embeddings specifically for the depression category in the mental disorder domain. We use contextual and non-contextual embeddings that are trained on domain-independent as well as clinical domain-specific data. We observe that embeddings carry bias for depression towards different gender groups depending on the type of embeddings. Moreover, we demonstrate that these undesired correlations are transferred to the downstream task for depression phenotype recognition. We find that data augmentation by simply swapping gender words mitigates the bias significantly in the downstream task. △ Less

Submitted 15 December, 2022; originally announced December 2022.

Comments: accepted to and published at "A Participatory Approach to AI for Mental Health (PAI4MH)" workshop, co-located with NeurIPS 2022

Error Analysis of Cooperative NOMA with Practical Constraints: Hardware-Impairment, Imperfect SIC and CSI

Authors: Beddiaf Safia, Khelil Abdellatif, Faical Khennoufa, Ferdi Kara, Hakan Kaya, Xingwang Li, Khaled Rabie, Halim Yanikomeroglu

Abstract: Non-orthogonal multiple access (NOMA) has been a strong candidate to support massive connectivity in future wireless networks. In this regard, its implementation into cooperative relaying, named cooperative-NOMA (CNOMA), has received tremendous attention by researchers. However, most of the existing CNOMA studies have failed to address practical constraints since they assume ideal conditions. Part… ▽ More Non-orthogonal multiple access (NOMA) has been a strong candidate to support massive connectivity in future wireless networks. In this regard, its implementation into cooperative relaying, named cooperative-NOMA (CNOMA), has received tremendous attention by researchers. However, most of the existing CNOMA studies have failed to address practical constraints since they assume ideal conditions. Particularly, error performance of CNOMA schemes with imperfections has not been investigated, yet. In this letter, we provide an analytical framework for error performance of CNOMA schemes under practical assumptions where we take into account imperfect successive interference canceler (SIC), imperfect channel estimation (ICSI), and hardware impairments (HWI) at the transceivers. We derive bit error rate (BER) expressions in CNOMA schemes whether the direct links between source and users exist or not which is, to the best of the authors' knowledge, the first study in the open literature. For comparisons, we also provide BER expression for downlink NOMA with practical constraints which has also not been given in literature, yet. The theoretical BER expressions are validated with computer simulations where the perfect-match is observed. Finally, we discuss the effects of the system parameters (e.g., power allocation, HWI level) on the performance of CNOMA schemes to reveal fruitful insights for the society. △ Less

Submitted 30 June, 2022; originally announced July 2022.

Comments: submitted to IEEE

A Hybrid Energy Harvesting Protocol for Cooperative NOMA: Error Performance Approach

Authors: Faical Khennoufa, Khelil Abdellatif, Ferdi Kara, Hakan Kaya, Xingwang Li, Khaled Rabie, Halim Yanikomeroglu

Abstract: Cooperative non-orthogonal multiple access (CNOMA) has recently been adapted with energy harvesting (EH) to increase energy efficiency and extend the lifetime of energy-constrained wireless networks. This paper proposes a hybrid EH protocol-assisted CNOMA, which is a combination of the two main existing EH protocols (power splitting (PS) and time switching (TS)). The end-to-end bit error rate (BER… ▽ More Cooperative non-orthogonal multiple access (CNOMA) has recently been adapted with energy harvesting (EH) to increase energy efficiency and extend the lifetime of energy-constrained wireless networks. This paper proposes a hybrid EH protocol-assisted CNOMA, which is a combination of the two main existing EH protocols (power splitting (PS) and time switching (TS)). The end-to-end bit error rate (BER) expressions of users in the proposed scheme are obtained over Nakagami-$m$ fading channels. The proposed hybrid EH (HEH) protocol is compared with the benchmark schemes (i.e., existing EH protocols and no EH). Based on the extensive simulations, we reveal that the analytical results match perfectly with simulations which proves the correctness of the derivations. Numerical results also show that the HEH-CNOMA outperforms the benchmarks significantly. In addition, we discuss the optimum value of EH factors to minimize the error probability in HEH-CNOMA and show that an optimum value can be obtained according to channel parameters. △ Less

Submitted 30 June, 2022; originally announced July 2022.

Comments: submitted to IEEE

Federated learning for violence incident prediction in a simulated cross-institutional psychiatric setting

Authors: Thomas Borger, Pablo Mosteiro, Heysem Kaya, Emil Rijcken, Albert Ali Salah, Floortje Scheepers, Marco Spruit

Abstract: Inpatient violence is a common and severe problem within psychiatry. Knowing who might become violent can influence staffing levels and mitigate severity. Predictive machine learning models can assess each patient's likelihood of becoming violent based on clinical notes. Yet, while machine learning models benefit from having more data, data availability is limited as hospitals typically do not sha… ▽ More Inpatient violence is a common and severe problem within psychiatry. Knowing who might become violent can influence staffing levels and mitigate severity. Predictive machine learning models can assess each patient's likelihood of becoming violent based on clinical notes. Yet, while machine learning models benefit from having more data, data availability is limited as hospitals typically do not share their data for privacy preservation. Federated Learning (FL) can overcome the problem of data limitation by training models in a decentralised manner, without disclosing data between collaborators. However, although several FL approaches exist, none of these train Natural Language Processing models on clinical notes. In this work, we investigate the application of Federated Learning to clinical Natural Language Processing, applied to the task of Violence Risk Assessment by simulating a cross-institutional psychiatric setting. We train and compare four models: two local models, a federated model and a data-centralised model. Our results indicate that the federated model outperforms the local models and has similar performance as the data-centralised model. These findings suggest that Federated Learning can be used successfully in a cross-institutional setting and is a step towards new applications of Federated Learning based on clinical notes △ Less

Submitted 17 May, 2022; originally announced May 2022.

Journal ref: Expert Systems with Applications Volume 199, 1 August 2022, 116720

Power-Time Channel Diversity (PTCD): A Novel Resource-Efficient Diversity Technique for 6G and Beyond

Authors: Ferdi Kara, Hakan Kaya, Halim Yanikomeroglu

Abstract: Diversity techniques have been applied for decades to overcome the effects of fading, which is one of the most challenging problems in wireless communications due to the randomness of the wireless channel. However, existing diversity techniques are resource-inefficient due to orthogonal resource usage, or they have high-power consumption due to multiple antennas and RF-chains which present an insu… ▽ More Diversity techniques have been applied for decades to overcome the effects of fading, which is one of the most challenging problems in wireless communications due to the randomness of the wireless channel. However, existing diversity techniques are resource-inefficient due to orthogonal resource usage, or they have high-power consumption due to multiple antennas and RF-chains which present an insurmountable constraint for small devices. To address this, this letter proposes a novel resource-efficient diversity technique called power-time channel diversity (PTCD). In PTCD, interleaved copies of the baseband symbols are transmitted simultaneously with weighted power coefficients. The PTCD provides a diversity order of the number of copies by implementing successive interference canceler at the receiver. To achieve this diversity, no additional resources are needed; hence, spectral efficient communication is guaranteed. Additionally, the power consumption at the transceivers is limited since the PTCD requires only one RF-chain. We provide an information-theoretic proof that the PTCD could have any diversity order. Based on extensive simulations, we reveal that PTCD can also outperform benchmarks without any additional cost. △ Less

Submitted 6 May, 2022; originally announced May 2022.

Comments: Accepted for IEEE WCL

Error Performance Analysis of Multi-user Detection in Uplink-NOMA with Adaptive $\mathcal{M}$-QAM

Authors: Hichem Semira, Ferdi Kara, Hakan Kaya, Halim Yanikomeroglu

Abstract: This work provides a generalized performance analysis for the multi-user uplink-NOMA system with adaptive square quadrature amplitude modulation (QAM) over Rayleigh fading channels. Motivated by the massive IoT connections and unavailability of orthogonal resources for each node, we consider a multi-access scheme where multi-users with single-antenna transmit data to a multiple-antenna base statio… ▽ More This work provides a generalized performance analysis for the multi-user uplink-NOMA system with adaptive square quadrature amplitude modulation (QAM) over Rayleigh fading channels. Motivated by the massive IoT connections and unavailability of orthogonal resources for each node, we consider a multi-access scheme where multi-users with single-antenna transmit data to a multiple-antenna base station through the same resource block. By taking advantage of combining diversity paths with the proposed joint maximum-likelihood detector (JMLD), a closed form expression for the upper bound of bit error rate (BER) is obtained. Despite the number of users or the order of modulation, the analytical results endorsed via computer simulations reveal the ability of the MRC-JMLD detector to discard the error floor completely. Moreover, the simulation results show that the MRC-JMLD surpasses its counterparts significantly and ensures a full diversity order. △ Less

Submitted 28 April, 2022; v1 submitted 25 April, 2022; originally announced April 2022.

Comments: accepted in IEEE WCL

Speech Analysis for Automatic Mania Assessment in Bipolar Disorder

Authors: Pınar Baki, Heysem Kaya, Elvan Çiftçi, Hüseyin Güleç, Albert Ali Salah

Abstract: Bipolar disorder is a mental disorder that causes periods of manic and depressive episodes. In this work, we classify recordings from Bipolar Disorder corpus that contain 7 different tasks, into hypomania, mania, and remission classes using only speech features. We perform our experiments on splitted tasks from the interviews. Best results achieved on the model trained with 6th and 7th tasks toget… ▽ More Bipolar disorder is a mental disorder that causes periods of manic and depressive episodes. In this work, we classify recordings from Bipolar Disorder corpus that contain 7 different tasks, into hypomania, mania, and remission classes using only speech features. We perform our experiments on splitted tasks from the interviews. Best results achieved on the model trained with 6th and 7th tasks together gives 0.53 UAR (unweighted average recall) result which is higher than the baseline results of the corpus. △ Less

Submitted 5 February, 2022; originally announced February 2022.

Comments: Conference, 5 pages, in Turkish language

A Lightweight Machine Learning Assisted Power Optimization for Minimum Error in NOMA-CRS over Nakagami-$m$ channels

Authors: Ferdi Kara, Hakan Kaya, Halim Yanikomeroglu

Abstract: Non-orthogonal multiple access based cooperative relaying system (NOMA-CRS) has been proposed to alleviate the decay in spectral efficiency of the conventional CRS. However, existing NOMA-CRS studies assume perfect successive interference canceler at the relay and mostly investigate sum-rate whereas the error performance has not been taken into consideration. In this paper, we analyze error perfor… ▽ More Non-orthogonal multiple access based cooperative relaying system (NOMA-CRS) has been proposed to alleviate the decay in spectral efficiency of the conventional CRS. However, existing NOMA-CRS studies assume perfect successive interference canceler at the relay and mostly investigate sum-rate whereas the error performance has not been taken into consideration. In this paper, we analyze error performance of the NOMA-CRS and the closed-form bit error probability (BEP) expression is derived over Nakagami-m fading channels. Then, thanks to the high performance of machine learning (ML) in challenging optimization problems, a joint power sharing-power allocation (PS-PA) scheme is proposed to minimize the bit error rate (BER) of the NOMA-CRS. The proposed ML-assisted optimization has a very low online implementation complexity. Based on provided extensive simulations, theoretical BEP analysis is validated. Besides, the proposed ML-aided PS-PA provides minimum BER (MBER) and outperforms previous PA strategies for the NOMA-CRS notably. △ Less

Submitted 28 August, 2021; originally announced August 2021.

Comments: accepted by IEEE TVT

Multi-user Joint Maximum-Likelihood Detection in Uplink NOMA-IoT Networks: Removing the Error Floor

Authors: Hichem Semira, Ferdi Kara, Hakan Kaya, Halim Yanikomeroglu

Abstract: The Internet of Things (IoT) framework requires a massive number of connection thus demanding spectral efficient solutions such as Non-Orthogonal Multiple Access (NOMA). However, the main drawback of NOMA with successive interference canceler (SIC)-based detectors is the error floor in the uplink. In this paper, a reliable multi-user detection in uplink IoT NOMA is guaranteed by a Joint Maximum-Li… ▽ More The Internet of Things (IoT) framework requires a massive number of connection thus demanding spectral efficient solutions such as Non-Orthogonal Multiple Access (NOMA). However, the main drawback of NOMA with successive interference canceler (SIC)-based detectors is the error floor in the uplink. In this paper, a reliable multi-user detection in uplink IoT NOMA is guaranteed by a Joint Maximum-Likelihood (JML) detector (i.e., optimum detection algorithm). We derive a closed-form upper bound of bit error rate (BER) of JML over Rayleigh fading channels for arbitrary number of IoT devices and an adaptive M-ary phase shift keying (M-PSK). Based on the extensive simulations, the derived expressions are validated and it is revealed that the JML improves the error performance in uplink NOMA and removes the error floor. Furthermore, regardless of the number of the IoT devices and modulation order, a full diversity order (i.e., number of receiving antennas) is guaranteed for each device. △ Less

Submitted 12 August, 2021; originally announced August 2021.

Comments: accepted in IEEE WCL

The INTERSPEECH 2021 Computational Paralinguistics Challenge: COVID-19 Cough, COVID-19 Speech, Escalation & Primates

Authors: Björn W. Schuller, Anton Batliner, Christian Bergler, Cecilia Mascolo, Jing Han, Iulia Lefter, Heysem Kaya, Shahin Amiriparian, Alice Baird, Lukas Stappen, Sandra Ottl, Maurice Gerczuk, Panagiotis Tzirakis, Chloë Brown, Jagmohan Chauhan, Andreas Grammenos, Apinan Hasthanasombat, Dimitris Spathis, Tong Xia, Pietro Cicuta, Leon J. M. Rothkrantz, Joeri Zwerts, Jelle Treep, Casper Kaandorp

Abstract: The INTERSPEECH 2021 Computational Paralinguistics Challenge addresses four different problems for the first time in a research competition under well-defined conditions: In the COVID-19 Cough and COVID-19 Speech Sub-Challenges, a binary classification on COVID-19 infection has to be made based on coughing sounds and speech; in the Escalation SubChallenge, a three-way assessment of the level of es… ▽ More The INTERSPEECH 2021 Computational Paralinguistics Challenge addresses four different problems for the first time in a research competition under well-defined conditions: In the COVID-19 Cough and COVID-19 Speech Sub-Challenges, a binary classification on COVID-19 infection has to be made based on coughing sounds and speech; in the Escalation SubChallenge, a three-way assessment of the level of escalation in a dialogue is featured; and in the Primates Sub-Challenge, four species vs background need to be classified. We describe the Sub-Challenges, baseline feature extraction, and classifiers based on the 'usual' COMPARE and BoAW features as well as deep unsupervised representation learning using the AuDeep toolkit, and deep feature extraction from pre-trained CNNs using the Deep Spectrum toolkit; in addition, we add deep end-to-end sequential modelling, and partially linguistic analysis. △ Less

Submitted 24 February, 2021; originally announced February 2021.

Comments: 5 pages

MSC Class: 68 ACM Class: I.2.7; I.5.0; J.3

DeepMuD: Multi-user Detection for Uplink Grant-Free NOMA IoT Networks via Deep Learning

Authors: Ahmet Emir, Ferdi Kara, Hakan Kaya, Halim Yanikomeroglu

Abstract: In this letter, we propose a deep learning-aided multi-user detection (DeepMuD) in uplink non-orthogonal multiple access (NOMA) to empower the massive machine-type communication where an offline-trained Long Short-Term Memory (LSTM)-based network is used for multi-user detection. In the proposed DeepMuD, a perfect channel state information (CSI) is also not required since it is able to perform a j… ▽ More In this letter, we propose a deep learning-aided multi-user detection (DeepMuD) in uplink non-orthogonal multiple access (NOMA) to empower the massive machine-type communication where an offline-trained Long Short-Term Memory (LSTM)-based network is used for multi-user detection. In the proposed DeepMuD, a perfect channel state information (CSI) is also not required since it is able to perform a joint channel estimation and multi-user detection with the pilot responses, where the pilot-to-frame ratio is very low. The proposed DeepMuD improves the error performance of the uplink NOMA significantly and outperforms the conventional detectors (even with perfect CSI). Moreover, this gain becomes superb with the increase in the number of Internet of Things (IoT) devices. Furthermore, the proposed DeepMuD has a flexible detection and regardless of the number of IoT devices, the multi-user detection can be performed. Thus, an arbitrary number of IoT devices can be served without a signaling overhead, which enables the grant-free communication. △ Less

Submitted 18 February, 2021; originally announced February 2021.

Comments: Submitted/accepted in IEEE WCL

Introducing a Central African Primate Vocalisation Dataset for Automated Species Classification

Authors: Joeri A. Zwerts, Jelle Treep, Casper S. Kaandorp, Floor Meewis, Amparo C. Koot, Heysem Kaya

Abstract: Automated classification of animal vocalisations is a potentially powerful wildlife monitoring tool. Training robust classifiers requires sizable annotated datasets, which are not easily recorded in the wild. To circumvent this problem, we recorded four primate species under semi-natural conditions in a wildlife sanctuary in Cameroon with the objective to train a classifier capable of detecting sp… ▽ More Automated classification of animal vocalisations is a potentially powerful wildlife monitoring tool. Training robust classifiers requires sizable annotated datasets, which are not easily recorded in the wild. To circumvent this problem, we recorded four primate species under semi-natural conditions in a wildlife sanctuary in Cameroon with the objective to train a classifier capable of detecting species in the wild. Here, we introduce the collected dataset, describe our approach and initial results of classifier development. To increase the efficiency of the annotation process, we condensed the recordings with an energy/change based automatic vocalisation detection. Segmenting the annotated chunks into training, validation and test sets, initial results reveal up to 82% unweighted average recall (UAR) test set performance in four-class primate species classification. △ Less

Submitted 25 January, 2021; originally announced January 2021.

Comments: 5 pages, 3 figures, 2 tables

An Audio-Video Deep and Transfer Learning Framework for Multimodal Emotion Recognition in the wild

Authors: Denis Dresvyanskiy, Elena Ryumina, Heysem Kaya, Maxim Markitantov, Alexey Karpov, Wolfgang Minker

Abstract: In this paper, we present our contribution to ABAW facial expression challenge. We report the proposed system and the official challenge results adhering to the challenge protocol. Using end-to-end deep learning and benefiting from transfer learning approaches, we reached a test set challenge performance measure of 42.10%. In this paper, we present our contribution to ABAW facial expression challenge. We report the proposed system and the official challenge results adhering to the challenge protocol. Using end-to-end deep learning and benefiting from transfer learning approaches, we reached a test set challenge performance measure of 42.10%. △ Less

Submitted 2 November, 2020; v1 submitted 7 October, 2020; originally announced October 2020.

Comments: Results on test dataset and acknowledgements were added

Is Everything Fine, Grandma? Acoustic and Linguistic Modeling for Robust Elderly Speech Emotion Recognition

Authors: Gizem Soğancıoğlu, Oxana Verkholyak, Heysem Kaya, Dmitrii Fedotov, Tobias Cadèe, Albert Ali Salah, Alexey Karpov

Abstract: Acoustic and linguistic analysis for elderly emotion recognition is an under-studied and challenging research direction, but essential for the creation of digital assistants for the elderly, as well as unobtrusive telemonitoring of elderly in their residences for mental healthcare purposes. This paper presents our contribution to the INTERSPEECH 2020 Computational Paralinguistics Challenge (ComPar… ▽ More Acoustic and linguistic analysis for elderly emotion recognition is an under-studied and challenging research direction, but essential for the creation of digital assistants for the elderly, as well as unobtrusive telemonitoring of elderly in their residences for mental healthcare purposes. This paper presents our contribution to the INTERSPEECH 2020 Computational Paralinguistics Challenge (ComParE) - Elderly Emotion Sub-Challenge, which is comprised of two ternary classification tasks for arousal and valence recognition. We propose a bi-modal framework, where these tasks are modeled using state-of-the-art acoustic and linguistic features, respectively. In this study, we demonstrate that exploiting task-specific dictionaries and resources can boost the performance of linguistic models, when the amount of labeled data is small. Observing a high mismatch between development and test set performances of various models, we also propose alternative training and decision fusion strategies to better estimate and improve the generalization performance. △ Less

Submitted 7 September, 2020; originally announced September 2020.

Comments: 5 pages, 1 figure, Interspeech 2020

Improved Error Performance in NOMA-based Diamond Relaying

Authors: Ferdi Kara, Hakan Kaya

Abstract: Non-orthogonal multiple access (NOMA)-based cooperative relaying systems (CRS) has emerged as a solution to the spectral inefficiency problem of the conventional cooperative relaying systems thanks to the NOMA integration. Thus, as a subset of NOMA-CRS, the NOMA-based diamond relaying network (NOMA-DRN) also provides a performance gain in terms of throughput. However, the NOMA-DRN has a poor error… ▽ More Non-orthogonal multiple access (NOMA)-based cooperative relaying systems (CRS) has emerged as a solution to the spectral inefficiency problem of the conventional cooperative relaying systems thanks to the NOMA integration. Thus, as a subset of NOMA-CRS, the NOMA-based diamond relaying network (NOMA-DRN) also provides a performance gain in terms of throughput. However, the NOMA-DRN has a poor error performance due to the second phase (uplink), indeed, it has a error floor regardless of the transmit power, power allocation and channel qualities. To address this problem, in this paper, we propose a novel NOMA-DRN schemes where joint maximum likelihood (JML) decoding is implemented at the destination. Then, we define the performance metrics (i.e., bit error rate (BER) and the diversity order ) of the NOMA-DRN with the JML and analyze the computational complexity. Moreover, we demonstrate that the new NOMA-DRN with JML can cope with the error floor penalty of the conventional NOMA-DRN. Hence, a spectral efficient NOMA-CRS scheme can be achieved with high data reliability. Specifically, this improvement can reach to $\sim20-30dB$ in the transmit power which is superb gain in terms of energy efficiency perspective. Furthermore, with the proposed NOMA-DRN with the JML, the full diversity order can be achieved in the low-medium SNR region. △ Less

Submitted 6 September, 2020; originally announced September 2020.

Comments: Accepted for IEEE MTTW'20

Pilot Interval Reduction by Deep Learning Based Detectors in Uplink NOMA

Authors: Ahmet Emir, Ferdi Kara, Hakan Kaya

Abstract: Non-Orthogonal Multiple Access (NOMA) has higher spectral efficiency than orthogonal multiple access (OMA) techniques. In uplink communication systems that the channel is not known at the receiver, pilot signals sent from each user in different time intervals have reduced the spectral efficiency of NOMA. In this study, in the uplink communication system, DL-deep learning based detectors which are… ▽ More Non-Orthogonal Multiple Access (NOMA) has higher spectral efficiency than orthogonal multiple access (OMA) techniques. In uplink communication systems that the channel is not known at the receiver, pilot signals sent from each user in different time intervals have reduced the spectral efficiency of NOMA. In this study, in the uplink communication system, DL-deep learning based detectors which are known to respond to the pilot signals sent from the users at the base station have been researched. It is aimed to maintain the spectral efficiency of NOMA by sending a single pilot from users, thus reducing the time interval in the DL detectors. △ Less

Submitted 26 April, 2020; originally announced April 2020.

Comments: in Turkish language. accepted for IEEE SIU 2020

Error Analysis of Decode-Forward Cooperative Relaying NOMA Schemes over Nakagami-m Fading Channels

Authors: Ferdi Kara, Hakan Kaya

Abstract: Non-orthogonal multiple access (NOMA) has attracted great recent attention due to its high spectral efficiency. Besides, NOMA can be easily implemented in physical layer, thus its interplays with other psychical layer techniques have been analyzed widely by researches. Cooperative NOMA schemes are the most investigated among these interplays. However, these studies mostly analyze cooperative NOMA… ▽ More Non-orthogonal multiple access (NOMA) has attracted great recent attention due to its high spectral efficiency. Besides, NOMA can be easily implemented in physical layer, thus its interplays with other psychical layer techniques have been analyzed widely by researches. Cooperative NOMA schemes are the most investigated among these interplays. However, these studies mostly analyze cooperative NOMA schemes in terms of achievable rate and outage probability whereas bit error probability (BEP) for those systems has not been studied well although it is one of the most important performance metrics. In this paper, we investigate the error performance of cooperative NOMA schemes where a decode-forward relay helps NOMA users. We derive exact BEP expressions in closed-form over Nakagami-m fading channels. All derived expressions are validated via computer simulations. △ Less

Submitted 26 April, 2020; originally announced April 2020.

Comments: in Turkish language, accepted for IEEE SIU 2020

Error Probability Analysis of Non-Orthogonal Multiple Access with Channel Estimation Errors

Authors: Ferdi Kara, Hakan Kaya

Abstract: Non-orthogonal multiple access (NOMA) is very promising for future wireless systems thanks to its spectral efficiency. In NOMA schemes, the effect of imperfect successive interference canceler (SIC) has dominant effect on the error performances. In addition to this imperfect SIC effect, the error performance will get worse with the channel estimation errors just as in all wireless communications s… ▽ More Non-orthogonal multiple access (NOMA) is very promising for future wireless systems thanks to its spectral efficiency. In NOMA schemes, the effect of imperfect successive interference canceler (SIC) has dominant effect on the error performances. In addition to this imperfect SIC effect, the error performance will get worse with the channel estimation errors just as in all wireless communications systems. However, all literature has been devoted to analyze error performance of NOMA systems with the perfect channel state information (CSI) at the receivers which is very strict/unreasonable assumption. In this paper, we analyze error performance of NOMA systems with imperfect SIC and channel estimation errors, much more practical scenario. We derive exact bit error probabilities (BEPs) in closed-forms. All theoretical analysis is validated via computer simulations. Then, we discuss optimum power allocation for user fairness in terms of error performance of users and propose a novel power allocation scheme which achieves maximum user fairness. △ Less

Submitted 26 April, 2020; originally announced April 2020.

Comments: accepted for IEEE BlackSeaCom 2020

Improved User Fairness in Decode-Forward Relaying Non-orthogonal Multiple Access Schemes with Imperfect SIC

Authors: Ferdi Kara, Hakan Kaya

Abstract: Non-orthogonal multiple access (NOMA) is one of the key technologies to serve in ultra-dense networks with massive connections which is crucial for Internet of Things (IoT) applications. Besides, NOMA provides better spectral efficiency compared to orthogonal multiple access (OMA) schemes. However, in NOMA, successive interference canceler (SIC) should be implemented for interference mitigation an… ▽ More Non-orthogonal multiple access (NOMA) is one of the key technologies to serve in ultra-dense networks with massive connections which is crucial for Internet of Things (IoT) applications. Besides, NOMA provides better spectral efficiency compared to orthogonal multiple access (OMA) schemes. However, in NOMA, successive interference canceler (SIC) should be implemented for interference mitigation and mostly in the literature, perfect SIC is assumed for NOMA involved systems. Unfortunately, this is not the case for practical scenarios and this imperfect SIC effect limits the performance of NOMA involved systems. In addition, it causes unfairness between users. In this paper, we introduce reversed decode-forward relaying NOMA (R-DFNOMA) to improve user fairness compared to conventional DFNOMA (C-DFNOMA) which is widely analyzed in literature. In the analysis, we define imperfect SIC effect dependant to channel fading and with this imperfect SIC, we derive exact expressions for ergodic capacity (EC) and outage probability (OP). Moreover, we evaluate bit error performance of proposed R-DFNOMA and derive bit error probability (BEP) in closed-form which has not been also studied well in literature. Then, we define user fairness index in terms of all key performance indicators (KPIs) (i.e., EC, OP and BEP). Based on extensive simulations, all derived expressions are validated, and it is proved that proposed R-DFNOMA provides better user fairness than C-DFNOMA in terms of all KPIs. Finally, we discuss the effect of power allocations at the both source and relay on the performance metrics and user fairness △ Less

Submitted 26 April, 2020; originally announced April 2020.

Comments: submitted to IEEE

Error Probability Analysis of NOMA-based Diamond Relaying Network

Authors: Ferdi Kara, Hakan Kaya

Abstract: Non-orthogonal multiple access (NOMA)-based cooperative relaying systems (CRS) are very promising to overcome spectral inefficiency of conventional cooperative communications. Although NOMA-CRS have great recent attention, almost all studies investigate NOMA-CRS only in terms of capacity and outage probability. Error performances of NOMA-CRS have not been well-studied. In this paper, we analyze er… ▽ More Non-orthogonal multiple access (NOMA)-based cooperative relaying systems (CRS) are very promising to overcome spectral inefficiency of conventional cooperative communications. Although NOMA-CRS have great recent attention, almost all studies investigate NOMA-CRS only in terms of capacity and outage probability. Error performances of NOMA-CRS have not been well-studied. In this paper, we analyze error performance of NOMA-based diamond relaying network (NOMA-DRN) with imperfect successive interference canceler (SIC) as a NOMA-CRS scheme. We derive exact bit error probability (BEP) for NOMA-DRN and provide a tight approximated BEP in the closed-form. In addition, high-SNR analysis is conducted to present that NOMA-DRN has an error floor. Moreover, it is proved that NOMA-DRN turns out to be a non-equiprobable communication system and we derive priori probabilities of symbols. All derived expressions are validated via computer simulations. △ Less

Submitted 10 January, 2020; originally announced January 2020.

Journal ref: in IEEE Transactions on Vehicular Technology, vol. 69, no. 2, pp. 2280-2285, Feb. 2020

Performance Analysis of SSK-NOMA

Authors: Ferdi Kara, Hakan Kaya

Abstract: In this paper, we consider the combination between two promising techniques: space-shift keying (SSK) and non-orthogonal multiple access (NOMA) for future radio access networks. We analyze the performance of SSK-NOMA networks and provide a comprehensive analytical framework of SSK-NOMA regarding bit error probability (BEP), ergodic capacity and outage probability. It is worth pointing out all anal… ▽ More In this paper, we consider the combination between two promising techniques: space-shift keying (SSK) and non-orthogonal multiple access (NOMA) for future radio access networks. We analyze the performance of SSK-NOMA networks and provide a comprehensive analytical framework of SSK-NOMA regarding bit error probability (BEP), ergodic capacity and outage probability. It is worth pointing out all analysis also stand for conventional SIMO-NOMA networks. We derive closed-form exact average BEP (ABEP) expressions when the number of users in a resource block is equal to i.e., $L=3$. Nevertheless, we analyze the ABEP of users when the number of users is more than i.e., $L\geq3$, and derive bit-error-rate (BER) union bound since the error propagation due to iterative successive interference canceler (SIC) makes the exact analysis intractable. Then, we analyze the achievable rate of users and derive exact ergodic capacity of the users so the ergodic sum rate of the system in closed-forms. Moreover, we provide the average outage probability of the users exactly in the closed-form. All derived expressions are validated via Monte Carlo simulations and it is proved that SSK-NOMA outperforms conventional NOMA networks in terms of all performance metrics (i.e., BER, sum rate, outage). Finally, the effect of the power allocation (PA) on the performance of SSK-NOMA networks is investigated and the optimum PA is discussed under BER and outage constraints. △ Less

Submitted 2 May, 2019; originally announced May 2019.

Journal ref: in IEEE Transactions on Vehicular Technology, vol. 68, no. 7, pp. 6231-6242, July 2019

Threshold-based Selective Cooperative-NOMA

Authors: Ferdi Kara, Hakan Kaya

Abstract: In this letter, we propose threshold-based selective cooperative-NOMA (TBS-C-NOMA) to increase the data reliability of conventional cooperative-NOMA (C-NOMA) networks. In TBS-C-NOMA, the intra-cell user forwards the symbols of cell-edge user after successive interference canceler (SIC) only if the signal-to-interference plus noise ratio (SINR) is greater than the pre-determined threshold value. He… ▽ More In this letter, we propose threshold-based selective cooperative-NOMA (TBS-C-NOMA) to increase the data reliability of conventional cooperative-NOMA (C-NOMA) networks. In TBS-C-NOMA, the intra-cell user forwards the symbols of cell-edge user after successive interference canceler (SIC) only if the signal-to-interference plus noise ratio (SINR) is greater than the pre-determined threshold value. Hence, the data reliability of the cell-edge user is increased by eliminating the effect of the error propagation. We derive closed-form end-to-end exact bit error probability (BEP) of proposed system for various modulation constellations. Then, the optimum threshold value is analyzed in order to minimize BEP. The obtained expressions are validated via simulations and it is revealed that TBS-C-NOMA outperforms C-NOMA and full diversity order is achieved. △ Less

Submitted 2 May, 2019; originally announced May 2019.

Journal ref: in IEEE Communications Letters, vol. 23, no. 7, pp. 1263-1266, July 2019

Spatial Multiple Access (SMA): Enhancing performances of MIMO-NOMA systems

Authors: Ferdi Kara, Hakan Kaya

Abstract: The error performance of the Non-Orthogonal Multiple Access (NOMA) technique suffers from the inter-user interference (IUI) although it is a promising technique for the future wireless systems in terms of the achievable sum rate. Hence, a multiple access technique design with limited IUI and competitive to NOMA in terms of spectral efficiency is essential. In this letter, we consider so-called spa… ▽ More The error performance of the Non-Orthogonal Multiple Access (NOMA) technique suffers from the inter-user interference (IUI) although it is a promising technique for the future wireless systems in terms of the achievable sum rate. Hence, a multiple access technique design with limited IUI and competitive to NOMA in terms of spectral efficiency is essential. In this letter, we consider so-called spatial multiple access (SMA) which is based on applying the principle of spatial modulation (SM) through the different users' data streams, as a strong alternative to MIMO-NOMA systems. The analytical expressions of bit error probability (BEP), ergodic sum rate and outage probability are derived for the SMA. The derivations are validated via computer simulations. In addition, the comparison of the SMA system with NOMA is presented. The results reveal that SMA outperforms to the NOMA in terms of the all performance metrics (i.e., bit error rate (BER), outage probability and ergodic sum rate) besides it provides low implementation complexity. △ Less

Submitted 18 October, 2018; originally announced October 2018.

Comments: 10 pages,4 figures

Effect of the Error Propagation on the Error Performance of Cooperative Communications with the Best Relay Selection Schemes

Authors: Ezgi Sanli, Ferdi Kara, Hakan Kaya

Abstract: In this paper, error performance of the cooperative communication systems with the best relay selection scheme is investigated in the presence of error propagation from role to destination. The error propagation expression is derived firstly when the best relay is selected within M relays. The derived end-to-end BER expression is verified with the computer simulations. It is shown that the best re… ▽ More In this paper, error performance of the cooperative communication systems with the best relay selection scheme is investigated in the presence of error propagation from role to destination. The error propagation expression is derived firstly when the best relay is selected within M relays. The derived end-to-end BER expression is verified with the computer simulations. It is shown that the best relay selection does not ensure M+1 diversity order under the error propagation unlike perfect decoding. In addition, the threshold selection for the relays has dominant effect on the error performance of the system. △ Less

Submitted 12 July, 2018; originally announced July 2018.

Comments: 26.IEEE Signal Processing and Communications Applications Conference,Izmir,Turkey,2017

MSC Class: 94-06

Derivation of the closed-form BER expressions for DL-NOMA over Nakagami-m fading channels

Authors: Ferdi Kara, Hakan Kaya

Abstract: NOMA is as a strong candidate for the Future Radio Access Network (FRA) due to its potential to support massive connectivity and high spectral efficiency. However, the most important drawback of NOMA is the error during Successive Interference Canceller (SIC) is implemented because of the inter-user interferences. In this paper, we derive closed-form exact Bit-Error Rate expressions for Downlink(D… ▽ More NOMA is as a strong candidate for the Future Radio Access Network (FRA) due to its potential to support massive connectivity and high spectral efficiency. However, the most important drawback of NOMA is the error during Successive Interference Canceller (SIC) is implemented because of the inter-user interferences. In this paper, we derive closed-form exact Bit-Error Rate expressions for Downlink(DL) NOMA over Nakagami-m fading channels in the presence of SIC errors. The derived expressions are validated by the computer simulations. It is shown that the m parameter still represents the diversity order like as OMA systems. Besides, the BER performances of users for NOMA have substantially depended on the power allocation coefficient. △ Less

Submitted 12 July, 2018; originally announced July 2018.

Comments: 26.IEEE Signal Processing and Communications Applications Conference,Izmir,Turkey

Explaining First Impressions: Modeling, Recognizing, and Explaining Apparent Personality from Videos

Authors: Hugo Jair Escalante, Heysem Kaya, Albert Ali Salah, Sergio Escalera, Yagmur Gucluturk, Umut Guclu, Xavier Baro, Isabelle Guyon, Julio Jacques Junior, Meysam Madadi, Stephane Ayache, Evelyne Viegas, Furkan Gurpinar, Achmadnoer Sukma Wicaksana, Cynthia C. S. Liem, Marcel A. J. van Gerven, Rob van Lier

Abstract: Explainability and interpretability are two critical aspects of decision support systems. Within computer vision, they are critical in certain tasks related to human behavior analysis such as in health care applications. Despite their importance, it is only recently that researchers are starting to explore these aspects. This paper provides an introduction to explainability and interpretability in… ▽ More Explainability and interpretability are two critical aspects of decision support systems. Within computer vision, they are critical in certain tasks related to human behavior analysis such as in health care applications. Despite their importance, it is only recently that researchers are starting to explore these aspects. This paper provides an introduction to explainability and interpretability in the context of computer vision with an emphasis on looking at people tasks. Specifically, we review and study those mechanisms in the context of first impressions analysis. To the best of our knowledge, this is the first effort in this direction. Additionally, we describe a challenge we organized on explainability in first impressions analysis from video. We analyze in detail the newly introduced data set, the evaluation protocol, and summarize the results of the challenge. Finally, derived from our study, we outline research opportunities that we foresee will be decisive in the near future for the development of the explainable computer vision field. △ Less

Submitted 28 September, 2019; v1 submitted 2 February, 2018; originally announced February 2018.

Comments: Preprint submitted to TAC

Prediction of the Optimal Threshold Value in DF Relay Selection Schemes Based on Artificial Neural Networks

Authors: Ferdi Kara, Hakan Kaya, Okan Erkaymaz, Ertan Ozturk

Abstract: In wireless communications, the cooperative communication (CC) technology promises performance gains compared to traditional Single-Input Single Output (SISO) techniques. Therefore, the CC technique is one of the nominees for 5G networks. In the Decode-and-Forward (DF) relaying scheme which is one of the CC techniques, determination of the threshold value at the relay has a key role for the system… ▽ More In wireless communications, the cooperative communication (CC) technology promises performance gains compared to traditional Single-Input Single Output (SISO) techniques. Therefore, the CC technique is one of the nominees for 5G networks. In the Decode-and-Forward (DF) relaying scheme which is one of the CC techniques, determination of the threshold value at the relay has a key role for the system performance and power usage. In this paper, we propose prediction of the optimal threshold values for the best relay selection scheme in cooperative communications, based on Artificial Neural Networks (ANNs) for the first time in literature. The average link qualities and number of relays have been used as inputs in the prediction of optimal threshold values using Artificial Neural Networks (ANNs): Multi-Layer Perceptron (MLP) and Radial Basis Function (RBF) networks. The MLP network has better performance from the RBF network on the prediction of optimal threshold value when the same number of neurons is used at the hidden layer for both networks. Besides, the optimal threshold values obtained using ANNs are verified by the optimal threshold values obtained numerically using the closed form expression derived for the system. The results show that the optimal threshold values obtained by ANNs on the best relay selection scheme provide a minimum Bit-Error-Rate (BER) because of the reduction of the probability that error propagation may occur. Also, for the same BER performance goal, prediction of optimal threshold values provides 2dB less power usage, which is great gain in terms of green communicationBER performance goal, prediction of optimal threshold values provides 2dB less power usage, which is great gain in terms of green communication. △ Less

Submitted 18 January, 2018; originally announced January 2018.

Comments: 6 pages,IEEE INnovations in Intelligent SysTems and Applications (INISTA), 2016 International Symposium on

Adaptive Mixtures of Factor Analyzers

Authors: Heysem Kaya, Albert Ali Salah

Abstract: A mixture of factor analyzers is a semi-parametric density estimator that generalizes the well-known mixtures of Gaussians model by allowing each Gaussian in the mixture to be represented in a different lower-dimensional manifold. This paper presents a robust and parsimonious model selection algorithm for training a mixture of factor analyzers, carrying out simultaneous clustering and locally line… ▽ More A mixture of factor analyzers is a semi-parametric density estimator that generalizes the well-known mixtures of Gaussians model by allowing each Gaussian in the mixture to be represented in a different lower-dimensional manifold. This paper presents a robust and parsimonious model selection algorithm for training a mixture of factor analyzers, carrying out simultaneous clustering and locally linear, globally nonlinear dimensionality reduction. Permitting different number of factors per mixture component, the algorithm adapts the model complexity to the data complexity. We compare the proposed algorithm with related automatic model selection algorithms on a number of benchmarks. The results indicate the effectiveness of this fast and robust approach in clustering, manifold learning and class-conditional modeling. △ Less

Submitted 22 October, 2015; v1 submitted 10 July, 2015; originally announced July 2015.

Comments: Pre-print has 30 pages including the appendix and references. A MATLAB tool of the proposed method is available (see the conclusions section)

ACM Class: G.3; I.5.4