-
Knowledge Distillation for Adaptive MRI Prostate Segmentation Based on Limit-Trained Multi-Teacher Models
Authors:
Eddardaa Ben Loussaief,
Hatem Rashwan,
Mohammed Ayad,
Mohammed Zakaria Hassan,
Domenec Puig
Abstract:
With numerous medical tasks, the performance of deep models has recently experienced considerable improvements. These models are often adept learners. Yet, their intricate architectural design and high computational complexity make deploying them in clinical settings challenging, particularly with devices with limited resources. To deal with this issue, Knowledge Distillation (KD) has been propose…
▽ More
With numerous medical tasks, the performance of deep models has recently experienced considerable improvements. These models are often adept learners. Yet, their intricate architectural design and high computational complexity make deploying them in clinical settings challenging, particularly with devices with limited resources. To deal with this issue, Knowledge Distillation (KD) has been proposed as a compression method and an acceleration technology. KD is an efficient learning strategy that can transfer knowledge from a burdensome model (i.e., teacher model) to a lightweight model (i.e., student model). Hence we can obtain a compact model with low parameters with preserving the teacher's performance. Therefore, we develop a KD-based deep model for prostate MRI segmentation in this work by combining features-based distillation with Kullback-Leibler divergence, Lovasz, and Dice losses. We further demonstrate its effectiveness by applying two compression procedures: 1) distilling knowledge to a student model from a single well-trained teacher, and 2) since most of the medical applications have a small dataset, we train multiple teachers that each one trained with a small set of images to learn an adaptive student model as close to the teachers as possible considering the desired accuracy and fast inference time. Extensive experiments were conducted on a public multi-site prostate tumor dataset, showing that the proposed adaptation KD strategy improves the dice similarity score by 9%, outperforming all tested well-established baseline models.
△ Less
Submitted 16 March, 2023;
originally announced March 2023.
-
Decentralized Model Dissemination Empowered Federated Learning in mmWave Aerial-Terrestrial Integrated Networks
Authors:
Mohammed S. Al-Abiad,
Md. Zoheb Hassan,
Md. Jahangir Hossain
Abstract:
It is anticipated that aerial-terrestrial integrated networks incorporating unmanned aerial vehicles (UAVs) mounted relays will offer improved coverage and connectivity in the beyond 5G era. Meanwhile, federated learning (FL) is a promising distributed machine learning technique for building inference models over wireless networks due to its ability to maintain user privacy and reduce communicatio…
▽ More
It is anticipated that aerial-terrestrial integrated networks incorporating unmanned aerial vehicles (UAVs) mounted relays will offer improved coverage and connectivity in the beyond 5G era. Meanwhile, federated learning (FL) is a promising distributed machine learning technique for building inference models over wireless networks due to its ability to maintain user privacy and reduce communication overhead. However, off-the-shelf FL models aggregate global parameters at a central parameter server (CPS), increasing energy consumption and latency, as well as inefficiently utilizing radio resource blocks (RRBs) for distributed user devices (UDs). This paper presents a resource-efficient FL framework, called FedMoD (\textbf{fed}erated learning with \textbf{mo}del \textbf{d}issemination), for millimeter-wave (mmWave) aerial-terrestrial integrated networks with the following two unique characteristics. Firstly, FedMoD presents a novel decentralized model dissemination algorithm that makes use of UAVs as local model aggregators through UAV-to-UAV and device-to-device (D2D) communications. As a result, FedMoD (i) increases the number of participant UDs in developing FL model and (ii) achieves global model aggregation without involving CPS. Secondly, FedMoD reduces the energy consumption of FL using radio resource management (RRM) under the constraints of over-the-air learning latency. In order to achieve this, by leveraging graph theory, FedMoD optimizes the scheduling of line-of-sight (LOS) UDs to suitable UAVs/RRBs over mmWave links and non-LOS UDs to available LOS UDs via overlay D2D communications. Extensive simulations reveal that decentralized FedMoD offers same convergence rate performance as compared to conventional FL frameworks.
△ Less
Submitted 28 February, 2023;
originally announced March 2023.
-
Energy Efficient Federated Learning in Integrated Fog-Cloud Computing Enabled Internet-of-Things Networks
Authors:
Mohammed S. Al-Abiad,
Md. Zoheb Hassan,
Md. Jahangir Hossain
Abstract:
We investigate resource allocation scheme to reduce the energy consumption of federated learning (FL) in the integrated fog-cloud computing enabled Internet-of-things (IoT) networks. In the envisioned system, IoT devices are connected with the centralized cloud server (CS) via multiple fog access points (F-APs). We consider two different scenarios for training the local models. In the first scenar…
▽ More
We investigate resource allocation scheme to reduce the energy consumption of federated learning (FL) in the integrated fog-cloud computing enabled Internet-of-things (IoT) networks. In the envisioned system, IoT devices are connected with the centralized cloud server (CS) via multiple fog access points (F-APs). We consider two different scenarios for training the local models. In the first scenario, local models are trained at the IoT devices and the F-APs upload the local model parameters to the CS. In the second scenario, local models are trained at the F-APs based on the collected data from the IoT devices and the F-APs collaborate with the CS for updating the model parameters. Our objective is to minimize the overall energy-consumption of both scenarios subject to FL time constraint. Towards this goal, we devise a joint optimization of scheduling of IoT devices with the F-APs, transmit power allocation, computation frequency allocation at the devices and F-APs and decouple it into two subproblems. In the first subproblem, we optimize the IoT device scheduling and power allocation, while in the second subproblem, we optimize the computation frequency allocation. For each scenario, we develop a conflict graph based solution to iteratively solve the two subproblems. Simulation results show that the proposed two schemes achieve a considerable performance gain in terms of the energy consumption minimization. The presented simulation results interestingly reveal that for a large number of IoT devices and large data sizes, it is more energy efficient to train the local models at the IoT devices instead of the F-APs.
△ Less
Submitted 7 July, 2021;
originally announced July 2021.
-
Task Offloading Optimization in NOMA-Enabled Multi-hop Mobile Edge Computing System Using Conflict Graph
Authors:
Mohammed S. Al-Abiad,
Md. Zoheb Hassan,
Md. Jahangir Hossain
Abstract:
Resource allocation is investigated for offloading computational-intensive tasks in multi-hop mobile edge computing (MEC) system. The envisioned system has both the cooperative access points (AP) with the computing capability and the MEC servers. A user-device (UD) therefore first uploads a computing task to the nearest AP, and the AP can either locally process the received task or offload to MEC…
▽ More
Resource allocation is investigated for offloading computational-intensive tasks in multi-hop mobile edge computing (MEC) system. The envisioned system has both the cooperative access points (AP) with the computing capability and the MEC servers. A user-device (UD) therefore first uploads a computing task to the nearest AP, and the AP can either locally process the received task or offload to MEC server. In order to utilize the radio resource blocks (RRBs) in the APs efficiently, we exploit the non-orthogonal multiple access for offloading the tasks from the UDs to the AP(s). For the considered NOMA-enabled multi-hop MEC computing system, our objective is to minimize both the latency and energy consumption of the system jointly. Towards this goal, a joint optimization problem is formulated by taking the offloading decision of the APs, the scheduling among the UDs, RRBs, and APs, and UDs' transmit power allocation into account. To solve this problem efficiently, (i) a conflict graph-based approach is devised that solves the scheduling among the UDs, APs, and RRBs, the transmit power control, and the APs' computation resource allocation jointly, and (ii) a low-complexity pruning graph-based approach is also devised. The efficiency of the proposed graph-based approaches over several benchmark schemes is verified via extensive simulations.
△ Less
Submitted 23 April, 2021;
originally announced April 2021.
-
A Joint Reinforcement-Learning Enabled Caching and Cross-Layer Network Code for Sum-Rate Maximization in F-RAN with D2D Communications
Authors:
Mohammed S. Al-Abiad,
Md. Zoheb Hassan,
Md. Jahangir Hossain
Abstract:
In this paper, we leverage reinforcement learning (RL) and cross-layer network coding (CLNC) for efficiently pre-fetching users' contents to the local caches and delivering these contents to users in a downlink fog-radio access network (F-RAN) with device-to-device (D2D) communications. In the considered system, fog access points (F-APs) and cache-enabled D2D (CE-D2D) users are equipped with local…
▽ More
In this paper, we leverage reinforcement learning (RL) and cross-layer network coding (CLNC) for efficiently pre-fetching users' contents to the local caches and delivering these contents to users in a downlink fog-radio access network (F-RAN) with device-to-device (D2D) communications. In the considered system, fog access points (F-APs) and cache-enabled D2D (CE-D2D) users are equipped with local caches for alleviating traffic burden at the fronthaul, while users' contents can be easily and quickly accommodated. In CLNC, the coding decisions take users' contents, their rates, and power levels of F-APs and CE-D2D users into account, and RL optimizes caching strategy. Towards this goal, a joint content placement and delivery problem is formulated as an optimization problem with a goal to maximize system sum-rate. For this NP-hard problem, we first develop an innovative decentralized CLNC coalition formation (CLNC-CF) algorithm to obtain a stable solution for the content delivery problem, where F-APs and CE-D2D users utilize CLNC resource allocation. By taking the behavior of F-APs and CE-D2D users into account, we then develop a multi-agent RL (MARL) algorithm for optimizing the content placements at both F-APs and CE-D2D users. Simulation results show that the proposed joint CLNC-CF and RL framework can effectively improve the sum-rate by up to 30\%, 60\%, and 150\%, respectively, compared to: 1) an optimal uncoded algorithm, 2) a standard rate-aware-NC algorithm, and 3) a benchmark classical NC with network-layer optimization.
△ Less
Submitted 22 March, 2021;
originally announced March 2021.
-
Throughput Maximization of Network-Coded and Multi-Level Cache-Enabled Heterogeneous Network
Authors:
Mohammed S. Al-Abiad,
Md. Zoheb Hassan,
Md. Jahangir Hossain
Abstract:
One of the paramount advantages of multi-level cache-enabled (MLCE) networks is pushing contents proximity to the network edge and proactively caching them at multiple transmitters (i.e., small base-stations (SBSs), unmanned aerial vehicles (UAVs), and cache-enabled device-to-device (CE-D2D) users). As such, the fronthaul congestion between a core network and a large number of transmitters is alle…
▽ More
One of the paramount advantages of multi-level cache-enabled (MLCE) networks is pushing contents proximity to the network edge and proactively caching them at multiple transmitters (i.e., small base-stations (SBSs), unmanned aerial vehicles (UAVs), and cache-enabled device-to-device (CE-D2D) users). As such, the fronthaul congestion between a core network and a large number of transmitters is alleviated. For this objective, we exploit network coding (NC) to schedule a set of users to the same transmitter. Focusing on this, we consider the throughput maximization problem that optimizes jointly the network-coded user scheduling and power allocation, subject to fronthaul capacity, transmit power, and NC constraints. Given the intractability of the problem, we decouple it into two separate subproblems. In the first subproblem, we consider the network-coded user scheduling problem for the given power allocation, while in the second subproblem, we use the NC resulting user schedule to optimize the power levels. We design an innovative \textit{two-layered rate-aware NC (RA-IDNC)} graph to solve the first subproblem and evaluate the second subproblem using an iterative function evaluation (IFE) approach. Simulation results are presented to depict the throughput gain of the proposed approach over the existing solutions.
△ Less
Submitted 5 February, 2021;
originally announced February 2021.
-
Cross-Layer Network Codes for Content Delivery in Cache-Enabled D2D Networks
Authors:
Mohammed S. Al-Abiad,
Md. Zoheb Hassan,
Md. Jahangir Hossain
Abstract:
In this paper, we consider the use of cross-layer network coding (CLNC), caching, and device-to-device (D2D) communications to jointly optimize the delivery of a set of popular contents to a set of user devices (UDs). In the considered D2D network, a group of near-by UDs cooperate with each other and use NC to combine their cached files, so as the completion time required for delivering all reques…
▽ More
In this paper, we consider the use of cross-layer network coding (CLNC), caching, and device-to-device (D2D) communications to jointly optimize the delivery of a set of popular contents to a set of user devices (UDs). In the considered D2D network, a group of near-by UDs cooperate with each other and use NC to combine their cached files, so as the completion time required for delivering all requested contents to all UDs is minimized. Unlike the previous work that considers only one transmitting UD at a time, our work allows multiple UDs to transmit simultaneously given the interference among the active links is small. Such configuration brings a new trade-off among scheduling UDs to transmitting UDs, selecting the coding decisions and the transmission rate/power. Therefore, we consider the completion time minimization problem that involves scheduling multiple transmitting UDs, determining their transmission rates/powers and file combinations. The problem is shown to be intractable because it involves all future coding decisions. To tackle the problem at each transmission slot, we first design a graph called herein the D2D Rate-Aware IDNC graph where its vertices have weights that judiciously balance between the rates/powers of the transmitting UDs and the number of their scheduled UDs. Then, we propose an innovative and efficient CLNC solution that iteratively selects a set of transmitting UDs only if the interference caused by the transmissions of the newly selected UDs does not significantly impact the overall completion time. Simulation results show that the proposed solution offers significant completion time reduction compared with the existing algorithms.
△ Less
Submitted 18 January, 2021;
originally announced January 2021.
-
Device-Clustering and Rate-Splitting Enabled Device-to-Device Cooperation Framework in Fog Radio Access Network
Authors:
Md. Zoheb Hassan,
Md. Jahangir Hossain,
Julian Cheng,
Victor C. M. Leung
Abstract:
Resource allocation is investigated to enhance the performance of device-to-device (D2D) cooperation in a fog radio access network (F-RAN) architecture. Our envisioned framework enables two D2D links to share certain orthogonal radio resource blocks (RRBs) by forming device-clusters. In each device-cluster, both content-holder device-users (DUs) transmit to the content-requester DUs via an enhance…
▽ More
Resource allocation is investigated to enhance the performance of device-to-device (D2D) cooperation in a fog radio access network (F-RAN) architecture. Our envisioned framework enables two D2D links to share certain orthogonal radio resource blocks (RRBs) by forming device-clusters. In each device-cluster, both content-holder device-users (DUs) transmit to the content-requester DUs via an enhanced remote radio head (eRRH) over the same RRBs. Such RRBs are shared with the uplink F-RAN as well. The intra device-cluster interference is mitigated by exploiting both uplink and downlink rate-splitting schemes, and the inter device-cluster interference is mitigated by using an orthogonal RRB allocation strategy. Our objective is to maximize the end-to-end sum-rate of the device-clusters while reducing the interference between D2D cooperation and the uplink F-RAN over the shared RRBs. Towards this objective, a joint optimization of device-clustering, transmit power allocations, assignment of device-clusters to the eRRHs, and allocation of RRBs among the device-clusters is presented. Since the joint optimization is NP-hard and intractable, it is decomposed into device-clustering and resource allocation sub-problems, and efficient solutions to both sub-problems are developed. Based on the solutions to the sub-problems, a semi-distributed and convergent algorithm, entitled rate-splitting for multi-hop D2D (RSMD), is proposed to obtain the device-clusters and resource allocation for these device-clusters. Through extensive simulations, efficiency of the proposed RSMD algorithm over several benchmark schemes is demonstrated
△ Less
Submitted 14 April, 2021; v1 submitted 7 July, 2020;
originally announced July 2020.
-
An Edge Computing Empowered Radio Access Network With UAV-Mounted FSO Fronthaul and Backhaul: Key Challenges and Approaches
Authors:
Yanjie Dong,
Md. Zoheb Hassan,
Julian Cheng,
Md. Jahangir Hossain,
Victor C. M. Leung
Abstract:
One promising approach to address the supply-demand mismatch between the terrestrial infrastructure and the temporary and/or unexpected traffic demands is to leverage the unmanned aerial vehicle (UAV) technologies. Motivated by the recent advancement of UAV technologies and retromodulator based free space optical communication, we propose a novel edge-computing empowered radio access network archi…
▽ More
One promising approach to address the supply-demand mismatch between the terrestrial infrastructure and the temporary and/or unexpected traffic demands is to leverage the unmanned aerial vehicle (UAV) technologies. Motivated by the recent advancement of UAV technologies and retromodulator based free space optical communication, we propose a novel edge-computing empowered radio access network architecture where the fronthaul and backhaul links are mounted on the UAVs for rapid event response and flexible deployment. The implementation of hardware and networking technologies for the proposed architecture are investigated. Due to the limited payload and endurance as well as the high mobility of UAVs, research challenges related to the communication resource management and recent research progress are reported.
△ Less
Submitted 16 March, 2018;
originally announced March 2018.
-
Dynamic scaling, data-collapse and self-similarity in Barabási-Albert networks
Authors:
M. Kamrul Hassan,
M. Zahedul Hassan,
Neeaj I. Pavel
Abstract:
In this article, we show that if each node of the Barabási-Albert (BA) network is characterized by the generalized degree $q$, i.e. the product of their degree $k$ and the square root of their respective birth time, then the distribution function $F(q,t)$ exhibits dynamic scaling $F(q,t\rightarrow \infty)\sim t^{-1/2}φ(q/t^{1/2})$ where $φ(x)$ is the scaling function. We verified it by showing tha…
▽ More
In this article, we show that if each node of the Barabási-Albert (BA) network is characterized by the generalized degree $q$, i.e. the product of their degree $k$ and the square root of their respective birth time, then the distribution function $F(q,t)$ exhibits dynamic scaling $F(q,t\rightarrow \infty)\sim t^{-1/2}φ(q/t^{1/2})$ where $φ(x)$ is the scaling function. We verified it by showing that a series of distinct $F(q,t)$ vs $q$ curves for different network sizes $N$ collapse onto a single universal curve if we plot $t^{1/2}F(q,t)$ vs $q/t^{1/2}$ instead. Finally, we show that the BA network falls into two universality classes depending on whether new nodes arrive with single edge ($m=1$) or with multiple edges ($m>1$).
△ Less
Submitted 7 April, 2011; v1 submitted 25 January, 2011;
originally announced January 2011.
