-
Land Classification in Satellite Images by Injecting Traditional Features to CNN Models
Authors:
Mehmet Cagri Aksoy,
Beril Sirmacek,
Cem Unsalan
Abstract:
Deep learning methods have been successfully applied to remote sensing problems for several years. Among these methods, CNN based models have high accuracy in solving the land classification problem using satellite or aerial images. Although these models have high accuracy, this generally comes with large memory size requirements. On the other hand, it is desirable to have small-sized models for a…
▽ More
Deep learning methods have been successfully applied to remote sensing problems for several years. Among these methods, CNN based models have high accuracy in solving the land classification problem using satellite or aerial images. Although these models have high accuracy, this generally comes with large memory size requirements. On the other hand, it is desirable to have small-sized models for applications, such as the ones implemented on unmanned aerial vehicles, with low memory space. Unfortunately, small-sized CNN models do not provide high accuracy as with their large-sized versions. In this study, we propose a novel method to improve the accuracy of CNN models, especially the ones with small size, by injecting traditional features to them. To test the effectiveness of the proposed method, we applied it to the CNN models SqueezeNet, MobileNetV2, ShuffleNetV2, VGG16, and ResNet50V2 having size 0.5 MB to 528 MB. We used the sample mean, gray level co-occurrence matrix features, Hu moments, local binary patterns, histogram of oriented gradients, and color invariants as traditional features for injection. We tested the proposed method on the EuroSAT dataset to perform land classification. Our experimental results show that the proposed method significantly improves the land classification accuracy especially when applied to small-sized CNN models.
△ Less
Submitted 21 July, 2022;
originally announced July 2022.
-
The potential of artificial intelligence for achieving healthy and sustainable societies
Authors:
B. Sirmacek,
S. Gupta,
F. Mallor,
H. Azizpour,
Y. Ban,
H. Eivazi,
H. Fang,
F. Golzar,
I. Leite,
G. I. Melsion,
K. Smith,
F. Fuso Nerini,
R. Vinuesa
Abstract:
In this chapter we extend earlier work (Vinuesa et al., Nature Communications 11, 2020) on the potential of artificial intelligence (AI) to achieve the 17 Sustainable Development Goals (SDGs) proposed by the United Nations (UN) for the 2030 Agenda. The present contribution focuses on three SDGs related to healthy and sustainable societies, i.e. SDG 3 (on good health), SDG 11 (on sustainable cities…
▽ More
In this chapter we extend earlier work (Vinuesa et al., Nature Communications 11, 2020) on the potential of artificial intelligence (AI) to achieve the 17 Sustainable Development Goals (SDGs) proposed by the United Nations (UN) for the 2030 Agenda. The present contribution focuses on three SDGs related to healthy and sustainable societies, i.e. SDG 3 (on good health), SDG 11 (on sustainable cities) and SDG 13 (on climate action). This chapter extends the previous study within those three goals, and goes beyond the 2030 targets. These SDGs are selected because they are closely related to the coronavirus disease 19 (COVID-19) pandemic, and also to crises like climate change, which constitute important challenges to our society.
△ Less
Submitted 11 February, 2022;
originally announced February 2022.
-
Aim in Climate Change and City Pollution
Authors:
Pablo Torres,
Beril Sirmacek,
Sergio Hoyas,
Ricardo Vinuesa
Abstract:
The sustainability of urban environments is an increasingly relevant problem. Air pollution plays a key role in the degradation of the environment as well as the health of the citizens exposed to it. In this chapter we provide a review of the methods available to model air pollution, focusing on the application of machine-learning methods. In fact, machine-learning methods have proved to important…
▽ More
The sustainability of urban environments is an increasingly relevant problem. Air pollution plays a key role in the degradation of the environment as well as the health of the citizens exposed to it. In this chapter we provide a review of the methods available to model air pollution, focusing on the application of machine-learning methods. In fact, machine-learning methods have proved to importantly increase the accuracy of traditional air-pollution approaches while limiting the development cost of the models. Machine-learning tools have opened new approaches to study air pollution, such as flow-dynamics modelling or remote-sensing methodologies.
△ Less
Submitted 30 December, 2021;
originally announced December 2021.
-
Interpretable deep-learning models to help achieve the Sustainable Development Goals
Authors:
Ricardo Vinuesa,
Beril Sirmacek
Abstract:
We discuss our insights into interpretable artificial-intelligence (AI) models, and how they are essential in the context of developing ethical AI systems, as well as data-driven solutions compliant with the Sustainable Development Goals (SDGs). We highlight the potential of extracting truly-interpretable models from deep-learning methods, for instance via symbolic models obtained through inductiv…
▽ More
We discuss our insights into interpretable artificial-intelligence (AI) models, and how they are essential in the context of developing ethical AI systems, as well as data-driven solutions compliant with the Sustainable Development Goals (SDGs). We highlight the potential of extracting truly-interpretable models from deep-learning methods, for instance via symbolic models obtained through inductive biases, to ensure a sustainable development of AI.
△ Less
Submitted 24 August, 2021;
originally announced August 2021.
-
Recurrent U-net for automatic pelvic floor muscle segmentation on 3D ultrasound
Authors:
Frieda van den Noort,
Beril Sirmacek,
Cornelis H. Slump
Abstract:
The prevalance of pelvic floor problems is high within the female population. Transperineal ultrasound (TPUS) is the main imaging modality used to investigate these problems. Automating the analysis of TPUS data will help in growing our understanding of pelvic floor related problems. In this study we present a U-net like neural network with some convolutional long short term memory (CLSTM) layers…
▽ More
The prevalance of pelvic floor problems is high within the female population. Transperineal ultrasound (TPUS) is the main imaging modality used to investigate these problems. Automating the analysis of TPUS data will help in growing our understanding of pelvic floor related problems. In this study we present a U-net like neural network with some convolutional long short term memory (CLSTM) layers to automate the 3D segmentation of the levator ani muscle (LAM) in TPUS volumes. The CLSTM layers are added to preserve the inter-slice 3D information. We reach human level performance on this segmentation task. Therefore, we conclude that we successfully automated the segmentation of the LAM on 3D TPUS data. This paves the way towards automatic in-vivo analysis of the LAM mechanics in the context of large study populations.
△ Less
Submitted 29 July, 2021;
originally announced July 2021.
-
Towards Autonomous Pipeline Inspection with Hierarchical Reinforcement Learning
Authors:
Nicolò Botteghi,
Luuk Grefte,
Mannes Poel,
Beril Sirmacek,
Christoph Brune,
Edwin Dertien,
Stefano Stramigioli
Abstract:
Inspection and maintenance are two crucial aspects of industrial pipeline plants. While robotics has made tremendous progress in the mechanic design of in-pipe inspection robots, the autonomous control of such robots is still a big open challenge due to the high number of actuators and the complex manoeuvres required. To address this problem, we investigate the usage of Deep Reinforcement Learning…
▽ More
Inspection and maintenance are two crucial aspects of industrial pipeline plants. While robotics has made tremendous progress in the mechanic design of in-pipe inspection robots, the autonomous control of such robots is still a big open challenge due to the high number of actuators and the complex manoeuvres required. To address this problem, we investigate the usage of Deep Reinforcement Learning for achieving autonomous navigation of in-pipe robots in pipeline networks with complex topologies. Moreover, we introduce a hierarchical policy decomposition based on Hierarchical Reinforcement Learning to learn robust high-level navigation skills. We show that the hierarchical structure introduced in the policy is fundamental for solving the navigation task through pipes and necessary for achieving navigation performances superior to human-level control.
△ Less
Submitted 8 July, 2021;
originally announced July 2021.
-
Remote sensing and AI for building climate adaptation applications
Authors:
Beril Sirmacek,
Ricardo Vinuesa
Abstract:
Urban areas are not only one of the biggest contributors to climate change, but also they are one of the most vulnerable areas with high populations who would together experience the negative impacts. In this paper, we address some of the opportunities brought by satellite remote sensing imaging and artificial intelligence (AI) in order to measure climate adaptation of cities automatically. We pro…
▽ More
Urban areas are not only one of the biggest contributors to climate change, but also they are one of the most vulnerable areas with high populations who would together experience the negative impacts. In this paper, we address some of the opportunities brought by satellite remote sensing imaging and artificial intelligence (AI) in order to measure climate adaptation of cities automatically. We propose a framework combining AI and simulation which may be useful for extracting indicators from remote-sensing images and may help with predictive estimation of future states of these climate-adaptation-related indicators. When such models become more robust and used in real life applications, they may help decision makers and early responders to choose the best actions to sustain the well-being of society, natural resources and biodiversity. We underline that this is an open field and an on-going area of research for many scientists, therefore we offer an in-depth discussion on the challenges and limitations of data-driven methods and the predictive estimation models in general.
△ Less
Submitted 12 July, 2022; v1 submitted 6 July, 2021;
originally announced July 2021.
-
Low-Dimensional State and Action Representation Learning with MDP Homomorphism Metrics
Authors:
Nicolò Botteghi,
Mannes Poel,
Beril Sirmacek,
Christoph Brune
Abstract:
Deep Reinforcement Learning has shown its ability in solving complicated problems directly from high-dimensional observations. However, in end-to-end settings, Reinforcement Learning algorithms are not sample-efficient and requires long training times and quantities of data. In this work, we proposed a framework for sample-efficient Reinforcement Learning that take advantage of state and action re…
▽ More
Deep Reinforcement Learning has shown its ability in solving complicated problems directly from high-dimensional observations. However, in end-to-end settings, Reinforcement Learning algorithms are not sample-efficient and requires long training times and quantities of data. In this work, we proposed a framework for sample-efficient Reinforcement Learning that take advantage of state and action representations to transform a high-dimensional problem into a low-dimensional one. Moreover, we seek to find the optimal policy mapping latent states to latent actions. Because now the policy is learned on abstract representations, we enforce, using auxiliary loss functions, the lifting of such policy to the original problem domain. Results show that the novel framework can efficiently learn low-dimensional and interpretable state and action representations and the optimal latent policy.
△ Less
Submitted 4 July, 2021;
originally announced July 2021.
-
Low Dimensional State Representation Learning with Robotics Priors in Continuous Action Spaces
Authors:
Nicolò Botteghi,
Khaled Alaa,
Mannes Poel,
Beril Sirmacek,
Christoph Brune,
Abeje Mersha,
Stefano Stramigioli
Abstract:
Autonomous robots require high degrees of cognitive and motoric intelligence to come into our everyday life. In non-structured environments and in the presence of uncertainties, such degrees of intelligence are not easy to obtain. Reinforcement learning algorithms have proven to be capable of solving complicated robotics tasks in an end-to-end fashion without any need for hand-crafted features or…
▽ More
Autonomous robots require high degrees of cognitive and motoric intelligence to come into our everyday life. In non-structured environments and in the presence of uncertainties, such degrees of intelligence are not easy to obtain. Reinforcement learning algorithms have proven to be capable of solving complicated robotics tasks in an end-to-end fashion without any need for hand-crafted features or policies. Especially in the context of robotics, in which the cost of real-world data is usually extremely high, reinforcement learning solutions achieving high sample efficiency are needed. In this paper, we propose a framework combining the learning of a low-dimensional state representation, from high-dimensional observations coming from the robot's raw sensory readings, with the learning of the optimal policy, given the learned state representation. We evaluate our framework in the context of mobile robot navigation in the case of continuous state and action spaces. Moreover, we study the problem of transferring what learned in the simulated virtual environment to the real robot without further retraining using real-world data in the presence of visual and depth distractors, such as lighting changes and moving obstacles.
△ Less
Submitted 4 July, 2021;
originally announced July 2021.
-
Low Dimensional State Representation Learning with Reward-shaped Priors
Authors:
Nicolò Botteghi,
Ruben Obbink,
Daan Geijs,
Mannes Poel,
Beril Sirmacek,
Christoph Brune,
Abeje Mersha,
Stefano Stramigioli
Abstract:
Reinforcement Learning has been able to solve many complicated robotics tasks without any need for feature engineering in an end-to-end fashion. However, learning the optimal policy directly from the sensory inputs, i.e the observations, often requires processing and storage of a huge amount of data. In the context of robotics, the cost of data from real robotics hardware is usually very high, thu…
▽ More
Reinforcement Learning has been able to solve many complicated robotics tasks without any need for feature engineering in an end-to-end fashion. However, learning the optimal policy directly from the sensory inputs, i.e the observations, often requires processing and storage of a huge amount of data. In the context of robotics, the cost of data from real robotics hardware is usually very high, thus solutions that achieve high sample-efficiency are needed. We propose a method that aims at learning a mapping from the observations into a lower-dimensional state space. This mapping is learned with unsupervised learning using loss functions shaped to incorporate prior knowledge of the environment and the task. Using the samples from the state space, the optimal policy is quickly and efficiently learned. We test the method on several mobile robot navigation tasks in a simulation environment and also on a real robot.
△ Less
Submitted 7 January, 2021; v1 submitted 29 July, 2020;
originally announced July 2020.
-
On Reward Shaping for Mobile Robot Navigation: A Reinforcement Learning and SLAM Based Approach
Authors:
Nicolò Botteghi,
Beril Sirmacek,
Khaled A. A. Mustafa,
Mannes Poel,
Stefano Stramigioli
Abstract:
We present a map-less path planning algorithm based on Deep Reinforcement Learning (DRL) for mobile robots navigating in unknown environment that only relies on 40-dimensional raw laser data and odometry information. The planner is trained using a reward function shaped based on the online knowledge of the map of the training environment, obtained using grid-based Rao-Blackwellized particle filter…
▽ More
We present a map-less path planning algorithm based on Deep Reinforcement Learning (DRL) for mobile robots navigating in unknown environment that only relies on 40-dimensional raw laser data and odometry information. The planner is trained using a reward function shaped based on the online knowledge of the map of the training environment, obtained using grid-based Rao-Blackwellized particle filter, in an attempt to enhance the obstacle awareness of the agent. The agent is trained in a complex simulated environment and evaluated in two unseen ones. We show that the policy trained using the introduced reward function not only outperforms standard reward functions in terms of convergence speed, by a reduction of 36.9\% of the iteration steps, and reduction of the collision samples, but it also drastically improves the behaviour of the agent in unseen environments, respectively by 23\% in a simpler workspace and by 45\% in a more clustered one. Furthermore, the policy trained in the simulation environment can be directly and successfully transferred to the real robot. A video of our experiments can be found at: https://youtu.be/UEV7W6e6ZqI
△ Less
Submitted 10 February, 2020;
originally announced February 2020.
-
A low-cost real-time 3D imaging system for contactless asthma observation
Authors:
Sheona M. M. D. P. Sequeira,
Beril Sirmacek
Abstract:
Asthma is becoming a very serious problem with every passing day, especially in children. However, it is very difficult to detect this disorder in them, since the breathing motion of children tends to change when they reach an age of 6. This, thus makes it very difficult to monitor their respiratory state easily. In this paper, we present a cheap non-contact alternative to the current methods that…
▽ More
Asthma is becoming a very serious problem with every passing day, especially in children. However, it is very difficult to detect this disorder in them, since the breathing motion of children tends to change when they reach an age of 6. This, thus makes it very difficult to monitor their respiratory state easily. In this paper, we present a cheap non-contact alternative to the current methods that are available. This is using a stereo camera, that captures a video of the patient breathing at a frame rate of 30Hz. For further processing, the captured video has to be rectified and converted into a point cloud. The obtained point clouds need to be aligned in order to have the output with respect to a common plane. They are then converted into a surface mesh. The depth is further estimated by subtracting every point cloud from the reference point cloud (the first frame). The output data, however, when plotted with respect to real time produces a very noisy plot. This is filtered by determining the signal frequency by taking the Fast Fourier Transform of the breathing signal. The system was tested under 4 different breathing conditions: deep, shallow and normal breathing and while coughing. On its success, it was tested with mixed breathing (combination of normal and shallow breathing) and was lastly compared with the output of the expensive 3dMD system. The comparison showed that using the stereo camera, we can reach to similar sensitivity for respiratory motion observation. The experimental results show that, the proposed method provides a major step towards development of low-cost home-based observation systems for asthma patients and care-givers.
△ Less
Submitted 3 November, 2019;
originally announced November 2019.
-
3D tissue reconstruction with Kinect to evaluate neck lymphedema
Authors:
Gerrit Brugman,
Beril Sirmacek
Abstract:
Lymphedema is a condition of localized tissue swelling caused by a damaged lymphatic system. Therapy to these tissues is applied manually. Some of the methods are lymph drainage, compression therapy or bandaging. However, the therapy methods are still insufficiently evaluated. Especially, because of not having a reliable method to measure the change of such a soft and flexible tissue. In this rese…
▽ More
Lymphedema is a condition of localized tissue swelling caused by a damaged lymphatic system. Therapy to these tissues is applied manually. Some of the methods are lymph drainage, compression therapy or bandaging. However, the therapy methods are still insufficiently evaluated. Especially, because of not having a reliable method to measure the change of such a soft and flexible tissue. In this research, our goal has been providing a 3d computer vision based method to measure the changes of the neck tissues. To do so, we used Kinect as a depth sensor and built our algorithms for the point cloud data acquired from this sensor. The resulting 3D models of the patient necks are used for comparing the models in time and measuring the volumetric changes accurately. Our discussions with the medical doctors validate that, when used in practice this approach would be able to give better indication on which therapy method is helping and how the tissue is changing in time.
△ Less
Submitted 2 November, 2019;
originally announced November 2019.
-
Sequential image processing methods for improving semantic video segmentation algorithms
Authors:
Beril Sirmacek,
Nicolò Botteghi,
Santiago Sanchez Escalonilla Plaza
Abstract:
Recently, semantic video segmentation gained high attention especially for supporting autonomous driving systems. Deep learning methods made it possible to implement real time segmentation and object identification algorithms on videos. However, most of the available approaches process each video frame independently disregarding their sequential relation in time. Therefore their results suddenly m…
▽ More
Recently, semantic video segmentation gained high attention especially for supporting autonomous driving systems. Deep learning methods made it possible to implement real time segmentation and object identification algorithms on videos. However, most of the available approaches process each video frame independently disregarding their sequential relation in time. Therefore their results suddenly miss some of the object segments in some of the frames even if they were detected properly in the earlier frames. Herein we propose two sequential probabilistic video frame analysis approaches to improve the segmentation performance of the existing algorithms. Our experiments show that using the information of the past frames we increase the performance and consistency of the state of the art algorithms.
△ Less
Submitted 29 October, 2019;
originally announced October 2019.
-
Semantic Segmentation of Skin Lesions using a Small Data Set
Authors:
Beril Sirmacek,
Max Kivits
Abstract:
Early detection of melanoma is difficult for the human eye but a crucial step towards reducing its death rate. Computerized detection of these melanoma and other skin lesions is necessary. The central research question in this paper is "How to segment skin lesion images using a neural network with low available data?". This question is divided into three sub questions regarding best performing net…
▽ More
Early detection of melanoma is difficult for the human eye but a crucial step towards reducing its death rate. Computerized detection of these melanoma and other skin lesions is necessary. The central research question in this paper is "How to segment skin lesion images using a neural network with low available data?". This question is divided into three sub questions regarding best performing network structure, training data and training method. First theory associated with these questions is discussed. Literature states that U-net CNN structures have excellent performances on the segmentation task, more training data increases network performance and utilizing transfer learning enables networks to generalize to new data better.
To validate these findings in the literature two experiments are conducted. The first experiment trains a network on data sets of different size. The second experiment proposes twelve network structures and trains them on the same data set. The experimental results support the findings in the literature. The FCN16 and FCN32 networks perform best in the accuracy, intersection over union and mean BF1 Score metric. Concluding from these results the skin lesion segmentation network is a fully convolutional structure with a skip architecture and an encoder depth of either one or two. Weights of this network should be initialized using transfer learning from the pre trained VGG16 network. Training data should be cropped to reduce complexity and augmented during training to reduce the likelihood of overfitting.
△ Less
Submitted 23 October, 2019;
originally announced October 2019.
