-
SLOctolyzer: Fully automatic analysis toolkit for segmentation and feature extracting in scanning laser ophthalmoscopy images
Authors:
Jamie Burke,
Samuel Gibbon,
Justin Engelmann,
Adam Threlfall,
Ylenia Giarratano,
Charlene Hamid,
Stuart King,
Ian J. C. MacCormick,
Tom MacGillivray
Abstract:
Purpose: To describe SLOctolyzer: an open-source analysis toolkit for en face retinal vessels appearing in infrared reflectance scanning laser ophthalmoscopy (SLO) images.
Methods: SLOctolyzer includes two main modules: segmentation and measurement. The segmentation module use deep learning methods to delineate retinal anatomy, while the measurement module quantifies key retinal vascular feature…
▽ More
Purpose: To describe SLOctolyzer: an open-source analysis toolkit for en face retinal vessels appearing in infrared reflectance scanning laser ophthalmoscopy (SLO) images.
Methods: SLOctolyzer includes two main modules: segmentation and measurement. The segmentation module use deep learning methods to delineate retinal anatomy, while the measurement module quantifies key retinal vascular features such as vessel complexity, density, tortuosity, and calibre. We evaluate the segmentation module using unseen data and measure its reproducibility.
Results: SLOctolyzer's segmentation module performed well against unseen internal test data (Dice for all-vessels, 0.9097; arteries, 0.8376; veins, 0.8525; optic disc, 0.9430; fovea, 0.8837). External validation against severe retinal pathology showed decreased performance (Dice for arteries, 0.7180; veins, 0.7470; optic disc, 0.9032). SLOctolyzer had good reproducibility (mean difference for fractal dimension, -0.0007; vessel density, -0.0003; vessel calibre, -0.3154 $μ$m; tortuosity density, 0.0013). SLOctolyzer can process a macula-centred SLO image in under 20 seconds and a disc-centred SLO image in under 30 seconds using a standard laptop CPU.
Conclusions: To our knowledge, SLOctolyzer is the first open-source tool to convert raw SLO images into reproducible and clinically meaningful retinal vascular parameters. SLO images are captured simultaneous to optical coherence tomography (OCT), and we believe our software will be useful for extracting retinal vascular measurements from large OCT image sets and linking them to ocular or systemic diseases. It requires no specialist knowledge or proprietary software, and allows manual correction of segmentations and re-computing of vascular metrics. SLOctolyzer is freely available at https://github.com/jaburke166/SLOctolyzer.
△ Less
Submitted 24 June, 2024;
originally announced June 2024.
-
Domain-specific augmentations with resolution agnostic self-attention mechanism improves choroid segmentation in optical coherence tomography images
Authors:
Jamie Burke,
Justin Engelmann,
Charlene Hamid,
Diana Moukaddem,
Dan Pugh,
Neeraj Dhaun,
Amos Storkey,
Niall Strang,
Stuart King,
Tom MacGillivray,
Miguel O. Bernabeu,
Ian J. C. MacCormick
Abstract:
The choroid is a key vascular layer of the eye, supplying oxygen to the retinal photoreceptors. Non-invasive enhanced depth imaging optical coherence tomography (EDI-OCT) has recently improved access and visualisation of the choroid, making it an exciting frontier for discovering novel vascular biomarkers in ophthalmology and wider systemic health. However, current methods to measure the choroid o…
▽ More
The choroid is a key vascular layer of the eye, supplying oxygen to the retinal photoreceptors. Non-invasive enhanced depth imaging optical coherence tomography (EDI-OCT) has recently improved access and visualisation of the choroid, making it an exciting frontier for discovering novel vascular biomarkers in ophthalmology and wider systemic health. However, current methods to measure the choroid often require use of multiple, independent semi-automatic and deep learning-based algorithms which are not made open-source. Previously, Choroidalyzer -- an open-source, fully automatic deep learning method trained on 5,600 OCT B-scans from 385 eyes -- was developed to fully segment and quantify the choroid in EDI-OCT images, thus addressing these issues. Using the same dataset, we propose a Robust, Resolution-agnostic and Efficient Attention-based network for CHoroid segmentation (REACH). REACHNet leverages multi-resolution training with domain-specific data augmentation to promote generalisation, and uses a lightweight architecture with resolution-agnostic self-attention which is not only faster than Choroidalyzer's previous network (4 images/s vs. 2.75 images/s on a standard laptop CPU), but has greater performance for segmenting the choroid region, vessels and fovea (Dice coefficient for region 0.9769 vs. 0.9749, vessels 0.8612 vs. 0.8192 and fovea 0.8243 vs. 0.3783) due to its improved hyperparameter configuration and model training pipeline. REACHNet can be used with Choroidalyzer as a drop-in replacement for the original model and will be made available upon publication.
△ Less
Submitted 23 May, 2024;
originally announced May 2024.
-
Choroidalyzer: An open-source, end-to-end pipeline for choroidal analysis in optical coherence tomography
Authors:
Justin Engelmann,
Jamie Burke,
Charlene Hamid,
Megan Reid-Schachter,
Dan Pugh,
Neeraj Dhaun,
Diana Moukaddem,
Lyle Gray,
Niall Strang,
Paul McGraw,
Amos Storkey,
Paul J. Steptoe,
Stuart King,
Tom MacGillivray,
Miguel O. Bernabeu,
Ian J. C. MacCormick
Abstract:
Purpose: To develop Choroidalyzer, an open-source, end-to-end pipeline for segmenting the choroid region, vessels, and fovea, and deriving choroidal thickness, area, and vascular index.
Methods: We used 5,600 OCT B-scans (233 subjects, 6 systemic disease cohorts, 3 device types, 2 manufacturers). To generate region and vessel ground-truths, we used state-of-the-art automatic methods following ma…
▽ More
Purpose: To develop Choroidalyzer, an open-source, end-to-end pipeline for segmenting the choroid region, vessels, and fovea, and deriving choroidal thickness, area, and vascular index.
Methods: We used 5,600 OCT B-scans (233 subjects, 6 systemic disease cohorts, 3 device types, 2 manufacturers). To generate region and vessel ground-truths, we used state-of-the-art automatic methods following manual correction of inaccurate segmentations, with foveal positions manually annotated. We trained a U-Net deep-learning model to detect the region, vessels, and fovea to calculate choroid thickness, area, and vascular index in a fovea-centred region of interest. We analysed segmentation agreement (AUC, Dice) and choroid metrics agreement (Pearson, Spearman, mean absolute error (MAE)) in internal and external test sets. We compared Choroidalyzer to two manual graders on a small subset of external test images and examined cases of high error.
Results: Choroidalyzer took 0.299 seconds per image on a standard laptop and achieved excellent region (Dice: internal 0.9789, external 0.9749), very good vessel segmentation performance (Dice: internal 0.8817, external 0.8703) and excellent fovea location prediction (MAE: internal 3.9 pixels, external 3.4 pixels). For thickness, area, and vascular index, Pearson correlations were 0.9754, 0.9815, and 0.8285 (internal) / 0.9831, 0.9779, 0.7948 (external), respectively (all p<0.0001). Choroidalyzer's agreement with graders was comparable to the inter-grader agreement across all metrics.
Conclusions: Choroidalyzer is an open-source, end-to-end pipeline that accurately segments the choroid and reliably extracts thickness, area, and vascular index. Especially choroidal vessel segmentation is a difficult and subjective task, and fully-automatic methods like Choroidalyzer could provide objectivity and standardisation.
△ Less
Submitted 5 December, 2023;
originally announced December 2023.
-
Statement: The Metaverse as an Information-Centric Network
Authors:
Dirk Kutscher,
Jeff Burke,
Giuseppe Fioccola,
Paulo Mendes
Abstract:
This paper discusses challenges and opportunities of considering the Metaverse as an Information-Centric Network (ICN). The Web today essentially represents a data-centric application layer: data named by URLs is manipulated with REST primitives. However, the semantic gap with the underlying host-oriented transport is significant, typically leading to complexity, centralization, and brittleness. P…
▽ More
This paper discusses challenges and opportunities of considering the Metaverse as an Information-Centric Network (ICN). The Web today essentially represents a data-centric application layer: data named by URLs is manipulated with REST primitives. However, the semantic gap with the underlying host-oriented transport is significant, typically leading to complexity, centralization, and brittleness. Popular interest in "the Metaverse" suggests that the end-user experience of the Web will evolve towards always-on eXtended Reality (XR). With the benefit of a historical perspective, computing advances, and decades of experience with a global network, there is an opportunity to holistically consider the Metaverse not as an application of the current network, but an evolution of the network itself, reducing rather than widening the gap between network architecture and application semantics. An ICN architecture offers the possibility to achieve this with less overhead, low latency, better security, and more disruption tolerance suitable to diverse uses cases, even those facing intermittent connectivity.
△ Less
Submitted 16 September, 2023;
originally announced September 2023.
-
An open-source deep learning algorithm for efficient and fully-automatic analysis of the choroid in optical coherence tomography
Authors:
Jamie Burke,
Justin Engelmann,
Charlene Hamid,
Megan Reid-Schachter,
Tom Pearson,
Dan Pugh,
Neeraj Dhaun,
Stuart King,
Tom MacGillivray,
Miguel O. Bernabeu,
Amos Storkey,
Ian J. C. MacCormick
Abstract:
Purpose: To develop an open-source, fully-automatic deep learning algorithm, DeepGPET, for choroid region segmentation in optical coherence tomography (OCT) data. Methods: We used a dataset of 715 OCT B-scans (82 subjects, 115 eyes) from 3 clinical studies related to systemic disease. Ground truth segmentations were generated using a clinically validated, semi-automatic choroid segmentation method…
▽ More
Purpose: To develop an open-source, fully-automatic deep learning algorithm, DeepGPET, for choroid region segmentation in optical coherence tomography (OCT) data. Methods: We used a dataset of 715 OCT B-scans (82 subjects, 115 eyes) from 3 clinical studies related to systemic disease. Ground truth segmentations were generated using a clinically validated, semi-automatic choroid segmentation method, Gaussian Process Edge Tracing (GPET). We finetuned a UNet with MobileNetV3 backbone pre-trained on ImageNet. Standard segmentation agreement metrics, as well as derived measures of choroidal thickness and area, were used to evaluate DeepGPET, alongside qualitative evaluation from a clinical ophthalmologist. Results: DeepGPET achieves excellent agreement with GPET on data from 3 clinical studies (AUC=0.9994, Dice=0.9664; Pearson correlation of 0.8908 for choroidal thickness and 0.9082 for choroidal area), while reducing the mean processing time per image on a standard laptop CPU from 34.49s ($\pm$15.09) using GPET to 1.25s ($\pm$0.10) using DeepGPET. Both methods performed similarly according to a clinical ophthalmologist, who qualitatively judged a subset of segmentations by GPET and DeepGPET, based on smoothness and accuracy of segmentations. Conclusions: DeepGPET, a fully-automatic, open-source algorithm for choroidal segmentation, will enable researchers to efficiently extract choroidal measurements, even for large datasets. As no manual interventions are required, DeepGPET is less subjective than semi-automatic methods and could be deployed in clinical practice without necessitating a trained operator.
△ Less
Submitted 29 October, 2023; v1 submitted 3 July, 2023;
originally announced July 2023.
-
Next Steps for Human-Centered Generative AI: A Technical Perspective
Authors:
Xiang 'Anthony' Chen,
Jeff Burke,
Ruofei Du,
Matthew K. Hong,
Jennifer Jacobs,
Philippe Laban,
Dingzeyu Li,
Nanyun Peng,
Karl D. D. Willis,
Chien-Sheng Wu,
Bolei Zhou
Abstract:
Through iterative, cross-disciplinary discussions, we define and propose next-steps for Human-centered Generative AI (HGAI). We contribute a comprehensive research agenda that lays out future directions of Generative AI spanning three levels: aligning with human values; assimilating human intents; and augmenting human abilities. By identifying these next-steps, we intend to draw interdisciplinary…
▽ More
Through iterative, cross-disciplinary discussions, we define and propose next-steps for Human-centered Generative AI (HGAI). We contribute a comprehensive research agenda that lays out future directions of Generative AI spanning three levels: aligning with human values; assimilating human intents; and augmenting human abilities. By identifying these next-steps, we intend to draw interdisciplinary research teams to pursue a coherent set of emergent ideas in HGAI, focusing on their interested topics while maintaining a coherent big picture of the future work landscape.
△ Less
Submitted 22 December, 2023; v1 submitted 27 June, 2023;
originally announced June 2023.
-
Deflectometry for specular surfaces: an overview
Authors:
Jan Burke,
Alexey Pak,
Sebastian Höfer,
Mathias Ziebarth,
Masoud Roschani,
Jürgen Beyerer
Abstract:
Deflectometry as a technical approach to assessing reflective surfaces has now existed for almost 40 years. Different aspects and variations of the method have been studied in multiple theses and research articles, and reviews are also becoming available for certain subtopics. Still a field of active development with many unsolved problems, deflectometry now encompasses a large variety of applicat…
▽ More
Deflectometry as a technical approach to assessing reflective surfaces has now existed for almost 40 years. Different aspects and variations of the method have been studied in multiple theses and research articles, and reviews are also becoming available for certain subtopics. Still a field of active development with many unsolved problems, deflectometry now encompasses a large variety of application domains, hardware setup types, and processing workflows designed for different purposes, and spans a range from qualitative defect inspection of large vehicles to precision measurements of microscopic optics. Over these years, many exciting developments have accumulated in the underlying theory, in the systems design, and in the implementation specifics. This diversity of topics is difficult to grasp for experts and non-experts alike and may present an obstacle to a wider acceptance of deflectometry as a useful tool in other research fields and in the industry.
This paper presents an attempt to summarize the status of deflectometry, and to map relations between its notable "spin-off" branches. The intention of the paper is to provide a common communication basis for practitioners and at the same time to offer a convenient entry point for those interested in learning and using the method. The list of references is extensive but definitely not exhaustive, introducing some prominent trends and established research groups in order to facilitate further self-directed exploration by the reader.
△ Less
Submitted 10 April, 2022;
originally announced April 2022.
-
Edge Tracing using Gaussian Process Regression
Authors:
Jamie Burke,
Stuart King
Abstract:
We introduce a novel edge tracing algorithm using Gaussian process regression. Our edge-based segmentation algorithm models an edge of interest using Gaussian process regression and iteratively searches the image for edge pixels in a recursive Bayesian scheme. This procedure combines local edge information from the image gradient and global structural information from posterior curves, sampled fro…
▽ More
We introduce a novel edge tracing algorithm using Gaussian process regression. Our edge-based segmentation algorithm models an edge of interest using Gaussian process regression and iteratively searches the image for edge pixels in a recursive Bayesian scheme. This procedure combines local edge information from the image gradient and global structural information from posterior curves, sampled from the model's posterior predictive distribution, to sequentially build and refine an observation set of edge pixels. This accumulation of pixels converges the distribution to the edge of interest. Hyperparameters can be tuned by the user at initialisation and optimised given the refined observation set. This tunable approach does not require any prior training and is not restricted to any particular type of imaging domain. Due to the model's uncertainty quantification, the algorithm is robust to artefacts and occlusions which degrade the quality and continuity of edges in images. Our approach also has the ability to efficiently trace edges in image sequences by using previous-image edge traces as a priori information for consecutive images. Various applications to medical imaging and satellite imaging are used to validate the technique and comparisons are made with two commonly used edge tracing algorithms.
△ Less
Submitted 5 November, 2021;
originally announced November 2021.
-
On the Global Minimizers of Real Robust Phase Retrieval with Sparse Noise
Authors:
Aleksandr Aravkin,
James Burke,
Daiwei He
Abstract:
We study a class of real robust phase retrieval problems under a Gaussian assumption on the coding matrix when the received signal is sparsely corrupted by noise. The goal is to establish conditions on the sparsity under which the input vector can be exactly recovered. The recovery problem is formulated as the minimization of the $\ell_1$ norm of the residual.
The main contribution is a robust p…
▽ More
We study a class of real robust phase retrieval problems under a Gaussian assumption on the coding matrix when the received signal is sparsely corrupted by noise. The goal is to establish conditions on the sparsity under which the input vector can be exactly recovered. The recovery problem is formulated as the minimization of the $\ell_1$ norm of the residual.
The main contribution is a robust phase retrieval counterpart to the seminal paper by Candes and Tao on compressed sensing ($\ell_1$ regression) [Decoding by linear programming. IEEE Transactions on Information Theory, 51(12):4203-4215, 2005]. Our analysis depends on a key new property on the coding matrix which we call the {Absolute Range Property} (ARP). This property is an analogue to the Null Space Property (NSP) in compressed sensing. When the residuals are computed using squared magnitudes, we show that ARP follows from a standard Restricted Isometry Property (RIP). However, when the residuals are computed using absolute magnitudes, a new and very different kind of RIP or growth property is required. We conclude by showing that the robust phase retrieval objectives are sharp with respect to their minimizers with high probability.
△ Less
Submitted 24 May, 2019;
originally announced May 2019.
-
Real-time marker-less multi-person 3D pose estimation in RGB-Depth camera networks
Authors:
Marco Carraro,
Matteo Munaro,
Jeff Burke,
Emanuele Menegatti
Abstract:
This paper proposes a novel system to estimate and track the 3D poses of multiple persons in calibrated RGB-Depth camera networks. The multi-view 3D pose of each person is computed by a central node which receives the single-view outcomes from each camera of the network. Each single-view outcome is computed by using a CNN for 2D pose estimation and extending the resulting skeletons to 3D by means…
▽ More
This paper proposes a novel system to estimate and track the 3D poses of multiple persons in calibrated RGB-Depth camera networks. The multi-view 3D pose of each person is computed by a central node which receives the single-view outcomes from each camera of the network. Each single-view outcome is computed by using a CNN for 2D pose estimation and extending the resulting skeletons to 3D by means of the sensor depth. The proposed system is marker-less, multi-person, independent of background and does not make any assumption on people appearance and initial pose. The system provides real-time outcomes, thus being perfectly suited for applications requiring user interaction. Experimental results show the effectiveness of this work with respect to a baseline multi-view approach in different scenarios. To foster research and applications based on this work, we released the source code in OpenPTrack, an open source project for RGB-D people tracking.
△ Less
Submitted 17 October, 2017;
originally announced October 2017.
-
Convex Geometry of the Generalized Matrix-Fractional Function
Authors:
James V. Burke,
Yuan Gao,
Tim Hoheisel
Abstract:
Generalized matrix-fractional (GMF) functions are a class of matrix support functions introduced by Burke and Hoheisel as a tool for unifying a range of seemingly divergent matrix optimization problems associated with inverse problems, regularization and learning. In this paper we dramatically simplify the support function representation for GMF functions as well as the representation of their sub…
▽ More
Generalized matrix-fractional (GMF) functions are a class of matrix support functions introduced by Burke and Hoheisel as a tool for unifying a range of seemingly divergent matrix optimization problems associated with inverse problems, regularization and learning. In this paper we dramatically simplify the support function representation for GMF functions as well as the representation of their subdifferentials. These new representations allow the ready computation of a range of important related geometric objects whose formulations were previously unavailable.
△ Less
Submitted 3 March, 2017;
originally announced March 2017.
-
Location-Based and Audience-Aware Storytelling
Authors:
Jeff Burke,
Jared J. Stein
Abstract:
While the daily user of digital, Internet-enabled devices has some explicit control over what they read and see, the providers fulfilling searches, offering options, and presenting material are using increasingly sophisticated real-time algorithms that tune and target content for the particular user. They redefine the historical relationships between tellers and users, providing a responsiveness p…
▽ More
While the daily user of digital, Internet-enabled devices has some explicit control over what they read and see, the providers fulfilling searches, offering options, and presenting material are using increasingly sophisticated real-time algorithms that tune and target content for the particular user. They redefine the historical relationships between tellers and users, providing a responsiveness paralleled only by forms of live performance incorporating elements of improvisation and audience interaction. The general accessibility of algorithmically driven content delivery techniques suggests significant untapped potential for new approaches to narrative beyond advertising and commercially orientated customization.
△ Less
Submitted 26 September, 2016;
originally announced September 2016.
-
The connection between Bayesian estimation of a Gaussian random field and RKHS
Authors:
Aleksandr Y. Aravkin,
Bradley M. Bell,
James V. Burke,
Gianluigi Pillonetto
Abstract:
Reconstruction of a function from noisy data is often formulated as a regularized optimization problem over an infinite-dimensional reproducing kernel Hilbert space (RKHS). The solution describes the observed data and has a small RKHS norm. When the data fit is measured using a quadratic loss, this estimator has a known statistical interpretation. Given the noisy measurements, the RKHS estimate re…
▽ More
Reconstruction of a function from noisy data is often formulated as a regularized optimization problem over an infinite-dimensional reproducing kernel Hilbert space (RKHS). The solution describes the observed data and has a small RKHS norm. When the data fit is measured using a quadratic loss, this estimator has a known statistical interpretation. Given the noisy measurements, the RKHS estimate represents the posterior mean (minimum variance estimate) of a Gaussian random field with covariance proportional to the kernel associated with the RKHS. In this paper, we provide a statistical interpretation when more general losses are used, such as absolute value, Vapnik or Huber. Specifically, for any finite set of sampling locations (including where the data were collected), the MAP estimate for the signal samples is given by the RKHS estimate evaluated at these locations.
△ Less
Submitted 17 July, 2013; v1 submitted 22 January, 2013;
originally announced January 2013.
-
Securing Instrumented Environments over Content-Centric Networking: the Case of Lighting Control
Authors:
Jeff Burke,
Paolo Gasti,
Naveen Nathan,
Gene Tsudik
Abstract:
Instrumented environments, such as modern building automation systems (BAS), are becoming commonplace and are increasingly interconnected with (and sometimes by) enterprise networks and the Internet. Regardless of the underlying communication platform, secure control of devices in such environments is a challenging task. The current trend is to move from proprietary communication media and protoco…
▽ More
Instrumented environments, such as modern building automation systems (BAS), are becoming commonplace and are increasingly interconnected with (and sometimes by) enterprise networks and the Internet. Regardless of the underlying communication platform, secure control of devices in such environments is a challenging task. The current trend is to move from proprietary communication media and protocols to IP over Ethernet. While the move to IP represents progress, new and different Internet architectures might be better-suited for instrumented environments. In this paper, we consider security of instrumented environments in the context of Content-Centric Networking (CCN). In particular, we focus on building automation over Named-Data Networking (NDN), a prominent instance of CCN. After identifying security requirements in a specific BAS sub-domain (lighting control), we construct a concrete NDN-based security architecture, analyze its properties and report on preliminary implementation and experimental results. We believe in securing a communication paradigm well outside of its claimed forte of content distribution. At the same time, we provide a viable (secure and efficient) communication platform for a class of instrumented environments exemplified by lighting control.
△ Less
Submitted 7 August, 2012;
originally announced August 2012.
