-
Beyond Thumbs Up/Down: Untangling Challenges of Fine-Grained Feedback for Text-to-Image Generation
Authors:
Katherine M. Collins,
Najoung Kim,
Yonatan Bitton,
Verena Rieser,
Shayegan Omidshafiei,
Yushi Hu,
Sherol Chen,
Senjuti Dutta,
Minsuk Chang,
Kimin Lee,
Youwei Liang,
Georgina Evans,
Sahil Singla,
Gang Li,
Adrian Weller,
Junfeng He,
Deepak Ramachandran,
Krishnamurthy Dj Dvijotham
Abstract:
Human feedback plays a critical role in learning and refining reward models for text-to-image generation, but the optimal form the feedback should take for learning an accurate reward function has not been conclusively established. This paper investigates the effectiveness of fine-grained feedback which captures nuanced distinctions in image quality and prompt-alignment, compared to traditional co…
▽ More
Human feedback plays a critical role in learning and refining reward models for text-to-image generation, but the optimal form the feedback should take for learning an accurate reward function has not been conclusively established. This paper investigates the effectiveness of fine-grained feedback which captures nuanced distinctions in image quality and prompt-alignment, compared to traditional coarse-grained feedback (for example, thumbs up/down or ranking between a set of options). While fine-grained feedback holds promise, particularly for systems catering to diverse societal preferences, we show that demonstrating its superiority to coarse-grained feedback is not automatic. Through experiments on real and synthetic preference data, we surface the complexities of building effective models due to the interplay of model choice, feedback type, and the alignment between human judgment and computational interpretation. We identify key challenges in eliciting and utilizing fine-grained feedback, prompting a reassessment of its assumed benefits and practicality. Our findings -- e.g., that fine-grained feedback can lead to worse models for a fixed budget, in some settings; however, in controlled settings with known attributes, fine grained rewards can indeed be more helpful -- call for careful consideration of feedback attributes and potentially beckon novel modeling approaches to appropriately unlock the potential value of fine-grained feedback in-the-wild.
△ Less
Submitted 24 June, 2024;
originally announced June 2024.
-
Private Gradient Descent for Linear Regression: Tighter Error Bounds and Instance-Specific Uncertainty Estimation
Authors:
Gavin Brown,
Krishnamurthy Dvijotham,
Georgina Evans,
Daogao Liu,
Adam Smith,
Abhradeep Thakurta
Abstract:
We provide an improved analysis of standard differentially private gradient descent for linear regression under the squared error loss. Under modest assumptions on the input, we characterize the distribution of the iterate at each time step.
Our analysis leads to new results on the algorithm's accuracy: for a proper fixed choice of hyperparameters, the sample complexity depends only linearly on…
▽ More
We provide an improved analysis of standard differentially private gradient descent for linear regression under the squared error loss. Under modest assumptions on the input, we characterize the distribution of the iterate at each time step.
Our analysis leads to new results on the algorithm's accuracy: for a proper fixed choice of hyperparameters, the sample complexity depends only linearly on the dimension of the data. This matches the dimension-dependence of the (non-private) ordinary least squares estimator as well as that of recent private algorithms that rely on sophisticated adaptive gradient-clipping schemes (Varshney et al., 2022; Liu et al., 2023).
Our analysis of the iterates' distribution also allows us to construct confidence intervals for the empirical optimizer which adapt automatically to the variance of the algorithm on a particular data set. We validate our theorems through experiments on synthetic data.
△ Less
Submitted 20 February, 2024;
originally announced February 2024.
-
The effect of variable labels on deep learning models trained to predict breast density
Authors:
Steven Squires,
Elaine F. Harkness,
D. Gareth Evans,
Susan M. Astley
Abstract:
Purpose: High breast density is associated with reduced efficacy of mammographic screening and increased risk of developing breast cancer. Accurate and reliable automated density estimates can be used for direct risk prediction and passing density related information to further predictive models. Expert reader assessments of density show a strong relationship to cancer risk but also inter-reader v…
▽ More
Purpose: High breast density is associated with reduced efficacy of mammographic screening and increased risk of developing breast cancer. Accurate and reliable automated density estimates can be used for direct risk prediction and passing density related information to further predictive models. Expert reader assessments of density show a strong relationship to cancer risk but also inter-reader variation. The effect of label variability on model performance is important when considering how to utilise automated methods for both research and clinical purposes. Methods: We utilise subsets of images with density labels to train a deep transfer learning model which is used to assess how label variability affects the mapping from representation to prediction. We then create two end-to-end deep learning models which allow us to investigate the effect of label variability on the model representation formed. Results: We show that the trained mappings from representations to labels are altered considerably by the variability of reader scores. Training on labels with distribution variation removed causes the Spearman rank correlation coefficients to rise from $0.751\pm0.002$ to either $0.815\pm0.006$ when averaging across readers or $0.844\pm0.002$ when averaging across images. However, when we train different models to investigate the representation effect we see little difference, with Spearman rank correlation coefficients of $0.846\pm0.006$ and $0.850\pm0.006$ showing no statistically significant difference in the quality of the model representation with regard to density prediction. Conclusions: We show that the mapping between representation and mammographic density prediction is significantly affected by label variability. However, the effect of the label variability on the model representation is limited.
△ Less
Submitted 8 October, 2022;
originally announced October 2022.
-
Pattern recognition in the nucleation kinetics of non-equilibrium self-assembly
Authors:
Constantine Glen Evans,
Jackson O'Brien,
Erik Winfree,
Arvind Murugan
Abstract:
Inspired by biology's most sophisticated computer, the brain, neural networks constitute a profound reformulation of computational principles. Remarkably, analogous high-dimensional, highly-interconnected computational architectures also arise within information-processing molecular systems inside living cells, such as signal transduction cascades and genetic regulatory networks. Might neuromorphi…
▽ More
Inspired by biology's most sophisticated computer, the brain, neural networks constitute a profound reformulation of computational principles. Remarkably, analogous high-dimensional, highly-interconnected computational architectures also arise within information-processing molecular systems inside living cells, such as signal transduction cascades and genetic regulatory networks. Might neuromorphic collective modes be found more broadly in other physical and chemical processes, even those that ostensibly play non-information-processing roles such as protein synthesis, metabolism, or structural self-assembly? Here we examine nucleation during self-assembly of multicomponent structures, showing that high-dimensional patterns of concentrations can be discriminated and classified in a manner similar to neural network computation. Specifically, we design a set of 917 DNA tiles that can self-assemble in three alternative ways such that competitive nucleation depends sensitively on the extent of co-localization of high-concentration tiles within the three structures. The system was trained in-silico to classify a set of 18 grayscale 30 x 30 pixel images into three categories. Experimentally, fluorescence and atomic force microscopy monitoring during and after a 150-hour anneal established that all trained images were correctly classified, while a test set of image variations probed the robustness of the results. While slow compared to prior biochemical neural networks, our approach is surprisingly compact, robust, and scalable. This success suggests that ubiquitous physical phenomena, such as nucleation, may hold powerful information processing capabilities when scaled up as high-dimensional multicomponent systems.
△ Less
Submitted 5 October, 2023; v1 submitted 13 July, 2022;
originally announced July 2022.
-
Dampen the Stop-and-Go Traffic with Connected and Automated Vehicles -- A Deep Reinforcement Learning Approach
Authors:
Liming Jiang,
Yuanchang Xie,
Danjue Chen,
Tienan Li,
Nicholas G. Evans
Abstract:
Stop-and-go traffic poses many challenges to tranportation system, but its formation and mechanism are still under exploration.however, it has been proved that by introducing Connected Automated Vehicles(CAVs) with carefully designed controllers one could dampen the stop-and-go waves in the vehicle fleet. Instead of using analytical model, this study adopts reinforcement learning to control the be…
▽ More
Stop-and-go traffic poses many challenges to tranportation system, but its formation and mechanism are still under exploration.however, it has been proved that by introducing Connected Automated Vehicles(CAVs) with carefully designed controllers one could dampen the stop-and-go waves in the vehicle fleet. Instead of using analytical model, this study adopts reinforcement learning to control the behavior of CAV and put a single CAV at the 2nd position of a vehicle fleet with the purpose to dampen the speed oscillation from the fleet leader and help following human drivers adopt more smooth driving behavior. The result show that our controller could decrease the spped oscillation of the CAV by 54% and 8%-28% for those following human-driven vehicles. Significant fuel consumption savings are also observed. Additionally, the result suggest that CAVs may act as a traffic stabilizer if they choose to behave slightly altruistically.
△ Less
Submitted 17 May, 2020;
originally announced May 2020.
-
Bold Hearts Team Description for RoboCup 2019 (Humanoid Kid Size League)
Authors:
Marcus M. Scheunemann,
Sander G. van Dijk,
Rebecca Miko,
Daniel Barry,
George M. Evans,
Alessandra Rossi,
Daniel Polani
Abstract:
We participated in the RoboCup 2018 competition in Montreal with our newly developed BoldBot based on the Darwin-OP and mostly self-printed custom parts. This paper is about the lessons learnt from that competition and further developments for the RoboCup 2019 competition. Firstly, we briefly introduce the team along with an overview of past achievements. We then present a simple, standalone 2D si…
▽ More
We participated in the RoboCup 2018 competition in Montreal with our newly developed BoldBot based on the Darwin-OP and mostly self-printed custom parts. This paper is about the lessons learnt from that competition and further developments for the RoboCup 2019 competition. Firstly, we briefly introduce the team along with an overview of past achievements. We then present a simple, standalone 2D simulator we use for simplifying the entry for new members with making basic RoboCup concepts quickly accessible. We describe our approach for semantic-segmentation for our vision used in the 2018 competition, which replaced the lookup-table (LUT) implementation we had before. We also discuss the extra structural support we plan to add to the printed parts of the BoldBot and our transition to ROS 2 as our new middleware. Lastly, we will present a collection of open-source contributions of our team.
△ Less
Submitted 22 April, 2019;
originally announced April 2019.
-
Towards a Live Anonymous Question Queue To Address Student Apprehension
Authors:
Lloyd Montgomery,
Guy Evans,
Francis Harrison,
Daniela Damian
Abstract:
In today's university climate many first and second year classes have over a hundred students. Large classrooms make some students apprehensive about asking questions. An anonymous method of submitting questions to an instructor would allow students to ask their questions without feeling apprehensive. In this paper we propose a Live Anonymous Question Queue (LAQQ), a system that facilitates anonym…
▽ More
In today's university climate many first and second year classes have over a hundred students. Large classrooms make some students apprehensive about asking questions. An anonymous method of submitting questions to an instructor would allow students to ask their questions without feeling apprehensive. In this paper we propose a Live Anonymous Question Queue (LAQQ), a system that facilitates anonymous question submissions in real time to mitigate student apprehension, increase student participation, and provide real-time feedback to the instructor. To study the necessary features of an LAQQ, we conducted a study of a system, namely Google Moderator, which best approached our concept of an LAQQ. We deployed Google moderator in large lectures and studied its support of a number of features that we envisioned for an LAQQ. Through our class observations, interviews with instructors, and surveys with the students, our results suggest that an LAQQ system must provide support for: notification of question submission to provide awareness for the instructor, and context for questions to allow an instructor to easily answer a question. Additionally our results suggest that an LAQQ system must be accessible and usable on multiple platforms. Finally our results suggest that in order to be successful in the classroom an LAQQ system must be fully adopted by the instructor and the classroom organizational structure must change to accommodate the use of the LAQQ.
△ Less
Submitted 4 January, 2019;
originally announced January 2019.
-
A Systems Approach to Achieving the Benefits of Artificial Intelligence in UK Defence
Authors:
Gavin Pearson,
Phil Jolley,
Geraint Evans
Abstract:
The ability to exploit the opportunities offered by AI within UK Defence calls for an understanding of systemic issues required to achieve an effective operational capability. This paper provides the authors' views of issues which currently block UK Defence from fully benefitting from AI technology. These are situated within a reference model for the AI Value Train, so enabling the community to ad…
▽ More
The ability to exploit the opportunities offered by AI within UK Defence calls for an understanding of systemic issues required to achieve an effective operational capability. This paper provides the authors' views of issues which currently block UK Defence from fully benefitting from AI technology. These are situated within a reference model for the AI Value Train, so enabling the community to address the exploitation of such data and software intensive systems in a systematic, end to end manner. The paper sets out the conditions for success including: Researching future solutions to known problems and clearly defined use cases; Addressing achievable use cases to show benefit; Enhancing the availability of Defence-relevant data; Enhancing Defence 'know how' in AI; Operating Software Intensive supply chain eco-systems at required breadth and pace; Governance and, the integration of software and platform supply chains and operating models.
△ Less
Submitted 28 September, 2018;
originally announced September 2018.
-
Loss-tolerant quantum secure positioning with weak laser sources
Authors:
Charles Ci Wen Lim,
Feihu Xu,
George Siopsis,
Eric Chitambar,
Philip G. Evans,
Bing Qi
Abstract:
Quantum position verification (QPV) is the art of verifying the geographical location of an untrusted party. Recently, it has been shown that the widely studied Bennett & Brassard 1984 (BB84) QPV protocol is insecure after the 3 dB loss point assuming local operations and classical communication (LOCC) adversaries. Here, we propose a time-reversed entanglement swapping QPV protocol (based on measu…
▽ More
Quantum position verification (QPV) is the art of verifying the geographical location of an untrusted party. Recently, it has been shown that the widely studied Bennett & Brassard 1984 (BB84) QPV protocol is insecure after the 3 dB loss point assuming local operations and classical communication (LOCC) adversaries. Here, we propose a time-reversed entanglement swapping QPV protocol (based on measurement-device-independent quantum cryptography) that is highly robust against quantum channel loss. First, assuming ideal qubit sources, we show that the protocol is secure against LOCC adversaries for any quantum channel loss, thereby overcoming the 3 dB loss limit. Then, we analyze the security of the protocol in a more practical setting involving weak laser sources and linear optics. In this setting, we find that the security only degrades by an additive constant and the protocol is able to verify positions up to 47 dB channel loss.
△ Less
Submitted 27 July, 2016;
originally announced July 2016.
-
Estimating Optimal Active Learning via Model Retraining Improvement
Authors:
Lewis P. G. Evans,
Niall M. Adams,
Christoforos Anagnostopoulos
Abstract:
A central question for active learning (AL) is: "what is the optimal selection?" Defining optimality by classifier loss produces a new characterisation of optimal AL behaviour, by treating expected loss reduction as a statistical target for estimation. This target forms the basis of model retraining improvement (MRI), a novel approach providing a statistical estimation framework for AL. This frame…
▽ More
A central question for active learning (AL) is: "what is the optimal selection?" Defining optimality by classifier loss produces a new characterisation of optimal AL behaviour, by treating expected loss reduction as a statistical target for estimation. This target forms the basis of model retraining improvement (MRI), a novel approach providing a statistical estimation framework for AL. This framework is constructed to address the central question of AL optimality, and to motivate the design of estimation algorithms. MRI allows the exploration of optimal AL behaviour, and the examination of AL heuristics, showing precisely how they make sub-optimal selections. The abstract formulation of MRI is used to provide a new guarantee for AL, that an unbiased MRI estimator should outperform random selection. This MRI framework reveals intricate estimation issues that in turn motivate the construction of new statistical AL algorithms. One new algorithm in particular performs strongly in a large-scale experimental study, compared to standard AL methods. This competitive performance suggests that practical efforts to minimise estimation bias may be important for AL applications.
△ Less
Submitted 5 February, 2015;
originally announced February 2015.
-
Targeting Optimal Active Learning via Example Quality
Authors:
Lewis P. G. Evans,
Niall M. Adams,
Christoforos Anagnostopoulos
Abstract:
In many classification problems unlabelled data is abundant and a subset can be chosen for labelling. This defines the context of active learning (AL), where methods systematically select that subset, to improve a classifier by retraining. Given a classification problem, and a classifier trained on a small number of labelled examples, consider the selection of a single further example. This exampl…
▽ More
In many classification problems unlabelled data is abundant and a subset can be chosen for labelling. This defines the context of active learning (AL), where methods systematically select that subset, to improve a classifier by retraining. Given a classification problem, and a classifier trained on a small number of labelled examples, consider the selection of a single further example. This example will be labelled by the oracle and then used to retrain the classifier. This example selection raises a central question: given a fully specified stochastic description of the classification problem, which example is the optimal selection? If optimality is defined in terms of loss, this definition directly produces expected loss reduction (ELR), a central quantity whose maximum yields the optimal example selection. This work presents a new theoretical approach to AL, example quality, which defines optimal AL behaviour in terms of ELR. Once optimal AL behaviour is defined mathematically, reasoning about this abstraction provides insights into AL. In a theoretical context the optimal selection is compared to existing AL methods, showing that heuristics can make sub-optimal selections. Algorithms are constructed to estimate example quality directly. A large-scale experimental study shows these algorithms to be competitive with standard AL methods.
△ Less
Submitted 30 July, 2014;
originally announced July 2014.
-
Neurally Implementable Semantic Networks
Authors:
Garrett N. Evans,
John C. Collins
Abstract:
We propose general principles for semantic networks allowing them to be implemented as dynamical neural networks. Major features of our scheme include: (a) the interpretation that each node in a network stands for a bound integration of the meanings of all nodes and external events the node links with; (b) the systematic use of nodes that stand for categories or types, with separate nodes for inst…
▽ More
We propose general principles for semantic networks allowing them to be implemented as dynamical neural networks. Major features of our scheme include: (a) the interpretation that each node in a network stands for a bound integration of the meanings of all nodes and external events the node links with; (b) the systematic use of nodes that stand for categories or types, with separate nodes for instances of these types; (c) an implementation of relationships that does not use intrinsically typed links between nodes.
△ Less
Submitted 18 March, 2013;
originally announced March 2013.
