Showing 1–8 of 8 results for author: Molina, F

Test Oracle Automation in the era of LLMs

Authors: Facundo Molina, Alessandra Gorla

Abstract: The effectiveness of a test suite in detecting faults highly depends on the correctness and completeness of its test oracles. Large Language Models (LLMs) have already demonstrated remarkable proficiency in tackling diverse software testing tasks, such as automated test generation and program repair. This paper aims to enable discussions on the potential of using LLMs for test oracle automation, a… ▽ More The effectiveness of a test suite in detecting faults highly depends on the correctness and completeness of its test oracles. Large Language Models (LLMs) have already demonstrated remarkable proficiency in tackling diverse software testing tasks, such as automated test generation and program repair. This paper aims to enable discussions on the potential of using LLMs for test oracle automation, along with the challenges that may emerge during the generation of various types of oracles. Additionally, our aim is to initiate discussions on the primary threats that SE researchers must consider when employing LLMs for oracle automation, encompassing concerns regarding oracle deficiencies and data leakages. △ Less

Submitted 21 May, 2024; originally announced May 2024.

Specification Inference for Evolving Systems

Authors: Renzo Degiovanni, Facundo Molina, Agustin Nolasco, Nazareno Aguirre, Mike Papadakis

Abstract: In this paper, we propose an assertion-based approach to capture software evolution, through the notion of commit-relevant specification. A commit-relevant specification summarises the program properties that have changed as a consequence of a commit (understood as a specific software modification), via two sets of assertions, the delta-added assertions, properties that did not hold in the pre-com… ▽ More In this paper, we propose an assertion-based approach to capture software evolution, through the notion of commit-relevant specification. A commit-relevant specification summarises the program properties that have changed as a consequence of a commit (understood as a specific software modification), via two sets of assertions, the delta-added assertions, properties that did not hold in the pre-commit version but hold on the post-commit, and the delta-removed assertions, those that were valid in the pre-commit, but no longer hold after the code change. We also present DeltaSpec, an approach that combines test generation and dynamic specification inference to automatically compute commit-relevant specifications from given commits. We evaluate DeltaSpec on two datasets that include a total of 57 commits (63 classes and 797 methods). We show that commit-relevant assertions can precisely describe the semantic deltas of code changes, providing a useful mechanism for validating the behavioural evolution of software. We also show that DeltaSpec can infer 88% of the manually written commit-relevant assertions expressible in the language supported by the tool. Moreover, our experiments demonstrate that DeltaSpec's inferred assertions are effective to detect regression faults. More precisely, we show that commit-relevant assertions can detect, on average, 78.3% of the artificially seeded faults that interact with the code changes. We also show that assertions in the delta are 58.3% more effective in detecting commit-relevant mutants than assertions outside the delta, and that it takes on average 169% fewer assertions when these are commit-relevant, compared to using general valid assertions, to achieve a same commit-relevant mutation score. △ Less

Submitted 29 January, 2023; originally announced January 2023.

Assertion Inferring Mutants

Authors: Aayush Garg, Renzo Degiovanni, Facundo Molina, Mike Papadakis, Nazareno Aguirre, Maxime Cordy, Yves Le Traon

Abstract: Specification inference techniques aim at (automatically) inferring a set of assertions that capture the exhibited software behaviour by generating and filtering assertions through dynamic test executions and mutation testing. Although powerful, such techniques are computationally expensive due to a large number of assertions, test cases and mutated versions that need to be executed. To overcome t… ▽ More Specification inference techniques aim at (automatically) inferring a set of assertions that capture the exhibited software behaviour by generating and filtering assertions through dynamic test executions and mutation testing. Although powerful, such techniques are computationally expensive due to a large number of assertions, test cases and mutated versions that need to be executed. To overcome this issue, we demonstrate that a small subset, i.e., 12.95% of the mutants used by mutation testing tools is sufficient for assertion inference, this subset is significantly different, i.e., 71.59% different from the subsuming mutant set that is frequently cited by mutation testing literature, and can be statically approximated through a learning based method. In particular, we propose AIMS, an approach that selects Assertion Inferring Mutants, i.e., a set of mutants that are well-suited for assertion inference, with 0.58 MCC, 0.79 Precision, and 0.49 Recall. We evaluate AIMS on 46 programs and demonstrate that it has comparable inference capabilities with full mutation analysis (misses 12.49% of assertions) while significantly limiting execution cost (runs 46.29 times faster). A comparison with randomly selected sets of mutants, shows the superiority of AIMS by inferring 36% more assertions while requiring approximately equal amount of execution time. We also show that AIMS 's inferring capabilities are almost complete as it infers 96.15% of ground truth assertions, (i.e., a complete set of assertions that were manually constructed) while Random Mutant Selection infers 19.23% of them. More importantly, AIMS enables assertion inference techniques to scale on subjects where full mutation testing is prohibitively expensive and Random Mutant Selection does not lead to any assertion. △ Less

Submitted 28 January, 2023; originally announced January 2023.

Fuzzing Class Specifications

Authors: Facundo Molina, Marcelo d'Amorim, Nazareno Aguirre

Abstract: Expressing class specifications via executable constraints is important for various software engineering tasks such as test generation, bug finding and automated debugging, but developers rarely write them. Techniques that infer specifications from code exist to fill this gap, but they are designed to support specific kinds of assertions and are difficult to adapt to support different assertion la… ▽ More Expressing class specifications via executable constraints is important for various software engineering tasks such as test generation, bug finding and automated debugging, but developers rarely write them. Techniques that infer specifications from code exist to fill this gap, but they are designed to support specific kinds of assertions and are difficult to adapt to support different assertion languages, e.g., to add support for quantification, or additional comparison operators, such as membership or containment. To address the above issue, we present SpecFuzzer, a novel technique that combines grammar-based fuzzing, dynamic invariant detection, and mutation analysis, to automatically produce class specifications. SpecFuzzer uses: (i) a fuzzer as a generator of candidate assertions derived from a grammar that is automatically obtained from the class definition; (ii) a dynamic invariant detector -- Daikon -- to filter out assertions invalidated by a test suite; and (iii) a mutation-based mechanism to cluster and rank assertions, so that similar constraints are grouped and then the stronger prioritized. Grammar-based fuzzing enables SpecFuzzer to be straightforwardly adapted to support different specification languages, by manipulating the fuzzing grammar, e.g., to include additional operators. We evaluate our technique on a benchmark of 43 Java methods employed in the evaluation of the state-of-the-art techniques GAssert and EvoSpex. Our results show that SpecFuzzer can easily support a more expressive assertion language, over which is more effective than GAssert and EvoSpex in inferring specifications, according to standard performance metrics. △ Less

Submitted 26 January, 2022; originally announced January 2022.

EvoSpex: An Evolutionary Algorithm for Learning Postconditions

Authors: Facundo Molina, Pablo Ponzio, Nazareno Aguirre, Marcelo Frias

Abstract: Software reliability is a primary concern in the construction of software, and thus a fundamental component in the definition of software quality. Analyzing software reliability requires a specification of the intended behavior of the software under analysis, and at the source code level, such specifications typically take the form of assertions. Unfortunately, software many times lacks such speci… ▽ More Software reliability is a primary concern in the construction of software, and thus a fundamental component in the definition of software quality. Analyzing software reliability requires a specification of the intended behavior of the software under analysis, and at the source code level, such specifications typically take the form of assertions. Unfortunately, software many times lacks such specifications, or only provides them for scenario-specific behaviors, as assertions accompanying tests. This issue seriously diminishes the analyzability of software with respect to its reliability. In this paper, we tackle this problem by proposing a technique that, given a Java method, automatically produces a specification of the method's current behavior, in the form of postcondition assertions. This mechanism is based on generating executions of the method under analysis to obtain valid pre/post state pairs, mutating these pairs to obtain (allegedly) invalid ones, and then using a genetic algorithm to produce an assertion that is satisfied by the valid pre/post pairs, while leaving out the invalid ones. The technique, which targets in particular methods of reference-based class implementations, is assessed on a benchmark of open source Java projects, showing that our genetic algorithm is able to generate post-conditions that are stronger and more accurate, than those generated by related automated approaches, as evaluated by an automated oracle assessment tool. Moreover, our technique is also able to infer an important part of manually written rich postconditions in verified classes, and reproduce contracts for methods whose class implementations were automatically synthesized from specifications. △ Less

Submitted 1 March, 2021; v1 submitted 26 February, 2021; originally announced February 2021.

A Blockchain based and GDPR-compliant design of a system for digital education certificates

Authors: Fernanda Molina, Gustavo Betarte, Carlos Luna

Abstract: Blockchain is an incipient technology that offers many strengths compared to traditional systems, such as decentralization, transparency and traceability. However, if the technology is to be used for processing personal data, complementary mechanisms must be identified that provide support for building systems that meet security and data protection requirements. We study the integration of off-cha… ▽ More Blockchain is an incipient technology that offers many strengths compared to traditional systems, such as decentralization, transparency and traceability. However, if the technology is to be used for processing personal data, complementary mechanisms must be identified that provide support for building systems that meet security and data protection requirements. We study the integration of off-chain capabilities in blockchain-based solutions moving data or computational operations outside the core blockchain network. We develop a thorough analysis of the European data protection regulation and discuss the weaknesses and strengths, regarding the security and privacy requirements established by that regulation, of solutions built using blockchain technology. We also put forward a methodological framework that helps systems designers in combining operational off-chain constructs with traditional blockchain functionalities in order to build more secure and privacy aware solutions. We illustrate the use of that framework presenting and discussing the design of a system that provides services to handle, store and validate digital academic certificates. △ Less

Submitted 24 October, 2020; originally announced October 2020.

Estado Actual de la Práctica de la Ingeniería de Software en México

Authors: Reyes Juárez-Ramírez, Karen Cortés Verdín, Beatriz Angélica Toscano de la Torre, Hanna Oktaba, Carlos Alberto Fernández-y-Fernández, Brenda Leticia Flores Ríos, Fabiola Angulo Molina

Abstract: The software engineering is a relatively new discipline compared to other sciences, since the origins of the term itself dates back to the years 1968 and 1969. At present, the market and the software industry have a significant relevance in several countries of the world; however, although Mexico is immersed in this race, has not even reached the level of success achieved in other countries in thi… ▽ More The software engineering is a relatively new discipline compared to other sciences, since the origins of the term itself dates back to the years 1968 and 1969. At present, the market and the software industry have a significant relevance in several countries of the world; however, although Mexico is immersed in this race, has not even reached the level of success achieved in other countries in this sector. This paper presents an overview of the situation that keeps the practice of software engineering in Mexico, with emphasis on the academic realm. It shows a compilation of scientific research activity carried out in universities, as well as a brief analysis of undergraduate educational programs including the software engineering discipline . At the end, future work to be done is proposed in order to find a point of convergence between academia and industry, and also to support the flourishing of this business which somehow will have a positive impact on the economy of our country. △ Less

Submitted 24 October, 2013; originally announced October 2013.

Journal ref: Congreso Internacional de Investigación e Innovación en Ingeniería de Software (Conisoft 2013), pp. 3-14, Xalapa, Veracruz, México, 2013. ISBN: 978-0-615-89523-9

IACTalks: an on-line archive of astronomy-related seminars

Authors: Johan H. Knapen, Jorge A. Pérez Prieto, Tariq Shahbaz, Anna Ferré-Mateu, Nicola Caon, Cristina Ramos Almeida, Brandon Tingley, Valentina Luridiana, Inés Flores-Cacho, Orlagh Creevey, Arturo Manchado Torres, Ignacio Trujillo, Maria Rosa Zapatero Osorio, Francisco Sánchez Martínez, Francisco López Molina, Gabriel Pérez Díaz, Miguel Briganti, Inés Bonet

Abstract: We present IACTalks, a free and open access seminars archive (http://iactalks.iac.es) aimed at promoting astronomy and the exchange of ideas by providing high-quality scientific seminars to the astronomical community. The archive of seminars and talks given at the Instituto de Astrofiísica de Canarias goes back to 2008. Over 360 talks and seminars are now freely available by streaming over the int… ▽ More We present IACTalks, a free and open access seminars archive (http://iactalks.iac.es) aimed at promoting astronomy and the exchange of ideas by providing high-quality scientific seminars to the astronomical community. The archive of seminars and talks given at the Instituto de Astrofiísica de Canarias goes back to 2008. Over 360 talks and seminars are now freely available by streaming over the internet. We describe the user interface, which includes two video streams, one showing the speaker, the other the presentation. A search function is available, and seminars are indexed by keywords and in some cases by series, such as special training courses or the 2011 Winter School of Astrophysics, on secular evolution of galaxies. The archive is made available as an open resource, to be used by scientists and the public. △ Less

Submitted 27 June, 2012; originally announced June 2012.

Comments: 2 pages, 2 figures