Showing 1–9 of 9 results for author: Morzy, M

Nie pozwól algorytmom rządzić Twoim koszykiem: systemy rekomendacyjne w dobie Omnibusa

Authors: Mikołaj Morzy, Mirosław Sobieraj, Sebastian Sikora

Abstract: The Omnibus Directive is an essential part of the European Union's New Deal for Consumers. The Directive introduces new regulations in trade, including e-commerce, with the main goal being to increase transparency, fairness and consumer protection. The authors critically draw attention to a significant oversight in the Omnibus Directive, namely the lack of consideration of recommendation systems.… ▽ More The Omnibus Directive is an essential part of the European Union's New Deal for Consumers. The Directive introduces new regulations in trade, including e-commerce, with the main goal being to increase transparency, fairness and consumer protection. The authors critically draw attention to a significant oversight in the Omnibus Directive, namely the lack of consideration of recommendation systems. Recommendation engines can be a source of potentially harmful practices affecting consumers, hence the need for a directive extension. The proposals presented in this article include the introduction of ethical supervision over recommendation systems to minimize the risk of negative effects of their recommendations, as well as a clear explanation of the criteria on which recommendations are made -- similar to search result rankings. -- Dyrektywa Omnibus stanowi istotną część Nowego Ładu dla Konsumentów (ang. \emph{New Deal for Consumers}) Unii Europejskiej. Dyrektywa wprowadza nowe regulacje w handlu, w tym e-commerce, których głównym celem jest zwiększenie przejrzystości, uczciwości i ochrony konsumentów. Autorzy krytycznie zwracają uwagę na istotne zaniedbanie w dyrektywie Omnibus, jakim jest brak uwzględnienia systemów rekomendacyjnych. Silniki rekomendacyjne mogą być źródłem potencjalnie szkodliwych praktyk uderzających w konsumentów, stąd niezbędne jest rozszerzenie dyrektywy. Propozycje przedstawione w niniejszym artykule obejmują wprowadzenie etycznego nadzoru nad systemami rekomendującymi, aby zminimalizować ryzyko negatywnych skutków ich rekomendacji, a także jasne wyjaśnienie kryteriów, na podstawie których dokonywane są rekomendacje -- analogicznie do rankingów wyników wyszukiwania. △ Less

Submitted 24 January, 2024; originally announced February 2024.

Comments: 7 pages, in Polish

Massively Multilingual Corpus of Sentiment Datasets and Multi-faceted Sentiment Classification Benchmark

Authors: Łukasz Augustyniak, Szymon Woźniak, Marcin Gruza, Piotr Gramacki, Krzysztof Rajda, Mikołaj Morzy, Tomasz Kajdanowicz

Abstract: Despite impressive advancements in multilingual corpora collection and model training, developing large-scale deployments of multilingual models still presents a significant challenge. This is particularly true for language tasks that are culture-dependent. One such example is the area of multilingual sentiment analysis, where affective markers can be subtle and deeply ensconced in culture. This w… ▽ More Despite impressive advancements in multilingual corpora collection and model training, developing large-scale deployments of multilingual models still presents a significant challenge. This is particularly true for language tasks that are culture-dependent. One such example is the area of multilingual sentiment analysis, where affective markers can be subtle and deeply ensconced in culture. This work presents the most extensive open massively multilingual corpus of datasets for training sentiment models. The corpus consists of 79 manually selected datasets from over 350 datasets reported in the scientific literature based on strict quality criteria. The corpus covers 27 languages representing 6 language families. Datasets can be queried using several linguistic and functional features. In addition, we present a multi-faceted sentiment classification benchmark summarizing hundreds of experiments conducted on different base models, training objectives, dataset collections, and fine-tuning strategies. △ Less

Submitted 13 June, 2023; originally announced June 2023.

Comments: submitted to NeurIPS 2023 Datasets and Benchmarks track. Dataset: https://huggingface.co/datasets/Brand24/mms Code: https://github.com/Brand24-AI/mms_benchmark

This is the way: designing and compiling LEPISZCZE, a comprehensive NLP benchmark for Polish

Authors: Łukasz Augustyniak, Kamil Tagowski, Albert Sawczyn, Denis Janiak, Roman Bartusiak, Adrian Szymczak, Marcin Wątroba, Arkadiusz Janz, Piotr Szymański, Mikołaj Morzy, Tomasz Kajdanowicz, Maciej Piasecki

Abstract: The availability of compute and data to train larger and larger language models increases the demand for robust methods of benchmarking the true progress of LM training. Recent years witnessed significant progress in standardized benchmarking for English. Benchmarks such as GLUE, SuperGLUE, or KILT have become de facto standard tools to compare large language models. Following the trend to replica… ▽ More The availability of compute and data to train larger and larger language models increases the demand for robust methods of benchmarking the true progress of LM training. Recent years witnessed significant progress in standardized benchmarking for English. Benchmarks such as GLUE, SuperGLUE, or KILT have become de facto standard tools to compare large language models. Following the trend to replicate GLUE for other languages, the KLEJ benchmark has been released for Polish. In this paper, we evaluate the progress in benchmarking for low-resourced languages. We note that only a handful of languages have such comprehensive benchmarks. We also note the gap in the number of tasks being evaluated by benchmarks for resource-rich English/Chinese and the rest of the world. In this paper, we introduce LEPISZCZE (the Polish word for glew, the Middle English predecessor of glue), a new, comprehensive benchmark for Polish NLP with a large variety of tasks and high-quality operationalization of the benchmark. We design LEPISZCZE with flexibility in mind. Including new models, datasets, and tasks is as simple as possible while still offering data versioning and model tracking. In the first run of the benchmark, we test 13 experiments (task and dataset pairs) based on the five most recent LMs for Polish. We use five datasets from the Polish benchmark and add eight novel datasets. As the paper's main contribution, apart from LEPISZCZE, we provide insights and experiences learned while creating the benchmark for Polish as the blueprint to design similar benchmarks for other low-resourced languages. △ Less

Submitted 23 November, 2022; originally announced November 2022.

Comments: 10 pages, 8 pages appendix

Journal ref: Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (NeurIPS 2022) - https://lepiszcze.ml

WER we are and WER we think we are

Authors: Piotr Szymański, Piotr Żelasko, Mikolaj Morzy, Adrian Szymczak, Marzena Żyła-Hoppe, Joanna Banaszczak, Lukasz Augustyniak, Jan Mizgajski, Yishay Carmiel

Abstract: Natural language processing of conversational speech requires the availability of high-quality transcripts. In this paper, we express our skepticism towards the recent reports of very low Word Error Rates (WERs) achieved by modern Automatic Speech Recognition (ASR) systems on benchmark datasets. We outline several problems with popular benchmarks and compare three state-of-the-art commercial ASR s… ▽ More Natural language processing of conversational speech requires the availability of high-quality transcripts. In this paper, we express our skepticism towards the recent reports of very low Word Error Rates (WERs) achieved by modern Automatic Speech Recognition (ASR) systems on benchmark datasets. We outline several problems with popular benchmarks and compare three state-of-the-art commercial ASR systems on an internal dataset of real-life spontaneous human conversations and HUB'05 public benchmark. We show that WERs are significantly higher than the best reported results. We formulate a set of guidelines which may aid in the creation of real-life, multi-domain datasets with high quality annotations for training and testing of robust ASR systems. △ Less

Submitted 7 October, 2020; originally announced October 2020.

Comments: Accepted to EMNLP Findings

Punctuation Prediction in Spontaneous Conversations: Can We Mitigate ASR Errors with Retrofitted Word Embeddings?

Authors: Łukasz Augustyniak, Piotr Szymanski, Mikołaj Morzy, Piotr Zelasko, Adrian Szymczak, Jan Mizgajski, Yishay Carmiel, Najim Dehak

Abstract: Automatic Speech Recognition (ASR) systems introduce word errors, which often confuse punctuation prediction models, turning punctuation restoration into a challenging task. These errors usually take the form of homonyms. We show how retrofitting of the word embeddings on the domain-specific data can mitigate ASR errors. Our main contribution is a method for better alignment of homonym embeddings… ▽ More Automatic Speech Recognition (ASR) systems introduce word errors, which often confuse punctuation prediction models, turning punctuation restoration into a challenging task. These errors usually take the form of homonyms. We show how retrofitting of the word embeddings on the domain-specific data can mitigate ASR errors. Our main contribution is a method for better alignment of homonym embeddings and the validation of the presented method on the punctuation prediction task. We record the absolute improvement in punctuation prediction accuracy between 6.2% (for question marks) to 9% (for periods) when compared with the state-of-the-art model. △ Less

Submitted 13 April, 2020; originally announced April 2020.

Comments: submitted to INTERSPEECH'20

Avaya Conversational Intelligence: A Real-Time System for Spoken Language Understanding in Human-Human Call Center Conversations

Authors: Jan Mizgajski, Adrian Szymczak, Robert Głowski, Piotr Szymański, Piotr Żelasko, Łukasz Augustyniak, Mikołaj Morzy, Yishay Carmiel, Jeff Hodson, Łukasz Wójciak, Daniel Smoczyk, Adam Wróbel, Bartosz Borowik, Adam Artajew, Marcin Baran, Cezary Kwiatkowski, Marzena Żyła-Hoppe

Abstract: Avaya Conversational Intelligence(ACI) is an end-to-end, cloud-based solution for real-time Spoken Language Understanding for call centers. It combines large vocabulary, real-time speech recognition, transcript refinement, and entity and intent recognition in order to convert live audio into a rich, actionable stream of structured events. These events can be further leveraged with a business rules… ▽ More Avaya Conversational Intelligence(ACI) is an end-to-end, cloud-based solution for real-time Spoken Language Understanding for call centers. It combines large vocabulary, real-time speech recognition, transcript refinement, and entity and intent recognition in order to convert live audio into a rich, actionable stream of structured events. These events can be further leveraged with a business rules engine, thus serving as a foundation for real-time supervision and assistance applications. After the ingestion, calls are enriched with unsupervised keyword extraction, abstractive summarization, and business-defined attributes, enabling offline use cases, such as business intelligence, topic mining, full-text search, quality assurance, and agent training. ACI comes with a pretrained, configurable library of hundreds of intents and a robust intent training environment that allows for efficient, cost-effective creation and customization of customer-specific intents. △ Less

Submitted 2 September, 2019; originally announced September 2019.

Comments: Accepted for Interspeech 2019

Towards Better Understanding of Spontaneous Conversations: Overcoming Automatic Speech Recognition Errors With Intent Recognition

Authors: Piotr Żelasko, Jan Mizgajski, Mikołaj Morzy, Adrian Szymczak, Piotr Szymański, Łukasz Augustyniak, Yishay Carmiel

Abstract: In this paper, we present a method for correcting automatic speech recognition (ASR) errors using a finite state transducer (FST) intent recognition framework. Intent recognition is a powerful technique for dialog flow management in turn-oriented, human-machine dialogs. This technique can also be very useful in the context of human-human dialogs, though it serves a different purpose of key insight… ▽ More In this paper, we present a method for correcting automatic speech recognition (ASR) errors using a finite state transducer (FST) intent recognition framework. Intent recognition is a powerful technique for dialog flow management in turn-oriented, human-machine dialogs. This technique can also be very useful in the context of human-human dialogs, though it serves a different purpose of key insight extraction from conversations. We argue that currently available intent recognition techniques are not applicable to human-human dialogs due to the complex structure of turn-taking and various disfluencies encountered in spontaneous conversations, exacerbated by speech recognition errors and scarcity of domain-specific labeled data. Without efficient key insight extraction techniques, raw human-human dialog transcripts remain significantly unexploited. Our contribution consists of a novel FST for intent indexing and an algorithm for fuzzy intent search over the lattice - a compact graph encoding of ASR's hypotheses. We also develop a pruning strategy to constrain the fuzziness of the FST index search. Extracted intents represent linguistic domain knowledge and help us improve (rescore) the original transcript. We compare our method with a baseline, which uses only the most likely transcript hypothesis (best path), and find an increase in the total number of recognized intents by 25%. △ Less

Submitted 21 August, 2019; originally announced August 2019.

Graph Energies of Egocentric Networks and Their Correlation with Vertex Centrality Measures

Authors: Mikołaj Morzy, Tomasz Kajdanowicz

Abstract: Graph energy is the energy of the matrix representation of the graph, where the energy of a matrix is the sum of singular values of the matrix. Depending on the definition of a matrix, one can contemplate graph energy, Randić energy, Laplacian energy, distance energy, and many others. Although theoretical properties of various graph energies have been investigated in the past in the areas of mathe… ▽ More Graph energy is the energy of the matrix representation of the graph, where the energy of a matrix is the sum of singular values of the matrix. Depending on the definition of a matrix, one can contemplate graph energy, Randić energy, Laplacian energy, distance energy, and many others. Although theoretical properties of various graph energies have been investigated in the past in the areas of mathematics, chemistry, physics, or graph theory, these explorations have been limited to relatively small graphs representing chemical compounds or theoretical graph classes with strictly defined properties. In this paper we investigate the usefulness of the concept of graph energy in the context of large, complex networks. We show that when graph energies are applied to local egocentric networks, the values of these energies correlate strongly with vertex centrality measures. In particular, for some generative network models graph energies tend to correlate strongly with the betweenness and the eigencentrality of vertices. As the exact computation of these centrality measures is expensive and requires global processing of a network, our research opens the possibility of devising efficient algorithms for the estimation of these centrality measures based only on local information. △ Less

Submitted 12 November, 2018; v1 submitted 31 August, 2018; originally announced September 2018.

Journal ref: Entropy 2018, 20(12), 916

Priority Attachment: a Comprehensive Mechanism for Generating Networks

Authors: Mikołaj Morzy, Tomasz Kajdanowicz, Przemysław Kazienko, Grzegorz Miebs, Arkadiusz Rusin

Abstract: We claim that networks are created according to the priority attachment mechanism and we show a simple model which uses the priority attachment to generate both synthetic and close to empirical networks. Priority attachment is a mechanism which generalizes previously proposed mechanisms, such as small world creation or preferential attachment, but we also observe its presence in a range of real-wo… ▽ More We claim that networks are created according to the priority attachment mechanism and we show a simple model which uses the priority attachment to generate both synthetic and close to empirical networks. Priority attachment is a mechanism which generalizes previously proposed mechanisms, such as small world creation or preferential attachment, but we also observe its presence in a range of real-world networks. In this paper we show that by using priority attachment we can generate networks of very diverse topologies, as well as recreate empirical networks. An additional advantage of the priority attachment mechanism is an easy interpretation of the latent processes of network formation. We substantiate our claims by performing numerical experiments on synthetic and empirical networks. The two main contributions of the paper are: the introduction of the priority attachment mechanism, and the design of the Priority Rank: a simple network generative model based on the priority attachment mechanism. △ Less

Submitted 20 June, 2018; v1 submitted 10 January, 2018; originally announced January 2018.

Journal ref: Scientific Reportsvolume 9, Article number: 3383 (2019)