Showing 1–50 of 132 results for author: Fu, F

Chromosomal Structural Abnormality Diagnosis by Homologous Similarity

Authors: Juren Li, Fanzhe Fu, Ran Wei, Yifei Sun, Zeyu Lai, Ning Song, Xin Chen, Yang Yang

Abstract: Pathogenic chromosome abnormalities are very common among the general population. While numerical chromosome abnormalities can be quickly and precisely detected, structural chromosome abnormalities are far more complex and typically require considerable efforts by human experts for identification. This paper focuses on investigating the modeling of chromosome features and the identification of chr… ▽ More Pathogenic chromosome abnormalities are very common among the general population. While numerical chromosome abnormalities can be quickly and precisely detected, structural chromosome abnormalities are far more complex and typically require considerable efforts by human experts for identification. This paper focuses on investigating the modeling of chromosome features and the identification of chromosomes with structural abnormalities. Most existing data-driven methods concentrate on a single chromosome and consider each chromosome independently, overlooking the crucial aspect of homologous chromosomes. In normal cases, homologous chromosomes share identical structures, with the exception that one of them is abnormal. Therefore, we propose an adaptive method to align homologous chromosomes and diagnose structural abnormalities through homologous similarity. Inspired by the process of human expert diagnosis, we incorporate information from multiple pairs of homologous chromosomes simultaneously, aiming to reduce noise disturbance and improve prediction performance. Extensive experiments on real-world datasets validate the effectiveness of our model compared to baselines. △ Less

Submitted 11 July, 2024; originally announced July 2024.

Unbending strategies shepherd cooperation and suppress extortion in spatial populations

Authors: Zijie Chen, Yuxin Geng, Xingru Chen, Feng Fu

Abstract: Evolutionary game dynamics on networks typically consider the competition among simple strategies such as cooperation and defection in the Prisoner's Dilemma and summarize the effect of population structure as network reciprocity. However, it remains largely unknown regarding the evolutionary dynamics involving multiple powerful strategies typically considered in repeated games, such as the zero-d… ▽ More Evolutionary game dynamics on networks typically consider the competition among simple strategies such as cooperation and defection in the Prisoner's Dilemma and summarize the effect of population structure as network reciprocity. However, it remains largely unknown regarding the evolutionary dynamics involving multiple powerful strategies typically considered in repeated games, such as the zero-determinant (ZD) strategies that are able to enforce a linear payoff relationship between them and their co-players. Here, we consider the evolutionary dynamics of always cooperate (AllC), extortionate ZD (extortioners), and unbending players in lattice populations based on the commonly used death-birth updating. Out of the class of unbending strategies, we consider a particular candidate, PSO Gambler, a machine-learning-optimized memory-one strategy, which can foster reciprocal cooperation and fairness among extortionate players. We derive analytical results under weak selection and rare mutations, including pairwise fixation probabilities and long-term frequencies of strategies. In the absence of the third unbending type, extortioners can achieve a half-half split in equilibrium with unconditional cooperators for sufficiently large extortion factors. However, the presence of unbending players fundamentally changes the dynamics and tilts the system to favor unbending cooperation. Most surprisingly, extortioners cannot dominate at all regardless of how large their extortion factor is, and the long-term frequency of unbending players is maintained almost as a constant. Our analytical method is applicable to studying the evolutionary dynamics of multiple strategies in structured populations. Our work provides insights into the interplay between network reciprocity and direct reciprocity, revealing the role of unbending strategies in enforcing fairness and suppressing extortion. △ Less

Submitted 29 May, 2024; originally announced May 2024.

Comments: 21 pages, 6 figures

Evolutionary game dynamics with environmental feedback in a network with two communities

Authors: Katherine Betz, Feng Fu, Naoki Masuda

Abstract: Recent developments of eco-evolutionary models have shown that evolving feedbacks between behavioral strategies and the environment of game interactions, leading to changes in the underlying payoff matrix, can impact the underlying population dynamics in various manners. We propose and analyze an eco-evolutionary game dynamics model on a network with two communities such that players interact with… ▽ More Recent developments of eco-evolutionary models have shown that evolving feedbacks between behavioral strategies and the environment of game interactions, leading to changes in the underlying payoff matrix, can impact the underlying population dynamics in various manners. We propose and analyze an eco-evolutionary game dynamics model on a network with two communities such that players interact with other players in the same community and those in the opposite community at different rates. In our model, we consider two-person matrix games with pairwise interactions occurring on individual edges and assume that the environmental state depends on edges rather than on nodes or being globally shared in the population. We analytically determine the equilibria and their stability under a symmetric population structure assumption, and we also numerically study the replicator dynamics of the general model. The model shows rich dynamical behavior, such as multiple transcritical bifurcations, multistability, and anti-synchronous oscillations. Our work offers insights into understanding how the presence of community structure impacts the eco-evolutionary dynamics within and between niches. △ Less

Submitted 7 June, 2024; v1 submitted 25 April, 2024; originally announced April 2024.

Comments: 8 figures, 2 tables

Journal ref: Bull Math Biol 86, 84 (2024)

Sentiment-oriented Transformer-based Variational Autoencoder Network for Live Video Commenting

Authors: Fengyi Fu, Shancheng Fang, Weidong Chen, Zhendong Mao

Abstract: Automatic live video commenting is with increasing attention due to its significance in narration generation, topic explanation, etc. However, the diverse sentiment consideration of the generated comments is missing from the current methods. Sentimental factors are critical in interactive commenting, and lack of research so far. Thus, in this paper, we propose a Sentiment-oriented Transformer-base… ▽ More Automatic live video commenting is with increasing attention due to its significance in narration generation, topic explanation, etc. However, the diverse sentiment consideration of the generated comments is missing from the current methods. Sentimental factors are critical in interactive commenting, and lack of research so far. Thus, in this paper, we propose a Sentiment-oriented Transformer-based Variational Autoencoder (So-TVAE) network which consists of a sentiment-oriented diversity encoder module and a batch attention module, to achieve diverse video commenting with multiple sentiments and multiple semantics. Specifically, our sentiment-oriented diversity encoder elegantly combines VAE and random mask mechanism to achieve semantic diversity under sentiment guidance, which is then fused with cross-modal features to generate live video comments. Furthermore, a batch attention module is also proposed in this paper to alleviate the problem of missing sentimental samples, caused by the data imbalance, which is common in live videos as the popularity of videos varies. Extensive experiments on Livebot and VideoIC datasets demonstrate that the proposed So-TVAE outperforms the state-of-the-art methods in terms of the quality and diversity of generated comments. Related code is available at https://github.com/fufy1024/So-TVAE. △ Less

Submitted 19 April, 2024; originally announced April 2024.

Comments: 27 pages, 10 figures, ACM Transactions on Multimedia Computing, Communications and Applications, 2024

Association schemes arising from non-weakly regular bent functions

Authors: Yadi Wei, Jiaxin Wang, Fang-Wei Fu

Abstract: Association schemes play an important role in algebraic combinatorics and have important applications in coding theory, graph theory and design theory. The methods to construct association schemes by using bent functions have been extensively studied. Recently, in [13], {Ö}zbudak and Pelen constructed infinite families of symmetric association schemes of classes $5$ and $6$ by using ternary non-we… ▽ More Association schemes play an important role in algebraic combinatorics and have important applications in coding theory, graph theory and design theory. The methods to construct association schemes by using bent functions have been extensively studied. Recently, in [13], {Ö}zbudak and Pelen constructed infinite families of symmetric association schemes of classes $5$ and $6$ by using ternary non-weakly regular bent functions.They also stated that constructing $2p$-class association schemes from $p$-ary non-weakly regular bent functions is an interesting problem, where $p>3$ is an odd prime. In this paper, using non-weakly regular bent functions, we construct infinite families of symmetric association schemes of classes $2p$, $(2p+1)$ and $\frac{3p+1}{2}$ for any odd prime $p$. Fusing those association schemes, we also obtain $t$-class symmetric association schemes, where $t=4,5,6,7$. In addition, we give the sufficient and necessary conditions for the partitions $P$, $D$, $T$, $U$ and $V$ (defined in this paper) to induce symmetric association schemes. △ Less

Submitted 13 June, 2024; v1 submitted 8 April, 2024; originally announced April 2024.

Self-Orthogonal Codes from Vectorial Dual-Bent Functions

Authors: Jiaxin Wang, Yadi Wei, Fang-Wei Fu, Juan Li

Abstract: Self-orthogonal codes are a significant class of linear codes in coding theory and have attracted a lot of attention. In \cite{HLL2023Te,LH2023Se}, $p$-ary self-orthogonal codes were constructed by using $p$-ary weakly regular bent functions, where $p$ is an odd prime. In \cite{WH2023Se}, two classes of non-degenerate quadratic forms were used to construct $q$-ary self-orthogonal codes, where $q$… ▽ More Self-orthogonal codes are a significant class of linear codes in coding theory and have attracted a lot of attention. In \cite{HLL2023Te,LH2023Se}, $p$-ary self-orthogonal codes were constructed by using $p$-ary weakly regular bent functions, where $p$ is an odd prime. In \cite{WH2023Se}, two classes of non-degenerate quadratic forms were used to construct $q$-ary self-orthogonal codes, where $q$ is a power of a prime. In this paper, we construct new families of $q$-ary self-orthogonal codes using vectorial dual-bent functions. Some classes of at least almost optimal linear codes are obtained from the dual codes of the constructed self-orthogonal codes. In some cases, we completely determine the weight distributions of the constructed self-orthogonal codes. From the view of vectorial dual-bent functions, we illustrate that the works on constructing self-orthogonal codes from $p$-ary weakly regular bent functions \cite{HLL2023Te,LH2023Se} and non-degenerate quadratic forms with $q$ being odd \cite{WH2023Se} can be obtained by our results. We partially answer an open problem on determining the weight distribution of a class of self-orthogonal codes given in \cite{LH2023Se}. As applications, we construct new infinite families of at least almost optimal $q$-ary linear complementary dual codes (for short, LCD codes) and quantum codes. △ Less

Submitted 19 March, 2024; originally announced March 2024.

Mathematics of multi-agent learning systems at the interface of game theory and artificial intelligence

Authors: Long Wang, Feng Fu, Xingru Chen

Abstract: Evolutionary Game Theory (EGT) and Artificial Intelligence (AI) are two fields that, at first glance, might seem distinct, but they have notable connections and intersections. The former focuses on the evolution of behaviors (or strategies) in a population, where individuals interact with others and update their strategies based on imitation (or social learning). The more successful a strategy is,… ▽ More Evolutionary Game Theory (EGT) and Artificial Intelligence (AI) are two fields that, at first glance, might seem distinct, but they have notable connections and intersections. The former focuses on the evolution of behaviors (or strategies) in a population, where individuals interact with others and update their strategies based on imitation (or social learning). The more successful a strategy is, the more prevalent it becomes over time. The latter, meanwhile, is centered on machine learning algorithms and (deep) neural networks. It is often from a single-agent perspective but increasingly involves multi-agent environments, in which intelligent agents adjust their strategies based on feedback and experience, somewhat akin to the evolutionary process yet distinct in their self-learning capacities. In light of the key components necessary to address real-world problems, including (i) learning and adaptation, (ii) cooperation and competition, (iii) robustness and stability, and altogether (iv) population dynamics of individual agents whose strategies evolve, the cross-fertilization of ideas between both fields will contribute to the advancement of mathematics of multi-agent learning systems, in particular, to the nascent domain of ``collective cooperative intelligence'' bridging evolutionary dynamics and multi-agent reinforcement learning. △ Less

Submitted 9 March, 2024; originally announced March 2024.

Comments: 8 pages, 1 figure

SiLVR: Scalable Lidar-Visual Reconstruction with Neural Radiance Fields for Robotic Inspection

Authors: Yifu Tao, Yash Bhalgat, Lanke Frank Tarimo Fu, Matias Mattamala, Nived Chebrolu, Maurice Fallon

Abstract: We present a neural-field-based large-scale reconstruction system that fuses lidar and vision data to generate high-quality reconstructions that are geometrically accurate and capture photo-realistic textures. This system adapts the state-of-the-art neural radiance field (NeRF) representation to also incorporate lidar data which adds strong geometric constraints on the depth and surface normals. W… ▽ More We present a neural-field-based large-scale reconstruction system that fuses lidar and vision data to generate high-quality reconstructions that are geometrically accurate and capture photo-realistic textures. This system adapts the state-of-the-art neural radiance field (NeRF) representation to also incorporate lidar data which adds strong geometric constraints on the depth and surface normals. We exploit the trajectory from a real-time lidar SLAM system to bootstrap a Structure-from-Motion (SfM) procedure to both significantly reduce the computation time and to provide metric scale which is crucial for lidar depth loss. We use submapping to scale the system to large-scale environments captured over long trajectories. We demonstrate the reconstruction system with data from a multi-camera, lidar sensor suite onboard a legged robot, hand-held while scanning building scenes for 600 metres, and onboard an aerial robot surveying a multi-storey mock disaster site-building. Website: https://ori-drs.github.io/projects/silvr/ △ Less

Submitted 11 March, 2024; originally announced March 2024.

Comments: Accepted at ICRA 2024; Website: https://ori-drs.github.io/projects/silvr/

Reinforcement Learning Jazz Improvisation: When Music Meets Game Theory

Authors: Vedant Tapiavala, Joshua Piesner, Sourjyamoy Barman, Feng Fu

Abstract: Live performances of music are always charming, with the unpredictability of improvisation due to the dynamic between musicians and interactions with the audience. Jazz improvisation is a particularly noteworthy example for further investigation from a theoretical perspective. Here, we introduce a novel mathematical game theory model for jazz improvisation, providing a framework for studying music… ▽ More Live performances of music are always charming, with the unpredictability of improvisation due to the dynamic between musicians and interactions with the audience. Jazz improvisation is a particularly noteworthy example for further investigation from a theoretical perspective. Here, we introduce a novel mathematical game theory model for jazz improvisation, providing a framework for studying music theory and improvisational methodologies. We use computational modeling, mainly reinforcement learning, to explore diverse stochastic improvisational strategies and their paired performance on improvisation. We find that the most effective strategy pair is a strategy that reacts to the most recent payoff (Stepwise Changes) with a reinforcement learning strategy limited to notes in the given chord (Chord-Following Reinforcement Learning). Conversely, a strategy that reacts to the partner's last note and attempts to harmonize with it (Harmony Prediction) strategy pair yields the lowest non-control payoff and highest standard deviation, indicating that picking notes based on immediate reactions to the partner player can yield inconsistent outcomes. On average, the Chord-Following Reinforcement Learning strategy demonstrates the highest mean payoff, while Harmony Prediction exhibits the lowest. Our work lays the foundation for promising applications beyond jazz: including the use of artificial intelligence (AI) models to extract data from audio clips to refine musical reward systems, and training machine learning (ML) models on existing jazz solos to further refine strategies within the game. △ Less

Submitted 25 February, 2024; originally announced March 2024.

Comments: 16 pages, 4 figures

SCott: Accelerating Diffusion Models with Stochastic Consistency Distillation

Authors: Hongjian Liu, Qingsong Xie, Zhijie Deng, Chen Chen, Shixiang Tang, Fueyang Fu, Zheng-jun Zha, Haonan Lu

Abstract: The iterative sampling procedure employed by diffusion models (DMs) often leads to significant inference latency. To address this, we propose Stochastic Consistency Distillation (SCott) to enable accelerated text-to-image generation, where high-quality generations can be achieved with just 1-2 sampling steps, and further improvements can be obtained by adding additional steps. In contrast to vanil… ▽ More The iterative sampling procedure employed by diffusion models (DMs) often leads to significant inference latency. To address this, we propose Stochastic Consistency Distillation (SCott) to enable accelerated text-to-image generation, where high-quality generations can be achieved with just 1-2 sampling steps, and further improvements can be obtained by adding additional steps. In contrast to vanilla consistency distillation (CD) which distills the ordinary differential equation solvers-based sampling process of a pretrained teacher model into a student, SCott explores the possibility and validates the efficacy of integrating stochastic differential equation (SDE) solvers into CD to fully unleash the potential of the teacher. SCott is augmented with elaborate strategies to control the noise strength and sampling process of the SDE solver. An adversarial loss is further incorporated to strengthen the sample quality with rare sampling steps. Empirically, on the MSCOCO-2017 5K dataset with a Stable Diffusion-V1.5 teacher, SCott achieves an FID (Frechet Inceptio Distance) of 22.1, surpassing that (23.4) of the 1-step InstaFlow (Liu et al., 2023) and matching that of 4-step UFOGen (Xue et al., 2023b). Moreover, SCott can yield more diverse samples than other consistency models for high-resolution image generation (Luo et al., 2023a), with up to 16% improvement in a qualified metric. The code and checkpoints are coming soon. △ Less

Submitted 15 April, 2024; v1 submitted 3 March, 2024; originally announced March 2024.

Comments: 22 pages, 16 figures

Retrieval-Augmented Generation for AI-Generated Content: A Survey

Authors: Penghao Zhao, Hailin Zhang, Qinhan Yu, Zhengren Wang, Yunteng Geng, Fangcheng Fu, Ling Yang, Wentao Zhang, Jie Jiang, Bin Cui

Abstract: Advancements in model algorithms, the growth of foundational models, and access to high-quality datasets have propelled the evolution of Artificial Intelligence Generated Content (AIGC). Despite its notable successes, AIGC still faces hurdles such as updating knowledge, handling long-tail data, mitigating data leakage, and managing high training and inference costs. Retrieval-Augmented Generation… ▽ More Advancements in model algorithms, the growth of foundational models, and access to high-quality datasets have propelled the evolution of Artificial Intelligence Generated Content (AIGC). Despite its notable successes, AIGC still faces hurdles such as updating knowledge, handling long-tail data, mitigating data leakage, and managing high training and inference costs. Retrieval-Augmented Generation (RAG) has recently emerged as a paradigm to address such challenges. In particular, RAG introduces the information retrieval process, which enhances the generation process by retrieving relevant objects from available data stores, leading to higher accuracy and better robustness. In this paper, we comprehensively review existing efforts that integrate RAG technique into AIGC scenarios. We first classify RAG foundations according to how the retriever augments the generator, distilling the fundamental abstractions of the augmentation methodologies for various retrievers and generators. This unified perspective encompasses all RAG scenarios, illuminating advancements and pivotal technologies that help with potential future progress. We also summarize additional enhancements methods for RAG, facilitating effective engineering and implementation of RAG systems. Then from another view, we survey on practical applications of RAG across different modalities and tasks, offering valuable references for researchers and practitioners. Furthermore, we introduce the benchmarks for RAG, discuss the limitations of current RAG systems, and suggest potential directions for future research. Github: https://github.com/PKU-DAIR/RAG-Survey. △ Less

Submitted 21 June, 2024; v1 submitted 29 February, 2024; originally announced February 2024.

Comments: Citing 353 papers, 22 pages, 1 table, 12 figures. Project: https://github.com/PKU-DAIR/RAG-Survey

Are Synthetic Time-series Data Really not as Good as Real Data?

Authors: Fanzhe Fu, Junru Chen, Jing Zhang, Carl Yang, Lvbin Ma, Yang Yang

Abstract: Time-series data presents limitations stemming from data quality issues, bias and vulnerabilities, and generalization problem. Integrating universal data synthesis methods holds promise in improving generalization. However, current methods cannot guarantee that the generator's output covers all unseen real data. In this paper, we introduce InfoBoost -- a highly versatile cross-domain data synthesi… ▽ More Time-series data presents limitations stemming from data quality issues, bias and vulnerabilities, and generalization problem. Integrating universal data synthesis methods holds promise in improving generalization. However, current methods cannot guarantee that the generator's output covers all unseen real data. In this paper, we introduce InfoBoost -- a highly versatile cross-domain data synthesizing framework with time series representation learning capability. We have developed a method based on synthetic data that enables model training without the need for real data, surpassing the performance of models trained with real data. Additionally, we have trained a universal feature extractor based on our synthetic data that is applicable to all time-series data. Our approach overcomes interference from multiple sources rhythmic signal, noise interference, and long-period features that exceed sampling window capabilities. Through experiments, our non-deep-learning synthetic data enables models to achieve superior reconstruction performance and universal explicit representation extraction without the need for real data. △ Less

Submitted 1 February, 2024; originally announced February 2024.

Towards Context-Stable and Visual-Consistent Image Inpainting

Authors: Yikai Wang, Chenjie Cao, Ke Fan Xiangyang Xue Yanwei Fu

Abstract: Recent progress in inpainting increasingly relies on generative models, leveraging their strong generation capabilities for addressing large irregular masks. However, this enhanced generation often introduces context-instability, leading to arbitrary object generation within masked regions. This paper proposes a balanced solution, emphasizing the importance of unmasked regions in guiding inpaintin… ▽ More Recent progress in inpainting increasingly relies on generative models, leveraging their strong generation capabilities for addressing large irregular masks. However, this enhanced generation often introduces context-instability, leading to arbitrary object generation within masked regions. This paper proposes a balanced solution, emphasizing the importance of unmasked regions in guiding inpainting while preserving generation capacity. Our approach, Aligned Stable Inpainting with UnKnown Areas Prior (ASUKA), employs a Masked Auto-Encoder (MAE) to produce reconstruction-based prior. Aligned with the powerful Stable Diffusion inpainting model (SD), ASUKA significantly improves context stability. ASUKA further adopts an inpainting-specialized decoder, highly reducing the color inconsistency issue of SD and thus ensuring more visual-consistent inpainting. We validate effectiveness of inpainting algorithms on benchmark dataset Places 2 and a collection of several existing datasets, dubbed MISATO, across diverse domains and masking scenarios. Results on these benchmark datasets confirm ASUKA's efficacy in both context-stability and visual-consistency compared to SD and other inpainting algorithms. △ Less

Submitted 17 March, 2024; v1 submitted 8 December, 2023; originally announced December 2023.

Comments: Project page: https://yikai-wang.github.io/asuka/ where full-size PDF with appendix is available. Dataset: https://github.com/Yikai-Wang/asuka-misato. Yikai Wang and Chenjie Cao contribute equally

Learning with Errors over Group Rings Constructed by Semi-direct Product

Authors: Jiaqi Liu, Fang-Wei Fu

Abstract: The Learning with Errors (LWE) problem has been widely utilized as a foundation for numerous cryptographic tools over the years. In this study, we focus on an algebraic variant of the LWE problem called Group ring LWE (GR-LWE). We select group rings (or their direct summands) that underlie specific families of finite groups constructed by taking the semi-direct product of two cyclic groups. Unlike… ▽ More The Learning with Errors (LWE) problem has been widely utilized as a foundation for numerous cryptographic tools over the years. In this study, we focus on an algebraic variant of the LWE problem called Group ring LWE (GR-LWE). We select group rings (or their direct summands) that underlie specific families of finite groups constructed by taking the semi-direct product of two cyclic groups. Unlike the Ring-LWE problem described in \cite{lyubashevsky2010ideal}, the multiplication operation in the group rings considered here is non-commutative. As an extension of Ring-LWE, it maintains computational hardness and can be potentially applied in many cryptographic scenarios. In this paper, we present two polynomial-time quantum reductions. Firstly, we provide a quantum reduction from the worst-case shortest independent vectors problem (SIVP) in ideal lattices with polynomial approximate factor to the search version of GR-LWE. This reduction requires that the underlying group ring possesses certain mild properties; Secondly, we present another quantum reduction for two types of group rings, where the worst-case SIVP problem is directly reduced to the (average-case) decision GR-LWE problem. The pseudorandomness of GR-LWE samples guaranteed by this reduction can be consequently leveraged to construct semantically secure public-key cryptosystems. △ Less

Submitted 1 December, 2023; v1 submitted 27 November, 2023; originally announced November 2023.

Comments: 45 pages

E-CORE: Emotion Correlation Enhanced Empathetic Dialogue Generation

Authors: Fengyi Fu, Lei Zhang, Quan Wang, Zhendong Mao

Abstract: Achieving empathy is a crucial step toward humanized dialogue systems. Current approaches for empathetic dialogue generation mainly perceive an emotional label to generate an empathetic response conditioned on it, which simply treat emotions independently, but ignore the intrinsic emotion correlation in dialogues, resulting in inaccurate emotion perception and unsuitable response generation. In th… ▽ More Achieving empathy is a crucial step toward humanized dialogue systems. Current approaches for empathetic dialogue generation mainly perceive an emotional label to generate an empathetic response conditioned on it, which simply treat emotions independently, but ignore the intrinsic emotion correlation in dialogues, resulting in inaccurate emotion perception and unsuitable response generation. In this paper, we propose a novel emotion correlation enhanced empathetic dialogue generation framework, which comprehensively realizes emotion correlation learning, utilization, and supervising. Specifically, a multi-resolution emotion graph is devised to capture context-based emotion interactions from different resolutions, further modeling emotion correlation. Then we propose an emotion correlation enhanced decoder, with a novel correlation-aware aggregation and soft/hard strategy, respectively improving the emotion perception and response generation. Experimental results on the benchmark dataset demonstrate the superiority of our model in both empathetic perception and expression. △ Less

Submitted 25 November, 2023; originally announced November 2023.

Comments: 19 pages, 6 figures

ACM Class: I.2.7

Journal ref: Conference on Empirical Methods in Natural Language Processing, EMNLP 2023

Linear codes with few weights from non-weakly regular plateaued functions

Authors: Yadi Wei, Jiaxin Wang, Fang-Wei Fu

Abstract: Linear codes with few weights have significant applications in secret sharing schemes, authentication codes, association schemes, and strongly regular graphs. There are a number of methods to construct linear codes, one of which is based on functions. Furthermore, two generic constructions of linear codes from functions called the first and the second generic constructions, have aroused the resear… ▽ More Linear codes with few weights have significant applications in secret sharing schemes, authentication codes, association schemes, and strongly regular graphs. There are a number of methods to construct linear codes, one of which is based on functions. Furthermore, two generic constructions of linear codes from functions called the first and the second generic constructions, have aroused the research interest of scholars. Recently, in \cite{Nian}, Li and Mesnager proposed two open problems: Based on the first and the second generic constructions, respectively, construct linear codes from non-weakly regular plateaued functions and determine their weight distributions. Motivated by these two open problems, in this paper, firstly, based on the first generic construction, we construct some three-weight and at most five-weight linear codes from non-weakly regular plateaued functions and determine the weight distributions of the constructed codes. Next, based on the second generic construction, we construct some three-weight and at most five-weight linear codes from non-weakly regular plateaued functions belonging to $\mathcal{NWRF}$ (defined in this paper) and determine the weight distributions of the constructed codes. We also give the punctured codes of these codes obtained based on the second generic construction and determine their weight distributions. Meanwhile, we obtain some optimal and almost optimal linear codes. Besides, by the Ashikhmin-Barg condition, we have that the constructed codes are minimal for almost all cases and obtain some secret sharing schemes with nice access structures based on their dual codes. △ Less

Submitted 30 October, 2023; originally announced October 2023.

Comments: 52 pages, 34 tables

Generative and Contrastive Paradigms Are Complementary for Graph Self-Supervised Learning

Authors: Yuxiang Wang, Xiao Yan, Chuang Hu, Fangcheng Fu, Wentao Zhang, Hao Wang, Shuo Shang, Jiawei Jiang

Abstract: For graph self-supervised learning (GSSL), masked autoencoder (MAE) follows the generative paradigm and learns to reconstruct masked graph edges or node features. Contrastive Learning (CL) maximizes the similarity between augmented views of the same graph and is widely used for GSSL. However, MAE and CL are considered separately in existing works for GSSL. We observe that the MAE and CL paradigms… ▽ More For graph self-supervised learning (GSSL), masked autoencoder (MAE) follows the generative paradigm and learns to reconstruct masked graph edges or node features. Contrastive Learning (CL) maximizes the similarity between augmented views of the same graph and is widely used for GSSL. However, MAE and CL are considered separately in existing works for GSSL. We observe that the MAE and CL paradigms are complementary and propose the graph contrastive masked autoencoder (GCMAE) framework to unify them. Specifically, by focusing on local edges or node features, MAE cannot capture global information of the graph and is sensitive to particular edges and features. On the contrary, CL excels in extracting global information because it considers the relation between graphs. As such, we equip GCMAE with an MAE branch and a CL branch, and the two branches share a common encoder, which allows the MAE branch to exploit the global information extracted by the CL branch. To force GCMAE to capture global graph structures, we train it to reconstruct the entire adjacency matrix instead of only the masked edges as in existing works. Moreover, a discrimination loss is proposed for feature reconstruction, which improves the disparity between node embeddings rather than reducing the reconstruction error to tackle the feature smoothing problem of MAE. We evaluate GCMAE on four popular graph tasks (i.e., node classification, node clustering, link prediction, and graph classification) and compare with 14 state-of-the-art baselines. The results show that GCMAE consistently provides good accuracy across these tasks, and the maximum accuracy improvement is up to 3.2% compared with the best-performing baseline. △ Less

Submitted 24 October, 2023; originally announced October 2023.

New Lower Bounds for the Minimum Distance of Cyclic Codes and Applications to Locally Repairable Codes

Authors: Jing Qiu, Weijun Fang, Fang-Wei Fu

Abstract: Cyclic codes are an important class of linear codes. Bounding the minimum distance of cyclic codes is a long-standing research topic in coding theory, and several well-known and basic results have been developed on this topic. Recently, locally repairable codes (LRCs) have attracted much attention due to their repair efficiency in large-scale distributed storage systems. In this paper, by employin… ▽ More Cyclic codes are an important class of linear codes. Bounding the minimum distance of cyclic codes is a long-standing research topic in coding theory, and several well-known and basic results have been developed on this topic. Recently, locally repairable codes (LRCs) have attracted much attention due to their repair efficiency in large-scale distributed storage systems. In this paper, by employing the singleton procedure technique, we first provide a sufficient condition for bounding the minimum distance of cyclic codes with typical defining sets. Secondly, by considering a specific case, we establish a connection between bounds for the minimum distance of cyclic codes and solutions to a system of inequalities. This connection leads to the derivation of new bounds, including some with general patterns. In particular, we provide three new bounds with general patterns, one of which serves as a generalization of the Betti-Sala bound. Finally, we present a generalized lower bound for a special case and construct several families of $(2, δ)$-LRCs with unbounded length and minimum distance $2δ$. It turns out that these LRCs are distance-optimal, and their parameters are new. To the best of our knowledge, this work represents the first construction of distance-optimal $(r, δ)$-LRCs with unbounded length and minimum distance exceeding $r+δ-1$. △ Less

Submitted 11 October, 2023; originally announced October 2023.

Comments: 35 pages

A Further Study of Vectorial Dual-Bent Functions

Authors: Jiaxin Wang, Fang-Wei Fu, Yadi Wei, Jing Yang

Abstract: Vectorial dual-bent functions have recently attracted some researchers' interest as they play a significant role in constructing partial difference sets, association schemes, bent partitions and linear codes. In this paper, we further study vectorial dual-bent functions $F: V_{n}^{(p)}\rightarrow V_{m}^{(p)}$, where $2\leq m \leq \frac{n}{2}$, $V_{n}^{(p)}$ denotes an $n$-dimensional vector space… ▽ More Vectorial dual-bent functions have recently attracted some researchers' interest as they play a significant role in constructing partial difference sets, association schemes, bent partitions and linear codes. In this paper, we further study vectorial dual-bent functions $F: V_{n}^{(p)}\rightarrow V_{m}^{(p)}$, where $2\leq m \leq \frac{n}{2}$, $V_{n}^{(p)}$ denotes an $n$-dimensional vector space over the prime field $\mathbb{F}_{p}$. We give new characterizations of certain vectorial dual-bent functions (called vectorial dual-bent functions with Condition A) in terms of amorphic association schemes, linear codes and generalized Hadamard matrices, respectively. When $p=2$, we characterize vectorial dual-bent functions with Condition A in terms of bent partitions. Furthermore, we characterize certain bent partitions in terms of amorphic association schemes, linear codes and generalized Hadamard matrices, respectively. For general vectorial dual-bent functions $F: V_{n}^{(p)}\rightarrow V_{m}^{(p)}$ with $F(0)=0, F(x)=F(-x)$ and $2\leq m \leq \frac{n}{2}$, we give a necessary and sufficient condition on constructing association schemes. Based on such a result, more association schemes are constructed from vectorial dual-bent functions. △ Less

Submitted 23 September, 2023; originally announced September 2023.

Some new constructions of optimal linear codes and alphabet-optimal $(r,δ)$-locally repairable codes

Authors: Jing Qiu, Fang-Wei Fu

Abstract: In distributed storage systems, locally repairable codes (LRCs) are designed to reduce disk I/O and repair costs by enabling recovery of each code symbol from a small number of other symbols. To handle multiple node failures, $(r,δ)$-LRCs are introduced to enable local recovery in the event of up to $δ-1$ failed nodes. Constructing optimal $(r,δ)$-LRCs has been a significant research topic over th… ▽ More In distributed storage systems, locally repairable codes (LRCs) are designed to reduce disk I/O and repair costs by enabling recovery of each code symbol from a small number of other symbols. To handle multiple node failures, $(r,δ)$-LRCs are introduced to enable local recovery in the event of up to $δ-1$ failed nodes. Constructing optimal $(r,δ)$-LRCs has been a significant research topic over the past decade. In \cite{Luo2022}, Luo \emph{et al.} proposed a construction of linear codes by using unions of some projective subspaces within a projective space. Several new classes of Griesmer codes and distance-optimal codes were constructed, and some of them were proved to be alphabet-optimal $2$-LRCs. In this paper, we first modify the method of constructing linear codes in \cite{Luo2022} by considering a more general situation of intersecting projective subspaces. This modification enables us to construct good codes with more flexible parameters. Additionally, we present the conditions for the constructed linear codes to qualify as Griesmer codes or achieve distance optimality. Next, we explore the locality of linear codes constructed by eliminating elements from a complete projective space. The novelty of our work lies in establishing the locality as $(2,p-2)$, $(2,p-1)$, or $(2,p)$-locality, in contrast to the previous literature that only considered $2$-locality. Moreover, by combining analysis of code parameters and the C-M like bound for $(r,δ)$-LRCs, we construct some alphabet-optimal $(2,δ)$-LRCs which may be either Griesmer codes or not Griesmer codes. Finally, we investigate the availability and alphabet-optimality of $(r,δ)$-LRCs constructed from our modified framework. △ Less

Submitted 9 July, 2023; originally announced July 2023.

Comments: 25 pages

Improving Automatic Parallel Training via Balanced Memory Workload Optimization

Authors: Yujie Wang, Youhe Jiang, Xupeng Miao, Fangcheng Fu, Shenhan Zhu, Xiaonan Nie, Yaofeng Tu, Bin Cui

Abstract: Transformer models have emerged as the leading approach for achieving state-of-the-art performance across various application domains, serving as the foundation for advanced large-scale deep learning (DL) models. However, efficiently training these models across multiple GPUs remains a complex challenge due to the abundance of parallelism options. Existing DL systems either require manual efforts… ▽ More Transformer models have emerged as the leading approach for achieving state-of-the-art performance across various application domains, serving as the foundation for advanced large-scale deep learning (DL) models. However, efficiently training these models across multiple GPUs remains a complex challenge due to the abundance of parallelism options. Existing DL systems either require manual efforts to design distributed training plans or limit parallelism combinations to a constrained search space. In this paper, we present Galvatron-BMW, a novel system framework that integrates multiple prevalent parallelism dimensions and automatically identifies the most efficient hybrid parallelism strategy. To effectively navigate this vast search space, we employ a decision tree approach for decomposition and pruning based on intuitive insights. We further utilize a dynamic programming search algorithm to derive the optimal plan. Moreover, to improve resource utilization and enhance system efficiency, we propose a bi-objective optimization workflow that focuses on workload balance. Our evaluations on different Transformer models demonstrate the capabilities of Galvatron-BMW in automating distributed training under varying GPU memory constraints. Across all tested scenarios, Galvatron-BMW consistently achieves superior system throughput, surpassing previous approaches that rely on limited parallelism strategies. △ Less

Submitted 24 February, 2024; v1 submitted 5 July, 2023; originally announced July 2023.

Comments: arXiv admin note: substantial text overlap with arXiv:2211.13878

OVLA: Neural Network Ownership Verification using Latent Watermarks

Authors: Feisi Fu, Wenchao Li

Abstract: Ownership verification for neural networks is important for protecting these models from illegal copying, free-riding, re-distribution and other intellectual property misuse. We present a novel methodology for neural network ownership verification based on the notion of latent watermarks. Existing ownership verification methods either modify or introduce constraints to the neural network parameter… ▽ More Ownership verification for neural networks is important for protecting these models from illegal copying, free-riding, re-distribution and other intellectual property misuse. We present a novel methodology for neural network ownership verification based on the notion of latent watermarks. Existing ownership verification methods either modify or introduce constraints to the neural network parameters, which are accessible to an attacker in a white-box attack and can be harmful to the network's normal operation, or train the network to respond to specific watermarks in the inputs similar to data poisoning-based backdoor attacks, which are susceptible to backdoor removal techniques. In this paper, we address these problems by decoupling a network's normal operation from its responses to watermarked inputs during ownership verification. The key idea is to train the network such that the watermarks remain dormant unless the owner's secret key is applied to activate it. The secret key is realized as a specific perturbation only known to the owner to the network's parameters. We show that our approach offers strong defense against backdoor detection, backdoor removal and surrogate model attacks.In addition, our method provides protection against ambiguity attacks where the attacker either tries to guess the secret weight key or uses fine-tuning to embed their own watermarks with a different key into a pre-trained neural network. Experimental results demonstrate the advantages and effectiveness of our proposed approach. △ Less

Submitted 25 June, 2023; v1 submitted 15 June, 2023; originally announced June 2023.

Steering control of payoff-maximizing players in adaptive learning dynamics

Authors: Xingru Chen, Feng Fu

Abstract: Evolutionary game theory provides a mathematical foundation for cross-disciplinary fertilization, especially for integrating ideas from artificial intelligence and game theory. Such integration offers a transparent and rigorous approach to complex decision-making problems in a variety of important contexts, ranging from evolutionary computation to machine behavior. Despite the astronomically huge… ▽ More Evolutionary game theory provides a mathematical foundation for cross-disciplinary fertilization, especially for integrating ideas from artificial intelligence and game theory. Such integration offers a transparent and rigorous approach to complex decision-making problems in a variety of important contexts, ranging from evolutionary computation to machine behavior. Despite the astronomically huge individual behavioral strategy space for interactions in the iterated Prisoner's Dilemma (IPD) games, the so-called Zero-Determinant (ZD) strategies is a set of rather simple memory-one strategies yet can unilaterally set a linear payoff relationship between themselves and their opponent. Although the witting of ZD strategies gives players an upper hand in the IPD games, we find and characterize unbending strategies that can force ZD players to be fair in their own interest. Moreover, our analysis reveals the ubiquity of unbending properties in common IPD strategies which are previously overlooked. In this work, we demonstrate the important steering role of unbending strategies in fostering fairness and cooperation in pairwise interactions. Our results will help bring a new perspective by means of combining game theory and multi-agent learning systems for optimizing winning strategies that are robust to noises, errors, and deceptions in non-zero-sum games. △ Less

Submitted 29 May, 2023; originally announced May 2023.

Comments: 8 pages, 3 figures, 2 tables

Accelerating Text-to-Image Editing via Cache-Enabled Sparse Diffusion Inference

Authors: Zihao Yu, Haoyang Li, Fangcheng Fu, Xupeng Miao, Bin Cui

Abstract: Due to the recent success of diffusion models, text-to-image generation is becoming increasingly popular and achieves a wide range of applications. Among them, text-to-image editing, or continuous text-to-image generation, attracts lots of attention and can potentially improve the quality of generated images. It's common to see that users may want to slightly edit the generated image by making min… ▽ More Due to the recent success of diffusion models, text-to-image generation is becoming increasingly popular and achieves a wide range of applications. Among them, text-to-image editing, or continuous text-to-image generation, attracts lots of attention and can potentially improve the quality of generated images. It's common to see that users may want to slightly edit the generated image by making minor modifications to their input textual descriptions for several rounds of diffusion inference. However, such an image editing process suffers from the low inference efficiency of many existing diffusion models even using GPU accelerators. To solve this problem, we introduce Fast Image Semantically Edit (FISEdit), a cached-enabled sparse diffusion model inference engine for efficient text-to-image editing. The key intuition behind our approach is to utilize the semantic mapping between the minor modifications on the input text and the affected regions on the output image. For each text editing step, FISEdit can automatically identify the affected image regions and utilize the cached unchanged regions' feature map to accelerate the inference process. Extensive empirical results show that FISEdit can be $3.4\times$ and $4.4\times$ faster than existing methods on NVIDIA TITAN RTX and A100 GPUs respectively, and even generates more satisfactory images. △ Less

Submitted 4 January, 2024; v1 submitted 27 May, 2023; originally announced May 2023.

Comments: AAAI 2024

OSDP: Optimal Sharded Data Parallel for Distributed Deep Learning

Authors: Youhe Jiang, Fangcheng Fu, Xupeng Miao, Xiaonan Nie, Bin Cui

Abstract: Large-scale deep learning models contribute to significant performance improvements on varieties of downstream tasks. Current data and model parallelism approaches utilize model replication and partition techniques to support the distributed training of ultra-large models. However, directly deploying these systems often leads to sub-optimal training efficiency due to the complex model architecture… ▽ More Large-scale deep learning models contribute to significant performance improvements on varieties of downstream tasks. Current data and model parallelism approaches utilize model replication and partition techniques to support the distributed training of ultra-large models. However, directly deploying these systems often leads to sub-optimal training efficiency due to the complex model architectures and the strict device memory constraints. In this paper, we propose Optimal Sharded Data Parallel (OSDP), an automated parallel training system that combines the advantages from both data and model parallelism. Given the model description and the device information, OSDP makes trade-offs between the memory consumption and the hardware utilization, thus automatically generates the distributed computation graph and maximizes the overall system throughput. In addition, OSDP introduces operator splitting to further alleviate peak memory footprints during training with negligible overheads, which enables the trainability of larger models as well as the higher throughput. Extensive experimental results of OSDP on multiple different kinds of large-scale models demonstrate that the proposed strategy outperforms the state-of-the-art in multiple regards. △ Less

Submitted 17 May, 2023; v1 submitted 17 May, 2023; originally announced May 2023.

Comments: An older version is in existence, and the article has been updated there. The URL for the updated version is arXiv:2209.13258

Singleton-Optimal LRCs and Perfect LRCs via Cyclic and Constacyclic Codes

Authors: Weijun Fang, Fang-Wei Fu, Bin Chen, Shu-Tao Xia

Abstract: Locally repairable codes (LRCs) have emerged as an important coding scheme in distributed storage systems (DSSs) with relatively low repair cost by accessing fewer non-failure nodes. Theoretical bounds and optimal constructions of LRCs have been widely investigated. Optimal LRCs via cyclic and constacyclic codes provide significant benefit of elegant algebraic structure and efficient encoding proc… ▽ More Locally repairable codes (LRCs) have emerged as an important coding scheme in distributed storage systems (DSSs) with relatively low repair cost by accessing fewer non-failure nodes. Theoretical bounds and optimal constructions of LRCs have been widely investigated. Optimal LRCs via cyclic and constacyclic codes provide significant benefit of elegant algebraic structure and efficient encoding procedure. In this paper, we continue to consider the constructions of optimal LRCs via cyclic and constacyclic codes with long code length. Specifically, we first obtain two classes of $q$-ary cyclic Singleton-optimal $(n, k, d=6;r=2)$-LRCs with length $n=3(q+1)$ when $3 \mid (q-1)$ and $q$ is even, and length $n=\frac{3}{2}(q+1)$ when $3 \mid (q-1)$ and $q \equiv 1(\bmod~4)$, respectively. To the best of our knowledge, this is the first construction of $q$-ary cyclic Singleton-optimal LRCs with length $n>q+1$ and minimum distance $d \geq 5$. On the other hand, an LRC acheiving the Hamming-type bound is called a perfect LRC. By using cyclic and constacyclic codes, we construct two new families of $q$-ary perfect LRCs with length $n=\frac{q^m-1}{q-1}$, minimum distance $d=5$ and locality $r=2$. △ Less

Submitted 10 March, 2023; originally announced March 2023.

Complex Systems of Secrecy: The Offshore Networks of Oligarchs

Authors: Ho-Chun Herbert Chang, Brooke Harrington, Feng Fu, Daniel Rockmore

Abstract: Following the invasion of Ukraine, the US, UK, and EU governments--among others--sanctioned oligarchs close to Putin. This approach has come under scrutiny, as evidence has emerged of the oligarchs' successful evasion of these punishments. To address this problem, we analyze the role of an overlooked but highly influential group: the secretive professional intermediaries who create and administer… ▽ More Following the invasion of Ukraine, the US, UK, and EU governments--among others--sanctioned oligarchs close to Putin. This approach has come under scrutiny, as evidence has emerged of the oligarchs' successful evasion of these punishments. To address this problem, we analyze the role of an overlooked but highly influential group: the secretive professional intermediaries who create and administer the oligarchs' offshore financial empires. Drawing on the Offshore Leaks Database provided by the International Consortium of Investigative Journalists (ICIJ), we examine the ties linking offshore expert advisors (lawyers, accountants, and other wealth management professionals) to ultra-high-net-worth individuals from four countries: Russia, China, the United States, and Hong Kong. We find that resulting nation-level "oligarch networks" share a scale-free structure characterized by heterogeneity of heavy-tailed degree distributions of wealth managers; however, network topologies diverge across clients from democratic versus autocratic regimes. While generally robust, scale-free networks are fragile when targeted by attacks on highly-connected nodes. Our "knock-out" experiments pinpoint this vulnerability to the small group of wealth managers themselves, suggesting that sanctioning these professional intermediaries may be more effective and efficient in disrupting dark finance flows than sanctions on their wealthy clients. This vulnerability is especially pronounced amongst Russian oligarchs, who concentrate their offshore business in a handful of boutique wealth management firms. The distinctive patterns we identify suggest a new approach to sanctions, focused on expert intermediaries to disrupt the finances and alliances of their wealthy clients. More generally, our research contributes to the larger body of work on complexity science and the structures of secrecy. △ Less

Submitted 6 March, 2023; originally announced March 2023.

Angel-PTM: A Scalable and Economical Large-scale Pre-training System in Tencent

Authors: Xiaonan Nie, Yi Liu, Fangcheng Fu, Jinbao Xue, Dian Jiao, Xupeng Miao, Yangyu Tao, Bin Cui

Abstract: Recent years have witnessed the unprecedented achievements of large-scale pre-trained models, especially the Transformer models. Many products and services in Tencent Inc., such as WeChat, QQ, and Tencent Advertisement, have been opted in to gain the power of pre-trained models. In this work, we present Angel-PTM, a productive deep learning system designed for pre-training and fine-tuning Transfor… ▽ More Recent years have witnessed the unprecedented achievements of large-scale pre-trained models, especially the Transformer models. Many products and services in Tencent Inc., such as WeChat, QQ, and Tencent Advertisement, have been opted in to gain the power of pre-trained models. In this work, we present Angel-PTM, a productive deep learning system designed for pre-training and fine-tuning Transformer models. Angel-PTM can train extremely large-scale models with hierarchical memory efficiently. The key designs of Angel-PTM are the fine-grained memory management via the Page abstraction and a unified scheduling method that coordinate the computations, data movements, and communications. Furthermore, Angel-PTM supports extreme model scaling with SSD storage and implements the lock-free updating mechanism to address the SSD I/O bandwidth bottlenecks. Experimental results demonstrate that Angel-PTM outperforms existing systems by up to 114.8% in terms of maximum model scale as well as up to 88.9% in terms of training throughput. Additionally, experiments on GPT3-175B and T5-MoE-1.2T models utilizing hundreds of GPUs verify the strong scalability of Angel-PTM. △ Less

Submitted 5 March, 2023; originally announced March 2023.

Bent Partitions, Vectorial Dual-Bent Functions and Partial Difference Sets

Authors: Jiaxin Wang, Fang-Wei Fu, Yadi Wei

Abstract: It is known that partial spreads is a class of bent partitions. In \cite{AM2022Be,MP2021Be}, two classes of bent partitions whose forms are similar to partial spreads were presented. In \cite{AKM2022Ge}, more bent partitions $Γ_{1}, Γ_{2}, Γ_{1}^{\bullet}, Γ_{2}^{\bullet}, Θ_{1}, Θ_{2}$ were presented from (pre)semifields, including the bent partitions given in \cite{AM2022Be,MP2021Be}. In this pa… ▽ More It is known that partial spreads is a class of bent partitions. In \cite{AM2022Be,MP2021Be}, two classes of bent partitions whose forms are similar to partial spreads were presented. In \cite{AKM2022Ge}, more bent partitions $Γ_{1}, Γ_{2}, Γ_{1}^{\bullet}, Γ_{2}^{\bullet}, Θ_{1}, Θ_{2}$ were presented from (pre)semifields, including the bent partitions given in \cite{AM2022Be,MP2021Be}. In this paper, we investigate the relations between bent partitions and vectorial dual-bent functions. For any prime $p$, we show that one can generate certain bent partitions (called bent partitions satisfying Condition $\mathcal{C}$) from certain vectorial dual-bent functions (called vectorial dual-bent functions satisfying Condition A). In particular, when $p$ is an odd prime, we show that bent partitions satisfying Condition $\mathcal{C}$ one-to-one correspond to vectorial dual-bent functions satisfying Condition A. We give an alternative proof that $Γ_{1}, Γ_{2}, Γ_{1}^{\bullet}, Γ_{2}^{\bullet}, Θ_{1}, Θ_{2}$ are bent partitions. We present a secondary construction of vectorial dual-bent functions, which can be used to generate more bent partitions. We show that any ternary weakly regular bent function $f: V_{n}^{(3)}\rightarrow \mathbb{F}_{3}$ ($n$ even) of $2$-form can generate a bent partition. When such $f$ is weakly regular but not regular, the generated bent partition by $f$ is not coming from a normal bent partition, which answers an open problem proposed in \cite{AM2022Be}. We give a sufficient condition on constructing partial difference sets from bent partitions, and when $p$ is an odd prime, we provide a characterization of bent partitions satisfying Condition $\mathcal{C}$ in terms of partial difference sets. △ Less

Submitted 2 January, 2023; originally announced January 2023.

Dormant Neural Trojans

Authors: Feisi Fu, Panagiota Kiourti, Wenchao Li

Abstract: We present a novel methodology for neural network backdoor attacks. Unlike existing training-time attacks where the Trojaned network would respond to the Trojan trigger after training, our approach inserts a Trojan that will remain dormant until it is activated. The activation is realized through a specific perturbation to the network's weight parameters only known to the attacker. Our analysis an… ▽ More We present a novel methodology for neural network backdoor attacks. Unlike existing training-time attacks where the Trojaned network would respond to the Trojan trigger after training, our approach inserts a Trojan that will remain dormant until it is activated. The activation is realized through a specific perturbation to the network's weight parameters only known to the attacker. Our analysis and the experimental results demonstrate that dormant Trojaned networks can effectively evade detection by state-of-the-art backdoor detection methods. △ Less

Submitted 2 November, 2022; originally announced November 2022.

Convergence analysis of a quasi-Monte Carlo-based deep learning algorithm for solving partial differential equations

Authors: Fengjiang Fu, Xiaoqun Wang

Abstract: Deep learning methods have achieved great success in solving partial differential equations (PDEs), where the loss is often defined as an integral. The accuracy and efficiency of these algorithms depend greatly on the quadrature method. We propose to apply quasi-Monte Carlo (QMC) methods to the Deep Ritz Method (DRM) for solving the Neumann problems for the Poisson equation and the static Schrödin… ▽ More Deep learning methods have achieved great success in solving partial differential equations (PDEs), where the loss is often defined as an integral. The accuracy and efficiency of these algorithms depend greatly on the quadrature method. We propose to apply quasi-Monte Carlo (QMC) methods to the Deep Ritz Method (DRM) for solving the Neumann problems for the Poisson equation and the static Schrödinger equation. For error estimation, we decompose the error of using the deep learning algorithm to solve PDEs into the generalization error, the approximation error and the training error. We establish the upper bounds and prove that QMC-based DRM achieves an asymptotically smaller error bound than DRM. Numerical experiments show that the proposed method converges faster in all cases and the variances of the gradient estimators of randomized QMC-based DRM are much smaller than those of DRM, which illustrates the superiority of QMC in deep learning over MC. △ Less

Submitted 28 October, 2022; originally announced October 2022.

Comments: 27 pages, 4 figures, 2 tables

MSC Class: 35J20; 35Q68; 65D30; 65N15; 68T07

OSDP: Optimal Sharded Data Parallel for Distributed Deep Learning

Authors: Youhe Jiang, Fangcheng Fu, Xupeng Miao, Xiaonan Nie, Bin Cui

Abstract: Large-scale deep learning models contribute to significant performance improvements on varieties of downstream tasks. Current data and model parallelism approaches utilize model replication and partition techniques to support the distributed training of ultra-large models. However, directly deploying these systems often leads to sub-optimal training efficiency due to the complex model architecture… ▽ More Large-scale deep learning models contribute to significant performance improvements on varieties of downstream tasks. Current data and model parallelism approaches utilize model replication and partition techniques to support the distributed training of ultra-large models. However, directly deploying these systems often leads to sub-optimal training efficiency due to the complex model architectures and the strict device memory constraints. In this paper, we propose Optimal Sharded Data Parallel (OSDP), an automated parallel training system that combines the advantages from both data and model parallelism. Given the model description and the device information, OSDP makes trade-offs between the memory consumption and the hardware utilization, thus automatically generates the distributed computation graph and maximizes the overall system throughput. In addition, OSDP introduces operator splitting to further alleviate peak memory footprints during training with negligible overheads, which enables the trainability of larger models as well as the higher throughput. Extensive experimental results of OSDP on multiple different kinds of large-scale models demonstrate that the proposed strategy outperforms the state-of-the-art in multiple regards. Our code is available at https://github.com/Youhe-Jiang/OptimalShardedDataParallel. △ Less

Submitted 18 May, 2023; v1 submitted 27 September, 2022; originally announced September 2022.

Comments: Accepted by IJCAI 2023

A multi-chain synchronization protocol that leverage zero knowledge proof to minimize communication trust base

Authors: Sinka Gao, Guo Qiang Li, Hong Fei Fu, Heng Zhang

Abstract: Delphinus cross-chain aggregator is a universal firmware which synchronise states between different smart contracts on different block-chains. In the world of block-chains, synchronization challenges are two-folded. Firstly, contracts from different main block-chain can not communicate with each other which makes it hard to establish a trustworthy communication channel for them to share and mainta… ▽ More Delphinus cross-chain aggregator is a universal firmware which synchronise states between different smart contracts on different block-chains. In the world of block-chains, synchronization challenges are two-folded. Firstly, contracts from different main block-chain can not communicate with each other which makes it hard to establish a trustworthy communication channel for them to share and maintain a universal state between each other. Secondly, transactions on different block-chains can hardly be ordered thus conflicts are common and we need a novel way to avoid and handle these conflicts. Delphinus cross-chain aggregator is a ZKSNARK based multi-block-chain layer on top of which rich cross chain applications can run safely and efficiently. △ Less

Submitted 7 September, 2022; originally announced September 2022.

Hilti-Oxford Dataset: A Millimetre-Accurate Benchmark for Simultaneous Localization and Mapping

Authors: Lintong Zhang, Michael Helmberger, Lanke Frank Tarimo Fu, David Wisth, Marco Camurri, Davide Scaramuzza, Maurice Fallon

Abstract: Simultaneous Localization and Mapping (SLAM) is being deployed in real-world applications, however many state-of-the-art solutions still struggle in many common scenarios. A key necessity in progressing SLAM research is the availability of high-quality datasets and fair and transparent benchmarking. To this end, we have created the Hilti-Oxford Dataset, to push state-of-the-art SLAM systems to the… ▽ More Simultaneous Localization and Mapping (SLAM) is being deployed in real-world applications, however many state-of-the-art solutions still struggle in many common scenarios. A key necessity in progressing SLAM research is the availability of high-quality datasets and fair and transparent benchmarking. To this end, we have created the Hilti-Oxford Dataset, to push state-of-the-art SLAM systems to their limits. The dataset has a variety of challenges ranging from sparse and regular construction sites to a 17th century neoclassical building with fine details and curved surfaces. To encourage multi-modal SLAM approaches, we designed a data collection platform featuring a lidar, five cameras, and an IMU (Inertial Measurement Unit). With the goal of benchmarking SLAM algorithms for tasks where accuracy and robustness are paramount, we implemented a novel ground truth collection method that enables our dataset to accurately measure SLAM pose errors with millimeter accuracy. To further ensure accuracy, the extrinsics of our platform were verified with a micrometer-accurate scanner, and temporal calibration was managed online using hardware time synchronization. The multi-modality and diversity of our dataset attracted a large field of academic and industrial researchers to enter the second edition of the Hilti SLAM challenge, which concluded in June 2022. The results of the challenge show that while the top three teams could achieve an accuracy of 2cm or better for some sequences, the performance dropped off in more difficult sequences. △ Less

Submitted 15 May, 2023; v1 submitted 21 August, 2022; originally announced August 2022.

Comments: Presented at IEEE Robotics and Automation (ICRA), 2023

Journal ref: IEEE Robotics and Automation Letters ( Volume: 8, Issue: 1, January 2023)

A Tool for Neural Network Global Robustness Certification and Training

Authors: Zhilu Wang, Yixuan Wang, Feisi Fu, Ruochen Jiao, Chao Huang, Wenchao Li, Qi Zhu

Abstract: With the increment of interest in leveraging machine learning technology in safety-critical systems, the robustness of neural networks under external disturbance receives more and more concerns. Global robustness is a robustness property defined on the entire input domain. And a certified globally robust network can ensure its robustness on any possible network input. However, the state-of-the-art… ▽ More With the increment of interest in leveraging machine learning technology in safety-critical systems, the robustness of neural networks under external disturbance receives more and more concerns. Global robustness is a robustness property defined on the entire input domain. And a certified globally robust network can ensure its robustness on any possible network input. However, the state-of-the-art global robustness certification algorithm can only certify networks with at most several thousand neurons. In this paper, we propose the GPU-supported global robustness certification framework GROCET, which is more efficient than the previous optimization-based certification approach. Moreover, GROCET provides differentiable global robustness, which is leveraged in the training of globally robust neural networks. △ Less

Submitted 15 August, 2022; originally announced August 2022.

Towards Communication-efficient Vertical Federated Learning Training via Cache-enabled Local Updates

Authors: Fangcheng Fu, Xupeng Miao, Jiawei Jiang, Huanran Xue, Bin Cui

Abstract: Vertical federated learning (VFL) is an emerging paradigm that allows different parties (e.g., organizations or enterprises) to collaboratively build machine learning models with privacy protection. In the training phase, VFL only exchanges the intermediate statistics, i.e., forward activations and backward derivatives, across parties to compute model gradients. Nevertheless, due to its geo-distri… ▽ More Vertical federated learning (VFL) is an emerging paradigm that allows different parties (e.g., organizations or enterprises) to collaboratively build machine learning models with privacy protection. In the training phase, VFL only exchanges the intermediate statistics, i.e., forward activations and backward derivatives, across parties to compute model gradients. Nevertheless, due to its geo-distributed nature, VFL training usually suffers from the low WAN bandwidth. In this paper, we introduce CELU-VFL, a novel and efficient VFL training framework that exploits the local update technique to reduce the cross-party communication rounds. CELU-VFL caches the stale statistics and reuses them to estimate model gradients without exchanging the ad hoc statistics. Significant techniques are proposed to improve the convergence performance. First, to handle the stochastic variance problem, we propose a uniform sampling strategy to fairly choose the stale statistics for local updates. Second, to harness the errors brought by the staleness, we devise an instance weighting mechanism that measures the reliability of the estimated gradients. Theoretical analysis proves that CELU-VFL achieves a similar sub-linear convergence rate as vanilla VFL training but requires much fewer communication rounds. Empirical results on both public and real-world workloads validate that CELU-VFL can be up to six times faster than the existing works. △ Less

Submitted 29 July, 2022; originally announced July 2022.

Comments: VLDB 2022

Constructions of linear codes with two or three weights from vectorial dual-bent functions

Authors: Jiaxin Wang, Zexia Shi, Yadi Wei, Fang-Wei Fu

Abstract: Linear codes with a few weights are an important class of codes in coding theory and have attracted a lot of attention. In this paper, we present several constructions of $q$-ary linear codes with two or three weights from vectorial dual-bent functions, where $q$ is a power of an odd prime $p$. The weight distributions of the constructed $q$-ary linear codes are completely determined. We illustrat… ▽ More Linear codes with a few weights are an important class of codes in coding theory and have attracted a lot of attention. In this paper, we present several constructions of $q$-ary linear codes with two or three weights from vectorial dual-bent functions, where $q$ is a power of an odd prime $p$. The weight distributions of the constructed $q$-ary linear codes are completely determined. We illustrate that some known constructions in the literature can be obtained by our constructions. In some special cases, our constructed linear codes can meet the Griesmer bound. Furthermore, based on the constructed $q$-ary linear codes, we obtain secret sharing schemes with interesting access structures. △ Less

Submitted 24 July, 2022; originally announced July 2022.

Bounds and Constructions of Singleton-Optimal Locally Repairable Codes with Small Localities

Authors: Weijun Fang, Bin Chen, Shu-Tao Xia, Fang-Wei Fu

Abstract: Constructions of optimal locally repairable codes (LRCs) achieving Singleton-type bound have been exhaustively investigated in recent years. In this paper, we consider new bounds and constructions of Singleton-optimal LRCs with minmum distance $d=6$, locality $r=3$ and minimum distance $d=7$ and locality $r=2$, respectively. Firstly, we establish equivalent connections between the existence of the… ▽ More Constructions of optimal locally repairable codes (LRCs) achieving Singleton-type bound have been exhaustively investigated in recent years. In this paper, we consider new bounds and constructions of Singleton-optimal LRCs with minmum distance $d=6$, locality $r=3$ and minimum distance $d=7$ and locality $r=2$, respectively. Firstly, we establish equivalent connections between the existence of these two families of LRCs and the existence of some subsets of lines in the projective space with certain properties. Then, we employ the line-point incidence matrix and Johnson bounds for constant weight codes to derive new improved bounds on the code length, which are tighter than known results. Finally, by using some techniques of finite field and finite geometry, we give some new constructions of Singleton-optimal LRCs, which have larger length than previous ones. △ Less

Submitted 12 July, 2022; originally announced July 2022.

BlindFL: Vertical Federated Machine Learning without Peeking into Your Data

Authors: Fangcheng Fu, Huanran Xue, Yong Cheng, Yangyu Tao, Bin Cui

Abstract: Due to the rising concerns on privacy protection, how to build machine learning (ML) models over different data sources with security guarantees is gaining more popularity. Vertical federated learning (VFL) describes such a case where ML models are built upon the private data of different participated parties that own disjoint features for the same set of instances, which fits many real-world coll… ▽ More Due to the rising concerns on privacy protection, how to build machine learning (ML) models over different data sources with security guarantees is gaining more popularity. Vertical federated learning (VFL) describes such a case where ML models are built upon the private data of different participated parties that own disjoint features for the same set of instances, which fits many real-world collaborative tasks. Nevertheless, we find that existing solutions for VFL either support limited kinds of input features or suffer from potential data leakage during the federated execution. To this end, this paper aims to investigate both the functionality and security of ML modes in the VFL scenario. To be specific, we introduce BlindFL, a novel framework for VFL training and inference. First, to address the functionality of VFL models, we propose the federated source layers to unite the data from different parties. Various kinds of features can be supported efficiently by the federated source layers, including dense, sparse, numerical, and categorical features. Second, we carefully analyze the security during the federated execution and formalize the privacy requirements. Based on the analysis, we devise secure and accurate algorithm protocols, and further prove the security guarantees under the ideal-real simulation paradigm. Extensive experiments show that BlindFL supports diverse datasets and models efficiently whilst achieves robust privacy guarantees. △ Less

Submitted 16 June, 2022; originally announced June 2022.

Comments: SIGMOD 2022

Spatial Games of Fake News

Authors: Matthew I Jones, Scott D. Pauls, Feng Fu

Abstract: To curb the spread of fake news on social media platforms, recent studies have considered an online crowdsourcing fact-checking approach as one possible intervention method to reduce misinformation. However, it remains unclear under what conditions crowdsourcing fact-checking efforts deter the spread of misinformation. To address this issue, we model such distributed fact-checking as `peer policin… ▽ More To curb the spread of fake news on social media platforms, recent studies have considered an online crowdsourcing fact-checking approach as one possible intervention method to reduce misinformation. However, it remains unclear under what conditions crowdsourcing fact-checking efforts deter the spread of misinformation. To address this issue, we model such distributed fact-checking as `peer policing' that will reduce the perceived payoff to share or disseminate false information (fake news) and also reward the spread of trustworthy information (real news). By simulating our model on synthetic square lattices and small-world networks, we show that the presence of social network structure enables fake news spreaders to be self-organized into echo chambers, thereby providing a boost to the efficacy of fake news and thus its resistance to fact-checking efforts. Additionally, to study our model in a more realistic setting, we utilize a Twitter network dataset and study the effectiveness of deliberately choosing specific individuals to be fact-checkers. We find that targeted fact-checking efforts can be highly effective, seeing the same level of success with as little as a fifth of the number of fact-checkers, but it depends on the structure of the network in question. In the limit of weak selection, we obtain closed-form analytical conditions for critical threshold of crowdsourced fact-checking in terms of the payoff values in our fact-checker/fake news game. Our work has practical implications for developing model-based mitigation strategies for controlling the spread of misinformation that interferes with the political discourse. △ Less

Submitted 8 June, 2022; originally announced June 2022.

Comments: 18 pages, 5 figures

On generalized quasi-cyclic codes over $\mathbb{Z}_4$

Authors: Jian Gao, Xiangrui Meng, Fang-Wei Fu

Abstract: Based on good algebraic structures and practicabilities, generalized quasi-cyclic (GQC) codes play important role in coding theory. In this paper, we study some results on GQC codes over $\mathbb{Z}_4$ including the normalized generating set, the minimum generating set and the normalized generating set of their dual codes. As an application, new $\mathbb{Z}_4$-linear codes and good nonlinear binar… ▽ More Based on good algebraic structures and practicabilities, generalized quasi-cyclic (GQC) codes play important role in coding theory. In this paper, we study some results on GQC codes over $\mathbb{Z}_4$ including the normalized generating set, the minimum generating set and the normalized generating set of their dual codes. As an application, new $\mathbb{Z}_4$-linear codes and good nonlinear binary codes are constructed from GQC codes over $\mathbb{Z}_4$. △ Less

Submitted 22 March, 2022; originally announced March 2022.

Post-quantum Multi-stage Secret Sharing Schemes using Inhomogeneous Linear Recursion and Ajtai's Function

Authors: Jing Yang, Fang-Wei Fu

Abstract: Secret sharing was firstly proposed in 1979 by Shamir and Blakley respectively. To avoid deficiencies of original schemes, researchers presented improvement schemes, among which the multi-secret sharing scheme (MSS) is significant. There are three categories of MSSs, however, we focus on multi-stage secret sharing scheme (MSSS) recovering secrets with any order in this work. By observing inhomogen… ▽ More Secret sharing was firstly proposed in 1979 by Shamir and Blakley respectively. To avoid deficiencies of original schemes, researchers presented improvement schemes, among which the multi-secret sharing scheme (MSS) is significant. There are three categories of MSSs, however, we focus on multi-stage secret sharing scheme (MSSS) recovering secrets with any order in this work. By observing inhomogeneous linear recursions (ILRs) in the literature, we conclude a general formula and divide ILRs into two types according to different variables in them. Utilizing these two kinds of ILRs, we propose four verifiable MSSSs with Ajtai's function, which is a lattice-based function. Our schemes have the following advantages. Firstly, our schemes can detect cheat of the dealer and participants, and are multi-use. Secondly, we have several ways to restore secrets. Thirdly, we can turn our schemes into other types of MSSs due to the universality of our method. Fourthly, since we utilize a lattice-based function to mask shares, our schemes can resist the attack from the quantum computer with computational security. Finally, although our schemes need more memory consumption than some known schemes, we need much less time consumption, which makes our schemes more suitable facing limited computing power. △ Less

Submitted 18 February, 2022; originally announced February 2022.

Comments: 36 pages,1 figure, 3 tables

MSC Class: 11T71; 68R01; 94A60; 94A62

New results on vectorial dual-bent functions and partial difference sets

Authors: Jiaxin Wang, Fang-Wei Fu

Abstract: Bent functions $f: V_{n}\rightarrow \mathbb{F}_{p}$ with certain additional properties play an important role in constructing partial difference sets, where $V_{n}$ denotes an $n$-dimensional vector space over $\mathbb{F}_{p}$, $p$ is an odd prime. In \cite{Cesmelioglu1,Cesmelioglu2}, the so-called vectorial dual-bent functions are considered to construct partial difference sets. In \cite{Cesmelio… ▽ More Bent functions $f: V_{n}\rightarrow \mathbb{F}_{p}$ with certain additional properties play an important role in constructing partial difference sets, where $V_{n}$ denotes an $n$-dimensional vector space over $\mathbb{F}_{p}$, $p$ is an odd prime. In \cite{Cesmelioglu1,Cesmelioglu2}, the so-called vectorial dual-bent functions are considered to construct partial difference sets. In \cite{Cesmelioglu1}, Çeşmelioǧlu \emph{et al.} showed that for vectorial dual-bent functions $F: V_{n}\rightarrow V_{s}$ with certain additional properties, the preimage set of $0$ for $F$ forms a partial difference set. In \cite{Cesmelioglu2}, Çeşmelioǧlu \emph{et al.} showed that for a class of Maiorana-McFarland vectorial dual-bent functions $F: V_{n}\rightarrow \mathbb{F}_{p^s}$, the preimage set of the squares (non-squares) in $\mathbb{F}_{p^s}^{*}$ for $F$ forms a partial difference set. In this paper, we further study vectorial dual-bent functions and partial difference sets. We prove that for vectorial dual-bent functions $F: V_{n}\rightarrow \mathbb{F}_{p^s}$ with certain additional properties, the preimage set of the squares (non-squares) in $\mathbb{F}_{p^s}^{*}$ for $F$ and the preimage set of any coset of some subgroup of $\mathbb{F}_{p^s}^{*}$ for $F$ form partial difference sets. Furthermore, explicit constructions of partial difference sets are yielded from some (non)-quadratic vectorial dual-bent functions. In this paper, we illustrate that almost all the results of using weakly regular $p$-ary bent functions to construct partial difference sets are special cases of our results. △ Less

Submitted 28 April, 2022; v1 submitted 8 February, 2022; originally announced February 2022.

Some Results on the Improved Bound and Construction of Optimal $(r,δ)$ LRCs

Authors: Bin Chen, Weijun Fang, Yueqi Chen, Shu-Tao Xia, Fang-Wei Fu, Xiangyu Chen

Abstract: Locally repairable codes (LRCs) with $(r,δ)$ locality were introduced by Prakash \emph{et al.} into distributed storage systems (DSSs) due to their benefit of locally repairing at least $δ-1$ erasures via other $r$ survival nodes among the same local group. An LRC achieving the $(r,δ)$ Singleton-type bound is called an optimal $(r,δ)$ LRC. Constructions of optimal $(r,δ)$ LRCs with longer code len… ▽ More Locally repairable codes (LRCs) with $(r,δ)$ locality were introduced by Prakash \emph{et al.} into distributed storage systems (DSSs) due to their benefit of locally repairing at least $δ-1$ erasures via other $r$ survival nodes among the same local group. An LRC achieving the $(r,δ)$ Singleton-type bound is called an optimal $(r,δ)$ LRC. Constructions of optimal $(r,δ)$ LRCs with longer code length and determining the maximal code length have been an important research direction in coding theory in recent years. In this paper, we conduct further research on the improvement of maximum code length of optimal $(r,δ)$ LRCs. For $2δ+1\leq d\leq 2δ+2$, our upper bounds largely improve the ones by Cai \emph{et al.}, which are tight in some special cases. Moreover, we generalize the results of Chen \emph{et al.} and obtain a complete characterization of optimal $(r=2, δ)$-LRCs in the sense of geometrical existence in the finite projective plane $PG(2,q)$. Within this geometrical characterization, we construct a class of optimal $(r,δ)$ LRCs based on the sunflower structure. Both the construction and upper bounds are better than previous ones. △ Less

Submitted 7 February, 2022; originally announced February 2022.

Comments: Submitted to the 2022 IEEE International Symposium on Information Theory (ISIT)

Polynomial-Time Key Recovery Attack on the Lau-Tan Cryptosystem Based on Gabidulin Codes

Authors: Wenshuo Guo, Fang-Wei Fu

Abstract: This paper presents a key recovery attack on the cryptosystem proposed by Lau and Tan in a talk at ACISP 2018. The Lau-Tan cryptosystem uses Gabidulin codes as the underlying decodable code. To hide the algebraic structure of Gabidulin codes, the authors chose a matrix of column rank $n$ to mix with a generator matrix of the secret Gabidulin code. The other part of the public key, however, reveals… ▽ More This paper presents a key recovery attack on the cryptosystem proposed by Lau and Tan in a talk at ACISP 2018. The Lau-Tan cryptosystem uses Gabidulin codes as the underlying decodable code. To hide the algebraic structure of Gabidulin codes, the authors chose a matrix of column rank $n$ to mix with a generator matrix of the secret Gabidulin code. The other part of the public key, however, reveals crucial information about the private key. Our analysis shows that the problem of recovering the private key can be reduced to solving a multivariate linear system over the base field, rather than solving a multivariate quadratic system as claimed by the authors. Solving the linear system for any nonzero solution permits us to recover the private key. Apparently, this attack costs polynomial time, and therefore completely breaks the cryptosystem. △ Less

Submitted 5 January, 2022; v1 submitted 31 December, 2021; originally announced December 2021.

K-Core Decomposition on Super Large Graphs with Limited Resources

Authors: Shicheng Gao, Jie Xu, Xiaosen Li, Fangcheng Fu, Wentao Zhang, Wen Ouyang, Yangyu Tao, Bin Cui

Abstract: K-core decomposition is a commonly used metric to analyze graph structure or study the relative importance of nodes in complex graphs. Recent years have seen rapid growth in the scale of the graph, especially in industrial settings. For example, our industrial partner runs popular social applications with billions of users and is able to gather a rich set of user data. As a result, applying K-core… ▽ More K-core decomposition is a commonly used metric to analyze graph structure or study the relative importance of nodes in complex graphs. Recent years have seen rapid growth in the scale of the graph, especially in industrial settings. For example, our industrial partner runs popular social applications with billions of users and is able to gather a rich set of user data. As a result, applying K-core decomposition on large graphs has attracted more and more attention from academics and the industry. A simple but effective method to deal with large graphs is to train them in the distributed settings, and some distributed K-core decomposition algorithms are also proposed. Despite their effectiveness, we experimentally and theoretically observe that these algorithms consume too many resources and become unstable on super-large-scale graphs, especially when the given resources are limited. In this paper, we deal with those super-large-scale graphs and propose a divide-and-conquer strategy on top of the distributed K-core decomposition algorithm. We evaluate our approach on three large graphs. The experimental results show that the consumption of resources can be significantly reduced, and the calculation on large-scale graphs becomes more stable than the existing methods. For example, the distributed K-core decomposition algorithm can scale to a large graph with 136 billion edges without losing correctness with our divide-and-conquer technique. △ Less

Submitted 25 December, 2021; originally announced December 2021.

Comments: 10 pages, 11 figures

Sound and Complete Neural Network Repair with Minimality and Locality Guarantees

Authors: Feisi Fu, Wenchao Li

Abstract: We present a novel methodology for repairing neural networks that use ReLU activation functions. Unlike existing methods that rely on modifying the weights of a neural network which can induce a global change in the function space, our approach applies only a localized change in the function space while still guaranteeing the removal of the buggy behavior. By leveraging the piecewise linear nature… ▽ More We present a novel methodology for repairing neural networks that use ReLU activation functions. Unlike existing methods that rely on modifying the weights of a neural network which can induce a global change in the function space, our approach applies only a localized change in the function space while still guaranteeing the removal of the buggy behavior. By leveraging the piecewise linear nature of ReLU networks, our approach can efficiently construct a patch network tailored to the linear region where the buggy input resides, which when combined with the original network, provably corrects the behavior on the buggy input. Our method is both sound and complete -- the repaired network is guaranteed to fix the buggy input, and a patch is guaranteed to be found for any buggy input. Moreover, our approach preserves the continuous piecewise linear nature of ReLU networks, automatically generalizes the repair to all the points including other undetected buggy inputs inside the repair region, is minimal in terms of changes in the function space, and guarantees that outputs on inputs away from the repair region are unaltered. On several benchmarks, we show that our approach significantly outperforms existing methods in terms of locality and limiting negative side effects. Our code is available on GitHub: https://github.com/BU-DEPEND-Lab/REASSURE. △ Less

Submitted 21 July, 2022; v1 submitted 14 October, 2021; originally announced October 2021.

Comments: 17 pages, 3 figures

Journal ref: International Conference on Learning Representations, 2022

Decoding Error Probability of the Random Matrix Ensemble over the Erasure Channel

Authors: Chin Hei Chan, Fang-Wei Fu, Maosheng Xiong

Abstract: Using tools developed in a recent work by Shen and the second author, in this paper we carry out an in-depth study on the average decoding error probability of the random matrix ensemble over the erasure channel under three decoding principles, namely unambiguous decoding, maximum likelihood decoding and list decoding. We obtain explicit formulas for the average decoding error probabilities of the… ▽ More Using tools developed in a recent work by Shen and the second author, in this paper we carry out an in-depth study on the average decoding error probability of the random matrix ensemble over the erasure channel under three decoding principles, namely unambiguous decoding, maximum likelihood decoding and list decoding. We obtain explicit formulas for the average decoding error probabilities of the random matrix ensemble under these three decoding principles and compute the error exponents. Moreover, for unambiguous decoding, we compute the variance of the decoding error probability of the random matrix ensemble and the error exponent of the variance, which imply a strong concentration result, that is, roughly speaking, the ratio of the decoding error probability of a random code in the ensemble and the average decoding error probability of the ensemble converges to 1 with high probability when the code length goes to infinity. △ Less

Submitted 22 April, 2024; v1 submitted 23 August, 2021; originally announced August 2021.

Comments: 4 figures

MSC Class: 94A40; 94B70

On pure MDS asymmetric entanglement-assisted quantum error-correcting codes

Authors: Ziteng Huang, Weijun Fang, Fang-Wei Fu

Abstract: Recently, Galindo et al. introduced the concept of asymmetric entanglement-assisted quantum error-correcting codes (AEAQECCs) from Calderbank-Shor-Steane (CSS) construction. In general, it's difficult to determine the required number of maximally entangled states of an AEAQECC, which is associated with the dimension of the intersection of the two corresponding linear codes. Two linear codes are sa… ▽ More Recently, Galindo et al. introduced the concept of asymmetric entanglement-assisted quantum error-correcting codes (AEAQECCs) from Calderbank-Shor-Steane (CSS) construction. In general, it's difficult to determine the required number of maximally entangled states of an AEAQECC, which is associated with the dimension of the intersection of the two corresponding linear codes. Two linear codes are said to be a linear l-intersection pair if their intersection has dimension l. In this paper, all possible linear l-intersection pairs of MDS codes are given. As an application, we give a complete characterization of pure MDS AEAQECCs for all possible parameters. △ Less

Submitted 8 July, 2021; originally announced July 2021.

Semilinear Transformations in Coding Theory: A New Technique in Code-Based Cryptography

Authors: Wenshuo Guo, Fang-Wei Fu

Abstract: This paper presents a new technique for disturbing the algebraic structure of linear codes in code-based cryptography. This is a new attempt to exploit Gabidulin codes in the McEliece setting and almost all the previous cryptosystems of this type have been completely or partially broken. To be specific, we introduce the so-called semilinear transformation in coding theory, which is defined through… ▽ More This paper presents a new technique for disturbing the algebraic structure of linear codes in code-based cryptography. This is a new attempt to exploit Gabidulin codes in the McEliece setting and almost all the previous cryptosystems of this type have been completely or partially broken. To be specific, we introduce the so-called semilinear transformation in coding theory, which is defined through an $\mathbb{F}_q$-linear automorphism of $\mathbb{F}_{q^m}$, then apply them to construct a public key encryption scheme. Our analysis shows that this scheme can resist all the existing distinguisher attacks, such as Overbeck's attack and Coggia-Couvreur attack. Meanwhile, we endow the underlying Gabidulin code with the so-called partial cyclic structure to reduce the public key size. Compared with some other code-based cryptosystems, our proposal has a much more compact representation of public keys. For instance, 2592 bytes are enough for our proposal to achieve the security of 256 bits, almost 403 times smaller than that of Classic McEliece entering the third round of the NIST PQC project. △ Less

Submitted 6 May, 2022; v1 submitted 7 July, 2021; originally announced July 2021.