-
Unexpected Benefits of Self-Modeling in Neural Systems
Authors:
Vickram N. Premakumar,
Michael Vaiana,
Florin Pop,
Judd Rosenblatt,
Diogo Schwerz de Lucena,
Kirsten Ziman,
Michael S. A. Graziano
Abstract:
Self-models have been a topic of great interest for decades in studies of human cognition and more recently in machine learning. Yet what benefits do self-models confer? Here we show that when artificial networks learn to predict their internal states as an auxiliary task, they change in a fundamental way. To better perform the self-model task, the network learns to make itself simpler, more regul…
▽ More
Self-models have been a topic of great interest for decades in studies of human cognition and more recently in machine learning. Yet what benefits do self-models confer? Here we show that when artificial networks learn to predict their internal states as an auxiliary task, they change in a fundamental way. To better perform the self-model task, the network learns to make itself simpler, more regularized, more parameter-efficient, and therefore more amenable to being predictively modeled. To test the hypothesis of self-regularizing through self-modeling, we used a range of network architectures performing three classification tasks across two modalities. In all cases, adding self-modeling caused a significant reduction in network complexity. The reduction was observed in two ways. First, the distribution of weights was narrower when self-modeling was present. Second, a measure of network complexity, the real log canonical threshold (RLCT), was smaller when self-modeling was present. Not only were measures of complexity reduced, but the reduction became more pronounced as greater training weight was placed on the auxiliary task of self-modeling. These results strongly support the hypothesis that self-modeling is more than simply a network learning to predict itself. The learning has a restructuring effect, reducing complexity and increasing parameter efficiency. This self-regularization may help explain some of the benefits of self-models reported in recent machine learning literature, as well as the adaptive value of self-models to biological systems. In particular, these findings may shed light on the possible interaction between the ability to model oneself and the ability to be more easily modeled by others in a social or cooperative context.
△ Less
Submitted 14 July, 2024;
originally announced July 2024.
-
Rethinking harmless refusals when fine-tuning foundation models
Authors:
Florin Pop,
Judd Rosenblatt,
Diogo Schwerz de Lucena,
Michael Vaiana
Abstract:
In this paper, we investigate the degree to which fine-tuning in Large Language Models (LLMs) effectively mitigates versus merely conceals undesirable behavior. Through the lens of semi-realistic role-playing exercises designed to elicit such behaviors, we explore the response dynamics of LLMs post fine-tuning interventions. Our methodology involves prompting models for Chain-of-Thought (CoT) reas…
▽ More
In this paper, we investigate the degree to which fine-tuning in Large Language Models (LLMs) effectively mitigates versus merely conceals undesirable behavior. Through the lens of semi-realistic role-playing exercises designed to elicit such behaviors, we explore the response dynamics of LLMs post fine-tuning interventions. Our methodology involves prompting models for Chain-of-Thought (CoT) reasoning and analyzing the coherence between the reasoning traces and the resultant outputs. Notably, we identify a pervasive phenomenon we term \emph{reason-based deception}, where models either stop producing reasoning traces or produce seemingly ethical reasoning traces that belie the unethical nature of their final outputs. We further examine the efficacy of response strategies (polite refusal versus explicit rebuttal) in curbing the occurrence of undesired behavior in subsequent outputs of multi-turn interactions. Our findings reveal that explicit rebuttals significantly outperform polite refusals in preventing the continuation of undesired outputs and nearly eliminate reason-based deception, challenging current practices in model fine-tuning. Accordingly, the two key contributions of this paper are (1) defining and studying reason-based deception, a new type of hidden behavior, and (2) demonstrating that rebuttals provide a more robust response model to harmful requests than refusals, thereby highlighting the need to reconsider the response strategies in fine-tuning approaches.
△ Less
Submitted 27 June, 2024;
originally announced June 2024.
-
Resolution Limits for Detecting Community Changes in Multilayer Networks
Authors:
Michael Vaiana,
Sarah Muldoon
Abstract:
Multilayer networks capture pairwise relationships between the components of complex systems across multiple modes or scales of interactions. An important meso-scale feature of these networks is measured though their community structure, which defines groups of strongly connected nodes that exist within and across network layers. Because interlayer edges can describe relationships between differen…
▽ More
Multilayer networks capture pairwise relationships between the components of complex systems across multiple modes or scales of interactions. An important meso-scale feature of these networks is measured though their community structure, which defines groups of strongly connected nodes that exist within and across network layers. Because interlayer edges can describe relationships between different modalities, scales, or time points, it is essential to understand how communities change and evolve across layers. A popular method for detecting communities in multilayer networks consists of maximizing a quality function known as modularity. However, in the multilayer setting the modularity function depends on an interlayer coupling parameter, $ω$, and how this parameter affects community detection is not well understood. Here, we expose an upper bound for $ω$ beyond which community changes across layers can not be detected. This upper bound has non-trivial, purely multilayer effects and acts as a resolution limit for detecting evolving communities. Further, we establish an explicit and previously undiscovered relationship between the single layer resolution parameter, $γ$, and interlayer coupling parameter, $ω,$ that provides new understanding of the modularity parameter space. Our findings not only represent new theoretical considerations but also have important practical implications for choosing interlayer coupling values when using multilayer networks to model real-world systems whose communities change across time or modality.
△ Less
Submitted 9 March, 2018;
originally announced March 2018.