-
Language Models are Surprisingly Fragile to Drug Names in Biomedical Benchmarks
Authors:
Jack Gallifant,
Shan Chen,
Pedro Moreira,
Nikolaj Munch,
Mingye Gao,
Jackson Pond,
Leo Anthony Celi,
Hugo Aerts,
Thomas Hartvigsen,
Danielle Bitterman
Abstract:
Medical knowledge is context-dependent and requires consistent reasoning across various natural language expressions of semantically equivalent phrases. This is particularly crucial for drug names, where patients often use brand names like Advil or Tylenol instead of their generic equivalents. To study this, we create a new robustness dataset, RABBITS, to evaluate performance differences on medica…
▽ More
Medical knowledge is context-dependent and requires consistent reasoning across various natural language expressions of semantically equivalent phrases. This is particularly crucial for drug names, where patients often use brand names like Advil or Tylenol instead of their generic equivalents. To study this, we create a new robustness dataset, RABBITS, to evaluate performance differences on medical benchmarks after swapping brand and generic drug names using physician expert annotations.
We assess both open-source and API-based LLMs on MedQA and MedMCQA, revealing a consistent performance drop ranging from 1-10\%. Furthermore, we identify a potential source of this fragility as the contamination of test data in widely used pre-training datasets. All code is accessible at https://github.com/BittermanLab/RABBITS, and a HuggingFace leaderboard is available at https://huggingface.co/spaces/AIM-Harvard/rabbits-leaderboard.
△ Less
Submitted 18 June, 2024; v1 submitted 17 June, 2024;
originally announced June 2024.
-
Enabling data-driven and bidirectional model development in Verilog-A for photonic devices
Authors:
Dias Azhigulov,
Zeqin Lu,
James Pond,
Lukas Chrostowski,
Sudip Shekhar
Abstract:
We present a method to model photonic components in Verilog-A by introducing bidirectional signaling through a single port. To achieve this, the concept of power waves and scattering parameters from electromagnetism are employed. As a consequence, one can simultaneously transmit forward and backward propagating waves on a single wire while also capturing realistic, measurement-backed response of p…
▽ More
We present a method to model photonic components in Verilog-A by introducing bidirectional signaling through a single port. To achieve this, the concept of power waves and scattering parameters from electromagnetism are employed. As a consequence, one can simultaneously transmit forward and backward propagating waves on a single wire while also capturing realistic, measurement-backed response of photonic components in Verilog-A. We demonstrate examples to show the efficacy of the proposed technique in accounting for critical effects in photonic integrated circuits such as Fabry-Perot cavity resonance, reflections to lasers, etc. Our solution makes electronic-photonic co-simulation more intuitive and accurate.
△ Less
Submitted 3 July, 2024; v1 submitted 15 February, 2024;
originally announced February 2024.
-
Numerical Modeling of the 2009 Impact Event on Jupiter
Authors:
Jarrad W. T. Pond,
Csaba Palotai,
Travis Gabriel,
Donald G. Korycansky,
Joseph Harrington,
Noemi Rebeli
Abstract:
We have investigated the 2009 July impact event on Jupiter using the ZEUS-MP 2 three-dimensional hydrodynamics code. We studied the impact itself and the following plume development. Eight impactors were considered: 0.5 km and 1 km porous (ρ= 1.760 g cm^{-3}) and non-porous (ρ= 2.700 g cm^{-3}) basalt impactors, and 0.5 km and 1 km porous (ρ= 0.600 g cm^{-3}) and non-porous ρ= 0.917 g cm^{-3}) ice…
▽ More
We have investigated the 2009 July impact event on Jupiter using the ZEUS-MP 2 three-dimensional hydrodynamics code. We studied the impact itself and the following plume development. Eight impactors were considered: 0.5 km and 1 km porous (ρ= 1.760 g cm^{-3}) and non-porous (ρ= 2.700 g cm^{-3}) basalt impactors, and 0.5 km and 1 km porous (ρ= 0.600 g cm^{-3}) and non-porous ρ= 0.917 g cm^{-3}) ice impactors. The simulations consisted of these bolides colliding with Jupiter at an incident angle of θ= 69 degrees from the vertical and with an impact velocity of v = 61.4 km s^{-1}. Our simulations show the development of relatively larger, faster plumes created after impacts involving 1 km diameter bodies. Comparing simulations of the 2009 event with simulations of the Shoemaker-Levy 9 events reveals a difference in plume development, with the higher incident angle of the 2009 impact leading to a shallower terminal depth and a smaller and slower plume. We also studied the amount of dynamical chaos present in the simulations conducted at the 2009 incident angle. Compared to the chaos of the SL9 simulations, where θis approximately 45 degrees, we find no significant difference in chaos at the higher 2009 incident angle.
△ Less
Submitted 23 March, 2012;
originally announced March 2012.
-
Enhancing tracer diffusivity by tuning interparticle interactions and coordination shell structure
Authors:
James Carmer,
Gaurav Goel,
Mark J. Pond,
Jeffrey R. Errington,
Thomas M. Truskett
Abstract:
This study uses a combination of stochastic optimization, statistical mechanical theory, and molecular simulation to test the extent to which the long-time dynamics of a single tracer particle can be enhanced by rationally modifying its interactions--and hence static correlations--with the other particles of a dense fluid. Specifically, a simulated annealing strategy is introduced that, when coupl…
▽ More
This study uses a combination of stochastic optimization, statistical mechanical theory, and molecular simulation to test the extent to which the long-time dynamics of a single tracer particle can be enhanced by rationally modifying its interactions--and hence static correlations--with the other particles of a dense fluid. Specifically, a simulated annealing strategy is introduced that, when coupled with test-particle calculations from an accurate density functional theory, finds interactions that maximize either the tracer's partial molar excess entropy or a related pair-correlation measure (i.e., two quantities known to correlate with tracer diffusivity in other contexts). The optimized interactions have soft, Yukawa-like repulsions, which extend beyond the hard-sphere interaction and disrupt the coordination-shell cage structure surrounding the tracer. Molecular and Brownian dynamics simulations find that tracers with these additional soft repulsions can diffuse more than three times faster than bare hard spheres in a moderately supercooled fluid, despite the fact that the former appear considerably larger than the latter by conventional definitions of particle size.
△ Less
Submitted 11 November, 2011; v1 submitted 11 October, 2011;
originally announced October 2011.
-
Mapping between long-time molecular and Brownian dynamics
Authors:
Mark J. Pond,
Jeffrey R. Errington,
Thomas M. Truskett
Abstract:
We use computer simulations to test a simple idea for mapping between long-time self diffusivities obtained from molecular and Brownian dynamics. The strategy we explore is motivated by the behavior of fluids comprising particles that interact via inverse-power-law pair potentials, which serve as good reference models for dense atomic or colloidal materials. Based on our simulation data, we presen…
▽ More
We use computer simulations to test a simple idea for mapping between long-time self diffusivities obtained from molecular and Brownian dynamics. The strategy we explore is motivated by the behavior of fluids comprising particles that interact via inverse-power-law pair potentials, which serve as good reference models for dense atomic or colloidal materials. Based on our simulation data, we present an empirical expression that semi-quantitatively describes the "atomic" to "colloidal" diffusivity mapping for inverse-power-law fluids, but also for model complex fluids with considerably softer (star-polymer, Gaussian-core, or Hertzian) interactions. As we show, the anomalous structural and dynamic properties of these latter ultrasoft systems pose problems for other strategies designed to relate Newtonian and Brownian dynamics of hard-sphere-like particles.
△ Less
Submitted 4 August, 2011;
originally announced August 2011.
-
Implications of the effective one-component analysis of pair correlations in colloidal fluids with polydispersity
Authors:
Mark J. Pond,
Jeffrey R. Errington,
Thomas M. Truskett
Abstract:
Partial pair-correlation functions of colloidal suspensions with continuous polydispersity can be challenging to characterize from optical microscopy or computer simulation data due to inadequate sampling. As a result, it is common to adopt an effective one-component description of the structure that ignores the differences between particle types. Unfortunately, whether this kind of simplified des…
▽ More
Partial pair-correlation functions of colloidal suspensions with continuous polydispersity can be challenging to characterize from optical microscopy or computer simulation data due to inadequate sampling. As a result, it is common to adopt an effective one-component description of the structure that ignores the differences between particle types. Unfortunately, whether this kind of simplified description preserves or averages out information important for understanding the behavior of the fluid depends on the degree of polydispersity and can be difficult to assess, especially when the corresponding multicomponent description of the pair correlations is unavailable for comparison. Here, we present a computer simulation study that examines the implications of adopting an effective one-component structural description of a polydisperse fluid. The square-well model that we investigate mimics key aspects of the experimental behavior of suspended colloids with short-range, polymer-mediated attractions. To characterize the partial pair-correlation functions and thermodynamic excess entropy of this system, we introduce a Monte Carlo sampling strategy appropriate for fluids with a large number of pseudo-components. The data from our simulations at high particle concentrations, as well as exact theoretical results for dilute systems, show how qualitatively different trends between structural order and particle attractions emerge from the multicomponent and effective one-component treatments, even with systems characterized by moderate polydispersity. We examine consequences of these differences for excess-entropy based scalings of shear viscosity, and we discuss how use of the multicomponent treatment reveals similarities between the corresponding dynamic scaling behaviors of attractive colloids and liquid water that the effective one-component analysis does not capture.
△ Less
Submitted 3 September, 2011; v1 submitted 25 July, 2011;
originally announced July 2011.
-
Generalizing Rosenfeld's excess-entropy scaling to predict long-time diffusivity in dense fluids of Brownian particles: From hard to ultrasoft interactions
Authors:
Mark J. Pond,
Jeffrey R. Errington,
Thomas M. Truskett
Abstract:
Computer simulations are used to test whether a recently introduced generalization of Rosenfeld's excess-entropy scaling method for estimating transport coefficients in systems obeying molecular dynamics can be extended to predict long-time diffusivities in fluids of particles undergoing Brownian dynamics in the absence of interparticle hydrodynamic forces. Model fluids with inverse-power-law, Gau…
▽ More
Computer simulations are used to test whether a recently introduced generalization of Rosenfeld's excess-entropy scaling method for estimating transport coefficients in systems obeying molecular dynamics can be extended to predict long-time diffusivities in fluids of particles undergoing Brownian dynamics in the absence of interparticle hydrodynamic forces. Model fluids with inverse-power-law, Gaussian-core, and Hertzian pair interactions are considered. Within the generalized Rosenfeld scaling method, long-time diffusivities of ultrasoft Gaussian-core and Hertzian particle fluids, which display anomalous trends with increasing density, are predicted (to within 20%) based on knowledge of interparticle interactions, excess entropy, and scaling behavior of simpler inverse-power-law fluids.
△ Less
Submitted 10 January, 2011;
originally announced January 2011.
-
Generalized Rosenfeld scalings for tracer diffusivities in not-so-simple fluids: Mixtures and soft particles
Authors:
William P. Krekelberg,
Mark J. Pond,
Gaurav Goel,
Vincent K. Shen,
Jeffrey R. Errington,
Thomas M. Truskett
Abstract:
Rosenfeld [Phys. Rev. A 15, 2545 (1977)] noticed that casting transport coefficients of simple monatomic, equilibrium fluids in specific dimensionless forms makes them approximately single-valued functions of excess entropy. This has predictive value because, while the transport coefficients of dense fluids are difficult to estimate from first principles, excess entropy can often be accurately p…
▽ More
Rosenfeld [Phys. Rev. A 15, 2545 (1977)] noticed that casting transport coefficients of simple monatomic, equilibrium fluids in specific dimensionless forms makes them approximately single-valued functions of excess entropy. This has predictive value because, while the transport coefficients of dense fluids are difficult to estimate from first principles, excess entropy can often be accurately predicted from liquid-state theory. Here, we use molecular simulations to investigate whether Rosenfeld's observation is a special case of a more general scaling law relating mobility of particles in mixtures to excess entropy. Specifically, we study tracer diffusivities, static structure, and thermodynamic properties of a variety of one- and two-component model fluid systems with either additive or non-additive interactions of the hard-sphere or Gaussian-core form. The results of the simulations demonstrate that the effects of mixture concentration and composition, particle-size asymmetry and additivity, and strength of the interparticle interactions in these fluids are consistent with an empirical scaling law relating the excess entropy to a new dimensionless (generalized Rosenfeld) form of tracer diffusivity, which we introduce here. The dimensionless form of the tracer diffusivity follows from knowledge of the intermolecular potential and the transport / thermodynamic behavior of fluids in the dilute limit. The generalized Rosenfeld scaling requires less information, and provides more accurate predictions, than either Enskog theory or scalings based on the pair-correlation contribution to the excess entropy. As we show, however, it also suffers from some limitations, especially for systems that exhibit significant decoupling of individual component tracer diffusivities.
△ Less
Submitted 1 October, 2009;
originally announced October 2009.
-
Composition and concentration anomalies for structure and dynamics of Gaussian-core mixtures
Authors:
Mark J. Pond,
William P. Krekelberg,
Vincent K. Shen,
Jeffrey R. Errington,
Thomas M. Truskett
Abstract:
We report molecular dynamics simulation results for two-component fluid mixtures of Gaussian-core particles, focusing on how tracer diffusivities and static pair correlations depend on temperature, particle concentration, and composition. At low particle concentrations, these systems behave like simple atomic mixtures. However, for intermediate concentrations, the single-particle dynamics of the…
▽ More
We report molecular dynamics simulation results for two-component fluid mixtures of Gaussian-core particles, focusing on how tracer diffusivities and static pair correlations depend on temperature, particle concentration, and composition. At low particle concentrations, these systems behave like simple atomic mixtures. However, for intermediate concentrations, the single-particle dynamics of the two species largely decouple, giving rise to the following anomalous trends. Increasing either the concentration of the fluid (at fixed composition) or the mole fraction of the larger particles (at fixed particle concentration) enhances the tracer diffusivity of the larger particles, but decreases that of the smaller particles. In fact, at sufficiently high particle concentrations, the larger particles exhibit higher mobility than the smaller particles. Each of these dynamic behaviors is accompanied by a corresponding structural trend that characterizes how either concentration or composition affects the strength of the static pair correlations. Specifically, the dynamic trends observed here are consistent with a single empirical scaling law that relates an appropriately normalized tracer diffusivity to its pair-correlation contribution to the excess entropy.
△ Less
Submitted 20 August, 2009;
originally announced August 2009.
-
Available states and available space: Static properties that predict dynamics of confined fluids
Authors:
Gaurav Goel,
William P. Krekelberg,
Mark J. Pond,
Jeetain Mittal,
Vincent K. Shen,
Jeffrey R. Errington,
Thomas M. Truskett
Abstract:
Although density functional theory provides reliable predictions for the static properties of simple fluids under confinement, a theory of comparative accuracy for the transport coefficients has yet to emerge. Nonetheless, there is evidence that knowledge of how confinement modifies static behavior can aid in forecasting dynamics. Specifically, molecular simulation studies have shown that the re…
▽ More
Although density functional theory provides reliable predictions for the static properties of simple fluids under confinement, a theory of comparative accuracy for the transport coefficients has yet to emerge. Nonetheless, there is evidence that knowledge of how confinement modifies static behavior can aid in forecasting dynamics. Specifically, molecular simulation studies have shown that the relationship between excess entropy and self diffusivity of a bulk equilibrium fluid changes only modestly when the fluid is isothermally confined, indicating that knowledge of the former might allow semi-quantitative predictions of the latter. Do other static measures, such as those that characterize free or available volume, also strongly correlate with single-particle dynamics of confined fluids? Here, we study this issue for both the single-component hard-sphere fluid and hard-sphere mixtures. Specifically, we use molecular simulations and fundamental measure theory to study these systems at approximately $10^3$ equilibrium state points. We examine three different confining geometries (slit pore, square channel, and cylindrical pore) and the effects of packing fraction and particle-boundary interactions. Although density fails to predict some key qualitative trends for the dynamics of confined fluids, we find that a new generalized measure of available volume for inhomogeneous fluids strongly correlates with the self diffusivity across a wide parameter space in these systems, approximately independent of the degree of confinement. An important consequence, which we demonstrate here, is that density functional theory predictions of this static property can be used together with knowledge of bulk fluid behavior to estimate the diffusion coefficient of confined fluids under equilibrium conditions.
△ Less
Submitted 19 January, 2009;
originally announced January 2009.
