Publications

We follow an Open Research & Reproducibility policy, because modern CFD research is not reproducible without it. All codes, scripts and logs alongside some derived data used in our publications are available from this website. However, using the codes for publications is not permitted, they are merely a reference. Use WOMBAT-public instead (MIT license) or contact us via email (address in the footer).

WENO-Wombat: Scalable Fifth-Order Constrained-Transport Magnetohydrodynamics for Astrophysical Applications

J. Donnert, H. Jang, P. Mendygral, G. Brunetti, D. Ryu, T. W. Jones

The Astrophysical Journal Supplement Series, Volume 241, Number 2 (Apr. 2019)

Due to increase in computing power, high-order Eulerian schemes will likely become instrumental for the simulations of turbulence and magnetic field amplification in astrophysical fluids in the next years. We present the implementation of a fifth order weighted essentially non-oscillatory scheme for constrained-transport magnetohydrodynamics into the code WOMBAT. We establish the correctness of our implementation with an extensive number tests. We find that the fifth order scheme performs as accurately as a common second order scheme at half the resolution. We argue that for a given solution quality the new scheme is more computationally efficient than lower order schemes in three dimensions. We also establish the performance characteristics of the solver in the WOMBAT framework. Our implementation fully vectorizes using flattened arrays in thread-local memory. It performs at about 0.6 Million zones per second per node on Intel Broadwell. We present scaling tests of the code up to 98 thousand cores on the Cray XC40 machine 'Hazel Hen', with a sustained performance of about 5 % of peak at scale.

Magnetic Field Amplification in Galaxy Clusters and its Simulation

J. Donnert,  F. Vazza, M. Brueggen,  J. ZuHone

Space Science Reviews, Volume 214, Issue 8, 49 pp. (2018)

We review the present theoretical and numerical understanding of magnetic field amplification in cosmic large-scale structure, on length scales of galaxy clusters and beyond. Structure formation drives compression and turbulence, which amplify tiny magnetic seed fields to the microGauss values that are observed in the intracluster medium. This process is intimately connected to the properties of turbu- lence and the microphysics of the intra-cluster medium. Additional roles are played by merger induced shocks that sweep through the intra-cluster medium and motions induced by sloshing cool cores. The accurate simulation of magnetic field amplification in clusters still poses a serious challenge for simulations of cosmological structure formation. We review the current literature on cosmological simulations that include magnetic fields and outline theoretical as well as numerical challenges.

Towards Exascale Simulations of the ICM Dynamo with WENO-Wombat

J. Donnert , H. Jang, P. Mendygral, G. Brunetti, D. Ryu and T. Jones

Proceeding: Galaxies, Special Issue "The Power of Faraday Tomography" (2018)

“In galaxy clusters, modern radio interferometers observe non-thermal radio sources with unprecedented spatial and spectral resolution. For the first time, the new data allows to infer the structure of the intra-cluster magnetic fields on small scales via Faraday tomography. This leap  forward demands new numerical models for the amplification of magnetic fields in cosmic structure formation - the cosmological magnetic dynamo. Here we present a novel numerical approach to astrophyiscal MHD simulations aimed to resolve this small-scale dynamo in future cosmological simulations. As a first step, we implement a fifth order WENO scheme in the new code WOMBAT .  We show that this scheme doubles the effective resolution of the simulation and is thus less expensive than common second order schemes. WOMBAT uses a novel approach to parallelization and load balancing developed in collaboration with performance engineers at Cray Inc. This will allow us scale simulation to the exaflop regime and achieve kpc resolution in future cosmological simulations of galaxy clusters. Here we demonstrate the excellent scaling properties of the code and argue that resolved simulations of the cosmological small scale dynamo within the whole virial radius are possible in the next years.“

WOMBAT: A scalable and high performance astrophysical MHD code,

P. Mendygral , N. Radcliffe , K. Kandalla , D. Porter , B. O'Neill , C. Nolting , P. Edmon , J. Donnert , and T. Jones ,

Astrophys. J., Suppl. Ser. 228, 23–45 (2017).

"We present a new code for astrophysical magnetohydrodynamics specifically designed and optimized for high performance and scaling on modern and future supercomputers. We describe a novel hybrid OpenMP/MPI programming model that emerged from a collaboration between Cray, Inc. and the University of Minnesota. This design utilizes MPI-RMA optimized for thread scaling, which allows the code to run extremely efficiently at very high thread counts ideal for the latest generation of multi-core and many-core architectures. Such performance characteristics are needed in the era of “exascale” computing. We describe and demonstrate our high-performance design in detail with the intent that it may be used as a model for other, future astrophysical codes intended for applications demanding exceptional performance."

Simulations of the Galaxy Cluster CIZA J2242.8+5301 I: Thermal Model and Shock Properties

J. Donnert, A. Beck, K. Dolag and H. J. A. Roettgering

Monthly Notices of the Royal Astronomical Society, Volume 471, Issue 4, p.4587-4605  2017

The giant radio relic in CIZA J2242.8+5301 provides clear evidence of a Mpc sized shock in a massive merging galaxy cluster. Here we present idealized SPH hydrodynamical and col- lisionless Dark Matter simulations, aiming to find a model that is consistent with that large range of observations of this galaxy cluster. We first show that in the northern shock, the observed radio spectral index profile and integrated radio spectrum are consistent with the observed upstream X-ray temperature. Using simulations, we first find that only a cool-core versus non-cool-core merger can lead to the observed elongated X-ray morphology. We then carry out simulations for two merging clusters assuming a range of NFW and β-model den- sity profiles and hydrostatic equilibrium. We find a fiducial model that mimics the overall morphology of the shock structures, has a total mass of 1.6 × 1015M⊙ and a mass ratio of 1.76. For this model the derived Mach number for the northern shock is 4.5. This is almost a factor 2 higher compared to observational determination of the Mach number using X-ray observations or measurements of the radio injection spectral index. We could not find numeri- cal models that both fit the X-ray properties and yielded such low Mach numbers. We discuss various ways of understanding this difference and argue that deep X-ray observations of CIZA J2242.8+5301 will be able to test our model and reconcile the differences.

Magnetic Field Evolution in Giant Radio Relics using the example of CIZA J2242.8+5301

J. Donnert, A. Stroe, G. Brunetti, D. Hoang and H. Roettgering

Monthly Notices of the Royal Astronomical Society, Volume 462, Issue 2, p.2014-2032 (2016)

Giant radio relics are the arc-shaped diffuse radio emission regions observed in the outskirts of some merging galaxy clusters. They are believed to trace shock-waves in the intra-cluster medium. Recent observations demonstrated that some prominent radio relics exhibit a steep- ening above 2 GHz in their radio spectrum. This challenges standard theoretical models be- cause shock acceleration is expected to accelerate electrons to very high energies with a power-law distribution in momentum. In this work we attempt to reconcile these data with the shock-acceleration scenario. We propose that the spectral steepening may be caused by the highest energy electrons emitting preferentially in lower magnetic fields than the bulk of synchrotron bright electrons in relics. We focus on a model with an increasing magnetic field behind the shock, which quickly saturates and then declines. We derive the time-evolution of cosmic-ray electron spectra in time variable magnetic fields and an expanding medium. We apply the formalism on the radio relic in the cluster CIZA J2242.8+5301. We show that under favourable circumstances of magnetic field amplification downstream, our model can explain the observed radio spectrum, the brightness profile and the spectral index profile of the relic. A possible interpretation for the required field amplification downstream is a dynamo acting behind the shock with an injection scale of magnetic turbulence of about 10 kpc. Our models require injection efficiencies of CRe - which are in tension with simple diffusive shock acceleration from the thermal pool. This problem can likely be alleviated considering pre-existing CRe.