Earth System Modeling Mission and Priorities
The Earth System Modeling (ESM) Program supports the development of innovative Earth system modeling capabilities, with the ultimate goal of providing accurate and computationally advanced representations of the fully coupled and integrated Earth system, as needed for energy and related sectoral infrastructure planning. Key examples of critical information for energy include accurate projections of water availability, drought incidence and persistence, temperature extremes including prolonged heat stress, probability of storms, opening of the Arctic Ocean, and sea level and storm-surge at coastal regions. In order to provide this information, considerable effort is needed to develop optimal-fidelity climate and Earth system simulations, with suitably-accurate representation of atmospheric dynamics, clouds and chemistry, ocean circulation and biogeochemistry, land biogeochemistry and hydrology, sea-ice and dynamic land-ice, and in each case including elements of human activities that affect these systems such as water management and land-use.
ESM utilizes the mathematical and computational expertise within the DOE-Laboratories to develop efficient, accurate and advanced algorithms for these Earth system processes, and to improve model initialization, optimal component coupling and uncertainty of system simulation and climate projections. The aim is to optimize the Earth system codes to run efficiently on DOE computers architectures, using modern and sustainable software and workflows, providing the Nation with a high-resolution coupled climate and Earth system simulation capability that is vital for accurately understanding how the climate and earth system evolve and also support DOE responsibilities involving energy planning.
Central to the ESM Program is the Accelerated Climate Modeling for Energy (ACME) project, launched by BER in 2014 to develop a high-resolution Earth system model that efficiently runs at high-resolution on DOE leadership computers, simulating the near-term past (for validation) and future (3-4 decades, to 2050) in support of the DOE science mission. ACME will design and perform high-resolution Earth system simulations, targeting the research community’s more challenging science questions, e.g., involving cloud-aerosol interactions, ice sheet physics, biogeochemistry, hydrology, ocean eddy dynamics, and the interdependence of low frequency variability and extreme weather. Other activities supported by ESM complement and enhance ACME, including the development of potential future-generation ACME capabilities within the Scientific Discovery through Advanced Computing (SciDAC) Program and supporting collaborative and community codes that are developed and used by multiple climate and weather groups.
The ESM Program contributes to the U.S. Global Change Research Program (USGCRP), and coordinates its activities with the climate modeling programs at other federal agencies, particularly the National Science Foundation (NSF) through the CESM project, the National Oceanic and Atmospheric Administration (NOAA), and the National Aeronautics and Space Administration (NASA).
Earth System Science and BER programmatic collaborations
ESM supports the development of all essential components of the coupled Earth-human system needed to simulate Earth system and climate by DOE’s research community. Each component (atmosphere, ocean, cryosphere and land) use advanced variable-resolution grids, allowing ultra-high resolution information and process resolution within particular regions of interest. One example is the placement of a very high resolution (e.g. 10 km) atmosphere near the location of a DOE ARM site, in order to overlap the global-model’s cloud-resolving capability with higher-resolution Large-Eddy-Simulations (LES’s) and with the ARM data; this integration of high resolution models with LES and ARM observations serves as a means of studying cloud processes and the coupling with the global system. Another example is the placement of high resolution modeling capabilities in the ocean surrounding Antarctica and along the margins of the Antarctic ice sheet. Without such high resolution, predictions would be unable to represent many of the critical processes controlling future change, e.g., involving the flow of the ocean up under the ice-sheet, and the dynamics in the ice sheet where the ocean and ice meet. A third example is configuring the land model using basin-gridding instead of rectangular grids in order to effectively study the water flow and supply changes within specific basins.
For global atmospheric models, ESM partners with BER’s Atmospheric System Research (ASR) Program to develop cloud and aerosol parameterizations needed to understand how clouds are shifting and influencing climate sensitivity. The multi-scale Climate Model Development and Validation (CMDV) Program also supports projects that study cloud and aerosol processes, spanning scales from Large-Eddy Simulation (LES) scale to global-model scale, and validating the simulations using Atmospheric Radiation Measurement ARM Facility data as well as other measurements.
For land modeling, ESM collaborates with Terrestrial Ecosystem Science to develop global land model parameterizations, incorporating modeling capabilities that are based on TES field investigations within particular regions. The Climate Model Development and Validation also supports liaisons that interconnect the modeling activities between ACME and the Next Generation Ecosystem Experiments (NGEE) in the Tropics and the Arctic. These joint efforts are contributing new hydrologic and dynamic ecosystem representations to the ACME model, using validation from field investigations.
ESM and Integrated Assessment Research (IAR) collaborate on the development of coupled human-natural systems, such as understanding how land and water management activities affect the Earth system, as needed for optimal detailed Earth system simulations. An important example is the alignment and coordination between the ACME and the Global Change Assessment Model (GCAM) integrated assessment model. The Integrated Earth System Model project directly coded and coupled the interactions between the climate model CESM and the integrated assessment model GCAM through the terrestrial-carbon cycle. The code from this project will be released to the community in 2017, and will be included in the ACME model for further development.
ESM and Regional and Global Climate Modeling (RGCM) are complementary Programs, with ESM supporting primarily model development, and RGCM focused on model analysis, intercomparison, metrics and validation, as well as using the models to evaluate system sensitivities and feedbacks. Important synergies include high-latitude modeling and research, biogeochemical feedbacks and the international Land Modeling Bencharking (iLAMB) project, use of climate-modeling metrics and diagnostics (e.g. Program for Climate Model Diagnosis and Intercomparison (PCMDI) metrics for ACME), and the use of high-resolution models such as ACME to study extremes.
Earth System Computation, and programmatic ASCR and BER collaborations
Essential to ESM is its collaboration with the Advanced Scientific Computing Research (ASCR) Office, in particular the SciDAC partnership program. SciDAC supports partnership between ASCR and the other Office of Science offices, in order to dramatically accelerate progress in scientific computing. Examples of advances in the BER-ESM-SciDAC projects include the development of: variable-resolution Earth system components, algorithms and mathematical methods to improve efficiency and accuracy of Earth system component simulation on advanced computational architectures, uncertainty characterization of modeling systems, and advances in code performance and portability.
Current BER-ESM-SciDAC projects are developing future capabilities needed in the ACME model, including next-generation dynamic and variable-resolution ice sheet models (Predicting Ice Sheet and Climate Evolution at Extreme Scales (PISCEES)), improved treatments of atmospheric convection and physics, and oceanic eddies that apply across their variable-mesh atmosphere and ocean components (Multiscale Methods for Accurate, Efficient, and Scale-Aware Models of the Earth System), and the next-generation version of ACME atmosphere, which will include non-hydrostatic dynamics, as needed when the model approaches very high-resolution (less than 10km) (A Non-hydrostatic Variable Resolution Atmospheric Model in ACME).
In order for ACME to develop sustainable and portable codes, state-of-science software development methods are important. The Climate Model Development and Validation Program’s ACME-SM: A Global Climate Model Software Modernization Surge is transforming ACME’s atmospheric and coupler codes, introducing new infrastructure capabilities and thorough and improved testing approaches within some of the most critical sub-modules. These developments are expected to be of broad benefit to Earth system as well as other complex, distributed and advanced computational modeling efforts.
Since ACME will be producing very-high resolution, and high frequency (sub-daily) model output, workflows to manage (download, move, store and analyze) large model outputs are needed. ESM collaborates with BER’s Data Management (DM) on the infrastructure and tools needed for managing large model ouput datasets. The Earth System Grid Federation (ESGF), supported by DM, is critical for hosting and sharing the data generated by ACME and other climate models. ACME’s support of UV-CDAT and similar model analytic tools are also of use to ESGF user communities.
The Accelerated Climate Modeling for Energy (ACME) project
The Accelerated Climate Modeling for Energy (ACME) project is central to ESM as well as many of the Climate and Environmental Sciences Division activities, as it is developing a computationally advanced coupled climate-energy model to investigate the challenges posed by the interactions of weather-climate scale variability with energy and related sectors. The ACME model simulates the fully coupled Earth system at high-resolution (15-25km, including higher resolution within regionally refined areas) and is incorporating coupling with energy, water, land-use and related energy-relevant activities, with a focus on near-term hind-casts (1970-2015) for model validation and a near-term projection (2015-2050) as needed for energy sector planning. The model further employs regional-refinement using variable mesh methodologies designed to provide high resolution in regions where the complex physical and dynamical processes require it, or where more detailed information is desired. The project is led by a collaboration of several DOE-National Laboratories and includes several academic and private partners. While ACME’s primary purpose is for scientific research, it will be available to support planning for National energy and related sectoral needs, for example by indicating the probability for regional changes in extreme temperature and precipitation, water availability, sea-level change and coastal impacts, Arctic ocean accessibility, and carbon exchange across atmosphere, land and ocean systems.
ACME’s scientific goals address three areas of importance to both climate and earth system research:
- Water cycle. The key water cycle question is: “How do the hydrological cycle and water resources interact with the climate system on local to global scales?” Understanding and developing the capability to project the evolution of water in the Earth’s systems is of fundamental importance both to climate-science and to societal and many energy-related processes, including coal-, nuclear-, biofuel-, and hydro-power potentials. Using river flow as a key indicator of hydrological changes from natural and human systems, ACME is testing the hypothesis that changes in river flow have been historically dominated by land management, water management, and aerosol forcing, but will shift to be increasingly dominated by greenhouse gas changes in coming decades. The initial phase of the project focuses on simulation of precipitation and surface water in orographically complex regions, including the western United States and Southeastern Asia. The longer-term goal is to understand how the hydrological cycle in the fully coupled climate system will evolve with climate change and the expected effect on local, regional, and national supplies of fresh water.
- Biogeochemistry. The key biogeochemistry question is: “How do biogeochemical cycles interact with global climate change?” The degree of carbon exchange among components is important for investigating human influences on atmospheric carbon dioxide, methane and elemental carbon particle concentrations, yet this exchange is in turn affected by climate change and nutrient availability. The early phase of ACME is examining how more complete treatments of nutrient cycles affect carbon–climate system feedbacks. ACME is adding phosphorus to its below-ground carbon-nitrogen nutrient system, since P availability may limit, e.g., tropical ecosystem production, and may play an important role in regulating global-scale feedbacks. Experiments will investigate the nutrient and climate interactions for the preindustrial through the 21st century. A longer-term goal is to study interactions between land and coastal ecosystems, combining coastal-zone biogeochemical cycling and its interaction with the silt, nutrients, and other substances transported by rivers and runoff.
- Cryosphere-ocean system. The key cryosphere-ocean question is: “How do rapid changes in cryosphere-ocean systems interact with the climate system?” As ACME builds and couples new dynamic ice sheet and ocean components, it will simulate the potential for ice sheet melt, destabilization and sea-level rise. Simulations will utilize ACME’s variable-mesh capabilities to enhance resolution in the ocean near the ice sheet and in active regions of the ice sheets, with particular focus on Antarctica. The Model Prediction Across Scales project, or MPAS-Ocean, will provide a new capability to dramatically influence the ability to resolve eddies to better represent the circumpolar deep water and dynamics associated with bringing this water onto the continental shelf under the ice sheet, with ocean model resolution attaining 5 km or less near the ice sheets, and the ice sheet resolution up to 500 m near the margins. Sea ice modeling is also crucial to capture the processes of buttressing at the ice shelf-sea ice boundary, including the development of ice calving dynamics and iceberg models. In the fully coupled system, climatic changes that influence the atmospheric general circulation will also influence the behavior of the Southern Ocean and sea ice. In the long-term, ACME will include components required to simulate impacts of sea-level change and storm surge on coastal regions, including wave models and focusing resolution in coastal and storm-track regions.
A major motivation for the ACME project is the paradigm shift in computing architectures and their related programming models as computational capabilities move towards the exascale era. DOE, through its science programs and early adoption of new computing architectures, traditionally leads many scientific communities, including Earth system simulation, through these disruptive changes in computing.
ACME is optimizing Earth system code performance for current and next-generation DOE computer facilities, particularly those at the Argonne Leadership Computing Facility, the Oak Ridge Leadership Computing Facility, and the Lawrence Berkeley National Laboratory National Energy Research Scientific Computing Center. To use these machines, the climate codes must support stricter memory management and more complex thread management. The ACME performance “gold-standard” is to maintain a coupled–model speed of five simulated years per wall-clock day, even while moving to higher resolution. ACME focuses on exposing increased concurrency throughout the model and on increasing the on-core performance of key computational kernels. Initially the project is implementing conventional approaches, such as threading and message-passing while increasingly employing the use of on-processor accelerators added in the latest machine designs. Redesigning code for better concurrency through the use of modularized kernels for accelerators will be beneficial for most envisioned exascale architectures. In the longer-term, ACME will explore dynamic auto-tuning and load balancing to minimize latency and make the model resilient to system disruptions anticipated on exascale architectures.
An important aspect of adaptation to new architectures is a substantial effort to improve software design and practice. Early priorities for ACME software engineering include maintaining build, test, and performance tools for the relevant computer platforms, and providing rapid development and debugging capabilities to the team. The ACME code repository both expedites the merging and testing of the fully coupled system and supports a distributed development environment where separate features are being co-developed at different sites. In the longer-term, ACME will expand use of regression testing, tools for code coverage, correctness analysis, debugging at scale, and traceability of code back to scientific requirements. Productivity will be enhanced by greater use of libraries, frameworks, and tools.
ACME also has a substantial workflow effort, to enable and automate model simulation, post-processing, analysis and validation. Building from the Ultrascale Visualization Climate Data Analysis Tools (UV-CDAT) software, ACME component and coupled simulation output will be processed on a single workflow platform. Importantly, the workflow software can accommodate the very large data sets from the ACME high-resolution simulations and it will enable “server-side” analysis of output rather than requiring porting of output to local machines. The analysis provenance will be captured, to enable replication of the process. Model output will be hosted and shared through the Earth System Grid Federation, using a Climate Model Intercomparison -friendly format. Model evaluation is initially based on well-established metrics developed by leading climate modeling centers. Availability of new observations, a focus on the ACME driving questions, and emphasis on high-resolution require development of new diagnostics and metrics. Metrics are being established that will track model improvement and realism of the coupled system.
Additional information is available on the ACME project website.
Community Projects supported by ESM
ESM supports many code and component developments that are used by multiple modeling groups, and in some cases supported jointly with other sponsors.
The Common Infrastructure for Modeling the Earth (CIME) is jointly developed between ACME and the CESM software engineering groups, to provide various tools and infrastructure for the CESM and ACME models.
The sea-ice (CICE) model is a collaborative activity led by scientists at Los Alamos National Laboratory together with scientists from several other climate and operational modeling centers and groups.
Ultrascale Visualization (UV-CDAT) is a software package for analysis, visualization and management of Earth system model output. It is part of the ACME project, but is used broadly and co-sponsored by NASA as well as DOE.
The Functionally Assembled Terrestrial Ecosystem Simulator (FATES) is a tropical ecosystem model under development as part of the Terrestrial Ecosystem Sciences (TES) Next Generation Ecosystem Experiment (NGEE) – Tropics and is co-supported by ESM.
The MARBL: Marine Biogeochemistry Library (led by Long, NCAR) is developing a modular ocean biogeochemistry (BGC) capability for use in both ACME and the Community Earth System Model (CESM), and to include options of BGC complexity as needed for a range of research projects.
The Community Emissions Data System (CEDS) (led by Smith, PNNL) is establishing a comprehensive emissions database for Earth system models, with a focus on short-lived species such as aerosols and ozone precursors, with the emissions subdivided by processes in a manner that will allow evaluation of emissions uncertainty.
The Cloud Layers Unified by Binormals (CLUBB) is a cloud and turbulence parameterization used in the ACME model as well as other Earth system models, and has been supported by NSF and NOAA as well as DOE.
Examples of Additional Projects Supported by ESM
Launching an Extreme-scale ACME Prototype-Transport (LEAP-T): (Led by Jones, LANL) to improve atmosphere and ocean computational performance and portability, with a particular focus on tracer-transport algorithm development.
Fire, Dust, Air and Water: Improving aerosol biogeochemistry interactions in ACME (Led by Mahowald, Cornell University) is developing and coupling iron, phosphorus and nitrogen sources and ecosystem effects in the atmosphere, land and ocean.
Improving the Interface Processes in the DOE/ACME Model (Led by Zeng, University of Arizona) is improving the accuracy and efficiency of coupling between various components in ACME, such as land-atmosphere and ocean-atmosphere.
Major Improvements on the Longwave Radiative Interactions Between Surface and Clouds in the Polar Regions in Atmospheric Global Circulation Model (GCM) (led by Huang, University of Michigan) is improving the long-wave radiative treatment for Earth system models, particularly in the polar regions and regions with bright surfaces.
Enabling Chemistry of Gases and Aerosols for Assessment of Short-Lived Climate Forcers: Improving Solar Radiation Modeling in the DOE-ACME and CESM Models (led by Prather, University of California-Irvine) is developing computationally efficient photochemistry and radiative transfer important for clouds and chemistry in the short-wave.
Modeling Long-Term Changes in Climate, Ice Sheets and Sea Level: Using the Paleo Record to Understand Possibilities for the Future(led by Otto-Bliesner, NCAR) is simulating dynamic ice sheets for very long periods, past and future, to simulate and study long-term changes in ice-sheets and sea-level.
Additional information about ESM is on the DOE climate modeling activity’s website.
Dr. Dorothy Koch
Climate and Environmental Sciences Division, SC-23.1
Department of Energy, GTN Bldg.
1000 Independence Ave, SW
Washington, DC 20585-1290
Phone: (301) 903-0105
Fax: (301) 903-8519