ASCR Monthly Computing News Report - March 2009
In this issue...
Noted PNNL Computational Scientist Jarek Nieplocha Dies
Jaroslaw "Jarek" Nieplocha, who led the Advanced Computing Technology Laboratory at Pacific Northwest National Laboratory, passed away in Richland, WA, on March 21, 2009. Niepolocha joined PNNL in 1993 and was a Laboratory Fellow and Deputy Director in the Computational Sciences and Mathematics Division, as well as the Chief Scientist for High Performance Computing in the division.
Nieplocha's research was in collective and one-sided communication on modern networks, runtime systems, parallel I/O, scalable programming models, multithreading, and component technology for scientific computing. He led development of Global Arrays, a portable shared memory programming toolkit widely used in scalable computational chemistry applications and other areas, the ARMCI runtime system, and GPSHMEM, the popular one-sided communication library developed by Cray, Inc. He worked with domain scientists on applying high-performance computing to several scientific disciplines including chemistry, biology, and electrical power grid. He received four best paper awards at leading conferences in high performance computing and an R&D-100 award for one of the top 100 scientific innovations. He shared a patent on event signaling in computer networks with IBM and authored or coauthored over 100 peer reviewed papers.
As an adjunct faculty in Computer Science, Nieplocha taught at Washington State University. He participated in MPI Forum in defining the MPI-2 standard. He was president of the Hewlett Packard Consortium for Advanced Scientific and Technical Computing as well as a member of editorial board of the International Journal of Computational Science and Engineering.
Advanced Networking Initiative: Research Project to Establish 100 Gbps Prototype
The Advanced Networking Initiative, a $69 million ASCR initiative funded under President Obama's American Recovery and Reinvestment Act, will establish a nationwide advanced network research demonstration prototype with 100 Gigabit per second (Gbps) throughput capability. The prototype will embrace at least four major sites, including the major SC Office of Advanced Scientific Computing Research (ASCR) computing facilities at Argonne National Laboratory, Oak Ridge National Laboratory and Lawrence Berkeley National Laboratory, along with the New York multi-agency peering point and gateway to transatlantic optical fiber.
This demonstration prototype has the potential to revolutionize the way scientists communicate in their day-to-day work and would have transformational impacts on commercial network providers. The effort will complement ASCR's Energy Sciences Network (ESnet), which is the primary network infrastructure connecting the distributed research elements in DOE. The current ESnet backbone supports multiple links at 10Gbps (the current state of the art) to exchange scientific data. However, the forecasts for network demand currently exceed the capacity available. The demonstration prototype will be complemented by an enhanced network research effort on critical network management and control plane issues for optical networks.
Through its partnerships with commercial vendors, this initiative can significantly enhance the competitiveness of the U.S. telecom industry by facilitating the creation of a next-generation optical backbone. The new technologies growing out of this initiative would enable telecom vendors to provide ten times the capacity without replacing the existing fiber plant. This would have substantial impact in urban areas, where conduit space is limited, in transatlantic fiber services, and in rural areas, where it would increase the value of installing fiber in sparsely populated areas.
Strength in Numbers: 15 LBNL Researchers Contribute to SIAM CSE09 Conference
Fifteen researchers from Lawrence Berkeley National Laboratory’s (LBNL’s) Computational Research Division (CRD) contributed in various ways to the SIAM Conference on Computational Science and Engineering held March 2–6 in Miami, Florida. Juan Meza was on the program committee, and David Bailey gave an invited talk on “Experimental Mathematics, Multicore Processors and Highly Parallel Computing.”
Tony Drummond of NERSC co-organized two mini-symposiums and gave three talks. The mini-symposiums were “Fostering Educational Activities for the Computational Science and Engineering Community,” at which he and Osni Marques gave the opening talk, “Outreach and Education Through Computation”; and “Multiphysics Modeling: Frameworks and Applications,” including his talk on “A Scalable Coupling Toolkit for Multiresolution Models.” Drummond’s third talk was “Use of High-Level User Interfaces for Software Sustainability.”
Other CRD researchers who organized, presented, or co-authored at CSE09 include:
Sandia Presents Novel Data Partitioning Scheme at SIAM CSE Conference
Sandia researchers have developed a novel data partitioning scheme to enable the representation of random variables with spectral polynomial chaos representations. The method uses k-center clustering to partition the data set of samples of the random variable into sets with unimodal distributions. These subsets are then further adaptively refined to improve the quality of the spectral representation, as measured by the Kullback-Leibler distance between the original data set and the sampled spectral representation. The resulting representation is a polynomial chaos mixture model. The approach has been demonstrated on stochastic reaction networks and presented at the SIAM Computational Science and Engineering conference in Miami, FL, March 2009. It is also the subject of a paper submitted to the SIAM Journal on Scientific Computing.
Using Neural Modeling to Gain New Insights into Epilepsy
To enrich our understanding of why epileptic seizures occur and how they propagate, scientists at Argonne National Laboratory have created a sophisticated model based on neural networks to model brain activity. Neural networks are not new, but traditionally they have treated each neuron as a fixed entity existing in only one of two states: firing or inactive. The more sophisticated model devised by the Argonne researchers treats each neuron as a data chain, where each link represents a different physical site on the cell. Moreover, the model classifies the neurons in the network into six categories based on their actual neurophysical role, and it sorts the connections between the cell bodies of different neurons into 32 types based on their electrical and chemical properties. Already, the model has produced results that call into question some commonly held assumptions about how epileptic seizures arise. In particular, scientists have traditionally linked the onset of seizures to overexcitation of the brain's network. However, the Argonne model shows more epileptiform activity when the neurons have a lower excitation strength.
Modeling of Microbial Pathways Could Lead to New Fuels, Environmental Cleanup
One key to utilizing microbes for applications such as biofuels production or environmental clean-up is understanding the complex metabolic pathways of these organisms. Computational modeling and simulation are among the latest in the arsenal of tools scientists have for probing the complex biochemical interactions that take place in living cells, providing new insights into the functioning of these organisms and a foundation for optimizing microbes for DOE mission-related applications. Costas Maranas of Pennsylvania State University and his colleagues have done pioneering work using computational methods to reconstruct pathways involving 40 percent of the genes of one of the smallest known self-replicating organisms. Their computational model of this organism, designated iPS189, includes 262 biochemical reactions and 274 metabolites and captures 87 percent of the genes essential for the microbe’s function. This work provides a roadmap for the automated construction of computer-based metabolic models for other organisms important for DOE mission needs. Details can be found in the journal PLoS Computational Biology, February 2009, Vol. 5, Issue (2). This work was jointly sponsored by DOE's Office of Biological and Environmental Science and the Office of Advanced Scientific Computing Research.
ANL Researchers Use Advanced Optimization to Cut Energy Consumption Cost
Researchers in Argonne’s Mathematics and Computer Science Division have developed a computational framework for the optimal management of integrated energy systems whose performance depends heavily on the environment. The approach is based on stochastic optimization and uncertainty quantification of weather prediction systems. The researchers applied stochastic optimization techniques to the thermal management of a model building in Pittsburgh, a choice based on the availability of data. By exploiting the day/night price differentials, the researchers were able to shift the energy load to nighttime to obtain a 25-75% cost reduction over current energy management approaches, while ensuring the satisfaction of operational, economic, and environmental constraints for all uncertainty scenarios. The Argonne researchers also constructed a model for uncertainty in data that is grounded in the underlying physical processes and removes the need for excessive empirical approximations; error localization techniques are also used to improve the forecast performance. The techniques, when applied to the Pittsburgh building, accurately rep-resented ambient predictions. The researchers hope to extend these investigations to determine whether the new computational framework achieves similar results in other buildings.
Sandia Researchers Publish on Least Squares Finite Element Methods
Applied math researcher Pavel Bochev submitted the paper “A Locally Conservative Least-Squares Finite Element Method for the Stokes Equations,” co-authored with Max Gunzburger (Florida State University), to a Springer Lecture Notes in Computer Science volume containing proceedings of the 7th Conference on Large-Scale Scientific Computing in Sozopol, Bulgaria. Bochev and Denis Ridzal also submitted the paper “Additive Operator Decomposition and Optimization-Based Reconnection with Applications” to the same volume. Finally, the book Least-Squares Finite Element Methods by Bochev and Gunzburger will appear on March 27th as volume 166 in the Springer Verlag Applied Mathematical Sciences Series.
PNNL Making Headway in Modeling and Simulation for the Subsurface
Researchers at PNNL are making headway in two projects under the ASCR-funded Science Application Partnerships. Under the “Component Software Infrastructure for Achieving High Level Scalability in Groundwater Reactive Transport Modeling and Simulation” project, researchers are refining a high-performance computational framework based on the Common Component Architecture (CCA). A smoothed particle hydrodynamics (SPH) capability has been extended from its initial implementation to support diffusive transport of species, and work is under way to incorporate reactions important in subsurface transport of radionuclides. PNNL researchers also have reached a major milestone by incorporating a continuum subsurface code into the framework.
Also through the ASCR-funded Science Application Partnership, PNNL researchers have developed a prototype modeling framework to support simplified organization, tracking, and analysis of studies composed of hundreds of simulations. The work is part of the "Process Integration, Data Management, and Visualization for Subsurface Simulations" project. The current framework supports running two models: the Subsurface Transport over Multiple Phases (STOMP) continuum code, and the CCA-based SPH code. The framework, however, is not specific to these codes. In addition, three visualizations tools have been integrated into the framework—TecPlot, General Mesh Viewer, and VisIt. With the integration of VisIt, millions of particles can be visualized through remote parallel visualization, thus avoiding transfer of large data files for analysis. The framework is currently being applied to a suite of SPH validation simulations.
HPCC RandomAccess Benchmark Highlighted in IPDPS Best Paper
HPCC RandomAccess Benchmark for Next Generation Supercomputers received Best Paper in the applications research track of the IEEE International Parallel & Distributed Processing Symposium. The paper examines the key elements determining the performance of the HPC Challenge RandomAccess benchmark on next generation supercomputers, including the Blue Gene/P at the Argonne Leadership Computing Facility (ALCF). Authors Vikas Aggarwal, Yogish Sabharwal, and Rahul Garg from the IBM India Research Lab and Philip Heidelberger from the IBM T.J. Watson Research Center presented a multi-core algorithm that distributes the benchmark workload on the quad-cores of each Blue Gene/P node. The algorithm gave a factor of approximately 3 speedup on the Blue Gene/P system. The ALCF won the HPCC Challenge Award at SC08 using this implementation on the Blue Gene/P. However, the load-balancing algorithm is general and can be applied to any system with k-cores by super-imposing a k-1 dimensional virtual torus topology on the system.
ORNL’s Zacharia Takes Science and Technology Posts
Thomas Zacharia, ORNL’s associate lab director for computing and computational sciences (CCSD), has been named deputy laboratory director for science and technology (S&T) and UT-Battelle senior vice president for S&T. As deputy lab director, Zacharia will coordinate ORNL’s major research and development programs, overseeing one of the nation’s largest R&D programs with annual expenditures in excess of $1.3 billion. His new duties begin the first of April. Zacharia’s appointment is part of a strategic reorganization of resources at ORNL.
Jeff Nichols, deputy associate lab director for CCSD, assumes the interim associate laboratory director position, while an international search is conducted for Zacharia’s replacement. Nichols currently leads efforts to deploy next-generation supercomputers for DOE, the National Science Foundation (NSF), and the Department of Defense (DOD).
Key Hires Boost Computational Leadership at Pacific Northwest National Laboratory
John Feo has been selected to be the director of the Center for Adaptive Computing Software (CASS-MT) at PNNL. The newly established center is exploring next-generation multithreaded architectures for high-performance graph analytics for knowledge discovery applications. Feo, who joined PNNL on March 16, comes from the Microsoft Corporation where he led the Application Framework Group in the Parallel Computing Platform Division. The division is responsible for defining and developing Microsoft's computing platform for many core processors. Feo was responsible for technical direction, identifying and communicating requirements, and choosing the best partners to work with. He also participated in several company-wide strategic initiatives on parallel computing. Prior to Microsoft, Feo held leadership positions at Cray, Inc. and Sun Microsystems. Feo has a Ph.D. in computer science from the University of Texas, Austin, and a master's degree in astronomy.
John Johnson has been selected to lead business development for technical computing for national security applications and will lead one of the new tasks in CASS-MT. Johnson has leadership experience in high performance computing and was a research program manager and group leader with the Center for Applied Scientific Computing (CASC) at Lawrence Livermore National Laboratory (LLNL). Johnson's focus at LLNL was on developing research programs to bring the Lab's high performance computing expertise to bear on national security problems in the data sciences. In this role, he worked with leaders of the defense community, academia, and industry to design and launch innovative programs demonstrating the profound power that scalable, high performance computing brings to this new class of problems. Johnson earned a master's in computer science from Johns Hopkins University and is currently working on his doctorate in computer science from the University of Chicago.
Sandia, ASCR Researcher Tamara Kolda Chairs SIAM CSE Group
Tamara Kolda (Sandia National Labs/CA) has been elected Chair of the SIAM Activity Group on Computational Science and Engineering (CS&E), which fosters collaboration and interaction among applied mathematicians, computer scientists, domain scientists and engineers in those areas of research related to the theory, development, and use of computational technologies for the solution of important problems in science and engineering. The activity group promotes computational science and engineering as an academic discipline and promotes simulation as a mode of scientific discovery on the same level as theory and experiment. The activity group organizes the biennial SIAM Conference on Computational Science and Engineering and maintains an electronic discussion group. This activity group is the largest in SIAM with 1,739 members at the end of 2008. The web page is: http://www.siam.org/activity/cse
Sandia's John Shadid Gives Plenary Presentation at SIAM-CSE Conference, Next Stop Japan
John Shadid of Sandia National Laboratories was recently selected – on very short notice – to give a replacement plenary presentation at the SIAM Computational Science and Engineering Conference held March 1-6 in Miami Fla. The focus of Shadid’s talk was on robust and efficient solution methods for multiple-time-scale multiphysics systems with examples taken from modeling magneto-hydrodynamic and transport/reactions systems.
Next, he will travel to Tokyo to present two invited talks at the 15th International Conference on Finite Elements in Flow Problems. The first talk, entitled “Solution Methods for Multiple-Time-Scale Multiphysics Systems,” is a keynote lecture in the “Coupled Flow and Transport Phenomena” minisymposium. Shadid’s second invited talk is entitled “A Comparison of a High-Resolution FCT and Godunov Method for Compressible Euler on a Set of Challenging Prototype Problems” that will be presented in a minisymposium on “Advanced Methods for Compressible Flow and Shock Hydrodynamics Computations.”
Shadidis a distinguished member of the technical staff in the Computational Science R&D Group at SNL and is a PI in the Department of Energy’s Office of Science Applied Mathematics Research program.
LBNL's David Bailey to Give Two Invited Talks at Conferences in Germany
David Bailey, chief technologist for Berkeley Lab’s Computational Research Division, will be traveling to meetings in eastern Germany in March and May to give invited talks. At the Computer Algebra and Particle Physics meeting to be held March 29-April 3 in Zeuthen, Bailey, one of 11 presenters in the “DESY School,” will give a talk on “Arbitrary Precision Numerics and the PSLQ Algorithm." In May, Bailey will be in Dresden to give an invited talk at the International Workshop on Random Geometry and Random Matrices: From Quantum Gravity to Econophysics. The meeting will be held May 18-22 at the Max-Planck Institute for Physics of Complex Systems.
PNNL's Xin Sun Receives Alumna Award
Pacific Northwest National Laboratory computational scientist Xin Sun received the 2009 Alumni Society Merit Award from the Department of Naval Architecture and Marine Engineering at the University of Michigan. The award recognizes Dr. Sun for her outstanding professional achievements and contributions to society.
Sun has a broad range of experience in the areas of applied mechanics and computational materials. Her expertise lies in applying the mechanics and materials basic principles in solving practical engineering problems associated with solid oxide fuel cell design and analyses, advanced laminated armor materials development, joining and forming of advanced lightweight materials for automotive and heavy vehicle applications, advanced high strength steel modeling development, and lightweight automotive glazing design and development.
Sun, a staff scientist in PNNL’s Computational Sciences & Mathematics Division, received her undergraduate degree in naval architecture and ocean engineering from China’s Shanghai Jiao Tong University in 1990. She then went on to earn two master’s degrees and her doctorate from the University of Michigan at Ann Arbor.
PNNL Scientists Provide Leadership for International Workshop on HEC
Computational scientists from Pacific Northwest National Laboratory are playing key roles in an international workshop focusing on the importance of software in high-end computing (HEC). The Second International Workshop on Parallel Programming Models and Systems Software for High-End Computing (P2S2) will be held in Vienna, Austria, in September 2009. PNNL’s Abhinav Vishnu is the workshop co-chair while fellow colleague Sriram Krishnamoorthy is on the program committee. As co-chair, Vishnu will be adjudicating the best overall papers to be recommended for a special issue of the International Journal of High Performance Computing Applications.
Now That's Entertainment: LBNL's David Bailey Cited for Most Entertaining Talk
LBNL’s David Bailey and his frequent collaborator Jonathan Borwein have been awarded a certificate from the Prize Committee for the “Most Entertaining Talk” at the Third Workshop on High-Dimensional Approximation held Feb. 16-20 at the School of Mathematics & Statistics of the University of New South Wales in Sydney, Australia. In back-to-back sessions, Bailey and Borwein presented “High-precision, high-performance numerical integration” parts I and II.
The workshop covered current research in all aspects of high dimensional integration and approximation. The topic is linked with (in no specific order): Monte Carlo and quasi-Monte Carlo methods, information-based complexity, sparse grid methods, tensor-product approximation and many more. According to Markus Hegland, one of the judges awarding the prize, “While it was done seriously there was a tongue-in-cheek component, especially since the prizes were TimTams (a popular Australian biscuit).”
ESnet Update: Hardware Upgrades to Metro Area Networks, Links to Sites
Throughout February and March, ESnet Engineers continued to upgrade hardware at its regional network sites across the country. Upgrades to the Bay Area Metropolitan Area Network (BAMAN) were completed earlier this month when Cisco routers at NERSC’s Oakland Scientific Facility and the DOE Joint Genome Institute in Walnut Creek were replaced with Juniper hardware. The new routers enable ESnet engineers to provide these sites with additional Science Data Network (SDN) circuits. The Chicago Metropolitan Area Network (CHI-MAN) was also enhanced as new routers were installed at Fermilab. These upgrades provide redundant hardware and a foundation for additional optical 10-gigabit interface circuits to be installed in the future. Each 10-gigabit line is capable of transferring the equivalent of 500-hours of digital music per second, which is essential for connecting DOE researchers to future large science experiments like the Large Hadron Collider. The same routers are currently being installed at Brookhaven National Laboratory (BNL), and an upgrade is planned for Argonne National Laboratory in April.
The Long Island Metropolitan Network (LIMAN) received two new 10-gigabit waves between ESnet hubs in New York City and BNL. Meanwhile, an additional 10-gigabit wave was added between the hubs in New York City. This doubles the LIMAN capacity from 20 to 40-gigabits. In February, one new 10-gigabit connection was added between the ESnet hub in Boston and MIT. This gives MIT researchers a shorter and additional route to the ESnet backbone. This new connection also enhances the current path between MIT to the hub in New York City. Lastly, a new DS3 circuit installed in March at the Oak Ridge Associated Universities (ORAU) in Tennessee provides a backup connection to the ESnet Washington D.C. hub in case the primary path to this site is down.
OUTREACH & EDUCATION:
LBNL's Meza, Drummond and CSGF's Chung a Big Hit at College Career Session in Miami
As part of the SIAM Conference on Computational Science and Engineering held March 2–6 in Miami, Florida, Juan Meza and Tony Drummond of Berkeley Lab and Julianne Chung, a DOE Computational Science Graduate Fellow who did her practicum at LBNL, participated in a career panel at Miami Dade College that was sponsored by the Krell Institute. The students were from the Miami Dade College Tools for Success program, an NSF-funded program aimed at increasing the number of under-represented students completing undergraduate degrees in science, technology, engineering, and math (STEM). After the March 4 panel session, Mary Ann Leung of the Krell Institute wrote, “Thanks so very much for participating in the Career Panel yesterday.....you guys were awesome. I know that the students really benefited from hearing your stories and I appreciate you sharing them with everyone. Everyone I spoke to at MDC said this was one of the best forums they have ever had and that it was exactly what their students needed".
ALCF Workshop Targeted at Non-INCITE Project Users
The Argonne Leadership Computing Facility (ALCF) held an Introduction to BG/P Workshop for non-INCITE project users on March 10-11 at Argonne National Laboratory. The workshop provided these users with an overview of ALCF services and resources, technical details on the ALCF Blue Gene/P architecture, and hands-on assistance in porting and tuning users’ applications onto the Blue Gene/P. The workshop offered an excellent opportunity to interact with the ALCF staff and learn more about the Blue Gene/P machine, as well as data analytics and visualization capabilities at the ALCF. Attendees took a tour of the Interim Supercomputing Support Facility and particularly appreciated the hands-on experience offered at the workshop.
University Profs, Students Win Access to OLCF Computing Resources
Four university professors and their students received $25,000 high-performance computing awards in February as winners in the first-ever computational research competition established by Oak Ridge Associated Universities (ORAU) and ORNL. The one-year, $25,000 grants may be extended for two more years, for a total of $75,000. Recipients will have access to the the Oak Ridge Leadership Computing Facility's (OLCF) Cray XT5 supercomputer Jaguar as well as computing staff.
The recipients are Shaikh Ahmed, an assistant professor in Electrical and Computer Engineering at Southern Illinois University at Carbondale, and his student Vamsi Mohan Gaddipati; Yongmei Wang, an associate professor in Chemistry at the University of Memphis, and students Jesse Ziebarth and Aaron Masur; Ming Ye, an assistant professor in Scientific Computing at Florida State University, and student Geoffery Miller; and Oleg Zikanov, an associate professor in Mechanical Engineering at the University of Michigan.
ORNL Connects K-12 Students with Breakthrough Science Research
SciEdTech (http://sciedtech.org), a new outreach and science website, aims to bring tomorrow’s science to today’s kindergarten through twelfth-grade students. Visualizations of the simulations done on ORNL supercomputers will be designed for the website to teach basic science concepts to students, as well as to introduce them to the kinds of projects ORNL scientists work on. Students and teachers can ask questions, post comments, and access other online resources to enhance classroom learning.
“We’re working on real-life science challenges of today and tomorrow, and we’re computing at scales that are unique,” said Ross Toedte of the OLCF’s visualization task group at ORNL. “We can [provide] science lessons based on these problems" The project’s partners include researchers and educators from the OLCF and Oak Ridge Associated Universities (ORAU).
Science Seminars at ORNL Tackle Research Needing Petascale Power
The Leadership Computing Facility at ORNL has resumed its monthly seminar series featuring speakers in sciences that would benefit from simulating their research problems on the XT5 Jaguar supercomputer. Guest speakers are typically researchers in the computational sciences fields from outside the center who collaborate with the center’s staff and may use Jaguar, recently upgraded to 1.64 petaflops.
Balint Joo, a physicist in lattice quantum chromodynamics (QCD) at Thomas Jefferson National Accelerator Facility in Newport News, Virginia, gave the first presentation in the series on March 2. On March 10, Cecilia Bitz, professor of atmospheric sciences at the University of Washington, presented her modeling of Arctic summer sea ice through the twenty-first century, using the Community Climate System Model. Videos of the seminars can be viewed at http://www.nccs.gov/category/podcast/ or subscribe to the podcast via RSS feed from the OLCF website or on iTunes, by searching for NCCS.
LBNL's Juan Meza Gives "Wizard" Talk to UC Students
Juan Meza, head of the High Performance Computing Research Department at LBNL was the keynote speaker at the UC LEADS (Leadership through Advanced Degrees) program in Sacramento on Sunday, March 8. About 100 people, mostly students from the various UC campuses,” attended. Meza was asked to give a talk on what he thought they should know to be able to succeed in a science and technology career. The UC LEADS program aims to educate California's future leaders by preparing promising students for advanced education in science, technology, mathematics and engineering (STEM). The program is designed to identify upper-division undergraduate students with the potential to succeed in these disciplines, but who have experienced situations or conditions that have adversely impacted their advancement in their field of study. “Since it was a breakfast talk, I decided to do something a bit different and fun to keep the audience entertained,” Meza said. He drew on “The Wizard of Oz” for his theme, reminding students that to succeed they need knowledge (a brain), passion (heart), confidence (courage) and community (home), mirroring the quest of the Scarecrow, Tin Woodman, Lion and Dorothy to find the source of all wisdom, the Wizard of Oz.