May 2006
Designing and Supporting Science-Driven Infrastructure
Thom H. Dunning, Jr, National Center for Supercomputing Applications, University of Illinois at Urbana-Champaign
Robert J. Harrison and Jeffrey A. Nichols, Computing and Computational Sciences Directorate, Oak Ridge National Laboratory

2. Development of NWChem

In the early 1990s, the development of quantum chemistry software that would scale to hundreds, if not thousands, of processors was a challenging task. It was not known how to parallelize some of the basic mathematical algorithms used in molecular computations, let alone how to parallelize the many algorithms specific to these computations. In addition, many basic software technologies (e.g., interprocessor communication) were still evolving. To address these issues, the NWChem Project Team included theoretical and computational chemists, computer scientists, and applied mathematicians. This long-term partnership, which continues to this day, was critical to meeting the goals of the NWChem Project and led to the development of new mathematical algorithms (e.g., PEIGS for diagonalizing matrices) as well as to new computer system technologies (e.g., Global Arrays for interprocessor communication). The multidisciplinary NWChem Project served as a model for the approach taken in DOE-SC’s Scientific Discovery through Advanced Computing (SciDAC) program.

The NWChem project was supported by the Office of Biological and Environmental Research (OBER) in DOE-SC as an integral part of the EMSL Project. The EMSL Project provided approximately $2 million per year for the period FY1992-7. During this same period, DOE-SC’s Office of Advanced Scientific Computing Research provided another $0.5 million per year to support a Grand Challenge project in computational chemistry. The NWChem project leveraged many of the results from the Grand Challenge project. In addition, the initial exploratory work was funded by the Laboratory Director’s Research and Development program at the Pacific Northwest Laboratory. The “core” NWChem project team involved five computational chemists and three computer scientists and applied mathematicians. In addition, 14 postdoctoral fellows were involved in the project. All told, it is estimated that approximately 100 person-years and $12 million were devoted to the development of NWChem v1.0, not including the effort required to develop the technology incorporated in NWChem from external sources.

Several goals were set for the NWChem software package. These included:

  1. NWChem would be based on algorithms that:
    • scale to hundreds, if not thousands, of processors, and,/li>
    • use the minimum number of flops consistent with the above.
  2. em would be affordable to develop, maintain, and extend.

  3. NWChem would be independent of computer architecture to the maximum extent possible:
    • a hardware abstraction layer would be used to isolate the most of the program from the details of the hardware, but
    • it would be possible to tune the code to some extent to optimize performance on the specific computer being used.

Achieving these goals required a combination of research to determine the best solutions to the above problems, modern software engineering practices to implement these solutions, and a world-wide set of collaborators to provide expertise and experience missing in the core NWChem software development team. Fifteen external collaborators were involved in the development of NWChem; seven from the US, seven from Europe, and one from Australia.

Pages: 1 2 3 4 5 6 7 8

Reference this article
Dunning, T. H., Harrison, R. J., Nichols, J. A. "NWChem: Development of a Modern Quantum Chemistry Program," CTWatch Quarterly, Volume 2, Number 2, May 2006. http://www.ctwatch.org/quarterly/articles/2006/05/nwchem-development-of-a-modern-quantum-chemistry-program/

Any opinions expressed on this site belong to their respective authors and are not necessarily shared by the sponsoring institutions or the National Science Foundation (NSF).

Any trademarks or trade names, registered or otherwise, that appear on this site are the property of their respective owners and, unless noted, do not represent endorsement by the editors, publishers, sponsoring institutions, the National Science Foundation, or any other member of the CTWatch team.

No guarantee is granted by CTWatch that information appearing in articles published by the Quarterly or appearing in the Blog is complete or accurate. Information on this site is not intended for commercial purposes.