November 2006 B
High Productivity Computing Systems and the Path Towards Usable Petascale Computing
D. E. Post, DoD High Performance Computing Modernization Program
R. P. Kendall, Carnegie Mellon University Software Engineering Institute

VI. Analyze computational results from production runs

Typical production runs from a large-scale computational project can produce TeraBytes or more of data. Analysis of the computational results is not only essential but can also be challenging.

VII. Make Decisions

The whole purpose of the computational system of computers, codes and results analysis is to provide information for the basis of decisions for scientific discovery, engineering design, prediction of operational conditions, etc. Often this is an iterative process. Analysis of the initial results will suggest several conclusions. Further production runs, and possibly further code development, will be needed to confirm those conclusions and suggest modifications. Finally, the decisions and the basis of the decisions need to be documented.


On the basis of many case studies of successful and unsuccessful Computational Science and Engineering Projects we have identified the steps that such projects follow from initial concept to final conclusions and decisions based on the results of the project. While none of the projects explicitly followed all these steps in an orderly fashion, the successful ones followed these steps either explicitly or implicitly. Key points emerged from the case studies.

  1. The level of resources involved in computational science and engineering is becoming large, and the potential impact of the decisions is large. A higher degree of organization and formality is, therefore, inevitable, both for the technical success of the projects and for ensuring that the decisions reached as a result of the projects are correct and accepted by all the stakeholders, especially the sponsors.
  2. Development of large scale projects is a highly iterative enterprise and almost always involve an element of research. While the project must be organized and run like a project, highly prescriptive formal designs and processes in the usual Information Technology sense and rigid software management processes are not practical.
  3. Development and application of large scale computational science and engineering projects is challenging. Existing development and application support tools are relatively immature compared to the scale of the challenge. Thus there are many opportunities for software and hardware vendors to develop support tools that can reduce the challenge and facilitate code development and production.
Acknowledgments The authors are grateful for suggestions and support from the members of the DARPA High Productivity Computing Systems (HPCS) Existing Code Analysis Team, Jeffrey Carver, Susan Halversen, Andrew Mark, Dolores Shaffer, and Susan Squires; the leaders of the HPCS productivity team, Jeremy Kepner and Robert Lucas; the members of the various code projects who allowed us to learn from their experiences; Frederick Johnson of the Department of Energy Office of Science; Cray Henry of the Department of Defense High Performance Computing Modernization Program; and Robert Graybill, the leader of the DARPA HPCS program.
1Post, D. E., Kendall, R. P., International Journal of High Performance Computing Applications, 18, 399 (2004); D. E. Post, R. P. Kendall, and E. M. Whitney, Report No. LA-UR-05-1593, 2005.
2Christensen, M. J., Thayer, R. H. The Project Manager's Guide to Software Engineering's Best Practices. IEEE Computer Society, Los Alamitos, CA, 2001.
3NASA - www.mscsoftware.com/products/nastran.cfm
4Post, D. E., Kendall, R. P., Lucas, R. F., In Advances in Computers, edited by M. V. Zelkowitz. Elsevier, Amsterdam, 2006. Vol. 66, pp. 239; Kweku Ewusi-Mensah, Software Development Failures: Anatomy of Abandoned Projects. MIT Press, Cambridge, Massachusetts, 2003.
5Yourdon, E. Death March. Prentice Hall PTR, Upper Saddle River, NJ, 1997.
6DeMarco, T. The Deadline. Dorset House Publishing, New York, New York, 1997; Rob Thomsett, Radical Project Management. Prentice Hall, Upper Saddle River, NJ, 2002.
7Thompson, J. F., Soni, B. K., Weatherill, N. P. Handbook of Grid Generation. CRC Press, Baca Raton, 1998.
8Oberkampf, W., Trucano, T. Progress in Aerospace Studies, 38, 209 (2002); Patrick J. Roache, Verification and Validation in Computational Science and Engineering. Hermosa Publishers, Albuquerque, 1998.
9Post, D. E., Votta, L. G. Physics Today, 58 (1), 35 (2005).

Pages: 1 2 3 4 5 6 7 8

Reference this article
"Large-Scale Computational Scientific and Engineering Project Development and Production Workflows," CTWatch Quarterly, Volume 2, Number 4B, November 2006 B. http://www.ctwatch.org/quarterly/articles/2006/11/large-scale-computational-scientific-and-engineering-project-development-and-production-workflows/

Any opinions expressed on this site belong to their respective authors and are not necessarily shared by the sponsoring institutions or the National Science Foundation (NSF).

Any trademarks or trade names, registered or otherwise, that appear on this site are the property of their respective owners and, unless noted, do not represent endorsement by the editors, publishers, sponsoring institutions, the National Science Foundation, or any other member of the CTWatch team.

No guarantee is granted by CTWatch that information appearing in articles published by the Quarterly or appearing in the Blog is complete or accurate. Information on this site is not intended for commercial purposes.