CTWatch
November 2006 A
High Productivity Computing Systems and the Path Towards Usable Petascale Computing
Jeffrey C. Carver, Mississippi State University
Lorin M. Hochstein, University of Nebraska, Lincoln
Richard P. Kendall, Information Sciences Institute, University of Southern California
Taiga Nakamura, University of Maryland, College Park
Marvin V. Zelkowitz, University of Maryland, College Park; Fraunhofer Center for Experimental Software Engineering
Victor R. Basili, University of Maryland, College Park; Fraunhofer Center for Experimental Software Engineering
Douglass E. Post, DoD High Performance Computing Modernization Office

3
3. Observations

Using the different methodologies and the different foci described in Section 2, we performed a cross-study analysis between the ASC codes and the MP codes to look for similar and different observations that can lead to deeper insight into the software development process. In this section, we present those observations along with the support (or lack of support) provided by each type of code (ASC and MP) during independent case studies. To facilitate the discussion of the observations, we have grouped them into high-level categories (each found in a subsection below) with a list of specific observations.

3.1 Goals and Drivers of Code Development

Code performance is not the driving force for developers or users; the science and portability are of primary concern: All of the projects studied were parallel programming projects, therefore code performance is clearly an important goal. But the primary interest of the users and developers is the science, not fast, scalable code (except where fast, scalable code is required to meet the scientific objective). As long as the overall project performance goals were met, the science and portability are of greater concern than the last 10-20% of speedup. For example, a member of one ASC Code team stated that metrics, like scalability, are recorded because the sponsor requests them, while “scientifically useful results per calendar time” would be a more appropriate productivity metric. Furthermore, the developers of the MP Codes indicated that because the codes are often used for decades, they focus more effort on portability than on speed and scalability for the current hardware platform. In fact, to the extent that an increase in performance can be achieved through new hardware, these developers believe that the portability of the software is far more important than the efficiency of the software.

Code success depends on customer satisfaction: For the MP Codes, success or failure depends on whether the code developers keep their customers (not always their sponsor) satisfied. Conversely, in the ASC Codes, the developers were their own customers, so they had no external customers to please.

3.2 Actions and Characteristics of Code Developers

Most developers are domain scientists or engineers, not computer scientists: The majority of the developers for the ASC Codes did not have any formal training in computer science or software engineering. On some (but not all) ASC Codes, the chief software architect had a background in computer science. The developers of the MP Codes found that it is easier to teach domain scientists and engineers how to write code than it is for computer scientists to comprehend the deep scientific or engineering phenomena being captured by the code, especially at the research level.

The distinction between developer and user is blurry: In the ASC Codes, there are no “external” customers whose needs must be met. The primary users of the code are the developers themselves, who were adding functionality to advance their own research. In some cases, there are external users of the code. But, because these codes may require additions or modifications to be useful, an external “user” may still need to do a certain degree of programming. The MP Codes have more external users than the ASC Codes, but the developers still constitute an important portion of the user-base.

There is high turnover in the development team: Because the ASC Codes are developed in academic environments, many project members (postdocs and grad students) are involved for only a few years (e.g., in one project, the two technical leads had been involved for less than four years). Most of the ASC Codes evolved from earlier codes that were written by scientists who had long since left the organization. Conversely, most of the MP Codes had a core set of developers that remained with the project for the duration, often for a decade or more.

Pages: 1 2 3 4 5

Reference this article
Carver, J. C., Hochstein, L. M., Kendall, R. P., Nakamura, T., Zelkowitz, M. V., Basili, V. R., Post, D. E. "Observations about Software Development for High End Computing," CTWatch Quarterly, Volume 2, Number 4A, November 2006 A. http://www.ctwatch.org/quarterly/articles/2006/11/observations-about-software-development-for-high-end-computing/

Any opinions expressed on this site belong to their respective authors and are not necessarily shared by the sponsoring institutions or the National Science Foundation (NSF).

Any trademarks or trade names, registered or otherwise, that appear on this site are the property of their respective owners and, unless noted, do not represent endorsement by the editors, publishers, sponsoring institutions, the National Science Foundation, or any other member of the CTWatch team.

No guarantee is granted by CTWatch that information appearing in articles published by the Quarterly or appearing in the Blog is complete or accurate. Information on this site is not intended for commercial purposes.