November 2006 B
High Productivity Computing Systems and the Path Towards Usable Petascale Computing
Andrew Funk, MIT Lincoln Laboratory
John R. Gilbert, UC Santa Barbara
David Mizell, Cray Inc.
Viral Shah, UC Santa Barbara

7. Conclusion

We believe that programmers go through an identifiable, repeated process when developing programs, which can be characterized by a directed graph model such as timed Markov models. We successfully gathered data and fit it to timed Markov models twice, in our 2004 and 2006 classroom experiments. The replay of programmer experience offline was one of the most important aspects of the data gathering process. The timed Markov models clearly indicate the most time intensive parts of the development cycle, quantitatively confirming our intuition – programmers spend most of their time debugging and, once they get a correct program, tuning it for performance is even more difficult. Our data also suggests, in this context, that programmers may introduce slightly fewer bugs in UPC programs and find it easier to optimize them, as compared to C/MPI programs.

Clearly, this is only the beginning. A lot more data needs to be collected before languages can be compared in a meaningful way. Towards this end, we are building various tools for the community at large. These tools will provide a general framework for data collection and model construction to study programmer productivity in a variety of ways.

1Chamberlain, B. L., Deitz, S. J., Snyder, L. "A comparative study of the nas mg benchmark across parallel languages and architectures," Supercomputing 2000, Proceedings of the 2000 ACM/IEEE conference on Supercomputing (CDROM), page 46, 2000.
2Hochstein, L., Basili, V. R. "An empirical study to compare two parallel programming models," In SPAA ’06, Proceedings of the eighteenth annual ACM symposium on Parallelism in algorithms and architectures, pages 114–114, New York, NY, USA, 2006. ACM Press.
3Post, D., Kendall, R. "Software project management and quality engineering, practices for complex, coupled multi-physics, massively parallel computational simulations: Lessons learned from asci," Los Alamos National Laboratory, 2003.
4Dongarra, J. J., Otto, S. W., Snir, M., Walker, D. "A message passing standard for MPP and workstations," Commun. ACM, 39(7):84–90, 1996.
5The UPC consortium. Upc language specifications v1.2. May 2005. upc.lbl.gov/docs/user/upc_spec_1.2.pdf
6Howard, R. Dynamic probabilistic systems. John Wiley, New York, 1971.
7Johnson, E. W., Brockman, J. B. "Measurement and analysis of sequential design processes," ACM Transactions on Design Automation of Electronic Systems, 3(1):1–20, 1998.
8Smith, R. P., Eppinger, S. D. "Identifying controlling features of engineering design iteration," Management Science, 43(3):276–293, Mar 1997.
9Smith, B., Mizell, D., Gilbert, J. R., Shah, V. "Towards a timed markov process model of softare development," Second International Workshop on Software Engineering for High Performance Computing System Applications, 2005.
10Shah, V., Gilbert, J. R., Aggarwal, V. "Productivity measurement in the classroom: Experiments and analysis," In preparation.
11Zelkowitz, M., Basili, B., Asgari, S., Hochstein, L., Hollingsworth, J., Nakamura, T. "Measuring productivity on high performance computers," In METRICS ’05, Proceedings of the 11th IEEE International Software Metrics Symposium (METRICS’05), page 6, Washington, DC, USA, 2005. IEEE Computer Society.
12Hochstein, L., Basili, V. R., Zelkowitz, M. V., Hollingsworth, J. K., Carver, J. "Combining self-reported and automatic data to improve programming effort measurement," In ESEC/FSE-13, Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering, pages 356–365, New York, NY, USA, 2005. ACM Press.
13 McCreary, C., Barowski, L. A. "VGJ: Visualizing graphs through java," In Graph Drawing, pages 454–455, 1998.
14Raitner, M. The gml file format. 2001. www.infosun.fmi.unipassau.de/Graphlet/GML/index.html

Pages: 1 2 3 4 5 6 7 8

Reference this article
"Modelling Programmer Workflows with Timed Markov Models ," CTWatch Quarterly, Volume 2, Number 4B, November 2006 B. http://www.ctwatch.org/quarterly/articles/2006/11/modelling-programmer-workflows-with-timed-markov-models/

Any opinions expressed on this site belong to their respective authors and are not necessarily shared by the sponsoring institutions or the National Science Foundation (NSF).

Any trademarks or trade names, registered or otherwise, that appear on this site are the property of their respective owners and, unless noted, do not represent endorsement by the editors, publishers, sponsoring institutions, the National Science Foundation, or any other member of the CTWatch team.

No guarantee is granted by CTWatch that information appearing in articles published by the Quarterly or appearing in the Blog is complete or accurate. Information on this site is not intended for commercial purposes.