【正文】 = Development Time + Execution Time Research Goal: Develop theories, hypotheses, and guidelines that allow us to characterize, evaluate, predict and improve how an HPC environment (hardware, software, human) affects the development of high end puting codes. Partners: MIT Lincoln Labs, MIT, UCSD, UCSB, UMD, USC, FCMD 31 HPCS Example Questions ? How does an HPC environment (hardware, software, human) affect the development of an HPC program? – What is the cost and benefit of applying a particular HPC technology? – What are the relationships among the technology, the work flows, development cost and the performance? – What context variables affect the development cost and effectiveness of the technology in achieving its product goals? – Can we build predictive models of the above relationships? – What tradeoffs are possible? – … 32 HPCS Example Hypotheses ? Effort to parallelize serial code is greater than effort to develop serial code ? Novices can achieve speedup ? The variation in execution time of MPI codes will be greater than the variation in execution time of OpenMP codes ? The variation in the speedup of MPI codes will increase with the number of processors ? … 33 HPCS Research Activities Empirical Data Development Time Experiments – Novices and Experts Predictive Models (Quantitative Guidance) General Heuristics (Qualitative Guidance) . Tradeoff between effort and performance: MPI will increase the development effort by y% and increase the performance z% over OpenMP . Scalability: If you need high scalability, choose MPI over OpenMP 34 HPCS Testbeds We are experimenting with a series of testbeds ranging in size from: – Classroom assignments (Array Compaction, the Game of Life, Parallel Sorting, LU Deposition, … to – Compact Applications (Combinations of Kernels, ., Embarrassingly Parallel, Coherence, Broadcast, Nearest Neighbor, Reduction) to – Full scientific applications (nuclear simulation, climate modeling, protein folding, ….) 35 Technology Transfer Future Combat Systems Project Goal: Support FCS Program Management Office in the development of the Future Combat Systems (FCS), focusing on the plex system of systems (software) development risk, ., acquisition, architecture, … and build lessons learned for future iterations of FCS and future CSoS. Research Goal: Build a risk experience Base and a Complex System of Systems Lessons Learned Experience Base. Partners: UMD, USC, FCMD, SEI, Sandia, LSI: Boeing, SAIC 36 FCS Technology Transfer Assumption: the technologies are mature enough and have been shown successful in other projects or anizations Example technologies being transferred: GQM to help define goals of various levels of project management for plex systems of systems Spiral life cycle model to the development of the system Experience base tracking problems associated with a plex system of systems to learn from early spirals of development and provide an experience base for future systems Activities: Observe, interview, tailor, train, support, learn, … Feedback: Take what has been learned and feed it back to identify research needs, immaturity in technologies, the importance of context variables, … 37 CeBASE ThreeTiered Empirical Research Strategy Basic Research Applied Research Practical applications NASA HDCP DoE HPCS . NSF Research DoD FCS Increasing success rates in developing agile, dependable, scalable IP applications. More mature, powerful ePackage. Faster technology maturation and transition. Evolving ePackage understanding and capabilities. Primary activities Evolving results Technology maturity 38 Conclusion ? This talk is about – The role of empirical study in software engineering – The synergistic relationship of research, applied research, and practice – The use of testbeds in experimental evaluations ? Software developers need to know what works and under what circumstances ? Technology developers need feedback on how well their technology works and under what conditions ? We need – to continue to collect empirical evidence – analyze and synthesize the data into models and theories – Collaborate to evolve software engineering into an engineering discipline