【正文】 ples with high dependability needs, packaged for use by the technologists, . TSAFE, SCRover ? M3. NASA offline: Part or all of a system previously developed for NASA, ., CTAS, EOSDIS ? M4. Live examples: Part or all of a system currently under development, ., MSL 29 HDCP Using testbeds to transfer technology ? Defining PackagedTestbeds – A packaged testbed is a set of artifacts and the infrastructure needed for running experiments on that system – A packaged testbed is not static。 understand Set goals Choose processes, methods, techniques, and tools Package amp。 manipulate the variables they only exist where developers build software systems Development needs to understand how to build systems better research can provide models to help Research and Development have a symbiotic relationship requires a working relationship between industry and academe 5 Motivation for Empirical Software Engineering For example, a software anization needs to ask: What is the right bination of technical and managerial solutions? What are the right set of process for that business? How are they tailored? How do they learn from their successes and failures? How do the demonstrate sustained, measurable improvement? More specifically: When are peer reviews more effective than functional testing? When is an agile method appropriate? When do I buy rather than make my software product elements? 6 Examples of Useful Empirical Results “Under specified conditions, …” Technique Selection Guidance ? Peer reviews are more effective than functional testing for faults of omission and incorrect specification (UMD, USC) ? Functional testing is more effective than reviews for faults concerning numerical approximations and control flow (UMD, USC) Technique Definition Guidance ? For a reviewer with an average experience level, a procedural approach to defect detection is more effective than a less procedural one. (UMD) ? Procedural inspections, based upon specific goals, will find defects related to those goals, so inspections can be customized. (UMD) ? Readers of a software artifact are more effective in uncovering defects when each uses a different and specific focus. (UMD) 7 Basic Concepts for Empirical Software Engineering This process of model building, experimentation and learning requires the development, tailoring and evolution of methods that support evolutionary learning, closed loop processes, well established measurement processes and the opportunity to build software core petencies As well as processes that support the development of software that is relevant to the needs of the anization can be predicted and estimated effectively satisfies all the stakeholders does not contain contradictory requirements 8 Basic Concepts for Empirical Software Engineering The following concepts have been applied in a number of anizations Quality Improvement Paradigm (QIP) An evolutionary learning paradigm tailored for the software business Goal/Question/Metric Paradigm (GQM) An approach for establishing project and corporate goals and a mechanism for measuring against those goals Experience Factory (EF) An anizational approach for building software petencies and supplying them to projects 9 Quality Improvement Paradigm Characterize amp。 it evolves as existing artifacts and infrastructure features are improved and added – There will be many different versions of these items. – For each experiment on the testbed, a set of artifacts and infrastructure features is selected, based on the design of the experiment and the characteristics of the technology – An important testbed artifact is the set of seeded faults that can be used to create a configuration of the testbed – Productoriented artifacts include source code, executables, and documentation such as requirements, test plans, test cases – Processoriented artifacts include project plans, descriptions of methods and techniques applied during software development and their results, . results of inspections and test 30 Applied Research DoE High Productivity Computing Systems Project Goal: Improve the buyers ability to select the high end puter for the problems to be solved based upon productivity, where productivity means Time to Solution