The development and the production of high-quality, reliable, complex computer software have become critical issues in the enormous worldwide computer technology market. The capability to efficiently engineer computer software development and production processes is central to the future economic strength, competitiveness, and national security of the United States. However, problems related to software quality, reliability, and safety persist, a prominent example being the failure on several occasions of major local and national telecommunications networks. It is now acknowledged that the costs of producing and maintaining software greatly exceed the costs of developing, producing, and maintaining hardware. Thus the development and application of cost-saving tools, along with techniques for ensuring quality and reliability in software engineering, are primary goals in today's software industry. The enormity of this software production and maintenance activity is such that any tools contributing to serious cost savings will yield a tremendous payoff in absolute terms.

At a meeting of the Committee on Applied and Theoretical Statistics (CATS) of the National Research Council (NRC), participants identified software engineering as an area presenting numerous opportunities for fruitful contributions from statistics and offering excellent potential for beneficial interactions between statisticians and software engineers that might promote improved software engineering practice and cost savings. To delineate these opportunities and focus attention on contexts promising useful interactions, CATS convened a study panel to gather information and produce a report that would (1) exhibit improved methods for assessing software productivity, quality, reliability, associated risk, and safety and for managing software development processes, (2) outline a program of research in the statistical sciences and their applications to software engineering with the aim of motivating and attracting new researchers from the mathematical sciences, statistics, and software engineering fields t o tackle these important and pressing problem areas, and (3) emphasize the relevance of using rigorous statistical and probabilistic techniques in software engineering contexts and suggest opportunities for further research in this direction.

To help identify important issues and obtain a broad range of perspectives on them, the panel organized an information-gathering forum on October 11-12, 1993, at which 12 invited speakers addressed how statistical methods impinge on the software development process, software metrics, software dependability and testing, and software visualization. The forum also included consideration of nonstandard methods and select case studies (see the forum program in the appendix). The panel hopes that its report, which is based on the panel's expertise as well as information presented at the forum, will contribute to positive advances in software engineering and, as a subsidiary benefit, be a stimulus for other closely related disciplines, e.g., applied mathematics, operations research, computer science, and systems and industrial engineering. The panel is, in fact, very enthusiastic about the opportunities facing the statistical community and hopes to convey this enthusiasm in this report.

The panel gratefully acknowledges the assistance and information provided by a number of individuals, including the 12 forum speakers—T.W. Keller, D. Card, V.R. Basili, J.C. Munson, J.C. Knight, R. Lipton, T. Yamaura, S. Zweben, M.S. Phadke, E.E. Sumne r, Jr., W. Hill, and J. Stasko—four anonymous reviewers, the NRC staff of the Board on Mathematical Sciences who supported the various facets of this project, and Susan Maurizi for her work in editing the manuscript.