TEAM Zarate Tarrani Notes From the Field

Reliability, Quality and Process. My entries here and in Postcards from the Revolution have a recurring theme: quality. Reliability is a function of quality, and process is the means of achieving quality. Moreover, quality is the main issue in my essay about the economic war in which we're engaged, whether or not we want to acknowledge that fact. However, I am not going to add to the essay in this entry; instead, I'm going to share documents about reliability, quality and process that anyone who is in our industry will find useful.

Reliability. No discussion of software quality would be complete without citing work by John Musa. He is considered to be the father of software reliability, and his influential (and now out-of-print) book, Software Reliability: Measurement, Prediction, Application, launched software reliability as a discipline. His most recent book, Software Reliability Engineered Testing, was published in 1998 and updates his original book with experience gained during the eleven years between the two books. My 11 May 2001 Amazon review of Software Reliability Engineered Testing gives a synopsis of the book, but you can learn more about the approach on Mr. Musa's SRET web page.

Victor Basili is another prominent figure in the software reliability community. His home page contains a list of publications to which he's contributed or authored, and conference proceedings at which he's presented.

If you're involved in software reliability you'll find the Tactical Software Reliability Guidebook to be an invaluable resource. Be aware that this 138-page PDF document is 11 MB and resides on a painfully slow server. If there is sufficient interest I'll place it on my server, which will support faster downloads.

Quality. This subject is vast and encompasses a wide range of subdisciplines. In keeping with my essay's theme I'm going to stick to a narrow view, starting with Cem Kaner's paper titled, Quality Cost Analysis: Benefits and Risks. Closely related to this is an interesting paper titled, A Quantitative Quality Index for Your Application Development Process. Don O'Neill has also contributed to the cost of quality body of knowledge, especially with his work with the National Software Quality Experiment. I also found his short paper on Software Inspections Return on Investment informative and a valuable source with which to make a business case for inspections and walkthroughs.

As luck would have it, the March 2002 issue of CrossTalk just arrived in my mailbox, and the entire issue is devoted to quality and metrics. The first article, Don Reifer's Let the Numbers Do the Talking is a treasure trove of data that addresses both quality and project management. Mr. Reifer is a favorite author whose published works include Software Management (see my 22 September review on Amazon), Practical Software Reuse (I also reviewed this on 22 September), and Making the Software Business Case: Improvement by the Numbers, which Linda reviewed on 22 September.

The article titled How CMM Impacts Quality, Productivity, Rework and the Bottom Line by Michael Diaz and Jeff King nicely ties together the relationship between process and quality.

Process. Believe it or not, but there are those in IT who are anti-process. I call them cowboys and cowgirls when I'm being politically correct. In private I usually refer to them as misguided (when I'm feeling charitable), or clueless (when I'm feeling less charitable). Certainly the foregoing article on how CMM impacts quality, productivity, rework and the bottom line makes a strong case for a process. However, processes do not spring up and take hold by magic or wishful thought. Suzanne Garcia's article titled, Are You Prepared for CMMI? is a short, well-written piece about adopting CMMI that can apply to any process. A good starting point for software engineering processes is SEI's page describing their SEI Software Engineering Process Management Programs. If you're seeking guidance on implementing the CMM, or any initiative of similar complexity, I strongly recommend CMM Implementation Guide: Choreographing Software Process Improvement by Kim Caputo. My 3 July 2001 Amazon review sums up why I think the book is valuable.

An additional source of information is a page about process improvement frameworks, which introduces a quality improvement paradigm, and briefly describes three techniques: Goal/Question/Metric, Assess, Analyze, Metricate, Improve, and the personal software process (PSP). This page is complemented by another page on Software measurement, which is the foundation of any software quality process.

The connection between quality and costs is shown in the following documents:

• Process Improvement as a Capital Investment: Risks and Deferred Paybacks
• Measuring the Value from Improved Predictions of Software Process Improvement Outcomes Using Risk-Based Discount Rates
• Estimating the Financial Benefit and Risk Associated with Process Changes
• Investigating Financial Measures for Planning of Software IV&V
• Assessing ROI for Independent Verification & Validation

All of these support the cost of quality (and non-quality) premise upon which my quality philosophy is built.

Parting Shot. A page from a University of Calgary software engineering course contains a few horror stories that stemmed from either lack of quality, process or both: Introduction to Software Engineering Process. It delights me to know that this is being taught in universities, and it reinforces my opinion that IT practitioners without degrees may be a part of our profession's reputation for failures and questionable value to the businesses that we're supposed to be supporting.

I'll be posting related material later today in Postcards from the Revolution. If you're not reading that weblog you're only getting half of the picture.