TEAM Zarate Tarrani Notes From the Field

Testing, Quality and Process. In our 13, 14 and 16 May entries Linda and I have taken turns discussing quality- and testing-related books.

The software testing profession came into its own in 1979 when Glenford Myers published The Art Software Testing. Although this book is still in print (a remarkable feat in itself), it's quaint when compared to what we now have in published works and the body of knowledge. What this book did for the profession is legitimize it as a valid career path and to portray software testing as a profession instead of an activity to which mediocre programmers were exiled. Myers deserves the credit bestowed, but there is an unsung hero in the software testing and quality movement whose prolific writing has had considerable influence: William E. Perry.

Perry was writing about maintenance, testing and quality before Myers' book arrived on the scene, and his 1991 book, Quality Assurance for Information Systems: Methods Tools, and Techniques, is an interesting blend of holistic IT quality and software testing. I still refer to my copy for ideas when I am researching metrics. This book is about mid-point in Perry's publishing career. While his subsequent books focused more on software testing, this one is among the first to cover both software quality assurance and software testing in a coherent manner.

William Lewis' Software Testing and Continuous Quality Improvement that both Linda and I have recently discussed here (and reviewed on Amazon) extends Perry's work with respect to a holistic view of software quality.

Testing vs. SQA. I make the distinction between testing and SQA as follows:

Testing is an activity to find or prevent defects in software using older inspection techniques or more modern preventive techniques. Note that I am not including value judgments in my definition, else I would have ignored the inspection approach. What I want to do is highlight differences between testing and SQA.

SQA is an oversight function that collects and analyzes quality data to be used in pursuit of process improvement.

Based on my definitions testing belongs in the application delivery domain and serves as the boundary between application delivery and service delivery (i.e., production). This is shown in the organizational diagram that Linda and I developed. SQA, in my opinion, should be a function of a program management office (an ideal spot for oversight), or an entirely separate function that reports directly to the CIO.

However, software testing is evolving to the point where testing and SQA are becoming blurred. In fact, to put it crudely, finding the boundary between testing and SQA is akin to picking fly shit of pepper. I apologize for that analogy, but it best describes the situation. The two books I've recently discussed, Systematic Software Testing and Introducing Software Testing each integrate testing and SQA, and it looks like the direction that software testing is going to take. There are some strengths and weaknesses to this:

• Strengths: end-to-end quality infrastructure, with a viewpoint that encompasses the entire systems life cycle (not just the development life cycle).
• Weaknesses: misses the big picture because testing is a narrow viewpoint of software quality. Other stakeholders in the service level management and project management domains have different viewpoints. See our Life Cycle Quality Gates document for an overview of metrics we deem important and you'll see why many will not be on the RADAR of a test organization.

I fall on the side of centralized SQA as an oversight function. I believe that Edward Deming was correct when he stated, [I]f the measurements you’re using are unfair, inconsistent and not within the control of the person being evaluated then you will demoralize and de-motivate your employees. Testers should be concerned with testing, not the politics of metrics. In fact, Craig and Jaskiel raise this as an issue (in different words) in Systematic Software Testing.

Clouds in My Coffee. The way I see it the maturity of the software testing profession, as evidenced by the two books I discussed yesterday, and the affinity of testing and SQA, are on a course that needs to be carefully considered. For small organizations this isn't such an important issue, but for large enterprises the strengths and weaknesses need to be more carefully examined and weighed than I've done in this entry. The good news is we have reached a point where quality is considered to be important and proactive approaches to achieving it are becoming more prevalent. Better yet, thses approaches are wrappd in process.

Where the issues become even more cloudy is in the growing (and excellent) body of knowledge and practices supporting test process improvement. My next entry will focus on that aspect of testing and quality before moving on to software reliability in a future entry.

Have a wonderful weekend!