TEAM Zarate Tarrani Notes From the Field

Picking Up. My last entry opened the door to test process improvement, which is summed up in a 2800 year old quote by Hesiod:

It is best to do things systematically, since we are only human, and disorder is our worst enemy.

Isn't it amazing how something uttered so long ago by a Greek poet is relevant to software testing?

It's beyond question that the Greeks made many lasting contributions to culture and civilization. In the world of test process improvement the lasting contributions may well be coming out of the Netherlands. As an aside, our Dutch brothers and sisters are also making significant contributions to service level management (see my 5 April 2002 entry in our sister weblog, Postcards from the Revolution). The reason I believe that the Dutch are leading the way in test process improvement is because the Test Process Improvement (TPI) and Test Management Approach (TMAP). Each of these approaches are documented in the following books:

Test Process Improvement: A Practical Step-by-Step Guide to Structured Testing. This book provides a coherent process improvement approach for software testing. It provides a model that supports the assessment of strengths and weaknesses of an existing software testing process and an approach for developing and implementing remedial action to rectify the weaknesses. As such this book is not useful to organizations that have not achieved a mature and stable testing process because the model will not apply. If you are seeking a book that will get your processes stable you will find Systematic Software Testing by Rick D. Craig and Stefan P. Jaskiel a better place to start.

However, if your processes are stable this book is among the best because it stays focused on improving the testing process and does so in the same manner that SEI's CMM does for software development. In fact, the TPI approach in this book is cross-referenced to the CMM, which gives you an approach that can be viewed as a testing maturity model that aligns nicely with the CMM (including the newer CMMI). This is one of the strong points of the book and TPI.

Another thing to know about this book is that it's written more like a specification than a narrative. Some readers may find this difficult, but if you are involved in mapping the TPI key process areas to the CMM (or SPICE, Bootstrap or PSM), you'll appreciate the format. Also, the book views TPI as a subset of software process improvement, and software process improvement as a subset of TQM. While the authors focus on the software testing process, they do not isolate it from the bigger picture. This allows you to view then entire quality process as a coherent whole when you're assessing the software testing process and developing improvement strategies.

I personally think this book adds considerably to the software testing body of knowledge, and that the approach the authors give is both practical and sensible. If you work in an organization that has a stable testing process or is at CMM level 2 or above this book is essential reading.

Software Testing: A Guide to the TMAP Approach My first introduction to TMAP was in the above book, which the author co-authored. It piqued my interest, but unfortunately all of the literature on TMAP was written in Dutch. This book makes this powerful test management approach available to English speaking readers, making it invaluable.

First, a little about TMAP to explain why I think the approach is important and useful: It views testing as a process instead of a collection of procedures. The advantage is that once a process is in place it can be stabilized and improved upon. The key to testing is repeatability, and without a process there can be no repeatability. TMAP consists of four elements that combine to form a cohesive test management model:

1. Testing life cycle that is aligned to the development life cycle. This life cycle is encapsulated within a planning and control framework that easily fits into the project management activities of the development life cycle.
2. Testing techniques - not the techniques used in the execution of test cases, but the techniques employed for defining a test strategy, developing test specifications, and the associated artifacts. This book does cover some basic test execution techniques, but they are not the focus of the book and are not covered in great detail.
3. Infrastructure and tools - addresses what are the minimums for an effective test process in the form of environments and tools. If you're establishing a test organization this aspect will be invaluable.
4. Organization - how the test organization is structured and how it relates to external functions, such as development, configuration and release management, project management and other major stakeholders.

Each of the above elements and their parts are covered in great detail, resulting in a sound framework for test management. That alone makes this book invaluable, but there are some additional gems that I especially liked:

• Test point analysis and estimation, which is an estimating method for test effort that is based on function point analysis. This is incredibly valuable because accurate estimation is one of the shortfalls in testing. This alone is reason to buy the book. For more information about Test Point Analysis you can download Test point analysis: a method for test estimation or look through the presentation slides from Conquest 2000, which also includes presentations on TPI and other items of interest. Although off topic, Test Effort Estimation Using Use Case Points is a related approach that fits nicely within the unified process.
• The wealth of checklists - I especially liked the comprehensive list of quality characteristics.
• Testing in maintenance situations - probably the most common situation for software testing and this book covers it well.

This book and the first one I discussed above combine to give a complete picture of test management and test process improvement.

There are a few other books about test process improvement that are worth reading:

• Software Testing in the Real World (see Linda's 21 March 2001 Amazon review for details).
• Lessons Learned in Software Testing
• Surviving the Top Ten Challenges of Software Testing: A People-Oriented Approach

While the last two are more slanted towards advice and examples, they do promote process improvement by showing what does and does not work.