TEAM Zarate Tarrani Notes From the Field

XML Resources. Because XML is so versatile, especially for enterprise applications integration, and as a core component of web services and e-commerce systems, I want to share some of the better XML books that are available.

Although you can download XML specifications from the W3C working groups, a single book that summarizes these specifications is worth the investment. XML Family of Specifications: A Practical Guide is such a book. It's a comprehensive and up-to-date (as of this review) reference on XML as defined by the W3C. Part I is more of a desk reference (with a lot of example code), which covers XML syntax, modeling and parsing, DTDs and schemas. Part II, also with many examples, is a complete treatment of parsing with APIs, with separate chapters on SAX, DOM, JDOM and JAXP. Transformation and display protocols are covered in Part III, including CSS2, XSLT and XPath. XSLFO for formatting is also covered in this part. Xlink and Xpointer to facilitate referencing operations are the subjects of Part IV, and the book wraps up the formal descriptions of the family of specifications in Part V, which covers XHTML and RDF. I have a personal interest in RDF, and found the chapter devoted to it complete, but terse. This characterizes all of the chapters in this book. What makes this book valuable is the way the information is displayed. Each chapter starts with either an overview or concepts, and each clearly explains each specification and gives clear examples to demonstrate how they work in practice.

Appendices at the back of the book are especially valuable because they summarize much of the information in the body of the book. For example, Appendix A depicts the family of specifications in a format that clearly shows the relationships among them. In addition, the web site that supports the book provides a lot of supplementary material, including over 900 links to related resources and an image map of the family of specifications that is one of the most visually appealing and informative resources one can have at their disposal. Note that the web site is not up-to-date - some information that was cited as coming in April and May were still not online as of late June.

This is not a book for learning XML as much as it's a reference. The main value over W3C material that is available over the web is the clear writing and many examples. It reads much better than dry specs and is complete in its coverage.

Now For Something Strange. As long as I'm dredging up old books that I think are still useful, here is one that is worth tracking down: Testing to Verify Design and Manufacturing Readiness This book, despite the editorial description on this page, is entirely about hardware/software integration as it pertains to managing acquisition risk for the buyer and the processes and procedures that need to be employed by the developer.

If you work within the framework of the FDA's General Principles of Software Validation or the FAA's DO-178B for safety-critical avionics the material is consistent with these governing documents, but is too outdated to be useful.

However, if you are working on integrated projects that are unregulated with respect to government controls you may find this book useful. It contains a wealth of useful guidelines for establishing and managing processes to support development of products that are based on embedded software or hardware/software integration, The core of this book is a collection of templates that were developed and proven in the DoD industry, and are designed to manage integrated testing, failure management and field feedback. Each element is applicable to commercial environments, especially for companies that are manufacturing intelligent network devices, data storage systems and specialty products such as digital control systems, sensors and other integrated hardware/software products.

The templates are introduced in Chapter 1, and each of the seven functional areas covered by the templates are discussed in separate chapters. These functional areas are: integrated testing, failure reporting, design limits, product life, test/analyze/fix process, uniform test reporting and field feedback. A chapter on applying these follows, but the material is slanted towards DoD issues. If you apply thought and imagination while reading this chapter you should get ideas on how to refactor the cases into your own environment.

Section 2 devotes three chapters to software design and test, which are based on the older waterfall development life cycle. However, this particular life cycle lends itself well to developing embedded systems, making this material valid and applicable to commercial environments.

Overall, this is a useful book for the intended audience I cited above if you can track down a copy. In particular, the checklists and overall framework are valuable, and much can be learned from the risk-based approach taken in the book.

Shifting Gears. Although I'll inevitably return to quality and reliability, I am going to shift to another topic in my next entry: XML. Also, most of the topics for the next few weeks will be in the form of book reviews instead of the tutorials and news items that we've been writing about. That will change as soon as things stabilize. We're all busy and haven't the time to do the research we normally do, nor the freedom to craft original essays on topics that are dear to us. That will change in due time, but until then please bear with us.

Oldies, But Goodies. One of my personal favorite books, and one that has had a profound influence on me, is Quality Assurance for Information Systems. This book represents a pivot point in Perry's prolific published works that date from 1981. What makes it pivotal is the fact that this book synthesizes his approach to IS quality assurance from a production support viewpoint and his future work which focuses on software testing.

Although over 11 years old the QA approach contained in this book is still valid. To get at the gems, though, you have to overlook a few things. For example, terminology common in the mainframe data center of past decades sounds quaint even to those of us who came from that environment. Also, the code examples used to illustrate quality problems are sure to confuse the younger generation of C++ and Java developers and test professionals who probably never heard of PL/I and only vaguely know about FORTRAN.

What I like about this book and the reason why I think it's still an important reference is the fact that application quality from an enterprise perspective is addressed. This goes beyond testing and release processes, as well as beyond project issues surrounding applications delivery and SQA. The focus is on production and maintenance, although testing, SQA and project metrics are addressed.

In addition to the focus, the book contains checklists, questionnaires and sample forms that can be updated to reflect modern computing environments - and you may be surprised to find that much of this 'ancient' material requires very little modification. Another aspect of this book that I like is the material on software maintenance, which seems to be a lost art, although it's as important now as it ever was.

Don't let the age of this book deter you if you're interested in quality assurance from a production support point of view. The best recommendation I can give is that this book has served me well in over a decade of consulting, and it probably will for years to come. However, it shouldn't be your only reference either.

More on Quality and Testing. In previous entries I covered most of the newer books on quality, reliability and testing. However, there are some older books that are still valid or contain enough information of value that they merit a mention.

• Testing Very Big Systems. After you've peeled back the layers of testing techniques that are better documented and more refined in more recent books, and archaic language that characterized the mainframe lingo that was dying out when this book was first written a decade ago you'll find gold.
  First, the way test case management is presented stands the test of time. The author is obviously well versed in managing complex system testing and it shows in his detailed approach to developing a test strategy and managing a large array of test cases. As good as this material is, it isn't a sufficient reason to track down a copy of this book because Rick Craig and Stefan Jaskiel have a more modern book, Systematic Software Testing that accomplishes the same goal.
  The real gold is in the way that this book integrates testing, issue management and metrics. Although there is a large body of knowledge on these topics, this book manages to sort out the complexities in the clearest terms I've encountered. I also think that the approach change management is excellent, and especially the way this is linked to issue management. On the subject of issue management, the taxonomy of issue types has served me as a model during numerous consulting engagements for service delivery and software engineering process development, and have been proven in the field.
  Additional gold is in the chapters on test documentation (especially the treatment of status reporting) and managing management. I also like the way that the author takes economic considerations into account, which was not much in vogue when this book was written in 1992.
  If you're an SQA or applications delivery practitioner I strongly recommend tracking down a copy of this book. Look past the archaic parts and you will find one nugget after the other of useful information. I wish this book would be rewritten to reflect today's environment and the lessons that the author learned in the decade since this book was first published because there is much in this book that you will not find elsewhere.
• Ensuring Software Reliability. Despite this book's age and the subsequent software reliability books that have since been published, it adds a perspective and information that is either not in more recent books, or is not given the same comprehensive treatment.
  If you are familiar with software reliability as a discipline and with any of the major books, such as John Musa's Software Reliability Engineered Testing, you'll probably not find anything new in Part I, although chapters 3 (software failures and failure processes), and 6 (reliability terms and definitions) add clear, succinct descriptions and definitions to these topics.
  Part II, however, is where this book shines and why I use this book as one of my principal references. Specifically, chapter 7, which covers software reliability data collection, is thorough and comprehensive. I especially like the way data collection is integrated into a reporting process, and the near exhaustive list of error, product and process metrics and their associated descriptions. Chapter 8 is another gem. It describes 12 major reliability models, ranging from Musa's models to predictive models. One of the most interesting models in this catalog is the 'Leone Test Coverage Model', which is based upon percentage of completion and coverage of specific development and testing tasks. For each model the author gives a summary description, provides assumptions and parameters of the model, and the associated math. Each model's summary contains strengths and weaknesses, and when in the life cycle the model is best employed.
  Overall, this book contains some invaluable information and information that has been superseded by newer books (especially the last chapters in Part II). If you're seeking information that I've highlighted above, this book is a worthwhile investment. If you're looking for a book that is more up-to-date I recommend Software Reliability Engineered Testing by John Musa. This book will remain an often referenced part of my library for some time to come.

In my next entry I'll provide additional books that I like in spite of their age.

Short Break. I am going to briefly break from the testing, SQA and reliability thread because I don't have time right now to devote to properly wrapping it up. I will offer an interesting article titled Use of Metrics in High Maturity Organizations to keep the pace alive until I return to the topic.

Wireless and M-Commerce Development. I just posted my take on a book titled Mobile Business Strategies: Understanding the Technologies and Opportunities in our sister weblog, Postcards from the Revolution.

That weblog focuses on service delivery and business/IT alignment issues, while this one is slanted towards software engineering and more technical topics. The book fit within our theme for Postcards from the Revolution, but there is a related book that is more suitable for this audience. The title is The Complete Wireless Internet & Mobile Business Programming Training Course (with CDROM), and the friend who called it to my attention was enthusiastic. It appears to be a complete training course in all aspects of wireless and mobile commerce development. Judging from the content of the thirty-four associated PowerPoint presentations that are available for free download this is, indeed, a complete training course. If you need to get yourself or your staff quickly up-to-speed and you have a constrained training budget this may be a cost-effective alternative.

Back to Quality. Before ending this entry I want to revisit quality. If you are pursuing the ASQ CSQE certification you may want to get a copy of Fundamental Concepts for the Software Quality Engineer. This book is published by the sponsor of the certification (ASQ), and the book editor is Taz Daughtrey, who is editor-in-chief of ASQ's peer-reviewed quarterly journal, Software Quality Professional.

More on SQA + Reliability. In my haste to provide SQA resources yesterday I left out two important ones that should be bookmarked and frequently visited by anyone who is interested in software quality assurance:

1. David F. Rico's home page.
2. Tantra Management Services.

These are my personal favorites, and I have been using them for years as primary resources.

Software Reliability - Short Version. I am still pressed for time, so this entry is going to be as terse as my last. In the same manner that I use a single book as my primary reference for SQA, I use Software Reliability Engineered Testing by John Musa as my primary reliability reference. My 11 May 2001 review on Amazon will show why I hold it in such high regard. That doesn't mean that it's the only book I use - I have a large collection of SQA and reliability books - it means that it's the first one to which I turn for authoritative information on the topic. On the web the first place I go is the Data and Analysis Center Software Reliability page, which points me to the resources I need for particular aspects of reliability.

Past Information. Reliability has been addressed in this weblog in many previous entries, so I am not going to repeat much of that material here. However, during the next few days (when I get a break in my routine) I am going to wrap up this thread with a few longer entries that describe my own views about SQA and reliability.

SQA. We've now come to SQA, and while most of my testing resources are books, there is only one book that I use as a primary reference for SQA: The Handbook of Software Quality Assurance by Gordon Schulmeyer and James McManus. My reasons for using this book as a primary reference are cited in my 18 April 2001 Amazon review. However, my most frequently used resources for SQA, and the ones which have shaped my thinking, are:

• CrossTalk Magazine.
• Software Technology and Support Center, which has a rich cache of technical documents, great material about software quality engineering and testing, as well as related topics.
• Data and Analysis Center for Software SQA page.

One interesting page I want to share that crosses SQA and software engineering practices is Nine Steps to Defect-Free Software, which should be made into a poster and placed in every cubicle in development.

I am pressed for time, so am going to abruptly end this without further commentary. I'll pick up where I left off tomorrow.

Closing In. This thread started with a brief set of reasons why I was enamored with Systematic Software Testing by Rick D. Craig and Stefan P. Jaskiel, and has grown into a series about testing, quality, SQA and reliability. I opened the last entry with a quote attributed to Hesiod, who remains an influential Greek poet and philosopher. The theme of this entry is metrics, so I am going to open with a quote by Albert Einstein:

Not everything that can be counted counts, and not everything that counts can be counted.

How true. Einstein's legacy of genius will live on for ages because he has influenced generations of mathematicians and physicists.

While perhaps not at the same level as Einstein, Robert B. Grady will remain in my memory because of the deep influence his work has had on my thinking. I first discovered Grady in 1992 when I read Practical Software Metrics for Project Management and Process Improvement (see Linda's 22 April 2001 Amazon review). This is Grady's first book and it sets the tone for his later two books discussed below. What makes this book so important is that it is one of the first to integrate software metrics with project management metrics.

What I particularly like about this book includes:

• Complete view of metrics that matter, and the chronicle of how these metrics evolved in a large company (Hewlett-Packard).
• Recognition that any software metrics initiative extends beyond the project that delivers the software - Grady examines post-production metrics and ties them back to not only the development life cycle, but the product life cycle as well. Ten years after this book was published there are still large organizations that are struggling with doing this, yet Grady's book provides a clear roadmap to achieving this elusive goal.
• Continuous improvement is the central theme in this book. Grady does not stop with collecting and analyzing metrics, but how to effectively employ them to spot improvement opportunities and develop a strategy to effect those improvements.

The book is written as both a story of how a successful metrics program evolved, complete with anecdotes that will prove helpful, and as a collection of data that illustrates what is and is not important to a comprehensive metrics program.

Among all of Grady's books I like this one the best; however, I recommend that his other two also be carefully read if software process improvement is your goal. He has much to say and backs it up with data and a chronicle of his experiences from real projects.

Five years later Grady wrote Successful Software Process Improvement, which followed-up on the foundation he laid in the first book by showing how his metrics-based approach can be leveraged into a viable process improvement program. This book uses the TQM Plan-Do-Check-Act framework as the basis for process improvement. However, he goes deep into the issues and factors to give a complete approach to developing and managing a continuous improvement posture.

Highlights of this book include:

• The same story telling approach he successfully used in his first book. The conversational writing style and the logical sequence of the book makes it easy to read. Moreover, the real life examples add credibility and make the content practical instead of merely blue sky theory.
• A complete survey of assessment methods, such as the CMM, Software Productivity Research's Software Quality and Productivity Assessment, and Hewlett-Packard's internal QUality Maturity System. The latter two are especially interesting because they are, in essence, balanced scorecards.
• Business-oriented - the approach taken never strays from cost/benefit and ROI.

The parts I especially liked included the chapter on software failure analysis (a personal interest), key lessons from adopting best practices, and moving past reasons not to succeed. In fact, if you get nothing else from this book the last part will make this book a worthwhile investment because he shows how to deal with the six most common excuses for not pursuing process improvement (or any other initiative for that matter).

In also like the wealth of metrics, data and examples. While this book is longer than his first one, it's still a manageable 314 pages and is highly readable. If you are involved with software process improvement initiatives this book should be on your short list.

His last book, Software Metrics: Establishing a Company-wide Program, is about how to establish a viable metrics program. See my 28 November 2000 review on Amazon for details.

There is one other book that has deeply influenced me, Software Excellence: A Total Quality Management Guide. This book is a collection of papers that were made into a text under the editorial control of Shigeichi Moriguchi. Mr. Moriguchi did a superb job of ensuring both readability and structuring the content in such a manner that it can actually be viewed as three books:

1. A textbook on software quality control.
2. Catalog of techniques used in testing and SQA.
3. Training guide for testers and SQA professionals.

More details can be read in my 20 February 2002 on Amazon.

Moving Along. Life is a journey, not a destination. This thread is going to imitate life because in the next entry I'll continue the journey, which will pass into the realm of SQA - a strange place inhabited by many cultures, and whose inhabitants are still trying to figure out who they are.