Rimu Research: Coaching: Testing

Coaching Test Automation

There has been a most surprising outcome of the development of Test Driven Development. Bringing the tests in earlier means that they are much more effective, for several reasons.

Feedback Delay

We are used to focusing on tests that check that a system is correct after it has been built. But when a problem is found, there can be a considerable delay in fixing it. First the problem has to be duplicated and understood. And then someone has to diagnose the cause of the problem, perhaps looking at code that hasn't been touched for several weeks or months. After they understand what went wrong, it can be corrected. Unfortunately, often the fix introduces further problems (in 16% of cases, on average).

Part of the problem is the delay of useful feedback. If the problem was identified within 30 minutes of the mistake being made, it would be much simpler to find and fix. If the delay is 10 seconds, the time lost in find and fix is almost nothing.

Requirements

Requirements documents are usually difficult to write and difficult to read. Someone has to understand the business requirements well and express them clearly. Programmers have to interpret the requirements, often having to make a best-guess at what is needed.

And then testers have to test the resulting system, based on their interpretation of the requirements and their understanding of the business domain and what the system does.

An Answer

One approach that has found to be effective is to change the view of testing, so that it happens all of the time, rather than after the software is done.

Tests are used as an alternative to requirements documents, by expressing the tests in a different form: so that they serve as executable specifications. Some of these are written before the corresponding code is written, and so drive the development.
- Functional, performance, and load tests can all be handled in this way
- As the system evolves, we regularly test that the system continues to meet the requirements by running the executable specifications.
Exploratory testing, a form of manual testing that is carried out as the software is developed, and aims to pick up problems that have been missed by the executable specifications.

Coaching/Training

The following are examples of what's possible. Consulting and coaching will be tailored to your needs.

1. One Day Assessment

A one day assessment of your project(s) can provide insight into your testing issues (both known and to be discovered), on how to solve your testing problems, and how to make a leap forward.

2. A Few Days Onsite Coaching

A review of your current approach
Overview of tests as executable specifications (why and how) to the whole team.
Half to one day coaching the Product Managers, Testers, and Business and Systems Analysts on writing executable specifications, working with project stories
Half to one day coaching the developers on driving development from executable specifications, driving the domain into the code

3. Multi-day Onsite Coaching

Present overview of tests as executable specifications (why and how) and Fit tables to the whole team.
Coach Product Managers, Testers, and Business and Systems Analysts for 2-4 days:
- Chose several requirements. Start with a simple example.

Initially I'll sketch Fit tables (storytests) on the whiteboard and alter them as we progress.
We may start with a simple workflow, based on use cases. Then we'll write a second storytest for a variant workflow. Then we'll focus on the underlying business rule and express that as a calculation rule.
After we'd tackled a few stories, the participants will gradually do more of the development of the storytests. I won't formally introduce the details of Fit tables, but will show by example what could be done.
I'll also show by example how to use FitNesse, a tool for editing and running storytests.

Coach testers for 1-3 days on the finer details of using storytests for doing non-functional testing.
Coach programmers for 2-5 days. Start with two programmers on one of the storytests. How to write fixtures. How to drive into the code. When to start test driving with unit tests. Driving domain into the code. Resolving differences between current code and storytests. Handling design smells.
Ongoing coaching, one day a week for several weeks, to locate issues and to help with those that arise.

This can take longer than initially expected because the storytests often expose problems in the way that the software is being developed. Coaching in OO design, TDD, and domain design may be needed as much as coaching in storytest driven development.

Rimu Research