SeaStar Quality Conference 2023

People sometimes suggest that the mission of testing is to compare the product to its specification — and therefore, if you don't have a specification, you can't test.

Imagine you've been given something to evaluate: a product, or a feature, or an API. Perhaps all you've got is a vague statement, or a couple of claims about what it should do. Perhaps there are some examples of intended behavior, expressed in the form "Given", "When", and "Then". Maybe you've been given a written document that describes a product or a feature — and when you look at it, you find out that the description is inconsistent, or out of date, or incomplete. Can you still test?

Of course, you can.

The real job of testing is to reveal the status of the product: to learn about it, to understand what it really does, to identify problems that might threaten its value, and to describe what you've learned. Although advance knowledge about the product might help, specifications, requirements documents, whiteboard sketches, and verbal descriptions are essentially rumors, wishes, or dreams until we've tested. The tester cannot simply depend on maps or descriptions; the tester must evaluate them skeptically, compare them to the product, challenge them, — or, sometimes, produce them when they are not already there.

In this class, we'll all see a product for the first time. We'll experience normal and necessary confusion, and through testing and map-making, and we'll feel the confusion lifting. We'll practice producing descriptions and representations of the product that will help us to find bugs that matter.

Key takeaways from Michael's tutorial:

- Testing any product well requires us to learn the product — which in turn requires us to test the product.
- Testing early on is messy and confusing; this is natural and normal; the mess and the confusion lift.
- No two products are alike, so the process of learning a product is always different to some degree. But there are systematic ways to organize the process of learning.