SeaStar Quality Conference 2023

Testers are supposed to be good at finding problems. Yet, there are many fawning, drooling, swirly-eyed demos and articles about how AI, and ChatGPT in particular, will revolutionize testing. These are often promoted by people who claim to be testers.

Although AI can do some impressive things, there is no basis to say that AI-based tools will revolutionize testing. The people who say that have no evidence or argument that it will. They are just kids getting excited about new toys.

Still, we have to be ready. We have to know enough about AI to be credible in conversations and debates about it. In this talk, I will lay out how to think critically about AI as a test tool. I will describe some of my experiments, and some of my reviews of other people's demos. I will also imagine how AI might be useful to testers, either now or in the future.