ER: presenting at the AST’s “Steel Yourselves” webinar (30th January 2023)

I was delighted to be invited to participate in a webinar by the Association for Software Testing as part of their “Steel Yourselves” series. The idea is based on the Steel Man technique and I was required to make the strongest case I could for a claim that I fundamentally disagree with – I chose to argue for “Shift Nowhere: A Testing Phase FTW”!

I had plenty of time to prepare for the webinar and to do my research on the use and abuse of testing phases. I also looked into the “shift left” and “shift right” movements as counterpoints to the traditional notion of the testing phase. Sorting through the various conflicting and contradictory ideas around testing phases was an interesting process in itself. I built a short PowerPoint deck and rehearsed it a couple of times (so thanks to my wife, Kylie, and good mate, Paul Seaman, for being my audience) to make sure I would comfortably fit my arguments for a testing phase into the ten-minute window I would have during the webinar.

January 30th came around quickly and the webinar was timed well for Europe (morning) and Australia (evening) as well as places in-between, so it was good to see an audience from various parts of the world. The session was ably facilitated by James Thomas for the AST and Anne-Marie Charrett went first, to make her case that “Crosby was Right. Quality is Free”. She did a great job, fielded the questions from audience really well and made some good observations on the experience – and concluded right on time at 30 minutes into the session.

I felt like I delivered my short presentation in defence of a testing phase pretty well, getting a few smiles and interesting body language from the audience along the way! There were plenty of questions from James and the audience to challenge my claims and I tried hard to stay “in character” when answering them! The final section of the webinar allowed me to remove the mask and speak freely on my real points of view in this area.

Preparing for and presenting this defence of a testing phase was a challenging and interesting task. As usual, if we’re willing to look past the dogma, there’s usually some useful ideas we can take away from most things. While I disagree that the lengthy, pre-planned, scripted test phases I was often involved in during the early stages of my testing career really offer much value, I think the noise around the “shift left” and “shift right” movements has left a gap in-between where we still need to take pause and allow some humans to interact with the software before unleashing it on customers. (I’ve written about this previously in my blog post, The Power of the Pause.) Thanks to the AST for the opportunity to present at this webinar and give myself a refresher on this particular area of testing.

A recording of this “Steel Yourselves” webinar, along with plenty more awesome content, can be found on the AST YouTube channel.

Snooker and test planning

I was lucky enough to spend the weekend watching the semi-finals and final of the Bendigo Goldfields Open snooker tournament. This world-ranking event is relatively new on the snooker calendar and I’ve taken the opportunity to attend each year, with the “finals” weekend now becoming an annual event for me. Coverage of snooker is sadly almost non-existent in Australia so this is my one chance each year to watch the sport I love, up close and personal. (I will use some snooker terminology in what follows, so if you’re not too familiar with the game, it might be worth a look at some basic rules and terms first.)

As I watched the deft skills of the players, it occurred to me that there are parallels between how they go about constructing a break (with the aim of scoring enough points to win the frame) and how we as exploratory testers go about testing a feature or product.

Break building is all about strategy and planning, with good players working out several shots ahead so that they maneuver the cue ball into the right place after each shot in order to make the next shot (and subsequent ones) as straightforward as possible. These plans don’t look too far ahead though and the actual outcome of each shot will often mean their plan for subsequent shots is no longer valid, so they need to adjust. Sometimes this will leave the player with a choice between deliberately ending the break now with a safety shot or taking on a more difficult or risky shot in order to continue the break. These risk assessments seem to improve with player experience (“match fitness”) and are also heavily influenced by the timing within the game (e.g. whether this player has a large advantage in terms of frames won over the opponent).

This is similar to test planning when we’re using exploratory testing – planning just enough ahead so we know where we’re heading, but being mindful that we will probably need to adjust as we go along, depending on what happens along the way. It is this ability to course adjust that gives power to exploratory testing and also allows the professional snooker player to deal with the flow of the balls during play.

The snooker player also has black swan events to worry about. One of the most notable of these events is the so-called kick – this is where either the cue ball or the object ball literally jumps in the air slightly after receiving contact from the cue or the cue ball respectively. This is almost always bad news for the player, since the angle on either ball is disturbed and contact is rarely clean. Many a seemingly simple shot has been scuppered by a kick. (This is why you will see players requesting balls – particularly the cue ball which becomes dirty with residual chalk from the tip of the cue – be cleaned frequently especially at critical points in a break.)

We can take a few cues (forgive the pun) from snooker when we’re thinking about our test execution/planning – consider your options just a few moves ahead, take considered risks, and be patient when you need to be. And those black swans are always lurking…

Oh, and for the sake of completeness, the tournament was won by Englishman Judd Trump (world number 6) who defeated Aussie Neil Robertson (world number 1). That’s right, the world number 1 snooker player is from Melbourne, surely the most under-rated sports professional in this country with zero media coverage (even when he won the World Championship!).