Testers, how good is your waggle dance?

Since returning from Europe, I’ve been enjoying a new commuting option in the shape of a ferry service across the bay and this relaxing 90-minute trip is proving to be a great alternative to my only previous option of a 30km drive plus one-hour train journey.

Cruising to Melbourne in this way has opened up some more time for reading and I’m just finishing The Wisdom of Crowds by James Surowiecki.

The book explores a simple idea: “large groups of people are smarter than an elite few, now matter how brilliant – better at solving problems, fostering innovation, coming to wise decisions, even predicting the future.”

It’s enjoyable stuff and, of course, I can’t help but draw connections between some of its content and software testing.

The following paragraphs from the book describe how bees go about finding good food sources for their hive (emphasis is mine):

Bees are remarkably efficient at finding food. According to Thomas Seeley, author of “The Wisdom of the Hive”, a typical bee colony can search six or more kilometres from the hive, and if there is a flower patch within two kilometres of the hive, the bees have a better-than half chance of finding it. How do the bees do this? They don’t sit around and have a collective discussion about where foragers should go. Instead, the hive sends out a host of scout bees to search the surrounding area. When a scout bee has found a nectar source that seems strong, he comes back and does a waggle dance, the intensity of which is shaped, in some way, by the excellence of the nectar supply at the site. The waggle dance attracts other forager bees, which follow the first forager, while foragers who have found less-good sites attract fewer followers and, in some cases, eventually abandon their sites entirely. The result is that bee foragers end up distributing themselves across different nectar sources in an almost perfect fashion, meaning that they get as much food as possible relative to the time and energy they put into searching. It is a collectively brilliant solution to the colony’s food problem.

What’s important, though, is the way the colony gets to that collectively intelligent solution. It does not get there by first rationally considering all the alternatives and then determining an ideal foraging pattern. It can’t do this, because it doesn’t have any idea what the possible alternatives – that is, where the different flower patches – are. So instead, it sends out scouts in many different directions and trusts that at least one of them will find the best patch, return, and do a good dance so that the hive will know where the food source is.

I immediately saw similarities with exploratory testing when I read this.

When we’re looking to identify interesting or risky areas of a product under test, our initial charters are quite loose, since we don’t necessarily have a good idea of where to look yet. Debriefing our sessions gives us the chance to narrow in on where to look next or where to return to as fertile ground for finding interesting information about the product.

So, as a tester returning information to your team, how good is your waggle dance?

(The Wisdom of Crowds is an interesting read – and not just for the story of the waggle dance!)

A new year and a new challenge, the EPIC TestAbility Academy

I’ve already reviewed 2016 in a previous blog post and it’s now time to look forward to a new year.

I expect I’ll make it to a few testing conferences (as usual) during the year, kicking off quite soon with the first CAST to be held outside of North America, the CASTx conference in Sydney. I’m looking forward to seeing some AST folks there as well as catching up with familiar faces from the Australia/New Zealand testing community – and hopefully making some new connections too. I have no conference speaking commitments lined up for 2017 yet, but that’ll probably change.

There will also inevitably be various work trips to cover the major offices to which my responsibilities extend, so at least West Coast US, China and Czech Republic are likely stamps in the diminishing free pages of my passport.

I’m most excited about a new community project I’m working on in 2017 with Paul Seaman. We had both been looking for an opportunity to give back to the community in some way and a lucky meeting with the good folks from the not-for-profit EPIC Assist organization has provided us with just that. EPIC Assist has a division called Recruit Assist, which does great work in matching candidates with employers, specifically candidates with disabilities who cannot or chose not to utilize Disability Employment Services.

We approached EPIC with the idea of running a software testing course so that they could potentially find new client companies in the IT sector who would have access to good candidates with a solid software testing training already behind them. It was pleasing to see how open EPIC were to the idea and, long story short, we expect to start the first run of our software testing training programme – to be known as the EPIC TestAbility Academy – very soon. (Note that EPIC Recruit Assist will be funding the costs of the programme, such as venue hire, etc., while Paul and I are offering our professional services on a voluntary basis.)

EPIC will source candidates on the autism spectrum to participate in the programme. Putting together the course and delivering it will be a huge challenge for both Paul and me, requiring us to build an understanding of the best ways to interact with the candidates and being sympathetic to their distinct learning style. While we both have considerable experience in presenting training to groups (Paul as a teacher in a past life), we expect there to be humps in the road with these groups and we both expect to learn a lot along the way.

I’m not setting success criteria around the work with EPIC Recruit Assist at this stage. It’ll be great if most of the students who start the first course make it through to the end, if there’s some engagement and maybe we inspire a few students to want to consider a career in testing or IT in general. If we manage to find genuine software testing employment for one or more of the students, that would be a fantastic achievement both for the student and for us as teachers of the material. We couldn’t do any of this without the support of EPIC Recruit Assist, of course, and their positive attitude from day one and their belief that Paul & I could do this have been very humbling.

I’ll blog more on ETA as the programme kicks off and we have experiences to share. It’s time to give back and this challenge will no doubt be a highlight of 2017.

(My friend and testing partner in this initiative, Paul Seaman, has also blogged about this new venture.)

RIP Rick Parfitt (12/10/1948-24/12/2016)

This blog typically relates to my professional life as a specialist in software testing, so I hope you will indulge me with a much more personal blog post than usual here today.

It’s not exactly a secret that I’m a massive fan of the UK rock band, Status Quo. I’ve been seeing them live for just over thirty years and collecting their records and memorabilia for even longer. The band in its various forms over the years has given me some of the most memorable times of my life and I’ve made the most incredible friendships as a result of following them.

My passion – sometimes referred to as an “obsession” – has brought me great joy, but on Christmas Eve 2016 it has now also brought me great sadness. With incredible timing, just the day after the “Last Night of the Electrics” tour concluded in Liverpool (UK), I received the news that Rick Parfitt had passed away. Being in the UK at the time, it was big news – and quite surreal watching the news stories talking about the rock hero we all thought was indestructible. I was unsure whether to blog immediately to capture my raw emotions but Christmas didn’t feel like the right time. Two weeks after the event, it feels like an appropriate time to take a moment for me to put virtual pen to virtual paper in some kind of tribute to Rick.

I appreciate that many of you reading this may not have heard of Rick, so a few words by way of historical record are in order.

It was a chance meeting at Butlin’s holiday camp in Minehead in 1965 that brought a young Rick Parfitt (then performing in a trio called The Highlights) together with a young Francis Rossi (then performing in a rock combo called The Spectres) with Rick officially joining the band that would become Status Quo in 1967. It was a partnership that lasted almost fifty years, quite incredible. Quo’s longevity is well documented and they remain the band with the most Top 40 hits in the UK (with an untoppable 57).

This early colour TV appearance shows a very young Rick in full flow, his face barely visible thanks to the long blonde hair:


The rock and roll excesses of the 70s and 80s certainly didn’t pass Rick by and Quo’s conformance to the sex, drugs and rock & roll mantra is also well documented.

Rick played hard on stage and lived life to the full off it. After decades of mistreatment, his body showed the first signs of cracking, with his first major health scare coming in 1997 when he underwent a quadruple heart bypass – but he was still back on stage a few weeks later. In 2005, he had a throat cancer scare and then it was more heart trouble in 2011 with surgery required after another heart attack and then another heart attack in 2014. His most recent heart attack came shortly after a very hot gig in Turkey in June 2016 and this ultimately led to him retiring from touring duties with the band.

He will perhaps be best remembered for his legendary rock star blonde locks (which stayed with him even in his sixties) and his incessant rhythm guitar skills on his faithful white Fender Telecaster. There are few rock rhythm players around who could go head to head with Rick and the following relatively recent clip (well, in Quo terms anyway, it’s from the 2009 Glastonbury festival) shows his power, opening just one of many thousands of Quo gigs with the iconic “Caroline”. Rick with his back to the “wall of death” of amps thrashing out the opening chords of this song is surely one of rock’s most recognizable images.

Rick was also a significant part of the songwriting ammunition for Quo, penning (and co-writing) a number of their best known songs, including “Whatever You Want”, “Rain”, “Again and Again”, “4500 Times”, “Backwater”, “Little Lady” and “Mystery Song”.

I’ve been lucky enough to see Quo live over 250 times and also very fortunate to have met the band, including Rick, on many occasions. He was always friendly and ready to crack a joke, always the rock star but also always just one of the lads.

Some of my most memorable meetings with Rick occurred on Australian shores over the last 20 years or so, where it’s easier for the band to mingle with the public than in Europe where they are much better known. Rick once said that his favourite place on Earth was in Australia – a place called Magnetic Island off the coast of Queensland – and he always seemed relaxed and happy being downunder. It is such a shame we will not get to welcome him to our shores again. The following photo comes from a meet & greet at the gig in Wollongong in 2006, happy memories indeed. I will miss seeing him up their doing what he did best – but the vast recorded legacy will always mean he is but a CD spin away.


Before leaving the UK to head back to Australia, I visited the tribute to Rick outside the Hammersmith Apollo (formerly Odeon). Each day I visited, more flowers and trinkets had been added to the tribute and different fans were there to pay their respects. Some took the chance to embrace fellow fans and let their emotions out, while others chose more solitary personal reflection at the site. This was an important thing for many of us, just somewhere to go to share our sadness with others who “get it”.


RIP Rick, keep on rocking, you will never be forgotten.

“Playing loud, playing clear
The song will never change
The memory will always be so near” (A Year, 1972)

(For an excellent collection of tributes to Rick, see the brilliant www.rickparfitt.de site.)

2016 in review

As another year begins, it’s a good time to look back on the last one and 2016 was another busy year for me professionally.

I managed 22 posts on this blog in 2016, well in excess of my (internal) target cadence of one post per month. My blogging was irregular, though, with not much activity in the last few months while I was travelling (more on that later). I’ve tried to focus on quality of posts over quantity and it’s pleasing to note that my blog received three times as many views in 2015 as it did in 2016, so hopefully I’m doing something right. (If there are topics you’d like to see me talking about here, please let me know.)

Community events

The testing community worldwide seemed to blossom during 2016, which was great to see. More and more meetups and other opportunities for testers to meet and discuss their craft can only be a good thing both for the industry and for the professional standing of testers too.

The first half of the year again saw me heavily involved in the TEAM meetup group here in Melbourne. Although I am no longer contributing to the group, it’s great to see how far it’s come from humble beginnings in 2015 to now being a very active and well-attended meetup. I hope it continues to provide an avenue for testers in Melbourne to come together and talk testing. (I blogged about these TEAM meetups here, herehere, and here.)

I also had the chance to attend an overseas meetup when I found myself in Belfast, Northern Ireland, in October. It was a warm welcome at the Belfast Testers Meetup and this young meetup group showed a lot of enthusiasm and passion for testing. (I blogged about my time in Belfast here.)

My last community event of the year saw me taking part in the invite-only Cambridge Exploratory Workshop on Testing (CEWT) in November. I was lucky enough to be in the UK at just the right time and readily accepted James Thomas’s invitation to be part of their peer conference. I always enjoy these small peer conferences as they allow for deep discussion and have always resulted in me making new connections with interesting testers from around the world – and this event was certainly no different in that regard. (I blogged about my attendance at CEWT here.)


2016 was a relatively quiet year for conferences for me, at least on paper, with just two testing conferences and one agile-centric one.

First up was the inaugural Australian Testing Days conference in Melbourne, for which I was a co-organizer. It was a huge amount of work pulling the conference together but also a lot of fun along the way and I’m proud of the great event that transpired in May. (I blogged about the experience of organizing this conference here.)

Next up was my first non-testing conference for many years, the Agile On The Beach conference held in Cornwall (UK) in September. It was interesting to hear talks of a more general nature and also on very different topics – and any conference with its group party event held on a beach has to be worth attending, right?! (I blogged about attending this conference here.)

Finally, it was over to California in October to attend and present at the massive  STARWest event. As my only conference speaking commitment of the year, I was really pleased with how the talk went and it was an enjoyable week at the Disneyland Resort taking in everything this enormous event has to offer. (I blogged about attending and presenting at STARWest here.)

Other stuff

I was pleased to be asked to act as guest editor for Testing Trapeze magazine for one edition in 2016 and it was an enjoyable experience bringing the articles together, getting them reviewed, and then finalizing the content in the magazine. Katrina Clokie has put a great team in place to ensure high quality in every edition and hopefully “my” edition added to its legacy in a positive way. (I blogged about my guest editor role here.)

Apart from my usual business and conference travel, I based myself in Europe for the last four months of 2016 and this gave me the chance to do some of the UK events I’ve noted above as well as catch up with friends and family in the UK. There were also some Status Quo gigs and the tragic loss of our rock and roll hero Rick Parfitt during this time (but that’s a story for another blog post).


Here’s to a successful and healthy 2017 – Happy New Year to you all!

ER: Attending the Cambridge Exploratory Workshop on Testing (CEWT)

One of the great things about being out of Australia for a while is the ability to experience testing community events in other parts of the world.

I recently attended a Belfast Testers meetup and, shortly afterwards, received an invitation from James Thomas to take part in the third Cambridge Exploratory Workshop on Testing – an invitation I readily accepted!

This peer workshop took place on Sunday 6th November and was held in the offices of games developer Jagex on the (enormous) Cambridge Science Park, with a total of 12 participants (the perfect size for such an event), as follows:

Michael Ambrose (Jagex)

James Thomas, Karo Stoltzenburg, Sneha Bhat, Aleksandar Simic (all from Linguamatics)

Alan Wallace (GMSL)

James Coombes (Nokia)

Neil Younger (DisplayLink)

Chris Kelly (Redgate)

Iuliana Silvasan

Chris George (Cambridge Consultants)

Lee Hawkins (Quest)

The workshop theme was “Why do we Test, and What is Testing (Anyway)?” and, after some introductions and housekeeping about how the workshop would be run, it was time for the first ten-minute talk, from Michael Ambrose with “Teach Them to Fish”. He talked about teaching developers to test at Jagex, as well as upskilling testers to be pseudo-developers. He said there was a technical need to cover more and more as well as a desire to get testers learning more (as a different approach to, say, pushing developers to do automation). Michael noted that there were a number of implications of these changes, including the perception of testers, working out what’s unique about what testers do, and knowing how far to go (getting testers to the level of junior developers might be enough). This was an interesting take on the current “testers need to be more technical” commentary in the industry and the twenty-minute discussion period was easily filled up.

Next up was James Coombes with “Who should do testing and what can they test?” He talked about the “I own quality” culture within Nokia and how he sees different roles being responsible for different aspects of quality. James suggested that developers should find most of the bugs and fix them, while QA then find the next highest number of bugs. Security testers act as specialists with generally few (but important) bugs being found. Documenters/trainers are well placed to find usability bugs, while customer support staff have good knowledge of how customers actually use their products and so can provide good testing tours. Alpha test engineers are responsible for integration/end-to-end testing and catch the low frequency bugs. Finally, customers are hopefully finding the very low frequency bugs. This was an interesting talk about getting everyone involved in the testing activity (and highlighted the “testing is an activity, not a role” idea). I particularly liked what James said about unit testing – “if someone changes your code and they don’t know they’ve broken it, it’s not their problem, it’s yours for not writing a good enough unit test”.

After a short break, I was up next with my talk “What is Testing? It depends …” I decided to tackle the latter half of the theme (i.e. the “what” rather than the “why”) and my idea was to discuss what testing means depending on the perspective of the stakeholder. We focus a lot of time and effort in the community on refining a definition of testing (and I favour the James Bach & Michael Bolton definition given towards the end of the Exploratory Testing 3.0 blog post) but this (or any other) definition is probably not very helpful to some stakeholders. I covered anumber of perspectives such as “Testing is a way to make money” (if you’re a testing tools vendor or a testing outsourcing services provider), “Testing is a cost centre” (if you’re a CFO) and “Testing is dead” (if you’re a CxO type reading some of the headline IT magazines & websites). There was a good discussion after my talk, mainly focused on the cost centre perspective and how this has impacted people in their day-to-day work. I was pleased with how my talk went (especially given the short time I had to prepare) and received some good feedback, particularly on the concise nature of the slides and the confidence with which it was presented. My slide deck can be seen at What is Testing? It depends…


The last session before lunch saw Aleksandar Simic with “A Two-day Testing Story”. He did a fine job of breaking down a two-day period in his work into various different activities, some testing-related (e.g. pairing on test design) and some not (e.g. working with IT support on a networking issue). Aleksandar’s level of self-inspection was impressive, as was his naming of the various activities, learning opportunities and challenges along the way. His “testing diary” seems to be working well for him in identifying and naming his testing activities and this would make an interesting conference talk with some further development.

Lunch provided a good chance for us all to chat and unwind a little after the intensive morning spent talking testing.

First up after the lunch break was Karo Stoltzenburg with “I test, therefore I am”. She had adopted the idea of substitution in preparing her talk so looked to answer the question “Why do I test?” and see where that took her. Karo’s answer was “Because I like it” and then she explored why she liked it, identifying investigation, learning, exploring, use of the scientific method, collaborating, thinking in different contexts and diversity as aspects of testing that appealed to her. I liked Karo’s closing remarks in which she said “I test because it makes me happy, because it’s interesting, challenging and varied work”. We really need more positive messages like Karo’s being expressed in the testing community (and wider still), so I’d love to see this become a full conference talk one day. She did a good job of communicating her passion for testing and there were some interesting discussions in the group following her talk, with a degree of agreement about why testing is so engaging for some of us.

The sixth and final talk of the day came from James Thomas with “Testing All the Way Down, and Other Directions” He walked through an in-depth analysis of Elisabeth Hendrickson’s “Tested = Checked + Explored” from her book, Explore It! James decided to explore this definition of testing using techniques from that definition which wouldn’t classify his actions as testing. He described how he’d interacted with Elisabeth on some of his questions after exploring the idea in this way and finally presented his proposed alternative definition of testing as “the pursuit of actual or potential incongruity” (Note that James more fully describes this talk in his blog post, Testing All the Way Down, and Other Directions) The main focus of discussion after James’s talk was around his proposed definition of testing and I’ll be following the broader community’s response to his proposal with interest.

A few discussion points arose during the day for which we didn’t have time to go deep between talks, so we dedicated ten minutes to each of the following topics to close out the workshop content:

  • Quality – what does it mean? (Weinberg definition, but are others more helpful?)
  • Domain knowledge (can bias you, can empathy with the end user be a disadvantage? How do we best adjust strategy to mitigate for any lack of domain knowledge?)
  • Evaluating success (how do we measure the success of spreading testing into development and other disciplines?)
  • Is testing just “the stuff that testers do”? (probably not!)
  • How do we make a difference? (blogging, workshops in our own workplaces, brown bag sessions, broader invitation list to peer conferences)

To wrap up, a short retrospective was held where we were all encouraged to note good things to continue, anything we’d like to stop doing, and suggestions for what we should start to do. There were some good ideas, briefly discussed by the group and I’d expect to see some of these ideas taken up by the CEWT organizers as part of their fourth incarnation.

The CEWT group standing outside number 10 Downing Street (or inside the Jagex office, maybe):


This was a really good day of deep-diving with a passionate group of testers, exactly what a peer conference should be all about. Thanks again to James for the invitation and thanks to all the participants for making me so welcome.

For reflections on the event from others, keep an eye on the CEWT blog at


Living up to my handle: rocking and testing in Belfast

I was lucky enough to find myself in Northern Ireland recently, with a trip based around attending one of Status Quo‘s final ‘electric’ shows in Belfast. This was my first trip to Northern Ireland and it was a very enjoyable few days. Visiting the stunning North coast around the Giant’s Causeway was a highlight, as was exploring the varied districts of the city of Belfast itself, from the colourful (both literally and figuratively) walls of the Falls/Shanklin area to the stunning City Hall.

So, firstly, the “rocker” part of the trip. The big SSE Arena would be home to Status Quo for one night on Friday 30th October. The Emerald Isle has always been good territory for the band and this gig would be no exception. Luckily, our hotel – the Premier Inn Titanic Quarter – was literally next door to the venue and our room had a view over the loading area at the back, so a perfect spot to keep an eye on proceedings during the day in the lead up to the gig. An early queue formed (as usual) and it was good to catch up with friends there and also make some new ones (one very generous bloke in the queue kindly gave my wife a free ticket so she could join me in attending the gig!). With doors opening a little later than usual (due to the Quo crew arriving late because of travel problems from the mainland to Ireland), the queue was very long by the time we were allowed in and it was a sprint down to the barrier, securing a spot centre stage.

Support came from Uriah Heep and they got a great reception, having not played live in this part of the world for over thirty years. While their brand of heavy rock (somewhat like Deep Purple to my ears) is not really to my taste, they did a good job of getting the 7-8000 strong audience involved and provided a good warm-up during their hour-long set.

At just after 9pm, it was finally time for Quo to take the stage and it was another top performance, with Rick Parfitt’s replacement, Richie Malone, getting a particularly warm welcome in his home territory. There were no setlist surprises (it is Status Quo after all!) but, as always with a live gig, there are subtle differences from night to night and the encouragable crowd made this a very enjoyable show.


Now, where does the “tester” fit into the trip? Well, luckily for me, the fairly new Belfast Tester meetup group had announced a meetup during one of the nights I was in Belfast (and thankfully not the same night as the Status Quo gig), so I decided to go along and see what this new testing community looks like.

It was the fourth meetup of the Belfast Testers Meetup group and it took place on the evening of Thursday 29th October, in the boardroom of Shopkeep. I was warmly welcomed as a “one off” attendee of the meetup, which drew a crowd of about 25 (many of whom were first-timers as you’d expect at such a new meetup). It’s always good to see a new community of testers being built and this one should do well, with strong leaders and a burgeoning IT industry in the city.

Meetup co-organizer Neill Boyd kicked off the meetup, after gathering up the crowd enjoying the hospitality and impressive surroundings of the Shopkeep office (all very “startuppy”) and funneling them into the boardroom. It was cosy, with 20+ of us in the room but it made for a good space for easy conversation and Q&A with the presenters.

His introduction was handy for newcomers, announced that the TestBash conference is adding Belfast to its growing list of city destinations in 2017, and outlined the agenda of two talks to fill the evening.

First up was Allan Hunter (Senior Tester at PwC Emerging Technologies team) with “User Focused Testing”. This short talk discussed how user research is a key ingredient in powering insightful testing. He talked on his experiences of user research and how this can be applied to more traditional testing activities. His basic message was that we as testers are well placed to test the problem and not just the solution. He packed a lot of cool content into ten minutes then held his own through a lengthy period of questioning.

Secondly, we had Ursula Wlodarczyk (a tester working in the startup, SaltDNA) with “Tester v s Designer: Why tester can make a great (UX) designer”. Ursula’s was a much longer talk and tried to cover a huge amount of ground, enough for at least a couple of good talks I’d say. She talked about exploring a tester’s skillset in the context of the User Experience field and, similarly to Allan, she recognizes that testers can provide great insights into design (not just at the user interface either) and organizations should be allowing testers the opportunities to engage in design meetings and provide their valuable insights and suggestions before the software is built. A very long talk for a meetup (it would actually make a decent conference talk), but some nice ideas and good resources.

The meetup kicks off  Allan Hunter presenting  Ursula Wlodarczyk presenting

So, thanks to Belfast and thanks to the two very different communities – Status Quo and testing – that continue to give me the chance to rock and test all over the world.

My ER of attending and presenting at STARWest 2016

I recently had the pleasure of heading to Southern California to attend and present at the long-running STARWest conference. Although the event is always held at the Disneyland Resort, it’s a serious conference and attracted a record delegation of over 1200 participants. For a testing conference, this is just about as big as it gets and was probably on a par with some recent EuroSTARs that I’ve attended.

My conference experience consisted of attending two full days of tutorials then two conference days, plus presenting one track session and doing an interview for the Virtual Conference event. It was an exhausting few days but also a very engaging & enjoyable time.

Rather than going through every presentation, I’ll talk to a few highlights:

  • Michael Bolton tutorial “Critical Thinking for Software Testers”
    The prospect of again spending a full day with Michael was an exciting one – and he didn’t disappoint. His tutorial drew heavily from the content of Rapid Software Testing (as expected), but this was not a big issue for his audience (of about 50) here as hardly anyone was familiar with RST, his work with James Bach, Jerry Weinberg, etc. Michael defined “critical thinking” to be “thinking about thinking with the aim of not getting fooled” and he illustrated this many times with interesting examples. The usual “checking vs. testing”, critical distance, models of testing, system 1 vs. system 2 thinking, and “Huh? Really? And? So?” heuristic familiar to those of us who follow RST and Bolton/Bach’s work were all covered and it seemed that Michael converted a few early skeptics during this class. An enjoyable and stimulating day’s class.
  • Rob Sabourin tutorial “Test Estimation in the Face of Uncertainty”
    I was equally excited to be spending half a day in the company of someone who has given me great support and encouragement – and without whose support I probably wouldn’t have made the leap into presenting at conferences. Whenever Rob Sabourin presents or teaches, you’re guaranteed passion and engagement and he did a fine job of covering what can be a pretty dry subject. In his audience of about 40, it was a 50/50 split between those on agile and waterfall projects and some of the estimation techniques he outlined suited one or other SDLC model better, while some were generic. He covered most of the common estimation techniques and often expressed his opinion on their usefulness! For example, using “% of project effort/spend” as a way of estimating testing required was seen as ignoring many factors that influence how much testing we need to do and also ignores the fact that small development efforts can result in big testing efforts. Rob also said this technique “belittles the cognitive aspects of testing”, I heartily agreed! Rob also cited the work of Steve McConnell on developer:tester ratios, in which he found that there was wide variability in this ratio, depending on the organization and environment (e.g. NASA has 10 testers to each developer for flight control software systems while it in business systems, Steve found ratios of between 3:1 and 20:1), rendering talk of an “industry standard” for this measurement seem futile. More agile-friendly techniques such as Wisdom of the Crowd, planning poker and T-shirt sizing were also covered. Rob finished off with his favourite technique, Hadden’s Size/Complexity Technique (from Rita Hadden), and this seemed like a simple way to arrive at decent estimates to iterate on over time.
  • Mary Thorn keynote “Optimize Your Test Automation to Deliver More Value”
    The second conference day kicked off with a keynote from Mary Thorn (of Ipreo). She based her talk around various experiences of implementing automation during her consulting work and, as such, it was really good practical content. I wasn’t familiar with Mary before this keynote but I enjoyed her presentation style and pragmatic approach.
  • Jared Richardson keynote “Take Charge of Your Testing Career: Bring Your Skills to the Next Level”
    The conference was closed out by another keynote, from Jared Richardson (of Agile Artisans). Jared is best known as one of the authors of the GROWS methodology and he had some good ideas around skills development in line with that methodology. He argued that experiments lead to experience and we gain experience both by accident and also intentionally. He also mentioned the Dreyfus model of skills acquisition. He questioned why we often compare ourselves as an industry to other “building” industries when we are very young compared to other building industries with hundreds or thousands of years of experience. He implored us to adopt a learner mentality (rather than an expert mentality) and to become “habitual experimenters”. This was an engaging keynote, delivered very well by Jared and packed full of great ideas.

Moving onto my track session presentation, my topic was “A Day in the Life of a Test Architect” and I was up immediately after lunch on the second day of the conference (and pitted directly against the legendary – and incredibly entertaining – Isabel Evans):

Room signage for Lee's talk at STARWest

I was very pleased to get essentially a full house for my talk and my initial worries about the talk being a little short for the one hour slot were unfounded as I ended up going for a good 45 minutes:


There was a good Q&A session after my talk too and I had to cut it to make way for the next speaker to set up in the same room. It was good to meet some other people in my audience with the title of “Test Architect” to compare notes.

Shortly after my talk, I had the pleasure of giving a short speaker interview as part of the event’s “Virtual Conference” (a free way to remotely see the keynotes and some other talks from the event), with Jennifer Bonine:

Lee being interviewed by Jennifer Bonine for the STARWest Virtual Conference

Looking at some of the good and not so good aspects of the event overall:


  • The whole show was very well-organized, everything worked seamlessly based on years of experience of running this and similar conferences.
  • There was a broad range of talks to choose from and they were generally of a good standard.
  • The keynotes were all excellent.

Not so good

  • The sheer size of the event was quite overwhelming, with so much going on all the time and it was hard for me to choose what to see when (and the resulting FOMO).
  • As a speaker, I was surprised not to have a dedicated facilitator for my room, to introduce me, facilitate Q&A, etc. (I had made the assumption that track talks – at such a large and mature event – would be facilitated, but there was nothing in the conference speaker pack to indicate that this would be the case.)
  • I’ve never received so much sponsor email spam after registering for a conference.
  • I generally stuck to my conference attendance heuristic of “don’t attend talks given by anyone who works for a conference sponsor”, this immediately restricted my programme quite considerably. There were just too many sponsor talks for my liking.

In terms of takeaways:


  • Continuous Delivery and DevOps was a hot topic, with its own theme of track sessions dedicated to it – there seemed to be a common theme of fear about testers losing their jobs within such environments, but also some good talks about how testing changes – rather than disappears – in these environments.
  • Agile is mainstream (informal polls in some talks indicated 50-70% of the audience were in agile projects) and many testers are still not embracing it. There seems to be some leading edge work from (some of) the true CD companies and some very traditional work in enterprise environments, with a big middle ground of agile/hybrid adoption rife with poor process, confusion and learning challenges.
  • The topic of “schools of testing” again came up, perhaps due to the recent James Bach “Slide Gate” incident. STARWest is a broad church and the idea of a “school of schools” (proposed by Julie Gardiner during her lightning keynote talk) seemed to be well received.
  • There is plenty of life left in big commercial testing conferences with the big vendors as sponsors – this was the biggest STARWest yet and the Expo was huge and full of the big names in testing tools, all getting plenty of interest. The size of the task in challenging these big players shouldn’t be underestimated by anyone trying to move towards more pragmatic and people-oriented approaches to testing.

Thanks again to Lee Copeland and all at TechWell for this amazing opportunity, I really appreciated it and had a great time attending & presenting at this event.