The first TestBash conference to be held in Australia/New Zealand took place in Sydney on October 19. The well-established conference brand of the Ministry of Testing ensured a sell-out crowd (of around 130) for this inaugural event, quite an achievement in the tough Australian market for testing conferences. The conference was held in the Aerial function centre at the University of Technology in Sydney.
The Twitter hashtag for the event was #testbash (from which I’ve borrowed the photos in this post) and this was very active across the conference and in the days after.
I was there to both attend and present at the conference. In fact, I would be co-presenting with Paul Seaman on our volunteer work teaching software testing to young adults on the autism spectrum. It was great to have this opportunity and we were humbled to be selected from the vast response the conference had to its call for papers.
The event followed the normal TestBash format, viz. a single day conference consisting of a single track with an opening and closing keynote plus a session of “99 second talks” (the TestBash version of lightning talks). Track sessions were 30 or 45 minutes in duration, generally with very little time after each talk for questions from the audience (especially in the case of the 30-minute slots).
Early arrivals were rewarded with the opportunity to participate in a Lean Coffee session out on the balcony at the Aerial function centre, a nice way to start the day in the morning sunshine (and with pretty good barista coffee too!).
The conference proper kicked off at 8.50am with a short opening address from the event MC, Trish Koo. She welcomed everyone, gave some background about the Ministry of Testing and also gave a shout out to all of the sponsors (viz. Enov8, Applitools, Gumtree, Tyro and Testing Times).
The opening keynote came from Maaret Pyhajarvi (from Finland) with “Next Level Teamwork: Pairing and Mobbing”. Maaret is very well-known for her work around mobbing and this was a good introductory talk on the topic. She mentioned that mobbing involves everyone in the team working together around one computer, which helps learning as everyone knows something that the others don’t. By way of contrast, she outlined strong-style pairing, in which “I have an idea, you take the keyboard to drive”. In this style, different levels of skill help, being unequal at the task is actually a good thing. Maaret said she now only uses pairing as a way to train people, not to actually test software. In a mobbing scenario, there is always one driver on the keyboard who is only following instructions and not thinking. A designated navigator makes decisions on behalf of the group. The roles are rotated ever four minutes and a retro is held at the end of every session. Maaret also noted the importance of mixing roles in the mob (e.g. testers, developers, automation engineers). This was a strong opening keynote with content pitched at just the right level for it to be of general interest.
Next up was a 30-minute talk from Alister Scott (from Automattic) with “How Automated E2E Testing Enables a Consistently Great User Experience on an Ever Changing WordPress.com”. He introduced his talk by giving some context about the way the company is organized – 800 people across 69 countries, with everyone remote (i.e. no offices!), and all internal communications being facilitated by WordPress (dogfooding). Alistair structured his talk as a series of problems and their solutions, starting with the problem of broken customer flows in production (when they moved to continuous delivery). Their solution to this problem was to add automated end-to-end testing of signup flows in production (and only in production). This solution led to the next problem, having non-deterministic end-to-end tests due to ever-changing A/B tests. The solution to this problem was an override of A/B tests during testing. The next problem was these new tests being too slow, too late (only in production) and too hidden, so they moved to parallel tests and adding “canaries” on merge (before deployment), simple tests of key features (signing up and publishing a page) designed to give fast feedback of major breaking changes. This led to the next problem, having to revert merges and slow local runs to which the solution was having live branch tests with canaries on every pull request. This led to the observation that, of course, canaries don’t find all the problems, so the solution then was to add optional full test suites on live branches. Even then, a problem persisted with Internet Explorer 11 and Safari 10 specific issues, so IE11 and Safari 10 canaries were added. The final problem is still current, in that people still break end-to-end tests! This was a nicely structured short talk about a journey of end-to-end testing and how solving one problem led to another (and ultimately has put them in a position of having no manual regression testing), good content.
A welcome break for morning tea and a chance to catch up with familiar faces came next before the delegates reconvened, with Enov8 getting the chance for a 99-second sponsor talk before sessions resumed.
First up after the break was a 30-minute session thanks to Michele Playfair (of YOW!) with “A Tester’s Guide to Changing Hearts and Minds”. Her key message was that the ability to change people’s opinions about testing was essentially a marketing exercise and she introduced the “4 P’s of marketing”, viz. Product, Price, Promotion and Placement. She argued that, as testers, we need to be better at defining our product (we should be able to answer questions like “what do you do here?”) and also promoting ourselves (by building relationships and networks, and revealing our value). This was a good short talk from Michele, a different angle on the topic of testers describing and showing their value.
Next up was Peter Bartlett (of Campaign Monitor) with a 45-minute talk on “Advancing Quality by Turning Developers into Quality Champions”. He defined a “quality champion” as “a developer who actively promotes quality in their team”, with this being a temporary role (typically lasting six months or so) which is rotated amongst the team. He generally selects someone who already displays a quality mindset or is an influencer within the team to take on the role initially and then trains them via one-on-one meetings, contextual training and against set goals. He encourages them to ask questions like “what areas are hard to test and why?”, “what can I do to make it easier for you to develop your code and be confident in its quality?”, and “what’s the riskiest piece of what you’re working on?”. Pete holds regular group meetings with all of the quality champions, these might be demo meetings, lean coffees or workshops/activities (e.g. how to write good acceptance criteria, dealing with automation flakiness, playing the dice game, introducing a new tool, how to use heuristics, live group testing). He has noted some positive changes as a result of using this quality champions model, including increased testability, a growth in knowledge and understanding around quality, new automation tests and performance tool testing research. Pete wrapped up with some tips, including starting small, taking time to explain and listen (across all project stakeholders), and to keep reviewing. This was a similar talk to Pete’s talk at the CASTx18 conference earlier in the year but it felt more fully developed here, no doubt as a result of another six months or so of trying this approach in Campaign Monitor.
As the clock struck noon, it was time for Paul Seaman (of Travelport Locomote) and I to take the big stage for our 30-minute talk, “A Spectrum of Difference – Creating EPIC Software Testers”. We outlined the volunteer work we’ve been doing with EPIC Assist to teach software testing to young adults on the autism spectrum (a topic on which I’ve already blogged extensively) and we were pleased with how our co-presenting effort went – and we thought we looked pretty cool in our EPIC polo shirts! We managed to finish up just about on time and the content seemed to resonate with this audience.
With our talk commitment completed, it was lunch hour (albeit with very limited vegan options despite pre-ordering) and it was good to get some fresh air and sunshine out on the venue’s balcony. Paul and I received lots of great feedback about our talk during lunch, it’s always so nice when people make the effort to express their thanks or interest.
Returning from lunch, it was Applitools’ turn to get their 99-seconds of fame as a sponsor before presentations resumed, in the form of a 45-minute session by Adam Howard (of TradeMe) with “Exploratory Testing: LIVE”. This was a really brave presentation, with Adam performing exploratory testing of a feature in the TradeMe website (New Zealand’s EBay) that had been deliberately altered by a developer in ways Adam was not aware of (via an A/B deployment in production). It was brave in many ways: he relied on internet connectivity and a stable VPN connection back to his office in New Zealand, and also exposed himself to testing a feature for the first time in front of 130 eagle-eyed testers! He applied some classic ET techniques and talked through everything he was doing in very credible terms, so this session served as an object lesson to anyone unfamiliar with what genuine exploratory testing looks like and how valuable it can be (Adam unearthed many issues, some of which probably weren’t deliberately introduced for the purposes of his session!). Great work from a solid presenter.
The following 30-minute talk was Paul Maxwell-Walters with “Avoid Sleepwalking to Failure! On Abstractions and Keeping it Real in Software Teams”. This was a really serious talk, high on well-researched content and it was a struggle to give all the content the coverage it deserved in such a short slot. He introduced the ideas of hyper-normalization and hyper-reality before getting into talking about abstractions, viz. “quality” and “measurement”. I particularly liked this quote from his talk, “bad metrics and abstractions are delusional propaganda”! This maybe would have been a better talk if he’d tried to cover less content, but nevertheless it was really engaging and interesting stuff.
The final break came next before we reconvened for the push to the finish. First up after the break was another 99-second sponsor talk, this time Anne-Marie Charrett (conference co-organizer) on her consultancy business, Testing Times.
The last 30-minute slot went to first-time conference presenter, Georgia de Pont (of Tyro), with “Test Representatives – An Alternative Approach to Test Practice Management” and she presented very confidently and calmly on her first outing. She outlined how Tyro moved to having testers embedded in agile teams and, while there lots of positives from doing this, there was also a lack of consistency in test practice across the teams and no way to consider practice-wide improvements. She went on to talk about the move to “test representatives” (who are themselves embedded testers in teams), one from each tribe, who have a mission to provide a community for testers and act as points of contact for initiatives impacting testing. Each representative then shares the outputs of the representatives group with their team. Initiatives the representatives have covered so far include clarifying the role of the embedded tester, improving the test recruitment process (via a pair testing exercise), onboarding new test engineers, performance criteria for test engineers, upskilling test engineers, co-ordinating engineering-wide test engineers and developing a Quality Engineering strategy. There is also a stretch goal for testers to operate across teams. Georgia’s recommended steps to implement such a model were to start small, look for volunteers over selection, communicate the work of the representatives across the organization, survey to get feedback, hold retros within the representatives group and foster support from engineering leadership. This was a solid talk, especially impressive considering Georgia’s lack of experience in this environment.
The final presentation of the day was a closing keynote thanks to Parimala Hariprasad (of Amadeus) with “Enchanting Experiences – The Future of Mobile Apps”. Her time on stage was pretty brief (using only a little over half of her 45-minute slot before Q&A) but was very engaging. She argued that designing great products isn’t about good screens, it’s about great – enchanting – experiences. She said we should think more about ecosystems than apps and screens as systems become more complex and interconnected. Her neat slides and confident presentation style made her messaging very clear and she also handled Q&A pretty well.
The last session of the conference was dedicated to “99 second talks”, the TestBash version of lightning talks in which each speaker gets just 99 seconds to present on a topic of their choice. There were plenty of volunteers so the short keynote was made up for by more 99s talks, some 18 in total, as follows:
- Sam Connelly – on depression (and introducing “spoon theory“)
- Amanda Dean – on why she believes testing is not a craft and should be thought of as a profession
- Maaret Pyhajarvi – live exploratory testing of an API (using the Gilded Rose example, as per her recent webinar on the same topic)
- Cameron Bradley – on why a common automation framework is a good thing (based on his experience of implementing one at Tabcorp)
- Dany Matthias – on experimenting with coffee!
- Melissa Ngau – on giving and receiving feedback
- Geoff Dunn – on conflict and how testers can help to resolve it
- Catherine Karena – on mentoring
- Nicky West – what is good strategy?
- Kim Nepata – Blockchain 101
- Sunil Kumar – mobile application testing: how, what and why?
- Said – on rotations and why they’re essential in development teams
- Melissa (Editor Boss at Ministry of Testing) – living a dream as a writer
- Leela – on transitioning from a small to a large company
- Haramut – demo of a codeless automation framework
- Trish Koo – promoting her test automation training course
- Anne-Marie Charrett – “Audience-Driven Speaking”
- Maaret Pyhajarvi – promoting the Speak Easy mentoring programme
After a brief closing speech from Trish Koo, the conference closed out. The action then moved to the nearby Knox Street Bar for a post-conference “meetup” with free drinks courtesy of sponsorship from Innodev. This was a fun evening, relaxing with old friends from the testing community and talking conference organizing with others involved in this, erm, fun activity!
I’ll finish off this blog post with some general thoughts on this conference.
The standard of presentations was excellent, as you might expect from a TestBash and the massive response to their call for papers (around 250). The mix of topics was also very good, from live exploratory testing (I would love to see something like this at every testing conference) to automation to coaching/training/interpersonal talks.
The single track format of all TestBash conferences means there is no fear of missing out, but the desire to pack as many talks as possible into the single day means very limited opportunity for Q&A (which is often where the really interesting discussions are). I personally missed the deep questioning that occurs post-presentations at conferences like CAST.
Although the sponsor talks were kept to short 99-second formats, I still find sponsor talks of any kind uncomfortable, especially at a relatively expensive conference.
Paul and I enjoyed presenting to this audience and the Ministry of Testing do an excellent job in terms of pre-gig information and speaker compensation (expensing literally door-to-door). We appreciated the opportunity to share our story and broaden awareness of our programme with EPIC Assist.
Rockin' and Testing All Over The World 