ER: Attending the Cambridge Exploratory Workshop on Testing (CEWT)

One of the great things about being out of Australia for a while is the ability to experience testing community events in other parts of the world.

I recently attended a Belfast Testers meetup and, shortly afterwards, received an invitation from James Thomas to take part in the third Cambridge Exploratory Workshop on Testing – an invitation I readily accepted!

This peer workshop took place on Sunday 6th November and was held in the offices of games developer Jagex on the (enormous) Cambridge Science Park, with a total of 12 participants (the perfect size for such an event), as follows:

Michael Ambrose (Jagex)

James Thomas, Karo Stoltzenburg, Sneha Bhat, Aleksandar Simic (all from Linguamatics)

Alan Wallace (GMSL)

James Coombes (Nokia)

Neil Younger (DisplayLink)

Chris Kelly (Redgate)

Iuliana Silvasan

Chris George (Cambridge Consultants)

Lee Hawkins (Quest)

The workshop theme was “Why do we Test, and What is Testing (Anyway)?” and, after some introductions and housekeeping about how the workshop would be run, it was time for the first ten-minute talk, from Michael Ambrose with “Teach Them to Fish”. He talked about teaching developers to test at Jagex, as well as upskilling testers to be pseudo-developers. He said there was a technical need to cover more and more as well as a desire to get testers learning more (as a different approach to, say, pushing developers to do automation). Michael noted that there were a number of implications of these changes, including the perception of testers, working out what’s unique about what testers do, and knowing how far to go (getting testers to the level of junior developers might be enough). This was an interesting take on the current “testers need to be more technical” commentary in the industry and the twenty-minute discussion period was easily filled up.

Next up was James Coombes with “Who should do testing and what can they test?” He talked about the “I own quality” culture within Nokia and how he sees different roles being responsible for different aspects of quality. James suggested that developers should find most of the bugs and fix them, while QA then find the next highest number of bugs. Security testers act as specialists with generally few (but important) bugs being found. Documenters/trainers are well placed to find usability bugs, while customer support staff have good knowledge of how customers actually use their products and so can provide good testing tours. Alpha test engineers are responsible for integration/end-to-end testing and catch the low frequency bugs. Finally, customers are hopefully finding the very low frequency bugs. This was an interesting talk about getting everyone involved in the testing activity (and highlighted the “testing is an activity, not a role” idea). I particularly liked what James said about unit testing – “if someone changes your code and they don’t know they’ve broken it, it’s not their problem, it’s yours for not writing a good enough unit test”.

After a short break, I was up next with my talk “What is Testing? It depends …” I decided to tackle the latter half of the theme (i.e. the “what” rather than the “why”) and my idea was to discuss what testing means depending on the perspective of the stakeholder. We focus a lot of time and effort in the community on refining a definition of testing (and I favour the James Bach & Michael Bolton definition given towards the end of the Exploratory Testing 3.0 blog post) but this (or any other) definition is probably not very helpful to some stakeholders. I covered anumber of perspectives such as “Testing is a way to make money” (if you’re a testing tools vendor or a testing outsourcing services provider), “Testing is a cost centre” (if you’re a CFO) and “Testing is dead” (if you’re a CxO type reading some of the headline IT magazines & websites). There was a good discussion after my talk, mainly focused on the cost centre perspective and how this has impacted people in their day-to-day work. I was pleased with how my talk went (especially given the short time I had to prepare) and received some good feedback, particularly on the concise nature of the slides and the confidence with which it was presented. My slide deck can be seen at What is Testing? It depends…

The last session before lunch saw Aleksandar Simic with “A Two-day Testing Story”. He did a fine job of breaking down a two-day period in his work into various different activities, some testing-related (e.g. pairing on test design) and some not (e.g. working with IT support on a networking issue). Aleksandar’s level of self-inspection was impressive, as was his naming of the various activities, learning opportunities and challenges along the way. His “testing diary” seems to be working well for him in identifying and naming his testing activities and this would make an interesting conference talk with some further development.

Lunch provided a good chance for us all to chat and unwind a little after the intensive morning spent talking testing.

First up after the lunch break was Karo Stoltzenburg with “I test, therefore I am”. She had adopted the idea of substitution in preparing her talk so looked to answer the question “Why do I test?” and see where that took her. Karo’s answer was “Because I like it” and then she explored why she liked it, identifying investigation, learning, exploring, use of the scientific method, collaborating, thinking in different contexts and diversity as aspects of testing that appealed to her. I liked Karo’s closing remarks in which she said “I test because it makes me happy, because it’s interesting, challenging and varied work”. We really need more positive messages like Karo’s being expressed in the testing community (and wider still), so I’d love to see this become a full conference talk one day. She did a good job of communicating her passion for testing and there were some interesting discussions in the group following her talk, with a degree of agreement about why testing is so engaging for some of us.

The sixth and final talk of the day came from James Thomas with “Testing All the Way Down, and Other Directions” He walked through an in-depth analysis of Elisabeth Hendrickson’s “Tested = Checked + Explored” from her book, Explore It! James decided to explore this definition of testing using techniques from that definition which wouldn’t classify his actions as testing. He described how he’d interacted with Elisabeth on some of his questions after exploring the idea in this way and finally presented his proposed alternative definition of testing as “the pursuit of actual or potential incongruity” (Note that James more fully describes this talk in his blog post, Testing All the Way Down, and Other Directions) The main focus of discussion after James’s talk was around his proposed definition of testing and I’ll be following the broader community’s response to his proposal with interest.

A few discussion points arose during the day for which we didn’t have time to go deep between talks, so we dedicated ten minutes to each of the following topics to close out the workshop content:

• Quality – what does it mean? (Weinberg definition, but are others more helpful?)
• Domain knowledge (can bias you, can empathy with the end user be a disadvantage? How do we best adjust strategy to mitigate for any lack of domain knowledge?)
• Evaluating success (how do we measure the success of spreading testing into development and other disciplines?)
• Is testing just “the stuff that testers do”? (probably not!)
• How do we make a difference? (blogging, workshops in our own workplaces, brown bag sessions, broader invitation list to peer conferences)

To wrap up, a short retrospective was held where we were all encouraged to note good things to continue, anything we’d like to stop doing, and suggestions for what we should start to do. There were some good ideas, briefly discussed by the group and I’d expect to see some of these ideas taken up by the CEWT organizers as part of their fourth incarnation.

The CEWT group standing outside number 10 Downing Street (or inside the Jagex office, maybe):

This was a really good day of deep-diving with a passionate group of testers, exactly what a peer conference should be all about. Thanks again to James for the invitation and thanks to all the participants for making me so welcome.

For reflections on the event from others, keep an eye on the CEWT blog at

http://cewtblog.blogspot.co.uk/2016/11/cewt-3-reflections.html

Living up to my handle: rocking and testing in Belfast

I was lucky enough to find myself in Northern Ireland recently, with a trip based around attending one of Status Quo‘s final ‘electric’ shows in Belfast. This was my first trip to Northern Ireland and it was a very enjoyable few days. Visiting the stunning North coast around the Giant’s Causeway was a highlight, as was exploring the varied districts of the city of Belfast itself, from the colourful (both literally and figuratively) walls of the Falls/Shanklin area to the stunning City Hall.

So, firstly, the “rocker” part of the trip. The big SSE Arena would be home to Status Quo for one night on Friday 30th October. The Emerald Isle has always been good territory for the band and this gig would be no exception. Luckily, our hotel – the Premier Inn Titanic Quarter – was literally next door to the venue and our room had a view over the loading area at the back, so a perfect spot to keep an eye on proceedings during the day in the lead up to the gig. An early queue formed (as usual) and it was good to catch up with friends there and also make some new ones (one very generous bloke in the queue kindly gave my wife a free ticket so she could join me in attending the gig!). With doors opening a little later than usual (due to the Quo crew arriving late because of travel problems from the mainland to Ireland), the queue was very long by the time we were allowed in and it was a sprint down to the barrier, securing a spot centre stage.

Support came from Uriah Heep and they got a great reception, having not played live in this part of the world for over thirty years. While their brand of heavy rock (somewhat like Deep Purple to my ears) is not really to my taste, they did a good job of getting the 7-8000 strong audience involved and provided a good warm-up during their hour-long set.

At just after 9pm, it was finally time for Quo to take the stage and it was another top performance, with Rick Parfitt’s replacement, Richie Malone, getting a particularly warm welcome in his home territory. There were no setlist surprises (it is Status Quo after all!) but, as always with a live gig, there are subtle differences from night to night and the encouragable crowd made this a very enjoyable show.

Now, where does the “tester” fit into the trip? Well, luckily for me, the fairly new Belfast Tester meetup group had announced a meetup during one of the nights I was in Belfast (and thankfully not the same night as the Status Quo gig), so I decided to go along and see what this new testing community looks like.

It was the fourth meetup of the Belfast Testers Meetup group and it took place on the evening of Thursday 29th October, in the boardroom of Shopkeep. I was warmly welcomed as a “one off” attendee of the meetup, which drew a crowd of about 25 (many of whom were first-timers as you’d expect at such a new meetup). It’s always good to see a new community of testers being built and this one should do well, with strong leaders and a burgeoning IT industry in the city.

Meetup co-organizer Neill Boyd kicked off the meetup, after gathering up the crowd enjoying the hospitality and impressive surroundings of the Shopkeep office (all very “startuppy”) and funneling them into the boardroom. It was cosy, with 20+ of us in the room but it made for a good space for easy conversation and Q&A with the presenters.

His introduction was handy for newcomers, announced that the TestBash conference is adding Belfast to its growing list of city destinations in 2017, and outlined the agenda of two talks to fill the evening.

First up was Allan Hunter (Senior Tester at PwC Emerging Technologies team) with “User Focused Testing”. This short talk discussed how user research is a key ingredient in powering insightful testing. He talked on his experiences of user research and how this can be applied to more traditional testing activities. His basic message was that we as testers are well placed to test the problem and not just the solution. He packed a lot of cool content into ten minutes then held his own through a lengthy period of questioning.

Secondly, we had Ursula Wlodarczyk (a tester working in the startup, SaltDNA) with “Tester v s Designer: Why tester can make a great (UX) designer”. Ursula’s was a much longer talk and tried to cover a huge amount of ground, enough for at least a couple of good talks I’d say. She talked about exploring a tester’s skillset in the context of the User Experience field and, similarly to Allan, she recognizes that testers can provide great insights into design (not just at the user interface either) and organizations should be allowing testers the opportunities to engage in design meetings and provide their valuable insights and suggestions before the software is built. A very long talk for a meetup (it would actually make a decent conference talk), but some nice ideas and good resources.

    

So, thanks to Belfast and thanks to the two very different communities – Status Quo and testing – that continue to give me the chance to rock and test all over the world.