Pre-launch announcements for my new projects

After six weeks or so of resetting following my unplanned exit from Quest, I’m getting close to publicly announcing more details on a couple of new projects.

One of these has been in the making for about a year, while the other has arisen as a direct result of leaving full-time employment.

I’ve always been drawn to the idea of writing a book and I will finally realize this idea with the release of a testing-related e-book very soon. It’s been a highly collaborative effort with input from many members of the testing community. Having more free time since finishing up at Quest has given me the opportunity to wrap up what I think is worthy of publishing as a first edition. I will return all proceeds from sales of this book to the testing community. Look out for more details of the book via this blog and my social media presences in the coming weeks!

My other project is a new boutique software testing consultancy business. The intention is to offer something quite different in the consulting space, utilizing my skills and experience from the last twenty years to help organizations to improve their testing practices. This consultancy won’t suit everyone but I hope that my niche offering will both help those who see the value in the way I think about testing and also give me the chance to share my knowledge and experience in a meaningful way outside of full-time corporate employment. I expect to launch this business before the end of the year, but feel free to express interest in securing my services now if you believe that my thinking around software testing could be of value in your organization. Note that I will not be making myself available full-time (as I’m deliberately carving out time for volunteer work and to focus on my wellbeing), so now is a good time to secure some of my limited future availability before the formal launch of the consultancy. Again, keep an eye on this blog and my socials for more details of the testing consultancy project.

ER of presenting at DDD Melbourne By Night meetup (10th September 2020)

In response to a tweet looking for speakers for an online meetup organized by DDD Melbourne By Night, I submitted an idea – “Testing is not dead!” – and it was accepted.

I had a few weeks to prepare for this short (ten-minute) talk and went through my usual process of sketching out the content in a mindmap first (using the free version of XMind), then putting together a short slide deck (in PowerPoint) to cover that content.

I find it harder to nail down my content for short talks like this than for a typical longer conference track talk. The restricted time forces focus and I landed on just a few key points: looking at the claims of “testing is dead”, defining what “testing” means to me (and contrasting with “checking”), where automation fits in, and wrapping up with a few tips for non-specialist testers (as this is primarily a meetup with a developer audience).

I did two practice runs of the talk over the same conference call technology that the meetup would be using (Zoom), even though my willing audience of one (my wife) was only in the next room at home! I find practice runs to be an essential part of my preparation and I was pleased to find both runs coming in very close to the ten-minute timebox.

The September DDD by Night meetup took place on the evening of 10th September and featured nine lightning talks with some preamble and also time for questions between each talk. I was third up on the bill and managed to whizz through my talk in a few seconds under ten minutes! The content seemed to be well received and some of my ideas were clearly new to this audience, so I was pleased to have the opportunity to spread my opinion about testing to a different part of the Melbourne tech community.

Lee kicking off his talk

It was also great to see Vanessa Morgan as a first-time presenter during this meetup and her talk was a very polished performance.

Thanks to the DDD Melbourne crew for putting on meetup events during these interesting times and, as a newcomer, the friendly community spirit in this group was obvious.

You can watch my talk on YouTube.

Decompression – phase 2

It’s already four weeks since my exit from Quest after some 21 years with the company and it’s amazing how quickly my “reset” is happening.

My calendar continues to be largely empty, apart from a number of video calls with friends both locally and overseas each week. My days are passing quickly and are filled with exercise (walking and cycling), well-being practices (meditation and yoga), reading, home/garden maintenance and relaxing. The welcome start of the Tour de France and the excellent daily highlights on SBS represent my only “TV time” during the day.

I’ve also been keeping abreast of what’s going on in the testing community and have a couple of small projects on the go – writing a guest blog post (details coming soon) and preparing a talk (“Testing is not Dead”) for the DDD Nights meetup on 10th September.

When my redundancy package payout appeared in the bank at the end of August, my ties to Quest finally came to an end. Reflecting on such an incredibly long time working in the same company, my main emotion is one of gratitude. Quest provided me with my first employment opportunity in Australia after migrating from the UK and, over the many years that followed, gave me manifold opportunities to grow professionally, contribute to the broader testing community, travel extensively, and build great relationships. I miss the banter with my good friends & colleagues and also my close collaborations with the wonderful folks over in our office in China, where I look back with great pride on the development programmes I helped to put in place and run over the years.

I continue to work on the testing-related project I mentioned in my previous blog post and expect to make an announcement on that in the next month or so, so stay tuned! I’ve also been taking part in an online business course and this may also lead to some news in terms of my next steps in the coming months.

Decompression – phase 1

It’s now been a couple of weeks since my 21-year career at Quest came to an abrupt end. My LinkedIn post about exiting Quest has by far the most engagement of all my contributions on that platform, with over 12,000 views and more than 100 reactions as I write. The irony of this fact is not lost on me.

I’ll blog specifically about my Quest exit later, but I’ve spent the time since then in “reset” mode, taking a step back from work and testing-related things in general.

An empty calendar is a real delight after so many years on the hamster wheel of frequent meetings. With my role having covered essentially every timezone, very early and very late meetings were the norm (although I’d become better at not accepting meetings starting before 7am or finishing after 11pm). Having long periods during the day without interruption from meetings has given me back opportunities for simple pleasures that have suffered greatly over the last decade (or more in some cases).

My wife and I have been practising meditation and yoga for several years but my daily practice has often failed to take precedence over the demands of work. It’s been great to put these important factors for my wellbeing back where they belong – as priorities.

Bike riding was another important and enjoyable pastime that I’ve neglected so it’s great to make a daily ride part of my routine (weather-permitting!). I’d forgotten how much I enjoy the feeling of freedom on two wheels.

I’ve also been reading a lot, some tech-related but plenty not (e.g. Status Quo gig history research, animal rights, vegan advocacy, some fiction).

It would have been nice to take the chance to travel for a while, but of course current COVID-19 restrictions make that impossible in this part of the world. But we’re at least thinking of where we might like to explore within Australia when our freedoms are returned to us.

I haven’t entirely disconnected from the testing world, though, and have invested some time into my LinkedIn profile (still work in progress) to ensure it more accurately reflects my experience and contributions in readiness for my next steps. I’ve also made some improvements to the configuration of this blog, so there is now an archive by month and I’ve categorized every post so hopefully it’s easier to find the stuff you’re interested in (and thanks to Paulo Lai for his feedback inspiring me to make these long overdue changes).

I’ve been working on a personal testing-related project for most of 2020 and have more time and energy to dedicate to it now, so it’s coming along nicely. I hope to release details of this project publicly very soon!

Everyone’s talking about testing!

I can’t remember a time in my life when “testing” has been such a hot topic of coverage in the media. It feels like every news item leads with some mention of the number of tests conducted to detect the COVID-19 coronavirus, whether it be it locally or further afield. This level of coverage of the topic of testing even exceeds that during Y2K (according to my memory at least), albeit in a very different context.

It was interesting to see the reaction when the President of the United States said that the US case numbers are high because so many tests have been conducted – and that a reduction in testing might be in order. This led Ben Simo to tweet on this idea in the context of software testing:

Stop testing your software! Bugs are the result of testing. Bugs that don’t kill users dead instantly aren’t really bugs. No testing is the key to zero defect software! If you don’t see it, it doesn’t exist. If you don’t count it, it doesn’t matter. No testing!

I felt similarly when I read some of the coverage of the worldwide testing efforts during the pandemic. “Testing” for COVID-19 is revealing valuable information and informing public health responses to the differing situations in which we find ourselves in different parts of the world right now. (In this context, “testing” is really “checking” as it results in an algorithmically-determinable “pass” or “fail” result.)

When we test software, we reveal information about it and some of that information might not be to the liking of some of our stakeholders. A subset of that information will be made up of the bugs we find. In testing “more”, we will likely unearth more bugs as we explore for different kinds of problems, while “more” in the COVID-19 sense means performing the same test but on more people (or more frequently on the same people).

We should remain mindful of when we’ve done “enough” testing. If we are genuinely not discovering any new valuable information, then we might decide to stop testing and move onto something else. Our findings from any test, though, represent our experience only at a point in time – a code change tomorrow could cause problems we didn’t see in our testing today and an unlucky person could acquire COVID-19 the day after a test giving them the all clear.

There is a balance to be struck in terms of what constitutes “enough” testing, be that in the context of COVID-19 or software. There comes a point where the cost of discovering new information from testing outweighs the value of that information. We could choose not to test at all, but this is risky as we then have no information to help us understand changing risks. We could try to test everyone every day for COVID-19, but this would be hugely expensive and completely overwhelm our testing capacity – and would be overkill given what we already understand about its risks.

Many of us are testing products in which bugs are potentially irritating for our users, but not life and death issues if they go undetected before release. The context is clearly very different in the case of detecting those infected by COVID-19.

As levels of COVID-19 testing coverage have increased, the risk of acquiring the virus has become better understood. By understanding the risks, different mitigation strategies have been employed such as (so-called) social distancing, progressively more stringent “lockdowns”, and mandatory mask wearing. These strategies are influenced by risk analysis derived from the results of the testing effort. This is exactly what we do in software testing too, testing provides us with information about risks and threats to value.

It’s also interesting to observe how decisions are being made by bearing in mind a broader context, not just taking into account the testing results in a particular area or country. Data from all across the world is being collated and research studies are being published & referenced to build up the bigger picture. Even anecdotes are proving to be useful inputs. This is the situation we find ourselves in as software testers too, the software in front of us is a small part of the picture and it’s one of the key tenets of context-driven testing that we are deliberate in our efforts to explore the context and not just look at the software in isolation. In this sense, anecdotes and stories – as perhaps less formal sources of information – are incredibly valuable in helping to more fully understand the context.

Test reporting continues to be a topic of great debate in our industry, with some preferring lightweight visual styles of report and others producing lengthy and wordy documents. The reporting of COVID-19 case numbers continues to be frequent and newsworthy, as people look to to form a picture of the situation in their locale. Some of this media reporting is very lightweight in the form of just new daily case and fatality numbers, while some is much deeper and allows the consumer to slice and dice worldwide data. Charts seem to be the reporting style of choice, sometimes with misleading axes that either exaggerate or down play the extent of the problem depending on the slant of the publisher. Different people react to the same virus infection reports in quite different ways, based on their own judgement, biases and influences. We see the same issue with software test reporting, especially when such reporting is purely based around quantitative measures (such as test case counts, pass/fail ratios, etc.) The use of storytelling as a means of reporting is nothing new in the media and I’d argue we would be well served in software testing to tell a story about our testing when we’re asked to report on what we did (see Michael Bolton’s blog for an example of how to tell a three-part testing story – a story about the product and its status, a story about how the testing was done, and a story about the quality of the testing work).

While I don’t normally focus on counting tests and their results, I’ll be happy to see more COVID-19 tests taking place and fewer new daily positive results in both my area of Australia and the world more generally. Stay safe.

(Many thanks to Paul Seaman for his review of this post, his sage feedback has made for a much better post than it otherwise would have been.)

Going meta: a blog post about writing a blog post

My recent experiences of authoring blog posts in WordPress have been less enjoyable than usual thanks to the use of their latest “block editor”, leading me to ask on Twitter:

WordPress seems to update their post editor very frequently so I just about learn the quirks of one when it is superseded by another.

This post will serve as my (long) answer to WordPress’s reply. I’m going to spend the next 45 minutes running an exploratory testing session, creating a blog post and noting issues as I come across them while using the block editor.

Session from Tuesday 21st July 2020 , on Windows 10 laptop (using keyboard and mouse controls only) using Chrome browser

4:10pm I’m starting my session by writing a very basic block of unformatted text. I note that when I move my mouse, a small toolbar appears which covers the end of the previous block (this could be an issue when in the flow of writing). The toolbar disappears as soon as I type and reappears on every mouse movement. The content of the toolbar seems very limited, maybe to just the most used formatting features (most used by the whole WordPress community or most used by me)? At least each icon in the toolbar has a tooltip. There’s a very odd control that only appears when hovering over the leftmost icon (to change block type or style) which appears to facilitate moving the whole block up or down in the post. I wonder why the toolbar is so narrow, snce there is plenty of room to add more icons to allow easier discovery of available options here. I’ve been distracted by the toolbar but now resume my mission to complete a basic paragraph of text.

OK, so hitting Enter gives me a new paragraph block, that makes sense. Let’s get more creative now, how about changing the colour of some text? The toolbar doesn’t appear to have a colour picker, oh, it’s tucked away under “More rich text controls”. I’ve typed some text, highlighted it and then selected a custom colour. That worked OK once I found the colour picker. The colour picker control seems to stay in the toolbar after using it – or does it? I’ll try it again but lo, it’s back under the hidden controls again. There’s probably a deliberate choice of behaviour here, but I’ll choose not to investigate it right now.

I’m trying to select some text across blocks using Shift+Arrow keys but that doesn’t work as I’d expect, being inconsistent with other text selection using this keyboard combination in other text processing applications. (Ctrl+Shift_Arrow keys suffers the same fate.) Shift+Page Up/Down only select within the current block, again not what I’d expect.

4:30pm After adding this new block (just by pressing Enter from the previous one), I’m intrigued by the array of block types to choose from when pressing the “+” button which appears in seemingly different spots below here (and I just spotted another “+” icon on the very top toolbar of the page and it looks like it does the same thing). There are many block types, so many that a search feature is provided (a testing rabbit hole I’ll choose not to go down at the moment). Some of the block types have names which indicate they require payment to use and the available block types are categorized (e.g. Text, Media, etc.) I decide to try a few of the different block types.

Adding a “quote” block now, which offers two areas, one for the quote and one for the citation. It appears that the citation cannot be removed and so more space is left below the quote text than I’d like (but maybe it doesn’t render the empty space when published?).

A test quote without citation

Moving on to adding a list and this works as I’d expected, offering a choice between bulleted and numbered with indentation (maybe there’s a limit on nesting here, but not investigated).

  • First item of my list
  • Next item of my list
    • Indented!

Even though I’ve been using this editor for my last few blog posts, I still tend to forget that auto-save is no longer a thing and I just happened to notice the “Save Draft” in the top right corner of the page, so let’s save.

In reality, my blog posts are mainly paragraphs of text with an occasional quote and image so exploring more block types doesn’t seem worth the effort. But looking at images feels like a path worth following.

Copying an image on the clipboard seems to work OK, though immediately puts cursor focus into the caption so I started typing my next bunch of paragraph text incorrectly as the image caption.

Options in the toolbar for the image make sense and I tried adding an image from a file with similar results (deleted from the post before publishing). Adding images into a post is straightforward and it’s good to see copying in directly from the clipboard working well as there have been issues with doing so in previous incarnations of the editor.

4:45pm Returning to simply writing text, I often add hyperlinks from my posts so let’s try that next. Ctrl+K is my usual “go to” for hyperlinks (from good ol’ Word) and it pops up a small edit window to add the URL and Enter adds it in: http://www.google.com Selecting some text and using the same shortcut does the same thing, allowing the text and the URL to be different. The hyperlinking experience is fine (and I note after adding the two hyperlinks here that there’s a “Link” icon in the toolbar also).

I remember to save my draft. As I resume typing, the toolbar catches my eye again and I check out “More options” under the ellipsis icon. I notice there are two very similar options, “Copy” and “Duplicate”, so I’ll try those. Selecting “Copy” changes the option to “Copied!” and pasting into Notepad shows the text of this block with some markup. I note that “Copied!” has now changed back to “Copy”. Selecting “Duplicate” immediately copies the content of this block right underneath (deleted for brevity), I’m not sure what the use case would be for doing that over and above the existing standard copy functionality. OK, I’ve just realised that I’ve been distracted by the toolbar yet again.

I just added this block via a “hidden” control, I’m not sure why products persist with undiscoverable features like this. Hovering just below an existing block halfway across the block reveals the “+” icon to add a block (though it often seems to get ‘blocked’ by, you’ve guessed it, that toolbar again).

My time is just about up. As I review my short session to create this blog post, I think it’s the appearing/disappearing toolbar that frustrates me the most during authoring of posts. I almost never use it (e.g. I always use keyboard shortcuts to bold and italicize text, and add hyperlinks) and, when I do, the option I’m after is usually tucked away.

Thanks to WordPress for responding to my tweet (and providing what is still generally a great free platforms for blogging!) and for giving me a good excuse to test, learn and document a session!

ER of attending my first virtual testing conference, Tribal Qonf (27 & 28 June 2020)

I spotted some promotion on Twitter for a new testing conference in India, Tribal Qonf, and the virtual nature of it (thanks to COVID-19) plus the impressive speaker line-up (including James Bach and Michael Bolton) spurred my interest. Looking into it further, the pricing was incredibly low so I decided to register for it (for around AU$30 at the time I registered).

Although the weekend scheduling of the conference and Indian timezone wasn’t ideal, the conference promised to provide all content via recordings so I didn’t tune into any of the presentations “live”, waiting instead the ten days or so for recordings to be made available. I then watched most of the presentations from the two-day event over a period of a few days.

The first presentation I watched was the opening talk from day 1 by James Bach, titled “Weaving Testing: Thread by Thread” This was a fascinating talk and it was great to see such a detailed analysis of what actually happens during good testing by skilled practitioners, especially compared to the mythology we’ve generally been conditioned with about what makes for ‘proper’ testing.

Next up, I opted for Pradeep Soundararajan‘s talk on “The Business Value of Testing”. I’ve unfortunately never managed to catch Pradeep presenting in person, but this virtual presentation displayed the passion I expected from him. It was also engaging and refreshingly honest about the challenges we face in terms of recognizing how different stakeholders view the “value” of what we provide as testers.

My next choice was “Adopting a simplified Risk-Based Testing Approach” by Nishi Grover Garg, in which she outlined the basics of the approach, very much in the style of practitioners like Rex Black. The approach was presented very clearly here and I liked the way Nishi contextualized the risk-based testing approach to her startup environment.

A nicely-crafted story came next thanks to Ajay Balamurugadas and his talk “Lessons from 14 Years of Software Testing Career”. He detailed his learnings from each of his testing jobs and offered practical suggestions for areas to focus on at different levels of experience in testing. This presentation reminded me very much of my “A Day In The Life Of A Test Architect” talk which I gave at STARWest in 2016 and again at CAST in 2017.

Rounding out the talks for day 1, I somewhat hesitantly tuned into the ‘expert panel’ on “Testing after 2020”. I’ve become a little jaded about panel sessions but I really enjoyed this one featuring Aprajita Mathur, Ashok Thiruvengadam, Rahul Verma and Pradeep Soundararajan. The panelists responses to the various questions were refreshingly down to earth and practical. I was particularly pleased to see the considered, reasonable and sensible discussions around AI/ML in testing, providing welcome relief from the usual Kool Aid drinkers around these topics in the industry at the moment. A shout out to Lalit Bhamare too for his skillful moderation of this panel session which was a significant factor in its success for me.

I kicked off my “day 2” viewing with the first talk from that day, viz. , Ashock Thiruvengadam with “Be in a Flow. Test Brilliantly” This was something a little different in terms of topic for a testing conference (which is always good to see), focusing on introducing the idea of “flow”. I was reminded of the importance of uninterrupted sessions when performing exploratory testing while listening to this talk.

Next, I opted for Mike Talks with “The Hard Lessons Learned in Test Automation”, in which he shared some interesting stories of lessons learned resulting from his chats with testers over coffee in his home city of Wellington (New Zealand). It was unsurprising to me that his chats resulted in a few very common themes, all of which were familiar territory from my various conversations about automation with testers from all over the world over the last twenty-odd years. It seems we have a long way to go in terms of learning these hard lessons, despite them being covered ad nauseam in blogs, articles, books and conference talks.

My next choice was “A Quick Recipe for Test Strategy” from Brijesh Deb and I immediately liked his take on the topic. He defined a test strategy simply as a “set of ideas that guide test design” and made it clear that we shouldn’t conflate this with a hefty “one size fits all” document of some sort. I also liked his focus on driving test strategy by asking questions, with not just a shout out to James Bach‘s Heuristic Test Strategy Model but also an example of using it in practice.

The penultimate talk I watched was “Who Are Your Stakeholders?” with Anna Royzman. We often hear the term “stakeholders” used in testing (and software development more generally) but rarely do we seem to agree on what this term means in the context of our projects. Anna gave a good introduction on how to identify different types of stakeholders and what kinds of information these different stakeholders might be looking for.

I concluded my binge watching of the conference talks with the closing session from the event, in the shape of a “Fireside Chat with Michael Bolton” with questions coming from Ajay Balamurugadas. I loved Michael’s answer to Ajay’s question “what has changed in testing from 1994 to 2020?”, “not enough!” This was a fun fifty minute session and a perfect way to wrap up the conference.

Obviously, “attending” a virtual conference is a completely different experience to an in-person event. I chose not to watch all of the presentation recordings but did watch most of them and the quality was high. I didn’t watch the recordings back-to-back either, rather spreading out my viewing across a few days alongside my usual work commitments. I also didn’t contribute to the conference’s Slack channels as the event had been over for two weeks or so by the time I got to the recordings.

I personally missed the in-person aspects that make traditional conferences so valuable, but it might not be the case that we have to choose one over the other as we move forward. I wonder if we’re entering a new era for conferences, driven by changes forced upon us by COVID-19. There are enormous accessibility benefits of the virtual model, thanks to lower pricing and the removal of the need to travel and spend time away from home & family. Such virtual events also open up opportunities for new voices who might be unable or unwilling to travel to a “normal” event, or are too uncomfortable to address a physical audience.

The selection of topics on offer during this event was good and the talks were of a high standard. It appeared to be well organized too, so thanks to Lalit and the Test Tribe crew for putting on a worthwhile testing event during these difficult times! I enjoyed the experience of this virtual conference and I am now considering attending other virtual testing conferences through 2020 before – maybe! – more normal service resumes in 2021…

My experience of working from home full-time during the pandemic

Thanks to Twitter, I spotted that a new testing-related ebook had been published recently, titled Software People Work From Home, in which a number of testers from around the world describe their personal experiences of working from home thanks to coronavirus-induced restrictions.

I really enjoyed reading this (free) ebook and felt inspired to share some of my own experiences based on more than two months of working full-time at home for Quest.

Firstly, some context around my “normal” work situation. Although I am based in our Melbourne city office, I’ve been working from home for three to four days a week for the last five years or so. Part of the reason for this mode of working is due to the long commute from my home to the Melbourne office (around two hours door-to-door, each way) and another factor is the many early morning/late night meetings I’m involved with thanks to my collaboration with Engineering teams across basically every timezone!

This hybrid model has worked well for me and for Quest over the last few years. The reduced requirement to be within “normal” commuting distance of Melbourne means I can live in a beautiful location, right on the beach with peace and fresh air – this has made a huge difference to my lifestyle and well-being. We always make time during the day for a long walk (typically around 5km) and having this as part of my routine continues to be really important to me.

I intended to resume this model of working following my return from some international travel back in mid-March. (I’ve blogged about the experience of travelling during the pandemic separately here and here.) Of course, on returning from the UK towards the end of March, our office had already been closed until further notice thanks to COVID-19 – and so began my immediate transition to working from home full-time.

It’s been an interesting three months in this new mode of working. In many ways, I am very fortunate. Firstly, I’ve kept a full-time job on the same salary as pre-COVID so there are no additional financial pressures resulting from this change. I’ve also had plenty of practice at working from home over the last several years, so the adjustment to full-time at home hasn’t been as significant as for many people. I’m lucky enough to have a dedicated room in our (albeit very small) home to work from (and the incredible water view from there never gets old!) and we are a couple with no kids so it’s pretty straightforward to maintain a quiet environment in which to concentrate on work.

Even adding the extra day or two at home every week compared to my usual routine has revealed some additional benefits of the arrangement. I’m settling into a great circadian rhythm (save for a few early morning meetings) and not commuting has freed up some more time to enjoy relaxing at home. We’re cooking together more often too.

While it’s generally a positive for me to work from home all the time, there are challenges too, the most significant of which for me is avoiding overworking. I spotted a great quote on Twitter recently:

Stop calling it ‘working from home’ and start calling it ‘living at work’

Heather De-Quincey

My boundaries between “work” and “not work” are – and never have been – very strict. I have access to all of the systems folks within Quest might use to contact me pretty much all of the time. My role operates across our entire business unit, which has people in timezones spanning the whole world.

Shortly after lockdowns became part of almost everyone’s life experience, we decided to convert what would have been a large in-person meeting in California into a virtual event. While it was a great success, organizing and running a 70+ person meeting across so many timezones was a huge effort by many people – I worked sixteen-hour days for three days straight during the event, resulting in severe over-tiredness (and unwelcome grumpiness on the home front).

There is always someone looking for something from me, so it can be hard to ignore those requests even when they’re out of what most would think of as “business hours”. The early morning and late night meetings necessitated particularly by the need to interact with folks in the US are draining, but I’ve learned to speak up more and refuse meetings before 7am or after 11pm so as to allow for a reasonably consistent sleep pattern.

There are some things I am missing as a result of not just working from home all the time, but also the general world situation thanks to the pandemic. I was looking forward to attending the CAST conference in Austin in August, but that of course has been cancelled. This is also the first time in recent memory that I haven’t had an overseas trip (or two or three!) in planning, either for work or leisure (or sometimes a combination of both). It feels strange not having these adventures to look forward to anytime soon.

Closer to home, my day or two up in Melbourne affords me a number of opportunities and benefits that I’d taken for granted. I miss my weekly coffee catch-ups with Paul Seaman, although we maintain close virtual contact. I make very good use of the Melbourne Library Service and usually have an interesting book or two on the go, but there’s been no opportunity to borrow books recently. I also miss the chance to have lunch or coffee with ex-colleagues, something I do almost every time I travel into the city. And, of course, the kitchen banter with my Quest colleagues is sadly lacking – and Teams meetings are just not the same!

Overall, I’m enjoying the experience of working from home full-time and the downsides certainly don’t outweigh the positives of both my work output and lifestyle. With Quest keeping its offices closed until at least September 2020, I’ll get to enjoy it for a while longer.

The Melbourne testing market and spotting “red flags”

One of the more popular job listing sites in Australia is Seek. A search on this site for Melbourne-based jobs under the Information & Communication Technology -> Testing & Quality Assurance classification yielded 70 results as I began writing this post on 29th May 2020.

Looking at these jobs, I’ve broadly categorized them as follows:

  • Automation (32)
    • Automation/SDET
    • Test Automation Engineer (x2)
    • Automation Test Consultant (x2)
    • Senior QA Automation Engineer
    • Test Automation Specialist (x2)
    • Automation Test Analyst (x5)
    • Automation Test Analyst – Tosca
    • Senior Automation Test Analyst
    • QA Engineer (Automation)
    • Automation Tester (x4)
    • Senior Test Automation Engineer/Lead (x2)
    • Java Development Engineer in Test (DevOps)
    • Lead/Sr.Automation Tester
    • Senior Test Automation Specialist
    • Automation Test Engineer
    • Automation Test Engineer (Embedded Software)
    • Applications Testing Automation Engineers (x4)
    • Technical Test Lead – Automation Lead Tosca
  • Management (7)
    • Test Manager
    • Performance Test Manager (x2)
    • Defect Manager (x4)
  • Specialists (14)
    • Performance Tester (x4)
    • Penetration Testing
    • Performance Test Consultant
    • Infrastructure and Security Test Analyst
    • Senior Test Analyst – Infrastructure testing
    • Performance Test Analyst (x2)
    • Performance Engineer (x2)
    • Jmeter Tester
    • Network Test Engineer (hardware-related)
  • “Manual” / Other (17)
    • Test Analyst
    • Test Analyst – Business Intelligence
    • Senior Test Analyst (x2)
    • UAT Tester
    • Mobile QA Engineer
    • Quality Assurance Engineer
    • QA Engineer
    • Graduate Test Analyst
    • Automation Specialists and Defect Managers
    • Junior Quality Assurance Tester Traineeship
    • Senior Software Test / V&V Engineers (Defence)
    • Validation & Verification Lead
    • Integration Testers (x4)

These ads are not representative of unique jobs, as exactly the same ad is sometimes posted at different times and/or the same job is typically posted by a number of different recruiters (especially for government roles).

The breakdown in terms of the number of opportunities didn’t surprise me. The focus at the moment seems to be around automation driven by CI/CD investments, DevOps/agile transformations – and, of course, the overestimation of what automation can and can’t do. Similarly, performance and security-related testing are topics de jour as represented by a swathe of ads in these areas. Test management doesn’t seem like a good place to be, with very few roles being advertised in recent times and this change has been heavily driven by agile adoption in my experience.

I generally take more interest in the ads focused on human testing to see what companies are looking for in this area. Most of the more “traditional” human testing roles (e.g. in government departments) also now mandate some degree of proficiency in tools commonly associated with “automated testing”. It’s pleasing to see requests for ISTQB certification becoming much less common and I’ll occasionally even spot a reference to “exploratory testing”.

But there are often “red flags” in these ads and I proffer a few examples from this most recent search of the “opportunities” on offer.

First up, a “Senior Test Analyst” role “to take accountability for the testing capabilities within the Microservices team.” The “Technical experience” requirements are listed first and include “Linux/Unix Shell Scripting, Java Programming, Clean Coding, Git, Jenkins2, Gradle, Docker, REST APIs, SQL, Unit Testing (Junit), Component Testing, BDD, Integration Testing” and then finally the “Testing Practices” requirements are presented, viz. “Exploratory Testing, Mind Mapping, Requirements Analysis, Peer Collaboration, Continuous Integration, Continuous Deployment, Stubbing and Mocking”.

There are a few red flags here for me. If this were a true microservice team, then it would have a clear sense of ownership of its microservice and a whole team approach to the testing and quality of that service. I’d be looking for clarification of what “accountability for the testing capabilities” really means in the context of this team. Another issue is the lack of clarity about whether this is a human testing role or a development (automation code) role, or a hybrid, or something else. The fact that the technical (mainly development) skills requirements are listed before the testing ones would immediately lead me to believe that more value is placed on automation than on deep human testing here. While it’s good to see exploratory testing explicitly listed (albeit as a “testing practice”), the other requirements listed around testing are much less convincing as to whether this organization would truly value the contribution of an excellent exploratory tester.

Next up, a “Quality Assurance Engineer” role where the successful applicant (who becomes just a “Quality Engineer” as the ad goes on) will “play a critical role in transforming quality practices… and work to develop a world-class test automation solution. [They will] work as part of a cross functional squad, acting as the QA expert, across new features, enhancements and bug fixes. [They’ll use their] testing experience to advise on automation opportunities and review requirements to ensure a fit for purpose solution in (sic) delivered.”

In describing what the day-to-day work entails for this role, there are some positive signs: “You’ll be passionate about testing and use your experience to identify critical defects. You’ll be naturally curious and explore the latest tools and techniques to continuously improve testing” But there are also some less positive signs: “You’ll work as a fully engaged member of cross functional squad including Developers, UX/UI Designers and Product Managers to ensure the quality of products. You’ll create, maintain and execute tests (manual or automated) to verify adherence to requirements.” Again, it’s good to see exploratory testing getting a mention, albeit in what comes across as a confused way (especially in light of the “verify adherence to requirements” elsewhere), “You’ll perform risk based exploratory testing.”

In terms of skills, they’re expecting “expertise in developing functional tests that cover 100% of requirements and execute manually or automated (with focus on least maintenance and faster script development time)” and possession of “strong skills on efficient test design”.

There are obviously a few red flags in this ad. It sounds like this organization has latched into the so-called “Spotify Model” since it mentions “squads” a number of times, even though this model was actually not successfully adopted within Spotify. It talks about “ensuring” quality and verifying “adherence to requirements”, while at the same time asking for exploratory testing skills (of the risk-based variety, of course). Covering “100% of requirements” completes a picture of an organization where verification, preferably by machine, is valued much more than human testing.

My final example is for a “QA Engineer” role which asks for background in “both manual and automated testing” The red flags come real early in this one: “Can you imagine how it would feel to be responsible for ensuring that our key products work well in all conditions? Are you interested in working in a Global Centre of Excellence…?” I’ll choose not to imagine how it would feel to be given an impossible mission.

This lucky candidate “will be responsible for both manual and automated testing as we move towards complete automation.” At least there is no doubt left here about the value of deep human testing as this organization seeks to “automate everything.”

I like the requirement to be “Excellent at finding the problems others miss”, but much less keen to see this followed by “Able to document test cases based on solution requirements provided. Understanding of Developing in an Agile environment”. Advanced-level “ISTBQ” is then included in their list of “desirable” skills.

I don’t understand why this organization believes this ad would attract an excellent human tester who could leverage exploratory testing to genuinely find “the problems others miss”. They describe themselves as Agile but want the successful candidate to write test cases, all the time trying to get to a point where “manual” testing is no longer required. Based on the ad alone, either this is an organization with a confused outlook on testing or they’re really looking for someone to take on an impossible mission while devaluing their actual testing skills – either way, this doesn’t sound like a great way for a decent tester to spend their working life (but I acknowledge the ad could just be very poorly-written and misrepresents the core beliefs of the organization when it comes to testing).

While it’s interesting to review occasionally what organizations are looking for when they’re advertising for testing-related roles, it’s also pretty depressing to see just how little value seems to now be placed on deep human testing skill. The “automate everything” bandwagon seems to roll on and is taking over this market, while opportunities for those genuinely skilled in testing as a craft seem to become fewer and fewer, at least in the Melbourne market represented by published job ads.

As the economic impact of COVID-19 takes its toll, a large number of folks are hitting the IT market at the same time. In just the last couple of weeks, large IT staff reductions have been reported in Melbourne by big tech names such as MYOB and Culture Amp – and more seem likely in the coming months. If there’s a bright side to this situation, it’s that there are no doubt lots of good people coming back into the market so it’s probably a great chance to pick up testing talent for those organisations able to do so. If you’re representing such an organisation and really want to hire skilled human testers, please take the time to construct better tester job ads! Making it sound like you understand the value of testing is likely to attract those really good testers who might have come back into the market thanks to COVID-19.

Remember that a good tester should be a strong critical thinker and will be testing your ad, so catch your own “red flags” before they do!

ER of international travel during the coronavirus pandemic (part 2)

This is the second part of my travelogue/ER about travelling between the UK and Australia during the coronavirus pandemic (the first part can be found here). I’m writing this post on day 5 of our enforced 14-day home quarantine period following our return to Australia.

This second week began on Monday 23rd March and you might recall that we were now based in Aberystwyth on the West coast of Wales. The day started with news from Cathay Pacific about our new return flight option, that being a departure from London on 18th April and only getting us back as far as Sydney as their routes to Melbourne were stopping completely. We suddenly felt like mid-April was a long way away given how fast things had changed in just the previous seven days, so we decided to look for alternatives to get us home earlier. (As it turned out, Hong Kong would soon announce the banning of transit passengers through its airport so these new return flights with Cathay would never happen anyway.)

Looking for flights back to Australia, we spotted some Emirates options via Dubai and booked new flights departing London on 1st April, costing almost AU$3000 for the two of us. This process wasn’t straightforward as these newly-announced flights were selling so fast that the website didn’t respond well to the load and we missed out on flights for several other earlier dates as the transactions failed part-way through. At least 1st April didn’t feel so far away and we felt comfortable waiting it out for a week-or-so in Aber before we could return home. With the flights booked, I refocused on my Quest work for a while before we made the most of the lovely sunny day with a walk along the Prom and South Beach. We bumped into my Mathematics PhD supervisor from all those years ago, Alun Morris, and his wife Mary while walking and it was great to see them. They stopped for a chat (at a distance, of course!) and it was our first interaction with someone we knew since we’d left Australia. Alun looked in great health (it’s that Aber sea air!) and seeing him again was a fillip to our morale. Lunch back at the apartment gave way to more work in the afternoon.

Work was soon interrupted, however, by an announcement from Emirates that they were suspending operations from 25th March (and, as it turned out shortly afterwards, Dubai would be prohibiting transit passengers), rendering our newly booked flights useless.

We were really disappointed and again looked for alternatives, this time coming across flights with Etihad from Manchester via Abu Dhabi to Melbourne leaving the next day. We booked these flights, accruing another AU$5000+ on our credit card, and started to make our plans for leaving Aber (including informing our host that we would now be leaving earlier than expected).

Now thoroughly distracted, we headed out into the late afternoon sunshine and took what we thought would be a last chance to climb Constitution Hill. The stunning views over this pretty town and coastline never fail to impress and we soaked them in, before again watching the starlings doing their thing at sunset over the Pier. Back at the apartment, we cooked up a big meal to use up at least some of the nice organic veggies we’d bought at the Farmers Market. It was shortly after devouring this feed that the news came through that Abu Dhabi (as part of the United Arab Emirates) was also banning transit passengers with almost immediate effect, so our latest flight booking was again rendered useless. Our host came up to see us about our early departure and, luckily for us, she was very understanding about the fact that we would yet again have to change our plans and stay on. We were thoroughly exhausted by the end of the day after all the ups and downs.

View over Aber from Constitution Hill

We woke early on Tuesday morning to read the news that Australia was enforcing a complete travel ban, even on its own citizens (this was an escalation of the previous restrictions on entry) so we resigned ourselves to the fact that we were in for the long haul with our stay in Aber. We considered ourselves fortunate to be in stable accommodation in a familiar place and my ability to continue working meant we were under no financial pressure. Many other stranded Aussies were in far worse situations across the world.

As resignation set in, we tried to reset mentally and I buried myself into work again. It was another sunny and very mild day so we enjoyed a nice walk back out to Tan-y-Bwlch beach (which was completely deserted) before returning for lunch in our apartment. Back into work in the afternoon, a strange Twitter DM arrived from Qantas re: a new flight option to replace our previous Emirates booking. This new flight would be via South Africa, leaving on 27th March. This was very confusing in light of the news we’d heard around border closure in Australia, so we called the Australian High Commission in London in the hope of gaining some clarity. It turned out that the new restrictions were only on Australian citizens leaving Australia and that citizens could still enter if they could find a flight to do so. The poor communication by Scott Morrison (Prime Minister) on this was unhelpful, but we were at least comforted that we could return to Australia if only we could find a way. We took up the option of the new Qantas flight through South Africa and were pleased that they waived the ~AU$3000 change cost on these new flights thanks to my Frequent Flyer status. We reset our plans and expectations yet again around a departure from the UK at the weekend.

A nice walk along the Prom and South beach late in the afternoon and a tasty dinner made up of food from the Parsnipship and Anuna bakery rounded out our day. We called it a night believing our luck had changed and we’d be right to get home fairly soon thanks to these new Qantas flights.

It was a frosty, clear and sunny start to Wednesday and forecast to head right up to 18C, beautiful! An early start secured a good few hours of work in the morning before we headed out for a walk, this time to try and find a modernist-style house we’d noticed from almost every vantage point over Aber. It turned out to be towards the top of Cae Melyn and it was great to see this unusual piece of architecture up close, we could only imagine what the 270-degree views out across Aber must look like from inside the place. A stroll through the lush greenery of Penglais Nature Park was great for our spirits and we walked back to the apartment along the Prom in time for lunch. Returning to work in the afternoon, we soon spotted news that South Africa was heading into lockdown from 26th March – of course! We failed to find any information on the impact of this on transit passengers so sought assistance from the Australian High Commission in South Africa. They were also unsure and suggested we ask our airline. We contacted Qantas and, you’ve guessed it, they suggested we seek government advice. By now we were exasperated and had no confidence that this set of flights would happen either. Desperation was certainly setting in as we searched again for any remaining options for flights from any UK airport to any Australian airport.

The only flights we could find now were with Qatar Airways via Doha. We’d seen their flights before but dismissed them based on long layovers in Doha, scared that regulations might change during the layover and leave us stranded somewhere we really didn’t want to be. We were encouraged, though, by the fact that some Australian relatives had successfully made the trip back with Qatar just the day before so we decided we’d book flights with them, from Birmingham (as the closest and easiest airport to return to from Aber) to Melbourne via Doha. Demand, of course, was really heavy for their flights as they were basically the last option flying into Australia so the flight prices were very high at around AU$5000 each. This presented us with our next problem. We’d been transferring cash over to our credit card as fast as we could, but the timezone difference to Australia meant that this basically took a day each time. We didn’t have enough credit left on our main card for both flights, but could cover one. A call to our bank asking for an emergency credit limit increase fell on deaf ears as it was during the night in Australia with no-one available to authorize such a request. I did have one more credit card in my wallet, unused for years and with enough of a credit limit (from memory) to also cover one ticket, but would this go through? We tried booking one ticket with this card… and it went through successfully! We could then book the other on our usual card. By now, we’d spent close to AU$20,000 on flights in a few days (and no refunds in sight, as Qantas/Emirates and Etihad all want to issue credit notes and not cash refunds), so this really was our last gasp attempt. (The eventual cancellation of our Qantas flights via South Africa was not communicated to us until after we left the UK, by the way.)

With our new flights in place, we needed some fresh air and we headed to the Treehouse to grab some supplies to sustain us on the very long trip home. They were doing a great job of continuing to service the community via their “shout your orders through the door” approach! We came away well armed with enough snacks to keep us going. A final (maybe?!) walk along South beach and the Prom on this lovely clear Spring afternoon was delightful, Aber looked resplendent in the sunshine. Dinner back in the apartment was an exercise in using up what supplies we already had open in order to save waste (we’d already decided to leave most of our haul of vegan organic goodies with the host, as a small token of our appreciation for her help and flexibility). She popped up later in the evening and we said our farewells (maybe?!).

View of the Old College, Prom and Constitution Hill

We had to make an early start on Thursday to pack, tidy the apartment and make the trip to Birmingham. It was with some trepidation that we first checked SMS and email, as well as the latest news updates, to see if Qatar had decided to end transit overnight, but all seemed well.

It was sunny and frosty as we headed out onto the deserted streets of Aber at 7.30am and loaded up our hire car. The drive over Pumlumon Fawr was just stunning with frost-covered paddocks, an abundance of newborn lambs and clear blue skies. The familiar drive back to the Midlands was effortless with so little traffic on the road so we comfortably covered the distance in under three hours including a fuel topup before returning the hire car to Budget at Birmingham airport. The agent at Budget mentioned that her only customers recently were people just like us, returning way too early (we still had sixteen days of our prepaid hire to go) and to the wrong location (we should have returned to Brighton), resulting in over three hundred early returns and basically no cars going out. Wandering down to the terminal, it showed all the signs of being closed – no cars, no passengers walking around, no signs of life. Even after entering the terminal, it was still deadly quiet and our Qatar flight was basically the only sizeable flight departing during the day. Check-in was easy at about 11am, leaving us no rush to make our 2pm flight. There wasn’t too much in the way of distraction during our wait, with only WH Smith’s and Boots being open in the entire terminal (and nowhere to source even a coffee!) The flight unsurprisingly left on time and the six-and-a-half hour flight down to Doha was very comfortable – and we experienced awesome service from an airline voted world’s best in recent times. As we’d booked our flights less than 24 hours before departure, we couldn’t order special (vegan) meals for this first flight. We notified cabin crew when we got on and they promised to look into it. We got amazing personal service from Melina who cobbled together a tray of vegan goodies from other meal trays to keep us going, then later delivered us a delicious vegan meze plate from business class. Impressive attention to detail and we actually felt like she cared about us, very much appreciated under the circumstances (and hopefully her employer does something nice for her based on the feedback we’ve given to them).

The flight arrived into Doha early at 11.20pm local time – and the airport was packed! It was strange to see so many people – for the first time in a couple of weeks – and attempts at social distancing during the lines for security checks weren’t very successful. Many people were in full hazmat gear from head to toe, we had no protective equipment at all as our attempts to source even a face mask in the UK had failed. On entering the main terminal, we were impressed by the spaciousness and feel of the place, but very surprised again to see all shops and eateries open seemingly as usual here, certainly in stark contrast to Birmingham airport.

As we passed into Friday in the airport, we had around twenty hours to kill before our (hopefully!) last flight from Doha direct to Melbourne. We managed to find a quiet spot with comfortable seating we could fashion into a makeshift bed and tag-teamed short spells of sleep. We had power and internet too, so could pass the time on our laptops even if most of that time was spent  following the latest news updates in the hope that nothing scuppered our plans during the long wait.

Of course, it wasn’t too long before another potential problem arose with the news that all arrivals into Australia would soon be subject to 14-day quarantine in hotels of the government’s choosing (to replace the existing home quarantine scheme). The messaging around this change of policy was again inconsistent and confusing – some reports said this new scheme would be in effect “by midnight on Saturday 28th” while others said it would take effect “from midnight on Saturday 28th”. We were due into Melbourne at about 6pm on Saturday so this small difference could potentially make a big difference to us. We didn’t get clarity on this point before the time finally rolled around for us to board the flight to Melbourne after our long, long wait.

The flight left Doha on time and landed early into a deserted Melbourne airport at about 5.30pm on Saturday. We still didn’t know what we’d find as we left the plane and headed to immigration. Thankfully, we had a lucky break and were one of the last flights from which passengers were allowed to head home to begin their 14-day quarantine period, so we were very thankful for that!

The journey home had been long but we were grateful to finally get back to our little house on the beach to begin our quarantine. The ups and downs of the previous few days had taken their toll and, five days later, we’re still adjusting to the new normal. Looking back on the events of last week while writing this blog, it almost doesn’t feel real and it feels all the more remarkable that we actually made it home at all. Attempts to obtain cash refunds from Qantas and Etihad continue…

There were many issues that made the process of finding a way home more complicated and stressful than it needed to be. Firstly, the information on changes to regulations coming out of government was not clear or well-communicated – the complete closing of the Australian border and the timing of the hotel quarantine scheme were two examples of this. Sourcing precise information in both of these cases was difficult as news outlets and even trusted sources like the High Commissions didn’t have consistent or reliable information at hand when the announcements were made.

Secondly, a number of IT systems had clearly failed to account for load and automated systems haven’t accounted for changes along the way. We have numerous examples of such problems, from the Emirates booking system failing part-way through flight bookings to automated “online check-in is now open” emails from Qantas just today about a flight already cancelled almost a week ago. The icing on this particular cake, though, has to go to Qantas again who sent us this SMS about a rescheduling of our South Africa-routed flight after it was cancelled due to the lockdown (asterisks and bold are mine):

We’ve now rebooked you onto flight QF9324 on Fri * Apr from Johannesburg at 10.00 arriving Fictitious Point at 11.00

It feels like we’re actually living at Fictitious Point right now, but we’re home and safe and, so far, feeling healthy. If there is a moral from this story, it’s probably that travelling during a pandemic is not a great idea!