Category Archives: Uncategorized

Everyone’s talking about testing!

I can’t remember a time in my life when “testing” has been such a hot topic of coverage in the media. It feels like every news item leads with some mention of the number of tests conducted to detect the COVID-19 coronavirus, whether it be it locally or further afield. This level of coverage of the topic of testing even exceeds that during Y2K (according to my memory at least), albeit in a very different context.

It was interesting to see the reaction when the President of the United States said that the US case numbers are high because so many tests have been conducted – and that a reduction in testing might be in order. This led Ben Simo to tweet on this idea in the context of software testing:

Stop testing your software! Bugs are the result of testing. Bugs that don’t kill users dead instantly aren’t really bugs. No testing is the key to zero defect software! If you don’t see it, it doesn’t exist. If you don’t count it, it doesn’t matter. No testing!

I felt similarly when I read some of the coverage of the worldwide testing efforts during the pandemic. “Testing” for COVID-19 is revealing valuable information and informing public health responses to the differing situations in which we find ourselves in different parts of the world right now. (In this context, “testing” is really “checking” as it results in an algorithmically-determinable “pass” or “fail” result.)

When we test software, we reveal information about it and some of that information might not be to the liking of some of our stakeholders. A subset of that information will be made up of the bugs we find. In testing “more”, we will likely unearth more bugs as we explore for different kinds of problems, while “more” in the COVID-19 sense means performing the same test but on more people (or more frequently on the same people).

We should remain mindful of when we’ve done “enough” testing. If we are genuinely not discovering any new valuable information, then we might decide to stop testing and move onto something else. Our findings from any test, though, represent our experience only at a point in time – a code change tomorrow could cause problems we didn’t see in our testing today and an unlucky person could acquire COVID-19 the day after a test giving them the all clear.

There is a balance to be struck in terms of what constitutes “enough” testing, be that in the context of COVID-19 or software. There comes a point where the cost of discovering new information from testing outweighs the value of that information. We could choose not to test at all, but this is risky as we then have no information to help us understand changing risks. We could try to test everyone every day for COVID-19, but this would be hugely expensive and completely overwhelm our testing capacity – and would be overkill given what we already understand about its risks.

Many of us are testing products in which bugs are potentially irritating for our users, but not life and death issues if they go undetected before release. The context is clearly very different in the case of detecting those infected by COVID-19.

As levels of COVID-19 testing coverage have increased, the risk of acquiring the virus has become better understood. By understanding the risks, different mitigation strategies have been employed such as (so-called) social distancing, progressively more stringent “lockdowns”, and mandatory mask wearing. These strategies are influenced by risk analysis derived from the results of the testing effort. This is exactly what we do in software testing too, testing provides us with information about risks and threats to value.

It’s also interesting to observe how decisions are being made by bearing in mind a broader context, not just taking into account the testing results in a particular area or country. Data from all across the world is being collated and research studies are being published & referenced to build up the bigger picture. Even anecdotes are proving to be useful inputs. This is the situation we find ourselves in as software testers too, the software in front of us is a small part of the picture and it’s one of the key tenets of context-driven testing that we are deliberate in our efforts to explore the context and not just look at the software in isolation. In this sense, anecdotes and stories – as perhaps less formal sources of information – are incredibly valuable in helping to more fully understand the context.

Test reporting continues to be a topic of great debate in our industry, with some preferring lightweight visual styles of report and others producing lengthy and wordy documents. The reporting of COVID-19 case numbers continues to be frequent and newsworthy, as people look to to form a picture of the situation in their locale. Some of this media reporting is very lightweight in the form of just new daily case and fatality numbers, while some is much deeper and allows the consumer to slice and dice worldwide data. Charts seem to be the reporting style of choice, sometimes with misleading axes that either exaggerate or down play the extent of the problem depending on the slant of the publisher. Different people react to the same virus infection reports in quite different ways, based on their own judgement, biases and influences. We see the same issue with software test reporting, especially when such reporting is purely based around quantitative measures (such as test case counts, pass/fail ratios, etc.) The use of storytelling as a means of reporting is nothing new in the media and I’d argue we would be well served in software testing to tell a story about our testing when we’re asked to report on what we did (see Michael Bolton’s blog for an example of how to tell a three-part testing story – a story about the product and its status, a story about how the testing was done, and a story about the quality of the testing work).

While I don’t normally focus on counting tests and their results, I’ll be happy to see more COVID-19 tests taking place and fewer new daily positive results in both my area of Australia and the world more generally. Stay safe.

(Many thanks to Paul Seaman for his review of this post, his sage feedback has made for a much better post than it otherwise would have been.)

Going meta: a blog post about writing a blog post

My recent experiences of authoring blog posts in WordPress have been less enjoyable than usual thanks to the use of their latest “block editor”, leading me to ask on Twitter:

WordPress seems to update their post editor very frequently so I just about learn the quirks of one when it is superseded by another.

This post will serve as my (long) answer to WordPress’s reply. I’m going to spend the next 45 minutes running an exploratory testing session, creating a blog post and noting issues as I come across them while using the block editor.

Session from Tuesday 21st July 2020 , on Windows 10 laptop (using keyboard and mouse controls only) using Chrome browser

4:10pm I’m starting my session by writing a very basic block of unformatted text. I note that when I move my mouse, a small toolbar appears which covers the end of the previous block (this could be an issue when in the flow of writing). The toolbar disappears as soon as I type and reappears on every mouse movement. The content of the toolbar seems very limited, maybe to just the most used formatting features (most used by the whole WordPress community or most used by me)? At least each icon in the toolbar has a tooltip. There’s a very odd control that only appears when hovering over the leftmost icon (to change block type or style) which appears to facilitate moving the whole block up or down in the post. I wonder why the toolbar is so narrow, snce there is plenty of room to add more icons to allow easier discovery of available options here. I’ve been distracted by the toolbar but now resume my mission to complete a basic paragraph of text.

OK, so hitting Enter gives me a new paragraph block, that makes sense. Let’s get more creative now, how about changing the colour of some text? The toolbar doesn’t appear to have a colour picker, oh, it’s tucked away under “More rich text controls”. I’ve typed some text, highlighted it and then selected a custom colour. That worked OK once I found the colour picker. The colour picker control seems to stay in the toolbar after using it – or does it? I’ll try it again but lo, it’s back under the hidden controls again. There’s probably a deliberate choice of behaviour here, but I’ll choose not to investigate it right now.

I’m trying to select some text across blocks using Shift+Arrow keys but that doesn’t work as I’d expect, being inconsistent with other text selection using this keyboard combination in other text processing applications. (Ctrl+Shift_Arrow keys suffers the same fate.) Shift+Page Up/Down only select within the current block, again not what I’d expect.

4:30pm After adding this new block (just by pressing Enter from the previous one), I’m intrigued by the array of block types to choose from when pressing the “+” button which appears in seemingly different spots below here (and I just spotted another “+” icon on the very top toolbar of the page and it looks like it does the same thing). There are many block types, so many that a search feature is provided (a testing rabbit hole I’ll choose not to go down at the moment). Some of the block types have names which indicate they require payment to use and the available block types are categorized (e.g. Text, Media, etc.) I decide to try a few of the different block types.

Adding a “quote” block now, which offers two areas, one for the quote and one for the citation. It appears that the citation cannot be removed and so more space is left below the quote text than I’d like (but maybe it doesn’t render the empty space when published?).

A test quote without citation

Moving on to adding a list and this works as I’d expected, offering a choice between bulleted and numbered with indentation (maybe there’s a limit on nesting here, but not investigated).

  • First item of my list
  • Next item of my list
    • Indented!

Even though I’ve been using this editor for my last few blog posts, I still tend to forget that auto-save is no longer a thing and I just happened to notice the “Save Draft” in the top right corner of the page, so let’s save.

In reality, my blog posts are mainly paragraphs of text with an occasional quote and image so exploring more block types doesn’t seem worth the effort. But looking at images feels like a path worth following.

Copying an image on the clipboard seems to work OK, though immediately puts cursor focus into the caption so I started typing my next bunch of paragraph text incorrectly as the image caption.

Options in the toolbar for the image make sense and I tried adding an image from a file with similar results (deleted from the post before publishing). Adding images into a post is straightforward and it’s good to see copying in directly from the clipboard working well as there have been issues with doing so in previous incarnations of the editor.

4:45pm Returning to simply writing text, I often add hyperlinks from my posts so let’s try that next. Ctrl+K is my usual “go to” for hyperlinks (from good ol’ Word) and it pops up a small edit window to add the URL and Enter adds it in: http://www.google.com Selecting some text and using the same shortcut does the same thing, allowing the text and the URL to be different. The hyperlinking experience is fine (and I note after adding the two hyperlinks here that there’s a “Link” icon in the toolbar also).

I remember to save my draft. As I resume typing, the toolbar catches my eye again and I check out “More options” under the ellipsis icon. I notice there are two very similar options, “Copy” and “Duplicate”, so I’ll try those. Selecting “Copy” changes the option to “Copied!” and pasting into Notepad shows the text of this block with some markup. I note that “Copied!” has now changed back to “Copy”. Selecting “Duplicate” immediately copies the content of this block right underneath (deleted for brevity), I’m not sure what the use case would be for doing that over and above the existing standard copy functionality. OK, I’ve just realised that I’ve been distracted by the toolbar yet again.

I just added this block via a “hidden” control, I’m not sure why products persist with undiscoverable features like this. Hovering just below an existing block halfway across the block reveals the “+” icon to add a block (though it often seems to get ‘blocked’ by, you’ve guessed it, that toolbar again).

My time is just about up. As I review my short session to create this blog post, I think it’s the appearing/disappearing toolbar that frustrates me the most during authoring of posts. I almost never use it (e.g. I always use keyboard shortcuts to bold and italicize text, and add hyperlinks) and, when I do, the option I’m after is usually tucked away.

Thanks to WordPress for responding to my tweet (and providing what is still generally a great free platforms for blogging!) and for giving me a good excuse to test, learn and document a session!

ER of attending my first virtual testing conference, Tribal Qonf (27 & 28 June 2020)

I spotted some promotion on Twitter for a new testing conference in India, Tribal Qonf, and the virtual nature of it (thanks to COVID-19) plus the impressive speaker line-up (including James Bach and Michael Bolton) spurred my interest. Looking into it further, the pricing was incredibly low so I decided to register for it (for around AU$30 at the time I registered).

Although the weekend scheduling of the conference and Indian timezone wasn’t ideal, the conference promised to provide all content via recordings so I didn’t tune into any of the presentations “live”, waiting instead the ten days or so for recordings to be made available. I then watched most of the presentations from the two-day event over a period of a few days.

The first presentation I watched was the opening talk from day 1 by James Bach, titled “Weaving Testing: Thread by Thread” This was a fascinating talk and it was great to see such a detailed analysis of what actually happens during good testing by skilled practitioners, especially compared to the mythology we’ve generally been conditioned with about what makes for ‘proper’ testing.

Next up, I opted for Pradeep Soundararajan‘s talk on “The Business Value of Testing”. I’ve unfortunately never managed to catch Pradeep presenting in person, but this virtual presentation displayed the passion I expected from him. It was also engaging and refreshingly honest about the challenges we face in terms of recognizing how different stakeholders view the “value” of what we provide as testers.

My next choice was “Adopting a simplified Risk-Based Testing Approach” by Nishi Grover Garg, in which she outlined the basics of the approach, very much in the style of practitioners like Rex Black. The approach was presented very clearly here and I liked the way Nishi contextualized the risk-based testing approach to her startup environment.

A nicely-crafted story came next thanks to Ajay Balamurugadas and his talk “Lessons from 14 Years of Software Testing Career”. He detailed his learnings from each of his testing jobs and offered practical suggestions for areas to focus on at different levels of experience in testing. This presentation reminded me very much of my “A Day In The Life Of A Test Architect” talk which I gave at STARWest in 2016 and again at CAST in 2017.

Rounding out the talks for day 1, I somewhat hesitantly tuned into the ‘expert panel’ on “Testing after 2020”. I’ve become a little jaded about panel sessions but I really enjoyed this one featuring Aprajita Mathur, Ashok Thiruvengadam, Rahul Verma and Pradeep Soundararajan. The panelists responses to the various questions were refreshingly down to earth and practical. I was particularly pleased to see the considered, reasonable and sensible discussions around AI/ML in testing, providing welcome relief from the usual Kool Aid drinkers around these topics in the industry at the moment. A shout out to Lalit Bhamare too for his skillful moderation of this panel session which was a significant factor in its success for me.

I kicked off my “day 2” viewing with the first talk from that day, viz. , Ashock Thiruvengadam with “Be in a Flow. Test Brilliantly” This was something a little different in terms of topic for a testing conference (which is always good to see), focusing on introducing the idea of “flow”. I was reminded of the importance of uninterrupted sessions when performing exploratory testing while listening to this talk.

Next, I opted for Mike Talks with “The Hard Lessons Learned in Test Automation”, in which he shared some interesting stories of lessons learned resulting from his chats with testers over coffee in his home city of Wellington (New Zealand). It was unsurprising to me that his chats resulted in a few very common themes, all of which were familiar territory from my various conversations about automation with testers from all over the world over the last twenty-odd years. It seems we have a long way to go in terms of learning these hard lessons, despite them being covered ad nauseam in blogs, articles, books and conference talks.

My next choice was “A Quick Recipe for Test Strategy” from Brijesh Deb and I immediately liked his take on the topic. He defined a test strategy simply as a “set of ideas that guide test design” and made it clear that we shouldn’t conflate this with a hefty “one size fits all” document of some sort. I also liked his focus on driving test strategy by asking questions, with not just a shout out to James Bach‘s Heuristic Test Strategy Model but also an example of using it in practice.

The penultimate talk I watched was “Who Are Your Stakeholders?” with Anna Royzman. We often hear the term “stakeholders” used in testing (and software development more generally) but rarely do we seem to agree on what this term means in the context of our projects. Anna gave a good introduction on how to identify different types of stakeholders and what kinds of information these different stakeholders might be looking for.

I concluded my binge watching of the conference talks with the closing session from the event, in the shape of a “Fireside Chat with Michael Bolton” with questions coming from Ajay Balamurugadas. I loved Michael’s answer to Ajay’s question “what has changed in testing from 1994 to 2020?”, “not enough!” This was a fun fifty minute session and a perfect way to wrap up the conference.

Obviously, “attending” a virtual conference is a completely different experience to an in-person event. I chose not to watch all of the presentation recordings but did watch most of them and the quality was high. I didn’t watch the recordings back-to-back either, rather spreading out my viewing across a few days alongside my usual work commitments. I also didn’t contribute to the conference’s Slack channels as the event had been over for two weeks or so by the time I got to the recordings.

I personally missed the in-person aspects that make traditional conferences so valuable, but it might not be the case that we have to choose one over the other as we move forward. I wonder if we’re entering a new era for conferences, driven by changes forced upon us by COVID-19. There are enormous accessibility benefits of the virtual model, thanks to lower pricing and the removal of the need to travel and spend time away from home & family. Such virtual events also open up opportunities for new voices who might be unable or unwilling to travel to a “normal” event, or are too uncomfortable to address a physical audience.

The selection of topics on offer during this event was good and the talks were of a high standard. It appeared to be well organized too, so thanks to Lalit and the Test Tribe crew for putting on a worthwhile testing event during these difficult times! I enjoyed the experience of this virtual conference and I am now considering attending other virtual testing conferences through 2020 before – maybe! – more normal service resumes in 2021…

My experience of working from home full-time during the pandemic

Thanks to Twitter, I spotted that a new testing-related ebook had been published recently, titled Software People Work From Home, in which a number of testers from around the world describe their personal experiences of working from home thanks to coronavirus-induced restrictions.

I really enjoyed reading this (free) ebook and felt inspired to share some of my own experiences based on more than two months of working full-time at home for Quest.

Firstly, some context around my “normal” work situation. Although I am based in our Melbourne city office, I’ve been working from home for three to four days a week for the last five years or so. Part of the reason for this mode of working is due to the long commute from my home to the Melbourne office (around two hours door-to-door, each way) and another factor is the many early morning/late night meetings I’m involved with thanks to my collaboration with Engineering teams across basically every timezone!

This hybrid model has worked well for me and for Quest over the last few years. The reduced requirement to be within “normal” commuting distance of Melbourne means I can live in a beautiful location, right on the beach with peace and fresh air – this has made a huge difference to my lifestyle and well-being. We always make time during the day for a long walk (typically around 5km) and having this as part of my routine continues to be really important to me.

I intended to resume this model of working following my return from some international travel back in mid-March. (I’ve blogged about the experience of travelling during the pandemic separately here and here.) Of course, on returning from the UK towards the end of March, our office had already been closed until further notice thanks to COVID-19 – and so began my immediate transition to working from home full-time.

It’s been an interesting three months in this new mode of working. In many ways, I am very fortunate. Firstly, I’ve kept a full-time job on the same salary as pre-COVID so there are no additional financial pressures resulting from this change. I’ve also had plenty of practice at working from home over the last several years, so the adjustment to full-time at home hasn’t been as significant as for many people. I’m lucky enough to have a dedicated room in our (albeit very small) home to work from (and the incredible water view from there never gets old!) and we are a couple with no kids so it’s pretty straightforward to maintain a quiet environment in which to concentrate on work.

Even adding the extra day or two at home every week compared to my usual routine has revealed some additional benefits of the arrangement. I’m settling into a great circadian rhythm (save for a few early morning meetings) and not commuting has freed up some more time to enjoy relaxing at home. We’re cooking together more often too.

While it’s generally a positive for me to work from home all the time, there are challenges too, the most significant of which for me is avoiding overworking. I spotted a great quote on Twitter recently:

Stop calling it ‘working from home’ and start calling it ‘living at work’

Heather De-Quincey

My boundaries between “work” and “not work” are – and never have been – very strict. I have access to all of the systems folks within Quest might use to contact me pretty much all of the time. My role operates across our entire business unit, which has people in timezones spanning the whole world.

Shortly after lockdowns became part of almost everyone’s life experience, we decided to convert what would have been a large in-person meeting in California into a virtual event. While it was a great success, organizing and running a 70+ person meeting across so many timezones was a huge effort by many people – I worked sixteen-hour days for three days straight during the event, resulting in severe over-tiredness (and unwelcome grumpiness on the home front).

There is always someone looking for something from me, so it can be hard to ignore those requests even when they’re out of what most would think of as “business hours”. The early morning and late night meetings necessitated particularly by the need to interact with folks in the US are draining, but I’ve learned to speak up more and refuse meetings before 7am or after 11pm so as to allow for a reasonably consistent sleep pattern.

There are some things I am missing as a result of not just working from home all the time, but also the general world situation thanks to the pandemic. I was looking forward to attending the CAST conference in Austin in August, but that of course has been cancelled. This is also the first time in recent memory that I haven’t had an overseas trip (or two or three!) in planning, either for work or leisure (or sometimes a combination of both). It feels strange not having these adventures to look forward to anytime soon.

Closer to home, my day or two up in Melbourne affords me a number of opportunities and benefits that I’d taken for granted. I miss my weekly coffee catch-ups with Paul Seaman, although we maintain close virtual contact. I make very good use of the Melbourne Library Service and usually have an interesting book or two on the go, but there’s been no opportunity to borrow books recently. I also miss the chance to have lunch or coffee with ex-colleagues, something I do almost every time I travel into the city. And, of course, the kitchen banter with my Quest colleagues is sadly lacking – and Teams meetings are just not the same!

Overall, I’m enjoying the experience of working from home full-time and the downsides certainly don’t outweigh the positives of both my work output and lifestyle. With Quest keeping its offices closed until at least September 2020, I’ll get to enjoy it for a while longer.

The Melbourne testing market and spotting “red flags”

One of the more popular job listing sites in Australia is Seek. A search on this site for Melbourne-based jobs under the Information & Communication Technology -> Testing & Quality Assurance classification yielded 70 results as I began writing this post on 29th May 2020.

Looking at these jobs, I’ve broadly categorized them as follows:

  • Automation (32)
    • Automation/SDET
    • Test Automation Engineer (x2)
    • Automation Test Consultant (x2)
    • Senior QA Automation Engineer
    • Test Automation Specialist (x2)
    • Automation Test Analyst (x5)
    • Automation Test Analyst – Tosca
    • Senior Automation Test Analyst
    • QA Engineer (Automation)
    • Automation Tester (x4)
    • Senior Test Automation Engineer/Lead (x2)
    • Java Development Engineer in Test (DevOps)
    • Lead/Sr.Automation Tester
    • Senior Test Automation Specialist
    • Automation Test Engineer
    • Automation Test Engineer (Embedded Software)
    • Applications Testing Automation Engineers (x4)
    • Technical Test Lead – Automation Lead Tosca
  • Management (7)
    • Test Manager
    • Performance Test Manager (x2)
    • Defect Manager (x4)
  • Specialists (14)
    • Performance Tester (x4)
    • Penetration Testing
    • Performance Test Consultant
    • Infrastructure and Security Test Analyst
    • Senior Test Analyst – Infrastructure testing
    • Performance Test Analyst (x2)
    • Performance Engineer (x2)
    • Jmeter Tester
    • Network Test Engineer (hardware-related)
  • “Manual” / Other (17)
    • Test Analyst
    • Test Analyst – Business Intelligence
    • Senior Test Analyst (x2)
    • UAT Tester
    • Mobile QA Engineer
    • Quality Assurance Engineer
    • QA Engineer
    • Graduate Test Analyst
    • Automation Specialists and Defect Managers
    • Junior Quality Assurance Tester Traineeship
    • Senior Software Test / V&V Engineers (Defence)
    • Validation & Verification Lead
    • Integration Testers (x4)

These ads are not representative of unique jobs, as exactly the same ad is sometimes posted at different times and/or the same job is typically posted by a number of different recruiters (especially for government roles).

The breakdown in terms of the number of opportunities didn’t surprise me. The focus at the moment seems to be around automation driven by CI/CD investments, DevOps/agile transformations – and, of course, the overestimation of what automation can and can’t do. Similarly, performance and security-related testing are topics de jour as represented by a swathe of ads in these areas. Test management doesn’t seem like a good place to be, with very few roles being advertised in recent times and this change has been heavily driven by agile adoption in my experience.

I generally take more interest in the ads focused on human testing to see what companies are looking for in this area. Most of the more “traditional” human testing roles (e.g. in government departments) also now mandate some degree of proficiency in tools commonly associated with “automated testing”. It’s pleasing to see requests for ISTQB certification becoming much less common and I’ll occasionally even spot a reference to “exploratory testing”.

But there are often “red flags” in these ads and I proffer a few examples from this most recent search of the “opportunities” on offer.

First up, a “Senior Test Analyst” role “to take accountability for the testing capabilities within the Microservices team.” The “Technical experience” requirements are listed first and include “Linux/Unix Shell Scripting, Java Programming, Clean Coding, Git, Jenkins2, Gradle, Docker, REST APIs, SQL, Unit Testing (Junit), Component Testing, BDD, Integration Testing” and then finally the “Testing Practices” requirements are presented, viz. “Exploratory Testing, Mind Mapping, Requirements Analysis, Peer Collaboration, Continuous Integration, Continuous Deployment, Stubbing and Mocking”.

There are a few red flags here for me. If this were a true microservice team, then it would have a clear sense of ownership of its microservice and a whole team approach to the testing and quality of that service. I’d be looking for clarification of what “accountability for the testing capabilities” really means in the context of this team. Another issue is the lack of clarity about whether this is a human testing role or a development (automation code) role, or a hybrid, or something else. The fact that the technical (mainly development) skills requirements are listed before the testing ones would immediately lead me to believe that more value is placed on automation than on deep human testing here. While it’s good to see exploratory testing explicitly listed (albeit as a “testing practice”), the other requirements listed around testing are much less convincing as to whether this organization would truly value the contribution of an excellent exploratory tester.

Next up, a “Quality Assurance Engineer” role where the successful applicant (who becomes just a “Quality Engineer” as the ad goes on) will “play a critical role in transforming quality practices… and work to develop a world-class test automation solution. [They will] work as part of a cross functional squad, acting as the QA expert, across new features, enhancements and bug fixes. [They’ll use their] testing experience to advise on automation opportunities and review requirements to ensure a fit for purpose solution in (sic) delivered.”

In describing what the day-to-day work entails for this role, there are some positive signs: “You’ll be passionate about testing and use your experience to identify critical defects. You’ll be naturally curious and explore the latest tools and techniques to continuously improve testing” But there are also some less positive signs: “You’ll work as a fully engaged member of cross functional squad including Developers, UX/UI Designers and Product Managers to ensure the quality of products. You’ll create, maintain and execute tests (manual or automated) to verify adherence to requirements.” Again, it’s good to see exploratory testing getting a mention, albeit in what comes across as a confused way (especially in light of the “verify adherence to requirements” elsewhere), “You’ll perform risk based exploratory testing.”

In terms of skills, they’re expecting “expertise in developing functional tests that cover 100% of requirements and execute manually or automated (with focus on least maintenance and faster script development time)” and possession of “strong skills on efficient test design”.

There are obviously a few red flags in this ad. It sounds like this organization has latched into the so-called “Spotify Model” since it mentions “squads” a number of times, even though this model was actually not successfully adopted within Spotify. It talks about “ensuring” quality and verifying “adherence to requirements”, while at the same time asking for exploratory testing skills (of the risk-based variety, of course). Covering “100% of requirements” completes a picture of an organization where verification, preferably by machine, is valued much more than human testing.

My final example is for a “QA Engineer” role which asks for background in “both manual and automated testing” The red flags come real early in this one: “Can you imagine how it would feel to be responsible for ensuring that our key products work well in all conditions? Are you interested in working in a Global Centre of Excellence…?” I’ll choose not to imagine how it would feel to be given an impossible mission.

This lucky candidate “will be responsible for both manual and automated testing as we move towards complete automation.” At least there is no doubt left here about the value of deep human testing as this organization seeks to “automate everything.”

I like the requirement to be “Excellent at finding the problems others miss”, but much less keen to see this followed by “Able to document test cases based on solution requirements provided. Understanding of Developing in an Agile environment”. Advanced-level “ISTBQ” is then included in their list of “desirable” skills.

I don’t understand why this organization believes this ad would attract an excellent human tester who could leverage exploratory testing to genuinely find “the problems others miss”. They describe themselves as Agile but want the successful candidate to write test cases, all the time trying to get to a point where “manual” testing is no longer required. Based on the ad alone, either this is an organization with a confused outlook on testing or they’re really looking for someone to take on an impossible mission while devaluing their actual testing skills – either way, this doesn’t sound like a great way for a decent tester to spend their working life (but I acknowledge the ad could just be very poorly-written and misrepresents the core beliefs of the organization when it comes to testing).

While it’s interesting to review occasionally what organizations are looking for when they’re advertising for testing-related roles, it’s also pretty depressing to see just how little value seems to now be placed on deep human testing skill. The “automate everything” bandwagon seems to roll on and is taking over this market, while opportunities for those genuinely skilled in testing as a craft seem to become fewer and fewer, at least in the Melbourne market represented by published job ads.

As the economic impact of COVID-19 takes its toll, a large number of folks are hitting the IT market at the same time. In just the last couple of weeks, large IT staff reductions have been reported in Melbourne by big tech names such as MYOB and Culture Amp – and more seem likely in the coming months. If there’s a bright side to this situation, it’s that there are no doubt lots of good people coming back into the market so it’s probably a great chance to pick up testing talent for those organisations able to do so. If you’re representing such an organisation and really want to hire skilled human testers, please take the time to construct better tester job ads! Making it sound like you understand the value of testing is likely to attract those really good testers who might have come back into the market thanks to COVID-19.

Remember that a good tester should be a strong critical thinker and will be testing your ad, so catch your own “red flags” before they do!

ER of international travel during the coronavirus pandemic (part 2)

This is the second part of my travelogue/ER about travelling between the UK and Australia during the coronavirus pandemic (the first part can be found here). I’m writing this post on day 5 of our enforced 14-day home quarantine period following our return to Australia.

This second week began on Monday 23rd March and you might recall that we were now based in Aberystwyth on the West coast of Wales. The day started with news from Cathay Pacific about our new return flight option, that being a departure from London on 18th April and only getting us back as far as Sydney as their routes to Melbourne were stopping completely. We suddenly felt like mid-April was a long way away given how fast things had changed in just the previous seven days, so we decided to look for alternatives to get us home earlier. (As it turned out, Hong Kong would soon announce the banning of transit passengers through its airport so these new return flights with Cathay would never happen anyway.)

Looking for flights back to Australia, we spotted some Emirates options via Dubai and booked new flights departing London on 1st April, costing almost AU$3000 for the two of us. This process wasn’t straightforward as these newly-announced flights were selling so fast that the website didn’t respond well to the load and we missed out on flights for several other earlier dates as the transactions failed part-way through. At least 1st April didn’t feel so far away and we felt comfortable waiting it out for a week-or-so in Aber before we could return home. With the flights booked, I refocused on my Quest work for a while before we made the most of the lovely sunny day with a walk along the Prom and South Beach. We bumped into my Mathematics PhD supervisor from all those years ago, Alun Morris, and his wife Mary while walking and it was great to see them. They stopped for a chat (at a distance, of course!) and it was our first interaction with someone we knew since we’d left Australia. Alun looked in great health (it’s that Aber sea air!) and seeing him again was a fillip to our morale. Lunch back at the apartment gave way to more work in the afternoon.

Work was soon interrupted, however, by an announcement from Emirates that they were suspending operations from 25th March (and, as it turned out shortly afterwards, Dubai would be prohibiting transit passengers), rendering our newly booked flights useless.

We were really disappointed and again looked for alternatives, this time coming across flights with Etihad from Manchester via Abu Dhabi to Melbourne leaving the next day. We booked these flights, accruing another AU$5000+ on our credit card, and started to make our plans for leaving Aber (including informing our host that we would now be leaving earlier than expected).

Now thoroughly distracted, we headed out into the late afternoon sunshine and took what we thought would be a last chance to climb Constitution Hill. The stunning views over this pretty town and coastline never fail to impress and we soaked them in, before again watching the starlings doing their thing at sunset over the Pier. Back at the apartment, we cooked up a big meal to use up at least some of the nice organic veggies we’d bought at the Farmers Market. It was shortly after devouring this feed that the news came through that Abu Dhabi (as part of the United Arab Emirates) was also banning transit passengers with almost immediate effect, so our latest flight booking was again rendered useless. Our host came up to see us about our early departure and, luckily for us, she was very understanding about the fact that we would yet again have to change our plans and stay on. We were thoroughly exhausted by the end of the day after all the ups and downs.

View over Aber from Constitution Hill

We woke early on Tuesday morning to read the news that Australia was enforcing a complete travel ban, even on its own citizens (this was an escalation of the previous restrictions on entry) so we resigned ourselves to the fact that we were in for the long haul with our stay in Aber. We considered ourselves fortunate to be in stable accommodation in a familiar place and my ability to continue working meant we were under no financial pressure. Many other stranded Aussies were in far worse situations across the world.

As resignation set in, we tried to reset mentally and I buried myself into work again. It was another sunny and very mild day so we enjoyed a nice walk back out to Tan-y-Bwlch beach (which was completely deserted) before returning for lunch in our apartment. Back into work in the afternoon, a strange Twitter DM arrived from Qantas re: a new flight option to replace our previous Emirates booking. This new flight would be via South Africa, leaving on 27th March. This was very confusing in light of the news we’d heard around border closure in Australia, so we called the Australian High Commission in London in the hope of gaining some clarity. It turned out that the new restrictions were only on Australian citizens leaving Australia and that citizens could still enter if they could find a flight to do so. The poor communication by Scott Morrison (Prime Minister) on this was unhelpful, but we were at least comforted that we could return to Australia if only we could find a way. We took up the option of the new Qantas flight through South Africa and were pleased that they waived the ~AU$3000 change cost on these new flights thanks to my Frequent Flyer status. We reset our plans and expectations yet again around a departure from the UK at the weekend.

A nice walk along the Prom and South beach late in the afternoon and a tasty dinner made up of food from the Parsnipship and Anuna bakery rounded out our day. We called it a night believing our luck had changed and we’d be right to get home fairly soon thanks to these new Qantas flights.

It was a frosty, clear and sunny start to Wednesday and forecast to head right up to 18C, beautiful! An early start secured a good few hours of work in the morning before we headed out for a walk, this time to try and find a modernist-style house we’d noticed from almost every vantage point over Aber. It turned out to be towards the top of Cae Melyn and it was great to see this unusual piece of architecture up close, we could only imagine what the 270-degree views out across Aber must look like from inside the place. A stroll through the lush greenery of Penglais Nature Park was great for our spirits and we walked back to the apartment along the Prom in time for lunch. Returning to work in the afternoon, we soon spotted news that South Africa was heading into lockdown from 26th March – of course! We failed to find any information on the impact of this on transit passengers so sought assistance from the Australian High Commission in South Africa. They were also unsure and suggested we ask our airline. We contacted Qantas and, you’ve guessed it, they suggested we seek government advice. By now we were exasperated and had no confidence that this set of flights would happen either. Desperation was certainly setting in as we searched again for any remaining options for flights from any UK airport to any Australian airport.

The only flights we could find now were with Qatar Airways via Doha. We’d seen their flights before but dismissed them based on long layovers in Doha, scared that regulations might change during the layover and leave us stranded somewhere we really didn’t want to be. We were encouraged, though, by the fact that some Australian relatives had successfully made the trip back with Qatar just the day before so we decided we’d book flights with them, from Birmingham (as the closest and easiest airport to return to from Aber) to Melbourne via Doha. Demand, of course, was really heavy for their flights as they were basically the last option flying into Australia so the flight prices were very high at around AU$5000 each. This presented us with our next problem. We’d been transferring cash over to our credit card as fast as we could, but the timezone difference to Australia meant that this basically took a day each time. We didn’t have enough credit left on our main card for both flights, but could cover one. A call to our bank asking for an emergency credit limit increase fell on deaf ears as it was during the night in Australia with no-one available to authorize such a request. I did have one more credit card in my wallet, unused for years and with enough of a credit limit (from memory) to also cover one ticket, but would this go through? We tried booking one ticket with this card… and it went through successfully! We could then book the other on our usual card. By now, we’d spent close to AU$20,000 on flights in a few days (and no refunds in sight, as Qantas/Emirates and Etihad all want to issue credit notes and not cash refunds), so this really was our last gasp attempt. (The eventual cancellation of our Qantas flights via South Africa was not communicated to us until after we left the UK, by the way.)

With our new flights in place, we needed some fresh air and we headed to the Treehouse to grab some supplies to sustain us on the very long trip home. They were doing a great job of continuing to service the community via their “shout your orders through the door” approach! We came away well armed with enough snacks to keep us going. A final (maybe?!) walk along South beach and the Prom on this lovely clear Spring afternoon was delightful, Aber looked resplendent in the sunshine. Dinner back in the apartment was an exercise in using up what supplies we already had open in order to save waste (we’d already decided to leave most of our haul of vegan organic goodies with the host, as a small token of our appreciation for her help and flexibility). She popped up later in the evening and we said our farewells (maybe?!).

View of the Old College, Prom and Constitution Hill

We had to make an early start on Thursday to pack, tidy the apartment and make the trip to Birmingham. It was with some trepidation that we first checked SMS and email, as well as the latest news updates, to see if Qatar had decided to end transit overnight, but all seemed well.

It was sunny and frosty as we headed out onto the deserted streets of Aber at 7.30am and loaded up our hire car. The drive over Pumlumon Fawr was just stunning with frost-covered paddocks, an abundance of newborn lambs and clear blue skies. The familiar drive back to the Midlands was effortless with so little traffic on the road so we comfortably covered the distance in under three hours including a fuel topup before returning the hire car to Budget at Birmingham airport. The agent at Budget mentioned that her only customers recently were people just like us, returning way too early (we still had sixteen days of our prepaid hire to go) and to the wrong location (we should have returned to Brighton), resulting in over three hundred early returns and basically no cars going out. Wandering down to the terminal, it showed all the signs of being closed – no cars, no passengers walking around, no signs of life. Even after entering the terminal, it was still deadly quiet and our Qatar flight was basically the only sizeable flight departing during the day. Check-in was easy at about 11am, leaving us no rush to make our 2pm flight. There wasn’t too much in the way of distraction during our wait, with only WH Smith’s and Boots being open in the entire terminal (and nowhere to source even a coffee!) The flight unsurprisingly left on time and the six-and-a-half hour flight down to Doha was very comfortable – and we experienced awesome service from an airline voted world’s best in recent times. As we’d booked our flights less than 24 hours before departure, we couldn’t order special (vegan) meals for this first flight. We notified cabin crew when we got on and they promised to look into it. We got amazing personal service from Melina who cobbled together a tray of vegan goodies from other meal trays to keep us going, then later delivered us a delicious vegan meze plate from business class. Impressive attention to detail and we actually felt like she cared about us, very much appreciated under the circumstances (and hopefully her employer does something nice for her based on the feedback we’ve given to them).

The flight arrived into Doha early at 11.20pm local time – and the airport was packed! It was strange to see so many people – for the first time in a couple of weeks – and attempts at social distancing during the lines for security checks weren’t very successful. Many people were in full hazmat gear from head to toe, we had no protective equipment at all as our attempts to source even a face mask in the UK had failed. On entering the main terminal, we were impressed by the spaciousness and feel of the place, but very surprised again to see all shops and eateries open seemingly as usual here, certainly in stark contrast to Birmingham airport.

As we passed into Friday in the airport, we had around twenty hours to kill before our (hopefully!) last flight from Doha direct to Melbourne. We managed to find a quiet spot with comfortable seating we could fashion into a makeshift bed and tag-teamed short spells of sleep. We had power and internet too, so could pass the time on our laptops even if most of that time was spent  following the latest news updates in the hope that nothing scuppered our plans during the long wait.

Of course, it wasn’t too long before another potential problem arose with the news that all arrivals into Australia would soon be subject to 14-day quarantine in hotels of the government’s choosing (to replace the existing home quarantine scheme). The messaging around this change of policy was again inconsistent and confusing – some reports said this new scheme would be in effect “by midnight on Saturday 28th” while others said it would take effect “from midnight on Saturday 28th”. We were due into Melbourne at about 6pm on Saturday so this small difference could potentially make a big difference to us. We didn’t get clarity on this point before the time finally rolled around for us to board the flight to Melbourne after our long, long wait.

The flight left Doha on time and landed early into a deserted Melbourne airport at about 5.30pm on Saturday. We still didn’t know what we’d find as we left the plane and headed to immigration. Thankfully, we had a lucky break and were one of the last flights from which passengers were allowed to head home to begin their 14-day quarantine period, so we were very thankful for that!

The journey home had been long but we were grateful to finally get back to our little house on the beach to begin our quarantine. The ups and downs of the previous few days had taken their toll and, five days later, we’re still adjusting to the new normal. Looking back on the events of last week while writing this blog, it almost doesn’t feel real and it feels all the more remarkable that we actually made it home at all. Attempts to obtain cash refunds from Qantas and Etihad continue…

There were many issues that made the process of finding a way home more complicated and stressful than it needed to be. Firstly, the information on changes to regulations coming out of government was not clear or well-communicated – the complete closing of the Australian border and the timing of the hotel quarantine scheme were two examples of this. Sourcing precise information in both of these cases was difficult as news outlets and even trusted sources like the High Commissions didn’t have consistent or reliable information at hand when the announcements were made.

Secondly, a number of IT systems had clearly failed to account for load and automated systems haven’t accounted for changes along the way. We have numerous examples of such problems, from the Emirates booking system failing part-way through flight bookings to automated “online check-in is now open” emails from Qantas just today about a flight already cancelled almost a week ago. The icing on this particular cake, though, has to go to Qantas again who sent us this SMS about a rescheduling of our South Africa-routed flight after it was cancelled due to the lockdown (asterisks and bold are mine):

We’ve now rebooked you onto flight QF9324 on Fri * Apr from Johannesburg at 10.00 arriving Fictitious Point at 11.00

It feels like we’re actually living at Fictitious Point right now, but we’re home and safe and, so far, feeling healthy. If there is a moral from this story, it’s probably that travelling during a pandemic is not a great idea!

ER of international travel during the coronavirus pandemic (part 1)

My next couple of blog posts are not my usual subject matter, but I wanted to tell this story (and, honestly, it feels cathartic right now to do so). These posts are perhaps best described as travelogues, serving as experience reports of what it was like travelling between Australia and the UK during the pandemic. I’m writing this one on day 4 of an enforced 14-day home quarantine period following our return to Australia.

Late last year, we were pleased to learn that a good friend in the UK was getting married in April 2020 and so planned a month-long trip from Australia to attend the wedding, do some travelling around the UK to catch up with friends and family, and take in a few events along the way.

As we looked for good flight options, Cathay Pacific offered a good deal and we were happy to use them given good experiences in the past. Since we would be transiting through Hong Kong, I decided to take the opportunity to break the trip there and go spend a few days in Quest’s Zhuhai office as I was long overdue a visit there anyway. For the rest of the trip, I planned to work a day or two a week during the trip to minimize the impact of my absence on Quest.

We built an itinerary of about a week in China followed by three weeks in the UK, mainly based in Wales then finishing up with the wedding and a few relaxing days in Brighton before the long trip home just after Easter.

The first spanner in the works came with the impact of the coronavirus outbreak in China, meaning a visit to the office was too dangerous (and also, as it later turned out, pointless as all our staff would be working remotely during isolation). So we changed our plans slightly to transit directly through Hong Kong, tacking on a few days in London to start the trip to replace the planned time in China.

As our departure date approached, the coronavirus news of course became much worse as the spread continued around the world. The UK seemed to be faring well, though, and Boris was persisting with his “keep calm and carry on”/herd immunity idea so we risk-assessed and decided to still make the trip, arriving in London very early on Monday 16th March.

It was a beautiful sunny & crisp day in Hyde Park as we killed time before checking into our hotel, La Suite West (with its very own vegan restaurant!). Exploring the local area again, Notting Hill was noticeably quiet but most shops, cafes and restaurants were open and it felt pretty much like “business as usual”. We were really tired by mid-afternoon and opted for a very early dinner at the excellent farm-to-table vegan eatery, Farmacy.

Back at the hotel by 5pm, we tuned in to the first of Boris’s daily missives to the people of the UK – and this is when everything really started to change. In this first speech, he announced that pubs and other places of social gathering would be stopped, the real start of social distancing in the UK.

We decided to avoid all forms of public transport for the rest of our time in London and so explored locally on foot on Tuesday. The Design Museum, housed in an amazing building in Kensington, was an excellent start and then we took in part of the glorious Victoria & Albert Museum, thinking we could come back to explore some more in the days ahead. There were quite a few people in the museums but their vast size made it feel safe in terms of keeping well apart from each other. We headed to the Brasserie at Cloud Twelve in Notting Hill for dinner at about 5pm, a lovely quiet and relaxing organic vegan eatery. We were warmly welcomed with very friendly service – we later found out we were their first customers for the day. We left with a pile of free vegan cakes, walking back through the very quiet local streets to our hotel for the next installment from Boris – this time finding out, among other things, that all the museums would now close, so much for our V&A revisit! If the Melbourne Cup is the race that stops the nation in Australia, Boris’s daily speeches quickly became those that stopped the UK as people tuned in to hear of the latest disruptions coming to their daily lives.

It was another sunny day on Wednesday so we kicked the day off with a long walk in Hyde Park around the Serpentine. There were plenty of people out exercising but respecting the social distancing rules. Heading to Kensington High Street, most of the shops along this usually busy street were closed but Wholefoods was doing good business and was a little too crowded for comfort but we stocked up on some supplies there while we had the chance. We had a relaxing coffee stop over at Cloud Twelve again (still very quiet and remarkable that it was still open) before walking back to the hotel along Westbourne Grove, which was eerily quiet but for a few food shops staying open (including the excellent Planet Organic where we gathered more supplies). We noted that Farmacy had signage to indicate even more reduced opening hours (closing at 5pm) so we made sure to head back there for a very early dinner at about 4pm, enjoying another excellent meal. It was almost empty  and we enjoyed nice conversations with the manager as he sought to work out what to do for the best in terms of staying open or closing up indefinitely. A post-dinner stroll in Hyde Park at dusk revealed many others making the most of the fresh air but the vastness of the park made social distancing very easy.

By now, it was clear day-by-day that more and more facilities and events were being cancelled around us and Boris’s daily updates clearly indicated stronger and stronger restrictions on what could stay open as well as limits on personal movements. All of the events we had planned – a Status Quo tribute band gig in a pub, Francis Rossi speaking in Aberystwyth and the “Only Fools and Horses” musical in the West End – were already cancelled and the postponement of our friend’s wedding, though not officially made yet, seemed likely.

Based on what we were seeing and hearing, we decided to cancel the various touring around the country we had planned to do and instead extend our stay in my old University town of Aberystwyth in Wales until our scheduled return date (14th April). With lower population density and no confirmed cases, Aber seemed like a good spot to take safe haven in familiar surroundings – I had the good fortune to live there for seven years and have returned almost every year for the last twenty years too. Cancelling our existing hotels and so on was straightforward and we could extend our AirBnB stay in Aber easily as all of the host’s other bookings had already cancelled for months ahead.

Our last full day in London on Thursday was a more typical drizzly, damp cold affair but we still donned our walking shoes and enjoyed a stroll around Little Venice, an area we’d discovered some years ago on a previous trip to London. It was a quiet locale with only a few locals on the tow paths around the canals. A return to Hyde Park followed and what would be our last coffee stop at Cloud Twelve, remarkably staying open while almost all around it had already closed indefinitely. We also stocked up on some more supplies at Planet Organic on the way back to the hotel. Our area was much quieter than even the previous day. Dinner was again taken early at Farmacy before they closed at 5pm, this time closing indefinitely. The manager was clearly upset at having to stand down all of his staff at this great restaurant. A final dusk walk in Hyde Park rounded out our outdoor time in London and we packed up ready to leave on Friday. Restrictions on the numbers for social gatherings such as weddings confirmed that our friend’s wedding would need to be postponed.

We had already booked a hire car to pick up from Heathrow on Friday as part of our original itinerary, so we opted to spend the extra for a private taxi back to the airport rather than travelling by tube. Terminal 3 was almost deserted as we caught the empty Budget car hire shuttle bus to collect our car. It was a lovely clear day for a drive and the roads (especially around Heathrow itself) were really quiet, so it was a relatively easy, if long, drive to Aber via the M4, Severn Bridge, Abergavenny, Brecon, Builth Wells, Rhayader and Llangurig. It was late afternoon by the time we concluded our almost five hour drive but we were relieved to be in Aber and our AirBnB apartment was great. Its central location, full kitchen and strong wi-fi boded well for a comfortable (and, if need be, longer term) stay. We figured we had enough time to make it to Dragonfly Bistro, a nice vegetarian eatery near the Castle, so headed there for a coffee and ordered lots of takeaway food to help us see out the coming days. Boris was giving his latest speech while we waited for our takeaway food and it ironically included the news that cafes and restaurants would now also need to close (apart from those offering takeaway and/or delivery services).

It was a stunning clear and sunny Saturday as we ventured out early on a shopping run in Aber. Our first port of call was to the newly-opened vegan deli, Iwtopia. Yes, a vegan deli in Aber, who’d have ever thought it!? This would be the last day for Iwtopia for a while so we took the chance to strongly support this great new place and stocked up on a heap of vegan goodies. We enjoyed a very long chat with the lovely owner too, we hope she can survive the downtime and come back to continue offering this great selection of vegan food to the people of Aber. It was especially pleasing to see that lots of the products were local and actively promoting other small vegan businesses. We were a little surprised that the splendid Farmers Market was still on (and practicing good social distancing too) so we also grabbed some quality fresh organic veggies, yummy sourdough bread from Anuna Bakery (and chatted with the owner who relocated from Melbourne to rural Wales!) and interesting food from the ingeniously named vegetarian producer, Parsnipship. Next stop was an old favourite, the stalwart organic shop, the Treehouse, where we stocked up on more quality produce as well as taking advantage of their new bulk supplies store. We’d had a big morning of shopping and felt like we were in a good position to not have to shop again for a week or two. But, more importantly, we’d enjoyed some great conversations with people and felt warmly welcomed – that familiar warm embrace of Aberystwyth was still there for me all these years later. We returned to the Farmers Market to grab lunch from the all-vegan Renegade Kitchen food van and again had an enjoyable conversation with the owner operators while they prepared our food.

With shopping and lunch sorted, we could finally take the opportunity to enjoy a long walk along the Prom and South beach. A hefty storm the week before had deposited some of the beach up onto the Prom. Aber was surprisingly crowded, it felt more like a Bank Holiday weekend than a town in semi-lockdown, but the expansive Prom gave everyone room to spread out (though not all chose to do so). We supported Dragonfly Bistro by grabbing a takeaway coffee and bought a few interesting Welsh ciders from the new Bottle and Barrel bottle shop. As sunset approached, we walked the Prom again and watched the murmuration of starlings over the Pier, always an amazing sight! Settling into our apartment, we assembled a big vegan dinner from our day’s shopping haul.

Some of the beach dumped onto Aber's Prom during a recent stormaber5The murmuration of starlings over Aber pier

Aber turned on another clear and sunny day on Sunday, albeit chilly out of the sun in a cool wind. We drove out the short distance to Penparcau to climb Pen Dinas, with its wonderful views across all of Aber and the Cardigan Bay coastline. A few hardy souls were hiking in this area but again it was easy to keep our distance from others. Heading downhill, we relaxed at Tan-y-Blwch beach, before the climb back to our car. Lunch back at the apartment was followed by a visit to Dragonfly Bistro for a takeaway coffee – and we then learned that the owner had decided that this would be her last day of opening for the foreseeable future. Back at the apartment, we felt like the world was closing around us with the vegan deli and vegetarian restaurant now gone. It was lovely in the sun down on the Prom and we sat there for a couple of hours to again watch the free show put on by the thousands of starlings heading home for the night under the Pier. Dinner back in the apartment was followed by the news that our return flight with Cathay Pacific (on 14th April) had been cancelled and to await news of an alternative flight home. Little did we know then that this would be the start of a series of flight-related highs and lows in the week ahead.

Pen Dinas monumentLooking down to Tan-y-Bwlch beach from Pen DinasThe Ystwyth at Tan-y-BwlchA stunning view of South Beach and the Castle from Tan-y-Bwlch

In my next blog post, I’ll cover one of the most uncertain, stressful and expensive weeks of our lives.

Solitude

One of the joys of reading is the books you come across by accident. Reading a couple of Tim Wu’s excellent books (viz. “The Attention Merchants” and “The Master Switch”) led me to books on solitude, including “Solitude: In Pursuit of a Singular Life in a Crowded World” by Michael Harris.

It seemed timely to read on this topic, as I’ve been implementing a “digital declutter” after recently reading Digital Minimalism by Cal Newport. I’m fortunate to live in a beautiful and peaceful location so I’m being much more mindful of making the most of the spot to deliberate separate myself from technology sometimes and take in the simple pleasures of time spent watching the ocean and listening to the birds.

The inspiration for writing “Solitude” came from the author reading about Dr Edith Bone. Hers is a remarkable story (and worth reading about in itself) of seven years spent in solitary confinement.

A little reading – and a hero in Dr. Bone – had turned malaise into a mission. I wanted to become acquainted again with the still night, with my own hapless daydreaming, with the bare self I had (for how long?) been running from. I kept asking myself: why I am so afraid of my own quiet company? This book is the closest I’ve come to an answer.

Aligning closely with Wu’s work, Harris discusses the rise of social media and the “connectedness” it was designed to create. But we all know by now that the “likes” and sharing are highly addictive, triggering small but frequent dopamine hits. This has had a devastating impact on our ability to find solitude:

We’re given opportunities to practise being alone every day, almost every hour. Go on a drive. Sit on a lawn. Stick your phone in a drawer. Once we start looking, we find solitude is always just below the surface of things. I thought at first that solitude was a lost art. Now I know that’s too pretty a term, too soft a metaphor.

Solitude has become a resource.

Like all resources, it can be harvested and hoarded, taken up by powerful forces without permission or inquiry, and then transformed into private wealth, until the fields of empty space we once took for granted first dwindle, then disappear.

Harris goes on to ask the question: what is solitude for? He comes up with three answers: the formulation of fresh ideas, self-knowledge, and (paradoxically) bonding with others.

Taken together, these three ingredients build a rich interior life. It turns out that merely escaping crowds was never the point of solitude at all: rather, solitude is a resource – an ecological niche – inside of which these benefits can be reaped. And so it matters enormously when that resource is under attack.

Our modern, hyperconnected, “always on” world sees solitude under constant threat and it takes a determined effort to find it in our lives:

Our online crowds are so insistent, so omnipresent, that we must now actively elbow out the forces that encroach on solitude’s borders, or else forfeit to them a large portion of our mental landscape.

It turns out that some research has already been done around daydreaming. MRI scanning reveals that daydreaming “constitutes an intense and heterogeneous set of brain functions” and:

…this industrious activity plays out while the conscious mind remains utterly unaware of the work – so our thoughts (sometimes really great thoughts) emerge without our anticipation or understanding. They emerge from the blue. Daydreaming thoughts may look like “pointless fantasizing” or “complex planning” or “the generation of creative ideas”. But, whatever their utility, they arrive unbidden.

Einstein believed that “the daydreaming mind’s ability to link things is, in fact, our only path toward fresh ideas.” Harris describes his own attempts to daydream during a three-hour wander and he says of this experience:

I start to see time-devouring apps like Candy Crush as pacifiers for a culture unwilling or unable to experience a finer, adult form of leisure. We believed those who told us that the devil loves idle hands. And so we gave our hands over for safekeeping. We long for constant proof of our effectiveness, our accomplishments. And perhaps it’s this longing for proof, for glittering external validation, that makes our solitude so vulnerable to those who would harvest it.

The addictive nature of social media (see ludic loops) has seen us giving up what few moments of spare time we have:

To a media baron looking for short-term profits, a daydreaming mind must look like an awful waste. All that time and attention left to wander, directionless! Making use of the blank spaces in a person’s life – draining the well of reverie – has become one of the missions of modernity.

But we do need to break out of this cycle and, bizarrely, doing so is seen as an odd and disruptive thing to do (e.g. I see the disbelief every time I mention to someone that I’m not, and never have been, “on Facebook”):

Choosing a mental solitude, then, is a disruptive act, a true sabotage of the schemes of ludic loop engineers and social media barons. Choosing solitude is a gorgeous waste.

Harris then discusses how we’ve all become part of the crowd and true marks of individualism are being eroded as a result:

…today we need to safeguard our inner weirdo, seal it off and protect it from being buffeted. Learn an old torch song that nobody knows; read a musty out-of-print detective novel; photograph a honey-perfect sunset and show it to no-one. We may need to build new and stronger weirdo cocoons, in which to entertain our private selves. Beyond the sharing, the commenting, the constant thumbs-upping, beyond all that distracting gilt, there are stranger things to be loved.

Harris explores the impact that technologies like Google Maps have had on our ability to truly lose ourselves and wander freely in nature, activities that have historically yielded great insights but are much more difficult to achieve in our hyper-connected and increasingly urban lives. He goes on to look at reading and writing – and the socialization of those activities. Proust once defined reading as “that fruitful miracle of a communication in the midst of solitude” but even this is under threat:

But that solitary reading experience is now endangered, and so is the empathy it fosters. Our stories are going social. We can assume that, in thirty years, readers and writers will use platform technologies to constantly interact with and shape each other, for better or worse. Authors will enlist crowd-sourcing and artificial intelligence to help them write their stories.

In his final chapter, Harris tells the story of his seven-day experience of solitude in a cabin in the woods, offline and alone:

Near the end of this lonely week my thoughts stop floating so much and return to the problem of solitude in a digital culture. Only now, out on the meditative trail I’ve been hiking before and after my crackers-and-apple lunch, I’m thinking about it differently, more expansively. Things here call for wide lenses.

From this dirt vantage, all that clicking and sharing and liking and posting looks like a pile of iron shackles. We are the ones creating the content, yet we’re never compensated with anything but the tremulous, fast-evaporating pleasures that social grooming delivers. Validation and self-expression, we are told, are far greater prizes than the measly cash that flows upward to platform owners…. [these] systems we live by can expropriate no value from solitude, and so they abhor it.

I enjoyed reading this book, it’s written in a very approachable style with many personal anecdotes (which you may or may not find interesting in themselves). I took this read as a reminder to make room for “daydreaming”, be that looking out over the ocean or simply not pulling out my phone during a short tram ride. Nicholas Carr says it well in the Foreword of the book:

Solitude is refreshing. It strengthens memory, sharpens awareness, and spurs creativity. It makes us calmer, more attentive, clearer headed. Most important of all, it relieves the pressure of conformity. It gives us the space we need to discover the deepest sources of passion, enjoyment, and fulfillment in our lives. Being alone frees us to be ourselves – and that makes us better company when we rejoin the crowd.

I also recently read another book on the same topic, but given a much more serious treatment by Raymond Kethledge & Mike Erwin, in the shape of “Lead Yourself First: Inspiring Leadership Through Solitude” – I highly recommend this book.

“The Influence of Organizational Structure on Software Quality: An Empirical Case Study” (Microsoft Research and subsequent blogs)

A Microsoft Research paper from back in 2008 has recently been getting a lot of renewed attention after a blog post about it did the rounds on Twitter, Reddit, etc. The paper is titled “The Influence of Organizational Structure on Software Quality: An Empirical Case Study” and it looks at defining metrics to measure organizational complexity and whether those metrics are better at predicting “failure-proneness” of software modules (specifically, those comprising the Windows Vista operating system) than other metrics such as code complexity .

The authors end up defining eight such “organizational metrics”, as follows:

  • Number of engineers – “the absolute number of unique engineers who have touched a binary and are still employed by the company”. The claim here is that higher values for this metric result in lower quality.
  • Number of ex-engineers – similar to the first metric, but defined as “the total number of unique engineers who have touched a binary and have left the company as of the release date of the software system”. Again, higher values for this metric should result in lower quality.
  • Edit frequency – “the total number times the source code, that makes up the binary, was edited”. Again, the claim is that higher values for this metric suggest lower quality.
  • Depth of Master Ownership – “This metric (DMO) determines the level of ownership of the binary depending on the number of edits done. The organization level of the person whose reporting engineers perform more than 75% of the rolled up edits is deemed as the DMO.” Don’t ask me, read the paper for more on this one, but the idea is that the lower the level of ownership, the higher the quality.
  • Percentage of Org contributing to development – “The ratio of the number of people reporting at the DMO level owner relative to the Master owner org size.” Higher values of this metric are claimed to point to higher quality.
  • Level of Organizational Code Ownership – “the percent of edits from the organization that contains the binary owner or if there is no owner then the organization that made the majority of the edits to that binary.” Higher values of this metric are again claimed to point to higher quality.
  • Overall Organization Ownership – “the ratio of the percentage of people at the DMO level making edits to a binary relative to total engineers editing the binary.” Higher values of this metric are claimed to point to higher quality.
  • Organization Intersection Factor – “a measure of the number of different organizations that contribute greater than 10% of edits, as measured at the level of the overall org owners.” Low values of this metric indicate higher quality.

These metrics are then used in a statistical model to predict failure-proneness of the over 3,000 modules comprising the 50m+ lines of source code in Windows Vista. The results apparently indicated that this organizational structure model is better at predicting failure-proneness of a module than any of these more common models: code churn, code complexity, dependencies, code coverage, and pre-release bugs.

I guess this finding is sort of interesting, if not very surprising or indeed helpful.

One startling omission from this paper is what constitutes a “failure”. There are complicated statistical models built from these eight organizational metrics and comparisons made to other models (and really the differences in the predictive power between all of them are not exactly massive), but nowhere does the paper explain what a “failure” is. This seems like a big problem to me. I literally don’t know what they’re counting – which is maybe just a problem for me – but, much more significantly, I don’t know whether what the different models are counting are the same things (which would be a big deal in comparing the outputs from these models against one another).

Now, a lot has changed in our industry since 2008 in terms of the way we build, test and deploy software. In particular, agile ways of working are now commonplace and I imagine this has a significant organizational impact, so these organizational metrics might not offer as much value as they did when this research was undertaken (if indeed they did even then).

But, after reading this paper and the long discussions that have ensued online recently after it came back into the light, I can’t help but ask myself what value we get from becoming better at predicting which modules have “bugs” in them. On this, the paper says:

More generally, it is beneficial to obtain early estimates of software quality (e.g. failure-proneness) to help inform decisions on testing, code inspections, design rework, as well as financial costs associated with a delayed release.

I get the point they’re making here but the information provided by this organizational metric model is not very useful in informing such decisions, compared to, say, a coherent testing story revealed by exploratory testing. Suppose I predict that module X likely has bugs in it, then what? This data point tells me nothing in terms of where to look for issues or whether it’s worth my while to do so based on my mission to my stakeholders.

We spend a lot of time and effort in software development as a whole – and testing specifically – trying to put numbers against things – perhaps as a means of appearing more scientific or accurate. When faced with questions about quality, though, such measurements are problematic and I thank James Bach for his very timely blog post in which he encourages us to assess quality rather than measure it – I suggest that taking the time to read his blog post is time better spent than trying to make sense of over-complicated and meaningless pseudo-science such as that presented in the paper I’ve reviewed here.

(The original 11-page MS Research paper can be found at https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/tr-2008-11.pdf)

2019 in review

It’s almost unbelievable that it’s time to close out my blogging for the year already! I published 13 blog posts during 2019, right on my target cadence of a post per month but down in number from 2017 and 2018. In terms of traffic, my blog attracted a very similar number of views to 2018 and I closed out the year with 1,000 followers on Twitter for the first time.

If there are particular topics you’d like to see me talking about here (especially to encourage more new readers), please feel free to reach out.

Working at Quest

I reached a milestone during 2019, notching up twenty years at Quest! It’s been an amazing journey since I started here in 1999 as a new migrant from the UK to Australia and I continue to enjoy a varied role working with dedicated people around the world. I travelled extensively again during the year and visited our folks in China, Austin (Texas) and the Czech Republic. The regular opportunities to travel and work with people from different cultures remains one of the most enjoyable (and sometimes most challenging!) aspects of my role.

lee20

I spent more time through 2019 helping teams to improve their agility, while still assisting widely around testing. As Quest modernizes both in terms of its products (e.g. new SaaS offerings) and processes, there is plenty to keep me busy helping the teams to deal with the different demands of more frequent delivery.

Conferences & meetups

I had another quieter year in terms of conference and meetup attendance. While I didn’t speak at a conference in 2019, I was lucky enough to co-organize the Association for Software Testing‘s third Australian conference, Testing in Context Conference Australia 2019 (TiCCA19). Working with Paul Seaman, we put together an excellent programme and the fifty-or-so delegates gave very positive feedback on what we offered. Although we had hoped to continue the TiCCA event as an annual conference, our small delegate numbers and ongoing challenges in attracting sponsorship unfortunately made it impossible for us to commit to the continuation of the event. It’s sad that we couldn’t build a sustainable true context-driven testing conference in a city as large as Melbourne, but Paul and I are happy to have tried hard with both CASTx18 and TiCCA19 providing great content for our local community.

The only other conference I attended was a non-IT event and something very different in many ways, the Animal Activists Forum in Melbourne. I contrasted the experience of attending this conference against the typical testing/IT conferences I’ve attended in my blog post, A very different conference experience.

I made it to a couple of meetups, the first being a pre-conference meetup we organized around TiCCA19. This meetup was enjoyable to organize and attend, featuring an excellent presentation by Aaron Hodder and a panel session with four TiCCA19 conference speakers – in the shape of Graeme Harvey, Aaron, Sam Connelly and Ben Simo – ably facilitated by Rich Robinson. The second meetup I attended was one of the high-quality Software Art Thou? series and saw the UK’s Kevlin Henney talking on “What do you mean?” (which he quickly modified to “WTF do you mean?”).

Community work

It was disappointing to learn that EPIC Assist had decided to pull out of the Melbourne market during 2019, resulting in the end of the software testing training course Paul Seaman and I had been delivering through them, the EPIC TestAbility Academy.

We would still love to share our knowledge and experience of software testing (and IT more generally) in a community setting and we continue to look for a partner organization to make this happen.

Other stuff

I’ve found myself reading a lot more books during 2019, a very welcome return to something I really enjoy and a useful way to reduce screen time (yes, I’m a physical book reader!). Many of the books came from the library and we are blessed with an excellent service in Melbourne (they purchased a number of books I requested through the year). Some of the books were purchased and shared with others in my office. I didn’t read testing books per se, but I became very interested in the subject of algorithms, AI and so on, reading a number of books in this area. Other areas of focus were leadership and knowledge acquisition.

I’ve also been spending more time to educate myself around animal rights and veganism, plus contributing in small ways to animal rights advocacy. It’s been an interesting change of tack to read books on these topics and also to see the reactions to my posts, tweets, etc. when this is the subject matter rather than my usual content! A handy summary of my thoughts around some of this can be found in my post, What becoming vegan taught me about software testing.

I hit another milestone early in 2019 when I acquired my first smartphone! I still find the form factor challenging and it seems unlikely I’ll ever become addicted to my phone, but I admit that it can be very handy when out and about – and Google Maps on the go during our travels made life a lot easier (though I was surprised offline maps don’t work in China, not a huge issue as we don’t drive there and taxis are incredibly cheap).

It felt like I had a much heavier workload during 2019 as well as some hefty stints of travel, so my outside projects didn’t get as much attention as in the previous few years. But I was glad to have the opportunity to organize the TiCCA19 conference as well as turning some work travel commitments into enjoyable holidays to see some new and interesting places. This time last year I was hinting at a new (personal) testing-related project that I hoped to kick off in 2019 and, while this didn’t eventuate, the project is still alive and I fully expect to get it up and running in 2020!

Thanks to my readers here and also followers on other platforms, I wish you all a very Merry Christmas & Happy New Year, and I hope you enjoy my posts to come through 2020. (And, remember, please let me know if there are any topics you particular want me to express opinions on, I’m happy to take suggestions!)