2022 in review

It feels like much less than a year since I was penning my review of 2021, but the calendar doesn’t lie so it really is time to take the opportunity to review my 2022.

I published just 10 blog posts this year, so didn’t quite meet my personal target cadence of a post every month. There were a few reasons for this, the main one being my unexpected re-entry into employment (more on that below). Perhaps due to my more limited output, my blog traffic dropped by about 40% compared to 2021. I continue to be grateful for the amplification of my blog posts via their regular inclusion in lists such as 5Blogs and Software Testing Weekly.

March was the biggest month for my blog by far this year, thanks to a popular post about a video detailing how testers should fake experience to secure roles. I note in writing this blog post now that the video in question has been removed from YouTube, but no doubt there are similar videos doing the rounds that encourage inexperienced testers to cheat and misrepresent themselves – to the detriment of both themselves and the reputation of our industry.

I again published a critique of an industry report in November (after publishing similar critiques in 2020 and 2021) and this was my second most popular post of the year, so it’s good to see the considerable effort that goes into these critique-style posts being rewarded by good engagement.

I closed out the year with about 1,200 followers on Twitter, steady year on year, but maybe everyone will leave Twitter soon if the outrage many are expressing recently isn’t fake!

Work life

For the first few months of 2022, I continued doing a small amount of consulting work through my own business, Dr Lee Consulting. It was good to work directly with clients to help solve testing challenges and I was encouraged by their positive feedback.

Quite unexpectedly, an ex-colleague from my days at Quest persuaded me to interview at SSW, the consultancy he joined after Quest. A lunch with the CEO and some formalities quickly led to an offer to become SSW’s first Test Practice Lead (on a permanent part-time basis). I’ve now been with SSW for about seven months and it’s certainly been an interesting journey so far!

The environment is quite different from Quest. Firstly, SSW is a consultancy rather than a product company and I’ve come to realise how different the approach is in the consulting world compared to the product world. Secondly, SSW is a small Australian company compared to Quest being a large international one, so meetings are all standard working hours (and I certainly don’t miss the very early and very late meetings that so frequently formed part of my Quest working day!).

I have been warmly welcomed across SSW and I’m spreading the word on good testing internally, as well as working directly with some of SSW’s clients to improve their approaches to testing and quality management.

Testing-related events

As I announced mid-2021, I was excited to be part of the programme for the in-person Testing Talks 2021 (The Reunion) conference in Melbourne, rescheduled for October 2022. Unfortunately, I had to give up my spot on the programme due to my COVID vaccination status – though, surprise surprise, all such restrictions had been removed by the time the event actually took place. But I did attend the conference and it was awesome to see so many people in the one place for a testing event, after the hiatus thanks to the pandemic and the incredibly harsh restrictions that resulted for Melbourne. (I blogged about my experience of attending Testing Talks 2022.)

In terms of virtual events, I was fortunate to be invited to act as a peer advisor for one of Michael Bolton’s virtual RST classes running in the Australian timezone. This was an awesome three-day experience and I enjoyed interacting with the students as well as sharpening my understanding of some of the RST concepts from Michael’s current version of the class.

Two very enjoyable virtual events came courtesy of the Association for Software Testing (AST) and their Lean Coffees. I participated in the May and September events suited to my timezone and they were enlightening and fun, as well as offering a great way to engage with other testers in an informal online setting.

I had an enjoyable conversation with James Bach too, forming part of his “Testing Voices” series on the Rapid Software Testing YouTube channel:

Although I’ve interacted with James online and also in person several times (especially during his visits to Melbourne), this was our most in-depth conversation to date and it was fun to talk about my journey into testing, my love of mathematics and my approach to testing. I appreciate James’s continued passion for testing and, in particular, his desire to move the craft forward.

Testing books

I didn’t publish an updated version of my book An Exploration of Testers during 2022, but may do in 2023.  I’m always open to additional contributions to this book, so please contact me if you’re interested in telling your story via the answers to the questions posed in the book!

I made good progress on the free AST e-book, Navigating the World as a Context-Driven Tester though. This book provides responses to common questions and statements about testing from a context-driven perspective, with its content being crowdsourced from the membership of the AST and the broader testing community. I added a further 10 responses in 2022, bringing the total to 16. I will continue to ask for contributions about once a month in 2023. The book is available from the AST’s GitHub.

Podcasting

Paul Seaman, Toby Thompson and I kicked off The 3 Amigos of Testing podcast in 2021 and produced three episodes in that first year, but we failed to reconvene to produce more content in 2022. There were a number of reasons for this, but we did get together to work up our next episode recently, so expect our next podcast instalment to drop in early 2023!

Volunteering for the UK Vegan Society

I’ve continued to volunteer with the UK’s Vegan Society both as a proofreader and also contributing to their web research efforts. I’ve learned a lot about SEO as a result of the web-related tasks and I undertook an interesting research project on membership/join pages to help the Society to improve its pages around joining with the aim of increasing new memberships.

I really enjoy working with The Vegan Society, increasing my contribution to and engagement with the vegan community worldwide. It was particularly rewarding and humbling to be awarded “Volunteer of the Season” and be featured in the Society’s member magazine, The Vegan, towards the end of the year.

In closing

As always, I’m grateful for the attention of my readers here and also followers on other platforms. I wish you all a Happy New Year and I hope you enjoy my posts and other contributions to the testing community to come through 2023 – the first public opportunity to engage with me in 2023 will be the AST’s Steel Yourselves webinar on January 30, when I’ll be arguing the case for a testing phase, I hope to “see you” there!

A tester’s critique of “The 2021 State of Software Quality: The View from Enterprise Leaders & Followers”

I really should know better, but I decided to watch a webinar titled The 2021 State of Software Quality: The View from Enterprise Leaders & Followers from MicroFocus and Enterprise Management Associates, Inc. The promo spiel for the webinar read as follows:

The rapid rise of the digital economy became twice as important after layering on a worldwide pandemic. With every company having to become a software company, enterprise application development speed, volume, cost, quality, and risk are key determinants that define who survives and who does not. The pressure on application development teams to build more software faster and cheaper often runs counter to the objectives of software quality and managing risk.

Join Steve Hendrick, Research Director at Enterprise Management Associates, to hear key findings from a recent worldwide survey about software quality. This webinar will look at the characteristics and differences between software quality leaders and followers. Key to this discussion of software quality is the impact that people, process, and products are having on enterprise software quality. Completing this view into software quality will be a discussion of best and worst practices and their differences across three levels of software quality leadership.

While the opening gambit of this promo literally makes no sense – “The rapid rise of the digital economy became twice as important after layering on a worldwide pandemic” – the webinar sounded like it at least held some promise in terms of identifying differences between those “leading” in software quality and those “following”.

The survey data presented in this webinar was formed from 316 responses by Directors, VPs and C-level executives of larger enterprises (2000 employees or more). The presenter noted specifically that the mean enterprise size in the survey was over 11,000 employees and that this was a good thing, since larger enterprises have a “more complex take on DevOps”. This focus on garnering responses from people far away from the actual work of developing software in very large enterprises immediately makes me suspicious of the value of the responses for practitioners.

Unusually for surveys and reports of this type, though, the webinar started in earnest with a slide titled “What is Software Quality”:

While the three broad software quality attributes seem to me to represent some dimensions of quality, they don’t answer the question of what the survey means when it refers to “software quality”. If this was the definition given in the survey to guide participants, then it feels like their responses are likely skewed to thinking solely about these three dimensions and not the many more that are familiar to those of us with a broader perspective aligned with, for example, Jerry Weinberg’s definition of quality as “Value to some person”.

The next slide was particularly important as it introduced the segmentation of respondents into Outliers, Laggards, Mainstreamers and Leaders based on their self-assessment of the quality of their products:

This “leadership segmentation” is the foundation for the analysis throughout the rest of the webinar, yet it is completely based on self-assessment! Note that over half (55%) self-assess their quality as 8/10, 9/10 or even 10/10, while only 11% rate themselves as 5/10 or below. This looks like a classic example of cognitive bias and illusory superiority. This poor basis for the segmentation used so heavily in the analysis which follows is troubling.

Moving on, imagine being faced with answering this question: “How does your enterprise balance the contribution to software quality that is made by people, policy, processes, and products (development and DevOps tools)?” You might need to read that again. The survey responses came back as follows:

Call me cynical but this almost impossible to answer question looks like it resulted in most people just giving equal weight to all of the five choices, so ending up with just about 20% in each category.

It was soon time to look to “agile methodologies” for clues as to how “adopting” agile relates to quality leadership segmentation:

It was noted here that the “leaders” (again, remember this is respondents self-assessing themselves as quality leaders) were most likely to represent enterprises in which “Nearly all teams are using agile methods”. A reminder that correlation does not imply causation feels in order at this point.

The revelations kept coming, let’s look at the “phases” in which enterprises are “measuring quality”:

The presenter made a big deal here about the “leaders” showing much higher scores for measuring quality in the requirements and testing management “phases” than the “mainstreamers” and “laggards”. Of course, this provided the perfect opportunity to propagate the “cost of change” curve nonsense, with the presenter claiming it is “many times more expensive to resolve defects found in production than found during development”. He also sagely suggested that the leaders’ focus on requirements management and testing was part of their “secret sauce”.

When the surveyed enterprises were asked about their “software quality journey over the last two years”, the results looked like this:

The conclusion here was that “leaders” are establishing centres of excellence for software quality. There was a question about this during the short Q&A at the end of the deck presentation, asking what such a function actually does, to which the presenter said a CoE is “A good way to jumpstart an enterprise thinking about quality, it elevates the importance of quality in the enterprise” and “raises visibility of the fact that software quality is important”. An interesting but overlooked part of the data on this slide in my opinion is that about 20% of enterprises (even the “leaders”) said that their “focus on agile and DevOps has not had any impact on software quality”. I assume this data didn’t fit the narrative for this highly DevOps-focused webinar.

Attention then turned to tooling, firstly looking at development tools:

I find it interesting that all of these different types of development tooling are considered “DevOps tools” and it’s surprising that only around half of the “laggards” even claim to use source code management tools (it’s not clear why “mainstreamers” were left off this slide) and only just over half of the “leaders” are using continuous integration tools. These statistics seem contrary to the idea that even the leaders are really mature in their use of tooling around DevOps. (It’s also worth noting that there is also considerable wiggle room in the wording of this question, “regularly used or will be used”.) Deployment, rather than development, tooling was also analyzed but I didn’t spot anything interesting there, apart from the very fine-grained breakdown of tooling types (resulting in an incredible 19 different categories).

The presenter then examined why software quality was improving:

Notice that the slide is titled “Why has your software quality been improving since 2019?” while the actual survey question was “Why has your approach to software quality improved since the beginning of 2019?” Improvements in approach may or may not result in quality improvements. Some of the choices for response to this question don’t really answer the question, but clearly the idea was to suggest that adding more DevOps process and tooling leads to quality improvements while the data suggests otherwise (more around business drivers).

Moving from the “why” to the “how” came next (again with the same subtle difference between the slide title and the survey question):

There are again business/customer drivers behind most of these responses, but increased automation and use of tooling also show up highly. A standout is the “leaders” highlighting that “our multifunctional teams have learned how to work more effectively together” was a way to improve quality.

Some realizations/revelations about quality followed:

There were at least signs here of enterprises accepting that improving quality takes significant effort, not just from additional testing and tooling, but also from management and the business. The presenter focused on the idea of “shifting left” and there was a question on this during the Q&A too, asking “how important is shift left?” to which the presenter said it was “very important to leaders, it’s a best practice and it makes intuitive sense”. But he also noted that there was an additional finding in the deeper data around this that enterprises found it to be a “challenge in piling more responsibility on developers, made it harder for developers to get their job done, it alienates them and gets them bogged down with activities that are not coding” and that enterprises were sensitive to these concerns. From that response, it doesn’t sound to me like the “leaders” have really grasped the concept of “shift left” as I understand it and are still not viewing some types of testing as being part of developers’ responsibilities. The final entry on this slide also stood out to me (but was not highlighted by the presenter), with 17% of the “leaders” saying that “software quality is a problem if it is too high”, interesting!

Presentations like this usually end up talking about best practices and this webinar was no different:

The presenter focused on the high rating given by the “leaders” “adoption of quality standards (such as ISO)” but overlooked what I took as one of the few positives from any of the data in the webinar, namely that adopting “a more comprehensive approach to software testing” was a practice generally seen as something worth continuing to do.

The deck wrapped up with a summary of the “Best Practices of Software Quality Leaders”:

These don’t strike me as actually being best practices, rather statements and dubious conclusions drawn from the survey data. Point 4 on this slide – “Embracing agile and improving your DevOps practice will improve your software quality” – was highlighted (of course) but is seriously problematic. Remember the self-assessed “leaders” claimed that their software quality was increasing due to expanding their “DevOps processes and toolchain”, but correlation does not imply causation as this point on this final slide implies. This apparent causality was reinforced by the presenter’s answer to a question during the Q&A also, when asked “what is one thing we can do to improve quality?”. He said his preference is to understand the impact that software quality has on the business, but his pragmatic answer is to “take stock of your DevOps practice and look for ways to improve it, since maturing your DevOps practice improves quality.”

There were so many issues for me with the methodology behind the data presented in this webinar. The self-assessment of software quality produced by these enterprises makes the foundation for all of the conclusions drawn from the survey data very shaky in my opinion. The same enterprises who probably over-rated themselves on quality are also likely to have over-rated themselves in other areas (which appears to be the case throughout). There is also evidence of mistakenly taking correlation to imply causation, e.g. suggesting that adding more DevOps process and tooling improves quality. (Even claiming correlation is dubious given the self-assessment problem underneath all the data.)

There’s really not much to take away from the results of this survey for me in helping to understand what differences in approach, process, practice, tooling, etc. might lead to higher quality outcomes. I’m not at all surprised or disappointed in feeling this way, as my expectations of such fluffy marketing-led surveys are very low (based on experiencing of critiquing a number of them over the last few years). What does disappoint me is not the “state of software quality” supposedly evidenced by such surveys, but rather the state of the quality of dialogue and critical thinking around testing and quality in our industry.

The webinar can be viewed from https://content.microfocus.com/optimize-devops-tb/2021-software-quality (note that registration is required).

A year has gone…

Almost unbelievably, it’s now been a year since I left my long stint at Quest Software. It’s been a very different year for me than any of the previous 25-or-so spent in full-time employment in the IT industry. The continuing impact of COVID-19 on day-to-day life in my part of the world has also made for an unusual 12 months in many ways.

While I haven’t missed working at Quest as much as I expected, I’ve missed the people I had the chance to work with for so long in Melbourne and I’ve also missed my opportunities to spend time with the teams in China that I’d built up such a strong relationship with over the last few years (and who, sadly, have all since departed Quest as well as their operations there were closed down this year).

I’ve deliberately stayed fairly engaged with the testing community during this time, including giving a talk at at meetup, publishing my first testing book, launching my own testing consultancy business, and blogging regularly (including a ten-part blog series answering the most common search engine questions around testing).

Starting to work with my first clients in a consulting capacity is an interesting experience with a lot of learning opportunities. I plan to blog on some of my lessons learned from these early engagements later in the year.

Another fun and testing-related project kicked off in May, working with my good friends from the industry, Paul Seaman and Toby Thompson, to start The 3 Amigos of Testing podcast. We’ve always caught up regularly to chat about testing and life in general over a cold one or two, and this new podcast has given us plenty of opportunities to talk testing again, albeit virtually. A new episode of this podcast should drop very soon after this blog post.

On more personal notes, I’ve certainly been finding more time for myself since ending full-time employment. There are some non-negotiables, such as daily one-hour (or more) walks and meditation practice, and I’ve also been prioritizing bike riding and yoga practice. I’ve been reading a lot too – more than a book a week – on a wide variety of different topics. These valuable times away from technology are foundational in helping me to live with much more ease than in the past.

I’ve continued to do volunteer work with The Vegan Society (UK). I started off performing proofreading tasks and have also now joined their web volunteers’ team where I’ve been leading research projects on how to reduce the carbon footprint of the Society’s website and also to improve its accessibility. These web research projects have given me the welcome opportunity to learn about areas that I was not very familiar with before, the “green website” work being particularly interesting and it has inspired me to pursue other opportunities in this area (watch this space!). A massive proofreading task led to the recent publication of the awesome Planting Value in the Food System reports, with some deep research and great ideas for transitioning UK farming away from animal-based agriculture.

Looking to the rest of 2021, the only firm commitment I have in the testing space – outside of consulting work – is an in-person conference talk at Testing Talks 2021 in Melbourne. I’ll be continuing with my considerable volunteering commitment with the Vegan Society and I have a big Status Quo project in the works too! With little to no prospect of long-distance travel in Australia or overseas in this timeframe, we will enjoy short breaks locally between lockdowns and also press on with various renovation projects on our little beach house.

(Given the title of this blog, I can’t waste this opportunity to include a link to one of my favourite Status Quo songs, “A Year” – this powerful ballad morphs into a heavier piece towards the end, providing some light amongst the heaviness of its parent album, “Piledriver”. Enjoy!)

Launching my testing consultancy, Dr Lee Consulting

As I mentioned in a recent blog post, I’ve been working on setting up my own software testing consultancy following my exit from Quest back in August.

I finally got all my ducks in a row and launched the business – Dr Lee Consulting – publicly on 21st October 2020!

I spent a few weeks focusing on business basics and refining the idea of what my consultancy should look like. I completed two of Pat Flynn‘s excellent courses in the process, viz. Will It Fly? (the book and companion course) and Smart From Scratch. Mindmaps were my friend during this ideation and refinement stage, and my thanks go to those connections I reached out to along the way for their valuable help and feedback.

Another great source of inspiration and ideas was the “Share What You Know Summit” run as a virtual event by Teach:able (on 22-24 September). This was an excellent three-day event and furnished me with some great tips around LinkedIn profile tweaks and ideas for content generation.

In terms of administrivia, I registered for an ABN and business name online, then purchased the corresponding domain name (business name and domain name need to closely match for a .com.au domain, unlike most other domain extensions). I built my website using a free WordPress site, mainly due to my familiarity with their platform after blogging there for many years. I’ll probably upgrade to a paid plan sometime soon to remove ads and allow me to more professionally map my domain to that site.

I feel like the time I invested in the ideation and refinement of the idea was well spent and I tried not to go overboard in perfecting my website – at some point you just need to pull the trigger and get the thing out there!

My aim has always been to share what I’ve learned about testing with other organizations and Dr Lee Consulting is now my vehicle to do this. While I realize I’m not the right fit for every organization, I hope there are organizations/teams out there who will see the value in my services – I’m ready and waiting to help!

Check out www.drleeconsulting.com.au for full details of my offering and how to contact me for a no obligation conversation about engaging my services.

Pre-launch announcements for my new projects

After six weeks or so of resetting following my unplanned exit from Quest, I’m getting close to publicly announcing more details on a couple of new projects.

One of these has been in the making for about a year, while the other has arisen as a direct result of leaving full-time employment.

I’ve always been drawn to the idea of writing a book and I will finally realize this idea with the release of a testing-related e-book very soon. It’s been a highly collaborative effort with input from many members of the testing community. Having more free time since finishing up at Quest has given me the opportunity to wrap up what I think is worthy of publishing as a first edition. I will return all proceeds from sales of this book to the testing community. Look out for more details of the book via this blog and my social media presences in the coming weeks!

My other project is a new boutique software testing consultancy business. The intention is to offer something quite different in the consulting space, utilizing my skills and experience from the last twenty years to help organizations to improve their testing practices. This consultancy won’t suit everyone but I hope that my niche offering will both help those who see the value in the way I think about testing and also give me the chance to share my knowledge and experience in a meaningful way outside of full-time corporate employment. I expect to launch this business before the end of the year, but feel free to express interest in securing my services now if you believe that my thinking around software testing could be of value in your organization. Note that I will not be making myself available full-time (as I’m deliberately carving out time for volunteer work and to focus on my wellbeing), so now is a good time to secure some of my limited future availability before the formal launch of the consultancy. Again, keep an eye on this blog and my socials for more details of the testing consultancy project.

Decompression – phase 2

It’s already four weeks since my exit from Quest after some 21 years with the company and it’s amazing how quickly my “reset” is happening.

My calendar continues to be largely empty, apart from a number of video calls with friends both locally and overseas each week. My days are passing quickly and are filled with exercise (walking and cycling), well-being practices (meditation and yoga), reading, home/garden maintenance and relaxing. The welcome start of the Tour de France and the excellent daily highlights on SBS represent my only “TV time” during the day.

I’ve also been keeping abreast of what’s going on in the testing community and have a couple of small projects on the go – writing a guest blog post (details coming soon) and preparing a talk (“Testing is not Dead”) for the DDD Nights meetup on 10th September.

When my redundancy package payout appeared in the bank at the end of August, my ties to Quest finally came to an end. Reflecting on such an incredibly long time working in the same company, my main emotion is one of gratitude. Quest provided me with my first employment opportunity in Australia after migrating from the UK and, over the many years that followed, gave me manifold opportunities to grow professionally, contribute to the broader testing community, travel extensively, and build great relationships. I miss the banter with my good friends & colleagues and also my close collaborations with the wonderful folks over in our office in China, where I look back with great pride on the development programmes I helped to put in place and run over the years.

I continue to work on the testing-related project I mentioned in my previous blog post and expect to make an announcement on that in the next month or so, so stay tuned! I’ve also been taking part in an online business course and this may also lead to some news in terms of my next steps in the coming months.

Decompression – phase 1

It’s now been a couple of weeks since my 21-year career at Quest came to an abrupt end. My LinkedIn post about exiting Quest has by far the most engagement of all my contributions on that platform, with over 12,000 views and more than 100 reactions as I write. The irony of this fact is not lost on me.

I’ll blog specifically about my Quest exit later, but I’ve spent the time since then in “reset” mode, taking a step back from work and testing-related things in general.

An empty calendar is a real delight after so many years on the hamster wheel of frequent meetings. With my role having covered essentially every timezone, very early and very late meetings were the norm (although I’d become better at not accepting meetings starting before 7am or finishing after 11pm). Having long periods during the day without interruption from meetings has given me back opportunities for simple pleasures that have suffered greatly over the last decade (or more in some cases).

My wife and I have been practising meditation and yoga for several years but my daily practice has often failed to take precedence over the demands of work. It’s been great to put these important factors for my wellbeing back where they belong – as priorities.

Bike riding was another important and enjoyable pastime that I’ve neglected so it’s great to make a daily ride part of my routine (weather-permitting!). I’d forgotten how much I enjoy the feeling of freedom on two wheels.

I’ve also been reading a lot, some tech-related but plenty not (e.g. Status Quo gig history research, animal rights, vegan advocacy, some fiction).

It would have been nice to take the chance to travel for a while, but of course current COVID-19 restrictions make that impossible in this part of the world. But we’re at least thinking of where we might like to explore within Australia when our freedoms are returned to us.

I haven’t entirely disconnected from the testing world, though, and have invested some time into my LinkedIn profile (still work in progress) to ensure it more accurately reflects my experience and contributions in readiness for my next steps. I’ve also made some improvements to the configuration of this blog, so there is now an archive by month and I’ve categorized every post so hopefully it’s easier to find the stuff you’re interested in (and thanks to Paulo Lai for his feedback inspiring me to make these long overdue changes).

I’ve been working on a personal testing-related project for most of 2020 and have more time and energy to dedicate to it now, so it’s coming along nicely. I hope to release details of this project publicly very soon!

My experience of working from home full-time during the pandemic

Thanks to Twitter, I spotted that a new testing-related ebook had been published recently, titled Software People Work From Home, in which a number of testers from around the world describe their personal experiences of working from home thanks to coronavirus-induced restrictions.

I really enjoyed reading this (free) ebook and felt inspired to share some of my own experiences based on more than two months of working full-time at home for Quest.

Firstly, some context around my “normal” work situation. Although I am based in our Melbourne city office, I’ve been working from home for three to four days a week for the last five years or so. Part of the reason for this mode of working is due to the long commute from my home to the Melbourne office (around two hours door-to-door, each way) and another factor is the many early morning/late night meetings I’m involved with thanks to my collaboration with Engineering teams across basically every timezone!

This hybrid model has worked well for me and for Quest over the last few years. The reduced requirement to be within “normal” commuting distance of Melbourne means I can live in a beautiful location, right on the beach with peace and fresh air – this has made a huge difference to my lifestyle and well-being. We always make time during the day for a long walk (typically around 5km) and having this as part of my routine continues to be really important to me.

I intended to resume this model of working following my return from some international travel back in mid-March. (I’ve blogged about the experience of travelling during the pandemic separately here and here.) Of course, on returning from the UK towards the end of March, our office had already been closed until further notice thanks to COVID-19 – and so began my immediate transition to working from home full-time.

It’s been an interesting three months in this new mode of working. In many ways, I am very fortunate. Firstly, I’ve kept a full-time job on the same salary as pre-COVID so there are no additional financial pressures resulting from this change. I’ve also had plenty of practice at working from home over the last several years, so the adjustment to full-time at home hasn’t been as significant as for many people. I’m lucky enough to have a dedicated room in our (albeit very small) home to work from (and the incredible water view from there never gets old!) and we are a couple with no kids so it’s pretty straightforward to maintain a quiet environment in which to concentrate on work.

Even adding the extra day or two at home every week compared to my usual routine has revealed some additional benefits of the arrangement. I’m settling into a great circadian rhythm (save for a few early morning meetings) and not commuting has freed up some more time to enjoy relaxing at home. We’re cooking together more often too.

While it’s generally a positive for me to work from home all the time, there are challenges too, the most significant of which for me is avoiding overworking. I spotted a great quote on Twitter recently:

Stop calling it ‘working from home’ and start calling it ‘living at work’

Heather De-Quincey

My boundaries between “work” and “not work” are – and never have been – very strict. I have access to all of the systems folks within Quest might use to contact me pretty much all of the time. My role operates across our entire business unit, which has people in timezones spanning the whole world.

Shortly after lockdowns became part of almost everyone’s life experience, we decided to convert what would have been a large in-person meeting in California into a virtual event. While it was a great success, organizing and running a 70+ person meeting across so many timezones was a huge effort by many people – I worked sixteen-hour days for three days straight during the event, resulting in severe over-tiredness (and unwelcome grumpiness on the home front).

There is always someone looking for something from me, so it can be hard to ignore those requests even when they’re out of what most would think of as “business hours”. The early morning and late night meetings necessitated particularly by the need to interact with folks in the US are draining, but I’ve learned to speak up more and refuse meetings before 7am or after 11pm so as to allow for a reasonably consistent sleep pattern.

There are some things I am missing as a result of not just working from home all the time, but also the general world situation thanks to the pandemic. I was looking forward to attending the CAST conference in Austin in August, but that of course has been cancelled. This is also the first time in recent memory that I haven’t had an overseas trip (or two or three!) in planning, either for work or leisure (or sometimes a combination of both). It feels strange not having these adventures to look forward to anytime soon.

Closer to home, my day or two up in Melbourne affords me a number of opportunities and benefits that I’d taken for granted. I miss my weekly coffee catch-ups with Paul Seaman, although we maintain close virtual contact. I make very good use of the Melbourne Library Service and usually have an interesting book or two on the go, but there’s been no opportunity to borrow books recently. I also miss the chance to have lunch or coffee with ex-colleagues, something I do almost every time I travel into the city. And, of course, the kitchen banter with my Quest colleagues is sadly lacking – and Teams meetings are just not the same!

Overall, I’m enjoying the experience of working from home full-time and the downsides certainly don’t outweigh the positives of both my work output and lifestyle. With Quest keeping its offices closed until at least September 2020, I’ll get to enjoy it for a while longer.

The Melbourne testing market and spotting “red flags”

One of the more popular job listing sites in Australia is Seek. A search on this site for Melbourne-based jobs under the Information & Communication Technology -> Testing & Quality Assurance classification yielded 70 results as I began writing this post on 29th May 2020.

Looking at these jobs, I’ve broadly categorized them as follows:

• Automation (32)
  • Automation/SDET
  • Test Automation Engineer (x2)
  • Automation Test Consultant (x2)
  • Senior QA Automation Engineer
  • Test Automation Specialist (x2)
  • Automation Test Analyst (x5)
  • Automation Test Analyst – Tosca
  • Senior Automation Test Analyst
  • QA Engineer (Automation)
  • Automation Tester (x4)
  • Senior Test Automation Engineer/Lead (x2)
  • Java Development Engineer in Test (DevOps)
  • Lead/Sr.Automation Tester
  • Senior Test Automation Specialist
  • Automation Test Engineer
  • Automation Test Engineer (Embedded Software)
  • Applications Testing Automation Engineers (x4)
  • Technical Test Lead – Automation Lead Tosca
• Management (7)
  • Test Manager
  • Performance Test Manager (x2)
  • Defect Manager (x4)
• Specialists (14)
  • Performance Tester (x4)
  • Penetration Testing
  • Performance Test Consultant
  • Infrastructure and Security Test Analyst
  • Senior Test Analyst – Infrastructure testing
  • Performance Test Analyst (x2)
  • Performance Engineer (x2)
  • Jmeter Tester
  • Network Test Engineer (hardware-related)
• “Manual” / Other (17)
  • Test Analyst
  • Test Analyst – Business Intelligence
  • Senior Test Analyst (x2)
  • UAT Tester
  • Mobile QA Engineer
  • Quality Assurance Engineer
  • QA Engineer
  • Graduate Test Analyst
  • Automation Specialists and Defect Managers
  • Junior Quality Assurance Tester Traineeship
  • Senior Software Test / V&V Engineers (Defence)
  • Validation & Verification Lead
  • Integration Testers (x4)

These ads are not representative of unique jobs, as exactly the same ad is sometimes posted at different times and/or the same job is typically posted by a number of different recruiters (especially for government roles).

The breakdown in terms of the number of opportunities didn’t surprise me. The focus at the moment seems to be around automation driven by CI/CD investments, DevOps/agile transformations – and, of course, the overestimation of what automation can and can’t do. Similarly, performance and security-related testing are topics de jour as represented by a swathe of ads in these areas. Test management doesn’t seem like a good place to be, with very few roles being advertised in recent times and this change has been heavily driven by agile adoption in my experience.

I generally take more interest in the ads focused on human testing to see what companies are looking for in this area. Most of the more “traditional” human testing roles (e.g. in government departments) also now mandate some degree of proficiency in tools commonly associated with “automated testing”. It’s pleasing to see requests for ISTQB certification becoming much less common and I’ll occasionally even spot a reference to “exploratory testing”.

But there are often “red flags” in these ads and I proffer a few examples from this most recent search of the “opportunities” on offer.

First up, a “Senior Test Analyst” role “to take accountability for the testing capabilities within the Microservices team.” The “Technical experience” requirements are listed first and include “Linux/Unix Shell Scripting, Java Programming, Clean Coding, Git, Jenkins2, Gradle, Docker, REST APIs, SQL, Unit Testing (Junit), Component Testing, BDD, Integration Testing” and then finally the “Testing Practices” requirements are presented, viz. “Exploratory Testing, Mind Mapping, Requirements Analysis, Peer Collaboration, Continuous Integration, Continuous Deployment, Stubbing and Mocking”.

There are a few red flags here for me. If this were a true microservice team, then it would have a clear sense of ownership of its microservice and a whole team approach to the testing and quality of that service. I’d be looking for clarification of what “accountability for the testing capabilities” really means in the context of this team. Another issue is the lack of clarity about whether this is a human testing role or a development (automation code) role, or a hybrid, or something else. The fact that the technical (mainly development) skills requirements are listed before the testing ones would immediately lead me to believe that more value is placed on automation than on deep human testing here. While it’s good to see exploratory testing explicitly listed (albeit as a “testing practice”), the other requirements listed around testing are much less convincing as to whether this organization would truly value the contribution of an excellent exploratory tester.

Next up, a “Quality Assurance Engineer” role where the successful applicant (who becomes just a “Quality Engineer” as the ad goes on) will “play a critical role in transforming quality practices… and work to develop a world-class test automation solution. [They will] work as part of a cross functional squad, acting as the QA expert, across new features, enhancements and bug fixes. [They’ll use their] testing experience to advise on automation opportunities and review requirements to ensure a fit for purpose solution in (sic) delivered.”

In describing what the day-to-day work entails for this role, there are some positive signs: “You’ll be passionate about testing and use your experience to identify critical defects. You’ll be naturally curious and explore the latest tools and techniques to continuously improve testing” But there are also some less positive signs: “You’ll work as a fully engaged member of cross functional squad including Developers, UX/UI Designers and Product Managers to ensure the quality of products. You’ll create, maintain and execute tests (manual or automated) to verify adherence to requirements.” Again, it’s good to see exploratory testing getting a mention, albeit in what comes across as a confused way (especially in light of the “verify adherence to requirements” elsewhere), “You’ll perform risk based exploratory testing.”

In terms of skills, they’re expecting “expertise in developing functional tests that cover 100% of requirements and execute manually or automated (with focus on least maintenance and faster script development time)” and possession of “strong skills on efficient test design”.

There are obviously a few red flags in this ad. It sounds like this organization has latched into the so-called “Spotify Model” since it mentions “squads” a number of times, even though this model was actually not successfully adopted within Spotify. It talks about “ensuring” quality and verifying “adherence to requirements”, while at the same time asking for exploratory testing skills (of the risk-based variety, of course). Covering “100% of requirements” completes a picture of an organization where verification, preferably by machine, is valued much more than human testing.

My final example is for a “QA Engineer” role which asks for background in “both manual and automated testing” The red flags come real early in this one: “Can you imagine how it would feel to be responsible for ensuring that our key products work well in all conditions? Are you interested in working in a Global Centre of Excellence…?” I’ll choose not to imagine how it would feel to be given an impossible mission.

This lucky candidate “will be responsible for both manual and automated testing as we move towards complete automation.” At least there is no doubt left here about the value of deep human testing as this organization seeks to “automate everything.”

I like the requirement to be “Excellent at finding the problems others miss”, but much less keen to see this followed by “Able to document test cases based on solution requirements provided. Understanding of Developing in an Agile environment”. Advanced-level “ISTBQ” is then included in their list of “desirable” skills.

I don’t understand why this organization believes this ad would attract an excellent human tester who could leverage exploratory testing to genuinely find “the problems others miss”. They describe themselves as Agile but want the successful candidate to write test cases, all the time trying to get to a point where “manual” testing is no longer required. Based on the ad alone, either this is an organization with a confused outlook on testing or they’re really looking for someone to take on an impossible mission while devaluing their actual testing skills – either way, this doesn’t sound like a great way for a decent tester to spend their working life (but I acknowledge the ad could just be very poorly-written and misrepresents the core beliefs of the organization when it comes to testing).

While it’s interesting to review occasionally what organizations are looking for when they’re advertising for testing-related roles, it’s also pretty depressing to see just how little value seems to now be placed on deep human testing skill. The “automate everything” bandwagon seems to roll on and is taking over this market, while opportunities for those genuinely skilled in testing as a craft seem to become fewer and fewer, at least in the Melbourne market represented by published job ads.

As the economic impact of COVID-19 takes its toll, a large number of folks are hitting the IT market at the same time. In just the last couple of weeks, large IT staff reductions have been reported in Melbourne by big tech names such as MYOB and Culture Amp – and more seem likely in the coming months. If there’s a bright side to this situation, it’s that there are no doubt lots of good people coming back into the market so it’s probably a great chance to pick up testing talent for those organisations able to do so. If you’re representing such an organisation and really want to hire skilled human testers, please take the time to construct better tester job ads! Making it sound like you understand the value of testing is likely to attract those really good testers who might have come back into the market thanks to COVID-19.

Remember that a good tester should be a strong critical thinker and will be testing your ad, so catch your own “red flags” before they do!

Two decades at Quest Software

Today (2nd August 2019) marks twenty years since I first sat down at a desk at Quest Software in Melbourne as a “Senior Tester”.

I’d migrated from the UK just a few weeks earlier and arrived in Australia in the middle of the late 90s tech boom. The local broadsheet newspaper, The Age, had a separate section once a week which was a hefty tome and packed full of IT jobs. I sent my CV to many different recruitment companies advertising in the newspaper and started to get some interest. My scatter gun approach was a response to the lack of opportunities for LISP developers (my previous skill from three years as a developer back in the UK, working on expert systems for IBM) but I did focus a little on openings for technical writers, believing I could string words together pretty well and had a decent grasp of technology.

One of the first interviews I secured for such a technical writing position was for a company I’d never heard of, Quest Software out in the Eastern suburbs of Melbourne (Ashburton, at that time). After some hasty company research, I remember catching a train there and following the recruiter’s directions to “take the staircase next to the bottle shop” to locate the Quest office (actually, one of two offices in the same street due to recent expansion). My interview would be with the head of the technical writing team and we started off with a chat over coffee in the kitchen. I didn’t even realize this was the interview, it was so relaxed and welcoming! At the end of the coffee/interview, he asked whether I’d also like to chat with the head of the testing team as she was looking for people too, so of course I took the opportunity to do so. This was again a very informal chat and I left the office with a technical writing task to complete. After completing the task, I was soon contacted to return to the Quest office to further my application for a software testing position, but not the technical writing one. A test case writing task formed part of this next slightly more formal interview, my first attempt at writing such a document! It was very shortly afterwards that the recruiter let me know I had an offer of a role as a “Senior Tester” and I couldn’t return the required paperwork fast enough – I’d found my first job in Australia!

I considered myself very fortunate to have secured a position so quickly after arriving into Australia. I was certainly lucky to find a great recruiter, Keith Phillips from Natural Solutions, and I recall visiting him in person for the first time after the deal was done with Quest, down at his office in South Melbourne. It turned out we had a common connection to the University of Wales in Aberystwyth, where I studied for both my undergraduate and doctoral degrees. We also studied in the same department (Mathematics) and, although Keith’s studies were some years before mine, many of the same department staff were still around during my time there as well. I believe Keith is still in the recruitment industry and I have fond memories of his kind, professional and unhurried approach to his work, something not common during my experiences with recruiters back then.

Back to 2nd August, 1999, then and my first day at the Quest office in Ashburton. Amidst the dotcom madness, Quest were growing rapidly and I was just one of many new starters coming through the door every week. We were sitting two to a desk for a while until we moved to bigger new digs in Camberwell, about three months after I joined. We grew rapidly and I enjoyed my time as a tester, slotting in well to a couple of different development teams and learning the ropes from other testers in the office. Being new to the testing game, I didn’t realize that we had a very “traditional” approach to testing in Quest at that time – I was part of an independent testing team under a Test Manager and spent a lot of my time writing and executing test cases, and producing lots of documentation (thanks, Rational Unified Process).

I was also learning the ropes of living in a new country and I’m indebted to my colleagues at the time for their patience and help in many aspects of me settling into a Melbourne life!

I worked across a few teams in my role as a “Senior Tester” from 1999 until 2004 when I was promoted to a “Test Team Lead” and given people management responsibility for the first time, leading a small group of testers as well as retaining hands-on testing commitments. I realize now that I was a classic “process cop” and quality gate fanatic, persisting with the very traditional ideas around testing and test management. This was an interesting and challenging time for me and, while I enjoyed some aspects of managing people, it was also not the most enjoyable aspect of my job.

It was during my time as test lead that Quest ran the Rapid Software Testing course in-house with Michael Bolton, in our Ottawa office in 2007. It was a very long way to travel to attend this course, but it was truly career-changing for me and opened my eyes to a new world of what testing was for and how it could be done differently. I returned to work in Melbourne inspired to change the way we thought about testing at Quest and took every chance I could to spread the word about the great new ideas I’d been exposed to. Looking back on it now, I banged this drum pretty hard and was probably quite annoying – but challenging the status quo seemed like the right thing to do.

During a shift to adopting Scrum within some of the Melbourne teams and a move away from the independent test team, I really saw an opportunity to bring in new testing ideas from Rapid Software Testing and so, in 2008, a new position was created to enable me to focus on doing so, viz. “Test Architect”. Evangelizing the new ideas and approaches across the Melbourne teams was the main job here and the removal of people management responsibility gave me a welcome chance to focus on effecting change in our testing approach. I enjoyed this new role very much over the next five years, during which time we moved to Southbank and Quest Software was acquired by Dell to form part of their new Software business.

My role expanded in 2013 to provide test architectural guidance across all of the worldwide Information Management group as “Principal Test Architect”. One of the great benefits of this promotion was the chance to work closely with colleagues in other parts of the world and I became a very regular visitor to our office in China, helping the talented and enthusiastic young testers there. I also started my conference presentation journey in 2014, a massive step outside my comfort zone! While attending a testing peer conference in Sydney in 2013, I was fortunate to meet Rob Sabourin (who was acting as content owner for the event) and he encouraged me to share my story (of implementing session-based exploratory testing with the teams in China) to a much wider audience, leading to my first conference talk at Let’s Test in Sweden the following year. This started a journey of giving conference talks all over the world, another great set of experiences and I appreciate the support I’ve had from Quest along the way in expanding the reach of my messages.

Dell sold off its software business in late 2016 and so I was again working for Quest but this time under its new owners, Francisco Partners.

My last promotion came in 2018, becoming “Director of Software Craft” to work across all of the Information Management business in helping to improve the way we develop, build and test our software. This continues to be both a challenging and rewarding role, in which I’m fortunate to work alongside supportive peers at the Director level as we strive for continuous improvement, not just in the way we test but the way we do software development.

My thanks go to the many great colleagues I’ve shared this journey with, some have gone onto other things, but a surprising number are still here with 20+ years of service. The chance to work with many of my colleagues on the ground across the world has been – and continues to be – a highlight of my job.

I’ve been fortunate to enjoy the support and encouragement of some excellent managers too, allowing me the freedom to grow, contribute to the testing community outside of Quest, and ultimately expand my purview across all of the Information Management business unit in my capacity as Director of Software Craft.

Little did I think on 2nd August 1999 that my first job in Australia would be the only one I’d know some twenty years later, but I consider myself very lucky to have found Quest and I’ve enjoyed learning & growing both personally & professionally alongside the company. My thanks to everyone along the way who’s made this two decade-long journey so memorable!