Category Archives: Books

“Changing Times – Quality for Humans in a Digital Age” (Rich Rogers)

It’s always good to see someone from our community of software testers taking the plunge to write their first book, as Rich Rogers has recently done with “Changing Times – Quality for Humans in a Digital Age” (available in both paperback and electronic formats).

The book is written in an easy to read style, with a story about a journalist called Kim running throughout. Her daily engagements with technology – as both positive and negative experiences – are described as she goes about her work and personal life. The story line makes it very easy to relate to the topics and Rich follows each chapter of her story with an exploration of its themes around quality and technology. It is these regular dives into the quality aspects of Kim’s experiences that makes his narrative so engaging and easy to consume.

Rich uses a model called the “Three Dimensions of Quality” and illustrates each dimension again by reference to Kim’s experiences. The three dimensions are Desirable, Dependable and Durable and he identifies a number of aspects within each dimension for further exploration (for example, “Dependable” is broken down into accurate, available, clear, private, protected, reactive, stable and tolerant).

For testers, I think this is a worthwhile read as it draws everything back to thinking about quality. But the book makes for a very enjoyable read for a much broader audience, from those with no real experience of the “nuts and bolts” of producing IT systems to anyone with an interest in “quality” and how we can improve it in the software we help to build. Well done, Rich!

The No Asshole Rule (Robert Sutton)

One of the few joys of long haul travel for business is time to browse that staple of airports everywhere, the bookshop. During a few hour stopover at Dallas Fort Worth airport recently, a couple of new paperbacks ended up in my hand luggage ready to help with the sixteen hour flight back to Australia. Though neither of the books actually made an appearance during the flight, I’ve managed to get through one of them since I got home, in the shape of The No Asshole Rule by Robert Sutton.

The author makes a distinction between “temporary assholes ” (people who are having a bad day or a bad moment) and “certified assholes” (persistently nasty and destructive jerks) and details the kinds of behaviours and damage done by them (not only to their direct victims, but also to bystanders, themselves and their organizations). He recommends implementing a “no asshole” rule and enforcing it, by “linking big policies to small decencies” (e.g. hiring and firing policies).

Tips for surviving nasty people and workplaces are also provided here: look for small wins, limit exposure, build pockets of safety, support & sanity, and fight & win the right small battles.

Robert also acknowledges the virtues of assholes, with Steve Jobs being used as a classic example of motivating fear-driven performance and perfectionism. These virtues are dangerous though given that the “weight of evidence shows that assholes, especially certified assholes, do far more harm than good”.

He also encourages us to look at ourselves and encourages us to find ways to “stop your “inner jerk” getting out”. I liked this mantra: “be slow to label others as assholes, but quick to label yourself”.

A couple of quotes sum up most of what this book is all about for me:

We all die in the end, and despite whatever “rational” virtues assholes may enjoy, I prefer to avoid spending my days working with mean-spirited jerks and will continue to question why so many of us tolerate, justify, and glorify so much demeaning behaviour from so many people

We are all given only so many hours here on Earth. Wouldn’t it be wonderful if we could travel through our lives without encountering people who bring us down with their demeaning remarks and actions?

This book is aimed at weeding out those folks and at teaching them when they stripped others of their esteem and dignity. If you are truly tired of living in Jerk City – if you don’t want every day to feel like a walk down Asshole Avenue – well, it’s your job to help build and shape a civilized workplace. Sure, you already know that. But isn’t it time to do something about it?

I’ve only worked for two employers in my twenty-odd years in the IT business and, after having read some of the stories in this book, I consider myself pretty lucky not to have encountered too much in the way of asshole behaviour. There have been a number of “temporary assholes” along the way, but I can only think of two “certified assholes” that have unfortunately crossed my path.

One was a manager who definitely went on the certification course and was only removed after a group of people were brave enough to “out” their awful behaviour (sadly, this person continues to be a people manager in a different company.) The other was a developer on a team for which I was the only tester and he let it be known that if I raised another bug against his work, he’d be waiting for me in the car park to exact his revenge. At the time, this was both amusing and of course somewhat frightening – and management did a good job of making sure he wasn’t with us too much longer.

While my own experiences are overwhelmingly positive in the IT industry, it’s obvious that many people (especially females) have a really hard time and at least some of the terrible behaviours are being publicly called out (e.g. the recent Uber stories). Closer to home, my good friend Paul Seaman recently wrote a blog post, The Standard You Walk Past , in which he clearly details the actions of what Sutton would deem a “certified asshole”.

We all deserve a safe and comfortable workplace, so it’s contingent on us to call out asshole behaviour whenever we see it (and that includes anyone who works with me!). This simple statement from the book says it all: “treat the person right in front of you, right now, in the right way”. That’s something we can all do to help make our workplaces and the world at large just that little bit better for everyone.

Creativity and testing

I’ve just finished reading Scott Berkun’s new book, The Dance of the Possible – “The Mostly Honest, Completely Irreverent Guide to Creativity”. As with his previous books, it makes for easy reading and he makes his points clearly and honestly. I read this book based on enjoying a couple of his other works – in the shapes of Confessions of a Public Speaker and Ghost of my Father – and wasn’t anticipating the amount of testing-related goodness I found in his new one!

In just the second chapter, Scott tackles the tricky topic of where to begin when starting a piece of creative work. He talks about taking an exploratory approach:

The primary goal when you’re starting creative work is to explore, and to explore demands you do things where you are not sure of the outcome. There will be false starts, twists, turns and pivots. These should be welcomed as natural parts of the experience, rather than resisted as mistakes or failures.

Exploratory testing, anyone?!  One of the joys of taking a session-based exploratory testing approach in my experience is the uncertainty of what information we’ll learn about our product in each session – this is so much more rewarding for the tester than knowing they’ll just report a “pass” or “fail” at the end of following a test case, for example.

As Scott moves on to methods for finding ideas (in chapter 4), one of my favourite tools for test planning and reporting makes an appearance:

Another approach to finding interesting combinations is called a mind map. On a large piece of paper write your main goal, subject or idea down in the center and circle it. Then think of an attribute, or an idea, related to the main one and write it down, drawing a line back to the main idea. Then think of another and another, connecting each one to any previous idea that seems most related.

Keep drawing lines and making associations. Soon you’ll have a page full of circles and lines capturing different ways to think about your main thought.

Exploratory testing puts the onus on the tester to come up with test ideas and this seems to be one of the biggest challenges for testers moving from a scripted approach, “how will I know what to test?” The skill of coming up with test ideas is one that requires practice and mind maps are a great way to both organize those ideas and give the tester a way to visualize their ideas, start to combine (or separate) them, and so on.

In chapter 5, Scott talks about creative projects being “a dance between two forces, expanding to consider more ideas and shrinking to narrow things down enough to finish” and how this very idea can be challenging for those who focus on efficiency:

Looking back on a finished project, you might think the time spent exploring ideas that didn’t get used was wasted. It’s easy to believe you should have known from the beginning which ideas would work best and which wouldn’t. This is an illusion. Creativity is exploration. You are going into the unknown on purpose. You can only learn about ideas as you develop them, and there’s no reliable predictor of which ones will pay off and which ones won’t. Certainly the more conservative you are in the ideas you pick, the more predictable the process will be, but by being more conservative you are likely being less creative and will discover fewer insights. Arguably the more creations you make the better your intuition gets, but you won’t find any successful creator, even the legends, who gets it right all the time.

People obsessed with efficiency have a hard time accepting this truth. They would also have a very hard time being at sea with Magellan, working with Edison in his lab or with Frida Kahlo in her art studio. They’d be stunned to see the “waste” of prototypes and sketches, and mystified by how many days Magellan had to spend at sea without discovering a single thing. Discovery is never efficient.

I see this “dance” a lot and it’s the natural tester dance between doing more testing (“consider more ideas”) and calling “good enough” (“narrow things down enough to finish”). I think these words from Scott are a great way to express the benefit of more creative testing in helping us better assess risks in our products:

Certainly the more conservative you are in the ideas you pick, the more predictable the process will be, but by being more conservative you are likely being less creative and will discover fewer insights.

Scott’s new book was a fairly short and enjoyable read. I always say that testing is a creative endeavour – seeking to counter the idea that testing is repetitive, boring work whenever I come across it – so Scott’s words on creativity will be a handy reference in this regard.

Testers, how good is your waggle dance?

Since returning from Europe, I’ve been enjoying a new commuting option in the shape of a ferry service across the bay and this relaxing 90-minute trip is proving to be a great alternative to my only previous option of a 30km drive plus one-hour train journey.

Cruising to Melbourne in this way has opened up some more time for reading and I’m just finishing The Wisdom of Crowds by James Surowiecki.

The book explores a simple idea: “large groups of people are smarter than an elite few, now matter how brilliant – better at solving problems, fostering innovation, coming to wise decisions, even predicting the future.”

It’s enjoyable stuff and, of course, I can’t help but draw connections between some of its content and software testing.

The following paragraphs from the book describe how bees go about finding good food sources for their hive (emphasis is mine):

Bees are remarkably efficient at finding food. According to Thomas Seeley, author of “The Wisdom of the Hive”, a typical bee colony can search six or more kilometres from the hive, and if there is a flower patch within two kilometres of the hive, the bees have a better-than half chance of finding it. How do the bees do this? They don’t sit around and have a collective discussion about where foragers should go. Instead, the hive sends out a host of scout bees to search the surrounding area. When a scout bee has found a nectar source that seems strong, he comes back and does a waggle dance, the intensity of which is shaped, in some way, by the excellence of the nectar supply at the site. The waggle dance attracts other forager bees, which follow the first forager, while foragers who have found less-good sites attract fewer followers and, in some cases, eventually abandon their sites entirely. The result is that bee foragers end up distributing themselves across different nectar sources in an almost perfect fashion, meaning that they get as much food as possible relative to the time and energy they put into searching. It is a collectively brilliant solution to the colony’s food problem.

What’s important, though, is the way the colony gets to that collectively intelligent solution. It does not get there by first rationally considering all the alternatives and then determining an ideal foraging pattern. It can’t do this, because it doesn’t have any idea what the possible alternatives – that is, where the different flower patches – are. So instead, it sends out scouts in many different directions and trusts that at least one of them will find the best patch, return, and do a good dance so that the hive will know where the food source is.

I immediately saw similarities with exploratory testing when I read this.

When we’re looking to identify interesting or risky areas of a product under test, our initial charters are quite loose, since we don’t necessarily have a good idea of where to look yet. Debriefing our sessions gives us the chance to narrow in on where to look next or where to return to as fertile ground for finding interesting information about the product.

So, as a tester returning information to your team, how good is your waggle dance?

(The Wisdom of Crowds is an interesting read – and not just for the story of the waggle dance!)

Reading matter & Simple Rules

I see two very common patterns among testers when it comes to their reading habits – most have never read a book dedicated to software testing and even fewer have thought to read books in other fields (such as psychology and social sciences) that would help them understand the problems of performing great testing. This is an amazing state of affairs for “professionals”, especially when there is so much excellent material just waiting to be read, absorbed and applied from good practitioners in both our field and related ones.

So, if you haven’t read any software testing books, I suggest you do so – something like Lessons Learned in Software Testing: A Context-Driven Approach (Cem Kaner, James Bach, Brett Pettichord) or Perfect Software: And Other Illusions about Testing (Jerry Weinberg) would be pretty good places to start. But also think about some broader reading, with Thinking Fast and Slow (Daniel Kahneman) and Tacit and Explicit Knowledge (Harry Collins) being regularly cited as helpful texts for testers to broaden their horizons.

As a case in point, I recently stumbled across the book Simple Rules: How to Thrive in a Complex World (by Donald Sull and Kathleen M. Eisenhardt) as it came on my wife’s radar in the trading world and she thought it might be interesting in relation to software testing too.

 

The book recognizes the complexity of modern life and how simple rules can help us to cut through some of that complexity, rather than coming up with increasingly complicated solutions.

For most of us, complexity is a problem we struggle to manage in our own lives every day… By limiting the number of guidelines, simple rules help maintain a strict focus on what matters most while remaining easy to remember and use. In a wide range of decisions, simple rules can guide choice while leaving ample room to exercise judgement and creativity.

I liked the idea that simplification leaves room for judgement and creativity, this resonating closely with the benefits I see in using exploratory testing (over detailed scripted tests).

Fighting complexity is an ongoing battle that can wear us down. Disheartened, people tolerate complicated solutions that don’t work, or cling to overly simplistic narratives… that deny the interdependencies characterizing modern life.

Again, I immediately thought of how the world of context-driven testing acknowledges this complexity and interdependency, but doesn’t try to come up with one size fits all hefty processes or standards to address these realities.

The authors talk about four key aspects of simple rules:

First off, simple rules consist of a handful of guidelines applied to a specific activity or decision… They’re intended to offer a limited amount of guidance, so there’s no need for a lot of them. Keeping the number of rules to a handful forces you to focus on what matters most. You might think that capping the number of rules would result in guidelines that are too simplistic to solve complex problems. Not so. In many situations, a handful of factors matter a great deal, while a long tail of peripheral variables can be safely ignored.

Second, simple rules are tailored to the situations of the particular people who will use them, versus one-size-fits-all rules that apply to everyone.

Third, simple rules are applied to a single well-defined activity or decision… Simple rules are most effective when they apply to critical activities or decisions that represent bottlenecks to accomplishing an important goal.

Finally, simple rules give concrete guidance without being overly prescriptive… Simple rules leave room to exercise creativity and pursue unanticipated opportunities.

They include some interesting examples of where such simple rules have been successful in battling complexity and resulted in good decision making, without being so overly prescriptive as to be useless or ignored.

Simple rules work best when flexibility matters more than consistency… Both flexibility and consistency have their advantages, but increasing one reduces the other. Then a large number of highly directive rules… are the best tools to use. Detailed rules are particularly useful for avoiding catastrophic errors, such as plane crashes, mishaps in nuclear power plants, and surgical deaths that result from known causes. Pilots are fond of saying that “checklists are written in blood”, a reference to how these lists are developed in the first place.

Checklists are a valuable tool in testing, just as they are for doctors and pilots. But flexibility is also very important in testing, acknowledging that there is much we don’t know about a system (despite whatever level of specification you might have) until we start to explore its behaviour.

Simple rules impose a threshold level of structure while avoiding the rigidity that results from too many restrictions. The resulting flexibility makes it easier to adapt to changing circumstances and seize fleeting opportunities. Simple rules can also produce better decisions than more complicated models can, particularly when time and information are limited.

Have you ever been in a situation during software development (including testing) where time and information were limited?! I like the idea of seizing “fleeting opportunities” as might be revealed during exploratory testing.

In the context-driven testing world, you will hear the term heuristics a lot and, for those of us who are practitioners, using heuristics turns out to be very powerful in our testing. The authors of this book also recognize the benefits of “rules of thumb”:

Simple rules enable people to make quick, reasonably accurate decisions that require less effort than more complicated approaches. When there is not much time or when information is at a minimum, these rules of thumb can save the day. Simple rules work because they focus on key aspects of a decision while ignoring peripheral considerations. By using simple rules, people can function without constantly stopping to rethink every aspect of a decision every time they make it.

Rules of thumb are often viewed a second-rate measures for use when people lack the time or information to come to a more considered judgement. Indeed, the term rule of thumb refers to a rough and practical approach that is not particularly accurate or reliable in every situation…

Counterintuitive as it may sound, simple rules can outperform more analytically complicated and information-intensive approaches even when there is ample time and information to make a decision. This is especially true in situations where links between cause and effect are poorly understood, when important variables are highly correlated, when a few factors matter most, and when a gap exists between knowing what to do and actually doing it. Simple rules do not trump complicated models every time, but they do so more often than you might think.

Heuristics are a powerful decision-making tool, often matching or even outperforming more sophisticated approaches. They are easy to remember and use, attributes that increase the odds that people will not only make the right choice, but translate their decision into action and stick with it over time.

The case for using simple rules seems pretty strong and I’ve certainly found heuristics to be a powerful means of generating test ideas during exploratory testing.

The authors ask the obvious question:

 

 

These rules also raise an important question: if simple rules are so effective in so many situations, why do complex solutions remain so prevalent? Regulators churn out ever more detailed rules, personnel departments promulgate thick policy manuals, and self-styled experts promote ever more arcane diet and exercise regimes. People crave simplicity and… simple rules often outperform more complicated approaches. Why aren’t simple rules even more common? What are the obstacles to simplifying our lives, corporations and societies? And, more importantly, how can we overcome these obstacles to achieve simplicity?

While most testers I’ve met are keen to simplify the way they work, they are often caught up in highly bureaucratic procedures and heavyweight documentation that result in less time spent actually testing than they would prefer.

The first obstacle is the effort required to develop simple rules. Like most worthwhile endeavours, it takes time and energy to get them right. The process of developing simple rules requires ruthless prioritization – honing in on the essential and decluttering the peripheral… The payoffs of simplification often dwarf the costs of getting there.

The people who benefit from complexity pose a second obstacle to simplicity. The costs of complex solutions are distributed across many people while the benefits of complexity tend to be concentrated in the hands of a few. These beneficiaries have, as a consequence, strong incentives to resist simplification. Much of the complexity of the US tax code, for example, exists because special interest groups secure tax breaks… that benefit a small number of individuals. These special interest groups obviously benefit from complexity, but so do the lobbyists who make their case to legislators, as well as the lawmakers themselves. After creating a labyrinth of rules, regulators and politicians often walk through the revolving door to join the companies they formally supervised.

As in many other industries, the software testing industry has seen the rise of certifications and standards that appear to resist simplicity and generate lots of work for the bodies overseeing them.

The third obstacle to simplicity is what we call the “myth of requisite complexity,” the mistaken belief that complex problems demand complicated solutions. There are, naturally, situations when complicated solutions are appropriate (e.g. detailed checklists used by pilots and surgeons). But detailed rules and regulations aren’t the only possible way to deal with complexity… Complicated solutions should be a considered choice, not the result of regulatory autopilot.

Often, complex rules and regulations arise out of a distrust of human nature. If people cannot be trusted to do the right thing, detailed regulations are necessary to prevent malfeasance. Many corporations, for example, rely on thick policy manuals to control people who might abuse their discretion. But these bad apples represent a tiny fraction of all employees.” Talking about Netflix as an example, “the company’s policy for expenses, travel, gifts, and conducting personal business at work… was reduced to four rules: (1) expense what you would not otherwise spend, (2) travel as if it were your own money, (3) disclose nontrivial gifts from vendors, and (4) do personal stuff at work when it is inefficient not to.

There are other good examples in the book of where simple solutions actually work, where complex problems don’t necessary demand complicated solutions. Good software testing is a complex problem (as we all know!) but we can tackle this problem in simple ways that add genuine value, rather than always resorting to complicated approaches, processes or standards of how to go about our work.

As they come to close out their book, the authors say:

..Simple rules work because they provide a threshold level of structure while leaving ample scope to exercise discretion. Complex rules, in contrast, attempt to anticipate every contingency and dictate what to do in each scenario, thereby reducing people to automatons who do what they are told. But human discretion is not a defect to be eliminated, it is our greatest hope in the battle against complexity. Close to the facts on the ground, individuals can draw on their judgement and creativity to manage risks and seize unexpected opportunities. The latitude to exercise discretion not only makes simple rules effective, it makes them attractive. People thrive given the opportunity to apply their judgement and creativity to the situations they face day to day. And if they benefit from simple rules, they are more likely to use them and use them well.

These are all great arguments for the use of more exploratory approaches to testing, where the tester is in control of their own destiny, “human discretion is not a defect to be eliminated”!

I really enjoyed this book and (as I’ve summarized above) saw many parallels with the world of software testing. It’s great to see a book that isn’t anything to do with software testing providing such insight and applicable ideas. Reading this also gave me hope that simplicity can be seen as a positive thing and that pragmatic approaches to software testing (for which I am a strong advocate) have the ability to add huge value to software development projects.