Finding the Intractable Bug

It doesn't happen too frequently. A lot of times it's a slow-build of customer complaints. The product is unstable. It keeps crashing. It slows down. It's freaking out.

But you just can't reproduce it. How can you get traction on an intractable bug?
Museums Victoria
You've got to start somewhere, and try to find a reproducible problem. Maybe it's not the whole problem, but some part of it. Review the reports, maybe even contact the original users and try to focus your efforts on specific places in the UI or the workflow.

You don't need to find the whole "unstable" problem. For example, let's imagine that the UI displays an unexpected screen. Just reproduce that. Just reproduce it in one place. You can start at screen X, under certain conditions, and try to go to screen Y, and end up somewhere else. If you have to write 1000 words of "here's how you get set up to reproduce this issue" then so be it, but find one thing you can reproduce reliably.

You don't really want to file a bug like "Product is unstable" anyway. How will developers fix that? How will you know if they succeeded?

Usually there are a number of root causes that are all interacting to give the impression of an "unstable" or "slow" system or some other intractable issue. Break them down, find things that are reproducible and get the engineers working on them.

Once the engineers can identify the part of the code or the system where your reproducible problem is occurring, they can not only fix the specific issue you've reported, but also review the nearby code for any obvious improvements that can be made.

After you have fixed a couple of these very narrow issues, and worked with your engineering team to "harden" the product a bit, you will probably find that the larger vague problem reports start to settle back down, and you have just "leveled up" in your own troubleshooting.

Comments

Post a Comment

Popular posts from this blog

The Boeing 737 MAX, a QA perspective

Image
Thanks to: Wikipedia.org user Acefitt The Boeing 737 MAX and its MCAS system (which tries to emulate the handling characteristics of earlier generations of the product) have been in the news a lot lately. Investigations are still ongoing, but the product is being blamed by many for two crashes which have resulted in the loss of almost 340 lives. That's a big quality problem. Clearly, the pressures on the Boeing are immense. The worldwide commercial aircraft market is over $191 billion , and dominated by cut-throat competition between Boeing and Airbus. The 737 has been in production since 1967, and if you are a pilot who was qualified to fly it in 1967, you still are today , despite vast numbers of changes to the aircraft. This is because Boeing has managed to keep the aircraft handling similar enough over time that the FAA (and other worldwide governmental authorities) has been willing to allow the "type certification" of pilots to be valid for all variants of t...
Read more

The risks of the "Golden" sample

Image
It's very tempting, and if you've ever worked with overseas contract manufacturers (CMs) they will almost prefer to use a "Golden Sample" instead of a detailed specification. Coyau /  Wikimedia Commons  /  CC BY-SA 3.0 In hardware manufacturing, it's easier to understand. I freely admit I've written incoming parts inspection instructions that don't have every detail of the part! Every dimension, the depth of any indentations, the gauge of any round protuberances, the color of all the materials, the hardness, even the chemical composition (important for fasteners and structural parts). If you simply take a prototype or early unit from the manufacturing line, give it a thorough inspection, and declare it to be a "Golden Sample," it's a heck of a shortcut, saving a lot of *your* time as the person who would otherwise have to write the documentation! There are risks, however, and you need to think about them, minimize them, and try to m...
Read more

Applicability of Economics to QA

Image
Years ago, I started listening to Russ Roberts' excellent podcast EconTalk ( http://www.econtalk.org ), and the more I've listened to it, the more I've come to understand that although the origins of economics and the study of quality may be different, the practice is very, very similar. What do we measure? We face a constant struggle trying to measure the right variables, and what is "right" changes over time. In economics, it seems like new indicators become popular all the time. After the Great Depression, the concept of a nation's Gross Domestic Product rose in popularity as economists worked on trying to explain how nations were affected by the depression, and how they might be able to avoid similar events in the future. However, GDP did nothing to predict the "Great Recession" of 2007-2009, and in fact trailed the crisis by a considerable time. We face similar issues with quality. Production or build test failure rates come to us p...
Read more