Code Blue!

How do you rank the severity of application defects? Some test teams assign severity/priority scores, but that’s arbitrary, and doesn’t reflect the real-world impact of bugs. How can you really assess the importance of something that’s rated “medium” for severity but “low” for priority?

More practical dev teams use expressions to communicate, through the defect database, change-management system, email or sticky notes, how important a defect is to the team. “This one’s a show stopper,” you might say. Or “If you can fix this before the next release, that would be great,” you might comment. Or “Who cares about a teeny-weeny typo?” you might write. Or “Sheesh, this one’s definitely gonna get us sued,” you might opine. Isn’t that better than “high,” “medium” and “low”?

However creative that approach is, the expression-based defect ranking system does leave things to interpretation. One person’s “it’s a teeny-weeny typo” is someone else’s “clean out your desk and be out of here before the cops come,” especially if that typo was in your CEO’s name, or in one of the digits in your upcoming Securities and Exchange Commission filing.

Similarly: While your CFO might issue a scathing four-letter expletive in both cases, which is worse: a bug that applies the wrong algorithms to stock-options pricing, or a bug that applies the wrong algorithms to a credit-scoring system? Selecting the “we’re totally screwed” button in the issue-defect system’s severity/priority may not accurately communicate the CFO’s displeasure.

The right solution, as I’m sure you will agree, is to hand these sorts of things to the U.S. Government. No, not the actually grading of your software’s bugs, you silly person: that’s your job, and you can’t get out of it. No, we should take a leaf from how our benevolent authorities have responded to airport security. You don’t hear the public address system at the airport say, “We’re at terror level we’re-going-to-die.” That would cause panic. Worse, it is ambiguous, since it doesn’t communicate who is going to die, when this will take place, and if you have time to buy a $5 Bloody Mary from a flight attendant beforehand.

Instead, as I’m sure you know, the monotone announcement on the P.A. system says something like, “Attention: We are at Homeland Security Threat Condition Orange.” This is more practical, and more useful, because it’s from the government and they know best.

That, my friends, is the model that software test/QA professionals should use when communicating with end users and other stakeholders about bugs, when assessing the bugs for ourselves, and when classifying said bugs in the defect database.

“Hey, Bob, looks like we’ve got a nice, juicy Yellow here,” you might hear shouted over a cubicle. “Are you sure it’s not Orange? We’re only fixing Oranges before the next beta,” you might shout back. And so-on and so-on.

The U.S. Government’s colorful Homeland Security Advisory System (HSAS) was enacted in March 2002. Forget about those silly “low,” “medium” and “high” scales that you see so often in defect-management systems: the HSAS goes much farther with five levels:

Red = Severe: Severe Risk of Terrorist Attacks
Orange = High: High Risk of Terrorist Attacks
Yellow = Elevated: Significant Risk of Terrorist Attacks
Blue = Guarded: General Risk of Terrorist Attacks
Green = Low: Low Risk of Terrorist Attacks

Brilliant, brilliant, I can hear you saying. Go ahead, say it again. Brilliant. Thank you. You can see instantly why this is appropriate for software development and test/QA. I would humbly propose the following scale for categorizing software threats. Actually, I would like to propose two scales, which I call the Software Defect Advisory System (SDAS). The first SDAS scale is the one that you tell your end users, managers and other stakeholders about, and which they use for reporting bugs to your test team:

Red = Severe: This Must Be Fixed Immediately
Orange = High: This Should Be Fixed Soon
Yellow = Elevated: Fix This When You Can
Blue = Guarded: Fix This Sometime, Maybe
Green = Low: Just Thought You Should Know

The other SDAS scale, of course, is more important, because it’s the one you use to categorize actionable issues in the defect database:

Red = Severe: This Will Get You Fired
Orange = High: This Probably Will Get You Fired
Yellow = Elevated: This Might Get You Fired
Blue = Guarded: This Probably Won’t Get You Fired
Green = Low: This Is Just Stupid

Follow this system, my friend, and you’ll never get scolded for misspelling the CEO’s name, or miscalculating stock option prices, ever ever again.