The Case For Dysfunctional Testing

You testers, QA specialists, performance jockeys, you poor ISO 9001 efficiency experts—you’ve got it wrong. All wrong.

All this talk about functional testing. It’s a load of hooey, and you know it.

What’s the point of functional testing? You don’t want to know if your code functions.

Your designers and architects are smart, they’ve listened to the end user and created some killer UML diagrams. The UI requirements are solid. Your programmers are top of the heap. Of course it functions. It’s just plain stupid to waste time and money, and mess up your delivery schedule, with so-called functional testing.

What you want to perform is dysfunctional testing. You don’t want to know what works in your n-tier application. You want to know what doesn’t work. As that funny-looking yellow guy on The Simpsons would say, “D’oh!” And Homer ought to know, he’s a nuclear engineer. (He kinda looks like my brother-in-law Fred, only better looking.)

I know that this blinding insight may come as a surprise. But it’s not your fault. Nobody talks about dysfunctional testing. Googling for dysfunctional testing brought up 11 results. Eleven! By contrast, Googling the phrase functional testing led to 253,000 results. That’s just a crime.

It’s clear that everyone has their priorities wrong.

What do I mean by dysfunctional testing? Finding out where the code breaks. It’s just that simple. Really. Stop looking to see what works, and instead see what doesn’t work. Imagine that your million-line client/server system is 98 percent bug-free.

When you’re doing functional testing, you’re testing 100 percent of the application. That could take hours during the nightly test run. Stupid! If you just test the two percent that doesn’t work, you’ve cut the test time down to minutes.

The advantages of dysfunctional testing are all the more compelling when your organization gets higher in the quality scale. Say you’re working you’re way up the Capability Maturity Model ladder, stuck between rungs 2 and 3, and your million lines of code is 99.8 percent functional. Imagine the bottom-line benefits of testing only the 0.2 percent of the code that’s dysfunctional—that’s only 2,000 lines of code.

How hard can it be to test 2,000 lines of code, remediate the dysfunctions, and then refactor like mad to improve the runtime performance? Not hard at all. You’ll be at Six Sigma in no time, guaranteed. Take that, W. Edwards Deming!

Before you nominate me for the Malcolm Baldrige National Quality Award, let’s talk about how to actually achieve dysfunctional testing within your enterprise test/QA organizations. Believe it or not, it’s not as simple as you might think.

A key component of dysfunctional testing is something that I call “root cause analysis” — that’s a term that I’ve just made up. What you need to do is find the root cause of your software defects. In most cases, it’s easy to determine the cause: bugs. Yes, that’s it. Sometimes it might be a design flaw, but I’d put serious money on it being bugs.

There are different ways you can get rid of bugs. One way is to use a debugger. “Yes, I just got de bugger,” is something that I say a lot, ha ha! (You’ve got to have a good sense of humor if you’re working on cleaning up 2,000 lines of code before your afternoon jog.) Another way is to do a code walk through. But that can take a long time, and if you’re not good at dysfunctional testing, you might look at the wrong 2,000 lines.

That’s definitely not recommended by the President’s Council on Physical Fitness and Sports. A better approach is to employ a number of techniques, such as overload testing — that is, you whack the code so hard that it shatters. Forensic analysis will show you where it broke.

(Don’t confuse overload testing with namby-pamby load testing, where you gradually increase the transaction load on an application server while monitoring its performance on a curve. Like, who has time for that?)

Another approach is to adopt seVere Programming, my agile methodology that applies the dysfunctional testing metaphor at the daily work level. In VP, developers work in teams of three: one writes the code, one tries to breaks the code, and the third programmer smacks the first programmer with a ruler whenever a defect is found. It’s foolproof! (If the first developer runs away, this is called the Scram methodology.)

In conclusion: Functional testing is stupid. Dysfunctional testing is smart.

The Nobel Foundation knows where to find me.

SCO Slaps Itself With Lawsuit

NEW BUSINESS MODEL SHOULD RESULT IN GAINS, LOSSES, CONFUSION

April 1, 2005 – The SCO Group is preparing to reverse its flagging fortunes by launching a Unix infringement lawsuit against itself, according to documents obtained by this reporter.

Today’s SCO Group consists of the original Caldera Group, the Unix assets purchased from Novell, and the SCO name bought from The Santa Cruz Operation. Prior to launching its lawsuits against the Linux community in 2003 — hitting companies such as AutoZone, DaimlerChrysler and IBM — Caldera was a major supporter of Linux. Indeed, in 2002 it formed a consortium with Conectiva, SuSE and TurboLinux to enhance and promote the open-source operating system.

According to confidential documents, Caldera may have inadvertently shared intellectual property it gained after the purchase of Unix with its own open-source developers. If true, that would be a violation of its own Unix license, and as such, would expose the company to legal charges and potentially huge damages.

“It’s possible,” pondered Ficus McMushroom, senior legal analyst with Guernsey, Jersey, Sark & Alderney. “If SCO’s Unix staff exchanged any information with their Linux staff, whether sitting at a conference table, on an internal bulletin board, or even while swilling beers at a bar, Unix license and intellectual property violations may have occurred.”

HAND OVER THE STICKYS

Is there a smoking gun? It’s impossible to know until the lawsuit strikes, but according to the secret report, the company is prepared to subpoena extensive documents, including e-mails, voice memos and rainbow-colored Super Sticky Post-It Notes written by SCO honcho Darl McBride.

“We’re clearly on unsteady legal ground,” observed McMushroom. “If the charges are true, the damages might run into the tens or even hundreds of millions of dollars.” Collecting those fees would significantly boost SCO’s revenue for 2005, the analyst noted, and could lead to a rise in stock price or dividend payments, as well as yet another round of restated financials.

The revenue from the suit would be offset by legal fees, and may be potentially covered by SCO’s liability insurance, McMushroom believes. Any additional costs would most likely be treated as an extraordinary charge against earnings.

But in any case, he expects SCO to come out ahead.

But will the suit see the courtroom? “Frankly, I don’t think so,” said McMushroom. “I expect SCO to settle before depositions are taken. But on the other hand, if they go through with the suit, they may set a legal precedent for the Unix lawsuits.”

SCO would not return calls about the rumored lawsuit.

Eclipse Gains Local, Interstellar Support

April 1, 2005 – The Eclipse Foundation today has announced 14 new strategic members, encompassing leading IT vendors, major technology consumers and at least one alien race.

Many of the new members will be extending existing Eclipse projects. For example, the Sesame Workshop will lead development of a new cookie-handling format for the Business Intelligence and Reporting Tools project.

“Frankly, we like working with BIRT,” smiled Ernie, the affable spokesperson for the Sesame Workshop, “because it reminds us of our friend Bert. Bert likes cookies.”

“This open-source project is sponsored by the letters A and F, and the number 5,” added Ernie.

Getting involved in the Eclipse Foundation’s infrastructure challenges are Plumbers and Steamfitters Local No. 43 and Sheet Metal Workers Local No. 16. “It takes a lot of plumbing and ducting to make enterprise computing work,” said shop steward Fergus McEwan, who will represent the unions on the foundation’s board of directors.

McEwan indicated that the two locals will host membership rallies for Eclipse developers.

“It’s time to bring the power of collective bargaining into the open-source community,” he explained, “to improve wages, working conditions and job security.”

One new member fits into its own category: Strategic Warrior. According to Commander Klotha, the Klingon Imperial Navy has pledged to preserve the honor of all Eclipse developers, committers and consumers, while also donating battle-tested source code to support two new projects.

The Eclipse Test and Performance Tools for Deployment of Disruptor Based Techniques in Combat, or ETaPTfDoDBTiC, will be used to create open-source frameworks for programming and tuning real-time system controls for high-intensity phased photonic beam emission systems. Meanwhile, the new pIqaD project will seek to port the Eclipse IDE’s user interface to become compatible with the Klingonaase script and triangular display devices used on distant Qo’noS.

The Imperial fleet also will propose a Stellar Modeling Framework project to be added to the Eclipse Modeling Framework and newly formed Graphical Editing Framework. The SMF would be incubated within the Eclipse Technology Project, and would have many applications in sixth-dimensional cartography, tachyon flux computation, warp-speed navigation and Organian peace treaty negotiations, said the commander.

“You have good software tools on this planet,” praised Klotha. “Kai Eclipse! Qapla’!”