Let’s continue discussing a few more details from our seminar, "Exterminate Your Bugs to Lower Software Costs: Top 5 things you can do right now!”  

#4- Reduce Churn.  The concept here is to “Stop spinning your wheels.”   Often times, the development and testing teams start feeling the pressure of the schedule and start cutting corners.  

We’ve all been there.  Pressure from management is building.  As a developer, you committed to getting your code completed by a certain date, but it’s taking longer than you expected, so what do you do?  You start reducing the amount of unit testing that you planned.  The code will probably work anyhow, right?  Besides, it doesn’t really affect you … at least not for awhile.  

Of course, the problem now becomes a team problem.  By making that decision, you handed your problem over to the test team.  Now, they need to find more bugs than they planned for.  Not only that, but it is very hard (and sometimes impossible) for a black-box test team to find certain coding bugs (that should have been discovered using white box techniques).   To be clear, I am not judging the developers.  We’ve made the same mistakes here at Lighthouse, but we are learning here and trying to understand the implications of our decisions.

What we’ve discovered is – if you use objective measures of quality, it helps to take the emotion out of the game.  Start requiring the use of automated unit testing and start tracking a few simple metrics like Number of Unit Test Cases executed, Number of Unit Test Defects Discovered, and Number of Coding Defects Discovered in System Test.  Eventually, you will start seeing a trend to let you know how many defects you should be finding during unit testing.   In addition, there are techniques that you can employ to forecast the number of defects expected during unit testing.  Then you can compare the actual discovered vs. the forecasted and take action if you start deviating significantly from plan.

In an earlier blog , we showed a graph of how many of you separate test environments.  It was really surprising to see only 13% had dedicated UAT environments.  The hardware and licenses are generally very inexpensive as compared to the labor and churn from trying to share environments.

There are a dozen other ways that teams create churn, but I will only focus on one more – Configuration Management (of your code and test data.)  Be sure you take the time to clearly identify which code gets compiled into each specific build.  Label your builds and label the code in your configuration management tool, and be sure you know exactly which defects were fixed in each build.  We’ve had too many clients who find a bug in UAT, then cannot recreate it in their Integration or System Test environments.    This causes a large amount of churn and frustration, especially when the bug is repeatedly discovered.

In addition, you need to tightly manage the configuration of your test data.  We’ve seen too many clients who had good CM of their code, but have no control of their test data.  Your test cases should very clearly indicate which data load you need and you should use manufactured test data for Integration and System Testing.  If you try to use live data, you will struggle because you don’t know what’s in it.  It may or may not contain the data needed to run your test case and then you don’t know your expected results.  Test teams are very efficient when they know their input data and have expected results.  There will be enormous churn if you try anything else.  Plus, when using live data, you can’t have confidence that you fully tested the system.

I’m not saying to never use live data, but it should be reserved for the end of your UAT, when you are confident that the code and integrations are working properly.  And you should definitely use it then to simulate the throughput and variations before going live.

That’s all for now.  Stay in touch and most definitely …

Keep having fun,
Jeff Van Fleet