Tuning Unit Tests

$$SWYNK$$

Tuning Unit Tests
I hate fads. Fads, to me, are software, organization structures, or practices that are done because everyone else is doing it, without understanding the principles behind then, or if they are applicable to your specific situation.

For example, SAP was a fad. It seems that every large company had to have SAP, regardless of how good the fit may be. Today the cloud is a fad. I had a Venture Capital individual walk away from a project simply because it didn’t use the cloud, even though the project didn’t justify the additional cost.

Other things are best practices because they have proven to be valuable in repeatable situations. In my experience, unit testing falls into a best practice category. Unit testing may become a fad when applied inappropriately.

For example, one shop blindly wanted to have 100% code coverage by unit tests. This meant that every single line of code was verified by at least one test. This extreme resulted in a lot of wasted effort. Many of the tests were simply validating that the language worked the way the language was designed.

Our reader submissions today do a great job of demonstrating value of unit testing as it is applied to their situation.

Greg:
I have been working with ETL for several years now. The application development for my projects has been specifically data-driven.

At my company, my experience at least is that we use a combination of TFD (Unit Testing) and TAD (System Testing). Our definitions of quality testing are:

Unit Testing: Tests done by the developer responsible for a change. These are usually limited to a specific defect or enhancement. The idea is to know that your change works as needed, and that it executes successfully as it did before but with the new change as well. Sometimes developers can get by just executing the specific module or script, rather than the entire job, especially if there is no small scrubbed file available.

System Testing: Tests done by another person, dedicated to QA, that verify the same as the Unit Testing but also while executing the job as a whole. They will run the entire job with either a small scrubbed file or with a full-sized one. Their tests normally just involve altering the data to see how the job handles it. Sometimes this will also involve performance testing, so that jobs with a design flaw could be found where they execute or hold locks onto resources far too long.

Regression Testing: Tests done by someone in QA to ensure that the new change does not break other existing functionality; i.e., other data feeds that need to execute. This may be done asynchronously from System Testing.

All of the projects I have been on have been agile-driven. While it can be well-intentioned for upper management to be able to see ‘progress’ in a short amount of time, there are some things that just either don’t fit into a 3-week sprint or need to be done up-front so that the deliverables can begin. My approach has been to ensure accuracy and quality, but not at the same speed as others who fly through work and just want to mark something ‘green’ or complete. I have never been able to experience the traditional requirements-driven model. Over time I have seen many examples where a delivery team that has over-promised has neglected to document anything at all, and if forced to it is not of the same quality as if it had been done during development. As teams come and go, and contractors are reassigned, the real knowledge fades away until an incident or a forklift/migration makes it necessary to go back and crack open the code.

I have tried to enforce standards on Unit Testing, but even with checklists that ask if very specific functionality works, they become rubber-stamped and the impetus comes back on the enforcer to really ask the tough questions. Now we have basic code checkers, but even that needs to be managed and updated over time. It is mainly a personal ethic thing… You either care enough to do it right the first time, or you care only enough to fill in the blanks, turn in your work and get your paycheck (or bonus) on time before going onto the next project.

Llewellyn:
As you might know, I’m the creator of an open source Assertion Framework called ApprovalTests. I wanted to touch on a couple of things that I thought could help fill out the TDD/BDD/Test First experience.

First there are 4 benefits of Unit Tests.

1) Specifications
Unit test provide a up to date, executable set of requirements as to what the code should do. Even if you never wrote a single unit test, simply sketching out the scenario to be tested would help to to code better.

2) Feedback
Ever write a main method to see how your code was running? or open up a local browser / query editor / simulator? all of these are about feed back, and you can usually manually achieve that feedback in 40-90 seconds with a bit of typing… automated unit test will give me feedback < 2 seconds. This is a big difference if I am working in small chunks (30 secs work + 2 secs test = 7% of my time is waiting for feedback. but 30 secs work + 40 secs manual tests = 58% of my time is waiting for feedback)

3) Regression
If it use to work, it should still work. This is the hidden requirement of almost every feature, and it is the violation of this requirement which is responsible for almost all of your bugs, because you usually at least manually test the new thing you are working on to make sure it works. Unit test can provide a large net of regression against your code base

4) Granularity
Ever get a bug report that says "the code is broken"? Compare that to "When a user enters a date of 2/4/2011 and a Total of$34.54 the tax is calculated as 3.42 instead of 3.23" Which one is going to take you less time to fix? But unit testing don’t just provide granularity of details, they also provide granularity of time. Wouldn’t be nice to get a bug report: "when line 294 of class InternalBussinessLogic was changed form var a = 15; to var a = 15.0; the code broke, would you like to undo that?" The ability to quickly and easily run your unit test gives many programmer just that kind of bug report every single day.

1) Write Test
2) Get test to pass
3) Refactor Code to improve readability, remove duplication, include new insight.
4) repeat

Design is still important, we just do in on the back side, without the Refactoring step, TDD loses this design.

Do you have software problems you haven’t been able to test? Are you afraid of introducing unit testing in your sphere of influence? Have you seen unit testing abused? Share your experience or questions with us by sending your comments to btaylor@sswug.org.

Cheers,

Ben

Featured Article(s)
Tips for using SQL Server 2008 Integration Services (Part 1)
In this article, you can find some helpful tips to performance tune and optimize SQL Server 2008 Integration Services.