An impressive article from Eric Jacobson that makes a lot of testers free. Here is his list of cases that could be skipped:
- Features that don’t go to production– My team has these every iteration. These are things like enhancements to error logging tables or audit reports to track production activity. On Agile teams these fall under the umbrella of Developer User Stories. The bits literally do not go to production and by their nature cannot directly affect users.
- Patches for critical production problems that can’t get worse– One afternoon our customers called tech support indicating they were on the verge of missing a critical deadline because our product had a blocking bug. We had one hour to deliver the fix to production. The programmer had the fix ready quickly and the risk of further breaking production was insignificant because production was currently useless. Want to be a hero? Don’t slow things down. Pass it through to production. Test it later if you need to.
- Cosmetic bug fixes with timely test setup– We fixed a spelling mistake that had shown up on a screen shot of a user error message. The user was unaware of the spelling mistake but we fixed it anyway; quick and easy. Triggering said error message required about 30 minutes of setup. Is it worth it?
- Straight forward configuration changes– Last year our product began encountering abnormally large production jobs it could not process. A programmer attempted to fix the problem with an obvious configuration change. There was no easy way to create a job large enough to cross the threshold in the QA environment. We made the configuration change in production and the users happily did the testing for us.
- Too technical for a non-programmer to test– Testing some functionality requires performing actions while using breakpoints in the code to reproduce race conditions. Sometimes a tester is no match for the tools and skills of a programmer with intimate knowledge of the product code. Discuss the tests but step aside.
- Non-tester on loan– If a non-tester on the team is willing to help test, or better yet, wants to help test a certain Feature, take advantage of it. Share test ideas and ask for test reports. If you’re satisfied, don’t test it.
- No repro steps– Occasionally a programmer will take a stab at something. There are often errors reported for which nobody can determine the reproduction steps. We may want to regression test the updated area, but we won’t prevent the apparent fix from deploying just because we don’t know if it works or not.
- Inadequate test data or hardware – Let’s face it. Most of us don’t have as many load balanced servers in our QA environment as we do in production. When a valid test requires production resources not available outside of production, we may not be able to test it.