Regression Testing of Software at SPEC India

One of the services provided by SPEC INDIA is Application Support and Maintenance. We have a vast experience in delivering quality off shore software application support. As a rule any time we modify an implementation within a program, we also do regression testing. We do this by rerunning existing test scripts against the modified code to determine whether the changes break anything that worked prior to the change and by writing new tests if necessary.

Sufficient coverage without wasting time is a primary concern when conducting regression tests. With effective planning we spend just enough time doing regression testing without reducing the probability of finding bugs in old, already tested code.

Our experience shows that:

• When software is fixed / change implemented, new and/or old bugs reoccur
• Many times a fix is "weak" that is, it fixes the problem in a typical scenario, where it was first observed but not in more general scenario which may arise later
• A fix for a problem in one area involuntarily causes a bug in another area
• Often in case of changes when some feature is redesigned, thru oversight the same mistakes that were made in the original implementation of the feature are made in the redesign

We consider the following aspects while regression test planning:

• Test fixed bugs/ changes first
• Write test scripts for each bug fixed
• Watch out for any adverse effects of fixes. The bug itself is fixed but a new bug created
• If there are similar tests, find the most effective and get rid of the others
• Identify tests that the program consistently passes and archive them
• Focus on functional issues
• Use different datasheets and find any resulting corruption

Building a repository
Our regression testing is based on a repository of tests developed from tests made up of base test case that can be run every time you build a new version of the program. With every new version of the program the test cases are updated to include all the feature changes and fixes. So before the next version is released for testing the scripts are ready for regression.

We also have the Automated Test Framework, which includes unit level and functional test cases. Unit level test scripts include functions to test basic validations, boundary conditions and timing for all the objects of the modules. While Functional tests provides generic routines and utilities useful even outside the context of a particular application. The Functions help with error detection, error correction, and synchronization. These Functions can readily use the application-specific data stored in the Data Table and test tables as necessary.

Periodic review of the regression test repository is done to eliminate redundant or unnecessary tests. Numerous tests are written and added to the regression test repository. These multiple tests are useful for fixing the bug, but when all traces of the bug and its variants are eliminated from the program, the best of the tests associated with the bug is selected and the rest removed from the repository.