Recently I have been talking with companies interested into getting into automated testing. They tend to be overwhelmed with the differences in the different products and the approach they take to testing.
Web application system testing has never been easy. Some sites are nearly impossible to test because of the way the html is built. Individual attributes as we would see them on the screen may not have a unique identifier on it; even when there are unique identifiers, sometimes the values assigned must be dynamic, and are guaranteed to be the same the next time the page is generated. So, even with an expensive testing suite, a web site may not lend itself to having automated testing.
For those web sites that can be automatically tested, what options do you have? There are some cool products with lots of features built in such as expected test results repositories, test history and reporting, or even stress testing. Different price tags cover different levels of capability.
Do you find the additional assets to be of value? Or, are you just as happy using a tool like Selenium or Watr/Want? While those tools don’t provide of themselves a big ecosystem they easily get the tests executed. Is it not enough simply to get a pass/fail message as part of an automated test execution? Do you really need bug tracking, a test repository, reporting, and all the other bells and whistles to have necessary test metrics? Or is it just enough to know on a regular basis that your application performs as expected?
What do you think? Is there a good rule of thumb to help make that decision for a testing? Share your thoughts with us here. Drop me an Email if that works better. Just send it to btaylor@sswug.org.
Cheers,
Ben