Software Management: Why Measure Automated App Testing?
Nowadays, many managers in charge of mobile app development projects expect automated testing for web and mobile apps to be a powerful tool; removing problems about test scheduling, the costs of testing, defect and failure reporting, and more. Automating testing can have positive impacts in many areas, and there are many success stories which prove that test automation helps to save money and to avoid some problems commonly produced during testing cycle.
How to measure testing projects
For reaching success it is necessary to set realistic expectations in management as well as to understand the benefits which test automation provide us. It’s possible to easily implement a cost justification for automation purposes if management requests numbers. However, automation generally does not benefit the organization any more than testing does. Cost benefit analysis give us useful information for taking decisions about how to best manage and invest in testing.
A useful measure supports effective analysis and decision making which are easily obtained. Sometimes it is not necessary to apply a complex measure because for some projects easy measures could be more useful even though they are not as accurate. The “Useful” term depends on objectives, letting you to know what you want to know.
Presentation of measures is very important because of the understanding of someone external in a project depending on how does the information looks like. Thus we can simply use organized information so that the people have a better vision for the objectives of that project. However, there are some metrics which give us useful results just with a simple analysis in a determined area. On the other hand, every metric can be abused if we do not take care of what and how we review it.
A particularly useful metric is EMTE (Equivalent Manual Test Effort) which is considered as a simple measure that can be understood easily and can be really useful in many ways. So we can say that a test case EMTE is equivalent to the amount of effort needed for running manual test cases. Therefore, it can relate with other measures with EMTE for better results, anf being more meaningful than simple averages.
Measuring Automated Testing
There are some important measures for automation. One of the options consists of an approach to the BENEFIT which is related to the main objectives of automation according to the project. It also proves that the effort spent on automating tests provides a higher value than if we were to spend time running tests manually. On the other hand we have to focus on RECURRING COSTS which is the biggest and most frequent spending. It comprises the build cost, which is the overall effort to automate testing.
The failure analysis cost helps to obtain results as effort from first recognition through to resumption of test execution, average hour per failed test case, % test failing, etc. Terefore, maintenance cost is related with effort updating testware which reveals the percentage of test cases requiring support, average effort per test case, percentage of equivalent manual test effort and frequency of software changes that impact automated tests.
Final Thoughts
Measuring automated testing in web and mobile app projects is important because it helps to obtain results which produce metrics for analyzing and improving the testing cycle and its impact on the quality of the software. We can conclude that there is perfect measure and measuring something is better than not measuring at all. So the objectives for automation indicate important metrics. Also it’s important to measure the benefits, being the most important since costs are more obvious. Finally, if we measure specific costs, then we know what and where to improve.
References
Fewster, M. (2006). BSC. Reviewed on December 8, 2013 http://www.bcs.org/upload/pdf/measurement.pdf
Hoffman, D. (1999). Software Quality Methods. Reviewed on December 8, 2013, http://www.softwarequalitymethods.com/papers/star99%20model%20paper.pdf
About the Author
Juan Pablo Santiago has been a tester for almost 7 months. He just graduated as a TI technician and now is a TI Engineering student. He works at iTexico as a Mobile Tester and Junior Web developer.
Post Your Comment Here