Requirement of maintenance effort to address test case flow changes
We have all graduated from record playback approach towards automation and rarely use it considering its limitations in terms of reusability. Let’s assume that we have scripted and parameterized the test cases and no test data is hard coded. So we can run the same scripts using multiple test data sets without having to spend effort in changing them. But what if additional fields are added and the test case flow changes. It would involve modifying the script code and it would involve engineers who are well versed with the tool’s scripting language.
Reduced automation coverage and potential inconsistencies between automated scripts and manual test process flows
Let’s take a step back; test automation is often done by engineers who are well versed with programming concepts and automation techniques, but not with business processes. The ideal situation would be business analysts or functional consultants automating the test process flows, but unfortunately, they do not have automation experience. Also, in SAP testing teams, automation engineers are usually not many in number and they are shared across projects, because of which the rate of automation and coverage is low.
Execution and management of the automated test cases
Many times, automated test scripts are executed and the results are manually updated in the test case management systems. This involves additional effort. The results are analyzed later and defects are logged. This again may delay the fix process for defects in sanity or core regression test cases run overnight.
In my subsequent posts, I will discuss approaches to address these challenges. Do let me know if you want me to cover any specific challenge and solution that you are presently facing.
There are also other challenges involved in automating test process flows that have integration with non-SAP applications. Generation of test data for various iterations of script execution is also a challenge in terms of the effort required. Apart from the effort, it is also difficult to get the confidence that we have covered all the required input data combinations optimally. I will discuss them in my future posts.