Following up with the previous blog on mobile testing, let’s have a brief look at what are the challenges for mobile testing as a practice.
The parameters to be considered are:
Devices:- Mobile devices differ in Screen Sizes, Input methods (QWERTY, Touch, Normal) with different Hardware and Software Configuration. It is expensive to buy all the devices for testing purpose.
Different Mobile Platforms/OS:- There are different Mobile OS available in the market among which the popular ones are Android, IOS, Blackberry, and Windows. Each operating System has their own limitations. Testing Single application across various platforms is a tedious job for a tester.
Network:- There are around 400 different network providers across the world out of which some are CDMA, some are GSM and others use less common networks (FOMA&TD-SCDMA). Since the application would be used on real devices from different locations which would have different bandwidths with 2G, 3G, 4G and Wi-Fi connection, we have to plan to cover all these networks.
Scripting:- Various devices differ in keystrokes, input methods, menu structure and display properties. Single script does not function on every device. Multiple scripts have to be written, to work on different devices.
Apart from testing the app functionality, testers need to concentrate on
Usability testing covers the following: Text visibility in English language, Screen navigation, Verification of functionality online/offline, Feedback from interaction with system i.e. downloaded app should be prompt with message, Validation of text and graphics formatting on various screen resolutions, Session Maintenance, App.
a) Integration testing:
Validation of buttons, text inputs, labels, etc., Validation of input error message handling, Validate screen layout on multiple resolutions, Validation of menu navigation flow, Validation of device orientation (portrait/landscape).
b) Compatibility testing:
Testing different OS versions, different devices, screen resolutions, screen sizes, checking for the app isolation with other apps on the device.
c) Installation testing:
Installation of the app, installing updates of app, uninstallation of app
d) Interrupt testing:
Incoming and outgoing of SMS and MMS, Incoming and Outgoing calls, Incoming Notifications, Cable Insertion and Removal for data transfer, Network outage and recovery, Media Player on/off, Device Power cycle
Security testing addresses the aspects of:
Insecure Data Storage, Weak Server Side controls, Insufficient Transport Layer Protection, Client Side Injection, Poor Authorization and Authentication, Improper Session Handling, Security Decisions via Untrusted Inputs, Side Channel Data Leakage, Broken Cryptography, Sensitive Information Disclosure
Performance testing includes:
a) End to end business critical scenarios based on the discussion and clarification responses provided by Customer.
b) Server Side: Measure server side response time
c) Load testing: Simultaneous access to application server by several users
d) Validation of and benchmarking API calls
e) Client Side: Measure end user response time on device
f) Validation of app connectivity and performance over 2G/3G/4G/Wi-Fi
g) Screen transition time or Page load time
h) Measure battery usage, memory usage, CPU usage
i) Manual testing – Test for touch sensitivity, scroll sensitivity
j) End to end business critical scenarios based on the discussion and clarification responses provided by Customer.
k) App and Web service load test and response times