Understanding the

Testing Challenges" in
Mobile Application
Presentation Objectives
Understanding the "Testing Challenges" in Mobile Applications by
differentiating between the different types of Mobile Applications,
the HCI Guidelines,
User Expectations,
Testing Methodologies and some practical examples.


Setting Up the Testing Environment


FACTS
ACCORDING TO SDTIMES SURVEY 6 IN EVERY 7 COMPANIES ARE PLANNING TO
DEVELOP A MOBILE APP WITHIN THE NEXT 18 MONTHS
SMART PHONES ARE RAPIDLY BECOMING THE PRIMARY METHOD OF
INTERACTION BETWEEN BUSINESS AND CONSUMERS
So What is at Stake?
Types of Mobile Apps
1. Mobile Native Apps


Types of Mobile Apps
2. Mobile Web Apps
Types of Mobile Apps
3. Hybrid Mobile App

Pros and Cons of Mobile Apps
Apples HUMAN COMPUTER INTERFACE
GUIDELINES
Users have one view at a time
Re-organize how information is to be displayed
from the desktop

Minimize number of user controls and input
elements
Do not display UI elements that are not absolutely
necessary

Provide tap-able items with a minimum dimension
of 44 x 44 points
Make it easier for the end user to interact with the
system

Apples HUMAN COMPUTER INTERFACE
GUIDELINES
User standard controls with which users have had prior experience
Steps with which users can go back or retrace their steps.

Users are going to use the application under various distractions
Design process to enable users to complete their tasks as quickly

Minimize text inputs
Use dropdowns/sliders/checkbox where applicable

Android UI Design GUIDELINES
Pictures are faster than words
Consider using pictures to explain ideas

Give me tricks that work everywhere
Make it easier to learn by leveraging design patterns from
other Android apps

Sprinkle Encouragement
Give feedback on actions, even if it's a subtle glow or a
slight vibration.


The power of App Users
USERS CAN AND WILL PROVIDE REVIEWS OF YOUR APPLICATION PUBLIC ON APP
STORE.










Users Control Reputation
FROM THE LIST BELOW WHICH APP IS THE MOST APPEALING TO THE USER?
WHAT FACTORS MAKE THE USERS WANT TO DOWNLOAD THEIR APPLICATION?
USABILITY. USER INTERFACE HAVE TO BE TOP NOTCH. APP SHOULD
PERFORM ITS MAIN PURPOSE SEAMLESSLY WITH GREAT ANIMATIONS.
APP SPEED . FUNCTIONAL SPEED ON DIFFERENT TYPES OF MOBILE
NETWORKS. [USERS ARE IN GENERAL IMPATIENT.]
DRAINAGE. DRAIN TOO MUCH BATTERY, USERS WILL UNINSTALL THE
APPLICATION.
SECURITY. USERS ARE SENSITIVE ON PRIVACY AND TRACKING.
APP AVAILABILITY. IN ALL TYPES OF HANDHELD DEVICES.
UPDATES. USERS WANT THE APP TO BE UPDATED REGULARLY FOR
IMPROVEMENTS AND BUG FIXES.

What does the Users Want?
What is the Ultimate Goal?
A COMPANYS BRAND STRENGTH AND REPUTATION IS DIRECTLY CO-RELATIONAL
TO THEIR MOBILE APPLICATIONS.




Type of App Content
Productivity Apps.
Users using app to organize data such as Personal Finance Apps
requiring multiple views.

Type of App Content
Utility Apps.
Users performing single tasks like watching videos on an attractive
UI.

Type of App Content
Immersive Apps.
Gaming/simulation Applications providing custom GUI.

TESTING ON MOBILE DEVICES
UI/USABILITY TESTING

The key to this is to make sure that HCI guidelines set forward by
Apple/Google are imposed on the applications.

Along with many questions, here are few we should consider asking:
Are the buttons too small to click on?*
Does the UI utilize good animations and transition effects?*
Is the application smooth and fast to use?*
Does the UI renders correctly on different OS versions?
Are proper error handling messages displayed*


Supporting Different Screen
Resolutions
Using Adaptable Layouts for
different screen sizes.

Is the Adaptable Layout
justifiable?

There is more space to display
more options to the user

What would be the
maximum/minimum screen
resolution?

The layouts are working
properly on different OS
versions?




Usability Issues -- 1
A user turns of the app and tries to log in. However, the internet
connection is disabled.

Usability Issues -- 1
Let the user know that the internet is turned off, else he/she will think
that something is wrong with the app.

Usability Issues -- 2
The user purchased over 150 Items from the website.
Via the API call during Log in to the app, all the Items and the Item Details
were Loaded.



Usability Issues -- 2
This user faced a big problem. It took him over 4/5 minutes to load
all the items.
On top of that, when tested on low configuration devices, the app
ended up crashing.


Usability Issues -- 2
Hence in order to rectify the issue, the API call were
modified to only send the Item List first.

Afterwards, on item click, each individual item
returned the Item Detail list via another API call.

This improved the loading time significantly to less
than 30 seconds
Trade-off: App now has to make more calls to the
server to retrieve data.
UI/USABILITY TESTING -- Events
TESTING ON MOBILE DEVICES
FUNCTIONAL TESTING

Are there too many functionalities on-screen to distract the end users?*
Can the tasks be broken down into simpler parts?
Are the functionalities working cross platform?*
Are the functionalities working even when the app is
resumed/restarted?
Functional Testing Issue -1
A video is being played by the user.
Then decides to click on the Back Button or Log Out Button.
The video should stop playing for both cases
Functional Testing Issue -2
Usually, during app load various JS files are downloaded from
the server.
When the internet was turned off when app is launches many
of the JS files were not loaded.
The biggest and most major problem was
BrightcoveExperiences.js was not loaded and hence
No Video or Audio Files were played
User had to kill app and restart app after activating internet
connection to be able to watch videos.

TESTING ON MOBILE DEVICES
INTERRUPT TESTING

How does the application handle:
Incoming Phone Calls
Text Messages
Notifications
Low Battery Alerts
TESTING ON MOBILE DEVICES
NETWORK TESTING
A decisive factor on Apps performance
Hence, testing under different/strained networks is a must.*
o Wi-Fi Network [High Speed/low Speed]
o 3G/2G
o Different Carriers

TESTING ON MOBILE DEVICES
CROSS-OS / CROSS-Device TESTING

Does the app performs satisfactorily in
IOS 5.0/5.1/6.0/6.1/7.0/7.1 in both iPhone & iPad?*
Android 4.0/4.1.4.2/4.3/Company Specific Rip?*
Other necessary OS versions on Windows Phones, Blackberry?


TESTING ON MOBILE DEVICES
PERFORMANCE AND STRESS TESTING
Does the application perform in low configuration devices?*
What if the user turns on this application while a large number of apps
are working on the background?*
Is there any graphics lag?*
Does animation slow down the application?*
Are all user information saved before crashing?


TESTING ON MOBILE DEVICES
RESOURCE ISSUES.
Does App cache over time leads to UI lags and application crashing?**
Test application on low configuration emulator/devices*
Check reliability of the application*


Emulators
ADVANTAGES OF TESTING ON EMULATORS




Cost effective solution as actual devices are expensive

Faster internet and has less latency than real devices

Can be easier to develop and test app in the initial development stages

Real-time debugging available on iOS Apps and Android Kit Kat





Emulators
DISADVANTAGES OF TESTING ON EMULATORS
Can be unreliable as it can cause false impression

Mobile computing resources are not emulated accurately to be able to
make judgment calls

Mobile/3G Networks affect application behavior

Can be very slow to use for testing. Can loose chain of thought.



Actual Devices
ADVANTAGES OF TESTING ON ACTUAL DEVICES

Allows Feel and touch sensations which is a big for understanding usability

Only true way of understanding the user experience, taking into account the
CPU, memory, screen size, etc.

Can test under different mobile Networks as it will affect the app.

Real-time debugging available on
o iOS Apps and,
o All upcoming Android versions starting from Android Kit-Kat



Actual Devices
DISADVANTAGES OF TESTING ON ACTUAL DEVICES
Physical Resources which need to be managed.

The logistics and costs involved in procuring and managing these devices
are significantly high.

Android Challenges
DEVICE SPECIFIC TESTING IS A MUST
Android Challenges
DEVICE SENSITIVE TESTING IS A MUST

SAMSUNG/NEXUS/HTC ALL USE MODIFIED OS VERSION

NETWORK/MEMORY CAPACITY/OTHER RESOURCES ARE
DIFFERENT ON EACH DEVICE

DIFFERENT DEVICES USES LOCAL RESOURCES DIFFERENTLY


Android Challenges
DEVICE SENSITIVE TESTING IS A MUST

IMPRACTICAL TO TEST ON ALL DEVICES.
HENCE, TEST ON POPULAR MOBILE PHONES
SAMSUNG S4 / NEXUS 5 / IPHONE 5

ALSO TEST ON POPULAR TABS NEXUS 10/SAMSUNG GALAXY
TAB/ IPAD 2

Bug Reporting Options
1. SCREENSHOTS CAN BE USED ALONG WITH FACE TO FACE
COMMUNICATION WITH DEVELOPERS TO VISUALLY GENERATE THE
GLITCH FOR BETTER UNDERSTANDING.

PROS AND CONS
IT IS THE FASTEST WAY TO REPORT BUGS.
TOUCH AND FEEL SENSATIONS CANNOT BE REPRESENTED PROPERLY.
GENERATE ISSUE IN FRONT OF DEVELOPER TO EXPLAIN THE TOUCH
AND FEEL FACTOR PROPERLY.



WHAT ARE THE OPTIONS?
Bug Reporting Options
2. USING ANDROID SCREEN RECORD APPS
A SHORT STORY OF THE SCENARIO FOLLOWED BY THE
SCREEN RECORDED SCENE.

PROS AND CONS
TIME CONSUMING AND VIDEOS ARE LARGE AND
LAGGY
CAN GENERATE REAL SCENARIO FOR DEVELOPER TO
VISUALIZE WITH TOUCH GESTURE RECORDING
HOWEVER, CAN BE CONFUSING TO INTERPRET.




PROBABLE OPTIONS
Setting Up the Test Environment
1. J ava Installation
Download and Install Java JRE from
http://www.oracle.com/technetwork/java/java
se/downloads/jre7-downloads-1880261.html
Add java JRE bin folder to the Path Variables


Download and Install Java JDK from
https://jdk7.java.net/download.html
Add java JDK bin folder to the Path Variables





Setting Up the Test Environment
2. ANDROID-SDK
Download SDK from
http://developer.android.com/sdk/index.html?hl=sk
Unzip the download on C:\





Setting Up the Test Environment
Add Android SDK tools folder to the Path Variables
Setting Up the Test Environment
3. Apache Ant
[Java-based build tool]
Download from http://ant.apache.org/bindownload.cgi
Unzip to any location. Eg. E:\Tools\Project\Build\apache-ant-x.x.x
Open environment variables (Control Panel System Advanced
Environment Variables) add a new system variable as displayed
below,


Setting Up the Test Environment
Set the %ANT_HOME%\bin; to the path as
displayed below,


Verify the installation of Ant by typing the
following command in the command
prompt,
Setting Up the Test Environment
4. NODEJS
[ Software platform for scalable server-side and
networking applications]
Download NODEJS from
http://nodejs.org/download/
Run command prompt
If nodejs is installed then npm command will be
recognized
Phone-Gap /Cordova
Phone-Gap/Cordova are application framework that allows developers to use HTML,
JavaScript and CSS to create apps for Mobile Market Place.
Adobe donated the code-base of Phone-Gap to Apache, which then renamed it
Cordova.

This was done so that PhoneGap can reap the benefits of open source, while at the same
time having the ability to be different.



Setting Up the Test Environment
5. Cordova/Phone-Gap
Open command prompt and type



Setting Up the Test Environment

Already have a MAC VM with all the necessary environments pre-
installed to be able to start testing immediately
Thank You