Case study of testing an e-learning web application

Context: Testing of release candidate & pre-beta testing Client: An award winning startup from Bengaluru, India

Pradeep Soundararajan sales@moolya.com / www.moolya.com

Copyright Moolya Software Testing Private Limited: www.moolya.com

Business challenge
An e-learning application developed by a client was scheduled to be released before school reopening. With less than three weeks before the code was planned to be pushed to production, finding important problems quickly was identified as the most important goal. With the developers at a different location and busy with the next sprint, an independent test team was required to help achieve the goal. Introducing the test team in between the project shouldnt create overheads to existing work schedules other than bug fixing.

Technical challenge
No time for the client to collate requirements documentation that can be shared with the test team because its all scattered over emails and notes while they were constantly changing. (Classic) Feedback from testing needs to be rapid and supportive to the dev team. The time to learn and test the application should not compromise the ability to provide a variety of information Test coverage to be focused towards: Functional Usability Security Accessibility Report should be lean and precise Reports should be easily understood by several stakeholders of the project Tests should cover all functions with deeper coverage on critical functions No access to code, backend or database

Copyright Moolya Software Testing Private Limited: www.moolya.com

Choosing practices suited for the context

Testers at Moolya brainstormed approaches and practices that might suit the context and the same was negotiated and agreed upon with the client. A list of practices chosen are listed below Progress & Coverage A daily meeting over a call between the client and Moolya All communication in a consistent format throughout the project duration Daily status report over email with the body of email containing the summary that helps all stakeholders grab a quick overview and for more details, the attachment would help. No documentation of tests but creation and use of checklists as necessary Mind maps for test coverage assessment Testers at Moolya seeking feedback for any changes needed to their work on a daily basis to make themselves more effective to the client Listing the testing blockers separately and highlighting it in the email as a top priority Accessibility tests as per Web Accessibility 508 compliance standard Regression testing for bug fix releases Exploratory & Heuristic approach to testing Tests done on staging server Cross browser testing in Firefox, IE & Chrome with add-ons OS Compatibility with Windows XP, Windows 7 and Apple Macintosh OS X 10.6.6 Types of coverage to be reported for each function tested Tracking the progress of the test coverage through mindmaps Emphasis on scenario based and role based testing for student tutor use

Bug Reporting Bug reporting in excel sheet with just three columns: Bug Summary Potential Risk to the user or business Color code indicating the severity No priority will be set or recommended by the tester for every bug reported Screenshot as an attachment to the report In a case where more support is required for a bug / a bug is intermittent, video recording of the bugs or steps to reproduce will be shared. All blockers to be communicated as and when the investigation is completed Version control of test and bug reports

Copyright Moolya Software Testing Private Limited: www.moolya.com

Project challenges faced

Release dates kept changing (Another classic) Unable to interact with the developers frequently Revised deadlines while goal remained same Delayed access to requirement changes Investigating intermittent bugs became a challenge, especially with no access to server logs Testability could have been better with more access

Things that made journey cool

View permissions of SQL queries on the staging server Freedom (with responsibility) bestowed on us from our client No enforcement of best practices throughout the project as achieving the goal was more important than following any best practices

Value added (as perceived by us)

25 high severity issues, 51 medium severity and 81 low severity issues reported in two weeks of testing All fixed bugs regressed and updated Test coverage provided through mind maps Inputs data coverage achieved Roles of Student Tutor Admin covered Single email thread maintained for all two weeks to provide status and progress update

For more information on how Moolya can help you in testing: sales@moolya.com

Copyright Moolya Software Testing Private Limited: www.moolya.com