The Endless Guide to Quality Assure Video Streaming Apps

Currently six in ten adults stream online to watch TV, says The Pew Research Center. The constant increase of streaming content makes it a key factor to have impeccable quality of service, where the consumers patience is limited and options for alternate subscriptions are plenty.

The exponential increase in the users of streaming video applications has led to multiple releases and updates of such apps across all devices, from mobile to smart TVs. Technology updates and innovations within such video-on-demand (VoD) or over-the-top (OTT) streaming apps have plateaued. This implies the need for rocksolid and stable services, which would help retain and grow consumer bases. 

Qualitative releases and planned testing can not only solve or reduce losses, but also earn the trust of the consumer, which is not easy nor cost-effective to gain back, once it has been lost. Consumers choose between a plethora of streaming TV Apps across many devices, and this mandates a thorough quality assurance process to ensure that apps are free of major defects.

The Endless List: Plans & Processes for Tests and Validation:

Any testing plan or quality assurance routine starts with understanding the target application and service, as consumed by users. Checking the design documentation and product goals helps for an initial outline of a Test Book. 

Every button, click, icon and page within an application should be browsed and listed as test cases and should be organised based on intended “functionality” as “features”. Writing an extensive number of positive and negative testing scenarios, as well as covering a reasonable number of edge cases is the ultimate goal of a Test Book. 

Quality Assurance engineers should assume and understand the role of a consumer, while using the apps. They should presume every possible scenario objectively and make sure edge cases are checked at the right moment. Based on such different assumptions, the validation of any release candidate needs to planned with some of the testing scenarios listed below:

Functional Testing/System Integration Testing

Functional Testing is defined as testing all the parts of an app to ensure that they satisfy the functional requirements whilst System Integration Testing focuses on the interactions between these features. 

This should be considered when writing Test Cases and Test Books, ensuring that all features adhere to the requirements both in isolation and together as a whole. 

Test cases should be written ahead of development. This way, the developers receive the acceptance criteria for each feature, as well as what test cases QA will perform. All whilst QA gets an in depth understanding of the functional requirements of the application.

These Test Books can then be condensed into Checklists, Smoke Tests and Sanity Tests based on the maturity of the project. 

UI/UX Testing

UI/UX Testing is defined as testing both the User Interface (UI) and User Experience (UX). This is an activity that is generally performed by a member of the Design team to ensure that the app’s UI is according to the design specifications as well as intuitive, whilst also ensuring that the user experience is up to scratch.

This type of testing is typically performed once the application is feature completed and ready for market, or once it is already in the market. 

At Norigin we practise Dog Fooding, where our designers, developers and QA use the applications we have developed at home, thus allowing them to use the app as a real end user would. This enables them to find usability issues within the app that they may not discover during functional testing. 

UI/UX Testing can also involve feedback from actual consumers, in the form of End User testing or conducting surveys with the actual customers on a routine basis. This is important because sometimes the real feedback from a consumer is more of a refreshing point of view!

Regression Testing defined as change-related testing to detect whether bugs have been introduced or uncovered in unchanged areas of the app during bug fixing or feature development. In other words, the QA team looks for any issues (or regressions) caused by new features or fixes.

At Norigin Media, this is a periodical activity, as the testing and development teams can discover any regressions whilst they verify and confirm bug fixes and perform feature testing.

Interoperability Testing defined as testing how the application interacts with other software components and/or operating systems. In other words, testing how the app performs across multiple platforms, devices, operating systems and/or browsers. 

A good example of this is when developing apps for legacy devices such as older Samsung Tizen or LG webOS Smart TVs, and when porting the application to Set Top Boxes as there are a number of different software and operating systems that interact with the application. 

Network Testing defined as testing your application under different network conditions to identify bugs and any potential performance issues. It can be used to either mimic  network conditions  similar  to production, which helps to reproduce and fix problems experienced by end-users, or test the applications performance under poor network conditions. 

At Norigin Media, we perform Network Testing when troubleshooting production issues, testing the apps performance as well as verifying certain features such as Adaptive Bitrate Streaming – the feature in the Player that switches between various streaming qualities based on your network conditions. 

Performance Testing defined as ​​testing the app to determine the overall smoothness and performance efficiency of the application. This is especially important, as users may lose interest in the application if it lacks performance, especially  if there is a delay in loading the live stream or a video on demand playback.

At Norigin Media, we test against KPIs for loading times during app navigation, page loading and stream loading, to ensure that the application performs at a high-level. 

Content is king, but what if the castle is falling apart and not checked up on for maintenance? The content needs to be streamed, but what if the app is buggy or offers a bad user experience? 

This can negatively affect the business as they can fail to retain existing customers or gain new ones. As technology advances, and steaming services multiply, it is important to offer end users the best experience including a stable application that will not frustrate them or drive them away.

As a quality assurance engineer at Norigin Media; I make sure that certification and quality assurance processes are planned or organised in a manner where apps are released onto any app store, only after a strong stamp of validation. The testing-cycles can be short or long, but when they are well planned, apps are always efficiently certified by all, including a Quality Assurance Engineer!

Norigin Media offers professional turnkey or customised testing service-packs for broadcasters or media companies. The Testing-aaS Solutions are customised, where we understand product and customer KPIs. Do you need help testing your Apps?