Delivering Quality Products:
With the easy availability of computing power, modern consumers expect an excellent experience from the services they consume: polished, smooth, and responsive. A cursory glance at many apps maintained by Fortune 2000 companies reveals a complete failure to deliver on these expectations. Mobile, with limited screen real estate and continually changing paradigms of user interaction, has proven particularly challenging for mature organizations to adopt. Increasingly, the practices of Quality Assurance (QA) and, in particular, User Experience (UX), have defined the success or failure of digital products and services.
With this decade-long shift in consumption, combined with the scattering and rapid development of new platforms and technologies to accommodate, businesses continue to search for answers in their digital strategies.
How do we ensure quality across hundreds of disparate platforms and form factors? What do we use to ensure a consistent experience across all of our properties? How can one grant his or her developers the velocity, agility, and risk-taking ability necessary to remain competitive while safeguarding the standards their customers have come to expect?
The answers lie in increasing levels of QA automation in the outsourcing process. As the complexity of UI-driven software expands, manual testing must become a last line of defense on the bleeding edge of product development, paving the way for automated systems to provide assurances against regression on the path beyond. More and more, larger IT departments are outsourcing both their automated QA testing tasks in addition to manual QA.
Unit QA Testing:
In-code unit testing has enjoyed a surge of popularity with the popularization of Agile practices, particularly Continuous Integration (CI/CD) and Test Driven Development (TDD). This level of QA starts with the development team themselves and tests the internals of the system. Functions are individually tested to handle chosen parameters and produce error-free, accurate outputs. A good metric to keep in mind is “code coverage”, a reference to the portion of the entire system that is covered with unit tests to protect against regression.
It should be noted that most initial unit testing is done to test correct functionality in a common use case. As defects are discovered, additional tests should be written to cover missed cases.
Unit tests should be executed prior to a developer moving changes into the repository to protect against regressions. Often, Automated and Unit tests are executed as a component to Development Operations (DevOps) process in an organization’s Continuous Integration pipeline.
Automated Testing:
Whether it’s conducted in-house or by outsourced teams, Automated Tests are owned by the QA Testing Engineers and are developed without requiring access to the codebase. They are the first level of “black box” tests.
Automated Tests focus specifically on critical areas of the application, which require extensive or repetitive GUI-based testing. Companies typically attack this problem in one of two ways: either a team of test engineers follows the core team by one sprint to automate tests for new functionality or, more commonly, automation resources focus on high-value areas prone to regression and core functionality that has matured to a point at which changes are rare.
Maintenance of a large suite can be costly, especially in applications that change frequently, as automated tests are tightly coupled to the UI. It is important to deploy this methodology strategically, preferably with a well-architected solution designed for ease of maintenance.
Automated QA Testing execution typically takes longer than unit testing due to the overhead of the client (usually a browser) and should be performed prior to a build’s release to a QA or Release Candidate (RC) environment.
Load Testing is commonly performed through simple, automated tests executed in parallel.
Manual Testing:
The final stage of testing involves time-honored manual tests performed by humans, rather than computers. Once all Automated Testing has been executed and passed, manual testers ferret out the last of the cases that were missed or left for later automation and create the appropriate defects for triage and remediation.
Well-written steps for defect reproduction are critical at this stage, as is incorporation of a test for the remediated defect into the unit and automated testing phases in order to protect against future regression.
