CDSA

After years as an ensemble player, testing is ready for its close-up thanks to automation.

With the media supply chain under duress, and content providers struggling with heavy workloads and intense pressure to cut costs, automated testing enables the fast, streamlined global delivery that’s essential to every media company’s business model.

But first content providers need to understand what automation can — and can’t — do for their testing function.

Make no mistake that testing is crucial all through the supply chain, from the A/B experimentation that helps determine what viewers see, to the source code validation of CI/CD pipelines that ensure application code is bug-free.

Quality control of content, however, occupies a pivotal role in the supply chain. Because it tests the media files themselves, QC of content is responsible for the ultimate experience of what the viewer sees and hears.

While storytelling is the primary driver of viewer engagement, testing and validation ensures the quality of the audio/visual media.

QC of content plays a vital role in the value viewers perceive. As a result, it’s critical to media companies’ ability to compete.

Content and media testing is also the least sexy, most problematic activity in today’s media supply chain — and the area in which automation can provide the greatest advance for content providers.

Anything seen on screen can benefit from automation to check on errors, whether it’s validating the integrity of video and audio signals, ensuring subtitle graphics sync properly with the actor’s voice, or using natural language processing to facilitate translation.

As media and entertainment companies seek to become less reliant on those checks being performed by costly “eyes and ears”— the term for manual QC — they’re wrestling with how far automation’s role can go.

Many are in a continual evaluation mode as they assess the maturity of emerging technology to enhance or replace traditional methods.

Not all automation is ready for prime time out of the box: An automated check for image distortion, for example, can return multiple false positives for manual operators to sift through — requiring more of their time in some cases than watching the full show in real time.

Testing is a complex, iterative process even before you factor in automation, with plenty of potential for missteps along the way.

Because media files pass through so many hands, it’s common for QC to be run multiple times on the same files, representing redundant costs that waste time and budget.

In addition, suppliers, and receivers of content—from production companies and internal supply chains to third-party distributors — regularly tussle over their roles and responsibilities for remedying issues identified in testing.

Topics such as rejection policies, stakeholder communication and expectations, and how to design loosely coupled technical capabilities so they can be leveraged from multiple points within the chain, are all critical to supply chain management — and constant sources of industry debate and planning.

The goal of a single, comprehensive QC, performed early in the supply chain to eliminate the need for downstream validation, is universal yet still elusive due to the additional processing and customisation carried out “after-market,” or after a show delivers its initial masters from a production.

Whether the single QC-to-rule-them-all will ever be fully achieved, progress towards this goal is widely viewed as critically important, even as companies grapple with tough decisions such as which steps in the supply chain make the best fit for automation and which areas are the most advantageous places for its insertion.

THREE MYTHS — AND REALITY CHECKS — OF AUTOMATED TESTING

Successfully automating the function of content QC requires understanding what automation can—and can’t—do for testing.

Here are three myths about automated testing that can help guide companies’ efforts:

Myth No. 1: Automated testing is one and done
The reality: Some automated checks work well but new requirements are rapidly emerging, and automation takes time to catch up.

For example, subtitles are evolving into a hot topic as creators seek to distribute video content into dozens of markets around the world.

Automated testing is emerging to validate many aspects of subtitling that affect the viewer experience, from read speeds to audio sync to language translations.

Similarly, growing into a big issue for catalogue owners is forced narrative, the text overlay that clarifies onscreen communications or alternate languages. (Think of the talking rocks in Everything, Everywhere, All at Once.) The text needs to be removed before subtitles in local languages are applied to avoid colliding graphics.

For programmers with extensive catalogues, detecting the presence of forced-narrative text is the kind of expensive, multilayered problem that AI can help mitigate.

What’s more, new formats and technical specifications have the potential to impact tools, processes and costs related to QC.

For example, high-resolution formats like ultra-high definition (UHD) increase the load on infrastructure, making it more difficult to execute effectively from the cloud.

High dynamic range (HDR) introduces challenges in the ability to automate down-converts to standard formats, requiring a second master to be delivered and QC’ed for each show, a repeat of a problem that was solved years ago for HD-to-SD conversion.

Myth No. 2: Automation solves all problems
The reality: Identifying a problem isn’t the same as fixing it.

Testing finds problems but remediation is another matter — and it often involves nuance and judgment.

Is the issue that’s been flagged a problem, or is it creative intent, from a mixing decision to the look of a video filter or colour grading?

While the goal may be to solve as many problems as possible programmatically, some fixes involve editorial review by creative or studio stakeholders.

For automated testing to fulfil its potential, it needs to combine finding anomalies with ways to solve them. QC is really about performance management: Finding and fixing.

Myth No. 3: Eventually all testing will be automated
The reality: Content is touched, processed, customised and transformed many times across the course of the end-to-end supply chain, and by many stakeholders, both internal and external to a given organisation.

Maintaining quality across this complex, often fragmented set of processes is a constant challenge, involving not only the testing methods themselves but also an accessible audit trail of the checks that have been performed. (As yet there is no blockchain-backed, immutable ledger for this information to be stored with the content wherever it resides.)

While machine learning and other advances will increase the reach and effectiveness of QC automation, it’s the eyes and ears of the viewer that we are testing for, and most industry participants believe that those same sensors will continue to be utilised in the testing process, hopefully in an increasingly machine-assisted manner.

As a result of advances in automation, testing is ready to step into the spotlight. Technology, however, is only part of the change.

Successful implementation takes education, preparation, and a big dose of reality.

* By Jeff Davidson, Chief Architect, Cognizant *

=============================================

Click here to download the complete .PDF version of this article
Click here to download the entire Winter 2022 M&E Journal