FROM THE INDUSTRY
is reasoning about a world where these problems do not exist.
September and you will drown in AI claims, ours included. “Does it use AI?” stopped being a useful question some time ago. Everything does, or says it does. Better questions: how much AI, where, and who decides? Can the same platform run a test at 100 percent, at 10 and at zero, depending on what the test is for? Is every AI decision tied to evidence from a real device? Can you see the cost of a run before you see the invoice? Ask those, and you will probably land where our customers have landed: AI available everywhere in the platform, and running in 10 to 20 percent of the execution. Engineering, unlike a demo, is the art of using exactly as much of a powerful thing as the job needs.
update prompt, an account verification that nobody asked for, a promotional overlay. The agent clears the obstruction, records what happened, and hands control back to the script. Each recovery also leaves the device clean for the next test in the queue, so one incident does not cascade into fifty false alarms overnight. That is 1 to 2 percent AI by volume. The effect on operations is out of all proportion, because the alternative was never “no AI”. The alternative was a person triaging a wall of red results every morning, most of them environmental noise rather than product defects. At that level of usage, the annual AI cost of a hundred real devices is a small fraction of what it costs to hand- maintain error handling for every possible interruption.
Real devices also make the AI accountable. Every agentic decision is recorded against video of what actually happened on screen, with the agent’s reasoning preserved step by step. When a test fails, evidence always points to timestamped footage, and there is no question of judgment as far as the agent is concerned; it is on the screen for you to see. A+E Global Media measured what this changes in practice. Their QA team used to spend most of its time rebuilding image captures after every user interface (UI) update. With agentic execution they ended up with roughly a hundred times fewer captures to maintain, and moved that effort into new test coverage instead. They shared the results themselves, in a public webinar with us. AI in this case did not take anyone’s job or even eat away at someone’s job decription; it actually absorbed tasks that freed the team up to better spend their time elsewhere.
Real devices keep the AI honest
www.witbe.net
None of this works in simulation, by the way. The failures that matter in video only exist on physical hardware: the 2018 television that behaves nothing like the 2024 model from the same brand, digital rights management (DRM) that renders wrong on one chipset, the firmware update that quietly breaks the programme guide. An agent reasoning about a simulated app
The question to ask in Amsterdam Walk any hall at the International Broadcasting Convention (IBC) this
84
SEPTEMBER 2026 Volume 48 No.3
Made with FlippingBook - Online magazine maker