Describe, Don’t Script: A New Mindset for Testing AI

SealingTech’s new Zepharis™ Software Suite helps military and commercial operators work more efficiently and effectively in disconnected, high-challenge environments,

Typical software testing entails specifying exact inputs and expected outputs. However, that approach falls short with non-deterministic AI which can deliver different outputs in different runs even with the same input. General randomness, probabilistic decisions, or multiple execution paths can all contribute to the unpredictable behavior of non-deterministic models.  

As a Systems Engineer and QA Automation Lead for SealingTech, I experienced non-deterministic AI while testing our new Zepharis™ Software Suite. To manage the AI’s unpredictability, I realized that I could describe the desired outcome along with the element I was looking for simply by using mabl agents. The agents also worked with unlabeled user interfaces (UIs). 

At this year’s Agentic Testing Conference: Experience, taking place October 20-22 in Boston, Massachusetts, I’ll share my experience in greater detail and demonstrate how this same descriptive approach can validate AI outputs when no fixed answer exists and locate UI elements with no IDs or labels—capabilities once thought impossible to automate.  

Agentic Testing Automation 

At SealingTech, our team once relied on manual ad-hoc testing for our internal tools. As we transitioned to more customer-facing applications, we needed a solution that would ensure the quality and user experience of our tools kept up with the speed of development. Knowing our developers were already at capacity; we sought a platform that could automate complex scenarios quickly without becoming a maintenance burden. 

We decided to collaborate with mabl to establish an agentic testing foundation that scales quality by allowing QA and developers to focus more on adding and testing new features and spending less time on rework and bug fixing. We experienced the following results: 

• 86% Reduction in Testing Time: Testing with mabl greatly reduced manual and scripted approach times, while maintaining complete coverage on new features and executing their regression suite. 

• Complex UI Automation: Successfully automated testing for SealingTech’s Kit Deployment Software, a product featuring complex drag-and-drop network diagrams that was traditionally difficult to automate. 

• Avoided Specialist Headcount: Enabled the existing team to handle the test creation, execution, and maintenance while enabling faster velocity—all without the expense of needing additional personnel. 

• 4x Faster Test Creation: Test creation cycles were reduced to just 15 minutes, compared to 60 minutes (and multiple reworks) using traditional methods. 

Read the complete case study: SealingTech: Securing the Federal “First Impression” with Agentic Test Automation. 

Building Quality in from the Beginning 

Zepharis™ Kit Deployer enables you to install and configure complex infrastructure faster and more efficiently in hours rather than days, ensuring rapid system readiness in air-gapped environments.

When we build software at SealingTech, it’s important that we build quality in early rather than play catch-up at the end of the development cycle. Our platforms need to handle high-risk, difficult-to-automate scenarios all within our team’s current headcount.  

Mabl’s agentic models helped us meet those goals, enabling us to increase product velocity and innovate faster. We also were able to identify bugs and abnormalities before they reached production, ensuring we maintain the highest standards of quality for our customers. 

I hope to see you at the Agentic Testing Conference Experience October 20-22 in Boston, Massachusetts. My presentation takes place October 21st at 11:15 AM – 12:00 PM in the State Room, 60 State Street in Downtown Boston.

To learn more about SealingTech’s Zepharis Software Suite, request a demo.