Manual testing practices hinder continuous delivery goals
QAD Redzone is a developer of cloud-based manufacturing and supply chain solutions. The company is on a mission to transform manufacturing by empowering frontline employees to make decisions in real time, helping its customers improve productivity and employee engagement. It currently serves 1,200+ manufacturing plants worldwide—including some of the largest manufacturers in the world—with 70,000+ monthly active users..
QAD Redzone’s development team values delivering new features and keeping up with the pace of innovation. However, its legacy testing practices could not keep up with business growth, and proved to be a bottleneck in its continuous delivery efforts. Despite having a highly efficient team, they were unable to strike the balance between the breadth and depth of testing needs. “We were finding defects in key workflows of the system that were impacting customer satisfaction,” says Ronald Davis, senior vice president of Product Engineering.
These testing challenges had consequences for the QAD Redzone engineering team and the business. Manual testing and validation processes were time-consuming, draining valuable resources and slowing the company’s development cycles. “In our effort to continuously deliver and increase release cadences, these manual efforts became increasingly unsustainable, creating bottlenecks in our CD pipeline,” says Davis.
They were also seeing a lot of false positives. “Our manual snapshot testing approach was prone to flaky and fragile tests, resulting in an overwhelming number of false positive alerts,” explains Davis. “This alert fatigue led to a lack of trust in our testing practice, as it became increasingly difficult to distinguish between genuine issues and false alarms.”
Without a comprehensive end-to-end testing solution, the company lacked visibility into the performance of its applications, which hindered its ability to identify and address defects proactively to improve the user experience. “Our testing practices were holding us back from truly embracing continuous delivery,” says Davis. “It became evident that we needed a more proactive and efficient approach to synthetic testing for our mobile applications.”
A testing solution that works right out of the box
Davis and his team evaluated multiple testing solutions but found many of them fragile. “Every little code change was breaking everything, and the engineers would spend more time trying to fix the test than actually fixing the code. We couldn’t make the tests work effectively or consistently,” Davis explains.
The company was already using several Datadog frontend monitoring products, and decided to try Datadog Synthetic Monitoring for end-to-end mobile testing for its core application functions. “One of the key things we love about Datadog is it allows us to create no-code synthetic tests— significantly reducing the barrier to creating end-to-end tests,” says Davis. “Our SRE easily set up no-code tests to monitor main customer flows which, if broken, would severely impact usage and revenue.”
“Our SRE easily set up no-code tests to monitor main customer flows which, if broken, would severely impact usage and revenue.”
The scalability of the platform was another key benefit. “I really like how Datadog scales with our business,” adds Davis. “As the complexity of our stack has grown, Datadog has been able to continually grow with us.”
The no-code approach also freed up engineers to focus on scaling, new features, and refactoring. “Because we’re a small engineering team, we just don’t have a lot of people or time,” Davis notes. “It was fast and easy to bring Datadog Synthetic Monitoring online. It was about getting value out of something really quickly.”
Accelerating release cycles with confidence
With comprehensive end-to-end synthetic testing, QAD Redzone transformed its continuous delivery pipeline, increasing release frequency from 3 to 4 releases per year to 2 to 4 releases per month. “A key piece of our continuous delivery process is the confidence we now have, allowing us to go faster to drive better product quality and improve team efficiency,” says Davis.
“A key piece of our continuous delivery process is the confidence we now have, allowing us to go faster to drive better product quality and improve team efficiency.”
By catching issues earlier in development, the team has also significantly reduced defects. “It’s reduced our SEV1 incidents by 90 percent,” explains Davis.
The solution has also minimized costs because they can now identify and resolve defects before they reach customers, avoiding the substantial costs associated with addressing issues in production environments. “We have not seen any disruptions to key features since deploying Datadog Mobile App Testing,” adds Davis. “This cost reduction has translated into significant savings, enabling us to reinvest resources into further enhancing our products and services.”
“This cost reduction has translated into significant savings, enabling us to reinvest resources into further enhancing our products and services.”
Ultimately, the improvements QAD Redzone has made ensure the company can continue to help its customers improve productivity and employee engagement. “By ensuring the stability and reliability of our mobile applications, we’ve empowered our frontline workers to interact with our solutions seamlessly, enhancing productivity and operational excellence across their organizations.”
