Handling multiple-step production processes like order-to-cash in an ERP system can be a hefty task that may affect user experience and even disrupt business operations due to potential defects or miscalculations. This is why it is crucial to have processes such as regression testing in place that aim to help mitigate that risk.
Regression testing allows companies to check that everything still works as expected after changes, such as updates, have been applied to software. There are two types of regression testing: manual and automated. The automation process uses test scripts, workflows and other processes within a regression testing technique. Meanwhile, manually performing regression tests involves human oversight and adaptability but can be time-consuming and prone to error.
Knowing the difference between the two and selecting the type of testing right for your business can be beneficial. This is shown through the case study of CertainTeed, a manufacturer of light and sustainable building products for both commercial and residential construction, who opted for Original Software’s automated regression testing platform in its search for a streamlined and efficient testing experience.
In the search for process stability
Marc Croquette, senior IT director at CertainTeed, joined the company around seven years ago while it was undergoing a major SAP transformation. At that point, the organization already had a team that was supporting its SAP journey, who were preparing for the upcoming change management and handling overall maintenance. These consultants were spread across different countries, including the US and India which, according to Croquette, allowed room for regression error on a manual front.
Undergoing a major transformation requires continuous changes in the system which is exactly what CertainTeed carried out – on a weekly and quarterly basis, the business was pushing for small and big changes to support the SAP transformation. Croquette explained that guaranteeing the business process stability was the first priority for the team. This was due to the fact that CertainTeed’s 4,000 business users needed to be able to log onto the system with no unexpected changes that could cause operational disruption.
When the company began performing tests on various business functions, such as the order-to-cash process and planned to install testing and insurance policies for 250 scenarios, expecting to spend a typical two to four hours on each, it soon realized that it instead took over a day to complete just a single one.
“So, multiply the complexity [of the order-to-cash process] by the number of scenarios and you get an idea of what we would have to do every time we were pushing a change to the system to make sure nothing broke,” Croquette said.
The realization of the infeasibility of this process, alongside the need to cater to user convenience, led CertainTeed to seek a solution in automated regression testing.
When running an ERP system, don’t leave your bags behind
Aiming to test the system en masse, the company required a solution that didn’t follow the conventional manual rules of regression testing. As Croquette explains, instead of having a manual process where you “replay a scenario, push that button, click here, do that, enter that value here and then play the process”, the team wanted to find a system that would provide them with a simple interface where resulting changes could be clearly displayed.
The system was described as similar to a “bag claiming process” by Croquette. “It’s almost like the bag claim that you have at the airport, where all the passengers are boarding off the flight and they go […] to pick up their bags.”
“If at the end of the process there are some bags left on that belt, then we get an issue – something [will go wrong because] everybody was supposed to pick up their bag. Same thing here. The analysts are coming up and picking up [the] differences [that were made in the system].”
After doing its research and conducting a request for proposal (RFP), CertainTeed came upon Original Software, a provider of automatic software testing products and services. Croquette explained that Original Software “was the only vendor who had 85 percent of the [software that CertainTeed] was looking for” and for the remaining 15 percent, the company promised to help however it could. This is how a seven-year partnership – and counting – began between the two organizations.
CertainTeed took their collaboration as an opportunity to build an SAP competency center with a “dedicated team of four.” Establishing a team that oversees business information and data to provide expertise for project or program support was important to Croquette and the team. This was as it would allow it to continue “enriching” its use of Original Software’s testing platform, building upon more tested scenarios each month. Croquette told ERP Today that this was especially crucial because of the ongoing mergers and acquisitions (M&A) transaction cycles that the organization was going through.
“That means that we’re onboarding new companies every three months to the SAP template,” Croquette added. “We started six years ago, with two [team members] in India, and we wanted to make sure that [would] be the competency center.
“[…] Since then, we grew it to four people and expanded the scope to more than just SAP, including a transportation management system. […] But what’s really important is that it took us barely a couple of weeks from the time we concluded the RFP and signed the contracts to the time we had the [competency center], which we’ve kept enriching [since].”
Securing the sleep insurance policy
Since the partnership launch and establishment of its competency center, CertainTeed and Original Software together achieved what Croquette calls a “sleep insurance policy” due to automation helping to prevent defects and errors when changes are pushed “between Thursday and Friday.” This allows Croquette (and his entire team) to sleep easy and not worry about any breakages coming his way by the end of the week.
As an example, the company has prevented over 20 critical defects and tests around 300 ensured scenarios on a weekly basis for the mainstream processes for six different business units. An additional benefit is that users don’t have to manually build hard-coded regression testing scripts themselves – instead, they can simply screen-record their process.
He added that the company is “enriching the insurance policy” by bringing in tools for regression, as tested on customer portals and the billing platform to ensure that the end-to-end process is untouched and CertainTeed can continue “delivering [its] commitment to business process stability.”
With Original Software’s automated no-code regression testing platform, users can keep up the testing framework with any business change. This way, they have breathing space away from thinking about a particular step in the daily production process.
Similarly, with a system that can automatically point out defects, it is possible to prevent an issue or error before it goes live, thus preventing any loss of efficiency, revenue and customers.
Opening up the world of opportunities
Looking to the future, CertainTeed is planning to continue leveraging Original Software’s automated regression testing when it “replaces its entire HRIS and payroll systems,” transforming the former in entire North America and the latter across the US and Canada. As Croquette explained, these systems cannot afford any hiccups as they support over 18,000 employees, so the “tremendous change” is scheduled for January 2026, with the company extending its automated regression testing insurance coverage.
CertainTeed’s decision to choose automation over manual regression testing proved to live up to the organization’s needs and goals, opening doors for even more ambitious projects and digital transformations. So, next time you’re lying in bed worried about potential breakages, perhaps automated regression testing is one solution to help you sleep easily.
