Pure Integration, LLC

The Challenge

A leading multiple system operator that was launching a wireless data card offering encountered time-to-market challenges due to extensive order brokering and provisioning testing during the integration test phase.  Each functional and software configuration release required approximately 20 business days to certify and deploy to the production environment. The client spent approximately 435 work-hours per release in regression testing, which consisted of manual execution of hundreds of test cases. While there were a relatively small number of test transactions, each one featured dozens of minor variations and permutations. As the business expressed a need for shorter release cycles, pureIntegration identified the opportunity to assist by developing a test automation platform to streamline the integration test process and reduce time-to-market.

The Solution

Following a comprehensive review of the client’s application and an end-to-end analysis of its system, pureIntegration designed a suite of automated tests that were accurate, effective and easy to maintain. The pureIntegration team identified the targeted test transactions for automation based on a combination of level of effort to automate, amount of time for manual execution, and stability of the test transaction. For each transaction, the team then developed a test template that performed the following tasks:

• Execution of the test case against the application under test
• Verification of the data in the database against expected values
• Verification of application message logs and responses from web services queries against expected values

Each test template enabled the team to quickly create dozens of automated test cases, which were all minor variations of one another.

HP’s QuickTest Professional (QTP) was used to code the automated tests, and HP’s Quality Center was used to store the automated tests, launch them, and store the results of test runs. The team made heavy use of QTP’s descriptive programming feature because the application under test was still in development and undergoing minor changes to the user interface.

The Business Benefits

pureIntegration created nearly 200 automated test cases, which the client could queue up and run sequentially with no human interaction. Furthermore, the test templates enabled the client to easily create additional test cases that were not originally part of its regression test set.

Each automated test case delivered the following benefits to the client:

• An automated test case needed approximately 30 minutes to run whereas manual testers needed 90 minutes to execute the same test.
• Database, application message log, and web services verifications performed faster and more consistently.
• Detailed results of test runs, including screenshots and all verifications, stored HP Quality Center.

For each software release, the automated tests reduced the regression test cycle by approximately 120 work-hours, which gave the client additional time to execute progression tests.