Test Automation: The Key to Increasing Velocity, Quality and Efficiency

Compuware, A BMC company, used this year’s Arcati Mainframe Yearbook to tell us about research they’d commissioned from independent research company Vanson Bourne, who’d conducted a global survey of 400 senior IT leaders responsible for application development at organizations with a mainframe. They were looking at testing and automation practices on the mainframe and their impact on the speed of innovation.

They said that world’s biggest and most successful enterprises continue to run their mission – critical workloads on the mainframe. They remain committed to the platform due to its unmatched digital strengths in reliability, performance, security and transactional efficiency; as well as its decades of proven intellectual property in the form of fine – tuned business logic and data.

What’s often overlooked, they suggest, is that the majority of customer – facing distributed, web and cloud – based applications are highly reliant on the mainframe. As a result, it’s critical that organizations are able to deliver change on the mainframe as quickly as they can in their distributed systems, to support the demands for a constant cycle of ‘new’ from the business and to satisfy their always beautifully, wonderfully dissatisfied customers.

Against this backdrop, speed and innovation have become the rallying call for IT departments across every industry. The shift to Agile and DevOps has improved the pace of software delivery and has become essential to competing in the Age of Software.

However, testing remains a critical development function where manual methods, especially within mainframe development environments, are delaying software delivery and the pressure for shortened lead times are threatening quality. Test automation coupled with a “shift – left” approach—where developers write unit tests at the same time as they write source code—is not widely embraced for mainframe application development.

Further, an increase in mainframe workloads and a growing shortage of skilled professionals are increasing the need to ensure code has been thoroughly tested, which can put it at odds with the need to increase the pace of delivery.

In the report, they examined the processes that organizations have in place to support testing on the mainframe, while exploring the challenges they face in simultaneously increasing quality, velocity and efficiency. The report shows that traditional, manual testing swallows up skilled developers’ time and in turn puts pressure on teams to take shortcuts. Ultimately, it shows that the current manual approach to testing is often difficult, costly and time – consuming, and highlights the need to increase the use of automation to alleviate these pressures.

The key highlights from the study include:

• 77% of application development managers find it increasingly difficult to simultaneously increase quality, velocity, and efficiency to meet business innovation goals when developing and testing mainframe application code.
• 53% of application development managers say the time required to conduct thorough testing is the biggest barrier to integrating the mainframe into Agile and DevOps.
• 80% of application development leaders say it is inevitable that unless they can automate more test cases, bad code will make its way into production.
• 90% of application development managers say that automating more test cases could be the single most important factor in their success as the pressure on IT to accelerate innovation increases.
• Only 7% of organizations automate the execution of test cases on mainframe code to support their ability to accelerate innovation.

IT teams are under constant pressure to deliver innovation at an ever – faster pace, but many are struggling under the weight of the challenge.

77% find it increasingly difficult to simultaneously increase quality, velocity, and efficiency to meet business innovation goals when developing and testing mainframe application code.

85% say that it is becoming harder to deliver innovation faster, without compromising on quality and increasing the risk of bugs in production.

The inordinate amount of time IT spends during the testing phase is a leading contributor to this challenge.

92% said their organization’s mainframe teams are spending more time testing code than they needed to in the past, to account for the growing complexity of cross platform application environments.

In fact, development teams spend, on average, more than half their time (51%) on testing during the release of a new mainframe application, feature, or functionality. By comparison, teams reported spending 39% of their time on testing when mainframe code is not involved, still a very large percentage.

These findings clearly indicate that traditional, manual methods of testing are amongst the biggest challenges that organizations face when it comes to accelerating the speed of innovation. In particular, organizations face significant challenges when it comes to testing mainframe code.

Manually testing mainframe code also hinders businesses’ ability to integrate the mainframe into wider DevOps initiatives, thereby slowing the pace of innovation across the business.

The traditional manual style of testing mainframe code also means IT teams are often forced to take shortcuts to deliver innovation at the speed the business requires. Yet application development leaders expressed concern that this pressure to cut corners to deliver innovation faster creates numerous business risks.

Organizations see automation of the testing process as the key to their ability to deliver innovation faster while maintaining quality.

While having obvious benefits in terms of speeding up the development process, test automation also helps to reduce human error, improve quality, and increase output and productivity. It also enables key workers to focus on more value – generating activities, such as user experience design and adding more business functionality.

Looking at the specific areas where automated testing on the mainframe stands to bring the most benefit to their organizations’ ability to accelerate innovation, application development leaders rated unit testing quite low, calling out the automation of functional testing (63%) and integration testing (53%) as priority focus areas, ahead of unit testing (37%).

This is in contrast to the ethos of the shift – left approach, where unit testing is performed in the earliest stages of development to enable developers to find bugs at the time they are introduced into their code, providing them with fast feedback and at a lower cost.

Automated testing/software quality tools have been available for mobile, web and distributed application code for several years, but mainframe code has not been as well supported by test automation capabilities. It was no surprise, therefore, that 86% of respondents said they find it difficult to automate the testing of mainframe code.

Fractured and fragmented testing practices also threaten the impact that organizations are set to achieve by automating more test cases on the mainframe. Many organizations are failing to test code at every stage of the development process and don’t empower their developers to test their own code. This creates an increased risk of bugs finding their way into production and increasing the frequency of software failures.

The last few years have seen automation become a major focus, as organizations look to accelerate software innovation and sharpen their competitive edge. Automation offers significant advantages through productivity, quality and efficiency improvements, and allows organizations to redeploy resources to support higher – value tasks that drive the business forward. It’s no surprise then that this research shows a growing interest in automated testing, as organizations look for a way to simultaneously accelerate the quality, velocity and efficiency of their efforts to bring new innovation to market, without fearing that faster speed equals lower quality.

However, despite the proven benefits of test automation, the research shows that many organizations are still conducting testing manually when it comes to the mainframe. Given the central role that the mainframe continues to play in powering modern digital services, these manual testing practices are creating a bottleneck that impacts quality, velocity, and efficiency. Ultimately, this is hindering the delivery of innovation and preventing organizations from meeting their business goals.

The research also highlights that businesses expect to achieve significant benefits by automating more testing on the mainframe at every stage of the development process. Not only does this remove the need for mainframe specialists to be directly involved every time that code is being worked on, it helps to improve quality, velocity and efficiency by enabling developers to fine tune their code as they’re working on it.

Ultimately, by “shifting left” with automated testing – and unit testing in particular – organizations can gain fast feedback on the mainframe, supercharging innovation without fearing the risk of introducing problems that disrupt operations, introduce security risks, hinder customer experiences or impact business revenues. Better still, they can improve quality, velocity, and efficiency on the mainframe in spite of the growing shortage of experienced developers; giving themselves a major advantage as they look to drive a competitive advantage through supercharged digital innovation.

You can read the full article here, including how Compuware suggests its products can help.