Migrating from manual to automated QA

Introduction

Quality assurance, or QA for short, is the process of proving that a product meets its requirements. We do this by testing the product to demonstrate and provide evidence that the stakeholders’ original requirements have been successfully implemented in a solution that delivers the expected business benefits. 

Overview

QA testing involves both manual and automated processes. Manual testing is most common, as it allows for quick turnaround. Whether at the early stages of a new product being developed or when adding to or improving an existing solution, the manual approach enables QA staff to quickly design and update tests as needed when requirements change, especially when those changes happen suddenly or at the last minute.

That said, automation also plays a valuable role in any comprehensive QA strategy. 

Let’s face it, QA testing involves a lot of repetitive tasks. These tasks are crucial, but they may not be particularly interesting, especially when performed ad infinitum. Nor do these tasks necessarily require a great deal of human expertise. In other words, they are well-suited to automation. If you’ve got a QA process that involves repetitive tasks, that’s the best place to start with automation. If you cannot easily and rapidly identify what needs to be automated, then stop and figure that out first. It may be helpful to hire a company well-versed in QA automation to help you through your migration process.

Introducing automation to the QA process requires careful thought and planning, because it necessitates the creation of large sets of instructions. Whether you do this through coding or via a helpful wizard, it still takes time — often longer than the product development effort itself.

Surely writing up a few tests can’t take that long, you may think. And if automated QA does take so long, why bother?

Let’s answer these questions one at a time. Planning automated QA to be implemented by a machine is time-intensive because it requires creating specific, detailed instructions for every validation or assertion that can be accomplished with a glance by a manual tester. Furthermore, these tests must cover a variety of positive and negative scenarios and incorporate all the supporting data. In short, designing and implementing an automated QA solution is a development effort in and of itself. As such, it needs to follow the same best practices used in any development process.

Although there are many off-the-shelf tools and boilerplate solutions available for conducting automated QA, they are rarely ideal for a particular purpose. To truly realize the benefits of automated QA, you need  an automation strategy that will support growth and change, allow for ease of maintenance, and be as future-proof as possible.

Considering test automation

Now that you have a sense of the complexity involved in automating QA, let’s talk about when and why it’s worthwhile. And let’s be clear, it’s not always worthwhile. There are instances when automating your QA process can be extremely beneficial as well as cases where it may not be a good investment of time and resources. And, of course, there is the huge space between those two extremes, where automated QA will likely pay off, but only in the long run.

This is why adding automation to your overall QA strategy should be a gradual process. Sure, you want it done yesterday, but unless you set aside a sufficient amount of time to do it right, you may never realize a return on your investment and all your efforts may be wasted. 

First, you’ll need a roadmap of immediate and long-term goals for adopting QA automation, defined and agreed to by your key stakeholders. This roadmap will be different for each organization, but there are a few basics that every roadmap should include. You’ll need to answer the following questions:

• When do we want to get this done?
• How will we accomplish it?
• Who will be responsible?
• What tool(s) will we need?
• What language(s) will we use?

To start seeing immediate benefits from your automation efforts, you’ll need to answer some additional questions. These questions will guide you in selecting the best candidates for automation from among your current units under test, commonly called UUTs. These selection criteria are intended to be platform-, technology- and solution-independent.

• What are the most stable parts of the UUTs? When we talk about stability, we are looking for functionality that is less likely to have bugs than other potential candidates.
• How often do the UUTs change? Change refers to an item’s maturity. If it is still undergoing development or there is a high likelihood that its behavior will change in the future, then it is not a good candidate for automated QA.
• Are any of the targeted UUTs considered core functionality? As a key component of your business’s overall solution, core functionality is obviously important to the business. It is usually one of the first UUTs to be implemented and may already be in use.
• Do your current manual processes involve any simple, extremely repetitive tests? As we discussed earlier, these are the easiest processes to automate.

When testing new solutions, stick with a manual process. Regardless of whether they represent core functionality, new solutions are not good candidates for automated QA because they tend to not be stable, are subject to significant change, and tend to have large numbers of defects. For all these reasons, it can be difficult to maintain an appropriate automation suite for testing a new solution.

Once you have implemented an automation solution for UUTs that meet the above criteria, you can add it to your continuous integration plans, so the automated QA processes run each time there’s a build. That’s a handy benefit, because now the resources that would have had to spend their time running these tests can focus instead on testing new solutions.

Challenges and benefits of QA automation

No new approach or process is without its challenges. Looking at QA automation and how it fits within your organization is no exception. But after overcoming the challenges, you are likely to realize significant benefits. The following lists of challenges and benefits should give you a good idea of what to expect when undertaking QA automation. 

Challenges

• Investing the necessary time.
• Identifying a starting strategy or expanding on the existing one.
• Setting realistic expectations.
• Deciding when and why to automate.
• Determining what and how much to automate.
• Choosing tools. You’ll need to decide whether to go with off-the-shelf tools, customize open source tools, or build everything from scratch, keeping the cost of each approach in mind.
• Deciding whether to hire experienced specialists or train your own resources.

Benefits 

• Enables you to move toward continuous integration/continuous deployment.
• Frees up resources.
• Reduces costs.
• Offers flexible, scalable smoke and regression test suites.
• Provides accurate, repeatable execution.
• Saves time, enabling you to get more done.
• Can be bundled and deployed.
• Provides a foundation to build on as you move toward nonfunctional test automation. 

How Prodigy can help

Whether you’re new to QA, looking at how it fits within your organization’s overall development process, or a seasoned QA veteran, Prodigy Labs can help.

Here are just some of the ways Prodigy Labs can help you with your QA processes:

• Putting best practices in place.
• Helping to start a QA team
• Assisting existing teams to improve on current practices and approaches to manual and automated testing.
• Adding automation to your QA arsenal.

Prodigy Labs assists clients with their quality assurance practices. In many cases, this is a multiyear, multistage endeavor that touches on technology, process, and — most importantly — people. Contact us today to see how we can help you strategize, plan and implement a transformational QA program.