The drive toward using agile as a development methodology that recognizes that testing is an integral part of development, not a separate phase, has transformed the software industry.
The trend of quick development iterations rather than long cycles supported by smaller teams of six to nine versus 50-plus people has changed the entire business model of software development. No longer do you need $10 million and a VC to back you. No longer are you developing for a year before releasing the product. No longer are you constrained by a software packaging model that requires people to download the software or go buy it at the store.
We’ve also seen many changes in the planning and management of the development cycle. As teams learn and adapt to agile, customized implementations in the process will further impact the industry.
One of the biggest changes is the drive toward short iterations. Quick iterations every two weeks makes a huge difference in how the project is managed from a QA perspective. With this approach, you’re biting off small pieces to develop, test, and deliver rather than focusing on one big release. Rather than attack a big software release, a few features, functions, or parts of functions at a time are the focus.
Once you demonstrate that your goals are met and defect-free, you can move forward to the next iteration. This also gives you a chance to reprioritize as you discover new requirements and learn new nuances about the work in progress. The objective is to get things done with higher velocity.
The industry has coined the term “shift left,” but what does this mean for testing? The short answer is automation, but what kind of automation? What must change in automation to accommodate agile methodology?
Before agile (the Agile Manifesto was released in 2001) when almost everyone used the waterfall development model, testing exercised the software’s functions in entirety including the user interface, but usually in a distinct test cycle at the end of the development cycle. Today, after each iteration, you have access to a feature or user story to test, but that’s it. Of course, you also have access to all previous iterations’ functions and features developed. Sometimes, you don’t have a user interface, or you have a user interface that is morphing and changing because the client may see something and want to change it. To speed up the development cycle, teams are also using a lot of third-party modules or functions so they don’t have to reinvent the wheel for noncore product functionality.
As shown in the figure above, that’s where API testing comes in. You can still test functions, but you don’t necessarily want to or have to test the UI. Instead, you want to call the function with various input ranges and check the expected output. You don’t care if it’s aligned on the right bottom corner of the form. You just want to check the returned output. And the same goes for any third-party plug-ins or modules you use. You want to exercise them the same way, providing input and checking the expected output. This way, you are not dependent on the UI to do our testing.
In an agile environment, it’s very likely that API’s, plug-ins, and modules are being broadly used, and they still need to be tested at each iteration because you’ll continue to develop functionality dependent on them. While you could rely on testing from past iterations, there will be calls to the API with different parameters and data depending on the functionality delivered. Because you’ll test the API on a repeated basis and changing the parameters, there are significant benefits realized through test automation. There are many tools to do this. Choosing the right tool as well as determining when to automate and when not to automate requires a certain degree of expertise and understanding, as with any form of test automation.
In addition to automating functionality testing, there are also the issues of performance and security that can be addressed with API testing as well. Like functionality testing where you test the expected outputs given various inputs, you can ramp up these calls to the API just as you would test performance with the user interface and vary the user scenarios. This will put strain on the various program components involved and serve as an independent assessment of the program and data logic apart from the user interface. For security testing, you can embark on the same or similar strategies that you use when executing security tests when the software is done, but at the data or API level. Making erroneous and black-hat calls to the software and determining how it behaves and blocks access.
API testing and test automation
With waterfall development models came a lot of manual testing as the QA team didn’t get the software to test until late in the game, leaving insufficient time to develop much automation. In some cases, automated test scripts were used for the major release and for subsequent releases to make sure no defects creeped into the software when fixes or new features were completed. The challenge was and still is to maintain the test scripts and balance that against their future usefulness. For example, you’d develop some Version 1 test scripts and pray that you could use your scripts for Version 2. Depending on the software design and the vision of product management, you must understand how to parse and structure the scripts carefully to ensure their later usefulness.
Now with agile, you’re testing features and functions as they are developed; in many cases the UI isn’t final. You can’t possibly test everything for every iteration, but you certainly must test enough to ascertain if the current build is good enough to progress to the next stage.
How do you get proper coverage with limited time? Automation! Next, determine what to automate and what not to automate. You’ll have to gauge the maturity of the user interface, the importance of those functions that have been delivered, and the reusability of the test scripts. That’s where automated API testing comes in handy. You can test the basic functionality of the software without the user interface. When the user interface is in final form and mature, you can then write UI based automated scripts.
So, as you can see, with agile, we still need to automate testing. In fact, you won’t be able to survive and keep up without it. Especially as more iterations give you more functions and features to test, you won’t be able to handle the regression manually. The key issue is to be judicious in the types of automation you use, when, and where.
This article is published as part of the IDG Contributor Network. Want to Join?