DevOps Test Automation: Enhancing the Agile Development Cycle

Nowaday, Agile is one of the most popular approaches to project management due to its flexibility, adaptability to change and high level of customer input. The way Agile methodology manages a project by breaking it down into several phases. At each stage, the release version can be sent to the stakeholders to track the current status. What the stakeholders really need to know is the quality of the release version. How many features have been deployed ? Or which one passed or failed ?… The test report can clearly show it to the internal and external teams. This article will introduce how Devops works on automation tests to ensure the quality of the project.

Introduction to testing

Software testing is a method to check whether the actual software product matches expected requirements. During the software development period, testing can help to detect errors, gaps or missing requirements in contrast to actual requirements. What the test team does is define the test case for the requirement features to make sure that the response from source code matches with the expectation.

Testing practices typically involve the following stages:

Unit testing: validates individual units of code, such as a function, so it works as expected
Integration testing: ensures several pieces of code can work together without unintended consequences
End-to-end testing: validates that the application meets the user’s expectations

As a DevOps engineer, I am more focused on automation testing in this article.

Test automation is the practice of automating the review and validation of applications under development by the testing team. In the CI/CD pipeline, the test team does not wait till the end to test the product quality. Continuous building and testing of the product go on till it is branded as fit for market release. Test automation in DevOps empowers businesses to build, test, and ship products faster and more reliably. It further supports streamlining incident responses and improves collaboration and communication across teams and stakeholders.

Why DevOps and Tester should work closely together

Following agile practices, software development life cycle will be divided into smaller phases to release products and get frequent customer feedback. This later drove the adoption of tools that enable continuous integration and continuous delivery to automate the build, test and deployment processes. However, key functions such as development, testing and delivery to production were performed by separate teams that operated within their own silos.This caused inefficiencies and slowed down the software development lifecycle. That is why DevOps will be the person in between to push the release process faster. Thanks to the test cases that are defined by test team, DevOps will integrate it in the CI/CD pipeline. This pipeline will automatically trigger when any new version of source code is deployed. Then, the test scripts will be run and a test report will be generated.

Thanks to collaboration between DevOps and test team, they bring some huge value to the project.

Test team no need to run automation script to get the test report at any new deployment anymore. So they have time to focus on writing test cases for each release.
Automate the execution of tests as part of the continuous integration and continuous delivery pipelines. This ensures that tests run automatically whenever changes are made, catching defects early in the development cycle
Collaboration between development, operations and testing teams to ensure that automated tests provide valuable feedback and contribute to the overall quality improvement.

How Devops handle automation test

This is software development life cycle of devops integration. In this module, the test phase won’t separate from the other phases any more. Each time a new version from the development team is pushed or deployed, the test phase will automatically run and generate a report. Then the test report can be downloaded or displayed on the dashboard with some tools which depend on DevOps integration. Following this approach, the development team can track the current code status if errors exist they can fix immediately. Moreover, stakeholders can keep track of the current status of the new feature through the report. There are some key points that DevOps need to do to integrate test in software development cycle.

Collaboration

Collaboration is a culture of DevOps which means they need to work closely with the team to release the product to the customer as soon as possible. A lot of projects follow Agile methodology, It means that all the changes have to be quickly adopted. Related to testing, If any changes such as automation test framework, test environment, test report visualization,… Both sides need communication to make sure there is no interruption in release process.

Guideline

Depending on framework and programming language utilized, they have their own execute command to generate test report. That is why DevOps need knowledge transfer from Tester to know how to get test reporting and integrate it in pipeline later. On the other hand, If test team does not have much knowledge about CI/CD tools, DevOps needs to show them how to trigger a pipeline and check the test report. I want to emphasize test team, because there is not much value if only the test lead can track the current status of testing.

Prepare environment

DevOps need to ensure provision and configuration for infrastructure, tools and resources for testing. DevOps need to deploy test environment, so that automation test can be run on it before moving forward to stage or production environment. Sometimes, test team needs access to the database server for checking data inside it or make up some data for testing purposes. So they will raise it and DevOps will deploy some tools for them to access. Another case is when testing process occurs, a lot of api call to get data from database servers, so devops will configure the database server to increase the number of connections. There are lot of configures and set up tools for executing test, that is why I want to emphasize collaboration one more time to adapt all requirements related to the environment efficiently.

DevOps test automation sample workflow

In order to show automation test integration in software development life cycle, I will choose Azure DevOps which is end-to-end software development platform as an example.

Test planning

Test planning should be the first step which is done by test team. They will define all features, test cases need to be executed in this sprint. Below is an example board with two test plans “Sample” and “Sample_1” in Doing column.

After defining all test plans and test cases for the Sprint, test team will start to write test script for it. During this time, DevOps will setup environment through CI/CD pipeline and integrate test in it.

CI/CD

Below is a pipeline that I integrated both unit testing and Integration testing. Unit testing will be defined by dev team to check functionality of individual units and to ensure that they produce the correct output given. Thanks to Unit testing, dev team can check their code coverage and track how many percent of test case have been passed. Then, the environment fThanks to Unit testing, dev team can check their code coverage and track how many percent of test cor integration test will be deployed. Integration test stage in CI/CD pipeline will execute test script and generate report.

Test result Analysis

Test result are analyzed, identifying failures or issues, and generating detailed reports with relevant metrics, such as pass/fail status, code coverage and performance.

Unit Test report

Integration test report with SpecFlow

Update automatically update status on board

Thanks to these reports, both internal team and stakeholders can check the status of the current process.

Conclusion

In conclusion, DevOps automation testing is a crucial aspect of software development and deployment processes. It helps organizations achieve faster release cycles, improve software quality and enhance collaboration between each team.

Would you like to read more articles by Tekos’s Team? Everything’s here.

References

Author

Harry

Harry is a DevOps engineer at Tekos Interactive. Interested in software development and infrastructures, Harry decided to focus his energy on learning and adopting new valuable technologies that could benefit projects. Sharing is also an important set of practice in the DevOps culture, reason why he enjoys sharing his experiences and knowledge with the community.