Continuous Delivery ensures that new codes are delivered as fast as possible. With the fast delivery, product quality should not be sacrificed. And to achieve that goal, testing plays a crucial role. For any code to work correctly, testers need to test the functionality of the codes. Different types of software testing help to achieve that objective. But remember not one shoe fits all. Similarly, different codes are meant to do different functions, so we need different types of testing to check the code functionality. This article will look at different kinds of software testing and its role in the CI/CD pipeline.
When we talk about different types of software testing on a high level, we think of manual testing and test automation. Manual testing is done manually where a tester has to create a test environment, author tests, execute them manually and then analyse the reports / root cause. Needless, to say it is expensive, time-consuming, and error prone because the tester has to verify and validate all the functionalities and features manually.
On the other hand, in test automation, a tool runs previously written test scripts which are coded using the automation tools. It is robust and more reliable. However, the quality of the test automation depends on how well your test scripts are written. Automated testing is the foundation of the CI/CD pipeline and plays a crucial role in scaling your QA process. But we need to keep in mind here that manual testing and test automation are not interchangeable but complementary to each other. While automated testing is the key component for CI/CD, manual testing is necessary as well for certain types of tests as we will see later in the article.
Unit Testing (At the code-build level)
Unit Testing is done by the developers at the code-build level of the application. Once the developers have done the coding, they conduct the Unit Testing to check the sanity and working of the build they have coded. These are basic testing done for individual functions and classes and are very easy to automate. A CI pipeline should have automated Unit Test that can be run quickly during continuous integration. This helps developers with a first-hand review if their code has broken any Unit Tests.
Integration Testing (Continuous integration level)
Integration Testing ensures that different modules are well integrated and functions the way they are supposed to do. For example, consider a coder is coding for app login, and another coder is coding for app sign-up. In that case, Integration Testing ensures that both the functions work correctly when integrated into the app. In CI/CD pipeline keep Integration Testing as a check to the developers if their code has broken any of the Integration Tests. For example, if the integrated build fails to pass a certain threshold, let’s say 85%, then integration is rejected with individual developers notified about the failure. This automated testing in the pipeline will ensures a certain level of quality of the code is achieved before publishing. If the code fails in the test, it is much easier to fix it at this level than failing at the deployment level.
End-to-end Testing (Continuous integration level)
End-to-End Testing tests the entire application from a user perspective by simulating the real user scenario and validating the system under test and its components for integration and data integrity. End-to-End testing offers expanded test coverage in the CI/CD pipeline and ensures correctness of the application. It also reduces the time to market, facilitating continuous delivery. It is useful to testers too because they can verify real-world user behaviour and how they can avoid achieve greater sense of accomplishment.
Regression Testing (On-going)
As more companies are adopting agile approaches for software development companies need to roll out new features at an accelerated pace. Regression Testing plays a crucial role here in ensuring that the application remains functional even with the new features amended. As each new amend has the potential to introduce new bugs, frequent regression testing ensures that the bugs are caught quickly and rectified without delaying the release. Due to its repetitive and predictable nature, regression testing is the ideal candidate for test automation. By automating Regression Testing, organizations can save resources, run regression testing 24/7, create instant feedback loops and reducing overall testing time and improving quality, enable scalability, and lowers the maintenance cost.
User Acceptance Testing (Pre-production level)
User Acceptance Testing determines whether a system satisfies the user needs and business requirements. Testers need to get into users’ shoes and conduct User Acceptance Testing to check if the application is meeting all the business requirements. Testers also need to measure the application’s performance and reject changes that do not meet the business requirement. Unit tests are typically automated as part of the Continuous Integration, but organizations still struggle to automate integration and acceptance testing. You can’t automate manual testing. At this stage, manual testing in form of exploratory testing should be carried out to save time and efforts that otherwise would have been spent on manual activities.
As you improve different features in your app, you need to do rigorous testing to ensure all the functionalities are working correctly. Exploratory Testing comes into play here. It is more random and works on unscripted sequences to find any errors. Different test management apps can help in Exploratory Testing. For example, QTM4J offers an Exploratory Testing add-on that empowers users to conduct Exploratory Testing while exploring and investigating the site.
Software Testing is crucial to achieving your Continuous Delivery pipeline. While test automation can save your time, effort, and cost, there are specific tests where manual tests must be done. For example, End-to-End Tests, Unit Tests, Integration Test can be automated, but Regression Testing must be done manually.
Disha Thakkar _QMetry_Marketplace Partner
This month the spotlight is on AppLiger. We caught up with Pavel Pavlovsky, CEO and Product Manager, to learn how the company started and what fuels the team's creativity. Atlassian:...
Connect with like-minded Atlassian users at free events near you!Find an event
Connect with like-minded Atlassian users at free events near you!
Unfortunately there are no Community Events near you at the moment.Host an event
You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events