You're on your way to the next level! Join the Kudos program to earn points and save your progress.
Level 1: Seed
25 / 150 points
1 badge earned
Challenges come and go, but your rewards stay with you. Do more to earn more!
What goes around comes around! Share the love by gifting kudos to your peers.
Keep earning points to reach the top of the leaderboard. It resets every quarter so you always have a chance!
Join now to unlock these features and more
The Atlassian Community can help you and your team get more value out of Atlassian products and practices.
Continuous Delivery ensures that new codes are delivered as fast as possible. With the fast delivery, product quality should not be sacrificed. And to achieve that goal, testing plays a crucial role. For any code to work correctly, testers need to test the functionality of the codes. Different types of software testing help to achieve that objective. But remember not one shoe fits all. Similarly, different codes are meant to do different functions, so we need different types of testing to check the code functionality. This article will look at different kinds of software testing and its role in the CI/CD pipeline.
When we talk about different types of software testing on a high level, we think of manual testing and test automation. Manual testing is done manually where a tester has to create a test environment, author tests, execute them manually and then analyse the reports / root cause. Needless, to say it is expensive, time-consuming, and error prone because the tester has to verify and validate all the functionalities and features manually.
On the other hand, in test automation, a tool runs previously written test scripts which are coded using the automation tools. It is robust and more reliable. However, the quality of the test automation depends on how well your test scripts are written. Automated testing is the foundation of the CI/CD pipeline and plays a crucial role in scaling your QA process. But we need to keep in mind here that manual testing and test automation are not interchangeable but complementary to each other. While automated testing is the key component for CI/CD, manual testing is necessary as well for certain types of tests as we will see later in the article.
Unit Testing (At the code-build level)
Unit Testing is done by the developers at the code-build level of the application. Once the developers have done the coding, they conduct the Unit Testing to check the sanity and working of the build they have coded. These are basic testing done for individual functions and classes and are very easy to automate. A CI pipeline should have automated Unit Test that can be run quickly during continuous integration. This helps developers with a first-hand review if their code has broken any Unit Tests.
Integration Testing (Continuous integration level)
Integration Testing ensures that different modules are well integrated and functions the way they are supposed to do. For example, consider a coder is coding for app login, and another coder is coding for app sign-up. In that case, Integration Testing ensures that both the functions work correctly when integrated into the app. In CI/CD pipeline keep Integration Testing as a check to the developers if their code has broken any of the Integration Tests. For example, if the integrated build fails to pass a certain threshold, let’s say 85%, then integration is rejected with individual developers notified about the failure. This automated testing in the pipeline will ensures a certain level of quality of the code is achieved before publishing. If the code fails in the test, it is much easier to fix it at this level than failing at the deployment level.
End-to-end Testing (Continuous integration level)
End-to-End Testing tests the entire application from a user perspective by simulating the real user scenario and validating the system under test and its components for integration and data integrity. End-to-End testing offers expanded test coverage in the CI/CD pipeline and ensures correctness of the application. It also reduces the time to market, facilitating continuous delivery. It is useful to testers too because they can verify real-world user behaviour and how they can avoid achieve greater sense of accomplishment.
Regression Testing (On-going)
As more companies are adopting agile approaches for software development companies need to roll out new features at an accelerated pace. Regression Testing plays a crucial role here in ensuring that the application remains functional even with the new features amended. As each new amend has the potential to introduce new bugs, frequent regression testing ensures that the bugs are caught quickly and rectified without delaying the release. Due to its repetitive and predictable nature, regression testing is the ideal candidate for test automation. By automating Regression Testing, organizations can save resources, run regression testing 24/7, create instant feedback loops and reducing overall testing time and improving quality, enable scalability, and lowers the maintenance cost.
User Acceptance Testing (Pre-production level)
User Acceptance Testing determines whether a system satisfies the user needs and business requirements. Testers need to get into users’ shoes and conduct User Acceptance Testing to check if the application is meeting all the business requirements. Testers also need to measure the application’s performance and reject changes that do not meet the business requirement. Unit tests are typically automated as part of the Continuous Integration, but organizations still struggle to automate integration and acceptance testing. You can’t automate manual testing. At this stage, manual testing in form of exploratory testing should be carried out to save time and efforts that otherwise would have been spent on manual activities.
As you improve different features in your app, you need to do rigorous testing to ensure all the functionalities are working correctly. Exploratory Testing comes into play here. It is more random and works on unscripted sequences to find any errors. Different test management apps can help in Exploratory Testing. For example, QTM4J offers an Exploratory Testing add-on that empowers users to conduct Exploratory Testing while exploring and investigating the site.
Software Testing is crucial to achieving your Continuous Delivery pipeline. While test automation can save your time, effort, and cost, there are specific tests where manual tests must be done. For example, End-to-End Tests, Unit Tests, Integration Test can be automated, but Regression Testing must be done manually.
Disha Thakkar _QMetry_
Product Marketing, QMetry
Ahmedabad, Gujarat, India
1 accepted answer