We’ve talked a bit about testing early and often. However, STLC in DevOps should not stop once the product passes QA. Continuously testing software in production is just as vital!
Shift Right refers to actions (such as testing) later in the software development life cycle. We should continuously test on production to catch bugs. If found, it makes rolling back to previous versions easier and lessens adverse effects. Just because the software and features pass quality assurance, we still must rigorously test the production environment. Why? “Software testing in a test environment is exactly that – a test environment,” explains Sheri Byrne-Haber, an Accessibility Architect at VMware. She elaborates, “It isn’t live data, and nothing stresses a system like use by real users. You can’t copy the production environment into a test environment without lots of security (plus permission from the owners of the data). Even then, stuff like this happens.”
There should be a separation between test data and production data. This is why both datasets should be tested. Byrne-Haber explains, “You can’t mix test data in a production environment because stuff like this happens.” She refers to a weird test article that accidentally went live in the New York Times and has since been retracted. Moreover, there is the fact that users can (and will) create totally unique use cases your team may not have thought of. “There is just so much that your users might be able to do that designers, developers, and product owners never thought of as valid use. But once your users have done it, that data is there forever, and you have to figure out how to make it work.”
This is why ZOZOTOWN tests their production application hourly using our no-code test automation tool. With easy-to-use tools like Autify, anyone on the QA team can handle test automation. No more relying on highly skilled test automation engineers- they can be freed up to focus on building software.
Test Automation is Paramount
Another important aspect of DevOps is automation testing. A time-saving avenue in software testing is shifting towards automation versus manual testing. It helps drastically speed up the delivery of superior software products. Though you should not completely eliminate manual testing, as it has its benefits. Testing with automated tools should make up the majority of test cases, while manual testing should be the minor. However, test automation benefits far outpace those done manually. Those include:
- Saving money, time, and resources
- Avoid the redundancy of manual efforts
- Execute more test cases at scale and improving test coverage
How, and why would you implement automation testing before the traditional QA stage? How you may ask… by making the leap and necessary investments in test automation. According to stats, those software development teams that do not make the investment in automation testing trail in comparison to competitors. That same study revealed that most companies allocate between 10%-49% of their overall QA budgets towards test automation expenditures. The aforementioned points justify the return on investment.
Test automation isn’t cheap, or easy for that matter but it’s a necessity for modern development. According to a survey, 71% of respondents search for new automation testing tools several times per year. One of the top complaints is maintenance at scale. Solving this issue solves the inflated man-hour problem in quality assurance.
Furthermore, investing in test automation leads to better software, which in turn, leads to happier customers. The study previously mentioned also revealed that faster release cycles also lead to happier customers. And happier customers retain longer thus accelerating revenue growth.
Communication & Collaboration with All Stakeholders are Key