State of DevOps Testing Statistics (2021)
Nov 26, 2020
Are you interested in knowing the state of DevOps for 2020 and 2021? We have carefully vetted and curated the top DevOps testing stats in an up-to-date list below.
Click the navigation below to jump quickly to any section:
- Job Functions
- Deployment Frequency
- DevOps Tools
- Test Automation
- Pain Points
- Customer Satisfaction
- Key Takeaways
State of DevOps Testing
“DevOps teams spending less than 10% of their QA budgets on test automation are trailing their peers. Most companies allocate between 10% and 49% of their overall QA budget to test automation related expenditures. (Kobiton-Infostretch)”
These are interesting DevOps stats we think you should know…
Nearly half of DevOps members have 16 or more years of experience. (Accelerate)
DevOps members with more than 10 years of experience from the U.S. and Canada tend to earn over $100K in annual salary. (PractiTest)
Only 10% of women make up DevOps teams. (Accelerate)
81.8% of companies surveyed have test teams of 2-25 people. (Kobiton-Infostretch)
37.9% of companies surveyed had annual revenues of $10-100M. (Kobiton-Infostretch)
One-quarter of testers move into testing from other departments. One-fifth of them migrate to the position by accident rather than traditional education. (PractiTest)
Most continue their education of testing by “just doing it,” utilizing testing books, or attending conferences, meetups, and seminars. (PractiTest)
Taking a look at the job functions among QA testers…
Other than QA, testers spend time doing other things like; test data management, test and development environments, documentation, test coaching & consulting. (SauceLabs)
Testers feel excluded overall, they want more inclusion in the whole process. Meaning, they want test plans introduced from the design stage. (SauceLabs)
In a survey by Kobiton-Infostretch regarding the global impact of Covid-19 on mobile QA; 44% of respondents believe the first priority of investment should be in the culture embracing what it means to be a remote team.
Testers feel their jobs are safe. Nearly 59% are not concerned about job security. (SauceLabs)
The stats below illustrate the current state of deployment frequency…
Nearly a quarter of teams ship weekly (23%), while more than one-fifth ship monthly (21%). This in contrast to fourteen percent who ship bi-weekly and seventeen percent who ship quarterly. (Mabl)
In the mobile app world specifically, weekly and monthly releases hold true as well. Yet the proportions are higher with 34.8% releasing weekly and 33.9% releasing monthly. (Kobiton-Infostretch)
Faster release cycles lead to happier customers, which has a direct correlation to the statistics above. The sweet spot seems to be weekly or monthly. This is also true for bug fixes found in production. As the industry moves forward, we see signals of transitioning towards faster daily to weekly release cycles. (Mabl & Autify)
22% of companies at the highest level of security integration have reached an advanced stage of DevOps evolution. (Puppet)
What tools are DevOps teams using, and are they satisfied with them?
Most DevOps teams use Selenium WebDriver for testing. We all know the maintenance challenges with this tool. (Mabl, Kobiton-Infostretch & Autify)
GitLab, Jenkins, and CircleCl are considered the top CI tool choices among DevOps teams. (Kobiton-Infostretch and Mabl)
Jira is by far the top choice for issue tracking according to DevOps stats. (Mabl)
While most teams use Bug tracking tools like Bugzilla, Jira, and Redmine to manage tests, surprisingly, 47% don’t use any dedicated test case management tool at all. However, these teams use a combination of Excel, Word, and email. (PractiTest & SauceLabs)
More than a quarter of respondents to a survey state “finding the right tools” as being the biggest barrier for entry to test automation. (Kobiton-Infostretch)
Here are some DevOps testing statistics that will blow your mind…
Most DevOps teams choose “improving product quality” and “time to market” as the top reasons for moving towards test automation. (Kobiton-Infostretch)
Most respondents indicated that their companies allocate between 10% and 49% of their overall QA budget to test automation related expenditures. (Kobiton-Infostretch)
Companies with more than 500 employees are 2.5x more likely to spend over 75% of their entire QA budget on test automation. (Kobiton-Infostretch)
When surveying mobile app developers, we found it takes teams on average 1 Day to 1 Week to update automation scripts for a new app release. (Kobiton-Infostretch)
It takes most QA testers on average 5-24 hours to code a test case using a framework of their choice. What if you could cut this time drastically by eliminating the need to “code” tests and let artificial intelligence handle the coding? (Kobiton-Infostretch & Autify)
Most testers run between 100-249 manual test cases with each mobile app release. This leaves plenty of tasks an automated tool can take off the hands of humans. Furthermore, most testers spend on average 3-5 days manually testing mobile apps before every release. (Kobiton-Infostretch)
Test automation presents a paradox… Organizations seek to release fast. Ideally daily or even weekly. However, it takes 1-3 days to initially write test cases, followed by another 1-day through 2-weeks to update automation scripts with each release. This makes daily or weekly releases incredibly challenging. Despite this paradigm, the ROI behind test automation is compelling! Since a large percentage of QA budgets are spent on automation, combined with the challenges of keeping up with the speed of releases, it shows automation is not cheap or easy. Though it is a necessity for innovation and a leap towards modern release frequencies. DevOps teams spending less than 10% of their QA budgets on test automation are trailing their peers. Seeking an easy test automation solution anyone on your team can use? Try our no cost demo today! (Kobiton-Infostretch & Autify)
It is important to note the top challenges for DevOps…
Regardless of company size, all organizations agree that the biggest struggle to start test automation is evaluating and choosing the right tools. (Kobiton-Infostretch)
The second biggest automation pain is “training/acquiring skilled automation engineers,” thus highlighting the inherent complexity in developing test scripts. To alleviate this pain, there are many no code test automation tools in the market. To help your organization choose the best tool, here is our roundup of the top 5 test automation tools. (Kobiton-Infostretch & Autify)
Is there a correlation between testing tools and customer satisfaction?
Teams with the happiest customers are not the happiest with their testing tools. (Mabl)
Half of DevOps teams are not satisfied with their testing tools. 71% search for new tools several times per year. (Mabl) If you are seeking a superior testing tool. Give Autify codeless AI-powered test automation tool a try!
We hope our analysis of the State of DevOps Testing reports are helpful in understanding the path and growth of the testing industry. We have discovered some key findings:
- The testing industry is stable, even in the shadow of the Covid-19 global crisis.
- Most teams ship weekly and monthly, however, they are striving for daily and weekly releases.
- Finding bugs quicker and having faster release cycles lead to happier customers.
- Most organizations are not complacent with their testing tools, thus searching for alternatives several times per year.
- Many DevOps teams use Selenium for testing, however, there is a large movement migrating towards no code test tools.
- Test automation factors a large portion of the modern QA team’s budget.
Lastly, the state of DevOps is holding strong and growing. Even in the face of Covid-19, teams have quickly adjusted, are more resilient by reprioritizing sprint objectives, and have a higher tolerance for bugs in the interest of speed of delivery. Aspiring towards modern goals to reach such as daily release cycles, identifying obstacles such as entry barriers and tooling, plus department heads noticing ROI benefits of test automation thus allocating massive portions of their budgets towards it.
Are you a part of the community? We would love to hear your opinions on the state of DevOps. Contact us to open a discussion!
Kobiton-Infostretch Covid-19 Global Impact on Mobile QA
Kobiton-Infostretch Test Automation 2020 Survey
SauceLabs Continuous Testing Benchmark Report 2020
PractiTest State of Testing Report
Accelerate State of DevOps Report
Puppet State of DevOps Report
Mabl Landscape Survey 2020