03 Jun Jon Cartu Report: Council Post: Software Testing Survival Tips
Desperate times call for desperate measures. In times of crisis, especially when social distancing is so prevalent, software can keep people connected, and as such, software vendors are competing with greater effort to win and keep customers. With higher competition, there is less room for error, especially when it comes to software quality. Any software bug today can have enormous damage on a company’s revenue and brand.
I’ve been speaking with many software executives every day, and I’d like to share with you some of the most common situations that executives are discussing with me.
I have to deliver 100% testing with only 25% personnel.
With the recent budget cuts, many companies have had to let go of large portions of their development teams. Still, you must do test automation (TA) as much as possible. The only way to utilize the time of your remaining team is to maximize their productivity per hour.
Software test automation is the practice of automating test procedures of the application under test (AUT). There are two main testing aspects: functional and performance.
Functional testing is oriented to validating AUT errors triggered by its functionality — primarily front-end (UI and API). As an example, clicking the login button in the AUT should navigate to the welcome page.
Performance testing involves validating overall performance and scalability of the system under load (SUL) by measuring end-to-end transaction time for data passed from the SUL front end to the back end and the sustainability of the SUL per number of users. Going back to our original example, a login transaction can take one second for 1,000 users.
My testers don’t have test automation experience.
Many QA teams are staffed by manual testers with no test automation or code writing experience. As painful as it might be for many leaders at this time, there’s no time to learn or experiment. Now is the time to reallocate your salaries toward personnel who can produce test automation — and consider hiring testers with extensive test automation experience.
We’ve lost budget on test automation in the past.
Many companies have tried to implement test automation over the past few years and have ended up spending more time on test script maintenance than regression testing. You have to be very focused on the test automation process to win the game. Make sure you are choosing the right test automation tool that supports your applications under test and your software development life cycle (SDLC). In many cases, new TA teams either jump into maximizing ROI, or they are afraid to invest in TA tools until they see the results. Often, these teams find free TA tools that are not a good fit for their team’s skill set, the AUT, or the development or quality assurance schedule.
We’re already doing TA and are falling behind.
Many companies have been doing TA for the past few years, but with fewer TA engineers, they can’t catch up with the regression testing schedule. If you are in this same boat, you need to look into your TA architecture. The key of high-ROI TA is the minimization of artifacts (i.e., the number of TA scripts, the size of TA scripts, the lines of code (steps), cross-platform portability, and the shareability of TA foundation components such as data sources, functional libraries and test objects. For example, one line of TA code costs, in my experience, one minute of script maintenance, depending on the TA tool. Chosen architecture managers can estimate the time required for TA maintenance.
Our AUT requires cross-platform testing with more scripts.
Often, AUT requires testing on multiple browsers and operating systems. Some companies believe there is no way they can sustain their current TA volume with 25% of their staff. For these companies, I’d suggest choosing the TA tool that offers a single, cross-platform test execution with one script. This way, you only need to develop one script instead of four or more. I’d also suggest you redesign your TA framework to be modular, event-driven, data-driven and rich-logic. This will allow you to encapsulate all functional business logic into a single test procedure that will be called into procedural tests with passing test data.
In the end, there are three main tenets you should take away from this dicussion:
- Automate your testing processes to optimize your team productivity.
- Choose TA tools wisely to match your team’s skills and your development schedule.
- Design TA architectures in a concise way to minimize ongoing maintenance.
Lastly, remember, you need to always be testing!