[Dev Infrastructure] Testing, QA, and Triage

Testing

Automation is at the core of good testing. If tests aren’t automatic, they are not going to get run. The way I’ve set up all of the projects I’ve had a hand in at our company is to add testing to the Continuous Integration pipeline. Whether it’s as simple as making sure everything compiles and the bare server code runs properly, or API tests that ensure expected input and output are achieved, to simulated browser tests with tools like Selenium, all of these get run every single time someone submits code to a branch. Branches are blocked from being merged until all tests have passed and the code has undergone peer review.

One thing that I wish I had done earlier, but haven’t been able to get around to for Epochtalk is documentation-based API tests. Since we’ve got documentation generators based on code comments, it would be really nice to have a hard, almost mathematical, link from what API docs say the API call will do and what it actually does. The overall idea there is to minimize chances for human error to cause a discrepancy between. In any case where a human error can be made, that’s a potential for a bug to be generated and that in turn is a potential for hours to be spent fixing instead of implementing.

QA

I haven’t worked on a team that uses a formal QA process for a while, but from what I remember from my previous job, the QA team does the hard work of making sure everything works as it’s supposed to. It was a lot of repetitive tasks and generating “bad input”, which is part of the bug spotting process. If we can give the QA team a way to generate input, whether it’s algorithmically or through templating, it reduces wear-and-tear on them and gives them a way to consistently do a good job and find new bugs quicker. The overarching idea here is that tests are hard to write as an afterthought, but an entire suite of them is easy to obtain if they’re written in tandem with development. When I was assigned to write tests for an existing untested monolithic project at my previous job, it was such a daunting task - not an experience I’d recommend!

Cooperation between QA and developers is difficult to achieve. There can be a lot of friction due to the fact that QA might think developers are incompetent because of the errant dumb mistakes we can make, and similarly, developers might think QA is incompetent because they might lack the technical expertise required to explain an issue well. One solution I can think of to aid in the cycle between QA and developers is to allow QA to create behavior driven tests. If they find a bug and create a test for it that assumes it’s fixed, they can then push that branch as a “fix” branch. That branch containing the test can only be merged once the tests pass and the bug is fixed. That doesn’t quite solve the mutually implied incompetence issue, but it does give both of them common ground to work off of.

Triage

How does triage work? It’s at the intersection of testing and QA, and it’s a problem that every dev team has. What do we fix first? How do we tell what’s the most important and what can be worked on later? What about all of these less important tasks that have been lying around for months? As a developer on an open source project using open source technologies, I’ve seen so many different solutions. One team in particular caught my eye; they immediately closed all submitted issues (strange to me, as it reduces visibility, but understandable) and had a voting system for closed issues that would tally up how many people had the issue. They would use the voting information to decide on task importance, and whether to work on them or not.

I’ve used so many triage/tasking tools in my career including Jira (plus Atlassian suite), Kanban, Clickup, GitHub Issues and a plethora of others I can’t even recall at this point. My favorite for our small team so far has been Kanban. I liked the style of tasks with columns and no hard deadlines. It gave me a sense of satisfaction to be able to drag a task over to the next column and watch it go from in-queue to in progress to review and finally completed. I personally believe that deadlines stunt creative growth and stifle innovation by restricting perspective and reducing an individual’s propensity to be intrinsically motivated.

In terms of intrinsic motivation, I want to add a little piece of opinion; food for thought perhaps… I’ll tie it back in to triage at the end:

Supervisors and managers tend to get in that position by being micromanagers. Being able to demand set deadlines for a feature or bug fix is of utmost importance to them, even when those tasks are not critically important. They see micromanaging as being productive and helping, but it can have quite the opposite effect. My ideal strategy, aligning with best practices of the new era of creatives, involves nurturing people's intrinsic motivation to maximize their potential for creative solutions.

This means relaxing supervision and looking at the results rather than the progress; a bit like the analogy of watching a pot of water boil. If you hire diligent, creative people who enjoy what they do, the job will get done; and if you want solid results, take care of them and they’ll take care of you. Psychologists and business analysts have all but proved it time and time again, the new workforce is about creativity and not labor. You can make a laborer work faster by “whipping” them for slow progress or rewarding them for good results, but you can’t make someone think harder or be more artistic by doing the same thing. If you’ve ever tried to force yourself to think, you'll know that it doesn’t work that way.

To tie it back in, I’ll talk about my experiences with task delegation and deadlines, which are key vulnerabilities to micromanagement. It’s my opinion that, if a feature or bug is not important or blocking, it can wait a while. I’m not claiming that this is correct, but I think it is a key factor in reaching a team’s full potential. With task picking, developers can choose what to do. When we’re allowed to work on what we feel is important, we can put more dedication into it. Speaking from a small team perspective, when tasks and deadlines are dictated by someone who lacks a perspective of the overall picture, that can greatly reduce motivation to get things done. I’ve been forcefully moved from critically important tasks to ones that should have taken a back seat, and the effect is so jarring.

The pinnacle of triage to me, to be philosophical for a bit, is the ability to let go. Some tasks are simply not worth doing; they will take much more time and effort than they will return in terms of benefit. Talking to friends in the medical field, in a dire situation the goal is to save as many people as possible, but also to prioritize those who can be saved. As much as it hurts to say “no” to a task, sometimes it’s the right call. Of course, medical situations and software development differ in urgency, so it follows that the perspective shifts accordingly.

Communication and understanding are key; to make the right decisions, we need the big picture as well as the small details. The overall point I’m trying to make is that there is no one solution to triage. It’s a constant effort of balance and rebalance, and the team has to work together on it. Those who understand this and are able to communicate well with all parties involved should be the ones making the call on what to do next.