It can be harder to justify in private tooling where you might only have a few dozen or hundred devs saving those seconds per each invocation.

But in public tooling, where the benefit is across tens of thousands or more? It's basically always worth it.

Obviously effort vs reward comes in here, but if you have 20 devs and you save 5 seconds per run, you save a context switch on every tool invocation possibly.

This is true, but I think the other side of it is that in most shops there is lower hanging fruit than 5 seconds per tool run, especially if it's not the tool that's in the build / debug / test loop but rather the workspace setup / packaging / lockfile management tool.

Like, I switched my team's docker builds to Depot and we immediately halved our CI costs and shed like 60% of the build time because it's a persistent worker node that doesn't have to re-download everything every time. I have no association with them, just a happy customer; I'm only giving it to illustrate how many more gains are typically on the table before a few seconds here and there are the next thing to seriously put effort into.