I think it's a lot more nuanced than this. Defense can be perceived as threatening and cause others to increase their defense, as per Balance of Threat theory and the security dilemma, creating a feedback loop (arms race) that leads to a lose-lose situation, primarily for the weak, but also eventually for the strong.

I am not advocating against defense spending as a category. I am saying it needs to be done skilfully and as a last resort, with the understanding that it is only coherent in a world without a unipolar security architecture, and is therefore hopefully temporary.