Yes it is, especially at scale. This is most obvious in the cloud where zero interest rate era companies blow ludicrous amounts of money running comically inefficient code. The cloud bills at some of these companies are hilarious.
Even on the desktop hardware is a constraint if you are trying to do anything that pushes the boundaries of the medium, which is why endpoint-first AI is only possible thanks to hyper-optimized code like llama.cpp.
I hate hearing this but I know it's true because I've tried to build my career as a software engineer around minimalism, simplicity, maintainability and efficiency at scale but it has constantly felt like a struggle; that there is no market for that. You're totally right in pointing out that a lot of valuable software is comically inefficient. Sometimes it almost seems like only bad, ridiculously inefficient software is allowed to succeed financially in this system. Freaky. I've literally seen projects with horrible code be worth billions and witnessed their market cap drop over time as the code quality improved. I don't know WTF is going on in this industry but it's weird.
Even on the desktop hardware is a constraint if you are trying to do anything that pushes the boundaries of the medium, which is why endpoint-first AI is only possible thanks to hyper-optimized code like llama.cpp.