Google is an ad company first. It stands to reason than their results optimize for ad profitability, not to have the best index.
If the goal was to have the best index, they would actively punish sites that use SEO to push sites that are nothing more than auto-generated affiliate links to Amazon products. I quit using Google a while ago, because that’s all I was getting. I use Kagi now, which had no ads. Without this conflict of interest, I find things go better.
I think it's reasonable to assume that google would favor the Ad Sense customer given two results that are otherwise of equal or near equal quality but the notion that google would exclude good results from domains that do not serve Ad Sense ads is absurd.
Why is it absurd? If it's still considered the go-to option by most users, they could afford the slightly worse quality for the extra revenue, no? That's the end result of monopolies, the provider starts to compromise on the service to capitalize on the user's lack of viable options.
The product is more than just the index! The UI has become cluttered with ads that are barely distinguishable from genuine results. The functionality around using quotation marks for exact-match results has now become broken and confusing.
This is the key.
I'm not saying Google couldn't be better. But if it were possible to build a better index of the Internet, Google would do it.
Spam garbage overrunning the web is the underlying issue.