I have some sympathy for people who are blindsided by surprising difference between a new tool and their old one.
This post is not eliciting sympathy. They're data consultants, who don't understand a very basic and fundamental aspect of the tool that they're using and recommending. If you're a consultant you have a responsibility to RTFM, and the docs are clear that LIMIT doesn't prune queries in BigQuery. And, also, the interface tells you explicitly how much data you're gonna query before you run it.
This post is also blaming Google rather than accepting their own part in this folly, and event admits the docs are clear on this matter. Cost-control in BigQuery is not difficult. One of the tradeoffs of BQ's design is that it must be configred explicitly, there's no "smart" or "automatic" query pruning, but that also makes it easier to guarantee and predict cost over time as new data arrives.
Yes the whole consultancy situation really is the icing on the cake - as the customer you pay for (alleged) experts in the field and get this as the result...
Wouldn't people who knew their tools perfectly well not even use a cloud service like BigQuery? At the level you expect them to use the tool, they could have created a big query engine themselves. Isn't the whole point of these tools is to make things easier?
Sorry but that's nonsense. Partitioning is THE central cost controlling mechanism in BigQuery and the docs clearly state this. And it's an easy to use feature, so I'm not sure what makes you think using that would be as challenging as building your own query engine.
