Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

There are just so, so many problems with these benchmarks. (This comment from the author of Servant, a Haskell framework, nicely sums up a lot of the criticism: https://www.reddit.com/r/haskell/comments/6b3dlt/techempower...) But cherry picking the single query benchmark in particular makes the issue worse. Like... how many web apps do you stand up where you have one client you want to serve? At least the "multiple requests" benchmark is sort of plausible for the kind of workload you'd want to handle.

In most cases, though, you care a lot more about the 95th or even 99th percentile response times than you do the mean; I'd trade quite a bit percentage-wise off my mean response time if it brought the 95th percentile way down.

(This isn't to hate on PHP; PHP is fine. But you shouldn't choose it because the "single query" benchmark says it's fast.)



While I agree that benchmark are somehow problematic, many of the critisms in the Reddit comment are pretty void once we do not targets a browser or theory. When I have an IoT device I do not care about content negotiation or the content type. I send my stuff up using some HTTP I use.

Is it fair in the comparison? Nope. Comparing oranges with apples is always bad. The question however is whether you want an Apple or an Orange?

TechEmpower has to be read like that: plaintext ensures the HTTP server stack and the base runtime. Json is about json encoding and emitting. Single query and multiple query about streaming data through. The benchmark is gradually testing your platform. .NET Core was bad at Plaintext. Now they are top 10. Now they realized the default json library (Newtonsoft.Json) of .NET is very nice but too slow. So they built a new one (System.Text.Json). Now the PostgreSQL Driver sucks. So they find and support the driver project. Like that you built a platform which is top notch.


> Like... how many web apps do you stand up where you have one client you want to serve?

I think you confused "single query" with "single request". The goal of the benchmark is to make one query to the DB per request. The number of requests (clients) is in the hundreds of thousands.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: