Do you think it's factually incorrect that the HN comment section is more likely to find problems which invalidate the conclusions of the paper than the journal-driven peer review process?
On reflection, I probably agree that the answer is "yes" to the question as I phrased it. I think that if you take a random paper, the peer reviewers probably do have much more useful feedback than HN would.
However, if you limit the question to "papers which make bold conclusions of the type that generates lots of discussion on HN", I think HN will be more likely to find methodological flaws in those papers than the peer review process would. I think that's mostly because papers are optimized pretty hard not to have any problems which would cause them to be rejected by the peer review process, but not optimized very hard to not have other problems.
Which means, on average, I expect the HN comment section to have more interesting feedback about a paper, given that it's the sort of paper that gets lots of HN discussion, and also given that the author put a lot of effort into anticipating and avoiding the concerns that would come up in the peer review process.
Which, to a reader of HN, looks like "a lot of peer-reviewed papers have obvious flaws that are pointed out by the HN commentariat".
I do think, on the object level, a pre-print which the author intends to publish in a reputable journal will be improved more by fixing any problems pointed out by HN commenters than by fixing any problems pointed out by peer reviewers, and as such I think "post the pre-print on HN and collect feedback before peer review" is still a good step if the goal is to publish the best paper possible.
This is a considerably more thoughtful comment and I appreciate your reflection. I also can see how my initial response was a little broad and over-generalizing. I do think there is an interesting conversion in there about whether a group of technically minded people outside the "in group" of the peer reviewer circle (of whatever paper in question) could offer different and potentially important feedback.
Although I should add I have no background in academia and don't feel prepared to have that discussion.
