This is what keeps getting me. People here keep posting benchmarks, bragging about 5x, 10x, 20x. None of the companies we work with are putting anything faster.
The evangelist response is to call it a skill issue, but looking around it seems like no one anywhere is actually pushing out new products meaningfully faster.
Appreciate this deep insider insight that... someone somewhere might someday use AI to make some good software that might be bad for SaaS platforms that also fail to improve, maybe.
Holy hell this is so uselessly vague. Honest take from someone on the inside indeed.
But most of the things you're describing as bad can also be described as a lack of soft skills. The person who can't take care of themselves and rolls over under pressure definitely has poor soft skills.
Not necessarily. The end result of googling a problem might be copying a working piece of code off of stack exchange etc. without putting any work into understanding it.
Some people will try to vibe out everything with LLMs, but other people will use them to help engage with their coding more directly and better understand what's happening, not do worse.
Eh, quite the contrary. A lot of anti AI people genuinely wanted to use AI but run into the factual reality of the limitations of the software. It's not that it's going to take my job, it's that I was told it would redefine how I do work and is exponentially improving only to find out that it just kind of sucks and hasn't gotten much better this year.
People keep saying stuff like this. That the improvements are so obvious and breathtaking and astronomical and then I go check out the frontier LLMs again and they're maybe a tiny bit better than they were last year but I can't actually be sure bcuz it's hard to tell.
sometimes it seems like people are just living in another timeline.
You might want to be more specific because benchmarks abound and they paint a pretty consistent picture. LMArena "vibes" paint another picture. I don't know what you are doing to "check" the frontier LLMs but whatever you're doing doesn't seem to match more careful measurement...
You don't actually have to take peoples word for it, read epoch.ai developments, look into the benchmark literature, look at ARC-AGI...
That's half the problem though. I can see benchmarks. I can see number go up on some chart or that the AI scores higher on some niche math or programming test, but those results don't seem to actually connect much to meaningful improvements in daily usage of the software when those updates hit the public.
That's where the skepticism comes in, because one side of the discussion is hyping up exponential growth and the other is seeing something that looks more logarithmic instead.
I realize anecdotes aren't as useful as numbers for this kind of analysis, but there's such a wide gap between what people are observing in practice and what the tests and metrics are showing it's hard not to wonder about those numbers.
I would really appreciate it if people could be specific when they say stuff like this because it's so crazy out of line with all measurement efforts. There are an insane amount of serious problems with current LLM / agentic paradigms, but the idea that things have gotten worse since 2023? I mean come on.
The evangelist response is to call it a skill issue, but looking around it seems like no one anywhere is actually pushing out new products meaningfully faster.
reply