Yes this is what I'm interested in. I think they might be right. I'm already finding myself thinking "well maybe a classifier would be useful here now that it's so easy to do...".
This probably just means that I could have been reaching for that tool more often already. But in practice I wasn't, and this has opened my eyes to the potential opportunities there.
I guess I'm circling toward this view. The question is, are there things that are 1. worth doing, 2. for which jev (or jev-like systems) works well, and 3. are not worth the effort to train a custom classifier. Probably yes, but it seems like it might be a pretty narrow path. But a lot depends on #2. The trade-off between #1 and #3 is less stark the more successful one shot models are at handling use cases successfully.
Scripts and debugging, one-off log parsing or filtering.
I saw an article about 2+ years ago of a researcher using a small local AI strapped into excel to evaluate the abstract and intro of 10000 papers for "papers that research X in domain of Y", and let it loose.
jev is probably more capable avd faster than that workflow was, but saved one dude a few very grindy weeks for a litteratur review.
It's amusing how long it took, and much hype it gets for someone releasing the least revolutionary ML architecture in a new package. But i can see a fair few uses.
Surely those industries are not relying on humans writing "good code" to make sure faults are not introduced, right? There are static analysis and automated testing and rigorous QA processes, surely?
I think too many people (very much including me) who are "passionate about the craft of software" allowed themselves to become too focused on the code itself as if that were the first order concern of the craft. But the craft has always been about the quality of the software, including how its quality changes as a function of time and adaptation. Code quality is only in support of that concern, it is not itself the primary concern.
It has been a fairly painful experience for me to shift my thinking on this, but it's a much better mindset. I still care about many of the same code quality concerns I always have, but I'm thinking a lot more about why I care than I once did.
I think that we should realistically expect that even a very rigorous testing and validation process is not fool proof, just as we anticipated that the code isn't.
I don't think my comment implied that any testing and validation process will be fool proof. There are two claims in the comment I replied two, first that people are creating "more bugs than ever". I take that to mean more bugs per line of code or more bugs per unit of functionality or something like that. If that's the case, then I think it implies that testing and validation has gotten worse per that same unit, which I think suggests that it could be improved, without expecting it to become fool proof. Then there is a causal claim, that these bugs are because people are not thinking about the code. Maybe so, but to me, it seems more likely to be caused by inadequate validation.
I think we'll probably get both kinds of companies, but that they will both be fringe approaches, and the predominant choice will be to have humans working with AI tools.
It seems like "luddite" is a reasonable description for a strict "NO-AI" policy. You softened it to "not wanting to hand off all your labor" in this comment, but the word "any" would be a more apt way to describe the quote in the comment you replied to.
Ah yeah, light bulb moment here. I do think the disconnect I sense in reading these comments and discussing this might be in my level of expectations for how good "code quality" has ever been or even really can be in non-trivial applications.
To me, AIs seem to write code with a normal (low) level of quality, but I now have so many more tools to make sure the software works nonetheless, to refactor quickly, and to encode better practices that (mostly) stick in the future.
So to me, this is all a huge net win. But I guess if I'd ever worked on a pristine perfectly engineered system, I might see this all differently.
To me, it's always been a mess, and I'm just ecstatic that I have more ways to manage the mess now.
This probably just means that I could have been reaching for that tool more often already. But in practice I wasn't, and this has opened my eyes to the potential opportunities there.
reply