Still waiting for these software factories to solve problems that aren't related to building software factories. I'm sure it'll happen sooner or later, but so far all the outputs of these "AI did this whole thing autonomously" are just tools to have AI build things autonomously. It's like a self reinforcing pyramid.
AI agents haven't yet figured out a way to do sales, marketing or customer support in a way that people want to pay them money.
Maybe that won't be necessary and instead the agent economy will be agents providing services for other agents.
With so much hype it's a valid question: "is this useful/practical, or just a fun rabbit hole/productivity porn". Money is the most obvious metric, feel free to inquire the parent about other possible metrics that might be useful to others instead of asking rhetorical questions.
I wonder if this is just a byproduct of factories being very early and very inefficient. Yegge and Huntley both acknowledge that their experiments in autonomous factories are extremely expensive and wasteful!
I would expect cost to come down over time, using approaches pioneered in the field of manufacturing.
For those of us working on building factories, this is pretty obvious because once you immediately need shared context across agents / sessions and an improved ID + permissions system to keep track of who is doing what.
I’ve been building using a similar approach[1] and my intuition is that humans will be needed at some points in the factory line for specific tasks that require expertise/taste/quality. Have you found that the be the case? Where do you find that humans should be involved in the process of maximal leverage?
To name one probable area of involvement: how do you specify what needs to be built?
Your intuition/thinking definitely lines up with how we're thinking about this problem. If you have a good definition of done and a good validation harness, these agents can hill climb their way to a solution.
But you still need human taste/judgment to decide what you want to build (unless your solution is to just brute force the entire problem space).
For maximal leverage, you should follow the mantra "Why am I doing this?" If you use this enough times, you'll come across the bottleneck that can only be solved by you for now. As a human, your job is to set the higher-level requirements for what you're trying to build. Coming up with these requirements and then using agents to shape them up is acceptable, but human judgment is definitely where we have to answer what needs to be built. At the same time, I never want to be doing something the models are better at. Until we crack the proactiveness part, we'll be required to figure out what to do next.
Also, it looks like you and Danvers are working in the same space, and we love trading notes with other teams working in this area. We'd love to connect. You can either find my personal email or shoot me an email at my work email: navan.chauhan [at] strongdm.com
This! It’s both-and. Literacy has been undeniably good, but we rarely consider the consequences of widespread literacy.
There’s a way of knowing something that can be recalled orally from memory that is different and valuable. But we even measure it using a yardstick for written knowledge (accuracy, breadth, etc).
I believe this overemphasis on written knowledge (really, it’s implicitly a denial that any other type exists) is part of what drives the hysteria about LLMs ending the world. LLM doomerism has to believe that written knowledge is at least the most important if not the only necessary form of knowledge.
Aside: it also helps for code review! Review bots can point out the diff between plan and implementation.
Some examples for the curious: https://github.com/sociotechnica-org/symphony-ts/tree/main/d...
reply