Is there any chance you might support a local-first version of this in the future? I've been interested in apps like this and Littlebird in particular seems very attractive. But I'm loathe to essentially send screenshots/summaries/etc of all my activity to a cloud solution, regardless of any claims you make about encryption. Any mistake you make could be catastrophic for me, which thoroughly dominates any upside to using your product. It's a non-starter.
We will for sure, but the issue is that without local LLMs, there's no way to offer a truly fully local version. And the local LLMs are dumb. So basically, you would still need to trust the LLM providers. Totally understand that this is a deal breaker for some people, but for many users, the theoretical risk is worth it. We do regular security audits, encrypt in transit and at rest, pen tests, etc.
Um, dismissing the tech as "the local LLMs are dumb" seems shortsighted. I can run some pretty impressive models on my local Mac, but it has >64gb of ram and an M3 Max.
Given the privacy benefit I wouldn't dismiss them so fast. I'd suggest picking one or two that your prompts will work well with and treating it as "we let you run with local models too, if you have a computer capable of that." This will (a) quiet the people who complain about everything and (b) get more people to try the cloud model knowing they could move to a local model for real usage.
I'm not dismissing them. I'm saying they're not there yet. As a startup, we have to prioritize. We can't do everything simultaneously, and it would be a substantial engineering effort to have a dual architecture as well as potentially more security holes. And the amount of people that want to run local LLMs is very small. I use local LLMs when I'm on flights, and that is my personal assessment. They are all benchmark-maxed and incapable of reliable tool calling or consistency over meaningfully long conversations.
Hey, I'm Dmitriy from Littlebird. For your use case, would you want the underlying LLM to be local as well, so that your data doesn't get sent to the "big dog" LLM providers? It's an important consideration because it wouldn't be nearly as smart then - though I totally understand if that's the only way it could work for you.
I'm not the OP but I came here to voice the same concern. I would love to use something like this. I also signed up for rewind.ai and Limitless and pre-ordered the pendant. But ultimately I cancelled it out of privacy concerns.
I wonder if it could be local storage and you could provide your own Open Router endpoint? That way it could be a local model or your own deployment of GPT/Claude in Azure/Bedrock/Vertex etc where you can control retention policies etc.
Basically, I want to know that you guys don't have access to view my stuff. I get that that limits your ability to improve the product and support issues, but when I'm sending everything it really starts to matter. Just thought I'd share what held me back from immediately signing up despite really wanting to use a product like this!
This is what I was going for with Treebeard[0]. It is sandbox-exec, worktrees, and COW/overlay filesystem. The overlay filesystem is nice, in that you have access to git-ignored files in the original directory without having to worry about those files being modified in the original (due to the COW semantics). Though, truthfully, I haven’t found myself using it much since getting it all working.
This approach is too complex for what is provided. You're better off just making a copy of the tree and simply using sandbox-exec. macFUSE is a shitshow.
The main issue I want to solve is unexpected writes to arbitrary paths should be allowed but ultimately discarded. macOS simply doesn't offer a way to namespace the filesystem in that way.
Completely agree; my approach was not the most practical. I mostly wanted to know how hard it would be and, as I said, haven’t used it much since. Yes, macFUSE is messy to rely upon.
I feel as though the right abstraction is simply unavailable on macOS. Something akin to chroot jails — I don’t feel like I need a particularly hardened sandbox for agentic coding. I just need something that will prevent the stupid mistakes that are particularly damaging.
pnpm’s minimumReleaseAge can help a ton with this. There’s a tricky balance, because allowing your dependencies to get stale makes you inherently more vulnerable to vulnerabilities in your packages. And, critically, fixing a vulnerability in an urgent situation (i.e. you were compromised) gets increasingly harder to address the more stale your dependencies are.
minimumReleaseAge strikes a good balance between protecting yourself against emerging threats like Shai-Hulud and keeping your dependencies up-to-date.
Because you asked: you can get another layer of protection through Socket Firewall Free (sfw), which prevents dependencies known to be malicious from being installed. Socket typically identifies malware very soon after its is published. Disclaimer: I’m the lead dev on the project, so obviously biased — YMMV.
I have been an AI-coding skeptic for some time. I always acknowledged LLMs as useful for solving specific problems and making certain things possible that weren't possible before. But I've not been surprised to see AI fail to live up to the hype. And I never had a personally magical moment - an experience that shifted my perspective à la the peak end rule.
I've been using GLM 4.6 on Cerebras for the last week or so, since they began the transition, and I've been blown away.
I'm not a vibe coder; when I use AI coding tools, they're in the hot path. They save me time when whipping up a bash script and I can't remember the exact syntax, or for finding easily falsifiable answers that would otherwise take me a few minutes of reading. But, even though GLM 4.6 is not as smart as Sonnet 4.5, it is smart enough. And because it is so fast on Cerebras, I genuinely feel that it augments my own ability and productivity; the raw speed has considerably shifted the tipping point of time-savings for me.
YMMV, of course. I'm very precise with the instructions I provide. And I'm constantly interleaving my own design choices into the process - I usually have a very clear idea in my mind of what the end result should look like - so, in the end, the code ends up how I would have written it without AI. But building happens much faster.
No affiliation with Cerebras, just a happy customer. Just upgraded to the $200/mo plan - and I'll admit that I was one that scoffed when folks jumped on the original $200/mo Claude plan. I think this particular way of working with LLMs just fits well with how I think and work.
I was AI skeptic too a year ago , but recently i wanted a windows exe program to do the same as a complicated bash script on linux.
i gave the bash script to claude code, which immediately started implementing something in the zig language. after a few iterations, i had zig source code that compiled in linux , produced a windows exe and perfectly mimicked the bash script.
Your post has inspired me to check them out. How do you use it, with their UI oe to power some other open source tool?
Do you suggest that this thing is so fast its simpler now to quickly work on one thing at a time instead of the 5 background tools running in parallel which might have been a pattern we invented because these things are so slow?
I’ve been using the crush TUI primarily. I like that I have the flexibility to switch to a smarter model on occasion - for awhile I hesitated to pick up AI coding at all, simply because I didn’t want to be locked into a model that could be immediately surpassed. It’s also customizable enough with sane defaults.
For anyone considering: I applied, didn’t get an offer, but had a fantastic experience. Authentic people, with the right mix of ambition and work-life balance. They respected the time required for me to engage in their hiring interviewing process, which was refreshing.
It is a bit above your price point, but I have been using Kagi.com (not affiliated, just impressed). They're in beta, but will charge ~$10 once they go GA. Like you, I tried DuckDuckGo for awhile, but resorted to g! so often that I started using it for everything out of habit.
In contrast, Kagi provides Google-quality results mosts of the time, better-than-Google semi-often, and worse-than-google rarely. They support g!, but I only use it a couple of times a week, usually for site-specific searches.
Additionally, I really like that I am their customer and not their product - incentives are aligned for them to continue respecting my privacy and preferences.
One only uses Google services if one absolutely needs to. Google on the other hand never needs you. You are absolutely unnecessary and super easy to replace. Whichever you chose, the next search engine really needs our queries. If we give them enough they might be able to create a competitive product. If they do google will dramatically improve. I'm sure they have plenty of ideas, the incentive is just not there.
After having used DuckDuckGo for a couple of years and habitually using g!, I switched to Kagi recently. Much better experience, I like that I’m a customer and not the product, great search results. I have been surprised and impressed.
reply