This has been fun we can task our nemotron-3-super model to run over night when our desktops are idle. 4070s and 96gb of ram works fine. Slow but does it's job.
This is how we design at HewesNguyen AI. We are both MIS so once LLMs came out we where like sweet whole teams that can be tasked for one thing done well. Thank you Unix Philosophy
The marvel cxl 2.0 ddr4 card Serve the Home used for kvcache speed ups. And I am personally looking forward to cxl 3 and memory coherence across my system builds.
A few reasons, "AI" as used by non-experts often has correctness and security issues. Even when it doesn't, its outputs are often not reproducible/predictable because they're probabilistic systems.
AI systems are also prone to writing code which they can't effectively refactor themselves, implying that many of these code bases are fiscal time bombs where human experts are required to come fix them. If the service being replaced has transactional behaviour, does the AI produced solution? Does the person using it know what that means?
The other side is that AI as an industry still needs to recoup trillions in investment, and enterprise users are potential whales for that. Good prices in AI systems today are not guaranteed to last because even with hardware improvements these systems need to make money back that has been invested in them.
Some of that latter part depends on how good and cheap open weight systems get. The ability to deploy your own will strictly limit the price of closed models if they aren't dominant in functionality.
Sweet I have about 500 of games on gog and I use heroic launcher on cachyos. Probably a total of 1.3k of titles across steam, gog and epic any idea gaming isn't Linux now is dumb thinking.
reply