AI seems to be growing without bounds, and with it, the wealth of opportunities available to us. Although AI isn't locally (being able to effectively run multiple models locally & performantly in a non-commercial setting, say on the average PC) viable just yet, it's pretty clear that its becoming more accessible, through the "trickle-down advancement" effect from corporate AI companies.

I'm seeing a growing surge in the concept of "local-home AI", and here's how I'm picturing it eventually happening:

1. A AI mainframe computer hosting inference for slew of local models from sites like HF, Civit, and other providers, that can be access through other computers to do more daily / privacy-demanding tasks (something you wouldn't drop your info into an API for) 2. Houses can pool together their compute to train distributed models, that are more community scoped. Literally, if you plan to train with your neighbors, figuratively if you'd like to band together to based on a common cause / challenge.

The early stages of this appear to already be commercially viable; Ran the idea by a director at AMD, and he pointed out we can already build 48 GB VRAM machines for ~$5000 (2 RTXx3090) .

Talked with Ion Stoica (founder of Databricks) on the idea, and he pointed out the following things:

- Quantization Techniques: How do we compress / larger scale models down into smaller formats that can be used with house-AI's smaller compute requirments? - Model Evaluation: There are thousands of models on HuggingFace, how can we figure out which ones are more practically effective for some specific thing, beyond just "this model has 50K monthly downloads and 40K likes on HF" (which mostly seem to be general purpose LLMs at the moment)

What do you think on the general concept? Are there any online communities out there specifically for this niche?