One possibility I like to imagine is a future where knowledge sources are used kind of like tools, i.e. the model never uses any preexisting knowledge from its training data (beyond what’s necessary to be fluent in English, coherent, logical, etc.), a “blank” intelligent being, tabula rasa. And for answering questions it uses various sources dynamically, like an agent would use tools.
I think this will let models be much smaller (and cheaper), but it would also enable a mechanism for monetizing knowledge. This would make knowledge sharing profitable.
For example, a user asks a question, the model asks knowledge sources if they have relevant information and how much it costs (maybe some other metadata like perceived relevance or quality or whatever), and then it decides (dynamically) which source(s) to use in order to compile an answer (decision could be based on past user feedback, similarly to PageRank).
One issue is that this incentivizes content users want to hear versus content they don’t want to hear but is true. But this is a problem we already have, long before AI or even the internet.
I think this will let models be much smaller (and cheaper), but it would also enable a mechanism for monetizing knowledge. This would make knowledge sharing profitable.
For example, a user asks a question, the model asks knowledge sources if they have relevant information and how much it costs (maybe some other metadata like perceived relevance or quality or whatever), and then it decides (dynamically) which source(s) to use in order to compile an answer (decision could be based on past user feedback, similarly to PageRank).
One issue is that this incentivizes content users want to hear versus content they don’t want to hear but is true. But this is a problem we already have, long before AI or even the internet.