Maybe this isn't possible for LLMs yet, but open source versions of AlphaZero have been trained on peer-to-peer networks.

https://zero.sjeng.org/

https://katagotraining.org/