Ask HN: Could we crowd-source training a LLM, like we do crypto-mining?

2 points

3 years ago

Could we train a LLM by using the computing power of thousands of personal computers, like we can do with mining crypto-currencies, or with other crowd-sourcing of computing power like the venerable SETI@home project (https://setiathome.berkeley.edu/)?

Is that possible technically?

Assuming one of the big issues for training a LLM is the computing cost, could this be an approach to avoid that advanced LLMs are only created by the biggest companies, encouraging even more capital concentration and leaving high-end IAs in their sole control?

Another problem is accessing the immense amount of data required to train the model, but building such datasets openly seems more feasible, if not already made available by some actors.

2 comments