Yandex, Hugging Face, HSE, Moscow Institute of Physics and Technology, University of Toronto and the Vector Institute propose a method to share compute power and ran a decentralized collaborative training of an Albert-like model in Bengali.https://arxiv.org/abs/2106.10207
https://github.com/yandex-research/DeDLOC
https://twitter.com/m_ryabinin/status/1406988626374762503