But I'm finding that most of the paid Google Colab options don't work. Instead of working on the LLM experiments, I'm wasting considerable time on infrastructure.
First, I purchased pay-as-you-go credits[1]. But about 80% of the time, I get a "Selected GPU unavailable" for A100.
Next, I tried the custom GCE VM option. But the Colab deployment marketplace app[2] keeps hitting me with quota denials for A2-CPUs and A100-GPUs.
Requesting quota increases in GCP is equally frustrating. They auto-approved 12 A2-CPUs for a zone but denied one A100-GPU for that zone!
There are some 50 or so zones and apparently I have to individually request quota increases each of them but they get denied.
-----
QUESTION 1:
How are you people testing LLMs like GPT-NeoX, LLaMA 66B, etc?
------
QUESTION 2:
Is there a simple, non-time-wasting notebook alternative for Colab where I can get A100 or better GPUs easily?
Is there some arrangement of using Colab with local runtime which somehow proxies to a better GPU service provider?
Kaggle NB is also frustrating in other ways. Their GPUs don't seem to work with these models. Also difficult to connect to GDrive, difficult to transfer files in and out, etc.
-----
QUESTION 3:
If there's no alternative to Colab, anybody here has some practical tips to avoid these quota denials?
------
[1]: https://colab.research.google.com/signup/pricing
[2]: https://research.google.com/colaboratory/marketplace.html