Ask HN: Google Colab alternatives for large lang models?

1 point

3 years ago

I just want to run some LLM experiments on GPUs. Since these are 10-20GB models, I need something like an A100 40GB GPU at a minimum.

But I'm finding that most of the paid Google Colab options don't work. Instead of working on the LLM experiments, I'm wasting considerable time on infrastructure.

First, I purchased pay-as-you-go credits[1]. But about 80% of the time, I get a "Selected GPU unavailable" for A100.

Next, I tried the custom GCE VM option. But the Colab deployment marketplace app[2] keeps hitting me with quota denials for A2-CPUs and A100-GPUs.

Requesting quota increases in GCP is equally frustrating. They auto-approved 12 A2-CPUs for a zone but denied one A100-GPU for that zone!

There are some 50 or so zones and apparently I have to individually request quota increases each of them but they get denied.

-----

QUESTION 1:

How are you people testing LLMs like GPT-NeoX, LLaMA 66B, etc?

------

QUESTION 2:

Is there a simple, non-time-wasting notebook alternative for Colab where I can get A100 or better GPUs easily?

Is there some arrangement of using Colab with local runtime which somehow proxies to a better GPU service provider?

Kaggle NB is also frustrating in other ways. Their GPUs don't seem to work with these models. Also difficult to connect to GDrive, difficult to transfer files in and out, etc.

-----

QUESTION 3:

If there's no alternative to Colab, anybody here has some practical tips to avoid these quota denials?

------

[1]: https://colab.research.google.com/signup/pricing

[2]: https://research.google.com/colaboratory/marketplace.html

5 comments