Show HN: Deploy Hugging Face Models to AWS Lambda

1 point

2 years ago

I've been working on Scaffoldly since 2020 to simplify AWS Lambda deployments. Recently discovered you can run Hugging Face models efficiently using EFS for caching. Here's what's interesting:

   - Uses EFS for model file persistence
   - Pre-downloads models after deployment for faster cold starts
   - Cold start: ~20s (model loading), warm requests: 5-20s (CPU inference)
   - Fully automated container builds and deployment
   - Works with private/gated models via HF_TOKEN

Example deployment:

  npx scaffoldly create app --template python-huggingface
  cd python-huggingface && npx scaffoldly deploy

Scaffoldly is Open Source and I'm excited for all feedback and contributions from the community!

https://github.com/scaffoldly/scaffoldly

https://github.com/scaffoldly/scaffoldly-examples/tree/pytho...