Show HN: Simple Crawling Server

2 points

a year ago

I’ve built a crawling server that you can use to crawl urls

It:

- Accepts requests via GET and responds with JSON data, including page contents, properties, headers, and more.

- Supports multiple crawling methods—use requests, Selenium, Crawlee, and more. Just specify the method by name!

- Perfect for developers who need a versatile and customizable solution for simple web scraping and crawling tasks

- Can read information about youtube links using yt-dlp

There is also a docker image.

1 comment