About a year ago, I built an ETL system to extract and process product data from e-commerce websites, mainly supplements. It maps product structures, automates data collection, cleans and classifies the information, and uploads it directly to Shopify through API integration.The system uses Python and Selenium for extraction, with transformation steps focused on cleaning, enrichment, and product classification. It was designed to be modular and easy to extend to new websites.
I paused development but I'm considering reviving it. Feedback or suggestions are welcome.
Repo: https://github.com/GustavoFortti/products-crawler