One year into my startup journey, I was going through pivot hell.
My life changed for good when I got a friend's call who wanted to connect me to their teammate looking to scrape an Indian apparel brand.
This brand wanted to scrape quite a lot of their competitors (inluding H&M and Zara type brands) from both the DTC websites and marketplaces.
This project gave us enough money to be in the game and not give up. Soon, working with them, we had gone into every depth possible of scraping e-commerce sites.
1. Identifying the product links (trust me, it's tough to fetch product links from raw HTML) 2. Scraping things like available sizes, the buttons and way of presentation is different for all websites. 3. Getting the data hidden on the page behind a button.
Our tech stack worked beautifully, maybe we over optimised because we had so much time.
So when the engagement with that apparel brand ended, we had time to actually work on this and make it work for each and every e-commerce website that there is.
I will launch it sooooon. But here is a taste of it, so sorry that I have already added sign up. I don't want to remove this now, but it's through google, so should be easy.
Any feedback would be genuinely helpful to me at this point, thanks!