Need ALL competitor products crawled / monitored?
Quite recently Price2Spy was hired to crawl and scrape all product pages from our client’s competitor website.
The task may sound simple, but there was a catch: there were more than 300K pages to scrape, not all of them were unique (complex content logic involved), and we had a deadline of 1 working week.
The task was tough, but we did make it – thanks to our experience in site scraping and crawling technologies, as well as our distributed server technology, which enabled us to crawl at a very fast rate, without causing suspicion on target website.
Anyway, what our client got from us was a series of Excel files, with more than 370 000 products, with product details such as
- product name
- product URL
- product description
- product category
- product price (list / sale price)
- brand information
- stock levels
- manufacturer part number (MPN)
The next step was on our client, who filtered out the product list (he selected only products that he needed monitored) – and we have loaded those products into Price2Spy. Today all those products price / availability data is checked on daily basis – which is something of a great value to our client.
Smart, isn’t it?
