Need ALL competitor products crawled / monitored?

18:07 September 12, 2012 in New Price2Spy features
Reading Time: < 1

Automated site crawlQuite recently Price2Spy was hired to crawl and scrape all product pages from our client’s competitor website.

The task may sound simple, but there was a catch: there were more than 300K pages to scrape, not all of them were unique (complex content logic involved), and we had a deadline of 1 working week.

The task was tough, but we did make it – thanks to our experience in site scraping and crawling technologies, as well as our distributed server technology, which enabled us to crawl at a very fast rate, without causing suspicion on target website.

Anyway, what our client got from us was a series of Excel files, with more than 370 000 products, with product details such as

  • product name
  • product URL
  • product description
  • product category
  • product price (list / sale price)
  • brand information
  • stock levels
  • manufacturer part number (MPN)

The next step was on our client, who filtered out the product list (he selected only products that he needed monitored) – and we have loaded those products into Price2Spy. Today all those products price / availability data is checked on daily basis – which is something of a great value to our client.

Smart, isn’t it?


About Price2Spy


Price2Spy is an online service that provides comprehensive and suitable solutions for eCommerce professionals including; retailers, brands/manufacturers and distributors in order to stay profitable in the current competitive market conditions. If you want to learn more about what Price2Spy can do for your business, please start your 30-day free trial.