Case Study: Web Scraping & Data Extraction for Ecommerce

Best practices in price monitoring, Price2Spy announcements 15.7.2019. Reading Time: 2 minutes

After 9 years in business, our client database has expanded greatly and so is the variety of businesses we help across the world. Besides the sites (over 70K), Price2Spy is monitoring for our clients (over 650) from different industries (furniture, music equipment, nursery, pet supplies, sports, foods & drinks and so on), it also helps them to get their desired data through web scraping / web crawling.

Over the years, Price2Spy team has mastered a web scraping process that could help your business gather any valuable data in bulk. To provide you some of the best practices and samples from Price2Spy’s technical capabilities, we thought that the following web scraping case study might be of interest to you.

Click here to see more web scraping case studies from Price2Spy.

Price2Spy harvests and prepares raw data with the help of web scraping for the leading household products brand, worldwide

This client is one of the world’s most famous manufacturer of household products. They are working with hundreds of retailers worldwide, also they are monitoring several hundreds of websites including their’s competitors. Most importantly, they keep track of product reviews closely, because, they want to determine consumer satisfaction.

When this client approached us a couple of years ago, their major problems were:

  • Capturing product reviews from retail websites, in order to determine consumer satisfaction towards their products
  • Detecting resellers who are not in accordance with strict MAP policy they contracted

One of our first tasks was to perform web scraping on the sites that the client provided, in order to capture product reviews.

This task may sound simple, but it wasn’t, there were dozens of pages to be scraped and not all of them were unique, there were technically very complex and bot-aware websites (like Amazon).

The task was tough, but we managed to do it in a short period of time – thanks to our experience in web scraping and crawling technologies, as well as our distributed server technology, which enabled us to perform web scraping at a very fast rate, without causing suspicion on the target websites.

Anyway, at the end of web scraping process, we provided the results in Excel files to our client. The client was very satisfied with the raw data that they got from us, and they decided to automate the web scraping procedure. We introduced repetitive web scraping as a part of their organization and still keep doing it on a monthly basis.

Moreover, Price2Spy’s data entry team performed product matching on the retailer and competitor sites for the products supplied by the client and entered them into the account. Today all those products price/availability data is checked in every 3 hrs (8 times per day), so the client is able to monitor and enforce MAP policy strictly.

If you also need to gather data from a retailer or competitor site (regardless of the complexity) in one go or if you need a detailed overview of your competitor’s assortment, along with full pricing/product description, category and brand information and etc., just let us know the list of websites you want to be scraped – and we’ll be happy to give you a quote for this task.


Cahide Gunes Pakay
Cahide Gunes Pakay is a Digital Marketing Manager at Price2Spy, an online price monitoring, pricing analytics, and repricing tool used by eCommerce professionals from all over the world. She loves reading, writing, and speaking about e-commerce, pricing, and competitive strategies. You can find her on LinkedIn.