Part 1: Creating Detail Page scraper

What user can learn after this 5-part tutorial:

This tutorial is meant to help through build two different types of scrapers in ProWebScraper and introduce you to the key concepts of ProWebScraper in short. By the end of the tutorial, you will be able to create chained scrapers and extract data from a list of URLs using ProWebScraper.         

Many websites have listings of products or a list of search results with links to each product page that has further more details. To fetch the details of all these products, you can build two scrapers, to create what we call chained scrapers. The first scraper (Parent scraper) will get a list of URLs (links) to the product pages.  The second scraper (Child scraper) makes use of the output from the first scraper to get the data for the individual products.

On the PWS Dashboard,

You can get started by clicking “New scraper”. Next, paste the URL of a detail page in the “Create scraper” screen, such as https://www.flipkart.com/redmi-note-7s-sapphire-blue-64-gb/p/itmfh75f4rhcjdfk,  and

Once the page is loaded, you can start configuring scraper.

Now, you can start fetching the data you need. All you need to do is just click directly on the different items of information.

When you select the data points, you will see that your selection is shown both in the “current selection” panel and on the page with green highlighted groups.

To extract the title of the mobile, just click on that.

You can see how the data you clicked on is now showing on the current selection panel.

To name this data point, click into the name either in the hovering data column or in the list of data columns at the top of the page.

To extract/add more data points (such as rating, price range, no. of reviews, seller info), you can click on “Add column”, and then click on that data point.

Once the data points have been selected, click the “Save” button to save your scraper. 

Now, name your scraper and then click “Save and Run”.

Once the scraper is saved, it will take you to the scraper Dashboard. There, you can download the results of the run once it is over.

After your first scraper is created, you’ll want to move on to Part 2: Creating a Listing scraper.