Part 4: Chaining Two scrapers

Now our listings and details scrapers have been created so you can chain the scrapers together to take the URLs collected in the listings scraper and extract data from them using the details scraper.

To do this, from scraper dashboard, you will need to go to the Details Scraper (“Flipkart Detail page”) and then under the “configure” tab, in the drop-down menu, choose URL from another scraper.

This will display the option to set a Parent scraper, which sets the scraper to pull the URLs from pages. After setting the option to your listings scraper, it will ask you to select the URL column that the URLs are present in.

Here, you will need to choose “Flipkart Listings” as a parent scraper and then select “Listings” as a column.

With the chain setup, you also have the option to set it to Always run the parent first. It means every time you trigger that scraper or its parent scraper, it will run them both in order. With that option enabled, you can then click the Run Chain button, which will run our listings scraper and then run our details scraper.

After the scrapers finish running, you will want to move on to Part 5: Downloading Your Data.