You can schedule your scraper to run on a regular basis to get the latest information from a webpage.
To schedule a scraper,
- Go to the scraper’s “Scheduler” tab
- Select the following options: hourly, daily, weekly, monthly, or advanced
- Select the advanced option to customize a schedule using cronjob GUI.
Set your schedule to Hourly
Select Frequency: Hourly to run your scraper at a specific minute of every hour.
Set your schedule to Daily
Select Frequency: Daily to run your extractor every day at a specific time of the day.
Set your schedule to Weekly
Select Frequency: Weekly to run your extractor every week at a specific time of a given day.
Set your schedule to Monthly
Select Frequency: Monthly to run your extractor once a month at a certain date and time.
Set Custom Schedule
Set your schedule to Advanced for a customized schedule, from months and days of the week up to the minute. For example:
- Run every hour between 09:45 AM and 3:45 PM on Tuesday, Thursday, and Saturday.
- Run on 5th during the months of June, July, and August.
Identifying Scheduled Extractors
You can quickly view and identify which scrapers are scheduled to run in your dashboard.
They're marked by the following icon:
Or they're marked by the following icon if the scrapers are chained and scheduled both: