How do I scrape sites with over 5000 pages?

Individual scrapes have a limit of 5000 pages per scrape.

In order to scrape very large sites, feed in several different base urls for portions of the target webpage.

1. Pick the "Crawl webpages" option from the scrape options

2. Feed in the base URL for a portion of the site and start the scrape.

3. Repeat step 1 and 2 with a different base URL.