Mastering Puppeteer: Comprehensive Guide on How to Scroll to the Bottom

Table of Contents

Table of Contents

Web scraping with Puppeteer often involves dealing with pages that necessitate scrolling to the bottom to load additional content, a common feature of infinite-scrolling pages. To effectively manage this task, integrating a reliable web scraping API can significantly enhance the efficiency and accuracy of your data collection efforts, providing advanced features to handle dynamic content and infinite scrolling seamlessly.

For scrolling in our Puppeteer browser, we can utilize a custom javascript function window.scrollTo(x, y), which scrolls the page to the designated coordinates. This method, combined with a powerful scraping API, ensures that you can navigate and extract data from complex websites with ease.

If we need to scroll to the absolute bottom of the page, a while loop can be employed to keep scrolling until the bottom is reached. Let’s examine an example by scraping

const puppeteer = require('puppeteer');

async function scrapeTestimonials() {
    const browser = await puppeteer.launch({headless: false});
    const page = await browser.newPage();

    await page.goto('');

    let prevHeight = -1;
    let maxScrolls = 100;
    let scrollCount = 0;

    while (scrollCount < maxScrolls) {
        // Scroll to the bottom of the page
        await page.evaluate('window.scrollTo(0, document.body.scrollHeight)');
        // Wait for page load
        await page.waitForTimeout(1000);
        // Calculate new scroll height and compare
        let newHeight = await page.evaluate('document.body.scrollHeight');
        if (newHeight == prevHeight) {
        prevHeight = newHeight;
        scrollCount += 1;

    // Collect all loaded data
    let elements = await page.$$('.testimonial');
    let results = [];
    for(let element of elements) {
        let text = await element.$eval('.text', node => node.innerHTML);

    console.log(`Scraped: ${results.length} results!`);

    await browser.close();


In the above example, we’re scraping an endless paging example from the website. We initiate a while loop and continue scrolling to the bottom until the browser’s vertical size ceases to change. Once the bottom is reached, we can commence parsing the content.

Related Questions

Related Blogs

Using Puppeteer for web scraping often involves navigating modal popups, such as Javascript alerts that conceal content and display messages upon page load. For developers...
In the world of automation and web scraping, Puppeteer stands out as a powerful tool for developers. Whether you’re automating routine tasks or collecting data...
Puppeteer stealth is a widely used extension for the Puppeteer browser automation framework. This plugin modifies Puppeteer’s runtime to reduce the likelihood of detection by...