Mastering Playwright: How to Click on Modal Alerts like Cookie Pop Up

Modal pop-ups, often seen as cookie consent or login requests, are created using custom JavaScript. They typically hide the page content upon loading and display a specific message. Navigating these pop-ups is a common challenge in web automation and scraping, requiring a toolset that can interact with them as seamlessly as a human user. Incorporating a web scraping API into your Playwright scripts can significantly enhance your ability to manage these modal alerts. By providing a streamlined approach to web scraping, these APIs enable developers to focus on extracting valuable data without getting bogged down by the intricacies of pop-up management, making your automation efforts more efficient and effective.

There are several strategies to manage modal pop-ups:

1. Clicking on an option such as “OK” or “Yes”
2. Removing the modal element from the Document Object Model (DOM)

For instance, consider the login page at web-scraping.dev, which presents a cookie pop-up upon loading:

from playwright.sync_api import sync_playwright, TimeoutError

with sync_playwright() as p:
    browser = p.chromium.launch(headless=False)
    page = browser.new_page()
    page.goto("https://web-scraping.dev/login")

    # Option #1 - use page.click() to click on the button
    try:
        page.click("#cookie-ok", timeout=2_000)
    except TimeoutError:
        print("no cookie popup")
    
    # Option #2 - delete the popup HTML
    #   remove pop up
    cookie_modal = page.query_selector("#cookieModal")
    if cookie_modal:
        cookie_modal.evaluate("el => el.remove()")
    #   remove grey backgdrop which covers the screen
    modal_backdrop = page.query_selector(".modal-backdrop")
    if modal_backdrop:
        modal_backdrop.evaluate("el => el.remove()")

The above example demonstrates two methods for handling modal pop-ups: clicking a dismissal button and directly removing them from the DOM. Generally, the first method is more reliable as clicking the actual button can trigger additional functions, such as setting a cookie to prevent the pop-up from reappearing. However, for situations involving login requirements or advertisements, the second method may be more appropriate.