How Can I Retrieve Data From A Web Page With A Loading Screen?

I am using the requests library to retrieve data from nitrotype.com/racer/insert_name_here about a user's progress using the following code: import requests base_url = 'https://ww

Solution 1:

This is likely because of dynamic loading and can easily be navigated by using selenium or pyppeteer.

In my example, I have used pyppeteer to spawn a browser and load the javascript so that I can attain the required information.

Example:

import pyppeteer
import asyncio

asyncdefmain():
    # launches a chromium browser, can use chrome instead of chromium as well.
    browser = await pyppeteer.launch(headless=False)
    # creates a blank page
    page = await browser.newPage()
    # follows to the requested page and runs the dynamic code on the site.await page.goto('https://www.nitrotype.com/racer/tupac')
    # provides the html content of the page
    cont = await page.content()
    return cont

# prints the html code of the user profiel: tupacprint(asyncio.get_event_loop().run_until_complete(main()))