How To Use Selenium To Go From One Url Tab To Another Before Scraping?

I have created the following code in hopes to open up a new tab with a few parameters and then scrape the data table that is on the new tab. #Open Webpage url = 'https://www.websi

Solution 1:

The problem lies here:

df_url = driver.switch_to_window(driver.window_handles[0])
page = requests.get(df_url).text

df_url is not referring to the url of the page. To get that, you should call driver.current_url after switching windows to get the url of the active window.

Some other pointers:

• finding elements by xpath is relatively inefficient (source)
• instead of time.sleep, you can look into using explicit waits

Solution 2:

Insert the url below the driver variable because first, the webdriver executes and then the url provided

driver=webdriver.Chrome(executable_path=r"C:\mypathto\chromedriver.exe")
url = "https://www.website.com"