python - Whats is the correct element type for this selenium project?

Question

I'm trying to find something out but nothing works. I'm trying to use a web scraper, to print all of the hot deal percentages on this website: ' https://shadowpay.com/en?price_from=0.00&price_to=34.00&game=csgo&hot_deal=true ' But I ran into an error (I have tried many ways to go about this, but I think that it's my lack of HTML) The error is that the element that I want to print ('percent-hot-deal__block'), isn't a class name, but I've tried a lot of find_element_by options, nothing worked, so I'm coming here. Code:

import pandas as pd
from bs4 import BeautifulSoup as bs
from selenium import webdriver
import requests
import time
#
perc = []
#
PATH = 'C:/Users/Matiss/Documents/chromedriver_win32/chromedriver.exe'
driver = webdriver.Chrome(PATH)
driver.get("https://shadowpay.com/en?price_from=0.00&price_to=34.00&game=csgo&hot_deal=true")
#
dealblock = driver.find_elements_by_tag_name("span")
for deal in dealblock:
    header = deal.find_element_by_class_name("percent-hot-deal__block")
    print(header.text)
#
time.sleep(15)

driver.quit()

Please help, if I need to edit anything, be sure to comment. Also, I know that importin so many things is useless, I was following other tutorials on the same file.

question from:https://stackoverflow.com/questions/65868329/whats-is-the-correct-element-type-for-this-selenium-project

Answer 1

Basically you want to wait for the page to load and grab the element.

You can do something like this.

driver.get("https://shadowpay.com/en?price_from=0.00&price_to=34.00&game=csgo&hot_deal=true")
driver.implicitly_wait(5)
dealblock = driver.find_elements_by_css_selector("span.percent-hot-deal__block")
#print(len(dealblock))
for deal in dealblock:
    print(deal.text)

The more ideal way is:

wait = WebDriverWait(driver, 10)
driver.get("https://shadowpay.com/en?price_from=0.00&price_to=34.00&game=csgo&hot_deal=true")
dealblock = wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "span.percent-hot-deal__block")))
#print(len(dealblock))
for deal in dealblock:
    print(deal.text)

Imports

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait 
from selenium.webdriver.support import expected_conditions as EC