Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
288 views
in Technique[技术] by (71.8m points)

python - Scraping Dynamic Information

I recently started with coding, I use Python and Pycharm. I Installed and imported the needed "Add-ons" like Selenium.
For my first project I tried to get the "address" information from this website:

https://randomstreetview.com/#fullscreen

When I use the Chrome developer tools (F12) the information shows up after the div'address'. After lots of trial and error I managed to scrape the website in multiple ways but nothing ever showed up behind the div.
Due to that I looked it up and think it's a dynamic information, so I started trying to use Selenium and other code that I found online.
Nothing worked or I didn't manage to get it working.
The address changes everytime when you open the website but the place stays the same (afaik).
Can anyone help me with a working code or help me to get on the right way?
I could provide my different approaches but I won't think that would help. See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

If you want the element's address just get the element and print it's text.

driver.get("https://randomstreetview.com/")
wait = WebDriverWait(driver, 10)
elem = wait.until(EC.presence_of_element_located((By.ID, "address")))
print(elem.text)

Element

<div id="address">Nordre Ringvej 97, 2600 Glostrup, D?nemark</div>

Outputs

Nordre Ringvej 97, 2600 Glostrup, D?nemark

Imports

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait 
from selenium.webdriver.support import expected_conditions as EC

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...