Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
3.1k views
in Technique[技术] by (71.8m points)

python - Scraping data from a site first returns the data but when I run the code again it returns None object

I am new to web scraping, using beautiful soup and selenium. I am trying to scrape the following webpage
'''
https://www.amazon.in/gp/product/B08GG7W329/ref=ewc_pr_img_3?smid=A14CZOWI0VEHLG&psc=1

'''
I am trying to scrape the title and price of the product using this code.
When I used this code to scrape the title on first run it returend the title correctly when I run the code again it return NONE object.
Why does this happen?

Below is the code I used.

''' python

from selenium import webdriver  
from bs4 import BeautifulSoup

url = 'https://www.amazon.in/gp/product/B08GG7W329/ref=ewc_pr_img_3?smid=A14CZOWI0VEHLG&psc=1'  
driver = webdriver.PhantomJS(executable_path=r'C:phantomjs-2.1.1-windowsphantomjs-2.1.1- 
windowsinphantomjs.exe')  
driver.get(url)  
page = driver.page_source  

soup = BeautifulSoup(page, 'lxml')  
title = soup.find('span', id = 'productTitle')  
print(title.text.strip())  
price = soup.find('tr', id = 'priceblock_ourprice_row').find('span', id = 'priceblock_ourprice').text  
print(price)  
converted_price = price[0:5]

'''
This is the HTML code for title part:

''' html

<div class="a-section a-spacing-none" id="titleSection">
         <h1 class="a-size-large a-spacing-none" id="title">
          <span class="a-size-large product-title-word-break" id="productTitle">
           Lenovo Legion 5 AMD Ryzen 7 4800H 15.6 inch Full HD Gaming Laptop (8GB/1TB HDD + 256GB 
      SSD/Windows 10/120 Hz/NVIDIA GTX 1650Ti 4GB GDDR6 Graphics/Phantom Black/2.3Kg), 82B500BMIN
          </span>
         </h1>
         <div class="a-section a-spacing-none expand aok-hidden" id="expandTitleToggle">
         </div>
        </div>  

'''


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)
等待大神答复

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...