I am trying to scrape google news headings along with their links for input term. But when I searched via find_all method for a class that contains all news headings, it returned an empty list.
I tried with parent divs with their id's but the result wasn't different.
import requests
from bs4 import BeautifulSoup
input_term = input("Enter a term to search:")
source = requests.get("https://www.google.com/search?q={0}&source=lnms&tbm=nws".format(input_term)).text
soup = BeautifulSoup(source, 'html.parser')
#here 'bkWMgd' is class that I found to be contained all search results.
heading_results = soup.find_all('div', class_ = 'bkWMgd')
print(heading_results)
I want to scrape all news headings and their respective links. I expected a list of all search result from the above code. But it returning an empty list.
See Question&Answers more detail:
os 与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…