Beautifulsoup Scrapping Object Selection

Question

I am trying to scrape by Beautifulsoup and new to this, I need table rows as you see enter image description here. The tables are coming from reactapp and then shown on the website. I need suggestion how to do this. I am struggling to create the beautifulsoup object and do not know what the actual class to grap to reach table rows and their content.

webpage = urlopen(req).read()
soup = bs(webpage, "html.parser")
table=soup.find('table', {'class': 'equity'})
rows=list()
for row in table.findAll("tr"):
     rows.append(row)

Need your help, really appreciated, having hard time to get it done!

Answer 1

You can grab the td elements with this code:

webpage = urlopen(req).read()
soup = bs(webpage, "lxml")
table=soup.find('table', {'class': 'table'}).find('tr')
rows=list()
for row in table.findAll("td"):
     rows.append(row)

I prefered using lxml as the parser because it has some advantages, but you can keep using html.parser

You can also use pandas, It will create, It's so much easier to learn from its documentaion (there is a lot).