python - Is there a better way to read many html url?

Question

Closed. This question needs debugging details. It is not currently accepting answers. question from:https://stackoverflow.com/questions/65863880/is-there-a-better-way-to-read-many-html-url

Answer 1

1. Add all your urls and the Sett. values in list of tuples.
2. Iterate over them using for loop.

This is the code:

import pandas as pd

urls = [('https://fbref.com/it/comp/11/calendario/Risultati-e-partite-di-Serie-A', 15),
        ('https://fbref.com/it/comp/26/calendario/Risultati-e-partite-di-Super-Lig', 15),
        ('https://fbref.com/it/comp/12/calendario/Risultati-e-partite-di-La-Liga', 13),
        ('https://fbref.com/it/comp/13/calendario/Risultati-e-partite-di-Ligue-1', 13),
        ('https://fbref.com/it/comp/20/calendario/Risultati-e-partite-di-Bundesliga', 13)
        ] #Add all URLS and 'Sett.' values

frames = []

for url, sett in urls:
    df2 = pd.read_html(url)[0][['Sett.', 'Data', 'Casa', 'Punteggio', 'Ospiti']]
    frames.append(df2[(df2['Punteggio'] == '0–0') & (df2['Sett.'] > sett)]) #Here will come in handy the 'sett' variable

result = pd.concat(frames)

I didn't manually add all the URLs into the list because there were so many of them.