Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
773 views
in Technique[技术] by (71.8m points)

python - Use Selenium to download file by clicking "javascript:__doPostBack('LeaderBoard1$cmdCSV','')"

There are a bunch of CSV files of baseball stats that I want to download via automation, which can be found at: https://www.fangraphs.com/leaders.aspx?pos=all&stats=bat&lg=all&qual=0&type=0&season=2020&month=0&season1=2020&ind=0&team=0&rost=0&age=0&filter=&players=0&startdate=2020-01-01&enddate=2020-12-31. The button to download the table as a CSV is labeled 'Export Data'.

HTML:

<div class="br_dby">
    <span style="float: left">
        <a href="javascript:ShowHide();">Show Filters</a>??
            |??
        <a href="#custom">Custom Reports</a>
    </span>
    <a href="javascript:__doPostBack('LeaderBoard1$cmdCSV','')" id="LeaderBoard1_cmdCSV">Export Data</a>
</div>

As you can tell, the button is not a redirect to a download page (in which case requests could be used to download the file), but is a process.

Code:

from selenium import webdriver
from selenium.webdriver.firefox.options import Options
binary = r'C:Program FilesMozilla Firefoxfirefox.exe'
options = Options()
options.headless = True
options.binary = binary
options.set_preference("browser.download.folderList",2)
options.set_preference("browser.download.manager.showWhenStarting", True)
options.set_preference("browser.download.dir", r"C:UsersjlpytDownloads")
driver = webdriver.Firefox(options=options, executable_path=r"C:Usersjlpytgeckodriver-v0.27.0-win32geckodriver.exe")
driver.get('https://www.fangraphs.com/leaders.aspx?pos=all&stats=bat&lg=all&qual=0&type=0&season=2020&month=0&season1=2020&ind=0&team=0&rost=0&age=0&filter=&players=0&startdate=2020-01-01&enddate=2020-12-31')
elem = driver.find_element_by_id('LeaderBoard1_cmdCSV')
elem.click()

Using this code, Selenium is able to click the button, but no download is initiated. Is there some way that I can use Selenium to click the button and download the file? Or, is there some alternative method that I have not thought of?

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

The element is __doPostBack enabled element, so to click on the element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:

  • Using ID:

    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.ID, "LeaderBoard1_cmdCSV"))).click()
    
  • Using LINK_TEXT:

    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.LINK_TEXT, "Export Data"))).click()
    
  • Using CSS_SELECTOR:

    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a#LeaderBoard1_cmdCSV"))).click()
    
  • Using XPATH:

    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@id='LeaderBoard1_cmdCSV']"))).click()
    
  • Note: You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support import expected_conditions as EC
    
  • Browser Snapshot:

fangraphs


References

You can find a couple of relevant discussions in:


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...