Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
540 views
in Technique[技术] by (71.8m points)

python - How to handle Google Authenticator with Selenium

I need to download a massive amount of excel-files (estimated: 500 - 1000) from sellercentral.amazon.de. Manually downloading is not an option, as every download needs several clicks until the excel pops up.

Since amazon cannot provide me a simple xml with its structure, I decided to automate this on my own. The first thing coming to mind was Selenium and Firefox.

The Problem:

A login to sellercentral is required, as well as 2-factor-authentication (2FA). So if I login once, i can open another tab, enter sellercentral.amazon.de and am instantly logged in. I can even open another instance of the browser, and be instantly logged in there too. They might be using session-cookies. The target URL to "scrape" is https://sellercentral.amazon.de/listing/download?ref=ag_dnldinv_apvu_newapvu .

But when I open the URL from my python-script with selenium webdrive, a new instance of the browser is launched, in which I am not logged in. Even though, there are instances of firefox running at the same time, in which I am logged in. So I guess the instances launched by selenium are somewhat different.

What I've tried:

I tried setting a timedelay after the first .get() (to open site), then I'll manually login, and after that redoing the .get(), which makes the script go on for forever.

from selenium import webdriver
import time


browser = webdriver.Firefox()

# Wait for website to fire onload event
browser.get("https://sellercentral.amazon.de/listing/download?ref=ag_dnldinv_apvu_newapvu")

time.sleep(30000)

browser.get("https://sellercentral.amazon.de/listing/download?ref=ag_dnldinv_apvu_newapvu")

elements = browser.find_elements_by_tag_name("browse-node-component")


print(str(elements))

What am I looking for?

Need solution to use the two factor authentication token from google authenticator.

I want the selenium to be opened up as a tab in the existing instance of the firefox browser, where I will have already logged in beforehand. Therefore no login (should be) required and the "scraping" and downloading can be done. If there's no direct way, maybe someone comes up with a workaround?

I know selenium cannot download the files itself, as the popups are no longer part of the browser. I'll fix that when I get there.

Important Side-Notes: Firefox is not a given! I'll gladly accept a solution for any browser.

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

Here is the code that will read the google authenticator token and used in the login. Used js to open the new tab. Install pyotp package before running the test code.

pip install pyotp

Test code:

from pyotp import *
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

driver = webdriver.Firefox()
driver.get("https://sellercentral.amazon.de/listing/download?ref=ag_dnldinv_apvu_newapvu")
wait = WebDriverWait(driver,10)
# enter the email
email = wait.until(EC.presence_of_element_located((By.XPATH, "//input[@name='email']")))
email.send_keys("email goes here")

# enter password
driver.find_element_by_xpath("//input[@name='password']").send_keys("password goes here")

# click on signin button
driver.find_element_by_xpath("//input[@id='signInSubmit']").click()

#wait for the 2FA feild to display
authField = wait.until(EC.presence_of_element_located((By.XPATH, "xpath goes here")))
# get the token from google authenticator
totp = TOTP("secret goes here")
token = totp.now()
print (token)
# enter the token in the UI
authField.send_keys(token)
# click on the button to complete 2FA
driver.find_element_by_xpath("xpath of the button goes here").click()
# now open new tab
driver.execute_script("""window.open("https://sellercentral.amazon.de/listing/download?ref=ag_dnldinv_apvu_newapvu")""")
# continue with your logic from here

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...