Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
150 views
in Technique[技术] by (71.8m points)

python - The website starts to refresh after a moment of scraping with Selenium

So I've been learning python for a while but I'm a newbie when it comes to web scraping so sorry if my question is kind of vague. The code below is basically an Instagram follower bot, where you follow the followers of a specific person and it. It actually works quite well, however after about 2 minutes the website refresh which means that it now has to start the whole process again, clicking cancel continuously now that the site has refreshed since it has to follow the people it has already followed. And after that, it refreshes again after 2 minutes so in the end it never can follow everyone. I appreciate every help! My apologies for my bad English!

     from selenium import webdriver
        from selenium.common.exceptions import ElementClickInterceptedException
        from selenium.webdriver.common.keys import Keys
        import time
        
        CHROME_DRIVER_PATH = "C:Developmentchromedriver.exe"
        SIMILAR_ACCOUNT = ""
        USERNAME = ""
        PASSWORD = ""
        
        class InstaFollower:
        
            def __init__(self, path):
                self.driver = webdriver.Chrome(executable_path=path)
        
            def login(self):
                self.driver.get("https://www.instagram.com/accounts/login/")
                time.sleep(5)
        
                username = self.driver.find_element_by_name("username")
                password = self.driver.find_element_by_name("password")
        
                username.send_keys(USERNAME)
                password.send_keys(PASSWORD)
        
                time.sleep(2)
                password.send_keys(Keys.ENTER)
        
            def find_followers(self):
                time.sleep(5)
                self.driver.get(f"https://www.instagram.com/{SIMILAR_ACCOUNT}")
                time.sleep(2)
                followers = self.driver.find_element_by_xpath(
                    '//*[@id="react-root"]/section/main/div/header/section/ul/li[2]/a')
                followers.click()
            def scroll(self):
                time.sleep(3)
                modal = self.driver.find_element_by_xpath('/html/body/div[5]/div/div/div[2]')
                self.driver.execute_script("arguments[0].scrollTop = arguments[0].scrollHeight", modal)
        
            def follow(self):
                time.sleep(2)
                buttons = self.driver.find_elements_by_css_selector("li button")
                buttonz = [x.text for x in buttons]
                print(buttonz)
                for button in buttons:
                    time.sleep(1)
                    try:
                        button.click()
                        time.sleep(3)
                    except ElementClickInterceptedException:
                        cancel = self.driver.find_element_by_xpath("/html/body/div[6]/div/div/div/div[3]/button[2]")
                        cancel.click()

  bot = InstaFollower("C:Developmentchromedriver.exe")
  bot.login()
  bot.find_followers()
  while True:
    bot.follow()
    bot.scroll()
        
question from:https://stackoverflow.com/questions/65932866/the-website-starts-to-refresh-after-a-moment-of-scraping-with-selenium

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)
Waitting for answers

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...