Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
159 views
in Technique[技术] by (71.8m points)

python - svg tag scraping from funnels

I am trying to scrape data from here but getting error. I have taken code from here Scraping using Selenium and python

This code was working perfectly fine but now I am getting error

 wait.until(EC.visibility_of_element_located((By.LINK_TEXT, "All Boards")))  
 raise TimeoutException(message, screen, stacktrace)
See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

After clicking on pe-funnel link , you can try with this code :

wait.until(EC.visibility_of_element_located((By.XPATH, "//*[name()='text' and @text-anchor='end']"))) 

all_data = driver.find_elements_by_xpath("//*[name()='text' and @text-anchor='end']")

print(len(all_data))

for data in all_data:
 print(data.text)  

UPDATE1 :

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC 
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.select import Select
import time

driver = webdriver.Chrome(executable_path = r'D:/Automation/chromedriver.exe')
driver.maximize_window()
driver.get("https://eu1.dashboard.clevertap.com/login.html")

wait = WebDriverWait(driver, 20)
action = ActionChains(driver)

driver.switch_to.default_content()

wait.until(EC.element_to_be_clickable((By.NAME, "email"))).send_keys("abhishe***")

wait.until(EC.element_to_be_clickable((By.NAME,"password"))).send_keys("***")  

wait.until(EC.element_to_be_clickable((By.ID,"submitBtn"))).click()

wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, 'div.ct-breadcrumb')))

driver.switch_to.default_content()  
action.move_to_element(driver.find_element_by_css_selector("div.sidebar__brand+ul>li:first-child>a")).perform()

wait.until(EC.visibility_of_element_located((By.LINK_TEXT, "All Boards")))  

wait.until(EC.element_to_be_clickable((By.LINK_TEXT,"All Boards"))).click()

wait.until(EC.element_to_be_clickable((By.LINK_TEXT,"pe-funnel"))).click()

time.sleep(1)

driver.execute_script("window.scrollTo(0,100)")

wait.until(EC.presence_of_all_elements_located((By.XPATH,"//*[name()='svg' and @class='highcharts-root']")))

all_charts = driver.find_elements_by_xpath("//*[name()='svg' and @class='highcharts-root']")
length_of_list = len(all_charts)
print(length_of_list)
i=0
while(i<len(all_charts)):
 wait.until(EC.presence_of_all_elements_located((By.XPATH,"//*[name()='svg' and @class='highcharts-root']")))
 all_charts = driver.find_elements_by_xpath("//*[name()='svg' and @class='highcharts-root']")
 all_charts[i].click()
 i=i+1
 try:
  print("Switch to frame")
  wait.until(EC.frame_to_be_available_and_switch_to_it((By.ID,"wiz-iframe-intent")))
  print("Switched to frame")
  wait.until(EC.element_to_be_clickable((By.XPATH, "//button[text()='OK' and @class='annoucement-popup__btn']"))).click()
  driver.switch_to.default_content()
  print("Clicked on Ok button")
 except:
  print("in catch block")
  pass
  print("last of CATCH BLOCK")

 driver.execute_script("window.scrollTo(0,1100)")
 ActionChains(driver).move_to_element(driver.find_element_by_css_selector("input[data-introp='View your analysis']")).click().perform()
 #wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,""))).click()
 #ActionChains(driver).move_to_element(driver.find_element_by_css_selector("label[for='funnelProgressionPercent']")).send_keys(Keys.END).perform()
 wait.until(EC.presence_of_all_elements_located((By.XPATH,"//*[name()='tspan' and @class='highcharts-text-outline']")))  
 all_values = driver.find_elements_by_xpath("//*[name()='tspan' and @class='highcharts-text-outline']")  
 for values in all_values:
  print(values.text)  
 driver.execute_script("window.history.go(-1)")
 driver.refresh()

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...