I want to download the content a whole page using scrapy.
With selenium this is quite easy:
import os,sys
reload(sys)
sys.setdefaultencoding('utf8')
from selenium import webdriver
url = 'https://es.wikipedia.org/wiki/Python'
driver = webdriver.Firefox()
driver.get(url)
content = driver.page_source
with open('source','w') as output:
output.write(content)
But selenium is much slower than scrapy.
Is it an simple way to do in scrapy?
I want to save the code of each page in a different file text, not as a csv or json file. Also, if posible without creating a project, which seems a bit of overkill for such a simple task.
See Question&Answers more detail:
os 与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…