Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
842 views
in Technique[技术] by (71.8m points)

python 3.x - urllib.urlretrieve with custom header

I am trying to retrieve a file using urlretrieve, while adding a custom header.

While checking the codesource of urllib.request I realized urlopen can take a Request object in parameter instead of just a string, allowing to put the header I want. But if I try to do the same with urlretrieve, I get a TypeError: expected string or bytes-like object as mentionned in this other post.

What I ended up doing is rewriting my own urlretrieve, removing the line throwing the error (that line is irrelevant in my use case).

It works fine but I am wondering if there is a better/cleaner way of doing it, rather than rewriting my own urlretrieve. If it is possible to pass a custom header to urlopen, it feels like it should be possible to do the same with urlretrieve?

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

I found a way where you only have to add a few extra lines of code...

import urllib.request

opener = urllib.request.build_opener()
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
urllib.request.install_opener(opener)
urllib.request.urlretrieve("type URL here", "path/file_name")

Should you wish to learn about the details you can refer to the python documentation: https://docs.python.org/3/library/urllib.request.html


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...