Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
97 views
in Technique[技术] by (71.8m points)

python - How can I access this type of site using requests?


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

This page needs to specify some HTTP headers to obtain the information (Accept-Language):

import requests
from bs4 import BeautifulSoup

headers = {'Accept-Language': 'en-US,en;q=0.5'}

def html(url):
    return BeautifulSoup(requests.get(url, headers=headers).content, "lxml")

url = "https://www.g2a.com/"

soup = html(url)

print(soup.prettify())

Prints:

<!DOCTYPE html>
<html lang="en-us">
 <head>
  <link href="polyfill.g2a.com" rel="dns-prefetch"/>
  <link href="images.g2a.com" rel="dns-prefetch"/>
  <link href="id.g2a.com" rel="dns-prefetch"/>
  <link href="plus.g2a.com" rel="dns-prefetch"/>

... and so on.

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...