python - Why doesn't the json() method on the requests module return anything?

Question

When I call the json() method on request response I get an error.

• Any suggestions to what could be wrong here?

My code:

import requests
import bs4

url = 'https://www.reddit.com/r/AskReddit/comments/l4styp/serious_what_is_the_the_scariest_thing_that_you/'

rsp = requests.get(url)

sc = rsp.json()

print(sc)

Output:

File "c:VS_Code1scrape.py", line 6, in <module>
     sc = rsp.json()
File "C:UsersUserAppDataLocalPackagesPythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0LocalCachelocal-packagesPython39site-packages
equestsmodels.py", line 900, in json
  return complexjson.loads(self.text, **kwargs)
File "C:Program FilesWindowsAppsPythonSoftwareFoundation.Python.3.9_3.9.496.0_x64__qbz5n2kfra8p0libjson\__init__.py", line 346, in loads
  return _default_decoder.decode(s)
File "C:Program FilesWindowsAppsPythonSoftwareFoundation.Python.3.9_3.9.496.0_x64__qbz5n2kfra8p0libjsondecoder.py", line 337, in decode
  obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:Program FilesWindowsAppsPythonSoftwareFoundation.Python.3.9_3.9.496.0_x64__qbz5n2kfra8p0libjsondecoder.py", line 355, in raw_decode
  raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 2 column 5 (char 5)

question from:https://stackoverflow.com/questions/65909340/why-doesnt-the-json-method-on-the-requests-module-return-anything

Answer 1

Your rsp is actually returning <Response [200]> which is not a JSON. If you want to read the content of the response, you can simply do:

rsp.text

What you get from the URL you posted here is HTML, and not JSON.