python - How to replace quotation marks within JSON dictionary value?

Question

I'm using Open Trivia DB's API to generate trivia questions.

import requests, json, urllib.parse
import 

url = "https://opentdb.com/api.php"

querystring = {"amount":"5","type":"multiple","encode":"url3986"}

response = requests.request("GET", url, params=querystring)
response_decoded = urllib.parse.unquote(response.text)
print(response_decoded)
response_dict = json.loads(response_decoded)
print(response_dict["results"][0])

However, I keep on running into an error one some occurrences, the error being:

Exception has occurred: JSONDecodeError
Expecting ',' delimiter: line 1 column 347 (char 346)

I've worked out that the error is because in some questions similar to Who is the main character in "gamename", there are quotation marks around gamename. In the context of the JSON I was returned it looks like this:

"question":"Who is the main character in "gamename"?",
"correct_answer":"maincharacter",
"incorrect_answers":["wrongname1","wrongname2","wrongname3"]

and the quotation marks around the gamename is messing the dictionary up.

Is there a way for me to replace the inner quotation marks only (around the gamename) to singular quotation marks, ', so that it doesn't mess up the dictionary structure?

question from:https://stackoverflow.com/questions/65895349/how-to-replace-quotation-marks-within-json-dictionary-value

Answer 1

Don't parse the response yourself. Use requests for that and it will handle the quotes:

import requests

resp = requests.get(
    "https://opentdb.com/api.php",
    params={"amount": "5", "type": "multiple", "encode": "url3986"},
)
resp.raise_for_status()  # raise an Exception if the HTTP response failed
data = resp.json()  # Parse JSON from the response
print(data["results"][0])  # print it

Looking at the data, some of the inner fields appear URL-encoded. Use urllib.parse.unquote for these:

import urllib.parse

print(urllib.parse.unquote("Entertainment%3A%20Music"))  # Entertainment: Music