google api - Python OAuth after few days fails refreshing tokens with "invalid_grant" error

Question

I have a Python app running on Raspberry Pi that starts a livestream to a YouTube channel that I manage. This is the code that I use to authenticate:

import google_auth_oauthlib.flow
import googleapiclient.discovery
import googleapiclient.errors
import google.auth.transport.requests
import google.oauth2.credentials
import requests

CLIENT_SECRETS_FILE = "client_secrets.json"
YOUTUBE_READ_WRITE_SCOPE = "https://www.googleapis.com/auth/youtube"
YOUTUBE_API_SERVICE_NAME = "youtube"
YOUTUBE_API_VERSION = "v3"

def get_authenticated_service(args):
    credentials = None
    credentials_json_file = "youtube-%s.json" % slugify(args.oauth2_name)
    if os.path.exists(credentials_json_file):
        # load credentials from file
        with open(credentials_json_file, encoding='utf-8') as f:
            credentials_json = json.load(f)
        credentials = google.oauth2.credentials.Credentials.from_authorized_user_info(credentials_json)
    if not credentials or not credentials.valid:
        # no credentials file or invalid credentials
        if credentials and credentials.expired and credentials.refresh_token:
            # refresh
            request = google.auth.transport.requests.Request()
            credentials.refresh(request)
        else:
            # re-authenticate
            flow = google_auth_oauthlib.flow.InstalledAppFlow.from_client_secrets_file(CLIENT_SECRETS_FILE, [YOUTUBE_READ_WRITE_SCOPE])
            credentials = flow.run_console()
        # save credentials to file
        credentials_json = credentials.to_json()
        with open(credentials_json_file, 'w', encoding='utf-8') as f:
            f.write(credentials_json)
    return googleapiclient.discovery.build(
        YOUTUBE_API_SERVICE_NAME, YOUTUBE_API_VERSION, credentials=credentials)

When I run my app a first time, I must authenticate. The credentials are stored in a JSON file that looks like this:

{
  "token": "...", 
  "refresh_token": "...", 
  "token_uri": "https://oauth2.googleapis.com/token", 
  "client_id": "....apps.googleusercontent.com", 
  "client_secret": "...", 
  "scopes": ["https://www.googleapis.com/auth/youtube"],    
  "expiry": "2021-02-28T09:27:44.221302Z"
}

When I re-run the app later on, I don't have to re-authenticate. That works fine.

But after 2-3 days, I get this error:

Traceback (most recent call last):
  File "./create_broadcast.py", line 188, in <module>
    youtube = get_authenticated_service(args)
  File "./create_broadcast.py", line 83, in get_authenticated_service
    credentials.refresh(request)
  File "/home/pi/.local/lib/python3.7/site-packages/google/oauth2/credentials.py", line 214, in refresh
    scopes,
  File "/home/pi/.local/lib/python3.7/site-packages/google/oauth2/_client.py", line 248, in refresh_grant
    response_data = _token_endpoint_request(request, token_uri, body)
  File "/home/pi/.local/lib/python3.7/site-packages/google/oauth2/_client.py", line 124, in _token_endpoint_request
    _handle_error_response(response_body)
  File "/home/pi/.local/lib/python3.7/site-packages/google/oauth2/_client.py", line 60, in _handle_error_response
    raise exceptions.RefreshError(error_details, response_body)
google.auth.exceptions.RefreshError: ('invalid_grant: Token has been expired or revoked.', '{
  "error": "invalid_grant",
  "error_description": "Token has been expired or revoked."
}')

The workaround is to delete the credentials file and re-authenticate. But I'd expect the credentials refresh to still work after a couple of days!

I do have NTP installed and running. I didn't manually revoke the token. I didn't change my Google password. I didn't generate a lot of other tokens elsewhere. I did none of the things that are told elsewhere to cause this error.

One thing to note: the app is not verified, because it's only meant for internal use. Still this shouldn't impact the lifespan of the refresh token, should it?

What could make that refreshing works after 1 day or after 2 days, but not anymore after 3 days?!

Best regards, Vic

Answer 1

I very much presume that your corresponding Google project shows -- from within the Developers Console, on the page OAuth consent screen -- your app to have its Publishing status set as Testing.

According to the official documentation, your refresh tokens are subject to the following restrictions:

Refresh token expiration

You must write your code to anticipate the possibility that a granted refresh token might no longer work. A refresh token might stop working for one of these reasons:

[...]

A Google Cloud Platform project with an OAuth consent screen configured for an external user type and a publishing status of "Testing" is issued a refresh token expiring in 7 days.

Until your app's publishing status becomes set to in production -- i.e. your app gets audited by Google --, the above restriction implies that you have to run to successful completion an OAuth authentication/authorization flow every week for to obtain new refresh tokens (that have a limited lifespan of only 7 days).