python - How to handle errors with boto3?

Question

I am trying to figure how to do proper error handling with boto3.

I am trying to create an IAM user:

def create_user(username, iam_conn):
    try:
        user = iam_conn.create_user(UserName=username)
        return user
    except Exception as e:
        return e

When the call to create_user succeeds, I get a neat object that contains the http status code of the API call and the data of the newly created user.

Example:

{'ResponseMetadata': 
      {'HTTPStatusCode': 200, 
       'RequestId': 'omitted'
      },
 u'User': {u'Arn': 'arn:aws:iam::omitted:user/omitted',
           u'CreateDate': datetime.datetime(2015, 10, 11, 17, 13, 5, 882000, tzinfo=tzutc()),
           u'Path': '/',
           u'UserId': 'omitted',
           u'UserName': 'omitted'
          }
}

This works great. But when this fails (like if the user already exists), I just get an object of type botocore.exceptions.ClientError with only text to tell me what went wrong.

Example: ClientError('An error occurred (EntityAlreadyExists) when calling the CreateUser operation: User with name omitted already exists.',)

This (AFAIK) makes error handling very hard because I can't just switch on the resulting http status code (409 for user already exists according to the AWS API docs for IAM). This makes me think that I must be doing something the wrong way. The optimal way would be for boto3 to never throw exceptions, but juts always return an object that reflects how the API call went.

Can anyone enlighten me on this issue or point me in the right direction?

Answer 1

Use the response contained within the exception. Here is an example:

import boto3
from botocore.exceptions import ClientError

try:
    iam = boto3.client('iam')
    user = iam.create_user(UserName='fred')
    print("Created user: %s" % user)
except ClientError as e:
    if e.response['Error']['Code'] == 'EntityAlreadyExists':
        print("User already exists")
    else:
        print("Unexpected error: %s" % e)

The response dict in the exception will contain the following:

• ['Error']['Code'] e.g. 'EntityAlreadyExists' or 'ValidationError'
• ['ResponseMetadata']['HTTPStatusCode'] e.g. 400
• ['ResponseMetadata']['RequestId'] e.g. 'd2b06652-88d7-11e5-99d0-812348583a35'
• ['Error']['Message'] e.g. "An error occurred (EntityAlreadyExists) ..."
• ['Error']['Type'] e.g. 'Sender'

For more information see:

• boto3 error handling
• botocore error handling

[Updated: 2018-03-07]

The AWS Python SDK has begun to expose service exceptions on clients (though not on resources) that you can explicitly catch, so it is now possible to write that code like this:

import botocore
import boto3

try:
    iam = boto3.client('iam')
    user = iam.create_user(UserName='fred')
    print("Created user: %s" % user)
except iam.exceptions.EntityAlreadyExistsException:
    print("User already exists")
except botocore.exceptions.ParamValidationError as e:
    print("Parameter validation error: %s" % e)
except botocore.exceptions.ClientError as e:
    print("Unexpected error: %s" % e)

Unfortunately, there is currently no documentation for these errors/exceptions but you can get a list of the core errors as follows:

import botocore
import boto3
[e for e in dir(botocore.exceptions) if e.endswith('Error')]

Note that you must import both botocore and boto3. If you only import botocore then you will find that botocore has no attribute named exceptions. This is because the exceptions are dynamically populated into botocore by boto3.

You can get a list of service-specific exceptions as follows (replace iam with the relevant service as needed):

import boto3
iam = boto3.client('iam')
[e for e in dir(iam.exceptions) if e.endswith('Exception')]

[Updated: 2021-09-07]

In addition to the aforementioned client exception method, there is also a third-party helper package named aws-error-utils.