elasticsearch - Proper access policy for Amazon Elastic Search Cluster

Question

I've recently started using the new Amazon Elasticsearch Service and I can't seem to figure out the access policy I need so that I can only access the services from my EC2 instances that have a specific IAM role assigned to them.

Here's an example of the access policy I currently have assigned for the ES domain:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::[ACCOUNT_ID]:role/my_es_role",
        ]
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:us-east-1:[ACCOUNT_ID]:domain/[ES_DOMAIN]/*"
    }
  ]
}

But as I said, this doesn't work. I log into the EC2 instance (which has the my_es_role role attached to it) and attempt to run a simple curl call on the "https://*.es.amazonaws.com" end point, I get the following error:

{"Message":"User: anonymous is not authorized to perform: es:ESHttpGet on resource: arn:aws:es:us-east-1:[ACCOUNT_ID]:domain/[ES_DOMAIN]/“}

Does anyone know what I have to change in the access policy in order for this to work?

Answer 1

You can lock access down to IAM-only, but how will you view Kibana in your browser? You could setup a proxy (see Gist and/or NPM module) or enable both IAM and IP-based access for viewing results.

I was able to get both IAM access IP-restricted access with the following Access Policy. Note the order is important: I could not get it working with the IP-based statement before the IAM statement.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::xxxxxxxxxxxx:root"
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:us-west-2:xxxxxxxxxxxx:domain/my-elasticsearch-domain/*"
    },
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:us-west-2:xxxxxxxxxxxx:domain/my-elasticsearch-domain/*",
      "Condition": {
        "IpAddress": {
          "aws:SourceIp": [
            "192.168.1.0",
            "192.168.1.1"
          ]
        }
      }
    }
  ]
}

My EC2 instance has an instance profile with the arn:aws:iam::aws:policy/AmazonESFullAccess policy. Logstash should sign requests using the logstash-output-amazon-es output plugin. Logstash running on my EC2 instance includes an output section like this:

output {
    amazon_es {
        hosts => ["ELASTICSEARCH_HOST"]
        region => "AWS_REGION"
    }
    # If you need to do some testing & debugging, uncomment this line:
    # stdout { codec => rubydebug }
}

I can access Kibana from the two IPs in the access policy (192.168.1.0 and 192.168.1.1).