elasticsearch - Elastic Search multi-value field aggregation

Question

My indexed documents have a schema:

{
  ...
  'authors': [{'first name': 'John', 'last name': 'Smith'},
              {'first name': 'Mark', 'last name': 'Spencer'}]
  ...
}

I would like to search them and aggregate by the individual authors, so get a list with top authors which occurred in my hits. Terms aggregation seems to be a match for my needs, but I'm not able to get it working for field with a list of values. Any help?

Answer 1

You will probably want to use a nested type, then you can use a nested aggregation on the author names.

As an example, I set up a simple index like this:

PUT /test_index
{
   "settings": {
      "number_of_shards": 1
   },
   "mappings": {
      "doc": {
         "properties": {
            "title": {
               "type": "string"
            },
            "authors": {
               "type": "nested",
               "properties": {
                  "first_name": {
                     "type": "string"
                  },
                  "last_name": {
                     "type": "string"
                  }
               }
            }
         }
      }
   }
}

Then added a couple of docs:

PUT /test_index/doc/1
{
    "title": "Book 1",
   "authors": [
      {
         "first_name": "John",
         "last_name": "Smith"
      },
      {
         "first_name": "Mark",
         "last_name": "Spencer"
      }
   ]
}

PUT /test_index/doc/2
{
   "title": "Book 2",
   "authors": [
      {
         "first_name": "Ben",
         "last_name": "Jones"
      },
      {
         "first_name": "Tom",
         "last_name": "Lawrence"
      }
   ]
}

Then I can get the list of (analyzed) author last names with:

POST /test_index/_search?search_type=count
{
   "aggs": {
      "nested_authors": {
         "nested": {
            "path": "authors"
         },
         "aggs": {
            "author_last_names": {
               "terms": {
                  "field": "authors.last_name"
               }
            }
         }
      }
   }
} 
...
{
   "took": 71,
   "timed_out": false,
   "_shards": {
      "total": 1,
      "successful": 1,
      "failed": 0
   },
   "hits": {
      "total": 2,
      "max_score": 0,
      "hits": []
   },
   "aggregations": {
      "nested_authors": {
         "doc_count": 4,
         "author_last_names": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
               {
                  "key": "jones",
                  "doc_count": 1
               },
               {
                  "key": "lawrence",
                  "doc_count": 1
               },
               {
                  "key": "smith",
                  "doc_count": 1
               },
               {
                  "key": "spencer",
                  "doc_count": 1
               }
            ]
         }
      }
   }
}

Here is the code I used:

http://sense.qbox.io/gist/ca94cc11a12f8e4fed5c62c52966128b9a6f58de