Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
254 views
in Technique[技术] by (71.8m points)

ruby on rails - Querying before Elasticsearch index update is finished returns old data

I'm using searchkick gem with option:

 searchkick callbacks: :async

There is a case where I delete a record and the client immediately calls the index endpoint to list all records. But since the ES index was not updated yet by the background job, it gets old data.

I'm looking for ideas on how to solve this. Is there a cheap way to check if an index is being updated?

question from:https://stackoverflow.com/questions/65925676/querying-before-elasticsearch-index-update-is-finished-returns-old-data

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

This happens cause of refresh_interval property of your index. If not set explicitly, it's default values is 1 (which I am assumging is your case).

From documentation:

By default, Elasticsearch periodically refreshes indices every second, but only on indices that have received one search request or more in the last 30 seconds.

Solution 1

You can use the refresh parameter of Elasticsearch to overcome your problem but at your own risk. I addressed a similar problem here.

curl -X PUT "localhost:9200/test/_doc/1?refresh&pretty" -H 'Content-Type: application/json' -d'
{"test": "test"}
'
curl -X PUT "localhost:9200/test/_doc/2?refresh=true&pretty" -H 'Content-Type: application/json' -d'
{"test": "test"}
'

Solution 2

There is a support to explicitly refresh your index.

curl -X POST "localhost:9200/my-index-000001/_refresh?pretty"

Note: Above mentioned solutions are not cheap, in-fact are expensive ones and can cause performance issues if you have large number of documents inside the index.

Solution 3

(If you have a backend and RDS layer in between of your client and ES) Once you have the response from ES, delete the entries from the final response which are not present in RDS.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...