Atomical reindexing: indexes sync

Hello !
By running an atomical reindexing, a temporary index index_name.tmp is created to index all records.
I would like to know if both indexes index_name and index_name.tmp are kept synchronised during the reindexing.

Ex: I have a Model with 500K entries, i run Model.reindex, the object with the id = 10 is indexed in the temporary index. While the rest of the records are indexed in the temporary, the object with id =10 is modified and indexed. Is the object is indexed in both indexes ? only in main index ?

Thanks.

Hello Stéphane!

There is no link between the temporary index and the final index, no sync mechanism.

What happens under the hood when you perform an atomic reindexing (either with the replaceAllObjects method with one of our API clients, or the reindex strategy of one of our integrations) is that we first create the temporary index, then index all the objects to it, then replace the old one with the new one.

Indexing operations are asynchronous. What you are actually doing when calling these methods is adding a new job to a queue: it is this job, and not the method, that actually performs the desired action. In most cases, the job is executed within seconds if not milliseconds. But it all depends on what is in the queue: if the queue has many pending tasks, the new job will need to wait its turn.

We do not recommend performing an atomic reindexing and an update at the same time. Depending on your implementation and the quality of your network, you could have a race condition resulting in inconsistent indexing: if your update task is ran after your reindex, but it reaches the queue sooner, then it will be overwritten by the reindexing happening right after, and will only be properly indexed during the next reindexing.

To ensure that tasks are queued in order, you should either rely on the asynchronous mechanisms of your programming language, or implement a queue on your end to ensure you don’t have concurrent processes running at the same time.

Does this help?

Best.

Thanks a lot Sarah for this detailed answer!

Actually I’m writing an Elixir library to integrate Algolia.
If I want to implement the atomic reindexing I could ensure that if an object is updated during the lifecycle of the temporary index, the given is first updated in the primary index but also in the temporary index.
This will decrease the risk to have a race condition and It would make the temporary index lot of more consistent.