Is it expected that waiting on updating a record in index can take many seconds? How to deal with this on UI?

Hello,

My team currently has a search UI page that is a list of user records which are the hits from searching our users index. User can also add new user records from this same page and we need to be able to show them the newly added record in the search results right after they add a new user (or update a user) record in our index.

To achieve this reactivity, we have tried following the suggested approach of waiting (.wait()) on the algolia partialUpdateObjects/saveObjects operation which we call on the server when a new record is created or updated. When the server operation finishes when then refresh our search query on the UI to see the newly updated search results, as described in the documentation Index Operations Are Asynchronous | In depth | Sending and Managing Data | Guide | Algolia Documentation

This shows the updated search results but unfortunately it takes way too long. The .wait() call on the write operation (saveObjects or partialUpdateObject) takes many seconds (6+ seconds at least) for us and this significantly above what is acceptable UX for our use case.

This leads to the following questions:

  • Is it expected that the wait() call on algolia updates can take many seconds?
  • What are the recommended approaches for handling this situation on such front-end UIs? (we are using react instantsearch)

Thanks in advance!

  • Carlos

Hi @carlos,

It’s possible that it takes a bit of time to get records fully updated. This depends on a lot of factors, such as the number of updates that are pending on each machine in your cluster, the size of your index, and more. That being said, there may be small changes you can make to make this process a bit faster. Which API client (language + version) are you using to perform the wait operation?

Regarding the front end: I don’t know your current UX, but would it be possible for you to programmatically create a hit and add it at the top of the list? This gives the user the impression of having their result added instantly. You can then use the wait logic to ensure the record has been added successfully.

Cheers,
Devin.

Hi @devin.beeuwkes - thanks for the response!

Regarding your first question about which API client we’re on:

Our server uses the Javascript API client to call to create/update the algolia record upon a front-end action. Our server uses the installed npm package algoliasearch at version 4.5.1.

We use the SearchIndex.partialUpdateObjects and SearchIndex.saveObjects methods like so when we wait on them:

await myIndex.partialUpdateObjects([... at most 4 records...]).wait()

I have profiled this call and it takes 5+ seconds regularly on our index.

Regarding your front-end suggestion:

That is an interesting approach, I could programmatically add a Hit for the created record but I would also have to make sure it matches on any subsequent instantsearch query on the front-end until we have the refreshed results. This seems to be a lot more complex/error-prone manual work than I expected for what seems to be a fairly common UI pattern – is this the recommended approach in these cases? is there any documentation on how to approach this? Also note that we cannot call .wait() on the front-end because we call the algolia write operation on the back-end.

Also, any information on how to best speed up the completion of indexing when we’re partially updating a field on a record would also help, as that seems like the ideal solution as then the wait flow would work in a reasonable amount of time.

Thanks again for taking a look!

1 Like

@devin.beeuwkes I’m interested in improvement/solution of this as well, any pointers either on Carlos’s question, or your idea of adding a hit on front end?

Thank You,

Hi @rustamd, There is little you can do to speed up the updates as all the indexing jobs are added to a queue and the speed will depend on the other jobs in the queue, size of the indices, search requests happening at the same time, etc. We have no SLA on indexing.

As for adding a hit on the front end, when you send the update, you should have some idea of what should be included in the results and should be able to optimistically add the change to your page. This would very much depend on your own UI as to how you would add this.