Using wait() on larger indexes

Open Q&A
, ,
#1

Hi!

I have a frontend table that renders my records from Algolia. In order to keep this UI up to date, i’m currently using saveObject.wait().then(...) and it’s working great - operations generally take just a hair longer than if I didn’t use it. My concern though is that my index is only at like 200 records at the moment. Will the time it takes to get a response from wait() be a lot longer if my index has thousands of documents in it, and if so, any ballpark idea of how much slower? I’m generally only waiting for single operations, doing one save/update at a time.

I’m aware that wait should be used sparingly, but it’s been the most reliable and clean way to do it so far.

Thanks!

#2

Hi Steve,

Can you share more details about why you need to wait and what goes into the thencallback?
If the batch is accepted and returned a 200, it will be indexed. Error checking (like record size) happens before accepting the batch.

It’s hard to give an estimate. I’d say that a few thousands records can still be considered small but a lot of things will influence the time to index: the load of the cluster, the size of the records, the complexity of the configuration and so on.

Please let me know if that helps.
Julien

#3

Thanks for the reply Julien! The context is I have a list of records in a table view where the user can perform various actions on a record. If they for example delete the record, I need the table to get rid of that record so that they can’t still see it after it was deleted etc. if there’s a better solution than waiting I’d love to hear it - before I knew there was a wait method I’ve had arbitrary asynchronous timeouts going before refreshing the table and that’s obviously a terrible solution. As it is right now I quite like the process of using wait as it’s still quick and my table is reliably up to date after each action. My only concern is if the time it takes will start being 30 sec instead of 2 sec. This seems like it must be a common issue that people need to tackle - are there better approaches? I’ve thought about if I can pull data for the table straight from my DB and still benefit from the algolia text search but not sure how that would work. Thanks again!

#4

I understand, I think it’s a good use case. It seems very similar to what we do in the dashboard.

I think you can week the wait(). It might happens that sometimes it’s longer but normally deleting/indexing a few records is very fast.

I’d recommending handling a loading state, like disabling the entire table row when something is being edited/deleted but that’s all .

About pulling data from the database, it’s something I’ve done personally and I think it’s a good solution. It would look like:

  1. search algolia and retrieve most relevant records based in the current state of the index. It means if somehting was deleted but not yet deleted in Algolia, you’d retrieve it.
  2. based on the results, you keep only your database primary key (often the objectID but it depends on your implementation)
  3. Fetch data from your DB based on those primary keys. The deleted model that you found in algolia won’t be retrieved from the database.

It’s a bit longer than using Algolia hits directly because it adds queries to the database but you get the benefit for working with actual models rather than json objects. (depends what you use in the backend I guess).

#5

Awesome response - you rock! Yup I have a loading state so you can’t do anything to the table until I get the fresh response from Algolia. But very interesting appracoh with the DB. I’ll probably stay with the wait() approach for now but great to know about an approach I can use elsewhere or if the current approach becomes to boggled down. Thanks again!!

1 Like