Backend search with custom/local Filtering

I’m considering Algolia for a use-case that would involve an InstantSearch frontend, with a custom backend search similar to the one illustrated here (Implement back-end search with InstantSearch.js | Algolia) but instead of modifying the records returned from Algolia’s servers, I’d like to remove selected records.

We’d be removing search hits based on custom made geolocation filterings that are still not possible using Algolia geo features.

Everything works correctly and I can send to the front end the right hits, the main problem is just pagination - because we’d be removing search results, pages as seen by the UI won’t align with pages from the index, and so in order to serve e.g. the second page of results to the UI, I have no idea what offset I’d need to start with when reading from the index unless there was a way to remember where I left off on the first page, and preserve that context between successive API calls from the UI.

Does such a capability exist, or are there other ways of approaching this problem that folks could point me to?

1 Like

Hey @iurienunu

Since the pagination is constructed and built within the Algolia search service. If you remove records from the API response, the pagination will always be off. You’ll need to pull in batches of records using your backend service then render your own pagination capability based on the modified result set. The back end service would have to keep track of page boundaries on both the front end and the back end to know when to request more records from the search service.

Out of curiosity, what is the missing geosearch feature that’s causing you to go down this path?

Hey @chuck.meyer
The missing geosearch feature is the following: basically I’ve got an app where I’ve got customers and sellers, the seller has got a Store Location, and a radius in kilometers within which he can ship his goods. The customer when makes a search for that good specify his address first, so he can retrieve the seller that has got that address within the radius from the selling point.
Eg: Seller A with location in New York with radius of ship of 500Km, if customer sets his address in Philadelphia and does the search, he will retrieve the seller A in the results. If customer sets his address in Los Angeles he won’t retrieve the seller A in the results.

So it’s basically a kind of reverse geo search by radius, that I don’t think we can do with Algolia, correct me if I’m wrong please :smiley:

Regarding your solution for the pagination, with your proposal I’m not sure how for example to know in advance the number of total hits (after my deletions).

Let’s say that I firstly use a custom searchClient for the Algolia search, how should I set the Configure component? How many hitsPerPage should I set? Because if I set like 10 hits per page and the total results after my filter are 5, I’m not sure how can I go on making a new Algolia request for the next 10 hits to try and complete the previous 5 results to reach 10 results per page? (I hope it makes sense) And with this approach there’s no way to understand the total amount of results in advance. The only way to do it seems to request an indefinite number of hits per page from Algolia (I guess the maximum is 1000 though) and then have all the batching logic in my back end. But it seems not efficient at all to always request the maximum number of hits… What do you think? What should exactly be the strategy/pattern here?

Thank you very much for the help @chuck.meyer

Got it. So the delivery radius extend from the stores/records and you only want the ones where the customer’s pin falls within the records delivery radius. yeah that’s an interesting one – let me ask around internally to see if folks have any ideas.

Per your other post – filtering the records yourself and keeping track of the pagination math between what Algolia sends you and what you display is going to be a real pain. You’ll basically have to go grab 3 or 4 pages from Algolia at a time, filter and serve the records with your hand-rolled pagination, and know when the server needs to go retrieve more records to fulfill the next page request. Seems messy.

Yes @chuck.meyer, that is exactly the geo behavior I want to achieve, I guess should be a common one for apps like JustEat Deliveroo etc… So yeah having a way to achieve this with Algolia would save me an incredible manual back end pain. Please let me know if you have some tips from the team for this, it would be much appreciated!

Hi @chuck.meyer, do you have any idea from the team about the geolocation issue?

We have a similar use case (not geo though) of wanting to further filter the results from Algolia based on logic from our server.

The problem even with the suggestion by Chuck to paginate several times, we’d still hit the 1000 record limit (or whatever it is) that Algolia would return, which could result in very few results actually being shown to the user, if any. (our index currently has over 50k records, and will continue to grow).

It would be nice if there was a way to inform Algolia somehow that these record IDs either do or do not match the criteria we’re using our backend to filter against, and for Algolia to do the other filtering that the index knows about…