Missing Facets due to query approximation

Open Q&A
,
#1

Hi folks,

We are having a strange issue with our index master_search.

We have several attributes for faceting and the main one is “modelType”. There are 8 possibilities for this facet.

However when loading our application it is frequent that we see only 3 or 4 of those possibilities (the “limit” variable in the refinement list React widget is set to 8). But some days there are all displayed without any change in the configuration or application code base.

After reading this section : Why are my facet and hit counts not accurate? | Configuration & Relevance FAQ | Docs Algolia it appears that Algolia does an approximation when the search query is too complex (= reach the timeout which is 50ms I believe).

Indeed our queries were returning : exhaustiveFacetsCount: and exhaustiveNbHits as false for most of our tries

Following this article we made some tests :

  • we have reduced the numbers of searchableAttributes from 30+ to 6
  • we have reduced the numbers of attributesForFaceting from 40+ to 10

We directly changed the index configuration on algolia dashboard, waited for the index to be properly updated (I can confirm since I was not seeing some facets on the app anymore) and ran some search in our app interface.

This didn’t change anything basically. We still got exhaustiveFacetsCount: and exhaustiveNbHits as false most of the times and some missing facets types

I believe I’m missing something in the big picture, therefore I have several questions :

  • Is there some sort of cache that could prevent the changes we made to the index to be reflected in the query complexity (I kind of doubt it, as the RefinementList in our app seemed to have the latest data)
  • Is there a magic number we should aim to reach for processingTimeMS like the timeout mentioned above ?
  • Are there any others obvious config that could add hidden complexity to the query ?
  • We tried to test the query from the browser tab in algolia dashboard and through our application. The results seem to differ quite a lot. processingTimeMS is roughly 20ms on the dashboard while 50/100 ms through our app (using react instantsearch), any explanation - since the doc specifically mentions that processingTimeMS excludes network time ?
  • Any other ideas :smile: ?

Bottom line we can deal with not exact count displaying approximation, but we would definitely need all facets to be present as they are basically the categories displayed to the user.

Any help would be greatly appreciated,

Thanks a lot, your product rock guys.

1 Like
#2

Hi @matthieu.veillon,

Thanks for your detailed question — I see you already tried some solutions highlighted in our documentation :slight_smile:

We tried to test the query from the browser tab in algolia dashboard and through our application. The results seem to differ quite a lot. processingTimeMS is roughly 20ms on the dashboard while 50/100 ms through our app (using react instantsearch)

Do you do extra filtering / optionalFiltering / faceting from your front-end? Could you share with us the payload of the query sent from React InstantSearch?

An alternative solution would be to use the searchForFacetValues method from our JS client (Search for Facet Values | Search | Method | API Reference | Algolia Documentation) but I’m not sure it would prevent the timeout.

Don’t hesitate to grant us Support access from your Algolia Dashboard > Account and to contact us on support@algolia.com so we can have a look at your index, records structure and configuration and better guide you.

Thanks!

#3

Hi Valentin,

Thanks for your answer.

I just granted you access to our dashboard. One of my teammate was already in contact with Jordan from the support team, but wanted to give a try on the forum. Should I contact back Jordan with the extra information provided here ? Otherwise I can provide you the app ID.

In the meantime to answer your question we have 2 Ui interface with slightly different queries which are based on the same index and suffer the same issue. Might help diagnosis the issue better !

We do have for one of them the optionalFilter enabled but it’s null by default. And since our issue is shared by the 2 apps…

Thanks a lot

App1 :

{“requests”:[{“indexName”:“master_search”,“params”:"highlightPreTag:
highlightPostTag:
maxValuesPerFacet: 20
query:
optionalFilters: null
facetingAfterDistinct: true
page: 0
facets: [“modelType”,“sector”,“theme”,“language”]
tagFilters: "}]}

App2 : (paying attention to it, this one seems weird with a query alike object within the filter.

{“requests”:[{“indexName”:“master_search”,“params”:“highlightPreTag:
highlightPostTag:
facetingAfterDistinct: true
getRankingInfo: true
query:
maxValuesPerFacet: 10
facets: [“modelType”,“language”,“updatedAt”]
tagFilters:
numericFilters: [“updatedAt<=1615417199”]”},{“indexName”:“master_search”,“params”:"highlightPreTag=
highlightPostTag:
facetingAfterDistinct: true
getRankingInfo: true
query:
maxValuesPerFacet: 10
hitsPerPage: 1
page: 0
attributesToRetrieve:
attributesToHighlight:
attributesToSnippet:
tagFilters:
analytics: false
clickAnalytics: false

#5

Hi Matthieu,

I had a look at your application, and to the former conversion with Jordan. Indeed it seems to be a problem related to the size of your index. As there is an important number of records in your index exhaustiveFacetsCount should be set to true.

As it relates to your index and configuration, the best place to continue this conversion is to go back on the discussion with our support team.