Building search ui for large data

I’m working on a site with large data. There are about 200k products and 150k categories. Is it possible to build a search ui with these requirements:

  • A hierarchical menu of categories
  • Auto refine the categories menu if query partial match one of them

For example I got this category data:

category: {
    level1: ['Ford', 'Toyota'],
    level2: ['Ford /// Falcon', 'Ford /// Fairlane', 'Toyota /// Hilux']
}

If user searches for falcon break, results will contains all break products, but how to auto refine the facet to Ford /// Falcon?

Hi @thungthudh,

A hierarchical menu of categories

It should be possible to build one. We provide a HierarchicalMenu in our InstantSearch UI libraries that should solve this use case. Note that this menu doesn’t display an exhaustive list of facet values. Only the top N values are returned (N depends on the settings).

Auto refine the categories menu if query partial match one of them

Algolia has a feature called Rules that should help to solve this use case. The rules are a way to detect an intent from the search. You can trigger an action (e.g. set a filter) when the search matches a given term. You can find more information about that in the documentation.

Hope that helps!

Hi @samuel.vaillant,
Thank you for your answer, but I’m afraid query rules will not work with hierarchical data. If I understand correctly, search query must contains the facet value to activate. Therefore I must search Ford /// Falcon brake to make the rule working.
Please correct me if I understand anything wrong.

Hi @thungthudh,

You have multiple matching criteria for a rule. Let’s say the query is falcon break. You can set up a rule with a condition: query contains falcon. The consequence would be to apply query parameters: a filter on category.level2:"Ford /// Falcon". You can find more capabilities for rules in the documentation about user intent.

Hope that it’s clearer!

Hi @samuel.vaillant,

I’m sorry I think that will not work for my case. As I said, I have about 150k categories and it is just impossible to add every levels as rules. I spent a lot of time digging in document but not found anything for my problem.

I already can achieve this with a smaller set of data by getting all facet values and build the hierarchical menu on my own. However due to the limits it will not work for large data since I can get only maximum 1000 facet values per facet.

If it’s not possible to do some magic with Algolia settings, is there anyway I can get all facet values for a query?

Thank you.

Hi @thungthudh,

Yes, you can use the parameter facets with a wildcard * value in your search. See docs

For example, this query "hatchimals" returns a response with all the matching facet values:

index.search('hatchimals', {
  facets: ['*']
}).then( res => {
  console.log(res);
});

// response
{
...
facets: {
    type: { gifts: 2 },
    brand: { HATCHiMALS: 2 },
    color: { 'Sparkly Penguala': 1 },
    price: { '9': 1, '59': 1 },
    featured: { false: 2 },
    categories: { Toy: 2 },
    productGroup: { Toy: 2 }
  }
}

Hi @ajay.david,

I understand your reply, but what I want is exhaustively get all values of a facet. In my case the vehicle facet can hit thousands values, and I can only get maximum 1000 values to do the calculation. I understand the performance purpose of that limit, but is there any trick to get all values of a single facet?

Thank you.

Hi there,

The 1000 max is a hard limit at the engine level that you’re not gonna be able to work around unfortunately :confused:

1 Like

Hi there,

Indeed, it’s as @vincent.lemeunier described and not doable through Algolia.

Another way to think about it is that all those values are indexed to Algolia because they existed in your database/source-of-truth. You can get them exhaustively from your own database.