Merge Multiple Query results and apply filters on that

Hi All,

Greetings !

I have one problem here, Any help will be much appreciated. I have two indexes one is parent and another is child.

For example
Parent index structure look like this

Child Index
image

So the relation is look like parent_group_key and neck_finish is used to group the result from child index and get all the records of child index to display on UI.

Parent index have multiple group keys associated with neck finish values.

I am able to get the data of child index using of multi query approach.

like first get all data from parent index and create dynamic query to child index as mentioned below.

product_group_key  = x AND ( neckfinish = a OR neckfinish = b .......) 
product_group_key  = y AND ( neckfinish = a1 OR neckfinish = b1 .......) 
...
...
..

Multi query in return gives me multiple result and from font-end I have merged all this Hits results and display to the page. now challenge is if I have to apply filter on these result how I can do that ? I have filter like “size”, “shape” etc. Also I need to use Facets for some specific attributes.

   try {
         data.results.forEach(result =>
           result.hits.forEach(hit => {
             filterGroupKey = hit.include_product_groups.map(groupKey => {
               var obj = {};
               obj = {
                 groupKey: `productGroupKey : ${groupKey.product_group_key}`,
                 neckfinish: groupKey.neck_finish
                   ? `( ${groupKey.neck_finish
                       .map(nf => `neckfinish : '${nf}'`)
                       .join(' OR ')} )`
                   : '',
               };
               return obj;
             });
           })
         );
       } catch (e) {
         console.log(e);
       }
 
       const assemlKey = filterGroupKey.map(x => {
         return x.neckfinish === ''
           ? x.groupKey
           : `${x.groupKey} AND ${x.neckfinish}`;
       });
 
       let queriesForMultiIndecies = [];
       assemlKey.forEach(x => {
         queriesForMultiIndecies.push({
           indexName: 'products',
           params: {
             filters: x,
           },
         });
       });
 
       console.log(queriesForMultiIndecies);
 
       return algoliaClient.search(queriesForMultiIndecies).then(res => {
         console.log('result of multi query');
         try {
           let composedHitArray = [];
           let responseFirstElement = res.results[0];
 
           res.results.forEach(result => {
             if (result.hits.length === 0) {
             } else {
               composedHitArray.push(...result.hits);
             }
           });
 
           console.table(composedHitArray.length);
           console.log(composedHitArray.map(x => x.productGroupKey).join(' '));
 
           return Promise.resolve({
             results: [
               {
                 hits: composedHitArray,
                 nbHits: responseFirstElement.nbHits,
                 nbPages: responseFirstElement.nbPages,
                 processingTimeMS: responseFirstElement.processingTimeMS,
               },
             ],
           });
         } catch (e) {
           console.log(e);
         }
       });

Thank you.

Hi @matt.newbill,

Thanks for contacting Algolia. If I understand your question correctly you are running a multi-query search, where first you query the Parent index and then use that data to follow-up queries to Child indices and merge results. However, your next goal is to use facet filters on these merged results. Is that correct?

In the current structure of your data, this approach to run multi-queries and then merge is the right approach. But filtering on this merged data out-of-the-box with Algolia will not be doable.

Merging is an anti-pattern in search. The recommended approach if you needed to run subsequent facet filtering on your results would be to merge your index.

Can you consider restructuring into a single, large index, such that you can get a single set of results and filter more-easily with an Algolia out-of-the-box widget?

1 Like

Hi @ajay.david, Thanks for your response message !

Actually merging the index would not solve our problem. we need a parent index that control the flow of child index data and decide which records needs to be show/hide. though instead of adding child index with extra attributes we decided to have parent index that control the behavior and can be maintain easily.

what you suggest if we some how restructure the parent index in more convenient so query can be easily apply (without uses of multi query) any suggestion would be warm welcome.

Thanks

Hi @matt.newbill, I’m not entirely sure I understand the restriction on data flow from parent to child index.

If you restructure as @ajay.david suggests (and as outlined here: https://docs.google.com/presentation/d/11fD2ilGIDSHCcXIg-AkkvBmM-3z1fY8FPdEnAr2pBTU/edit?usp=sharing) could you not apply the correct filters to get only the products with the correct productGroupKey and neckFinish you are looking for? And from there apply any other facet filters necessary?

In a nutshell, there would be just one index (the parent), with nested information for each child.

Looking forward for any further specifics you can give on the restriction.

@ajay.david Also one more thing, if I will use single index to get the result. then how i can do it like I have multiple group key AND neckfinish attribute that i have to use to get the data.

this is like ProductGroupKey AND Neckfinish with different values.

Hi @matt.newbill, if I understand correctly, does each productGroupKey have its own particular neckfinish values?

Instead of what I suggested earlier ( one index (the parent), with nested information for each child.) I believe you would need to create one record per child, and decorate them with the data from the parent.

So each record would look like the child record you showed, but also include the information from the parent (e.g. the “name” attribute).

Would that work?

@maria.schreiber, I also think about flat data structure for parent index but updating back to child index with that will be work i think , Actually my child index already have lot of data that i have to update Is it possible to connect quickly and discuss more on that ?

Hey @matt.newbill, to discuss further it would probably be better to reach out to support@algolia.com (I’m one of the folks on that mailbox), feel free to reference this discourse thread for context.

Also feel free to continue discussing here for anyone else who might be facing similar issues. :slight_smile:

When you say flat data structure for your parent – do you mean having one record per child in a master index?

And if so, indeed, there would be need for more updates since any update you would normally make to a record in the parent index would now need to be reflected as updates for each child in a master index.

@maria.schreiber, Sure I will more coordinate on the mail mentioned above. Yes you are right need to maintain more update if I will have flat data structure. Meantime I will follow up the approach you mentioned above and see how it works. Thank you for your help and support. Have a great day :slight_smile:

1 Like

@maria.schreiber, Just little help here, can you define structure for above problem with your suggested approach ( one index (the parent), with nested information for each child.) I believe you would need to create one record per child, and decorate them with the data from the parent.

Thanks

Hi @matt.newbill,

I believe Maria is suggesting one of two different solutions.

  1. one index (the parent), with nested information for each child, or
  2. create one record per child and decorate them with the data.
  1. {
    “parent”: “name”,
    “children”: {
    “child1”: “data”,
    “child2”: “data”
    }
    }

  2. {
    “parent”: “name”,
    “child”: “data for child 1”
    },
    {
    “parent”:“name”, // same parent information as the record above
    “child”:“data for child2” //different child information
    }

Let us know how this works for you or if you have more questions!