Searching multiple indices with different fields; loading indicator

I’m currently implementing an react-instantsearch-powered UI where there are a handful of results “tabs”:

  • All
  • Foo
  • Bar
  • Baz
  • Quux

We have have a Posts index that all our content resides in. Each tab is used to refine the results by way of the facet filters. This works well; we can limit each tab’s items to whatever criteria we like. I’m using a lot of the provided connectors to integrate the provided InstantSearch component into our existing search results page.

In the “Bar” tab, we’d like to rank the results of type Bar differently, so we created a separate Bar index, which contains only posts of type Bar. The Bar index has a few additional fields that are indexed, including one new field (bar_title) that is to be used as the “title” of each item displayed in the search results. All of the Bar posts will continue to exist in the Posts index and will be displayed in the “All” tab, but they will be ranked and displayed slightly differently (different title) in the “Bar” tab. Hope this makes sense.

So now we’re dealing with multiple indices and things get tricky. Since it seemed straightforward and appropriate, I initially made the indexName prop dynamic on the InstantSearch component:

const target_index = (current_tab.index === undefined) ?
  'Post' : current_tab.index;

(where current_tab is a hash of all the tabs)

It’s then used in InstantSearch like so:

<InstantSearch
  // ...
  indexName={target_index}
  // ...
>

This is a good place to note that the search results page utilizes react-router (my top-level InstantSearchController component is wrapped in withRouter() and each change in the search state then pushes the new URL on to the history stack. So, the InstantSearchController gets new props from react-router every time the tab and other aspects of the search UI state changes. (This contributes to and exacerbates the issue.)

The above approach generally worked well until I attempted to implement the Bar-optimized title (bar_title) for the “Bar” tab. I had to then determine which field to display as the “title” visible to the user. To solve this, I added the notion of “field mappings” that are applied to each tab. Right now only the “Bar” tab has custom field mappings defined. It looks something like this for the “Bar” tab:

field_mappings: { title: 'bar_title' }

with the default being, of course,

field_mappings: { title: 'title' }

So here’s the main issue with the above. (There’s actually two separate issues).

My InstantSearchController component is driven by react-router and re-renders when the tab changes. During that re-render, the current tab is changed, which then changes the field_mapping. So when starting off on the “All” tab, here’s the order of events:

  • User loads page with some term pre-filled (e.g. “blargh”) and is on the default tab (“All”)
  • field_mappings is set to use the default title attribute in the hits.
  • Results render nicely without error
  • User clicks “Bar” tab; URL updates (&filter=bar) via react-router, thereby activating the “Bar” tab
  • field_mappings change to use bar_title
  • React generates a bunch of errors about the bar_title prop not being on the first hit. This makes sense, because React is trying to render the same data using a new field mapping. In other words, the new response data as a result of querying the “Bar” index has not loaded yet.
  • After the XHR for querying the Bar index completes, the results display without error.

This is clearly an issue because the UI essentially has two masters: react-rotuer and InstantSearch, each exerting their own influence over the UI, and the changes to the render tree are not in sync.

Now’s a good time to talk about the related minor issue with this approach. While I was technically querying two different indices, I was not doing it at the same time, nor was the InstantSearch component aware, per se, that I was switching the index. Interestingly, this seemed to cause multiple (3) XHRs in quick succession for the second tab viewed. Again, React’s re-rendering of the results component didn’t have any adverse affects; it’s most about latency and over-the-wire data size.

I then switched the results rendering to be encapsulated by the Index component with a dynamic indexName prop, so it looks something like this:

<Index indexName={target_index}>
  <ConnectedInstantSearchResults
    field_mappings={current_tab.field_mappings}
  />
</Index>

and changed the InstantSearch component to look like this:

<InstantSearch
  // ...
  indexName="Post"
  // ...
>

At this time, I also discovered createConnector and realized that’s my ticket to figuring out if the results have loaded yet, thereby allowing me to hide/remove the current results while we’re awaiting the results to return for the newly-selected tab. I duplicated the connectHits connector and created my own connectHitsLoading connector that just adds a loading prop, like so:

const connectHitsLoading = createConnector({
  // ...
  getProvidedProps(props, searchState, searchResults) {
    return {
      hits: (searchResults.results && searchResults.results.hits) ? searchResults.results.hits : [],
      loading: searchResults.searching
    };
  },
  // ...
});

So this was interesting. With my newfound loading property available, I am able to conditionally render the search results. BUT… the shape of the searchResults.results object changes based on what “tab” state the page starts in and goes to. Lemme 'splain…

  • If you load the page with the "All"tab active, the results object looks like this:
{
  // ...
  hits: Array(20).
  hitsPerPage: 20,
  index: "Post"
  // ...
}
  • if you load the page with the “Bar” tab active, it looks like this:
{
  Post: {
    // ...
    hits: Array(20).
    hitsPerPage: 20,
    index: "Post"
    // ...
  },
  Bar: {
    // ...
    hits: Array(20).
    hitsPerPage: 20,
    index: "Bar"
    // ...
  }
}
  • Loading the page with the “All” tab and switching to the “Bar” tab yields odd behavior. The searchResults.searching value flip-flops (falsetruefalsetrue) without changing the results data. Then the XHR for searching the “Bar” index returns, searchResults.searching is set to false, and results changes to:
{
  // ...
  hits: Array(20).
  hitsPerPage: 20,
  index: "Bar"
  // ...
}

With all of the above, I now have to

  1. have connectHitsLoading be aware of the current_tab and look at the searchResults.results.index property to ensure it’s logically aligned when searchResults.searching flip-flops back and forth (the “Bar” tab results are rendering with the stale “Post” data which lacks the bar_title prop), and

  2. also account for a different data shape of searchResults.results based on what order the UI state-changes execute. (!!!)

So I guess my two main questions are:

  • Am I implementing multiple indices correctly? Remember, I just want to display results from one index at a time.
    • Related, given my use case, it feels awkward to me to specify a single indexName on InstantSearch.
  • Are there other approaches to figuring out if results are still loading?
    • Related, having loading and errors as props in the default connectHits connector seems like a no-brainer.

Hope that all makes sense, and thanks for reading this far!

Hi @wrksprfct,

The typical use case for the <Index/> feature is when you want to have one searchbox used to search into multiple indices, meaning displaying hits from different indices. It seems that in your case, each tab is a different route (so everything gets unmounted when you change the route), then it would be better to use several <InstantSearch/> instance.

If you need to share some filters/query between those instances you can rely on onSearchStateChange to get the searchState, and pass this searchState to another instance. See this guide for more information: https://community.algolia.com/react-instantsearch/guide/Routing.html

Then I’m not sure to see what would be the issue using multiple <InstantSearch/>. Maybe you can provide a small repo reproducing your issue, it will help me visualise it?

Also, about the loading and errors being only accessible from the createConnector api => It’s currently on our backlog to work on that and propose an easier and declarative api for it.

@marielaure.thuret: Thanks for the reply to my lengthy post!

So I guess I was right in my initial feeling that the Index component didn’t apply to my use case—I’m not simultaneously displaying multiple indices. I’ll pull that out since that is the thing that’s messing with the shape of the results object.

The entire search page is under the same route:

<Router history={browserHistory}>
  <Route
    path="/search"
    component={withRouter(InstantSearchController)}
    onChange={handleSearchRouteChange}
  />
</Router>

…so the InstantSearchController and therefore InstantSearch don’t get unmounted during search UI changes (tabs, searchState, etc.) BTW, that handleSearchRouteChange just updates the document.title and sends a Google Analytics pageview event.

Your suggestion of separate InstantSearch instances—one per tab to keep things sane, I guess?—feels like a bit heavy-handed since the entire search UI is inside InstantSearch. (Though maybe I’m misunderstanding the approach (I’m kinda new to React).

The most straightforward approach seems to be my initial one: dynamically update the indexName prop of InstantSearch for any tab that requires searching a separate index.

Thanks again. I’ll try to follow up after I get a chance to revisit this, hopefully with an isolated test case to demonstrate the behavior. And glad to hear the loading and errors are on the list of things to address!

Oh, I thought it was different routes. Sorry about that.
So indeed you’re approach seems ok. Would that mean that those tab all look the same and all use the same refinements?

No matter what, having a repo illustrating what’s wrong for you will definitely help. It would be easier for me to work on actual code :slight_smile:

@marielaure.thuret I’m back with some code! I had some time to revisit this and create an isolated-ish test case on CodePen: https://codepen.io/wrksprfct/full/GENVeq/

Now, a few caveats:

  • It has a bit more code than I’d prefer, but that’s what’s required to demonstrate the issue.
  • The results rendering is barebones (1 field, no pagination, etc.)
  • I had to jump through a bunch of hoops to get all external libs on CodePen.
  • The URL bar does not update due to the page being in an <iframe>. Everything else works as designed, but you just won’t see the URL bar update.
  • The pen only works in “full page” mode because react-router is matching on that path. Also, the Algolia API key I’m using is referral-limited to that URL.

So the “Recipes” tab here is the one that uses the separate index (FauxRecipes, what I referred to in my post as Bar). To see things work, search for “negroni” or “eggs”. Notice that when you switch tabs from All to Recipes, the items that are recipes have their “Recipe:” prefix removed. (That is the due to using the recipe-specific search_recipes_title field.) In order to simulate the issues we were experiencing, I’ve adjusted the display to include the string length of the recipe title. So when you switch to the Recipes tab, you get a console error (Cannot read property 'length' of undefined) on the initial component render because that property is indeed temporarily undefined. (Deeper-nested components expect the property to be ever-present and operate on it, truncating it to fit in our card-like layout.)

One update: In the time since I originally posted this question, we changed our multiple index strategy slightly for other reasons. The FauxRecipes index is now a replica index of Post, and Post contains the additional recipe-specific fields. In other words, the search_recipe_title attribute (bar_title in my example) is present in both indexes.

(The reason, btw, that it’s named FauxRecipes, is because we’re changing our internal data model for recipes in the near future. Right now, a Recipe is identical to a Post with unstructured recipe data embedded in the body text. But I digress…)

Even with the attribute present in both indexes, we still have the issue where the newly-clicked tab is “activated” before the Algolia results are loaded and rendered. This still means the results component is re-rendered and our new tab config could be set to read a different property from the results, which is undefined in some states. (Need to track down exactly why… I forget.)

The good news is I was actually able to mitigate this whole in a rather straightforward way by only passing hits if it was not “searching”:

getProvidedProps(props, searchState, searchResults) {
  return {
    hits: (!searchResults.searching && searchResults.results && searchResults.results.hits) ? searchResults.results.hits : [],
    loading: searchResults.searching
  };

The effect of this, of course, is a brief hiding of the current results while we switch tabs (via the LoadingMessage custom component), which I feel is actually the UI appropriate behavior. I now need to make custom connectors for the Pagination and Stats components so that they are also hidden while the results are loading.

Also, I see that you just released react-instantsearch v4.0.4 which looks like it includes a fix that might address the odd mono-/multi-index thing I saw, right?

Small questions here: why are you creating a custom connectors for every component to deal with the loading issue?

Why not have something like:

const content = createConnector({
    displayName: 'ContentApp',
    getProvidedProps(props, searchState, searchResults) {
      return {loading: searchResults.searching};
    },
})(({loading}) => {
    const content = loading
      ? <div>We are loading</div>
      :  <div>
             <Hits />
             <Pagination />
         </div>;
    return <div>{content}</div>;
 });

const App = <InstantSearch ...>
               <Content />
            </InstantSearch>

Also, the fix I released is concerning the usage of the Multi Index API. If you were to switch between tabs (and conditionally choose to render some component instead of others) and one tabs was using multiples indices, and one other tab was using only one indices, they were a problem of results mapping. They might have still other issues with this use case pending though. We’re working on it.

@marielaure.thuret I guess I’m still wrapping around how exactly to use the custom connectors. It didn’t occur to me to use a connector to wrap something other than a Hits/Pagination/etc. component, but now I clearly see that’s a reasonable thing to do. Thx!

(Also, I somehow missed the Conditional Display page the docs, which lays out each use case I care about! Ugh, sorry.)

Also, our search/results UI is currently spread across a handful of upper-level DOM elements. I’ll see if I can consolidate the UI to make use of a single connector like the above pattern.

Anyway…back to the question above:

Is this hiding-results-while-loading approach appropriate? Specifically, only setting hits if we’re not loading. It feels like it’s the only way to avoid the “outer search UI” (our tabs, not driven or managed by react-instantsearch) getting out-of-sync with the react-instantsearch-driven UI. This is our use case because we wanted to abstract the faceting capabilities to “tabs” (each preset with specific facet filters) instead of presenting the user with your default search UI that contains each facet listed out with the available values. I went with the “unmanaged” tab approach because it seemed there was no way to achieve something like that with the available components/patterns.

Thanks again for your continued help!

@marielaure.thuret So I’ve spent some time playing around with the loading indicator pattern and I’ve encountered some odd behavior I wanted to bring to your attention.

Using the example connector provided above and in your docs:

const connectLoading = createConnector({
  displayName: 'ConditionalError',
  getProvidedProps(props, searchState, searchResults) {
    return {
      loading: searchResults.searching
    };
  }
});

I then create a component that renders a message when loading, and its children when complete:

const LoadingMessage = connectLoading(({ loading, children }) =>
  <div>
    {loading ? <p>Loading...</p> : children }
  </div>
);

Now we make use of it in a minimal UI where we just display the stats:

<InstantSearch ...>
  <SearchBox />

  <div className="SearchResults">
    <LoadingMessage>
      <p>FINISHED!</p>
      <Stats />
    </LoadingMessage>
  </div>

</InstantSearch>

You can see this example here in a codepen: https://codepen.io/wrksprfct/full/jwmmox/. This works as expected. Load the page and you see “Loading…” and then “FINISHED! X results found in Yms”. Ensure you have web console open and you’ll see the XHRs fly by and searchResults.searching toggle between true and false when things change state. Type “avocado” into the box: the UI updates and the console logs as you’d expect.

Now here’s the fun part. Let’s replace the <Stats /> component with the <Hits /> component, so it looks like this:

<InstantSearch ...>
  <SearchBox />

  <div className="SearchResults">
    <LoadingMessage>
      <p>FINISHED!</p>
      <Hits />
    </LoadingMessage>
  </div>
</InstantSearch>

You can see this example in another codepen: https://codepen.io/wrksprfct/full/yXbXNQ. !!!BEWARE!!! before loading that page, be prepared to kill it via Chrome’s Task Manager or your OS’s equivalent because it gets into an infinite loop and consumes CPU & gobs of memory quickly. Again, ensure you have the web console open.

So, upon loading that page, you’ll see two XHRs instead of one (identical, AFAICT), and then searchResults.searching flip between false and true ad infinitum.

If, in this problematic example, you hardcode the loading prop to false:

getProvidedProps(props, searchState, searchResults) {
  return {
    loading: false
  };
}

…everything works. Well, except for seeing the loading message… which is the whole point of this exercise!

Again, the only difference is the switch from <Stats /> to <Hits />. I looked at their sources, but could not find anything interesting.

Interestingly, the same problematic behavior occurs with <Pagination />. See https://codepen.io/wrksprfct/full/zzzOWm/. I did not test any others yet.

Any ideas as to what’s going on? Am I missing some basic concept?

Thanks again!

That’s clearly a bug :slight_smile:

Can you open an issue on the github repo that way you’ll be able to follow his resolution?

@marielaure.thuret As requested…

Also, note that I added more info at bottom: The behavior occurs only when rendering children. Using static content for the ‘loading finished’ state works just fine. :-/