recordExtractor function doesn't get called at all

Hello,
I have noticed that the function assigned to the recordExtractor field in my config doesn’t get called at all. For example in my config I have the following:

actions: [
    {
      indexName: "elumity",
      pathsToMatch: ["https://elumity.com/resources/documentation/**"],
      recordExtractor: ({ helpers }) => {
        console.log('Does not get called at all');
        return helpers.docsearch({
          recordProps: {
            lvl1: ["header h1", "article h1", "main h1", "h1", "head > title"],
            content: ["article p, article li", "main p, main li", "p, li"],
            lvl0: {
              selectors: "",
              defaultValue: "Documentation",
            },
            lvl2: ["article h2", "main h2", "h2"],
            lvl3: ["article h3", "main h3", "h3"],
            lvl4: ["article h4", "main h4", "h4"],
            lvl5: ["article h5", "main h5", "h5"],
            lvl6: ["article h6", "main h6", "h6"],
          },
          aggregateContent: true,
          recordVersion: "v3",
        });
      },
    },
  ],

I’ve added the console.log in the function, but when I test any url, for the LOGS I get

Nothing was logged from recordExtractor

Does anybody know what am I doing wrong, or am I wrong for expecting the function to be run?

same here.
How you solved it?

Hello Hasan,
in my case we had a <meta name="robots" content="noindex"> tag on our pages which prevents the Algolia crawler to index our pages.
One solution was to remove this meta tag, but you can also specify in your config ignoreNoIndex: true and the crawler will index the page. The second approach worked for us.
Hope it helps, but for me it was this case. I didn’t know we had this tag, because it was added by another colleague.

1 Like