Docsearch fails to crawl site. nbHits 0

Hi, I’ve been using Docsearch for our site for a while, and re-index it using the crawler running in a docker container (as documented on your site). Our site is: https://docs.thoughtspot.com.

Today, I tried to re-index, and for some reason, the crawler does not run properly. Here’s what I get:

> Ignored: from start url https://docs.thoughtspot.com/software/latest/

Crawling issue: nbHits 0 for search-manager

search-manager is the name of my index.

And here’s the configuration file I’m using:

{
  "index_name": "search-manager",
  "start_urls": [
    {
      "url": "https://docs.thoughtspot.com/(?P<component>.*?)/(?P<version>.*?)/",
      "variables": {
        "component": [
          "software"
        ],
        "version": [
          "latest"
        ]
      }
    }
  ],
  "stop_urls": [
    "^((?!\\.html).)*$",
    "geomap-"
  ],
  "selectors": {
    "default": {
      "lvl0": {
        "global": true,
        "selector": ".context .title, .context .version"
      },
      "lvl1": ".doc > h1.page",
      "lvl2": ".doc .sect1 > h2:first-child, .doc > h1.sect0",
      "lvl3": ".doc .sect2 > h3:first-child",
      "lvl4": ".doc .sect3 > h4:first-child",
      "text": ".doc dt, .doc td.context, .doc .admonitionblock"
    }
  },
  "selectors_exclude": [
    "tableblock"
  ],
  "custom_settings": {
    "attributesForFaceting": [
      "component",
      "version"
    ],
    "attributesToRetrieve": [
      "anchor",
      "content",
      "hierarchy",
      "url",
      "component",
      "version"
    ],
    "customRanking": [
      "desc(weight.page_rank)",
      "desc(version)",
      "desc(weight.level)",
      "asc(weight.position)"
    ]
  },
  "nb_hits": 10000
}

I’ve confirmed that I’m using the correct Application ID and API_KEY in my .env file.

Any ideas on what I’m doing wrong?