Local crawler issue: nbHits 0

Hello,
I’m trying to create an index using te local crawler.
It seems to crawl all links in the website, but reports 0 records on each crawled page.

The index is being created, but has 0 records.

My config. json is as follows:

{
    "index_name": "NETASPECT",
    "start_urls": [
      "http://www.netaspect.nl"
    ],
    "stop_urls": [],
    "selectors": {
      "lvl0": "FIXME h1",
      "lvl1": "FIXME h2",
      "lvl2": "FIXME h3",
      "lvl3": "FIXME h4",
      "lvl4": "FIXME h5",
      "lvl5": "FIXME h6",
      "text": "FIXME p, FIXME li"
    }
  }

I would be very happy if someone can help.

Regards, Jacques

Hi @jvantende, welcome to the forum!

I understand you’re using our Docsearch scraper to try to crawl your site.

The selectors in your config are all prefixed with “FIXME”, which means we are looking for <h1> html elements within a “FIXME” html element, which presumable does not exist.

More likely, the selectors should look something like this:

“lvl0”: “.docs h1”,
“lvl1”: “.docs h2”,
“lvl2”: “.docs h3”,
“lvl3”: “.docs h4”,
“lvl4”: “.docs h5”,
“text”: “.docs p, .docs li”

That is, looking for <h1> and other html elements within a “docs” class HTML element.

Hope this helps!

Hi Maria,

Thanks for the feedback. It works fine now.

Jacques