Create search for entire website?

Insofar, I’ve set up Algolia to search within items indexed in my website’s database. I’m interested, however, in also creating a search for the entire website, (i.e. all its pages) something to my understanding neither Algolia nor my website’s framework (Django) has easy provisions for. One idea I had was have Algolia search through my website’s sitemap, but are there any easier approaches I’m not thinking of?

Hi @blee025432 indeed that’s not something Algolia provides (a self serve site search crawler), we do provide a website crawler to enterprise users.

You could also try to run DocSearch, our open-source crawler on your website and see how it goes.

Alternatively, you could make your own little crawler or reuse one and index your website manually since you know well the page structures.

Let us know how it goes.

I had created my own crawler that uses the C# library HTML Agility Pack to grab content off of each html page based on a sitemap.xml and form that into a JSON object to send to Algolia that has worked decently.