Algolia + web crawler


I’m looking for a search solution to replace our current Google Site Search.
Algolia look great but it seems to only work by importing or synchronizing our data.
Is there a way to have data indexed by a web crawler? As we need a full text search and it will very complicated to extract a well structured data as it comes from many sources.

Thanks for your help!


1 Like

Hey Yvan,

Do you mind contacting me by email: lucas.cerdan at algolia dot com? I’m interested to review what you need together.


1 Like

Hi Lucas,

Thanks for your swift reply, I’ll send you an email right away…



I have the exact same question and found this thread via a Google Search. Why not just publicly answer the question so that others like me who find this thread don’t all have to email you for an answer?


Thanks for your message, and you’re totally right.

Algolia does provide a crawler today, as you can see in the details on our Pricing page.
That being said, it is quite different from GSS. Our web crawler is an add-on that is customised exclusively for customers on our Enterprise plan. This is why I asked Yvan to reach out, as we need to evaluate the scope of the project to provide a more definitive answer.

Should you also be interested, please contact our team, we would be happy to help!

Thanks for the info. Unfortunately, as a local government entity, I think the Enterprise plan is out of our budget.

You might want to look at Algolia’s Doc Search as a possible solution.