Content-Type not supported: text/html in Crawler


I’ve taken over an old company Docusaurus v1 project and have been looking to upgrade it to a more contemporary version. As a part of this, I noticed that the last time the Crawler had re-indexed the site was in February of last year - I’ve been working to get this working so we can include all new content since then in the search. However, I’m running into issues with trying to configure the crawler - every URL seems to return a "Content-Type not supported: “text/html;” regardless of settings inside of the Editor.

I’ve tried with our old config as well as beginning with a brand new one but that doesn’t seem to change anything. I’m also not seeing anything obvious on the site which would prevent a crawler from proceeding (robots.txt, etc).

This is what I get when trying to run out of the Editor:

Let me know if there’s other information that would be helpful.