I am trying to store blog posts in Algolia, but they are around 30kb each. As such, I keep getting error messages saying that my documents are too large. I don’t really care if the content is searchable, but I need the content to be returned with the search results. Is there any way I can increase the size limit of the documents?
Yes indeed, record size is limited to 10kb in current Algolia plans (Community, Starter and Pro) unless you are on the Enterprise plan when we can set a custom record size.
The reason for this limit is that there is a significant performance impact going along with increasing the record size, so we recommend to all of our users to keep there record size below this limit, and it is not possible to manually update it on these plans.
Indexing blog post is quite a common issue though, and we do have solutions:
- Some of our clients will split their large objects in multiple objects so that the size of one object is manageable: https://www.algolia.com/doc/guides/managing-results/refine-results/grouping/#handling-large-records
- You can do so using the distinct attribute: https://www.algolia.com/doc/guides/ranking/distinct/#distinct-to-index-large-records
- Make sure to read our documentation on indexing long documents: https://www.algolia.com/doc/guides/sending-and-managing-data/prepare-your-data/how-to/indexing-long-documents/
These two blogposts also explain in depth the benefits of handling large records this way:
I hope you can find some directions here!