we’re indexing several different categories of quite large text chunks in individual indexes. That’s all great and can lead to indexes containing several thousand records.
We also have some smaller chunks of data, which are in general of the same type (text) but need to be filtered by the user who created the relevant text. So user A should not be able to see content that was created by user B.
We are wondering if it makes sense to create one index per user, which can be anything between empty or very large. With a growing number of subscribed users however we might end up with hundreds or even thousands of users / indexes.
The alternative would be to bunch all this together into one index, add a user / group id to the records and use filtering when querying that index to retrieve only records that the relevant user should see.
What’s the most idiomatic approach here and what would be best performance wise?