Indexing: getting more than 1000 records using getObjects

Context: Index contains a product catalog, with an attribute inventory which contains branch-wise stock information (maxPrice, batches available in each branch, etc)

What we’re trying to accomplish: We get periodic notifications of inventory changes of multiple items, which we need to update into the index.

How we tried to do it: Used getObjects() method and passed the ObjectIDs to get the inventory attribute of each item/object. Can’t use browseObjects() since you can’t pass ObjectIDs.

Problem we faced: Getting an error “cannot fetch more than 1000 records”. No option to overcome the max-1000-records limit even after setting the paginationLimitedTo value to 200,000 and increasing hitsPerPage value.

Solution we found: To chunk the process in batches of 1000 records.

This seems to be a rather common indexing use-case, I’m thinking maybe there is some other way to accomplish it? Maybe something I missed in the docs?

Any inputs in the right direction appreciated. Thanks!