Too many indexing operations

We have Magento 1 shop with about 700 products. It also has an integration to another system from where some of the product data is updated many times in a day. We just installed the Algolia and it has now been running for half a day. The number of indexing operations is currently about 58 000. I’m presuming that every time the product data is updated from another system to Magento it also results product data to be completely reindexed again to Algolia.

What would be a good way to reduce the operations? Should we change to manual reindexing in Algolia’s extension settings? Is there a way to make it happen only once a day and not every time our other integration updates product data to Magento?

I guess I found an answer. Setting up a cron job to run this for example once a day should do the trick after setting the Algolia indexes to be run manually, disabling the queue, and enabling automatic indexing in Algolia settings:
php -f /absolute/path/to/magento/shell/indexer.php – -reindex algolia_search_indexer

Nope. Not the right settings. After these I got 20 000 operations in an hour. Any ideas?

What does the setting enable indexing actually do? Does it automatically index via magento’s normal cron? I simply need once a day indexing to minimize operations.

I hope it does not stack the updates made for products…

Hello @Kuono .

If you disable queue indexing, each product save will trigger an update operation sent to Algolia as the indexing is “plugged” to the product save before event. I don’t know how your catalog management system works exactly, but if it is generating a lot of product updates, it’s the cause of the issue, for sure.

In your case, I think it’s much better to activate the indexing queue because the indexing jobs are merged before being sent to the Algolia’s servers.

May I ask what could justify that amount of product updates performed by your system for such a small catalog ?

Thank you for you reply.

There is an integration plugin that integrates Magento into POS system that our client use as main system in their physical stores. They use the POS system to update product data, categories and stock qty’s. Stock qty’s is imported into Magento every 30 minutes so I guess that is the cause for product saves that trigger the update to Algolia.

It is essential to update stock that often because the stock needs to be up to date in both systems.

1 Like

I’d be interested to see if there’s a solution here. I will probably end up in a similar situation eventually. Either that, or become Algolia’s best customer…

Hello @Boogs and @Kuono,

the solution would be to modify the import script to turn off Algolia indexing and reindex only products changed by script. Or don’t propagate those changes to Algolia at all, but then you might end up with inconsistent data and that’s probably something you really don’t want to do.