Magento 1 reindex keeps blanking the records

Hi,

We are setting up the extension in Magento 1 but the Indices in Algolia keep resetting to 0 and then get re imported again from magento. This is making the search unavailable for 10+ minutes at a time and is causing a lot of index operations to occur.

Thanks

Hello Christopher,

please, enable indexing queue. That technique will not erase indices.
Would that work for you?

Hi Jan,

We have enabled the index queue, this is when the problem started to occur, I have stopped the cron from running and it has not blanked the indices since.

The admin shows the following:

Algolia Search - Indexing queue information: Number of queued jobs: 1286, all queued jobs will be processed in appr. 10 hours 45 minutes, more information about how the indexing queue works you can find in the documentation: Indexing queue

Which seems like a lot of jobs to have queued?

Thanks

Hello Christopher,

yes, this can be a lot of jobs, but it depends on how many products you want to index.
How many products do you have in your catalog?

3.5k products are currently on the site, the number of jobs has increased by about 100 with no product changes and the index queue still off, is this normal as well?

Hi Jan,

Any update on this? I just re enabled the cron for it and it has blanked and re imported the products again, it has been on for 3 hours and the number of jobs has not gone down it just keeps going up!

We need to get this working so the boss will sign off on paying for one of the packages so we can get all of our products on it. If we cannot solve this soon we will need to start looking for another solution.

Thanks

Hello @christopher.lewthwai,

it looks like you have a re-indexing job in a crontab, so it reindexes you catalog regularly. That’s why jobs are increasing.
Can you send me how you run the queue runner?

Hi,
Btw, I can confirm a bug somewhere in the queuing, however so far we are unable to reproduce it. I’ve set an internal bounty (crate of beer) for anyone who gets it reproduced.

Regards
Alex

Hey Alex,

you mean, when you have indexing queue enabled, hit the “reindex” for products it blanks the index until the new temporary one is moved to production?

Hi,

no, not really. I see sometimes orphaned rows in the queue table, where jobs have died, with no exception in the logs.
Also I sometime get reports that updates to products did not show up in algolia, albeit re-index has run.
Both are unreproducable so far, though.

regards
Alex

It’s super weird Alex.

Do you know what jobs are those left in the queue? Is it completely random or are they still the same / similar jobs?

It’s not often enough so far to answer. I’ll keep an eye on the issue and report back.

1 Like

This is happening to us since we upgraded to magento 1.9.3.7 and algolia 1.13
We have less than 10k products (9200 actually) and we are on free plan. There is definitely a bug in queuing. We are getting error in api call on algolia dashboard. It says error 403 exceeding quota, change plan or delete records.

so my guess is that queuing is not working or free plan is not allocation or not calculation total records.