Nightly index update with synonyms

Hello,

I’ve implemented a nightly update of the index and this has been working well. However, some synonyms have recently been added via the dashboard; so I amended the nightly update to call get_synonym() and batch these into the temp index (as per instructions here).

This worked too, but … I don’t necessarily know the objectID of each synonym to pass to get_synonym(). As synonyms can be created in the dashboard by non-developers, this means that I need to be told a synonym has been added in order to include it in the nightly update.

As an alternative I’m using the results of search_synonyms() (no object IDs needed), and I cleanup the results returned so that batch_synonyms() doesn’t throw an exception; but this feels clunky too.

Is there another way of doing this? For example, it would be nice to have a get_synonyms() method that returns all the synonyms for an index (no object IDs needed), and then I can pass the results straight to batch_synonyms() to set on the temp index.

Is there a better way of doing this - maybe I’m missing something here?

Hello and thanks for the question and feedback,

You are doing things exactly as they should be done. I agree with you that it seems a bit clunky having to manually move synonyms as well as settings. Your feedback is very valuable to us to see how we can improve that part, actually.

search_synonyms is the right way to get all the synonyms. We don’t have a get_synonyms method as there is no real limit to the number of synonyms you can implement and downloading a list of 10k synonyms in one call might not be a pleasant experience. Instead we thought that with search_synonyms and its pagination mechanism we gave developers the best way to download them at their own pace. But we’d like to hear your take on that.

I’m curious about the cleanup you had to do before sending data to batch_synonyms? Could you tell me more about that, this is something we might want to improve (either directly in the API or the documentation).

I think our documentation should contain a part about how to properly move synonyms from on index to another. With all the information you can provide (maybe even sharing your code?) I could update the doc accordingly.

Hi @pixelastic and thanks for your quick response - reassuring to know that I’m doing the right thing using search_synonyms.

‘cleanup’ is probably too strong a word, what I did was this

   result = list()
   wanted = ["objectID", "type", "synonyms", "input"]
   syns = index.search_synonyms("")["hits"]
   for syn in syns:
        result.append({key: value for (key, value) in syn.items() if key in wanted})
   temp_index.batch_synonyms(result, False, True)

and what I thought would be nicer to do is something like this

temp_index.batch_synonyms(index.search_synonyms("")["hits"], False, True)

But if I implement it this way I get:

algoliasearch.helpers.AlgoliaException: Invalid attributes, allowed attributes name are objectID|type|input|synonyms|word|corrections|placeholder|replacements

on the call to batch_synonyms, presumably because it doesn’t like some of the attributes returned from search_synonyms.

I agree, to have the documentation updated to reflect the task of moving all synonyms for nightly update would be helpful - thanks again.

You have a point. It would be better if synonyms could be passed directly from search to batch. I’ll see what we can do about that, and I’ve filed internally a ticket to update the documentation to give clearer examples.

Thanks for the feedback, really appreciate that.