Save and restore index settings in go(lang)

Hi there,

I’m using Go(lang) for my projects.

i want to write a small index managing tool that allows me to save and restore index settings and as a side effect allows me to create a set of indices for a new algolia app.
I can read the index settings, using index.Getsettings() just fine and have saved them as a json file. but when i try to apply them to an index I get the following error:
panic: {"message":"Invalid object attributes: AdvancedSyntax near line:1 column:17","status":400}

After reading and unmarshalling the json file content, I converted it to a Map, but when it’s handed over to index.SetSettings(optionsMap), it causes the above error.

here is the code in question:

// open json file with settings
raw, err := ioutil.ReadFile(in + f)

// convert into algolia Settings struct
var indexOptions algoliasearch.Settings
err1 := json.Unmarshal(raw, &indexOptions)
check(err1)

//convert Settings struct into a Map
optionsMap := structs.Map(indexOptions)

// initialize index
index := client.InitIndex(indexName)

// try to update index settings
_, err2 := index.SetSettings(optionsMap)
check(err2) 

Any pointers much appreciated!

Hello @searchtech,

This error seems to come from the engine itself. I’m wondering if the JSON that you read and try to send does contain the parameters with a first capitalized letter? If so, the engine would indeed reject them.

Here, it seems to have an issue with AdvancedSyntax whereas the correct setting name would be advancedSyntax. Could you check the case of the setting names you are trying to send?

Also, if the previous statement is correct, it also means that the way you’re serializing the settings in the first place is probably wrong. It’s weird because I’ve checked the code of the client and the advancedSyntax JSON tag attribute is using the correct case. I’ll be happy to assist you on that if you need help.

Salut Anthony,

thanks for your help, much appreciated!

Indeed, the attribute names in the map are upper cased.
The json looks fine:

{
   "allowCompressionOfIntegerArray": false,
   "attributeForDistinct": "article_nmbr",
   "attributesForFaceting": [
  "searchable(article_nmbr)"
  ],
  "attributesToIndex": [
  "article_title",
  "article_text",
  "article_nmbr",
  "post_id"
  ], ...

when I pull the json back in using:

// convert into algolia Settings struct
var indexOptions algoliasearch.Settings
err1 := json.Unmarshal(raw, &indexOptions)

It’s populating the algoliaserch.Settings struct.
In the conversion to the Map then:

//convert Settings struct into a Map
optionsMap := structs.Map(indexOptions)

The struct variable names are being used by the looks of it and they are all exportable.
I’m using
github.com/fatih/structs
for the struct -> map conversion.
Is there a better approach?

Thanks,
Olaf

Yes, it precisely comes from the algoliasearch.Settings -> structs.Map conversion. As written on their documentation, the fields are converted with the first letter being uppercased.

If you do not use the strucs.Map elsewhere, you may want to do the following instead (forgetting the error handling for ease of reading):

var settings algoliasearch.Map
json.Unmarshal(raw, &settings)
index.SetSettings(settings)

This way, you would directly read the settings as-is from the JSON file, that seems correctly formatted already.

Let me know if you need anything else or if it doesn’t suit your needs.

Thanks, that got me a bit further. Now I get errors for type mismatches like:
panic: 'minWordSizefor2Typos' should be of type 'int'
and
panic: 'ranking' should be of type '[]string'

Should I try to save the settings to files in a different format instead of json?

Merci beaucoup,
olaf

It probably has to do with the JSON deserialization. For instance, the integer field may have been decompressed as float64 instead of int. At this point, you may need to check what every field type is in the map.

Otherwise, I just recall we have a toMap() function on the Settings object but it’s missing some fields as of 2.23.0. We’re gonna publish a new version soon with up-to-date implemantation. This should be prefered for what you are doing.

Thanks, that helps a lot already!
Settings.ToMap() indeed worked. GetSettings get’s all settings but I don’t need really all of them. TheSearchable Attributes, Attributes for faceting, Attributes to retrieve, snippet and highlight are what I care most about and they are all there.

Do you have a very rough estimate for when a new version that includes an updated ToMap() function might be available? Don’t want a guarantee, just an estimate really.

cheers,
olaf

Thanks, that helps a lot already!
Settings.ToMap() indeed worked.

Ok perfect! Happy to have been helpful here. Don’t hesitate if something is still missing.

Do you have a very rough estimate for when a new version that includes an updated ToMap() function might be available? Don’t want a guarantee, just an estimate really.

Following the discussion we had, I just submitted the fix on Github a few minutes ago. If it gets reviewed quickly and on the basis that Travis will run the tests shortly, you could expect a 2.23.1 release today or on Monday worst-case scenario.

edit: We’ve just released 2.23.1 with a fixed and up-to-date version of Settings.ToMap.

1 Like

Wow - thanks!!! I thought it’ll be great to have it after the summer perhaps…