Export Index contents

I am trying to get an extract of all the records in one of our indices. I see some instructions here:
https://www.algolia.com/doc/tutorials/indexing/exporting/how-to-export-data-of-an-index-to-a-file/?language=ruby#exporting-the-index

However, I am not sure where I can run this code from. Can someone direct me where in the dashboard I can run a query to get the full data dump? Is it under the “Custom” tab under the Browse wizard? When i tried running the export code, i get some errors which i am sure is just syntax that i am not familiar with.

Thanks!

Hi @adri,

To export your index data you’ll have to run it outside the Dashboard using your preferred programming language.

The browse method is indeed the way to get all your data via the API. There is no direct way to request a data dump from the Dashboard.

What code did you try and run and what errors did you receive?

Please feel free to include them here!

Best regards,

I was trying to run the example snippet from the link above:

require ‘json’
require ‘algoliasearch’

Algolia.init(application_id: ‘AJ0P3S7DWQ’,
api_key: ‘ce1181300d403d21311d5bca9ef1e6fb’)

index = Algolia::Index.new(‘best_buy’)

hits =

index.browse do |hit|
hits.push(hit)
end

File.write(‘your_filename’, hits.to_json)

Obviously using our apikey, index name etc…but i have a feeling i am not running it from the right window?

Hi @adri,

You selected the Ruby example so lets debug starting from there with a step-by-step.

Did you:

  • Copy-and-Paste the exact Ruby example into any text editor (Atom, Sublime, etc.)
  • Save the file with a “your_filename.rb” extension
  • In terminal, navigate to the parent folder of that file
  • In terminal, run gem install algoliasearch to install the gem
  • In terminal, run ruby your_filename.rb

How does that work out?

Hi @ajay.david

I installed the gem. I just want to make sure that the following line in the code is not going to create a new index with the name of our current index, correct?

index = Algolia::Index.new(‘ourindexname’)

If not, what is that line exactly doing? The “new” part is throwing me off thats all.
Also once i run it, am i able to export the results into a CSV or some other format?

Thanks

Hi @adri,

Yes, that’s correct!

The line you noted initializes an instance of the Algolia::Index class and targets an existing index with the name ourindexname or, as noted in the docs, will create a new index with that name only if one does not already exist.

There is no way to export your index data into a downloadable format from the Dashboard.

However, as noted above , you can use the browse method to retrieve all your data. It will be returned to you in JSON format and you can handle it as you wish, including exporting into a CSV file, but the best approach for that will be determined by you.

Let us know how it goes!

Hi @ajay.david

I am getting the following error. Must be related to indentation?

syntax error, unexpected tINTEGER, expecting end-of-input
api_key: ‘xxxxxxxxxxxxxxxxxxxxxx’)

Here is how my file body looks:

require 'json’
require ‘algoliasearch’

Algolia.init(application_id: ‘xxxxxxxxxx’,
api_key: ‘xxxxxxxxxxxxxxxxxxxxx’)

index = Algolia::Index.new(‘indexname’)

hits =

index.browse do |hit|
hits.push(hit)
end

File.write(‘algolia_records’, hits.to_json)

Hi @ajay.david

Any update on helping me out here? Am i missing an ‘end’ somewhere?

Hi @adri,

The code looks fine except that I can’t verify any syntax issues in your AppID or API Key. It’s fine to share that publicly because on your front-end website, that will generally be available anyway (which is why we differentiate access with the Search API Key and the Admin API Key).

Can you try this out - it’s a copy-and-paste of the sample code using our Algolia sample index. Works with no issues on our end:

require 'json'
require 'algoliasearch'

Algolia.init(application_id: 'AJ0P3S7DWQ',
         api_key:        'ce1181300d403d21311d5bca9ef1e6fb')

index = Algolia::Index.new('best_buy')

hits = []

index.browse do |hit|
    hits.push(hit)
end

File.write('your_filename', hits.to_json)

Let us know how it goes!

thanks @ajay.david

used your example and it worked but now im running into the following error:

`perform_request’: Cannot GET to https://XXXXXXXX-dsn.algolia.net/1/indexes/my_search/browse?cursor=: {“message”:“Method not allowed with this API key”,“status”:403} (Algolia::AlgoliaProtocolError)
(403)

Should I be using our Admin API Key or Write API Key to run this command? I was using the Search API Key.

Hey @adri, you got it - give it a shot with your Admin API Key.

Hi @ajay.david thanks for your help on this. I was able to export the index contents. However it is a massive file. Do you know if i can manipulate my code to just extract the “object_ID” fields? it will make it much easier for me. It is the only field i really need.

Hi @adri,

You can use the parameter attributesToRetrieve, as described in our docs, to limit the attributes returned by your browse call to object_ID:

Include it as one of the browseParameters (Note: If you use it in the setSettings, it will override your index settings)

Hi @ajay.david

Can you give me a quick thumbs up before i run this?

require ‘json’
require ‘algoliasearch’

Algolia.init(application_id: ‘xxxxxxxxxxx’,
api_key: ‘xxxxxxxxxxxxxxxxxxxxxx’)

index = Algolia::Index.new(‘levelup_search’)

hits =

index.browse do |hit|
hits.push(hit)
attributesToRetrieve: [‘merchantID’,‘objectID’]
end

File.write(‘algoliaexport’, hits.to_json)

Hey @ajay.david can i get a confirmation i have this correct before i can run it?

Thanks!
Adri

@ajay.david
I ran the following but it still returned other attributes from the index:

require ‘json’
require ‘algoliasearch’

Algolia.init(application_id: ‘xxxxxxxxxx’,
api_key: ‘xxxxxxxxxxxxxxxxxx’)

index = Algolia::Index.new(‘my_search’)

hits =

index.browse do |hit|
hits.push(hit)
end

index.browse({
attributesToRetrieve: [‘merchantID’,‘objectID’]
}
)
File.write(‘algoliaprodexport’, hits.to_json)

However, when i ran the same one for a smaller index we have it worked properly. Any idea what i might be missinng here?

Hi @adri,

Thanks for sending through the snippet. There is a little tweaking you can do.

As it stands, you are calling the browse method two separate times, but in the one that you are using to write hits to your document it is not getting the attributesToRetrieve parameter.

Take a look at this and see how you can set attributesToRetrieve a little differently per our documentation:

index.browse({ query: '', attributesToRetrieve: ['objectID'] }) do |hit|
    hits.push(hit)
end 

Hope this helps!

thanks @ajay.david that did the trick! I will need to look into doing bulk-delete or bulk-add records to an index next. Can you link me to relevant documentation?