The Musicologist: Search as a Conversation 🗣📱

Hello World!

I’d like to share my Musicologist, a prototype I built to explore the intersection of conversational interfaces and search UIs :slightly_smiling_face:


What does it do?

The Musicologist can answer your questions about an artist, a genre, or a theme to find in the song’s title or album.
You can then ask it to play those songs, using your installed music apps.

How does it work?

This project is built with three components:

  • A front-end written in Kotlin using InstantSearch Android :iphone:
  • A conversational agent developed on :robot:
  • A Node.js back-end hosted on Heroku :factory:

You can find it released open-source on GitHub! Here are direct links to the front-end, the agent and the back-end :wink:

Join the conversation

Have a look at the code, it’s pretty easy to leverage Algolia to build a conversational interface that can understand your users’ questions and provide them with relevant answers!

Feel free to share here your feedback and ideas, on this project or on other applications you might have for conversational voice search! We’ll keep exploring and writing about this topic, and we are eager to see what great search experiences you can create with this approach :slightly_smiling_face:


Wow, this is mind blowing!

Could you share a bit more about the challenges behind conversational search in the context of your application?


That’s huge ! Any chance we get the apk to try it ?

Agree with Ray - this is super cool and I’d <3 to see some more exposition around this tool. What inspired you, what difficulties you encountered, etc.

Hi folks, thanks for the warm reaction and the kind words :blush:

@rayrutjes, there are quite a few areas in which it was challenging:

  • getting the agent to understand the user correctly
  • connecting the agent to Algolia
  • handling different forms of speech responses depending on the Algolia results
  • recognizing entities in a typo-tolerant way

I’m currently working on a blog post to describe these challenges in more details and to help the next ones that will explore this path :wink:

@pierre.aurele.martin: Thanks for your interest! I am considering a public release of the application on the Play Store, but I want to improve the dataset first (today we only have 170 songs mostly from the 70s, not enough to be fun to play with…
In the meantime, feel free to checkout the code and run cd android && ./gradlew installDebug, which will build and install the app on your device :slight_smile:

@phrawzty: I’m looking forward to the public release to expose this tool further, and I’ll make sure to discuss the motivation/inspiration along with the challenges of building it when I’ll write the blog post!


@phrawzty & @rayrutjes, you can now read more about the journey behind building a Musicologist on our blog:

I also presented the project in front of the Paris NLP Meetup, you can find the video on YouTube:


What an awesome idea!!! In my own personal projects I do some video editing to music, and sometimes try to relate the song’s title/lyrics/themes to the actual video content - would love to try this out once there is more music on it, and use it to find music that is relevant to what I’m editing!

1 Like