Howdy y'all, new member here.
You use Slack, yes? Slack has pretty good search, but it doesn't (yet) index the contents of links included in messages (except the tiny snippets that appear in a link unfurl). Link Minder is a bot I built not too long ago that helps with this. The bot sits in a channel (you have to invite it) and watches for messages with URLs in them. When it finds one, it loads the page, indexes the contents, and makes it available for searching by messaging the bot directly.
You can install to try it out yourself (I've got it running on a free Algolia account, so I don't know how well this is going to scale):
or examine the source code directly:
(you can see I used to call it something else )
Would love feedback from y'all!