Search KCD page

[kentcdodds]

I'd like a page where you can search through all my content everywhere. This has been discussed quite a bit: #107

The biggest challenge is creating a way to search through and index all that content. Not sure of the best way to go about it...

[dane-stevens]

redisearch would handle this nicely. Here's a blog post describing the basics behind it: https://redis.com/blog/building-real-time-full-text-site-search-with-redisearch/

Let me know if this is something I should look into further.

I do have question about the current redis implementation with primary and replica clients. Could you briefly explain the setup?

[kentcdodds]

Basically I have two connections, one to the primary region redis for writes and one to the replica region for reads. Fly handles automatic replication for me. That's it.

[kentcdodds]

The biggest challenge isn't text search. The biggest challenge is all the APIs (or non-APIs) you'd have to use across all my content distribution channels. Mostly YouTube, Egghead, Frontend Masters, podcasts I've been a guest on, and my own site.

[dane-stevens]

Interesting. Yeah, sounds like you would need a combination of cron-jobs, webhooks (if the APIs support them when new content is added) and some manual URL entry for scraping to get the data into Redis. Once the data is all ingested, the search part is easy. It is probably something that could be started with one data type and more added later. Youtube push notifications: https://developers.google.com/youtube/v3/guides/push_notifications Not sure if Egghead or frontend masters have an API or not.