IndieWeb Search results are also feeds

Published on January 13, 2022 under the IndieWeb category.Toggle Memex mode

IndieWeb Search has been intentionally designed to be open. The source code for the project is open. From the early days, there was a basic API that let you retrieve content and featured snippets from search results. I often chat about the search engine and how it is designed. The project is well documented. Another way in which the search engine is open is that all search result pages come with feeds. I hope to see more search engines do this too.

I decided to write feeds to accompany search result pages so that I could subscribe to content that matches a particular query. For example, I currently subscribe to the search page for my domain name and "coffee." This lets me monitor changes in the search results. If new content makes its way onto the first page, that content will show in my feed reader, no matter when it was published. This is a whole new way to discover content. If a page has managed to make its way to page one on a term I care about, the content is probably going to be at least somewhat interesting to me (assuming the term is competitive).

Making search pages feeds also opens up the world of programmatic access to the engine. In a previous blog post, I talked about how you could parse an IndieWeb Search microformats h-feed in 15 lines of code. The applications of this are numerous. You could build a tool that monitors when an article you have written becomes prominent on the search engine. You could build a tool that aggregates search results for queries you find interesting. You could create a utility that archives the top 10 results on every search result. Again, there are so many possibilities.

At the moment, IndieWeb Search pages come with four types of feeds:

jf2 feed
JSON Feed feed
RSS feed
h-feed feed

The engine supports all four of these feed types as I understand they can each be used in different scenarios. RSS support opens up the possibility to subscribe to search pages in pretty much any feed reader. h-feed support lets the engine embrace microformats. h-feeds can easily be turned into structured JSON. JSON Feed and jf2 are less widely used but also have their own applications.

Support for subscribing to feeds is still early but I am excited by the promise of this idea. I don't get regular updates that tell me when a search page has changed because the index has not been crawling much content lately. But, over time, I'll likely get a few notifications every now and again for the terms that interest me.

I also use the h-feed markup to enable a full, native IndieWeb Search integration into my feed reader. I can search for new blogs to read in my feed reader using the IndieWeb Search index without actually having to go to IndieWeb Search and submit a query.

To subscribe to a feed, paste a search page into your feed reader. Most feed readers should automatically find the feed for a page. Here's an example of a search page:

https://indieweb-search.jamesg.blog/results?query=eleventy

The corresponding feeds to this page are:

You can also subscribe to different pages on a search result page by putting the specific page URL in your feed reader, like so:

https://indieweb-search.jamesg.blog/results?query=eleventy&page=2

If you end up subscribing to a search result on IndieWeb Search, do let me know. Send me an email at readers@jamesg.blog. I'd also love to see applications that use the feed support in novel ways. Maybe you could build a command line interface that works with the search engine, for example.

Responses

Comment on this post

Respond to this post by sending a Webmention.

Have a comment? Email me at readers@jamesg.blog.