IndieWeb

How I am participating in the IndieWeb.

Sharing links to web pages that I find interesting.
Submitting webmentions to notify the sites that I have linked to them.
Following #indieweb hashtags on Mastodon and Bluesky.
Being supportive by letting people know I like their site or page or post.
Joined the IndieWeb slack, though I am mostly just lurking. I do find it difficult to join in existing conversations.

Still learning here. The only way to learn is to do.

The glue of the modern IndieWeb social fabric is Webmentions.

This website currently uses the Webmention.io service. Eventually I will build this service myself but for now I think it’s fine to use this external service. Also, they seem like they are excellent web citizens and so I am happy to support them!

The stack for this site:

Ruby my favorite programming language
Middleman a static site generator
Rake a build tool
Webmention.io a webmention service

Some of the guardrails and guiding principles for this are, simply:

I don’t care for javascript and I don’t want to add too much of it to the site. The site should work without it.
No tracking or analytics.
Need to store the webmentions in a format I can use as a data source for Middleman.
Use Ruby.
Build all of the tools I need and host them myself.

And of course, it’s not easy. It will be a constant work in progress. For example, this site depends on many 3rd party services and APIs.

The implementation is relatively simple. Using the gem webmention I fetch the webmentions and store them in a YAML file when I build the site. I store a timestamp of the last time I found new webmentions and then use this value to fetch new webmentions since that time.

The code I am using to fetch and store webmentions.

The directory structure is as follows:

lib/
    fetch_webmentions.rb        # Main module entry point
    fetch_webmentions/
        api_client.rb           # Client to interact with Webmention.io API
        config.rb               # Configuration
        repository.rb           # Loads/saves data file
        service.rb              # Main orchestration logic
    tasks/
        fetch-webmentions.rake  # Rake tasks
data/
    webmentions.yaml            # Persistent storage for incoming webmentions

When I build the site a process called FetchWebmentionsStep is run. This process fetches the webmentions and stores them in the data/webmentions.yaml file. FetchWebmentionsStep is part of a publishing plugin I have built for Middleman to help me publish my pages, posts, and notes. I can also run it manually as a rake task.

I am not quite ready to share the code for the publishing plugin yet, but I will soon (soon is a vague term, let’s just say sometime in the future). It is not yet ready for public consumption.

Rake task to fetch the webmentions:

rake webmentions:fetch

lib/tasks/fetch-webmentions.rake

# frozen_string_literal: true

require_relative '../fetch_webmentions'

namespace :webmentions do
  task :environment do
    raise 'WEBMENTION_API_KEY is not set' unless ENV['WEBMENTION_API_KEY']
  end

  desc 'Fetch all web mentions'
  task fetch: [:environment] do
    logger = Logger.new($stdout)
    logger.level = Logger::DEBUG

    config = FetchWebmentions::Config.new
    api_client = FetchWebmentions::ApiClient.new(config: config, logger: logger)
    repository = FetchWebmentions::Repository.new(
      file_path: config.data_file,
      logger: logger
    )

    service = FetchWebmentions::Service.new(
      config: config,
      api_client: api_client,
      repository: repository,
      logger: logger
    )

    count = service.fetch_and_store
    logger.info "Processed #{count} webmentions"
  rescue StandardError => e
    logger.error "Error: #{e.message}"
    logger.error e.backtrace.join("\n")
    raise
  end
end

lib/fetch_webmentions/service.rb

# frozen_string_literal: true

require 'uri'
require 'date'

module FetchWebmentions
  class Service
    def initialize(config:, api_client:, repository:, logger: Logger.new($stdout))
      @config = config
      @api_client = api_client
      @repository = repository
      @logger = logger
    end

    def fetch_and_store
      webmentions = @repository.load
      since = webmentions[:since]

      @logger.info "Fetching new webmentions since #{since}"

      mentions = @api_client.fetch_mentions(since: since)

      if mentions.empty?
        @logger.info "No new webmentions found"
        return 0
      end

      process_mentions(mentions, webmentions)

      webmentions[:since] = DateTime.now.iso8601
      @repository.save(webmentions)

      mentions.size
    end

    private

    def process_mentions(mentions, webmentions)
      mentions.each do |mention|
        target = URI(mention['wm-target'])
        @logger.info "Processing: #{target.path}"

        next unless valid_mention?(mention, target)

        add_mention(target.path, mention, webmentions)
      end
    end

    def valid_mention?(mention, target)
      unless @config.allowed_domains.include?(target.host)
        @logger.error "Unexpected host: #{target.host} for #{target.path}"
        return false
      end

      if @config.ignored_paths.include?(target.path)
        @logger.debug "Skipping ignored path: #{target.path}"
        return false
      end

      if @config.ignored_webmention_ids.include?(mention['wm-id'])
        @logger.debug "Skipping ignored webmention ID: #{mention['wm-id']}"
        return false
      end

      true
    end

    def add_mention(path, mention, webmentions)
      webmentions[:urls][path] ||= []
      webmentions[:urls][path] << mention
      @logger.debug "Added webmention for #{path}"
    end
  end
end

lib/fetch_webmentions/repository.rb

# frozen_string_literal: true

require 'yaml'
require 'fileutils'

module FetchWebmentions
  class Repository
    def initialize(file_path:, logger: Logger.new($stdout))
      @file_path = file_path
      @logger = logger
    end

    def load
      return default_data unless File.exist?(@file_path)

      content = IO.read(@file_path)
      return default_data if content.empty?

      data = YAML.load(content, symbolize_names: true)
      @logger.debug "Loaded webmentions | Last: #{data[:since]} | URLs: #{data[:urls].size}"
      data
    end

    def save(data)
      # Ensure directory exists
      FileUtils.mkdir_p(File.dirname(@file_path))

      # Convert symbol keys to strings recursively
      stringified_data = stringify_keys(data)
      File.write(@file_path, stringified_data.to_yaml)
    end

    private

    def default_data
      { since: nil, urls: {} }
    end

    def stringify_keys(obj)
      case obj
      when Hash
        obj.each_with_object({}) do |(key, value), result|
          result[key.to_s] = stringify_keys(value)
        end
      when Array
        obj.map { |item| stringify_keys(item) }
      else
        obj
      end
    end
  end
end

lib/fetch_webmentions/api_client.rb

# frozen_string_literal: true

require 'net/http'
require 'json'
require 'uri'

module FetchWebmentions
  class ApiClient
    ENDPOINT = 'https://webmention.io/api/mentions.jf2'

    def initialize(config:, logger: Logger.new($stdout))
      @config = config
      @logger = logger
    end

    def fetch_mentions(since:)
      uri = build_uri(since)
      @logger.debug "Fetching from: #{uri}"
      response = Net::HTTP.get(uri)
      JSON.parse(response)['children'] || []
    rescue URI::InvalidURIError => e
      @logger.error "Could not fetch from #{uri} | #{e.message}"
      raise e
    end

    private

    def build_uri(since)
      uri = URI(ENDPOINT)
      uri.query = URI.encode_www_form(
        'per-page': 1000,
        'domain': @config.domain,
        'token': @config.api_key,
        'since': since
      )
      uri
    end
  end
end

lib/fetch_webmentions/config.rb

# frozen_string_literal: true

module FetchWebmentions
  class Config
    attr_reader :domain, :allowed_domains, :api_key, :data_file

    def initialize(
      domain: 'roylindauer.com',
      allowed_domains: %w[www.roylindauer.com roylindauer.com],
      api_key: ENV['WEBMENTION_API_KEY'],
      data_file: 'data/webmentions.yaml'
    )
      @domain = domain
      @allowed_domains = allowed_domains
      @api_key = api_key
      @data_file = data_file
    end

    def ignored_webmention_ids
      []
    end

    def ignored_paths
      ['', '/']
    end
  end
end

Example datafile structure:

---
since: '2025-10-23T17:32:55-07:00'
urls:
  "/2025/01/01/some-post.html":
  - {...WEBMENTION_DATA}
  "/2025/01/02/another-post.html":
  - {...WEBMENTION_DATA}