Storing and retrieving webmentions with Firebase

October 7th, 2019

I've recently transformed my Statamic powered site into a statically hosted version, mostly because services like Netlify and Firebase offer free hosting for static files, which means I don't need to manage and pay for my own server. The only dynamic piece that was missing was a way for people to react or comment to my blog entries.

For this I've set up webmentions. If you want to know more about webmentions or the Webmention standard, you can learn more about it here.

For more information on how to set up webmentions for Twitter posts and receiving webmention webhooks through webmention.io, Freek has a very good blogpost about how he set up webmentions on his Laravel powered blog.

Receiving webmentions with Firebase

Receiving the webmention.io webhook is the first step, you can do this on a Firebase hosted website by using the firebase console and setting op Firebase Functions in the root of your project. To get started, we'll need to install the Firebase tools.

npm install -g firebase-tools #Install the Firebase tools
firebase init functions # Initialize the functions

When initialising the functions, Firebase will ask you if you want to use JavaScript or TypeScript for writing your function. In my case I've chosen JavaScript but you can choose whichever you prefer.

Setting up the shared webhook secret

Webmention.io requires you to set up a secret when using the webhook, to verify that you're receiving a valid request. This is not something you want to hardcode in your codebase, so Firebase allows you to set config variables for functions using the Firebase Tools

firebase functions:config:set webmention.secret="my_secret"

Afterwards you can retrieve this config variable using

functions.config().webmention.secret

Processing the request and storing data in Firestore

After our configuration for the secret is done, it's time to process the POST request we receive from webmention.io. Firebase will automatically create the collection for you in your account once you've created a database for your project.

First we'll capture the request, to do this we'll need to import the firebase-functions and firebase-admin libraries and initialize our app.

import * as functions from 'firebase-functions';
import * as admin from "firebase-admin";

// Initialize the admin credentials to use Firestore
admin.initializeApp();

Once that's done, we can capture the webhook request

import * as functions from 'firebase-functions';
import * as admin from "firebase-admin";

admin.initializeApp();

export const webmentions = functions.https.onRequest((request, response) => {
  // Captured the request
});

To verify that the request comes from Webmention.io we'll now have to retrieve our secret from the configuration, and for an extra safety measure we'll check if the request is a POST request.

import * as functions from 'firebase-functions';
import * as admin from "firebase-admin";

admin.initializeApp();

const secret = functions.config().webmention.secret;

export const webmentions = functions.https.onRequest((request, response) => {
  if (request.method !== 'POST') {
    return response.status(500).send('Not Allowed');
  }

  if (secret !== request.body.secret) {
    return response.status(400).send("Invalid secret");
  }
});

Once we know we're dealing with a valid request from Webmention.io, we can add our webmention to the Firestore

import * as functions from 'firebase-functions';
import * as admin from "firebase-admin";

admin.initializeApp();

const secret = functions.config().webmention.secret;

export const webmentions = functions.https.onRequest((request, response) => {
  if (request.method !== 'POST') {
    return response.status(500).send('Not Allowed');
  }

  if (secret !== request.body.secret) {
    return response.status(400).send("Invalid secret");
  }
  
  return admin.firestore().collection('webmentions').add({
    type: request.body.post['wm-property'],
    webmention_id: request.body.post['wm-id'] || '',
    author_name: request.body.post.author.name,
    author_photo_url: request.body.post.author.photo,
    author_url: request.body.post.author.url,
    post_url: request.body.target,
    interaction_url: request.body.source,
    text: request.body.post.content
    ? request.body.post.content.text
    : '',
    created_at: request.body.post.published
    ? new Date(request.body.post.published)
    : new Date(),
  }).then(result => {
    return response.send('Webmention added');
  }).catch(error => {
    return response.status(500).send(error);
  });
});

As an extra safety measure, I don't want to add a webmention if we've received it before, this can be done by checking the Firestore for the webmention_id before adding the new webmention

import * as functions from 'firebase-functions';
import * as admin from "firebase-admin";

admin.initializeApp();

const secret = functions.config().webmention.secret;

export const webmentions = functions.https.onRequest((request, response) => {
  if (request.method !== 'POST') {
    return response.status(500).send('Not Allowed');
  }

  if (secret !== request.body.secret) {
    return response.status(400).send("Invalid secret");
  }
  
  return admin.firestore()
    .collection('webmentions')
    .where('webmention_id', '==', request.body.post['wm-id'])
    .get()
    .then(querySnapshot => {
      if (querySnapshot.docs.length > 0) {
        return response.send('Already added');
      }

      return admin.firestore().collection('webmentions').add({
        type: request.body.post['wm-property'],
        webmention_id: request.body.post['wm-id'] || '',
        author_name: request.body.post.author.name,
        author_photo_url: request.body.post.author.photo,
        author_url: request.body.post.author.url,
        post_url: request.body.target,
        interaction_url: request.body.source,
        text: request.body.post.content
        ? request.body.post.content.text
        : '',
        created_at: request.body.post.published
        ? new Date(request.body.post.published)
        : new Date(),
      }).then(result => {
        return response.send('Webmention added');
      }).catch(error => {
        return response.status(500).send(error);
      });
  });
});

Deploying the function

Once your function is ready, you can deploy it using the Firebase cli tool

firebase deploy

Firebase will then set up the necessary HTTP endpoint you can use to configure the webhooks on webmention.io. Your function will live on a similar URL to https://us-central1-your-project.cloudfunctions.net/webmentions

Retrieving and displaying the webmentions

Once we've received some webmentions in our store, we can start displaying them using the Firebase JavaScript library. The first step is initialising the library with the necessary configuration.

import * as firebase from "firebase";
import "firebase/firestore";

const firebaseConfig = {
    apiKey: '### FIREBASE API KEY ###',
    authDomain: '### FIREBASE AUTH DOMAIN ###',
    projectId: '### CLOUD FIRESTORE PROJECT ID ###',
    databaseURL: "https://your-app.firebaseio.com",
};

firebase.initializeApp(firebaseConfig);

Once everything is set up, you can start querying your Firestore, it will warn you that an index is needed to query on both post_url and sort on created_at, the error in your browser console will link straight to your project to create the necessary index.

const db = firebase.firestore();
const url = 'https://url-to-your-blogpost.com'; // Up to you how you retrieve this

db.collection("webmentions")
    .where('post_url', '==', url)
    .orderBy('created_at', 'desc')
    .onSnapshot((querySnapshot) => {
  	let webmentions = [];
        querySnapshot.forEach(doc => {
            webmentions.push(doc.data());
        });
        renderWebmentions(webmentions);
    });

The renderWebmentions function then receives all webmentions for the specific url, up to you how you want to display them. A really nice feature of using the Firestore this way is that updates are realtime, as soon as a webmention comes in, it's displayed on the page without having to refresh the page.

Index confuguration in code

You can also manage your indexes in code if you prefer by adding them to your firestore.indexes.json config file, make sure to run firebase init firestore, it will create the necessary config files for you.

{
  "indexes": [
    {
      "collectionGroup": "webmentions",
      "queryScope": "COLLECTION",
      "fields": [
        {
          "fieldPath": "post_url",
          "order": "ASCENDING"
        },
        {
          "fieldPath": "created_at",
          "order": "DESCENDING"
        }
      ]
    }
  ]
}

You can deploy changes to your indexes using firebase deploy --only firestore

You can comment or react to this post by replying to this Tweet
MENU