Wikipedia and IFTTT: A Technical Guide

Here at Hatnote, we build on Wikipedia a lot. And while we love building projects like Listen to Wikipedia and The Weeklypedia, we have to admit programming, integrating, and maintaining reliable services can be a lot of work. Creating cool and functional Wikipedia projects remains out of reach of most busy Internet denizens. Until today.

With the aim of bringing Wikipedia to the wider web, Stephen and I are pleased to have worked with Wikimedia and IFTTT to build the brand-new Wikipedia IFTTT channel. This post is your 10-minute usage and technical guide to all things Wikipedia+IFTTT. For the official announcement, see the Wikimedia blog.

IFTTT Logo

IFTTT connects web services, so when something happens on one service (the Trigger), it pushes an update to another service (the Action). With the new Wikipedia channel, so you can create customized combinations, called Recipes, to plug Wikipedia into your Internet ecosystem. With this new channel, Wikipedia offers eight new Triggers that can be connected to IFTTT’s 200+ other channels, including email, phone, Android, iOS, Twitter, Tumblr, Slack, and many more. There are already dozens of example Wikipedia Recipes ready for activation.

Even if you’re already familiar with IFTTT, a few things set Wikipedia apart from other channels. Foremost, whereas most other services provide an API to data and information, Wikipedia provides a direct interface to curated knowledge. So trust us when we say it pays to invest in learning the special nature of Wikipedia, as seen through the lens of the new IFTTT Triggers.

The Daily Triggers

The simplest of the new Triggers, these three update once per day around midnight UTC. Their simplicity makes them the perfect candidates to introduce Wikipedia Fact #1:

Fact #1: There are many Wikipedias! Most people are familiar with Wikipedia as one site. But unlike other IFTTT channel providers, the fact is there is one Wikipedia per language, each with its own conventions and quirks. We’ve made all the new Triggers customizable for all Wikipedia’s 290 languages. So keep in mind for future triggers, a recipe that works well on one language, may not apply at all to the others. As a multilingual Wikipedia editor, I’ve found clicking the interlanguage links on individual articles and referring to the Weeklypedia archives are good ways to get acquainted with the differences.

The Daily Triggers are are an exception to the interlanguage differences. Almost all Wikipedias, regardless of size and age, follow the convention of featured articles. So feel free to get a weekly dose of free knowledge in your language of choice!

Edits to an Article

When most people talk about Wikipedia, they’re talking about the articles, Wikipedia’s fundamental unit of knowledge organization. Articles are written by many users contributing what knowledge they can, edit by edit.

On the surface, the Edits to an Article Trigger is simple. If you have a favorite article, IFTTT will monitor it for changes and notify you when an edit occurs. But of course there are caveats.

Fact #2: Wikipedia is a uniquely open platform. That openness is reflected in the data passing through IFTTT APIs. Don’t be surprised to see raw or non-constructive changes made (and usually reverted) by Wikipedia editors, often in quick succession.

Fact #3: Wikipedia is a community with many subcommunities. Quality, approaches, and discipline will vary between these subcommunities. For instance, to match naming conventions, it’s common for articles to be renamed (“moved”) multiple times, especially when they’re brand new.

If you’re learning new things from the facts and think you might have some better solutions, Wikipedia wants to hear about them. In the meantime, you can have IFTTT DM you on Twitter when there are updates to the discussion or new terms that might help in the debate.

Edits from a User

Editors write articles. You don’t need to be logged in to edit Wikipedia, but it helps with site customizations and better edit history maintenance. Now, using the Edits from a User Trigger you can easily connect a given user’s edit activity with other social sites, making Wikipedia more personal than ever.

Other than Stephen and me, finding interesting users to follow is a topic for another post. But there is one type of user that has garnered a lot of attention in the past: the Bots. Wikipedia’s Bots are more administrative power tool than autonomous overlord, but people are still obsessed. Just as well, because this brings us to the next fact.

Fact #4: Wikipedia grows at different speeds. Some pages change quickly, some slowly. Generally the larger the item, the faster it changes, unless it’s brand new. This is even visible at the site level: English Wikipedia sees about 100,000 changes per day, French has 20,000, and Farsi has 200. If your Trigger event is too high-velocity, occurring more than 50 times per hour, it is likely not a good fit for IFTTT, and you could get temporarily blocked by a Recipe’s Action’s service. If it’s too low-velocity, you might be disappointed with an otherwise sound recipe.

Some bots, like ClueBot, are very active, and would not play well with IFTTT. Other bots edit only a couple times a day, like SpaceFactsBot. It updates space-related world records in progress, and would be a fine IFTTT Trigger candidate. Tweet its edits (or your own), by filling in this recipe.

Edits with a Hashtag

Hatnote is a big fan of hashtagging on Wikipedia. Hashtags are the best way to unify edits with a common purpose across articles and editors. If you’re editing with friends, colleagues, or Editathon participants, easy-to-use hashtags build momentum and record your hard work.

Fact #5: Set up earlier rather than later. This is hardly specific to Wikipedia, but whenever possible, setting up a recipe beforehand is way easier than retrieving the data later. Don’t forget to test them out to make sure they work! That’s what the Sandbox is for.

With a little bit of forethought and the Edits with a Hashtag Trigger, you can automatically build that record into a blog or other manifest. You can even push it to a Google Spreadsheet, and access that as a form of JSON API, giving you a whole API backend in no time.

Article added to a Category

We’ve saved the best for last, because now we’re getting into the most complex and exciting Triggers in this release. The Article added to Category Trigger is one of our favorites because it is so open-ended. You can get really creative with them. Just look at these Recipes:

In case you hadn’t interacted with them before, Categories can be found by scrolling to the bottom of any Wikipedia article. An article’s associated Talk page often references administrative categories, which are interesting in their own right. In fact, due to technical limitations for this recipe and the next, Talk page changes are automatically resolve to their “main” namespace counterparts. Talk goes to the Article namespace, User talk to User, Project talk to Project, etc. It is not possible to monitor a Talk page exclusively.

Categories are full of promise, and we eagerly anticipate the community’s leveraging of this Trigger, but that promise comes with a price.

Fact #6: Wikipedia Categories can be problematic. First, the caveats of the previous facts apply doubly to Categories. Category systems and conventions vary greatly between communities and languages. Some Categories are huge and change frequently, others are small or even empty, despite having broad-sounding names. Research is necessary.

Second, Categories often assume an unpredictable tree structure, wherein parent categories do not automatically contain their subcategories’ members. Furthermore, cycles can exist. Finally, due to the way pages are added and removed to Categories, it is possible for a page to appear to be added to a Category multiple times, simply due to editing snafus. Do not actually rely on the Presidential phone call for your think tank’s next strategy summit.

Most of these complexities are manageable through other means. For instance, if one wanted to monitor a Category’s Subcategories, one would simply have to do a bit more mousework, and set up multiple Recipes to perform the same Action, as we’ve done in the examples for this next Trigger.

Edits to Articles in a Category

Some would argue the most complex Trigger, but for those who have been following along so far, you can probably guess exactly what this will and won’t do. Use the Edits to Articles in a Category Trigger to simultaneously monitor multiple articles related to a specific topic.

From Fact #4, we know that if the topic is too broad, the edit speed may outpace IFTTT’s effectiveness. From Fact #2 and #3 we know that Categories mean different things to different communities. From Fact #6 we know that a given Category may not always be what it appears. And finally, from Fact #1 we know that Categories don’t apply across languages, so we recommend disabling the language customization field for published Category-based recipes.

Add in the Talk-page caveat from the previous Trigger and it’s been a long road, but we’re already reaping the returns. For instance, using the Tweet about Wikipedia Updates in a Category Recipe, there are already three special-interest Twitter accounts running solely off Wikipedia with no specific programming overhead whatsoever:

When it’s this easy, one might wonder where all the complexity went. Well, if you’re really curious about the technical details, read on.

Behind the Scenes

Hard at work at Hatnote HQ:

Stephen at Sugarlump with Snacks

The Article Edit Trigger, User Edit Trigger, and Daily Triggers are based on basic queries to the Wikipedia API. The Wikipedia channel includes a daily update on the Wikipedia Article of the Day, Wikimedia Commons Photo of the Day, and the Wiktionary Word of the Day. All three of these feeds are also available as RSS feeds from the API.

The Hashtag Trigger and Category Triggers are all based on SQL queries to the Wikipedia database replicas that are openly available on Tool Labs. The service that does all the API calls, SQL queries, and serves all the IFTTT requests is written in Python.

The very astute might notice that all the Wikipedia IFTTT features are Triggers for reading as opposed to Actions for editing. While Wikipedia supports OAuth, there are existing community conventions and affordances for automated editing that require further consideration and discussion.

Thanks

Many thanks to Ori, Dario, Ed, Niki, Yuvi, Kirsten, and many others for their hard work and feedback during the Wikipedia+IFTTT development.

We hope you found the guide useful. If you build anything, tweet it out and CC us, our handles are linked below. Happy automating!

Mahmoud and Stephen

  1. capsulebase reblogged this from superdrivel
  2. superdrivel reblogged this from hat-note and added:
    Also featuring Persian snacks:...
  3. hat-note posted this