While we were building the new Lullabot site, we decided that we wanted a slideshow of 'bots out and about doing their thing. Kicking ass and having fun at events is a big part of the Lullabot culture, and we wanted to show it off. We thought it would be fun to show a slideshow of the 20 or 30 most recent photos from the Lullabot team pictures Flickr group on our Who We Are page, to give a taste of what its like to be a bot. However, we didn't want to import all those photos into our site, wasting storage and duplicating data. Well, you know what they say: the Views module is the cause of, and solution to, all life's problems!
Beginning with Views 3, you can write your own plugin to replace Views' built-in SQL query engine. This means that you can make Views query against any kind of data source. The most common use case is to create Views that query remote web service; that sounded like a great match for our needs, and it's what we're going to explain.
This topic is pretty deep, so to keep things manageable I've divided it into three parts:
- Planning and modeling your data
- Creating a basic query plugin
- Exposing configuration options and handling arguments and filters
While this is not meant to be an in-depth guide to writing Views plugins, these articles will touch on a variety of different plugin types, and after reading it you will have at least a general understanding of how Views plugins work in addition to all the steps to create a query plugin specifically. The Drupalize.Me series Coding For Views is an excellent resource for a more general understanding of the Views architecture.
Let's get started!
Modeling your plugin data
One of the first things you need to do before coding your plugin is sit down and think about how the data being returned from your API maps to the data that Views expects. Views is designed to represent tabular data, the basis of which is a row of fields. Many API endpoints do not follow this model. For example, a single request may contain a lot of nested data. When writing a query plugin for the last.fm events API, I found that the results for a single event contain a nested list of places to buy tickets for that event. This obviously doesn't map well to the row/field model. The proper way to handle this would be to create a relationship plugin to link that data to its event. However I didn't want to add the complexity, and the data wasn't that important, so I simply chose not to expose it in my plugin.
With Flickr I had the opposite problem - the data I wanted was scattered across multiple API functions, each one requiring a single request. Lets look at this example in detail, since that is what we're going to build here. In the end, we will want to generate a list of photos from a group, with the following information about each photo
- The photo's title (for alt/title tag use)
- The URL of the source image
- The URL of the photo's Flickr page (for linking each image)
- The photo itself, run through a specified image style
From a Views standpoint, we want to expose each of these pieces of data as a field. So lets see what we need to do to get all these pieces of data from the Flickr API. To get photos from a group pool, we need to use the flickr.groups.pool.getPhotos API call call. This returns a set of photos for a specified group. However it does not give much data about the photos other than their IDs.
Looking through the API some more, it appears we can use flickr.photos.getInfo to get the photo title and a link to the photo page, but even there we still can't get the URL of the photo itself. In order to get that we need to call flickr.photos.getSizes and choose which size we want. It seems like the list of sizes is unpredictable, but assumedly every photo will have an 'Original' size, so we'll use that as a standard choice.
So what does this mean for our views plugin? Well, in order to get the four fields we want, we will have to get a list of photos from flickr.groups.pool.getPhotos, then for each photo we will need to call flickr.photo.GetInfo and flickr.photo.getSizes. Once we've done that, we will have the info we need. The downside, of course, is that if we want to grab 20 photos from the group, we need to make 41 API calls (two per photo plus one for the group listing.)
This is pretty typical of the query plugins I've written. The APIs are rarely written to conveniently map data to a single row as neatly as Views might need it. This is why it is worth taking sometime to investigate the API you are using and figuring out how the data it provides maps to your use case before sitting down and starting to write any code around it. In some cases you might find the data you want isn't available at all, or that the data needs some significant massaging before it can be used the way you want it.
Other considerations
Given that we will have to be making a large number of API calls in order to get data about a single photo, we will probably want to investigate a caching strategy to reduce round trips. Another consideration I had at the beginning of the project was that I preferred not to interact with the API directly, but to instead use a module or library that abstracted a lot of that work away for me. In particular, it would be nice not to have to write the Flickr authentication code myself.
Thankfully Drupal contrib provided a solution to both these problems. The Flickr API module provides an object that wraps all the functions in the Flickr API, as well as providing built in caching and easy authentication through the Drupal admin UI. What an enormous amount of code I no longer have to worry about! It's really great when you find a module like this that does exactly what you need.
Note: in order to use Flickr APIs and the flickrapi module you need to create a Flickr API key.
Conclusion
When writing any piece of sufficiently complex code, taking the time to think about your problem and the best way to solve it will pay great dividends down the road. Writing a set of Views plugins is no exception. You need to think about the data you have, the data Views expects, and how to deal with the complications that arise when the two don't fit together perfectly. If you're designing your own system from scratch, you have the luxury of building the APIs just so to fit your desired use case. Sadly, life is rarely so neat. Resist the temptation to dive straight into code, and first figure out what it is you need to build.
In part 2 of this series, we'll go through the steps of building our plugin, ending up with the simple use case of displaying a list of photo titles. Stay tuned!