Three years ago Greg Dunlap wrote a series of articles about building Views query plugins in Drupal 7. A lot has changed since then. Drupal 8 has been out for a year now and Views is in core. I recently had an opportunity to write a Views query plugin for a Drupal 8 project, and it worked out surprisingly well. So, how has the process of implementing a Views query plugin changed? As in the original series, we’ll take a look at how you build one from scratch.
The original article series used Flickr as an example of a remote service you could expose to Views via a query plugin. The Flickr API module that was used does not have a Drupal 8 port, so I needed to port something for the series. We’re big Fitbit users at Lullabot, myself included, so I thought it might be fun to use the Fitbit API as an example of what can be done with Views query plugins. We’ll use the Fitbit API, exposed via a Views query plugin, to build our own custom leader board for our Drupal site.
What’s a Views query plugin?
Since Views 3 in Drupal 7, you could write your own plugin to replace Views’ built-in SQL-query engine. This means that you can make Views query against any kind of data source, using the same Views UI your site administrators are accustomed to. The most common use case is to create views that query a remote web service.
The big picture of how you write a Views query plugin hasn’t changed much. You’ll still go through the same steps as in Drupal 7, so we’ll follow the original article series and divide it into three parts:
- Planning and modeling your data
- Creating a basic query plugin
- Exposing configuration options and handling arguments and filters
Probably the biggest change in writing Views query plugins in Drupal 8 is the use of Drupal 8’s plugin system. It’s helpful to have a general understanding of Drupal 8 plugins as we’ll skip over some of those details. The Drupalize.me Drupal 8 Module Development Guide is an excellent source for general information about Drupal 8 plugins.
Let’s do it!
Modeling your plugin data
One of the first things you need to do before coding your plugin is sit down and think about how the data being returned from your API maps to the data that Views expects. There are a lot of moving parts when writing a Views query plugin, so it’s good to resist the temptation to dive right into code. Let's first figure out the nature of the remote API and what we need to do to transform the data into a Views-friendly format.
Views is designed to represent tabular data, the basis of which is a row of fields. Many API endpoints do not follow this model. For example, a single request may contain a lot of nested data. The query plugin I wrote on a recent project wrapped a Search endpoint that returned faculty member search results. The results contained arrays of research interests and degrees. It was easy enough to flatten these arrays into comma separated lists and present to Views a single field, but it’s not difficult to imagine more complex data. Perhaps the results should include details about the Department the faculty member belongs to. Complex nested data may necessitate a Views relationship plugin.
The Fitbit API exposes a number of endpoints. We’ll first hone in on the User endpoint. It’s pretty straightforward in that it returns nearly tabular data. Issue a request and you get back a list of key-value pairs. It’s also got just enough data in the response to put together a simple leader board based on users’ average daily steps. Here is an example of what it returns (clipped for brevity):
{
"user": {
"age": 32,
"avatar": "https://d6y8zfzc2qfsl.cloudfront.net/B8395E1F-346C-1E31-E74C-0AE2512A38BD_profile_100_square.jpg",
"avatar150": "https://d6y8zfzc2qfsl.cloudfront.net/B8395E1F-346C-1E31-E74C-0AE2512A38BD_profile_150_square.jpg",
"averageDailySteps": 7334,
"displayName": "Matthew O.",
"topBadges": [
{
"image100px": "https://static0.fitbit.com/images/badges_new/100px/badge_daily_steps30k.png",
"name": "Trail Shoe (30,000 steps in a day)",
},
{
"image100px": "https://static0.fitbit.com/images/badges_new/100px/badge_lifetime_miles1997.png",
"name": "India (3,213 lifetime kilometers)",
}
],
}
}There are a few values with nested data, in particular topBadges, but we’ll see that it’s not that difficult to mange these. From a Views standpoint, we want to expose each of these pieces of data as a field.
There is a wrinkle when it comes to the Fitibit API and our ability to translate it for Views, perhaps you’ve already noticed it. Each response contains only a single user. The API does not allow for multiple users’ data to be returned by a single request. This is because the data can be very personal; values like gender, height, and weight are included in some responses, but only with the individual user's explicit permission. Fitbit’s API follows the OAuth 2.0 Authorization Code Grant. Under the OAuth 2.0 Authorization Code Grant scheme, your application, in this case our Drupal module, has to be registered with Fitbit. Then, in order to query the API on behalf of users, each user must grant access to the application. When a user grants access, they have the option of selectively granting scope, for example they can choose to share activity data, such as steps, distance, calories burned, and active minutes but choose to omit profile data, which includes values like height and weight. Authentication is outside the scope of this article but take a look at the Fitbit base module if you’re interested.
What does this mean for our Views implementation? Well unlike a SQL-query backend which can receive multiple rows in a single request, we’ll need to write our Views query plugin such that we loop over all of our authenticated users and make a single API request per user to aggregate the “table” we give back to Views. This kind of complexity is the trickiest part of writing a Views query plugin. Remember, Views deals in tabular data, so you have to analyze your remote data source and, anywhere it doesn’t fit the bill, you need to translate it for Views.
Data spread across more than one API request represents another common hang up. Here too you have to aggregate the data and then translate into a tabular format. There are a couple ways you can do this, you can either hit each endpoint in turn and present the aggregate fields to Views as a single “table,” or you can implement a Views relationship plugin. By implementing a Views relationship plugin you can defer the decision of which API endpoints are hit to the site administrator.
Take our Fitbit module for example. The Fitbit API has an endpoint to retrieve user profile data, and a separate endpoint to retrieve daily activity summary data. To surface data from both endpoints to Views, we could just hit both endpoints all the time and give the aggregate result back to Views, however, that could be wasteful. The site administrator may have no interest in daily activity summary data. Instead, we can use Views relationships to give the site administrator the option to opt in to daily activity summary data. When the relationship is present, our Views query plugin will know to hit both endpoints.
Other considerations
If the number of authenticated Fitbit users on our site grew, we’d reach a point where performance and rate limiting would become a concern. We will probably want to investigate a caching strategy to reduce the number of round trips. Another concern I had at the beginning of the project was that I preferred not to interact directly with the API, but to instead use a module or library that abstracted a lot of that work away for me. In particular, I thought it would be nice to not have to write the Fitbit authentication code myself.
There wasn’t any module available for Drupal 8, but I did find a Composer library that I could use to do a lot of that work for me. The OAuth 2.0 Client Library together with the Fitbit provider provided a great stepping stone. So in true Drupal 8 fashion, I stepped off the island and harnessed the power of these external dependencies to take care of the heavy lifting around authentication. The author of the Fitbit provider was even open to pull requests, making the experience much more fun.
Conclusion
When writing any piece of sufficiently complex code, taking the time to think about your problem and the best way to solve it will pay dividends down the road. Writing a set of Views plugins is no exception. You need to think about the data you have, the data Views expects, and how to deal with the complications that arise when the two don’t fit together perfectly. If you’re designing your own system from scratch, you have the luxury of building the APIs 'just so' to fit your desired use case. Sadly, life is rarely so neat. Resist the temptation to dive straight into code, and first figure out what it is you need to build.
In part 2 of this series, we’ll go through the steps of building our plugin, ending up with the simple use case of having a Fitbit leaderboard. Until next time!