Internal API Design for Distributed Teams

My first real project with multiple distributed teams and a timeline measured in weeks (not months!) was the launch of The Tonight Show with Jimmy Fallon. With six weeks to go from nothing to a CMS, a website, and multiple apps, we were forced to figure out the best way to keep all the teams always moving. I wrote then about Legally Binding your Web APIs, but I think it’s time to clarify and update some of it’s points for maximising your team’s productivity.

Introduce the teams

Before features are defined, APIs are designed, and work is started, it’s really important to have some sort of team introduction. This doesn’t need to be a long meeting. What’s important is to get at least one technical person who will need to create or use APIs from each distinct team introduced. At a high level, identify the API providers and consumers from a business perspective. Determine who owns the APIs and their specifications. Make sure each technical person understands the goals of the project. Then, unless you are one of those technical people, get out of the way! Leave the meeting, drop the call, and let those people get to work.

Lay the API Foundation

Every API makes assumptions - about the basic structure of data, about authentication, and about maintenance and updates. Before determining your core objects, figure these things out. Discuss if you’re implementing an RPC API or a REST API. Decide if you’re supporting JSON, XML, or both. For REST APIs, find a suitable specification that supports the Hypermedia Factors you need, like JSON API or HAL, and use it to structure your data and API to simplify client and server implementations. Decide on authentication protocols, like OAuth or HTTP basic. Decide on a versioning strategy such as versioned URLs or custom MIME types. Steal from existing internal implementations where possible. Finally, start writing code.

The problem with beginning implementations now is that there is no source of truth for the API. It’s easy for teams to start making bad assumptions, leading to rework and putting your launch at risk. Instead, use a tool built for managing API documentation like Apiary’s Blueprint or Swagger. Ideally, the tools will let you easily generate API stubs too. Doing so will unblock client implementations, as they won’t be dependent on the production API to start their work. Best yet, when there is disagreement between implementations, the docs become the arbiter of what is right. Sure, documentation can be wrong or miss key details, but it’s much easier to understand a spec than someone else’s code. Best yet, since the first API consumer will be an involved participant in writing the docs, your API will have docs ready to go when that second consumer comes along.

Now, your teams can start writing code, pulling in the core libraries for authentication and data structures.

Design API Objects

Now that you have the basic API details figured out, you can start to design objects (or in REST parlance ‘resources’). At this stage, you are modelling data, and not actions. Assume every object has a GET call, even if it’s not immediately clear that it’s needed. Letting your clients inspect the state of objects will help them to debug your API and their code without relying on expensive calls and meetings that suck up time from getting things done. Design the object schema to be as “obvious” as possible—don’t use ‘image’ in one object attribute and ‘picture’ in another, unless they mean different things. Use REST best practices such as using URLs as canonical identifiers of objects. A critical step many teams miss is determining who manages unique IDs. With the inherent concurrency and assumed unreliability of HTTP requests, identifying which systems manage which IDs is something not to forget.

Again, at this point you should have new documentation and stubs, but no code. Let your documentation continue to be the source of truth.

Design API Operations

Using your work to this point, API operations should start to have intuitive implementations, especially if you are using REST best practices. Map your various CRUD operations to GET / POST / PATCH / PUT / DELETE as required. Find good HTTP status codes to communicate the results of REST calls (my favourite new discovery is 409 Conflict).

One common mistake services teams make at this point is to create a “wide-open POST endpoint” that accepts arbitrary data, and then to reverse-engineer the data into an implementation. Remember, our goal is to keep any sort of dependencies between the teams to a minimum. This sort of development methodology maximizes the dependencies between the teams, and requires implementations that might be totally wrong. It makes what should be an explicit API contract implicit. When the API calls break, there’s no documentation to fall back on, forcing calls and meetings that block all the teams.

Implement All The Things!

All the work so far has been to get to this point of productivity. By now, there should be no hard dependencies between the teams. Stubs can be replaced by implementations as development continues. New APIs can be added quickly by following the existing patterns. API consumers can even make assumptions about how an API would likely be designed before they contact the API team. Questions and iterations can be focused on tickets and documentation comments, helping to preserve the feasibility of the launch date. Stakeholders, project managers, developers, all win.

Header image from The simplest foundation, a padstone