Designing a Beautiful REST API with JSON

There is a lot of documentation about how to build REST API using JSON formatted data in internet and I will try to summarize a little bit what I gather as knowledge about what is the right way to build RESTful API with JSON.

Why we use APIs?

Building APIs allows you to clear application layers and give you the freedom to separate the server side as a separate application.
Having a separate application for data access  allows you to sell product without front-end, profiting from your ideas and in the same time allowing other developers to build rich user interfaces on different platforms and devices.

Why REST?

REST bring into your application saleable web content, general paradigms and protocols not related to the clients programmable language, independence of components and releases, latency (because of caching), security and encapsulation.

Why JSON?

Currently JSON is the preferred language for 55% of the developers working on all type of applications.
It’s simple, readable, scalable and flexible.

It’s important to know several things about REST API before you start.
Representational State Transfer (REST) is well described in this small Wiki article.

Regardless of it seems easy, developing REST API with JSON may not be so easy. In most cases developers run into building JSON-RPC protocols instead of REST API.
To avoid troubles you shall make and follow some guidelines.
According to the recommendations in “RESTful Web APIs” by Leonard Richardson, Mike Amundsen and Sam Ruby, it’s best if you use standardized JSON format. And only in case you can’t find one fitting you data you shall use your own standard, but make a documentation and share it for standardization.

What is important to think about when you design a REST API?

First, think about resources – Your API will manipulate resources through their representations delivered to the client.
So your REST shall be designed in this way that client can discover everything what he can do with the resource from what he received in response of the initial request.

In simple rules you should think in “Nouns, not Verbs” when you think about your resources. You shall make your resources “Coarse Grained, not File Grained” because fine gaining a resource will soon make the need of a different version and you will end up with tenths of methods exporting same resource with slight differences.

Keep it simple!
Design only Collection and Instance resources.

Collection resource for example is the resource you can request on a specific URL like /emails. By looking only to the URL you see it’s plural and this is already giving you the knowledge that you will receive multiple things in response.

A Instance resource is accessible with URL like /emails/123. Reading this URL you understand that you will receive an item that is a child into the Collection resource.

Resources in REST are manipulated using a short list of methods and this list of methods is inherited from HTTP protocol: GET, PUT, POST, DELETE and HEAD.
There PATCH Method for HTTP in standardization – designed to add a feature to do a partial resource modification. (RFC 5789)

Remark that GET, PUT, POST and DELETE are inherited from HTTP, but not similar.
As GET is a read, DELETE is a delete and HEAD is for receiving only headers without body – it’s correct to say those are one to one correspondence to the HTTP methods behavior.

PUT and POST can both be used for creating and updating resources.
PUT always must contain the whole resource representation. This makes it idempotent operation.

As it’s not commonly used to use PUT for creating, it may look not obvious how this can be done. The key is that it can be used only in cases when you allow client to generate unique ID for the resource recorded.

PUT /emails/{clientGeneratedId}

{
  ...  entire resource data ...
}

POST is commonly used for create, but it’s used on parent resource.

POST /emails
{
   "email" : "x@y.com",
   "dne" : true
}

Response:
201 Created
Location: http://api.vlaevski.com/emails/345

Important is that you will not receive HTTP response 200 but 201 that will indicate something has been created. This response shall contain Location header with full path to the new created resource.

It’s important when you design the API to follow the right response rules. To respond the right exceptions according to HTTP protocol.

POST used as update allows you to update only one property of a resource and it’s works with instance resource URL.

POST /email/345
{
   "dne": true
}

Response:
200 OK

 

Use the right media types!

As we speak about REST with JSON the correct request shall contain Accept header set to application/json and the correct response shall return Content-Type set to application/json.

In some cases yo may build an API that response ont only into JSON format. For this reason client shall use Accept header in request so that server knows what type of response to generate. Some clients are limited and in those cases they can use suffix at the end of the resource.

//Request with Accept header showing the client can parse both JSON and CSV format
GET /emails/345
Accept: application/json, text/plain

//Example of using suffix into the URL
/emails/345.json
/emails/345.csv

Remark: Conventional suffix approach shall be taken with priority in front of the Accept header approach.

Beautiful REST

Now I will recommendations some solutions that are not required for REST API to be REST, but they are the changes that will elevate your RESTful API to be a beautiful API.

Use simple base URLs.
“http://api.foo.com/” is better then
“http://www.foo.com/dev/service/api/rest”.
As you can see the second one is not wrong and you may find a lot base URLs like it in usage, but is not the easiest for adoption and not intuitive.

Use version control.
In case your project may grow to a point when you will need to change resource representation, you shall plan a version control element from the beginning. This will allow your clients to use both old and new resource representation until the old one is no longer supported. Avoid making a lot of version changes.
The chip visioning is made adding version to base URL. (like “http://api.foo.com/v1/”), but the right way is to use Content-Type header.

Use camelCase.
As this is JSON you shall use camelCase not other naming conversions.

Use ISO 8601 for dates and time stamps.
This is the “2015-01-30T00:21:24:343Z” format in UTC (GMT).

Use HREF instead IDs.
Client shall be able to address resources you have. For this reason each resource shall have global and unique URL. Delivering ID to the client require client to know how to build the URL to access a resource. But delivering a HREF will make things simple.

{
   "href":"https://api.foo.com/v1/emails/343",
   ...
}

Note that it’s recommended to avoid using sequential numbering when creating a resource. This way you will protect your resources from users pooling random data through URL generations.

Using non sequential numbering is also good for clustering your API.

Always return body (on POST)
It’s a good approach if you plan to return the resource after a POST request. In some cases this may be the same data you just received from the client and this may only load traffic.
If you want to avoid this design an override: skip the body when POST is send with ?_body=false in query strings.

Linking and referencing resources in REST
As I wrote above it’s important that after your client receives the first response, he shall be able to find all needed information about linked or referenced resources from the received data.
In JSON there is no W3C or RFC standards about how to make resource linking, so you have to find your way.
For me one effective approach is to use a complex object with one property – the HREF property containing the URL to get to that resource.

GET /emails/345

200 OK
{
  "href" : "https://api.foo.com/v1/emails/345",
  "email" : "x@y.com",
  ...
  "subscriptions" : {
      "href" : "https://api.foo.com/v1/subscriptions"
  },
  "lastSource" : {
      "href" : "https://api.foo.com/v1/sources/555"
  }
}

It’s easy to use with both collection and instance linking.

Reference Expansion
Also named as Entity expansion or Link expansion this is all bout receiving several lined instances or collections with one request.

A parameter into query string will force the API to replace the HREF object into the response with the whole instance or collection you need.

GET /emails/345?expand=lastSource

200 OK
{
  "href" : "https://api.foo.com/v1/emails/345",
  "email" : "x@y.com",
  ...
  "lastSource" : {
      "href" : "https://api.foo.com/v1/sources/555",
      "name" : "Sofia",
      "description" : "Sofia is capital city of Bulgaria.",
      ...
      "type" : {
         "href" : "https://api.foo.com/v1/types/555"         
      }
  }
}

In this approach you can specify how deep can the expansion go. For example:

GET /emails/345?expand=lastSource(type)

200 OK
{
  "href" : "https://api.foo.com/v1/emails/345",
  "email" : "x@y.com",
  ...
  "lastSource" : {
      "href" : "https://api.foo.com/v1/sources/555",
      "name" : "Sofia",
      "description" : "Sofia is capital city of Bulgaria.",
      ...
      "type" : {
         "href" : "https://api.foo.com/v1/types/555",
         "name" : "Default Type"
      }
  }
}

 

Partial Representation
Depending of the client visualization you may not need to deliver the whole resource data by limiting the representation.

GET /emails/345?fields=email,lastSource(name)

200 OK
{
  "href" : "https://api.foo.com/v1/emails/345",
  "email" : "x@y.com",
  "lastSource" : {
      "href" : "https://api.foo.com/v1/sources/555",
      "name" : "Sofia"
  }
}

This will give the UI developer a good level of control over the data flow. It is important to remark that in this request fields parameter requesting name from linked object will automatically lead to reference expansion.

Server side paging
Collection resources need server side pagination for limiting the export and avoid timeouts or large responses. Adding parameters to the query string allow you to define the offset and the limit, but it’s important to note some prerequisites:

• All collection resources must have default sorting in order to calculate the offset in the correct way when there is no sorting requested through the query string.
• As I mentioned before the resource must contain all needed information about where and what can the client do with received information. This is why the response shall contain properties like “first”, “previous”, “next” and “last”.
• In some cases you may auto start paging. This is when client makes a request without paging parameters and server is responding with a limited set of instance resources.

Acceptable is to use HREF-templates instead of “first”, “previous”, “next” and “last” or in combination with them.

GET /emails

200 OK
{
  "href" : "https://api.foo.com/v1/emails",
  "offset": 0, 
  "limit": 25, 
  "total" : 2500,
  "first": { 
     "href": "https://api.foo.com/v1/emails?offset=0"
  }, 
  "previous": null,
  "next": { 
     "href": "https://api.foo.com/v1/emails?offset=25"
  }, 
  "last": { 
     "href": "https://api.foo.com/v1/emails?offset=100"
  },
  "template": { 
     "href": "https://api.foo.com/v1/emails?offset={+offset}&limit={+limit}"
  },
  "items": [  
     {
      "href" : "https://api.foo.com/v1/emails/1",
      "email" : "x1@y.com",
      ...
     },
     {
      "href" : "https://api.foo.com/v1/emails/2",
      "email" : "x2@y.com",
      ...
     }
     ...
  ]  
}

 

Many to Many
This type of relationship often causes a lot trouble when you design an API. What is the correct way to show a many to many relationship into an instance resource.

I will explain this with an example.
In my scenario I have emails and sources. Each email can be linked with multiple Sources. And each Source can have multiple emails. The resource responsible for this connection I name emailOrigin. This resource contains link to each of the instance recourse (email and source) plus several other properties (in this case creation date, user responsible, etc.)

Usually REST APIs are exporting an expanded collection of the linked resource skipping the connection resource. Or they export only the connection resource.

In this example this is represented by /edit/123?expand=source but you will not have access the emailOrigin properties.
Or the second scenario is represented by /edit/123?expand=emailOrigin but in this case client have to make a separate request for source resource data.

In my case I decided to export both linked resources – source and emailOrigin. With this approach my clients can decide how to represent the data on their visualization.

GET /emails/345?expand=source,emailOrigin

200 OK
{
  "href" : "https://api.foo.com/v1/emails/345",
  "email" : "x@y.com",
  ...
  "sources" : {
      "href" : "https://api.foo.com/v1/sources?email_id=345",
      "offset" : 0,
      "limit" : 25,
      "total" : 250,
      "first" : { 
          "href" : "https://api.foo.com/v1/sources?email_id=345&offset=0" 
      },
      "previous" : null,
      "next" : { 
          "href" : "https://api.foo.com/v1/sources?email_id=345&offset=25" 
      },
      "last" : { 
          "href" : "https://api.foo.com/v1/sources?email_id=345&offset=10" 
      },
      "template" : { 
          "href" : "https://api.foo.com/v1/sources?email_id=345&offset={+offset}&limit={+limit}" 
      },
      "items" : [
          { 
            "href" : "https://api.foo.com/v1/sources/678",
            "name" : "Plovdiv",
            ...
          },
          ....
      ]
  },
  "emailOrigin" : {
      "href" : "https://api.foo.com/v1/emailorigin?email_id=345",
      "offset" : 0,
      "limit" : 25,
      "total" : 250,
      "first" : { 
          "href" : "https://api.foo.com/v1/emailorigin?email_id=345&offset=0" 
      },
      "previous" : null,
      "next" : { 
          "href" : "https://api.foo.com/v1/emailorigin?email_id=345&offset=25" 
      },
      "last" : { 
          "href" : "https://api.foo.com/v1/emailorigin?email_id=345&offset=10" 
      },
      "template" : { 
          "href" : "https://api.foo.com/v1/emailorigin?email_id=345&offset={+offset}&limit={+limit}" 
      },
      "items" : [
          { 
            "href" : "https://api.foo.com/v1/emailorigin/111",
            "creationDate" : "30-01-2014T10:25:00:000Z",
            "source" : {
                "href" : "https://api.foo.com/v1/sources/678",
                "name" : "Plovdiv",
                ...
            },
            ...
          },
          ....
      ]
  }
}

 

Errors
In case when errors appear you shall be as descriptive as possible and give as much information as possible because REST API users are in the most cases developers.

This is a common error reporting approach used by several companies providing REST APIs.

POST /emails 

409 Conflict 
{ 
   "status": 409, 
   "code": 40924, 
   "property": "email", 
   "message": "An email „x@y.com? already exists.", 
   "developerMessage": "An email „x@y.com? already exists. If you have a stale local cache, please expire it now.", 
   "moreInfo": "https://www.vlaevski.com/docs/api/errors/40924" 
}

Note that status code is same as HTTP status code returned with the response.

In conclusion I just want to remark this post is not including anything about security, caching, concurrency control, redirects, maintenance and browser compatibility.