stoney core: REST API

REST API

The REST API will be implemented as a first-class citizen
- It provides all the available functions and data to its clients
- Serves as a data and business logic abstraction layer
The REST API will be implemented using HTTPS and REST principles
- Clients are required to validate the certificate (at least via CA)
The REST API uses JSON as the primary data interchange format (serialization of data structures should be abstracted), other formats should be possible in the future.
Authentication via Basic HTTP-Auth
Multiple authentication methods can be added in the future (possibly Web-Server assisted):
- X509 Certificate based authentication
- Kerberos
- API key with shared secret
- Access tokens
- OAuth
versioned API:
- starting with one version number in the URI, for example: https://api.selfcare.com/v1/customer , corresponding to the major version in SemVer
- minor version will be added via Request-Header-Field in future (as-needed)
All API calls need to be fully nonblocking. If an expensive call has to be made to a backend system, the client needs to be provided with a status URI which can be checked for the current status or preferably be notified via WebSockets.
Input validation must be performed for all data (validation of data happens twice: in the API and the client)
- JSON (or XML) validation has to be done before everything else and the client needs to be informed if he passed invalid syntax (see function.json-last-error and function.json-last-error-msg)
Meaningful error message will be presented to the client
All API functions are to be documented using an accepted documentation standard (doxygen (preferred), phpDocumentor or Sami)
The API will be based on existing, proven and tested open source modules and components, coming either from a framework are as stand alone implementations,

Why a REST API?

Separation and abstraction of presentation and business logic
- Faster development/test cycles for business logic
- Smaller development packages
Support for multiple clients with the same code base
- HTML/JS/CSS for selfcare Web GUI
- Command line interface for easy scripting
- Integration into third party provisioning systems for resellers
Automatic testing of functionality
Base for responsive resp. Mobile First Web-Applications/-Design

Yii related API modules

On the Yii PHP Framework Homepage: Extensions tagged with "rest"
- On the Yii PHP Framework Homepage: RestfullYii or on GitHub: RestfullYii
On the Yii PHP Framework Homepage: Extensions tagged with "api"
yii-apiauth

Service implementation details

Base URI

The RESTful web service has to be accessible via a secure HTTP (HTTPS) base URI, for instance https://api.example.com/v1. The definition of the base URI is up to the provider of the service. The only requirements are the use of HTTPS and the presence of the service's version information, so that further changes are possible without breaking existing clients.

Client authentication and authorization

The service needs to authenticate each client via HTTP basic authentication by a user name and a corresponding password. If a unauthenticated client tries to access the service, it will response with a 401 (Unauthorized) HTTP error code.

Furthermore the service must retrieve the authenticated users role and object ownership and respect their respective value when returning collections and elements and acting on HTTP methods. If a client tries to get, modify or delete an element for which it is not authorized, the services will response with a 403 (Forbidden) HTTP error code and includes a descriptive authorization validation message within the JSON error object.

Data interchange format

The service needs to accept and send all data in the JSON data interchange format via HTTP, encoded as UTF-8. Thus a client needs to accept and use the application/json media type. Further media types might be supported in the future.

This results in the following required request and respons headers:

Request header	Response header
`Accept: application/json`	`Content-Type: application/json; charset=UTF-8`

If the client sends an Accept header with an unsupported value (at the moment only application/json is supported), the service will respond with a 406 (Not Acceptable) error code.
If no Accept header is sent, the server will use JSON, possibly pretty-printed and annotated.
If the client sends a Content-Type other than application/json on a POST, PUT or PATCH request, the service will respond with a 415 (Unsupported Media Type) error code.
If the client sends invalid JSON, the service will response with a 400 (Bad Request) HTTP error code and a descriptive error message within the error object.

Future extension: Client may supply application/vnd.org.stoney-cloud.api+json as Content-Type to declare the requested schema/format of the data. This can then also be used to introduce additional versioning.

Error codes and responses

The service returns appropriate HTTP status codes for every request, the following table lists the commonly used codes:

HTTP status code	Text	Description
200	OK	Success.
201	Created	A new resource was successfully created.
400	Bad Request	The request was invalid. A descriptive error message will be sent within the response body.
404	Not Found	The requested resource could not be found but may be available again in the future.
406	Not Acceptable	The requested resource is only capable of generating content not acceptable according to the Accept headers sent in the request.
401	Unauthorized	The client has failed or not yet tried to authenticate.
403	Forbidden	The client is not allowed to access the requested resource.
415	Unsupported Media Type	The request entity has a media type which the server or resource does not support. For example, sending XML instead of JSON in a POST, PUT or PATCH method.
422	Unprocessable Entity	The request was well-formed but was unable to be followed due to semantic errors. For example, a client sends a invalid field value (numbers instead of characters) in a JSON object.
428	Precondition Required	The client did not provide a proper ETag and/or Last-modification header when updating an object via PUT, see http://tools.ietf.org/html/rfc6585#section-3
429	Too Many Requests	There were too many requests within a given time-period, see http://tools.ietf.org/html/rfc6585#section-4
500	Internal Server Error	An internal error succoured. A descriptive error message will be sent within the response body.
503	Service Unavailable	The service is temporary unavailable, because it is overloaded or down for maintenance

Additionally the service returns a descriptive error object in case a HTTP error was returned (4xx) within the message body of the response. An error object consists of an error code and a human readable error message, with further detailed error messages if applicable.

{ 
  "error": { 
    "code": 123,
    "message": "Validation failed"
    "details" : [
      {
        "code" : 5432,
        "field" : "firstName",
        "message" : "First name cannot be longer than 35 characters"
      },
      {
        "code" : 5123,
        "field" : "password",
        "message" : "Password cannot be blank"
      }
    ]
  }
}

Mandatory headers

Besides the above mentioned headers, the following headers are mandatory.

every answer to a GET reqest should always include ETag and Last-Modified header. This allows a proxy to cache requests and a client to revalidate already fetched data.
the service must recognize ETag, Last-Modified and Cache-Control: none provided by the client and act accordingly.
every answer to a GET request must include proper Cache-Control headers
every PUT request to update an object must include the ETag provided by the GET request to fetch the object initially. The API must respond with an 428 (Precondition Required) if the ETag is missing.

Implementation notes:

one could use the internal LDAP attributes entryCSN and/or modifyTimestamp to generate an ETag via a hash function. In the case of business objects where multiple LDAP objects are aggregated for one exposed object, the hash can be generated over all constituent objects
the Last-Modified header can be used to directly limit the search results when hitting the LDAP via the modifyTimestamp internal attribute. Ex. modifytimestamp>=20060301000000Z

Resources and HTTP methods

Resources are always nouns, and specified in plural (such as resellers, customers, users etc.), this prevents one from dealing with irregular pluralizations such as person/people.

The manipulation of resources happens via the HTTP request methods such as GET, POST, PUT, DELETE and PATCH. The following example illustrates the concept with a fictive user resource:

HTTP request	Description
`GET /users`	Retrieves a list of users
`GET /users/12345678`	Retrieves a specific user with user ID 12345678
`POST /users`	Creates a new user
`PUT /users/12345678`	Updates the user with user ID 12345678
`PATCH /users/12345678`	Partly updates the user with user ID 12345678
`DELETE /users/12345678`	Deletes the user with user ID 12345678
`DELETE /users`	Deletes all users

POST

On successful creation of an element, the service must return an URI string to the newly created element.

Example:

Request:

POST /v1/users/ HTTP 1.1
HOST: api.example.com

Accept: application/json
Content-Type: application/json

{
  "usersName": "Mueller",
  "usersType": "reseller",
}

Answer:

HTTP/1.1 201 Created

Content-Type: application/json; charset=UTF-8 
Location: https://api.example.com/v1/users/67890

{
  "id": 67890,
  "location": "https://api.example.com/v1/users/67890",
}

Relations

If a relation can only exist within another resource, it will be represent by its URI, for example: /threads/123/messages/45. This URI represents the message with ID #45 of the forum thread with ID #123.

If a resources can stand by its own, such as users it won't be added as a sub-resource. Relations are always returned as URIs, which the client can hit.

@TBD: Shall there be a functionality to request the embedding of elements, for relations that are commonly requested alongside the resource (to save requests)?

Filtering, sorting and searching

Filter, sort and search requests are added as query parameters to the resource URI.

For filtering the objects returned by a resource URI, the name of an object's attribute is added as a query parameter with the required value.

For example, get all active user elements GET /v1/users?status=active

For sorting the objects returned by a resource URI, the query parameter sort is added with the object's sort attribute(s) as the value.

For example, sort all users by their last and first name GET /v1/users?sort=lastname,firstname

For full text search the objects returned by a resource URI, the query parameter q is added with the value to search for.

For example, GET /v1/users?q=Muell will return users named Mueller as well as the ones living at Muellhaldenstrasse.

Pagination

Responses with multiple items will be limited and paginated to 30 items per default (defined on server-side). Further items can be accessed by appending the page query parameter. The number of items to be returned can be raised to a maximum of 100 (defined on server-side), by specifying the per_page query parameter.

For example, to request page number 3 with 40 items per page, a client would send the following GET request:

GET /v1/users?page=3&per_page=40

If pagination is requested by the client and/or enforced by the server (e.g. if the number of available records is larger than the default count and no pagination requested), the service returns official registered link relation types (next, prev, first, last) within the HTTP Link header field for pagination use:

Link: <https://api.example.com/v1/users?page=1&per_page=40>; rel="first",
  <https://api.example.com/v1/users?page=2&per_page=40>; rel="prev",
  <https://api.example.com/v1/users?page=4&per_page=40>; rel="next",
  <https://api.example.com/v1/users?page=10&per_page=40>; rel="last"

The client MUST use those pagination links, rather than constructing the URLs by itself.

Furthermore the service sets a custom header X-Total-Count containing the (estimation of) total number of records.

Field specifications and limitations

@TBD: Do we want a possibility to specify which fields should be return on a GET request? This could either be used to save further requests to element URIs, if one fetches items from a collection URL, or to reduce the required amount of data to be transferred if one only uses a few attributes from a response. If yes, a fields query parameter should be added to a resource which takes a comma separated list of field names, such as https://api.example.com/v1/users?fields=firstname,lastname.

Maybe. In a collection there shouldn't be as many elements returned such that this may be come a problem. On the other hand, if we return large sets, we should rather use caching properly. Making it possible to specify fields makes cachign even harder. See for example https://blog.apigee.com/detail/restful_api_design_can_your_api_give_developers_just_the_information/ --Tiziano (talk) 11:47, 11 December 2013 (CET)

Input validations

The service validates all input it receives from a client and returns a 422 (Unprocessable Entity) HTTP error code with a descriptive error object.

REST API documentation

There are two types of API calls:

Fully RESTful Ressources and Collections (the default), addressed using "nouns"
Helpers, commands and actions, addressed using "verbs"

With this we are following a pragmatic API design.

Auth action

To give a client the possibility of verifying username and password, a pseudo-ressource is provided, the only method implemented is the GET.

Auth retrieval (GET)

Auth retrieval (GET) example

To verify the authentication the clients sends a HTTP GET request on the auth's resource URI https://api.example.com/v1/auth. The service responds with a HTTP status code:

200 (OK) on success
401 (Unauthorized) on authentication failure
429 (Too Many Requests)

The service must never return 403 or similar to avoid attacks which try to figure out which users exist and which do not.

Request:

GET /v1/auth/ HTTP 1.1
HOST: api.example.com
AUTHORIZATION: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==

Accept: application/json

Answer:

HTTP/1.1 200 OK

Content-Type: application/json; charset=UTF-8

(no content for now)

Search helper

We want to provide an omni-search/ElasticSearch style search function and the most flexible approach is by doing the search completely on the server-side.

Therefore we are gonna copy the leader in search and define the URL for searching this way (see [1]):

https://api.example.com/v1/search?q=fluffy+dragon.

Reseller resource

Resource representing a collection of resellers or a specific reseller element.

Reseller creation (POST)

To create a new reseller the client needs to send a HTTP POST request on the reseller collection resource URI https://api.example.com/v1/resellers, including the associated reseller informations. The service will generate a new reseller and responds with a HTTP status code 201 (Created) on success. The newly created reseller URI is returned within the HTTP location header, which can be used by the client to gather informations about the new reseller.

TBD: Table of all attributes (including optional ones)

Reseller creation (POST) example

Request:

POST /v1/resellers/ HTTP 1.1
HOST: api.example.com

Accept: application/json
Content-Type: application/json

{
  "isCompany": true,
  "billingAddress":
  {
    "organizationName": "Reseller Ltd.",
    "gender": 'm',
    "givenName": "Name",
    "surname": "Surname",
    "postalAddress": "Street Number",
    "countryCode": "CH",
    "postalCode": "Postal Code",
    "localityName": "Locality",
    "preferredLanguage": "en-GB",
    "mail": "name.surname@example.com",
    "telephoneNumber": "+41 00 000 00 00",
    "mobileTelephoneNumber": "+41 00 000 00 00",
    "websiteURL": "https://www.example.com/"
  }
}

Answer:

HTTP/1.1 201 Created

Content-Type: application/json; charset=UTF-8 
Location: https://api.example.com/v1/resellers/4000001

{
  "id": 4000001,
  "location": "https://api.example.com/v1/resellers/4000001",
}

Reseller retrieval (GET)

Reseller collection retrieval (GET) example

To retrieve existing resellers, the client needs to send a HTTP GET request on the reseller's collection resource URI https://api.example.com/v1/resellers. The service responds with a HTTP status code 200 (OK) on success and returns the various resellers.

Request:

GET /v1/resellers/ HTTP 1.1
HOST: api.example.com

Accept: application/json

Answer:

HTTP/1.1 200 OK

Content-Type: application/json; charset=UTF-8

[
  {
    "id": 4000000,
    "location": "https://api.example.com/v1/resellers/4000000",
    "isCompany": true,
    "descriptiveName": "stepping stone GmbH"
  },
  {
    "id": 4000001,
    "location": "https://api.example.com/v1/resellers/4000001",
    "isCompany": true,
    "descriptiveName": "Company Name or Givenname Surname"
  }
]

Reseller element retrieval (GET) example

To retrieve an existing reseller and fetch the informations associated with it, the client needs to send a HTTP GET request on the reseller's element resource URI (such as https://api.example.com/v1/resellers/4000001. The service responds with a HTTP status code 200 (OK) on success and returns the associated reseller informations.

Request:

GET /v1/resellers/4000001 HTTP 1.1
HOST: api.example.com

Accept: application/json

Answer:

HTTP/1.1 200 OK

Content-Type: application/json; charset=UTF-8

{
  "id": 4000001,
  "isCompany": true,
  "billingAddress":
  {
    "organizationName": "Reseller Ltd.",
    "gender": 'm',
    "givenName": "Name",
    "surname": "Surname",
    "postalAddress": "Street Number",
    "countryCode": "CH",
    "postalCode": "Postal Code",
    "localityName": "Locality",
    "preferredLanguage": "en-GB",
    "mail": "name.surname@example.com",
    "telephoneNumber": "+41 00 000 00 00",
    "mobileTelephoneNumber": "+41 00 000 00 00",
    "websiteURL": "https://www.example.com/"
  }
  "shippingAddresses":
  [
    TBD,
  ]
}

Reseller update (PUT)

To updates an existing reseller, the client needs to send a HTTP PUT request on the reseller's element resource URI (such as, https://api.example.com/v1/resellers/4000001). The service responds with a HTTP status code 200 (OK) on success and returns an empty body.

The PUT method requires one to sent the complete record, thus the work-flow is normally as follows:

A GET request on the Reseller element URI will be made to fetch the whole document.
Update the fields which content has changed
Send a PUT request with all the reseller data

Request:

PUT /v1/resellers/4000001 HTTP 1.1
HOST: api.example.com

Accept: application/json
Content-Type: application/json

{
  "id": 4000001,
  "isCompany": true,
  "billingAddress":
  {
    "organizationName": "Reseller Ltd.",
    "gender": 'm',
    "givenName": "Name",
    "surname": "Surname",
    "postalAddress": "Street Number",
    "countryCode": "CH",
    "postalCode": "Postal Code",
    "localityName": "Locality",
    "preferredLanguage": "en-GB",
    "mail": "name.surname@example.com",
    "telephoneNumber": "+41 00 000 00 00",
    "mobileTelephoneNumber": "+41 00 000 00 00",
    "websiteURL": "https://www.example.com/"
  }
  "shippingAddresses":
  [
    TBD,
  ]
}

Answer:

HTTP/1.1 200 OK

Content-Type: application/json; charset=UTF-8

Reseller partly update (PATCH)

To update fields of an existing reseller, the client needs to send a HTTP PATCH request on the reseller's element resource URI (such as, https://api.example.com/v1/resellers/4000001). The service responds with a HTTP status code 200 (OK) on success and returns an empty body.

In contrast to the PUT method, only the changed fields are to be included in the request.

Request:

PATCH /v1/resellers/4000001 HTTP 1.1
HOST: api.example.com

Accept: application/json
Content-Type: application/json

{
  "billingAddress":
  {
    "postalAddress": "New Street Number",
    "preferredLanguage": "de-CH"
  }
}

Answer:

HTTP/1.1 200 OK

Content-Type: application/json; charset=UTF-8

Reseller deletion (DELETE)

To delete an existing reseller, the client needs to send a HTTP DELETE request on the reseller's element resource URI (such as, https://api.example.com/v1/resellers/4000001). The service responds with a HTTP status code 200 (OK) on success and returns an empty body.

Request:

DELETE /v1/resellers/4000001 HTTP 1.1
HOST: api.example.com

Accept: application/json

Answer:

HTTP/1.1 200 OK

Content-Type: application/json; charset=UTF-8

stoney core: REST API

Contents

REST API

Yii related API modules

Service implementation details

Base URI

Client authentication and authorization

Data interchange format

Error codes and responses

Mandatory headers

Resources and HTTP methods

POST

Relations

Filtering, sorting and searching

Pagination

Field specifications and limitations

Input validations

REST API documentation

Auth action

Auth retrieval (GET)

Auth retrieval (GET) example

Search helper

Reseller resource

Reseller creation (POST)

Reseller creation (POST) example

Reseller retrieval (GET)

Reseller collection retrieval (GET) example

Reseller element retrieval (GET) example

Reseller update (PUT)

Reseller partly update (PATCH)

Reseller deletion (DELETE)

Navigation menu

Search