Extended RESTful URL Design¶

CCEP	7
Title	Extended RESTful URL Design
Version	1
Author	Markus Holtermann
Date	2020-10-07
Status	Accepted

Introduction¶

CCEP-0005 lays the basis for the current CrateDB Cloud API in terms of API design and URL routing. Over the course of the last year, some cases have occurred that are not explicitly covered by said CCEP, which led to different, competing approaches being taken. The goal of this CCEP is to decide on an approach to move forward.

Problem Description¶

A key confusion and inconsistency is in the listing and filtering of resources. As an example, this CCEP uses the listing of cluster resources.

There is an API endpoint GET /api/v2/clusters/ which returns all clusters across all projects a user is a member of. This is the most basic result set in terms of complexity.

However, sometimes it is desired to only show clusters that are within a specific project, assuming the user belongs to that project. For that, there is GET /api/v2/clusters/?project_id=... which uses a query string argument to filter the result set. But, additionally, there is also GET /api/v2/projects/:project_id/clusters/ which does the same. Technically, the two views are using the same implementation. Which results in the confusing scenario where one requests GET /api/v2/projects/p1/clusters/?project_id=p2: does this filter on project ID p1 or p2? The code documents the behavior and explicitly states that it use p1. But that is not obvious to a user.

Furthermore, the list of clusters can currently be filtered by the product name and soon allows to be filtered by the subscription they belong to. Both options are (going to be) implemented using query string filtering but not using sub-resource API endpoints.

More interesting becomes the case of listing users. There are dedicated views for listing all users (GET /api/v2/users/), all users in an organization (GET /api/v2/organizations/:organization_id/users/), or all users in a project (GET /api/v2/projects/:project_id/users/).

The three API endpoints are implemented independently. Which, upon a closer look, doesn’t make too much sense, since they do very similar filtering and permission checking.

Proposal¶

There are essentially three ways forward:

Keep picking the API endpoints “at will” and add or use either query string filtering or sub-resource paths.

Taking this route means there SHOULD NOT be any changes. While this causes the least amount of work, in the long run it leads to more inconsistencies between API endpoints. However, given the somewhat small API surface CrateDB Cloud has at this time, this MAY be fine.
Use query string filtering only.

Gradually all missing filtering options MUST be added to existing API endpoints and all others SHOULD eventually be deprecated and removed. While this creates some additional work in the short term, there is an added benefit: all API endpoints behave the same and automatic query filtering COULD be added, by specifying some Marshmallow schema and mapping that onto a query.

As a user of the API, selecting all clusters MUST become GET /api/v2/clusters/ with the option to apply several filters: for example by project_id, or product_name, or subscription_id. And those filters COULD be combined “at will”.

The implementation for the aforementioned example of listing users might become more complex, but in the end it MUST be a single API endpoint that encapsulates that complexity.
Use sub-resource filtering only.

This approach is somewhat contrary to how RESTful APIs look like. It also makes it almost impossible to add arbitrary filters. What would the API endpoint be that shows all clusters in a specific project that belong to a specific subscription? GET /api/v2/projects/:project_id/subscriptions/:subscription_id/clusters/ or GET /api/v2/subscriptions/:subscription_id/projects/:project_id/clusters/?

It is the author’s believe, that using sub-resource API endpoints causes more confusion over time and limits the flexibility of the API.

All three options provide the currently required feature set, but only approach 2 allows Cloud API to grow without running into barriers and obstacles around the endpoint design and filtering capabilities.

As such, the next section of this CCEP elaborates on the filtering capabilities mentioned in the description of approach 2.

Filtering Result Sets¶

There are three degrees on the amount of filtering the API may support:

Allow filtering on a fixed and pre-defined list of filter criteria:

Each criterion MAY be part of the result object. For example, taking the cluster listing at GET /api/v2/clusters/, a “cluster object” has a project_id but no region attribute. However, listing all clusters in a region, regardless of the project MAY be useful.

Choosing this option means that all filtering would be implemented explicitly and as required.
Allow filtering on all direct attributes.

Each “list” API endpoint SHALL allow filtering on any attribute available on a result object. It’s to be decided and researched how filtering on nested objects would work (such as the dc attribute).
Allow filtering on all direct and nested attributes.

Each “list” API endpoint SHALL allow filtering on any attribute available on a result object, as it would be in option 2. Additionally, filtering SHALL be possible on each attribute of related objects, such that a superuser COULD list all clusters belonging to an organization with a given name. This would follow the relationships between objects: a cluster belongs to a project and a project belongs to an organization. Hence, a cluster belongs to and organization.

Conclusion¶

Balancing the pros and cons of the three approaches, the second one, which uses query string based filtering, seems to be the most reliable one, with the best long-term perspective.

Futhermore, providing explicit filtering options, as suggested in the first one seems more doable and on-point with the current requirements. The filtering part of the CCEP SHALL be revisited when requirements change, that make implementing the second or third option more appropriate.