Table of contents

• Introduction
  • Restful API
  • Bulk Operation
• Examples From the real world
• Flexibility
  • Strict
  • Flexible
• Reuse
  • Same route
  • Different route
• Proposal
  • Bulk Update Example
• References

Introduction

Restful API

This paper will not get involved on how to create a robust restful api. We focus on how to apply bulk operations on a restful api. We based on the straightforward approach about the restful api. We assume that if we want to solely operate on a collection, we will use the following routes:

For instance, if we want to update a user, we can PATCH a request on /user/:id, providing the fields that we want to update as JSON object in the body of the request.

PATCH /user/:id
Content-Type: application/json
{
    "firstName": "my first name"
}

Bulk Operation

The discussion under a bulk operation comes when we have to apply the same functionality in more than one entries.

A trivial approach is to send one request for each entry. Obviously, this is a waste of resources and it is too slow. What we are searching for, is a way to use one request for a bundle of entries.

There are many articles on how to organize a bulk operation. We will approach the bulk architecture from different aspects.

Difference with batch operation

We do not want to confuse with the batch operations. A batch operation is a bundle of different actions and routes in one request. Those routes does not have any coherence. Potentially, each route could be completely different from the others. A bulk request is usually a bundle of entries that we want to apply the same functionality in one request.

Examples From the real world

Facebook

graph-api

Facebook does not support bulk operations. It only uses batch operations.

curl \
    -F ‘access_token=…’ \
    -F ‘batch=[{“method”:”GET”, “relative_url”:”me”},{“method”:”GET”, “relative_url”:”me/friends?limit=50”}]’ \
    https://graph.facebook.com

Jira

create-issues

• Route: Uses additional links, /rest/api/2/issue/bulk.
• Request parameters: The request provides an array of entries.

{
    "issueUpdates": [
        {
            "update": {
                "worklog": ...
            }
        }
    ]
}

Zendesk

new bulk api

• Route: Comma separated ids on url.
• Request parameters: One entry as body parameter that will be applied to the ids.

curl https://{subdomain}.zendesk.com/api/v2/organizations/update_many.json?external_ids=external_1,external_2 \  
-H "Content-Type: application/json" -X PUT\  
-v -u {email_address}:{password} \  
-d '{"organization": {"notes": "Something interesting"\}\}'

Gmail

Mark as unread

• Route: Uses a special parameter to set the bulk action.
• Request parameters: Ids in the body of request as form data.

https://mail.google.com/mail/u/0/?ui=2&ik=3g4f2h34j23&rid=mail%3Amau.4ef1.1.0&at=sdfgasfsdvfsdvfasd&view=up&act=ur&_reqid=2344&pcd=1&mb=0&rt=j&search=inbox

form data:
t:3452524524534
t:3453453453454

Workable

all bulk activities

• Route: A single route that commit all the bulk actions.
• Request parameters: The bulk actions are described by a JSON object.

https://encodegroup.workable.com/backend/api/activities/bulk

{
  "bulk_action": "read",
  "activity_ids": [
    41928042,
    41886730
  ],
  "include_archived": true
}

Gitlab

issues bulk update

• Route: Additional route for the bulk operation.
• Request parameters: Ids are provided via form data.

https://gitlab.encode.local/enorasys-sa/enorasys-sa-webui/issues/bulk_update

update[issuable_ids]:2330,2327
update[add_label_ids][]:41

Flexibility

There are two approaches regarding the bulk flexibility.

Strict

Apply only one action on many ids.
The request payload decreases because we send only the fields to be updated and the ids to be applied.

Flexible

Apply different actions on many ids.
The request payload increases because we send an array of fields to be updated for each id.

Reuse

Either reuse the same route for a single operation (with different parameters) or create a completely different route.

Same route

Reuse the same route that we use for the single operation and separate the singularity with additional parameters. Those parameters could be:
* Array of the objects on the body of the request (For bulk operation), instant of one object (For single operation)
* Parameters on the request header. For instance, use one header argument to define the bulk or the singularity.

Different route

Create one additional route for each single route in order to apply the bulk operation.

Proposal

My proposal is to reuse the same routes for the bulk operations. The only difference in comparison with a single operation is that the url for the route will not contain the /:id suffix. The /:id is useless because the action will be applied on many requests.

The requests will be marked with the X-Action: bulk header. The header provides an additional intuition that the request addresses a bulk operation.

The body of the request will contain an array of objects, in JSON format. Each object in one entry, in the same format with a single action. The entry will contain:

Bulk Update Example

Apply update on many entries, selected by id

PATCH /user
X-Action: bulk
[
    {
        "firstName": "my first name (1)",
        "Id": "id1"
    },
    {
        "firstName": "my first name (2)",
        "Id": "id2"
    }    
]

Apply update on a group of entries

PATCH /user?optionallyFilter1=value1
X-Action: bulk
{
    "firstName": "my first name (1)"
}   

References

• Implementing bulk updates within restful services, http://restlet.com/blog/2015/05/18/implementing-bulk-updates-within-restful-services/
• Facebook multiple-requests, https://developers.facebook.com/docs/graph-api/making-multiple-requests)
• Foursquare multi api, https://developer.foursquare.com/docs/multi/multi
• Zendesk bulk api, https://developer.zendesk.com/blog/from-100-requests-to-1-introducing-our-new-bulk-and-batch-apis
• Jira api section, https://docs.atlassian.com/jira/REST/cloud/#api/2/issue-createIssues
• Gmail, https://mail.google.com
• Workable api, https://workable.readme.io/docs
• Gitlab api, https://docs.gitlab.com/ee/api/README.html
• Taptica api, http://wiki.taptica.com/index.php/Bulk_API
• Stackoverflow discussions
  • http://stackoverflow.com/questions/28638071/how-to-update-a-rest-resource-collection
  • http://stackoverflow.com/a/33768160
  • http://stackoverflow.com/a/1630897