Agility Bulk API Overview

What is the Bulk API tool?

The Agility Bulk API allows you to make a high volume of edits to work items all at one time, through the programmatic use of query writing.

Use Cases

• A very large US company had over 50,000 notifications accumulated on a single admin account.
• Another famous international sneaker company can now use the Bulk API for the automated creation and pre-population of assets that correspond to new employees or making modifications to every Story in an Agility instance that meets some specified criteria whereby using the bulk script like a template.
• Quickly remove 200,000+ Stories created in an Agility instance created to test the SDK performance at a massive International Gaming company.
• A large Health Care company created thousands of test Member assets when they first purchased Agility. They will be able to use the Bulk API to modify one of the attributes of each of their test assets to be used in their production instance.

All of the above scenarios are ideal cases for the power of the Agility Bulk API.

Overview

These samples will help guide the usage of the Agility Bulk API endpoint. The Digiital.ai Rest API is our original and most general-purpose endpoint for querying and manipulating all of Agility's assets. As with any endpoint or software tool in general, its scope of functionality can be robust but could have some limitations. One limitation with Rest API is that it cannot perform bulk updates without and is bound to a single operation at a time.

For example, let's say I have 50,000 messages that I would like to delete. The traditional way to delete these assets would be to create a script that reads the OIDS of 50,000 messages that you would like to delete and process them one at a time. Using the Rest-API or the Agility .Net SDK, because deletion of this magnitude would take approximately 11-13 hours since we are forced to repeat the following chunk of pseudo-code.

OID oidList = getOidsToDelete(queryCriteria);
for each (OID x in oidList){
    deleteOid(x); //Called 50,000 times!!!
}

The complexity of the client code is very linear and network intensive since it requires 50K individual API calls so the average time in our testing has been one deletion per second using the SDK or API. The round trip time latency reduction gives the best case theoretical model would be a reduction of 35-40% time saved per transaction. We were able to witness an average throughput of 0.65 seconds/deletion or approximately 8.5 hours to process the 50K records given favorable network conditions and eliminating any of the code's bookkeeping required for track and report progress in real-time.

Using the Bulk Endpoint, the server is doing all of the work deleting assets closer to the database eliminating multiple calls to the API. This deletion is now a single POST to the bulk endpoint. The best-case 8.5 hour deletion time is now approaching 2 hours.