CloudSoda facilitates the transfer of files from one storage to another. This documentation describes the secure public API for accessing a subset of CloudSoda functionality.

The primary use case supported by the API is to allow a client to programmatically track and manage transfers. As such, the API is not designed to be a one-to-one replacement of the CloudSoda web interface.

CloudSoda is continuously deployed and as such the API will change, but all changes will be backward compatible. If a change needs to be made that breaks backward compatibility, a new major version of the API will be released under a different endpoint to expose the new functionality. Ample notice will be given that the existing API is being deprecated and allow for integrated applications to migrate.

However, the current API version can be found by making an authenticated request to the root API URL.

At a high level, CloudSoda deployments allow users to transfer files from one storage to another. File transfers occur by running file transfer jobs. Jobs can be one time transfers, but users also have the ability to schedule future jobs by creating recurring job templates called policies.

Deployments are top-level entities under which all resources are grouped. As a result, various resources, such as jobs, groups, and users, have a deploymentId property. This id uniquely identifies the deployment to which a resource belongs.

Jobs describe the instructions to copy or move files. Storages act as sources and targets for file transfer jobs. Users are provided credentials to sign in and interact with the system via the WebUI.

The CloudSoda API can be accessed by using the .api.cloudsoda.io subdomain. For example, if your URL is https://acme-us-1.cloudsoda.io, your API can be reached at https://acme-us-1.api.cloudsoda.io.

Pagination for all API resources is done in accordance with the JSON:API specification. Details on JSON:API pagination can be found here.

Any third party client must authenticate against the API using the client id and secret. The client credentials authentication flow must only be used in cases where the client application can be completely trusted to securely store the secret, which is effectively a password. If the secret is exposed, new client credentials must be generated because malicious clients could make authorized requests with compromised credentials.

A third party client seeking to integrate against the API can request credentials from the on-site CloudSoda product administrator. Specifically, the client id and the client secret can be generated in the API section of the CloudSoda web interface.

Groups are used to organize CloudSoda users, their actions, and incurred expenses.

Users can use their credentials to interact with CloudSoda resources through the UI.

Two classes of users exist in the system. The primary class of user has login credentials and can interact with the system through the UI. Alternatively, users can be registered without a password, preventing anyone from signing in with that user account. However, the user still exists as a stub, which can be used for tracking purposes. For instance, those accounts might correspond to users interacting with the system through a third party integration rather than directly the web portal.

Jobs are used to transfer files or objects from one storage to another.

Jobs are customizable and offer flexibility in executing transfers. The job object contains information describing a transfer, which specifies which files will be transferred, which storages to transfer files to and from, and how files should be transferred. In addition to information related to the transfer, the job object may contain information about the user who initiated the transfer or the related group/deployment.

Once initiated, a job goes through various stages of preparation and execution through to completion or failure. The job object's status property keeps track of the given stage a job is going through.

Before transferring files, a job begins by looking through a storage and identifying the files to be transferred. During this initial stage, the job's status will be scanning. When a job completes its scanning stage and is transferring files its status will change to running. A job that has been paused will have a status of paused. If a job is interrupted due to resources not being available to execute it, the job's status will be set to stuck. Canceled jobs have their status set to canceled. Once the file transfer is complete, the job's status will be set to finished. If the job fails to complete its status will change to failed.

Jobs can either be executed as one time quick transfers, or scheduled in advance using policies. All jobs can be similarly handled using the API, irrespective of how they were triggered.

It is also possible to have a job execute as a dry run and not move any data. In this case, it simply returns information on what would have been transferred and the cost of doing so.

Files can either be copied to a new storage, or moved over without retaining copies in the original storage. The transferType property specifies the type of transfer a job will execute.

Setting the transfer type property to copy will result in preserving all data on the source storage. A copy of the files will be created on the target storage. If the transfer type is move, files will be moved over to the target, and then removed from the source. Finally, the sync transfer type will synchronize the files between the source, and the target, by making the target mirror the source. Please note that all files on the target that are not on the source will be deleted. This is also true when using rules: all files not matching the rules on the target will be deleted. For example, with a rule set to sync all .txt files from a source to a target, then, after the transfer, previously existing files on the target will be deleted, and only the synced .txt files will be present on the target.

A naming conflict occurs when a target directory contains files of the same name as one or more of the files to be transferred into that directory. With its conflictHandling property, a job must specify how such conflicts will be handled if they occur. Valid values for conflict handling include: skip, rename, overwrite, and null. The conflictHandling property must not be null, unless the transferType is sync. When transferType is sync, conflictHandling must be null.

skip will cause existing files not to be replaced by remote ones with the same name. Such remote files will not be transferred to the target storage. rename will result in both files being present in the target storage after the transfer, but with different names. For example, a file named file.txt, would be saved as file_copy-1.txt. overwrite will cause a previously existing file to be removed and replaced by the remote one if the two have the same name. null must only be used when the transferType property is set to sync. In that case, all data will be synchronized between the source and the target storage, and therefore, the manner in which naming conflicts will be handled has already been specified.

There are some limitations when working with the job object. Please see the limitations enumerated below.

• The request payload cannot exceed 1MB.
• Scheduled jobs (policies) cannot be created, only quick transfers.
• A job created with the "files" scope cannot be cloned in the UI.

Storages store files and act as sources and targets in file transfer jobs. Currently, these resources are read-only.