LiquidFiles Documentation

Attachment (File Upload) API

The most common use of the API is to use it to send files.

Before sending our first file, a little bit a background information is required. When LiquidFiles sends a file, it really sends a message, with files attached to it. Same with a FileLink, it's a FileLink object with a file (attachment) attached to it. The Message or FileLink is what has all the meta data such as expiration and it's when the message expires that the files attached to it gets deleted as well.

Sending large files is difficult. If it was easy, a product such as LiquidFiles wouldn't be needed in the first place. To upload the actual file to the LiquidFiles system, we have three different methods — Binary Uploads, HTML Form Based Uploads or JSON based uploads.

Upload Method Overview

JSON Based Uploads

The typical modern web based API format is JSON, which is a text based format that is also used by LiquidFiles for all LiquidFiles API functions. It's a simple, well documented format that is strictly text based, which means that we have to Base64 encode binary data (which basically stretches 3 bytes of binary data over 4 bytes only using text characters) that is a lot less efficient than either the binary or the HTML based upload methods, but has the benifit of being able to be easily defined in API. If you're sending smaller files (sensitive PDF documents that are fairly small for instance), this is the easiest to use.

Binary Uploads

This is the method that's used by all the builtin LiquidFiles functions. So if you upload a file using the web interface, the Outlook plugin, Mac plugin and so on, it's using the binary upload method. It's basically sending the raw binary data to the LiquidFiles system and is the most efficient way of uploading files to a web server. It is also probably the most complex/least supported in terms of support from external libraries if you want to incorporate this into an existing project.

HTML Form Based Uploads Deprecated

This is the previous method used by the builtin LiquidFiles functions and mimics what happens when someone uses a HTML (multipart/form-data) form element to upload files. This is now considered legacy and you should if possible use the Binary Upload method instead. The HTML Form Based Upload is the method that's been used on the web for uploading files since it was first introduced way back in the 1990s. It's a very well documented way of uploading files, but adds some preamble to the actual data and requires web server plugins to process the file on the server side so is not as efficient as uploading files using the binary upload method.

Please see the Form Based Attachment Upload documenation for further information.

JSON based uploads

JSON is what's being used for all other aspects of the LiquidFiles but when it comes to sending files it poses a bit of a problem. Consider the following example request:

{"message":
  {"attachments": [{
      "filename":"logo.gif",
      "data":?????
    }]
  }
}

Since JSON is a text based protocol, we can't just insert binary data in it in the "data" tag. The standard programatical solution to problems like this is to Base64 encode the binary data into something that can be transmitted over text. This works, but using Base64 encoding has several problems though, some of which include:

  • Aproximately 33% file size increase. We're taking 3 bytes or binary data and spreading over 4 bytes using only the text writable characters. With large files, this leads to a significant increase.
  • It's impossible to do any fancy server side file handling. The entire JSON request will be loaded into the web application. Normally when files are sent using LiquidFiles, the web server (nginx) takes care of the binary file data and the web application only deals with moving files around but the actual file data never passes through the web application. This keeps LiquidFiles fast and efficient. If you sent a 1Gb file using this method, first it would be 1.3Gb transferred (Base64) and this would be loaded (twice - raw Base64 data and decoded) into the memory of LiquidFiles before it could write it to disk. This is very slow and inefficient.
  • With these limitations, the maximum message size (all files combined) you can send with this method is 100 Mb. While this should still be plenty for most applications, if you send files near that size on a regular basis, it's the recommendation to switch to the binary based upload mechanism instead.

When sending files using JSON, you add it directly into the message. Please see the Sending Files API documentation for an example how to send files directly using Base64 encoded file data.

Binary Data Uploads

There's two parts to a HTML POST statement, it's the body of the POST and it's the parameters. In a traditional HTML post, the parameters are hidden within the body of the POST, but there's no reason they have to. With the LiquidFiles binary upload method, we're actually using GET style parameters to pass additional information. It looks like this:

POST https://liquidfiles.company.com/attachments/binary_upload?filename=somefile.ext
RAW BINARY FILEDATA
Request Info
Info Value
Request URL /attachments/binary_upload
Request VERB POST
Request Parameters
Parameter Type Description
filename String The filename of the uploaded file.
filename String (Optional) Content-Type of the uploaded file. Will detect using the Linux `file` command if not provided.
pool_id Integer (optional) If this file is for a pool.
chunks Integer (optional) When uploading files in chunks (in pieces), these are the total number of pieces there is.
chunk Integer optional and required when chunks is set) When uploading files in chunks, this is the current chunk number, with the first chunk being chunk number 0 (zero).

Please see the Attributes documentiation for a description of the Response Parameters.

Example Using curl

The Response has been formatted for readability (added indendation and linebreaks).

curl -X POST --user "ojubiigauS2TxTy4ovk6Ph:x" -H "Accept: application/json" \
  -H "Content-Type: application/json" --data-binary "@someimage.gif" \
  https://liquidfiles.company.com/attachments/binary_upload?filename=someimage.gif

{"attachment":
  {
    "id":"VodoHHSnzG0DltkXihNPSF",
    "filename":"someimage.gif",
    "size":45324,
    "size_human":"44.3 KB"
    "content_type":"image/gif",
    "checksum":"2900b619fa004ba8030195b70e661df1fe6d0480cab3dabc71634cd4679bde4b",
    "crc32":"edac54e4",
    "assembled":true,
    "virus_scan_required":true,
    "virus_scanned":true,
    "virus_scanned_at":"2020-09-02T03:49:21.004Z",
    "content_blocked":false,
    "content_blocked_message":null,
    "actionscript":null,
    "actionscript_scanned_at":null,
    "actionscript_scanned":false,
    "processed":false,
    "processed_at":null,
    "expires_at":null,
    "created_at":"2020-09-02T03:49:21.004Z",
    "available":true
  }
}