LiquidFiles Documentation
LiquidFiles Documentation
Updated v3.7.13

Upload File Shares API

Before uploading our first file, a little bit a background information is required. To upload the actual file to the LiquidFiles system, we have three different methods — Binary Uploads, HTML Form Based Uploads or JSON based uploads.

Upload Method Overview

Binary Uploads

This is the method that's used by all the builtin LiquidFiles functions. So if you upload a file using the web interface, the Outlook plugin, Mac plugin and so on, it's using the binary upload method. It's basically sending the raw binary data to the LiquidFiles system and is the most efficient way of uploading files to a web server. It is also probably the most complex/least supported in terms of support from external libraries if you want to incorporate this into an existing project.

JSON Based Uploads

The typical modern web based API format is JSON, which is a text based format that is also used by LiquidFiles for all LiquidFiles API functions. It's a simple, well documented format that is strictly text based, which means that we have to Base64 encode binary data (which basically stretches 3 bytes of binary data over 4 bytes only using text characters) that is a lot less efficient than either the binary or the HTML based upload methods, but has the benifit of being able to be easily defined in API. If you're sending smaller files (sensitive PDF documents that are fairly small for instance), this is the easiest to use.

Binary Data Uploads

There's two parts to a HTML POST statement, it's the body of the POST and it's the parameters. In a traditional HTML post, the parameters are hidden within the body of the POST, but there's no reason they have to. With the LiquidFiles binary upload method, we're actually using GET style parameters to pass additional information. It looks like this:

POST https://liquidfiles.example.com/shares/someshare/folders/somefolder/files/upload?name=somefile.ext
RAW BINARY FILEDATA
Request Info
Info Value
Request URL /shares/_share_id_/folders/_folder_id_/files/upload
Request VERB POST
Request Parameters
Parameter Type Description
name String The filename of the uploaded file.
chunks Integer (optional) When uploading files in chunks (in pieces), these are the total number of pieces there is.
chunk Integer optional and required when chunks is set) When uploading files in chunks, this is the current chunk number, with the first chunk being chunk number 0 (zero).
Response Parameters
Parameter Type Description
id String The Unique ID for the share.
folder_id String The ID for the parent folder.
name String The Name of the Share.
size Integer The size of the file in Bytes.
size_to_human String The size of the file in human readable format.
content_type String The Content-Type of the file.
checksum String The SHA-256 Checksum of the file.
crc32 String The CRC32 Checksum of the file.
av_scanned Boolean If the file has been AV scanned or not.
av_infected Boolean If the file is AV infected.
deleted Boolean If the file has been deleted or not.
created_at DateTime When the file was created
update_at DateTime When the file was last updated

Example Using curl

The Response has been formatted for readability (added indendation and linebreaks).

curl -X POST --user "DaDMcRpYEFUsxSUjUozMoU:x" \
     -H "Accept: application/json" \
     -H "Content-Type: image/gif" \
     --data-binary "@someimage.gif" \
     https://liquidfiles.example.com/shares/project-1/folders/folder-1/files/upload?name=someimage.gif

{"file":
  {
    "id":"someimage-gif",
    "folder_id":"folder-1",
    "name":"someimage.gif",
    "size":49656,
    "size_to_human":"48 KB",
    "content_type":"image/gif",
    "checksum":"a5f21ee04ff2ff7ac191a76bf5bf432ac32eda5f2599bb2e876568d1376a0984",
    "crc32":"9d378437",
    "av_scanned":false,
    "av_infected":false,
    "deleted":false,
    "created_at":"2021-04-28T22:35:08.983Z",
    "updated_at":"2021-04-28T22:35:08.983Z",
    "processed":false,
    "processed_at":null
  }
}

Checking Individual Chunks before each upload

This is the same method that's described in the Attachments Chunks documentation. Before a chunk is uploaded, we can query the server and if we get a "partial content" response (HTTP reponse code 206) that means that the chunk doesn't exist and we can go ahead and upload it. If the server response with an "ok" message (HTTP response code 200) that means that the chunk already exist and we can move to the next one.

Request Info
Info Value
Request URL /shares/_share_id_/folders/_folder_id_/files/upload
Request VERB GET
Request Parameters
Parameter Type Description
name String The filename of the uploaded file.
chunk Integer The individual chunk you want to test.
chunks Integer The total number of chunks.

Example Request using curl

The following example will check if the first chunk (chunk 0) already exist on the system. If the reponse is a 206 response the chunk doesn't exist, if the response is 200, the chunk already exists.

curl -w "%{http_code}\n" --user "Y9fdTmZdv0THButt5ZONIY:x" \
     -H "Accept: application/json" \
     -H "Content-Type: application/zip" \
     https://liquidfiles.company.com/shares/test-share/folders/test-folder/files/upload?name=bigfile.zip&chunks=10&chunk=0

206

Or a more complete (bash) example how this would actually be used:

if [[ `curl -w "%{http_code}\n" --user "ojubiigauS2TxTy4ovk6Ph:x" \
     -H "Accept: application/json" \
     -H "Content-Type: application/zip" \
     https://liquidfiles.company.com/shares/test-share/folders/test-folder/files/upload?ame=bigfile.zip&chunks=10&chunk=0` -eq 206 ]]; then

  curl -X POST --user "ojubiigauS2TxTy4ovk6Ph:x" \
       -H "Accept: application/json" \
       -H "Content-Type: application/zip" \
       --data-binary "@bigfile.zip.00" \
       https://liquidfiles.company.com/shares/test-share/folders/test-folder/files/uploadd?ame=bigfile.zip&chunks=10&chunk=0
fi

So we first do a GET request for the filename, chunks and chunk to see if the chunk exist and if it doesn't we upload it.

Available Chunks

As an alternative to the above, you can also check all chunks that are available for a given file. Uploaded chunks they are stored for 5 days before being removed (if the file was not uploaded completely). One thing to note is that a chunk (or a file for that matter) is interrupted during transit, the partially uploaded chunk (or file) will be discarded. If you want to resume uploads, you can query the server using this API call to see what chunks are available and upload any missing chunks.

Request

Request Info
Info Value
Request URL /shares/_share_id_/folders/_folder_id_/files/available_chunks
Request VERB GET
Request Parameters
Parameter Type Description
name String The filename of the uploaded file.
Response Parameters
Parameter Type Description
chunks Array An array with the numerical chunk ID and the chunk size.

Example Request using curl

#!/bin/sh

# Some nice variables
api_key="Y9fdTmZdv0THButt5ZONIY"
server="https://liquidfiles.company.com"

curl -X GET --user "$api_key:x" -H "Accept: application/json" \
  $server//shares/project-alpha/folders/root/files/available_chunks?name=Presentation1.pptx

{"chunks":[
  {
    "id":0,
    "size":104857600
  }, {
    "id":1,
    "size":104857600
  }]
}

In this case the first 2 chunks have been uploaded, both with 100Mb size. You can now continue with chunk 3 (chunk id 2, when starting from 0) starting at 200Mb.

JSON based uploads

JSON is what's being used for all other aspects of the LiquidFiles API but when it comes to sending files it poses a bit of a problem. Consider the following example request:

{"file":
  {
    "name":"logo.gif",
    "data":?????
  }
}

Since JSON is a text based protocol, we can't just insert binary data in it in the "data" tag. The standard programatical solution to problems like this is to Base64 encode the binary data into something that can be transmitted over text. This works, but using Base64 encoding has several problems though, some of which include:

  • Aproximately 33% file size increase. We're taking 3 bytes or binary data and spreading over 4 bytes using only the text writable characters. With large files, this leads to a significant increase.
  • It's impossible to do any fancy server side file handling. The entire JSON request will be loaded into the web applications memory. Normally when files are sent using LiquidFiles, the web server (nginx) takes care of the binary file data and the web application only deals with moving files around but the actual file data never passes through the web application. This keeps LiquidFiles fast and efficient. If you where to send a 1Gb file using this method, first it would be 1.3Gb transferred (Base64) and this would be loaded (twice - raw Base64 data and decoded) into the memory of LiquidFiles before it could write it to disk. This is very slow and inefficient.
  • With these limitations, the maximum message size you can upload with this method is 100MB. While this should still be plenty for a lot of applications, if you upload files near that size on a regular basis, it's the recommendation to switch to the form based upload mechanism instead.

Request Info and Parameters for JSON based Uploads

Request Info
Info Value
Request URL /shares/_share_id_/folders/_folder_id_/files
Request VERB POST
Request Parameters
Parameter Type Description
name String The filename of the file you're uploading.
data String The data of the file, encoded in Base64 encoding (max size 100MB).
content_type String (Optional) The Content-Type of the uploaded file. If not present the server will calculate the Content-Type.
checksum String (Optional) The SHA-256 checksum of the uploaded file. If not present the server will calculate the SHA-256 checksum.
crc32 String (Optional) The CRC32 checksum of the uploaded file. If not present the server will calculate the CRC32 checksum.

To upload files using Base64 Encoding, please see the following example using curl and bash:

cat <<EOF | curl -s -X POST \
     -H "Accept: application/json" \
     --user "nkpIxMK9ucUUE7FvfNpdAf:x" -d @- \
     https://liquidfiles.example.com/shares/project-alpha/folders/root/files

{"file":
  {
    "name":"Presentation1.pptx",
    "data":"`base64 /path/to/Presentation1.pptx`"
  }
}     
EOF