Known limitations

This page documents known limitations of Transfer Appliance. If you anticipate ordering many appliances for a large transfer and cannot follow the limitations listed below, you must perform an initial proof-of-concept transfer with a single Transfer Appliance and representative data to ensure performance is suitable before you order multiple appliances.

Maximum file size

Cloud Storage supports a maximum single-object size of 5 tebibytes. Transfers from Transfer Appliance to Cloud Storage fail for objects larger than 5 TiB.

Maximum files per directory

Cloud Storage supports a maximum of 500,000 files or directories in a single directory to avoid poor ingest performance.

Maximum path length

The maximum object name length for data copied to Cloud Storage is 1024 bytes. The object prefix for the destination object is included in the length limitation because the object prefix is incorporated in the object's name in Cloud Storage.

Minimum file size

The minimum file size for files copied onto the appliance is 1MB. To copy many smaller files, we recommend that you archive the files before copying them.

Supported file metadata

Transfer Appliance preserves symlinks, numeric MODE and the last modified time (mtime) from Transfer Appliance file system to copy into the corresponding Cloud Storage destination object. Other file metadata are not preserved in the transfer.

Supported file names

File names should be Unicode-compatible and not contain newlines. If your source directory contains file names with newlines, the copy from the appliance to Cloud Storage for that directory fails.

We strongly recommend that you replace newlines in your file names before copying data to Transfer Appliance.

For more information, see Cloud Storage's requirements for object naming.

Supported file types

Transfer Appliance supports transferring regular files. Non-regular files, such as devices, named pipes, or sockets are not supported.

Transfer Appliance does not create empty directories in Cloud Storage when we transfer from the appliance to your Cloud Storage bucket, because objects don't reside within subdirectories within a bucket. For more information, see Object namespace.

Object Versioning

Using Transfer Appliance with a bucket where object versioning is enabled may result in multiple versions of the same object being created and storage costs associated with these versions. We recommend that you disable object versioning for the bucket you specified as the destination at the time of ordering your Transfer Appliance.

To learn more, see Object versioning. Follow these instructions to disable object versioning for your bucket.