Storage Transfer Service V1 API - Class Google::Cloud::StorageTransfer::V1::HttpData (v0.2.0)

Reference documentation and code samples for the Storage Transfer Service V1 API class Google::Cloud::StorageTransfer::V1::HttpData.

An HttpData resource specifies a list of objects on the web to be transferred over HTTP. The information of the objects to be transferred is contained in a file referenced by a URL. The first line in the file must be "TsvHttpData-1.0", which specifies the format of the file. Subsequent lines specify the information of the list of objects, one object per list entry. Each entry has the following tab-delimited fields:

• HTTP URL — The location of the object.

• Length — The size of the object in bytes.

• MD5 — The base64-encoded MD5 hash of the object.

For an example of a valid TSV file, see Transferring data from URLs.

When transferring data based on a URL list, keep the following in mind:

• When an object located at http(s)://hostname:port/<URL-path> is transferred to a data sink, the name of the object at the data sink is <hostname>/<URL-path>.

• If the specified size of an object does not match the actual size of the object fetched, the object is not transferred.

• If the specified MD5 does not match the MD5 computed from the transferred bytes, the object transfer fails.

• Ensure that each URL you specify is publicly accessible. For example, in Cloud Storage you can share an object publicly and get a link to it.

• Storage Transfer Service obeys robots.txt rules and requires the source HTTP server to support Range requests and to return a Content-Length header in each response.

• ObjectConditions have no effect when filtering objects to transfer.