Method: projects.locations.dataStores.branches.documents.import

Bulk import of multiple Documents. Request processing may be synchronous. Non-existing items are created.

Note: It is possible for a subset of the Documents to be successfully updated.

HTTP request

POST https://discoveryengine.googleapis.com/v1/{parent=projects/*/locations/*/dataStores/*/branches/*}/documents:import

The URL uses gRPC Transcoding syntax.

Path parameters

Parameters
parent

string

Required. The parent branch resource name, such as projects/{project}/locations/{location}/collections/{collection}/dataStores/{dataStore}/branches/{branch}. Requires create/update permission.

Request body

The request body contains data with the following structure:

JSON representation 
{
  "errorConfig": {
    object (ImportErrorConfig)
  },
  "reconciliationMode": enum (ReconciliationMode),
  "updateMask": string,
  "autoGenerateIds": boolean,
  "idField": string,
  "forceRefreshContent": boolean,

  // Union field source can be only one of the following:
  "inlineSource": {
    object (InlineSource)
  },
  "gcsSource": {
    object (GcsSource)
  },
  "bigquerySource": {
    object (BigQuerySource)
  },
  "fhirStoreSource": {
    object (FhirStoreSource)
  },
  "spannerSource": {
    object (SpannerSource)
  },
  "cloudSqlSource": {
    object (CloudSqlSource)
  },
  "firestoreSource": {
    object (FirestoreSource)
  },
  "alloyDbSource": {
    object (AlloyDbSource)
  },
  "bigtableSource": {
    object (BigtableSource)
  }
  // End of list of possible types for union field source.
}
Fields
errorConfig

object (ImportErrorConfig)

The desired location of errors incurred during the Import.
reconciliationMode

enum (ReconciliationMode)

The mode of reconciliation between existing documents and the documents to be imported. Defaults to ReconciliationMode.INCREMENTAL.
updateMask

string (FieldMask format)

Indicates which fields in the provided imported documents to update. If not set, the default is to update all fields.

This is a comma-separated list of fully qualified names of fields. Example: "user.displayName,photo".
autoGenerateIds

boolean

Whether to automatically generate IDs for the documents if absent.

If set to true, Document.ids are automatically generated based on the hash of the payload, where IDs may not be consistent during multiple imports. In which case ReconciliationMode.FULL is highly recommended to avoid duplicate contents. If unset or set to false, Document.ids have to be specified using idField, otherwise, documents without IDs fail to be imported.

Supported data sources:
idField

string

The field indicates the ID field or column to be used as unique IDs of the documents.

For GcsSource it is the key of the JSON field. For instance, my_id for JSON {"my_id": "some_uuid"}. For others, it may be the column name of the table where the unique ids are stored.

The values of the JSON field or the table column are used as the Document.ids. The JSON field or the table column must be of string type, and the values must be set as valid strings conform to RFC-1034 with 1-63 characters. Otherwise, documents without valid IDs fail to be imported.

Only set this field when autoGenerateIds is unset or set as false. Otherwise, an INVALID_ARGUMENT error is thrown.

If it is unset, a default value Id is used when importing from the allowed data sources.

Supported data sources:
forceRefreshContent

boolean

Optional. Whether to force refresh the unstructured content of the documents.

If set to true, the content part of the documents will be refreshed regardless of the update status of the referencing content.
Union field source. Required. The source of the input. source can be only one of the following:
inlineSource

object (InlineSource)

The Inline source for the input content for documents.
gcsSource

object (GcsSource)

Cloud Storage location for the input content.
bigquerySource

object (BigQuerySource)

BigQuery input source.
fhirStoreSource

object (FhirStoreSource)

FhirStore input source.
spannerSource

object (SpannerSource)

Spanner input source.
cloudSqlSource

object (CloudSqlSource)

Cloud SQL input source.
firestoreSource

object (FirestoreSource)

Firestore input source.
alloyDbSource

object (AlloyDbSource)

AlloyDB input source.
bigtableSource

object (BigtableSource)

Cloud Bigtable input source.

Response body

If successful, the response body contains an instance of Operation.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.

IAM Permissions

Requires the following IAM permission on the parent resource:

  • discoveryengine.documents.import

For more information, see the IAM documentation.