- 0.61.0 (latest)
- 0.60.0
- 0.59.0
- 0.58.0
- 0.57.0
- 0.56.0
- 0.55.0
- 0.54.0
- 0.53.0
- 0.52.0
- 0.51.0
- 0.50.0
- 0.49.0
- 0.48.0
- 0.47.0
- 0.46.0
- 0.45.0
- 0.44.0
- 0.43.0
- 0.42.0
- 0.41.0
- 0.40.0
- 0.39.0
- 0.38.0
- 0.37.0
- 0.36.0
- 0.35.0
- 0.34.0
- 0.33.0
- 0.32.0
- 0.31.0
- 0.30.0
- 0.29.0
- 0.28.0
- 0.27.0
- 0.26.0
- 0.25.0
- 0.24.0
- 0.23.0
- 0.22.0
- 0.21.0
- 0.20.0
- 0.19.0
- 0.18.0
- 0.17.0
- 0.16.0
- 0.15.0
- 0.14.0
- 0.13.0
- 0.12.0
- 0.11.0
- 0.10.0
- 0.9.1
- 0.8.0
- 0.7.0
- 0.6.0
- 0.5.0
- 0.4.0
- 0.3.0
- 0.2.0
- 0.1.0
Specifies Vertex AI owned input data to be used for training, and possibly evaluating, the Model.
Inherits
- Object
Extended By
- Google::Protobuf::MessageExts::ClassMethods
Includes
- Google::Protobuf::MessageExts
Methods
#annotation_schema_uri
def annotation_schema_uri() -> ::String-
(::String) — Applicable only to custom training with Datasets that have DataItems and
Annotations.
Cloud Storage URI that points to a YAML file describing the annotation schema. The schema is defined as an OpenAPI 3.0.2 Schema Object. The schema files that can be used here are found in gs://google-cloud-aiplatform/schema/dataset/annotation/ , note that the chosen schema must be consistent with metadata of the Dataset specified by dataset_id.
Only Annotations that both match this schema and belong to DataItems not ignored by the split method are used in respectively training, validation or test role, depending on the role of the DataItem they are on.
When used in conjunction with annotations_filter, the Annotations used for training are filtered by both annotations_filter and annotation_schema_uri.
#annotation_schema_uri=
def annotation_schema_uri=(value) -> ::String-
value (::String) — Applicable only to custom training with Datasets that have DataItems and
Annotations.
Cloud Storage URI that points to a YAML file describing the annotation schema. The schema is defined as an OpenAPI 3.0.2 Schema Object. The schema files that can be used here are found in gs://google-cloud-aiplatform/schema/dataset/annotation/ , note that the chosen schema must be consistent with metadata of the Dataset specified by dataset_id.
Only Annotations that both match this schema and belong to DataItems not ignored by the split method are used in respectively training, validation or test role, depending on the role of the DataItem they are on.
When used in conjunction with annotations_filter, the Annotations used for training are filtered by both annotations_filter and annotation_schema_uri.
-
(::String) — Applicable only to custom training with Datasets that have DataItems and
Annotations.
Cloud Storage URI that points to a YAML file describing the annotation schema. The schema is defined as an OpenAPI 3.0.2 Schema Object. The schema files that can be used here are found in gs://google-cloud-aiplatform/schema/dataset/annotation/ , note that the chosen schema must be consistent with metadata of the Dataset specified by dataset_id.
Only Annotations that both match this schema and belong to DataItems not ignored by the split method are used in respectively training, validation or test role, depending on the role of the DataItem they are on.
When used in conjunction with annotations_filter, the Annotations used for training are filtered by both annotations_filter and annotation_schema_uri.
#annotations_filter
def annotations_filter() -> ::String-
(::String) — Applicable only to Datasets that have DataItems and Annotations.
A filter on Annotations of the Dataset. Only Annotations that both match this filter and belong to DataItems not ignored by the split method are used in respectively training, validation or test role, depending on the role of the DataItem they are on (for the auto-assigned that role is decided by Vertex AI). A filter with same syntax as the one used in ListAnnotations may be used, but note here it filters across all Annotations of the Dataset, and not just within a single DataItem.
#annotations_filter=
def annotations_filter=(value) -> ::String-
value (::String) — Applicable only to Datasets that have DataItems and Annotations.
A filter on Annotations of the Dataset. Only Annotations that both match this filter and belong to DataItems not ignored by the split method are used in respectively training, validation or test role, depending on the role of the DataItem they are on (for the auto-assigned that role is decided by Vertex AI). A filter with same syntax as the one used in ListAnnotations may be used, but note here it filters across all Annotations of the Dataset, and not just within a single DataItem.
-
(::String) — Applicable only to Datasets that have DataItems and Annotations.
A filter on Annotations of the Dataset. Only Annotations that both match this filter and belong to DataItems not ignored by the split method are used in respectively training, validation or test role, depending on the role of the DataItem they are on (for the auto-assigned that role is decided by Vertex AI). A filter with same syntax as the one used in ListAnnotations may be used, but note here it filters across all Annotations of the Dataset, and not just within a single DataItem.
#bigquery_destination
def bigquery_destination() -> ::Google::Cloud::AIPlatform::V1::BigQueryDestination-
(::Google::Cloud::AIPlatform::V1::BigQueryDestination) —
Only applicable to custom training with tabular Dataset with BigQuery source.
The BigQuery project location where the training data is to be written to. In the given project a new dataset is created with name
dataset_<dataset-id>_<annotation-type>_<timestamp-of-training-call>where timestamp is in YYYY_MM_DDThh_mm_ss_sssZ format. All training input data is written into that dataset. In the dataset three tables are created,training,validationandtest.- AIP_DATA_FORMAT = "bigquery".
AIP_TRAINING_DATA_URI = "bigquery_destination.dataset_
AIP_VALIDATION_DATA_URI = "bigquery_destination.dataset_
AIP_TEST_DATA_URI = "bigquery_destination.dataset_
#bigquery_destination=
def bigquery_destination=(value) -> ::Google::Cloud::AIPlatform::V1::BigQueryDestination-
value (::Google::Cloud::AIPlatform::V1::BigQueryDestination) —
Only applicable to custom training with tabular Dataset with BigQuery source.
The BigQuery project location where the training data is to be written to. In the given project a new dataset is created with name
dataset_<dataset-id>_<annotation-type>_<timestamp-of-training-call>where timestamp is in YYYY_MM_DDThh_mm_ss_sssZ format. All training input data is written into that dataset. In the dataset three tables are created,training,validationandtest.- AIP_DATA_FORMAT = "bigquery".
AIP_TRAINING_DATA_URI = "bigquery_destination.dataset_
AIP_VALIDATION_DATA_URI = "bigquery_destination.dataset_
AIP_TEST_DATA_URI = "bigquery_destination.dataset_
-
(::Google::Cloud::AIPlatform::V1::BigQueryDestination) —
Only applicable to custom training with tabular Dataset with BigQuery source.
The BigQuery project location where the training data is to be written to. In the given project a new dataset is created with name
dataset_<dataset-id>_<annotation-type>_<timestamp-of-training-call>where timestamp is in YYYY_MM_DDThh_mm_ss_sssZ format. All training input data is written into that dataset. In the dataset three tables are created,training,validationandtest.- AIP_DATA_FORMAT = "bigquery".
AIP_TRAINING_DATA_URI = "bigquery_destination.dataset_
AIP_VALIDATION_DATA_URI = "bigquery_destination.dataset_
AIP_TEST_DATA_URI = "bigquery_destination.dataset_
#dataset_id
def dataset_id() -> ::String- (::String) — Required. The ID of the Dataset in the same Project and Location which data will be used to train the Model. The Dataset must use schema compatible with Model being trained, and what is compatible should be described in the used TrainingPipeline's [training_task_definition] [google.cloud.aiplatform.v1.TrainingPipeline.training_task_definition]. For tabular Datasets, all their data is exported to training, to pick and choose from.
#dataset_id=
def dataset_id=(value) -> ::String- value (::String) — Required. The ID of the Dataset in the same Project and Location which data will be used to train the Model. The Dataset must use schema compatible with Model being trained, and what is compatible should be described in the used TrainingPipeline's [training_task_definition] [google.cloud.aiplatform.v1.TrainingPipeline.training_task_definition]. For tabular Datasets, all their data is exported to training, to pick and choose from.
- (::String) — Required. The ID of the Dataset in the same Project and Location which data will be used to train the Model. The Dataset must use schema compatible with Model being trained, and what is compatible should be described in the used TrainingPipeline's [training_task_definition] [google.cloud.aiplatform.v1.TrainingPipeline.training_task_definition]. For tabular Datasets, all their data is exported to training, to pick and choose from.
#filter_split
def filter_split() -> ::Google::Cloud::AIPlatform::V1::FilterSplit- (::Google::Cloud::AIPlatform::V1::FilterSplit) — Split based on the provided filters for each set.
#filter_split=
def filter_split=(value) -> ::Google::Cloud::AIPlatform::V1::FilterSplit- value (::Google::Cloud::AIPlatform::V1::FilterSplit) — Split based on the provided filters for each set.
- (::Google::Cloud::AIPlatform::V1::FilterSplit) — Split based on the provided filters for each set.
#fraction_split
def fraction_split() -> ::Google::Cloud::AIPlatform::V1::FractionSplit- (::Google::Cloud::AIPlatform::V1::FractionSplit) — Split based on fractions defining the size of each set.
#fraction_split=
def fraction_split=(value) -> ::Google::Cloud::AIPlatform::V1::FractionSplit- value (::Google::Cloud::AIPlatform::V1::FractionSplit) — Split based on fractions defining the size of each set.
- (::Google::Cloud::AIPlatform::V1::FractionSplit) — Split based on fractions defining the size of each set.
#gcs_destination
def gcs_destination() -> ::Google::Cloud::AIPlatform::V1::GcsDestination-
(::Google::Cloud::AIPlatform::V1::GcsDestination) —
The Cloud Storage location where the training data is to be written to. In the given directory a new directory is created with name:
dataset-<dataset-id>-<annotation-type>-<timestamp-of-training-call>where timestamp is in YYYY-MM-DDThh:mm:ss.sssZ ISO-8601 format. All training input data is written into that directory.The Vertex AI environment variables representing Cloud Storage data URIs are represented in the Cloud Storage wildcard format to support sharded data. e.g.: "gs://.../training-*.jsonl"
- AIP_DATA_FORMAT = "jsonl" for non-tabular data, "csv" for tabular data
AIP_TRAINING_DATA_URI = "gcs_destination/dataset-
AIP_VALIDATION_DATA_URI = "gcs_destination/dataset-
AIP_TEST_DATA_URI = "gcs_destination/dataset-
#gcs_destination=
def gcs_destination=(value) -> ::Google::Cloud::AIPlatform::V1::GcsDestination-
value (::Google::Cloud::AIPlatform::V1::GcsDestination) —
The Cloud Storage location where the training data is to be written to. In the given directory a new directory is created with name:
dataset-<dataset-id>-<annotation-type>-<timestamp-of-training-call>where timestamp is in YYYY-MM-DDThh:mm:ss.sssZ ISO-8601 format. All training input data is written into that directory.The Vertex AI environment variables representing Cloud Storage data URIs are represented in the Cloud Storage wildcard format to support sharded data. e.g.: "gs://.../training-*.jsonl"
- AIP_DATA_FORMAT = "jsonl" for non-tabular data, "csv" for tabular data
AIP_TRAINING_DATA_URI = "gcs_destination/dataset-
AIP_VALIDATION_DATA_URI = "gcs_destination/dataset-
AIP_TEST_DATA_URI = "gcs_destination/dataset-
-
(::Google::Cloud::AIPlatform::V1::GcsDestination) —
The Cloud Storage location where the training data is to be written to. In the given directory a new directory is created with name:
dataset-<dataset-id>-<annotation-type>-<timestamp-of-training-call>where timestamp is in YYYY-MM-DDThh:mm:ss.sssZ ISO-8601 format. All training input data is written into that directory.The Vertex AI environment variables representing Cloud Storage data URIs are represented in the Cloud Storage wildcard format to support sharded data. e.g.: "gs://.../training-*.jsonl"
- AIP_DATA_FORMAT = "jsonl" for non-tabular data, "csv" for tabular data
AIP_TRAINING_DATA_URI = "gcs_destination/dataset-
AIP_VALIDATION_DATA_URI = "gcs_destination/dataset-
AIP_TEST_DATA_URI = "gcs_destination/dataset-
#predefined_split
def predefined_split() -> ::Google::Cloud::AIPlatform::V1::PredefinedSplit-
(::Google::Cloud::AIPlatform::V1::PredefinedSplit) — Supported only for tabular Datasets.
Split based on a predefined key.
#predefined_split=
def predefined_split=(value) -> ::Google::Cloud::AIPlatform::V1::PredefinedSplit-
value (::Google::Cloud::AIPlatform::V1::PredefinedSplit) — Supported only for tabular Datasets.
Split based on a predefined key.
-
(::Google::Cloud::AIPlatform::V1::PredefinedSplit) — Supported only for tabular Datasets.
Split based on a predefined key.
#stratified_split
def stratified_split() -> ::Google::Cloud::AIPlatform::V1::StratifiedSplit-
(::Google::Cloud::AIPlatform::V1::StratifiedSplit) — Supported only for tabular Datasets.
Split based on the distribution of the specified column.
#stratified_split=
def stratified_split=(value) -> ::Google::Cloud::AIPlatform::V1::StratifiedSplit-
value (::Google::Cloud::AIPlatform::V1::StratifiedSplit) — Supported only for tabular Datasets.
Split based on the distribution of the specified column.
-
(::Google::Cloud::AIPlatform::V1::StratifiedSplit) — Supported only for tabular Datasets.
Split based on the distribution of the specified column.
#timestamp_split
def timestamp_split() -> ::Google::Cloud::AIPlatform::V1::TimestampSplit-
(::Google::Cloud::AIPlatform::V1::TimestampSplit) — Supported only for tabular Datasets.
Split based on the timestamp of the input data pieces.
#timestamp_split=
def timestamp_split=(value) -> ::Google::Cloud::AIPlatform::V1::TimestampSplit-
value (::Google::Cloud::AIPlatform::V1::TimestampSplit) — Supported only for tabular Datasets.
Split based on the timestamp of the input data pieces.
-
(::Google::Cloud::AIPlatform::V1::TimestampSplit) — Supported only for tabular Datasets.
Split based on the timestamp of the input data pieces.