Data sync is the process of refreshing or synchronizing the data in a feature view within an online store from the feature data source in BigQuery. To sync the data for the entire online store, you need to sync the data for all of its feature views.
There are two ways in which the data sync occurs for a feature view:
Scheduled data sync: The feature data is automatically refreshed, based on the sync schedule configured for the feature view using the
FeatureView.sync_config
parameter.Manual data sync: You can skip the wait between two scheduled data sync operations by manually initiating the data sync for a feature view.
Data sync might involve costs for BigQuery resource usage. For information about how to optimize costs while setting the sync schedule for a feature view, see Optimize costs during sync.
Before you begin
Authenticate to Vertex AI, unless you've done so already.
To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.
Install the Google Cloud CLI, then initialize it by running the following command:
gcloud init
For more information, see Authenticate for using REST in the Google Cloud authentication documentation.
Manually start a data sync
Use the following sample to manually start the data sync in a feature view. Note that only one data sync operation can be active at any point of time for a feature view. If you try to manually start the data sync while another sync is in progress, then the new sync operation starts only after the ongoing sync operation ends.
REST
To manually start a data sync in a
FeatureView
,
send a POST
request by using the
featureViews.sync
method.
Before using any of the request data, make the following replacements:
- LOCATION_ID: Region where the online store is located, such as
us-central1
. - PROJECT_ID: Your project ID.
- FEATUREONLINESTORE_NAME: The name of the online store containing the feature view.
- FEATUREVIEW_NAME: The name of the feature view where you want to manually start the data sync.
HTTP method and URL:
POST https://LOCATION_ID-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/featureOnlineStores/FEATUREONLINESTORE_NAME/featureViews/FEATUREVIEW_NAME:sync
To send your request, choose one of these options:
curl
Execute the following command:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d "" \
"https://LOCATION_ID-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/featureOnlineStores/FEATUREONLINESTORE_NAME/featureViews/FEATUREVIEW_NAME:sync"
PowerShell
Execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-Uri "https://LOCATION_ID-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/featureOnlineStores/FEATUREONLINESTORE_NAME/featureViews/FEATUREVIEW_NAME:sync" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "featureViewSync": "projects/PROJECT_ID/locations/LOCATION_ID/featureOnlineStores/FEATUREONLINESTORE_NAME/featureViews/FEATUREVIEW_NAME/featureViewSyncs/OPERATION_ID" }
What's next
Learn how to view a list of all data sync operations executed for a feature view.
Learn how to update a feature view