Update a context cache

You can update when a context cache expires. The default expiration time of a context cache is 60 minutes after its creation time. An expired context cache is deleted during a garbage collection process and can't be used or updated. To update the time when an unexpired context cache expires, update one of its following properties:

  • ttl - The number of seconds and nanoseconds that the cache lives after it's created or after the ttl is updated before it expires. When you set the ttl, the expireTime of the cache is updated.

  • expire_time - A Timestamp that specifies the absolute date and time when the context cache expires.

Update the context cache using its ttl parameter

The following is an example of a curl command that updates its expiration time by 3,600 seconds.

Python

To learn how to install or update the Vertex AI SDK for Python, see Install the Vertex AI SDK for Python. For more information, see the Vertex AI SDK for Python API reference documentation.

Streaming and non-streaming responses

You can choose whether the model generates streaming responses or non-streaming responses. For streaming responses, you receive each response as soon as its output token is generated. For non-streaming responses, you receive all responses after all of the output tokens are generated.

For a streaming response, use the stream parameter in generate_content.

  response = model.generate_content(contents=[...], stream = True)

For a non-streaming response, remove the parameter, or set the parameter to False.

Sample code

import vertexai
from datetime import datetime as dt
from datetime import timezone as tz
from datetime import timedelta

from vertexai.preview import caching

# TODO(developer): Update and un-comment below lines
# PROJECT_ID = "your-project-id"
# cache_id = "your-cache-id"

vertexai.init(project=PROJECT_ID, location="us-central1")

cached_content = caching.CachedContent(cached_content_name=cache_id)

# Option1: Update the context cache using TTL (Time to live)
cached_content.update(ttl=timedelta(hours=3))
cached_content.refresh()

# Option2: Update the context cache using specific time
next_week_utc = dt.now(tz.utc) + timedelta(days=7)
cached_content.update(expire_time=next_week_utc)
cached_content.refresh()

print(cached_content.expire_time)
# Example response:
# 2024-09-11 17:16:45.864520+00:00

Go

Before trying this sample, follow the Go setup instructions in the Vertex AI quickstart. For more information, see the Vertex AI Go SDK for Gemini reference documentation.

To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

Streaming and non-streaming responses

You can choose whether the model generates streaming responses or non-streaming responses. For streaming responses, you receive each response as soon as its output token is generated. For non-streaming responses, you receive all responses after all of the output tokens are generated.

For a streaming response, use the GenerateContentStream method.

  iter := model.GenerateContentStream(ctx, genai.Text("Tell me a story about a lumberjack and his giant ox. Keep it very short."))

For a non-streaming response, use the GenerateContent method.

  resp, err := model.GenerateContent(ctx, genai.Text("What is the average size of a swallow?"))

Sample code

import (
	"context"
	"fmt"
	"io"
	"time"

	"cloud.google.com/go/vertexai/genai"
)

// updateContextCache shows how to update the expiration time of a cached content, by specifying
// a new TTL (time-to-live duration)
// contentName is the ID of the cached content to update
func updateContextCache(w io.Writer, contentName string, projectID, location string) error {
	// location := "us-central1"
	ctx := context.Background()

	client, err := genai.NewClient(ctx, projectID, location)
	if err != nil {
		return fmt.Errorf("unable to create client: %w", err)
	}
	defer client.Close()

	cachedContent, err := client.GetCachedContent(ctx, contentName)
	if err != nil {
		return fmt.Errorf("GetCachedContent: %w", err)
	}

	update := &genai.CachedContentToUpdate{
		Expiration: &genai.ExpireTimeOrTTL{TTL: 2 * time.Hour},
	}

	_, err = client.UpdateCachedContent(ctx, cachedContent, update)
	fmt.Fprintf(w, "Updated cached content %q", contentName)
	return err
}

REST

You can use REST to create a update the context cache by using the Vertex AI API to send a PATCH request to the publisher model endpoint. The following example shows how to update the expiration date using the ttl parameter.

Before using any of the request data, make the following replacements:

  • PROJECT_ID: Your project ID.
  • LOCATION: The region where the request to create the context cache was processed.
  • CACHE_ID: The ID of the context cache. The context cache ID is returned when you create the context cache. You can also find context cache IDs by listing the context caches for a Google Cloud project using. For more information, see create a context cache and list context caches.
  • SECONDS: A float that specifies the seconds component of the duration before the cache expires.
  • NANOSECONDS: A float that specifies the nanoseconds component of the duration before the cache expires.

HTTP method and URL: 

PATCH https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/cachedContents/CACHE_ID

Request JSON body: 

{
  "seconds":"SECONDS",
  "nanos":"NANOSECONDS"
}

To send your request, choose one of these options:

curl

Save the request body in a file named request.json, and execute the following command: 

curl -X PATCH \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/cachedContents/CACHE_ID"

PowerShell

Save the request body in a file named request.json, and execute the following command: 

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method PATCH `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/cachedContents/CACHE_ID" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

Example curl command

PROJECT_NUMBER="PROJECT_NUMBER"
LOCATION="us-central1"
PROJECT_ID="PROJECT_ID"
CACHE_ID="CACHE_ID"

curl \
-X PATCH \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json"\
"https://${LOCATION}-aiplatform.googleapis.com/v1/projects/${PROJECT_NUMBER}/locations/${LOCATION}/cachedContents/${CACHE_ID}" -d \
'{
   "ttl": {"seconds":"3600","nanos":"0"}
}'

Update the context cache using its expire_time parameter

The following is an example of a curl command that uses the expire_time parameter to update its expiration time to 9 AM on June 30, 2024.

REST

You can use REST to create a update the context cache by using the Vertex AI API to send a PATCH request to the publisher model endpoint. The following example shows how to update the expiration date using the expire_time parameter.

Before using any of the request data, make the following replacements:

  • PROJECT_ID: Your project ID.
  • LOCATION: The region where the request to create the context cache was processed.
  • CACHE_ID: The ID of the context cache. You can find the ID in the response when you create the context cache.
  • EXPIRE_TIME: A Timestamp that specifies the time when the context cache expires.

HTTP method and URL: 

PATCH https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/cachedContents/CACHE_ID

Request JSON body: 

{
   "expire_time":"EXPIRE_TIME"
}

To send your request, choose one of these options:

curl

Save the request body in a file named request.json, and execute the following command: 

curl -X PATCH \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/cachedContents/CACHE_ID"

PowerShell

Save the request body in a file named request.json, and execute the following command: 

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method PATCH `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/cachedContents/CACHE_ID" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

Example curl command

PROJECT_NUMBER="PROJECT_NUMBER"
LOCATION="us-central1"
PROJECT_ID="PROJECT_ID"
CACHE_ID="CACHE_ID"

curl \
-X PATCH \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json"\
"https://${LOCATION}-aiplatform.googleapis.com/v1/projects/${PROJECT_NUMBER}/locations/${LOCATION}/cachedContents/${CACHE_ID}" -d \
'{
   "expire_time":"2024-06-30T09:00:00.000000Z"
}'

What's next