Get information about a context cache

You can learn about the time a context cache was created, the time it was most recently updated, and the time it expires. To get information about every context cache associated with a Google Cloud project, including their cache IDs, use the command to list context caches. If you know the cache ID of a context cache, you can get information about just that context cache.

Get a list of context caches

To get a list of the context caches associated with a Google Cloud project, you need the region where you created and the ID of your Google Cloud project. The following shows you how to get a list of context caches for a Google Cloud project.

Python

Before trying this sample, follow the Python setup instructions in the Vertex AI quickstart using client libraries. For more information, see the Vertex AI Python API reference documentation.

To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment. 

import vertexai

from vertexai.preview import caching

# TODO(developer): Update & uncomment line below
# PROJECT_ID = "your-project-id"
vertexai.init(project=PROJECT_ID, location="us-central1")

cache_list = caching.CachedContent.list()
# Access individual properties of a CachedContent object
for cached_content in cache_list:
    print(f"Cache '{cached_content.name}' for model '{cached_content.model_name}'")
    print(f"Last updated at: {cached_content.update_time}")
    print(f"Expires at: {cached_content.expire_time}")
    # Example response:
    # Cached content 'example-cache' for model '.../gemini-1.5-pro-001'
    # Last updated at: 2024-09-16T12:41:09.998635Z
    # Expires at: 2024-09-16T13:41:09.989729Z

Go

Before trying this sample, follow the Go setup instructions in the Vertex AI quickstart using client libraries. For more information, see the Vertex AI Go API reference documentation.

To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment. 

import (
	"context"
	"fmt"
	"io"

	"cloud.google.com/go/vertexai/genai"
	"google.golang.org/api/iterator"
)

// listContextCaches retrieves all context caches associated with the specified
// Google Cloud project and region
func listContextCaches(w io.Writer, projectID, location string) error {
	// location := "us-central1"
	ctx := context.Background()

	client, err := genai.NewClient(ctx, projectID, location)
	if err != nil {
		return fmt.Errorf("unable to create client: %w", err)
	}
	defer client.Close()

	cacheList := client.ListCachedContents(ctx)
	// `cacheList` is a standard Google API iterator.
	// See https://pkg.go.dev/google.golang.org/api/iterator#example-package for more details
	for {
		item, err := cacheList.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			return fmt.Errorf("listContextCaches failed: %w", err)
		}

		fmt.Fprintf(w, "Cache %q will expire at %v\n", item.Name, item.Expiration.ExpireTime.String())
		// Example response:
		// Cache "projects/.../locations/.../cachedContents/12345678900000000" will expire at 2024-10-25 09:13:58.67004 +0000 UTC
	}

	return nil
}

REST

The following shows how to use REST to list the context caches associated with a Google Cloud project by sending a GET request to the publisher model endpoint.

Before using any of the request data, make the following replacements:

HTTP method and URL: 

GET https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/cachedContents

To send your request, choose one of these options:

curl

Execute the following command: 

curl -X GET \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     "https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/cachedContents"

PowerShell

Execute the following command: 

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method GET `
    -Headers $headers `
    -Uri "https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/cachedContents" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

Example curl command

LOCATION="us-central1"
PROJECT_ID="PROJECT_ID"

curl \
-X GET \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
https://${LOCATION}-aiplatform.googleapis.com/v1beta1/projects/${PROJECT_ID}/locations/${LOCATION}/cachedContents

Get information about a context cache

To get information about one context cache, you need its cache ID, the Google Cloud project ID with which the context cache is associated, and the region where the request to create the context cache was processed. The cache ID of a context cache is returned when you create the context cache. You can also get the cache ID of each context cache associated with a project using the context cache list command.

The following shows you how to get information about one context cache.

Python

To learn how to install or update the Vertex AI SDK for Python, see Install the Vertex AI SDK for Python. For more information, see the Vertex AI SDK for Python API reference documentation.

Streaming and non-streaming responses

You can choose whether the model generates streaming responses or non-streaming responses. For streaming responses, you receive each response as soon as its output token is generated. For non-streaming responses, you receive all responses after all of the output tokens are generated.

For a streaming response, use the stream parameter in generate_content.

  response = model.generate_content(contents=[...], stream = True)

For a non-streaming response, remove the parameter, or set the parameter to False.

Sample code

import vertexai

from vertexai.preview import caching

# TODO(developer): Update and un-comment below lines
# PROJECT_ID = "your-project-id"
# cache_id = "your-cache-id"

vertexai.init(project=PROJECT_ID, location="us-central1")

cached_content = caching.CachedContent(cached_content_name=cache_id)

print(cached_content.resource_name)
# Example response:
# projects/[PROJECT_ID]/locations/us-central1/cachedContents/1234567890

Go

Before trying this sample, follow the Go setup instructions in the Vertex AI quickstart. For more information, see the Vertex AI Go SDK for Gemini reference documentation.

To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up ADC for a local development environment.

Streaming and non-streaming responses

You can choose whether the model generates streaming responses or non-streaming responses. For streaming responses, you receive each response as soon as its output token is generated. For non-streaming responses, you receive all responses after all of the output tokens are generated.

For a streaming response, use the GenerateContentStream method.

  iter := model.GenerateContentStream(ctx, genai.Text("Tell me a story about a lumberjack and his giant ox. Keep it very short."))

For a non-streaming response, use the GenerateContent method.

  resp, err := model.GenerateContent(ctx, genai.Text("What is the average size of a swallow?"))

Sample code

import (
	"context"
	"fmt"
	"io"

	"cloud.google.com/go/vertexai/genai"
)

// getContextCache shows how to retrieve the metadata of a cached content
// contentName is the ID of the cached content to retrieve
func getContextCache(w io.Writer, contentName string, projectID, location string) error {
	// location := "us-central1"
	ctx := context.Background()

	client, err := genai.NewClient(ctx, projectID, location)
	if err != nil {
		return fmt.Errorf("unable to create client: %w", err)
	}
	defer client.Close()

	cachedContent, err := client.GetCachedContent(ctx, contentName)
	if err != nil {
		return fmt.Errorf("GetCachedContent: %w", err)
	}
	fmt.Fprintf(w, "Retrieved cached content %q", cachedContent.Name)
	return nil
}

REST

The following shows how to use REST to list the context caches associated with a Google Cloud project by sending a GET request to the publisher model endpoint.

Before using any of the request data, make the following replacements:

  • PROJECT_ID: Your project ID.
  • LOCATION: The region where the request to create the context cache was processed.
  • CACHE_ID: The ID of the context cache. The context cache ID is returned when you create the context cache. You can also find context cache IDs by listing the context caches for a Google Cloud project using. For more information, see create a context cache and list context caches.

HTTP method and URL: 

GET https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/cachedContents/CACHE_ID

To send your request, choose one of these options:

curl

Execute the following command: 

curl -X GET \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     "https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/cachedContents/CACHE_ID"

PowerShell

Execute the following command: 

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method GET `
    -Headers $headers `
    -Uri "https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/cachedContents/CACHE_ID" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

Example curl command

LOCATION="us-central1"
PROJECT_ID="PROJECT_ID"
CACHE_ID="CACHE_ID"

curl \
-X GET \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
https://${LOCATION}-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/${LOCATION}/${CACHE_ID}