Deploy an agentic AI application on GKE with the Agent Development Kit (ADK) and Vertex AI

Prepare the environment

This tutorial uses Cloud Shell to manage resources hosted on Google Cloud. Cloud Shell comes preinstalled with the software you need for this tutorial, including kubectl, terraform, and the Google Cloud CLI.

To set up your environment with Cloud Shell, follow these steps:

1. In the Google Cloud console, launch a Cloud Shell session and click Activate Cloud Shell. This action launches a session in the bottom pane of the Google Cloud console.

2. Set the default environment variables:

   gcloud config set project PROJECT_ID
   export GOOGLE_CLOUD_LOCATION=REGION
   export PROJECT_ID=PROJECT_ID
   export GOOGLE_CLOUD_PROJECT=$PROJECT_ID
   export WORKLOAD_POOL=$PROJECT_ID.svc.id.goog
   export PROJECT_NUMBER=$(gcloud projects describe --format json $PROJECT_ID | jq -r ".projectNumber")
   

   Replace the following values:

   • PROJECT_ID: your Google Cloud project ID.
   • REGION: the Google Cloud region (for example, us-central1) to provision your GKE cluster, Artifact Registry, and other regional resources.

Clone the sample project

1. From your Cloud Shell terminal, clone the tutorial's sample code repository:

   git clone https://github.com/GoogleCloudPlatform/kubernetes-engine-samples.git
   

2. Navigate to the tutorial directory:

   cd kubernetes-engine-samples/ai-ml/adk-vertex
   

Create and configure Google Cloud resources

To deploy your agent to GKE, provision the necessary Google Cloud resources. You can deploy the agent by using either the gcloud CLI for direct command-line execution or Terraform for infrastructure as code.

Configure kubectl to communicate with your cluster

If you haven't configured kubectl to communicate with your cluster, run the following command:

gcloud container clusters get-credentials CLUSTER_NAME \
    --location=${GOOGLE_CLOUD_LOCATION}

Replace CLUSTER_NAME with the name of your GKE cluster.

Configure Workload Identity Federation for GKE access

You can skip this step if you are using Terraform. This process links a Kubernetes Service Account (KSA) to an IAM service account to securely grant your agent access to Google Cloud services.

1. Create a service account for Vertex AI access:

   gcloud iam service-accounts create vertex-sa \
       --project=$PROJECT_ID
   

2. Grant the service account the aiplatform.user role. This allows the service account to interact with Vertex AI.

   gcloud projects add-iam-policy-binding $PROJECT_ID \
       --member "serviceAccount:vertex-sa@$PROJECT_ID.iam.gserviceaccount.com" \
       --role "roles/aiplatform.user"
   

3. Create a KSA in your cluster. Make sure to follow the instructions in Configure kubectl to communicate with your cluster before running this command.

   kubectl create serviceaccount vertex-sa
   

4. Annotate the KSA to link it to your service account:

   kubectl annotate serviceaccount vertex-sa \
       iam.gke.io/gcp-service-account=vertex-sa@$PROJECT_ID.iam.gserviceaccount.com
   

5. Grant your service account permissions to act as a Workload Identity Federation for GKE user for the KSA:

   gcloud iam service-accounts add-iam-policy-binding vertex-sa@$PROJECT_ID.iam.gserviceaccount.com \
       --role roles/iam.workloadIdentityUser \
       --member "serviceAccount:$PROJECT_ID.svc.id.goog[default/vertex-sa]"
   

Deploy and configure the agent application

After setting up your Google Cloud resources, prepare your agent application for deployment and configure its runtime on GKE. This involves defining your agent's code, creating a Dockerfile to containerize it, and writing a Kubernetes manifest to deploy it to your cluster.

1. Understand the agent application structure: The /app directory contains the core files for your agent application:
   • main.py: The FastAPI application entry point, responsible for exposing the agent's API.
   • agent.py: Contains the ADK agent's logic, defining how it uses Vertex AI and handles requests.
   • __init__.py: Initializes the Python package.
   • requirements.txt: Lists all Python dependencies for your agent.
   • Dockerfile: Defines how your application is packaged into a Docker image.

2. Create the agent deployment manifest: In the tutorials-and-examples/adk/vertex directory, create the following manifest as a file named agent-deployment.yaml:

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: adk-agent-deployment
     labels:
       app: adk-agent
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: adk-agent
     template:
       metadata:
         labels:
           app: adk-agent
       spec:
         serviceAccountName: vertex-sa
         containers:
         - name: adk-agent
           image: IMAGE_URL
           ports:
           - containerPort: 8000
           env:
           - name: GOOGLE_CLOUD_PROJECT_ID
             value: PROJECT_ID
           - name: GOOGLE_CLOUD_LOCATION
             value: REGION
           - name: GOOGLE_GENAI_USE_VERTEXAI
             value: "1"
           - name: PORT
             value: "8000"
           resources:
             requests:
               memory: "512Mi"
               cpu: "500m"
             limits:
               memory: "1Gi"
               cpu: "1"
   

   Replace the following values:

   • IMAGE_URL: the full URL of the Docker image you pushed to Artifact Registry (for example, us-central1-docker.pkg.dev/${PROJECT_ID}/adk-repo/adk-agent:latest). You can get this value from the output of the gcloud artifacts docker images list command in the previous section. Alternatively, run a command like the following: sed -i "s|image: IMAGE_URL|image: $IMAGE_URL|" agent-deployment.yaml.
   • PROJECT_ID: your Google Cloud project ID.
   • REGION: the Google Cloud region where your GKE cluster is provisioned.

3. Apply the deployment manifest:

   kubectl apply -f agent-deployment.yaml
   

   This command creates the Deployment and associated Pods for your agent application in the GKE cluster.

4. Expose the agent: You can use these methods to expose your agent for access.

   • To access your agent from outside the cluster, create a Kubernetes Service. For a web API, a LoadBalancer type service is common.

   • For local development and testing, use the kubectl port-forward command to access your agent directly.

   LoadBalancer

   1. Create the following manifest as a file named agent-service.yaml:

      apiVersion: v1
      kind: Service
      metadata:
        name: adk-agent-service
      spec:
        selector:
          app: adk-agent
        type: LoadBalancer # Creates an external IP address for access
        ports:
        -   protocol: TCP
          port: 80
          targetPort: 8000 # Matches the containerPort exposed in the Deployment
      

   2. Apply the manifest:

      kubectl apply -f agent-service.yaml
      

      It might take a few minutes for the external IP address to provision.

   3. Verify that the IP address is provisioned:

      kubectl get service adk-agent-service
      

      Look for the EXTERNAL-IP value in the output. After the value is available, use this IP address to interact with your deployed agent.

   port-forward

   If you primarily use port-forward, consider using a ClusterIP Service type instead of LoadBalancer as the ClusterIP Service is internal and consumes fewer resources.

   POD_NAME=$(kubectl get pods -l app=adk-agent -o jsonpath='{.items[0].metadata.name}')
   kubectl port-forward $POD_NAME 8000:8000
   

   This command blocks your terminal, but it forwards traffic from localhost:8000 on your machine to the agent running inside the GKE cluster.

Test your deployed agent

Now that your agent application is deployed and exposed, test its functionality.

This section describes how to identify your agent's endpoint and test the agent's API.

1. Identify the agent's endpoint: Depending on how you chose to expose your agent in the previous section, identify its accessible endpoint:

   LoadBalancer

   1. Retrieve the external IP address:

      kubectl get service adk-agent-service -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
      

   2. Store the external IP address in an environment variable for easier use:

      export AGENT_IP=$(kubectl get service adk-agent-service -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
      

      Your agent's base URL is http://${AGENT_IP}.

   port-forward

   Ensure your kubectl port-forward command runs in a separate terminal. Your agent's base URL is http://localhost:8000.

2. Test the agent's API: Test the agent's API by making a curl request to its /run endpoint. The agent expects a JSON payload with a message field. Replace AGENT_BASE_URL with either http://${AGENT_IP} (for LoadBalancer) or http://localhost:8000 (for port-forward).

   1. Create a new session. This prepares the agent for a new conversation.

      # The user ID and session ID are arbitrary.
      # The appName must match the agent's Python package directory name (in this case, "capital-agent").
      curl -X POST AGENT_BASE_URL/apps/capital-agent/users/user-123/sessions/session-123
      

      You can list available apps with the following command:

      curl -X GET AGENT_BASE_URL/list-apps
      

   2. Send a query to the agent by using the session details from the previous step.

      curl -X POST AGENT_BASE_URL/run \
      -H "Content-Type: application/json" \
      -d '{
        "appName": "capital-agent",
        "userId": "user-123",
        "sessionId": "session-123",
        "newMessage": {
          "role": "user",
          "parts": [{
            "text": "Hello, agent! What can you do for me?"
          }]
        }
      }'
      

      You should receive a JSON response from your agent, indicating that it successfully processes requests and interacts with the Gemini model through Vertex AI. The response contains the agent's generated reply based on your message.

3. Access the agent's web UI (if applicable): If your agent includes a web-based user interface, access it by navigating to your AGENT_BASE_URL in a web browser. The ADK typically includes a basic web UI for interaction and debugging. For example, if you exposed your agent via a LoadBalancer and its EXTERNAL-IP is 34.123.45.67, navigate to http://34.123.45.67 in your browser.