Model Armor integration with Google Cloud services

Model Armor integrates with various Google Cloud services:

  • Google Kubernetes Engine (GKE) and Service Extensions
  • Vertex AI

GKE and Service Extensions

Model Armor can be integrated with GKE through Service Extensions. Service Extensions allow you to integrate internal (Google Cloud services) or external (user-managed) services to process traffic. You can configure a service extension on application load balancers, including GKE inference gateways, to screen traffic to and from a GKE cluster. This verifies that all interactions with the AI models are protected by Model Armor. For more information, see Integration with GKE.

Vertex AI

Model Armor can be directly integrated into Vertex AI using either floor settings or templates. This integration screens Gemini model requests and responses, blocking those that violate floor settings. This integration provides prompt and response protection within Gemini API in Vertex AI for the generateContent method. You need to enable Cloud Logging to get visibility into the sanitization results of prompts and responses. For more information, see Integration with Vertex AI.

Before you begin

Enable APIs

You must enable Model Armor APIs before you can use Model Armor.

Console

  1. Enable the Model Armor API.

    Enable the API

  2. Select the project where you want to activate Model Armor.

gcloud

Before you begin, follow these steps using the Google Cloud CLI with the Model Armor API:

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. Run the following command to set the API endpoint for the Model Armor service.

    gcloud config set api_endpoint_overrides/modelarmor "https://modelarmor.LOCATION.rep.googleapis.com/"

    Replace LOCATION with the region where you want to use Model Armor.

Run the following command to enable Model Armor.

  gcloud services enable modelarmor.googleapis.com --project=PROJECT_ID

Replace PROJECT_ID with the ID of the project.

Options when integrating Model Armor

Model Armor offers the following integration options. Each option provides different features and capabilities.

Integration option Policy enforcer/detector Configure detections Inspect only Inspect and block Model and cloud coverage
REST API Detector Only using templates Yes Yes All models and all clouds
Vertex AI (Preview) Inline enforcement Using floor settings or templates Yes Yes Gemini (non-streaming) on Google Cloud
Google Kubernetes Engine (Preview) Inline enforcement Only using templates Yes Yes Models with Open AI format on Google Cloud

For the REST API integration option, Model Armor functions as a detector only using templates. This means it primarily identifies and reports potential policy violations based on predefined templates, rather than actively preventing them.

With the Vertex AI integration option, Model Armor provides inline enforcement using floor settings or templates. This means Model Armor actively enforces policies by intervening directly in the process without requiring modifications to your application code.

Similar to Vertex AI, the GKE integration option also offers inline enforcement only using templates. This indicates that Model Armor can enforce policies directly within the inference gateway without requiring modifications to your application code.