Model Armor integrates with various Google Cloud services:
- Google Kubernetes Engine (GKE) and Service Extensions
- Vertex AI
GKE and Service Extensions
Model Armor can be integrated with GKE through Service Extensions. Service Extensions allow you to integrate internal (Google Cloud services) or external (user-managed) services to process traffic. You can configure a service extension on application load balancers, including GKE inference gateways, to screen traffic to and from a GKE cluster. This verifies that all interactions with the AI models are protected by Model Armor. For more information, see Integration with GKE.
Vertex AI
Model Armor can be directly integrated into Vertex AI using either
floor settings or
templates.
This integration screens Gemini model requests and responses, blocking
those that violate floor settings. This integration provides prompt and response
protection within Gemini API in Vertex AI for the
generateContent method. You need to enable Cloud Logging to get visibility
into the sanitization results of prompts and responses. For more information, see
Integration with Vertex AI.
Before you begin
Options when integrating Model Armor
Model Armor offers the following integration options. Each option provides different features and capabilities.
| Integration option | Policy enforcer/detector | Configure detections | Inspect only | Inspect and block | Model and cloud coverage |
|---|---|---|---|---|---|
| REST API | Detector | Only using templates | Yes | Yes | All models and all clouds |
| Vertex AI (Preview) | Inline enforcement | Using floor settings or templates | Yes | Yes | Gemini (non-streaming) on Google Cloud |
| Google Kubernetes Engine (Preview) | Inline enforcement | Only using templates | Yes | Yes | Models with Open AI format on Google Cloud |
For the REST API integration option, Model Armor functions as a detector only using templates. This means it primarily identifies and reports potential policy violations based on predefined templates, rather than actively preventing them.
With the Vertex AI integration option, Model Armor provides inline enforcement using floor settings or templates. This means Model Armor actively enforces policies by intervening directly in the process without requiring modifications to your application code.
Similar to Vertex AI, the GKE integration option also offers inline enforcement only using templates. This indicates that Model Armor can enforce policies directly within the inference gateway without requiring modifications to your application code.