Imagen on Vertex AI brings Google's state of the art generative AI capabilities to application developers. As an early-stage technology, Imagen on Vertex AI's evolving capabilities and uses create potential for misapplication, misuse, and unintended or unforeseen consequences. For example, Imagen on Vertex AI could generate output that you don't expect, such as images that are offensive, insensitive, or contextually incorrect.
Given these risks and complexities, Imagen on Vertex AI is designed with Google's AI Principles in mind. However, it is important for developers to understand and test their models to deploy them safely and responsibly. To aid developers, Imagen on Vertex AI has built-in safety filters to help customers block potentially harmful outputs within their use case. See the safety filters section for more.
When Imagen on Vertex AI is integrated into a customer's unique use case and context, additional responsible AI considerations and model limitations may need to be considered. We encourage customers to use fairness, interpretability, privacy, and security recommended practices.
Safety filters
Text prompts provided as inputs and images (generated or uploaded) through
Imagen on Vertex AI are assessed against a list of safety filters, which
include 'harmful categories' (for example, violence, sexual, derogatory,
and toxic).
These safety filters aim to filter out (generated or uploaded) content that
violates our Acceptable Use Policy (AUP),
Generative AI Prohibited Use Policy or our
AI Principles.
If the model responds to a request with an error message such as "The prompt couldn't be submitted" or "it might violate our policies", then the input is triggering a safety filter. If fewer images than requested are returned, then some generated output are blocked for not meeting safety requirements.
You can choose how aggressively to filter sensitive content by adjusting the
safetySetting parameter.
Safety attributes
Safety attributes and safety filters don't have a one-to-one mapping
relationship. Safety attributes are the set of attributes that we return to user
when includeSafetyAttributes is set. Safety filters are the set of filters we
use to filter content. We don't filter on all safety attribute categories. For
example, for the safety attribute category "Health", we don't filter content
based on the health confidence score. Also, we don't expose the confidence
scores for some of our internal sensitive safety filters.
Configure safety filters
There are several safety filtering parameters you can use with the
imagegeneration model. For example, you can let the model report safety filter
codes for blocked content, disable people or face generation, adjust the
sensitivity of content filtering, or return rounded safety scores of list of
safety attributes for input and output. For more technical information about individual fields, see the
imagegeneration model API reference.
The response varies depending on which parameters you set; some parameters affect the content produced, while others affect content filtering and how filtering is reported to you. Additionally, the output format depends on if the input data is filtered, or if the generated image output is filtered.
Parameters that filter content
The following optional parameters affect content filtering or how filtering is reported to you:
safetySetting* - Lets you set how aggressively to filter for potentially sensitive output content.includeRaiReason- Provides more verbose information on filtered output.personGeneration- A setting that allows you more control over the generation of people, faces, and children.disablePersonFace- Deprecated. A choice to allow person and face generation or not. Users should setpersonGenerationinstead.includeSafetyAttributes- Gives you full safety attribute information for input text, input image (for editing), and all generated images. This information includes safety category (for example,"Firearms & Weapons","Illicit Drugs", or"Violence") and the confidence scores.
* Available for imagegeneration@006 model only.
Filtered input
If your text input or input image (for editing) is filtered, you get a response
with a 400 error code. A request with RAI-filtered input returns this output
format if you set either includeRaiReason or includeSafetyAttributes.
Output depends on the model version you use. The following shows output when the input is filtered for different model versions:
Model
{
"error": {
"code": 400,
"message": "Image generation failed with the following error: The prompt could not be submitted. This prompt contains sensitive words that violate Google's Responsible AI practices. Try rephrasing the prompt. If you think this was an error, send feedback."
"status": "INVALID_ARGUMENT",
"details": [
{
"@type": "type.googleapis.com/google.rpc.DebugInfo",
"detail": "[ORIGINAL ERROR] generic::invalid_argument: Image generation failed with the following error: The prompt could not be submitted. This prompt contains sensitive words that violate Google's Responsible AI practices. Try rephrasing the prompt. If you think this was an error, send feedback. [google.rpc.error_details_ext] { message: \"Image editing failed with the following error: The prompt could not be submitted. This prompt contains sensitive words that violate Google's Responsible AI practices. Try rephrasing the prompt. If you think this was an error, send feedback.\" }"
}
]
}
}
Models
{
"error": {
"code": 400,
"message": "Image generation failed with the following error: The prompt could not be submitted. This prompt contains sensitive words that violate Google's Responsible AI practices. Try rephrasing the prompt. If you think this was an error, send feedback.",
"status": "INVALID_ARGUMENT",
"details": [
{
"@type": "type.googleapis.com/google.rpc.DebugInfo",
"detail": "[ORIGINAL ERROR] generic::invalid_argument: Image generation failed with the following error: The prompt could not be submitted. This prompt contains sensitive words that violate Google's Responsible AI practices. Try rephrasing the prompt. If you think this was an error, send feedback. [google.rpc.error_details_ext] { message: \"Image generation failed with the following error: The prompt could not be submitted. This prompt contains sensitive words that violate Google\\'s Responsible AI practices. Try rephrasing the prompt. If you think this was an error, send feedback.\" }"
}
]
}
}
Filtered output
The contents of filtered output vary depending on the RAI parameter you set.
The following output examples show the result of using the includeRaiReason
and includeSafetyAttributes parameters.
Filtered output using includeRaiReason
If you don't add includeRaiReason or set includeRaiReason: false, your
response only includes generated image objects that aren't filtered. Any
filtered image objects are omitted from the "predictions": [] array. For
example, the following is a response to a request with "sampleCount": 4, but
two of the images are filtered and consequently omitted:
{
"predictions": [
{
"bytesBase64Encoded": "/9j/4AAQSkZJRgABA[...]bdsdgD2PLbZQfW96HEFE/9k=",
"mimeType": "image/png"
},
{
"mimeType": "image/png",
"bytesBase64Encoded": "/9j/4AAQSkZJRgABA[...]Ct+F+1SLLH/2+SJ4ZLdOvg//Z"
}
],
"deployedModelId": "MODEL_ID"
}
If you set includeRaiReason: true and several output images are filtered, your
response includes generated image objects and raiFilteredReason objects for
any filtered output images. For example, the following is a response to a
request with "sampleCount": 4 and includeRaiReason: true, but two of the
images are filtered. Consequently, two objects include generated image
information and the other object includes an error message.
Model
{
"predictions": [
{
"bytesBase64Encoded": "/9j/4AAQSkZJRgABA[...]bdsdgD2PLbZQfW96HEFE/9k=",
"mimeType": "image/png"
},
{
"mimeType": "image/png",
"bytesBase64Encoded": "/9j/4AAQSkZJRgABA[...]Ct+F+1SLLH/2+SJ4ZLdOvg//Z"
},
{
"raiFilteredReason": "Your current safety filter threshold filtered out 2 generated images. You will not be charged for blocked images. Try rephrasing the prompt. If you think this was an error, send feedback."
},
],
"deployedModelId": "MODEL_ID"
}
Models
{
"predictions": [
{
"bytesBase64Encoded": "/9j/4AAQSkZJRgABA[...]bdsdgD2PLbZQfW96HEFE/9k=",
"mimeType": "image/png"
},
{
"mimeType": "image/png",
"bytesBase64Encoded": "/9j/4AAQSkZJRgABA[...]Ct+F+1SLLH/2+SJ4ZLdOvg//Z"
},
{
"raiFilteredReason": "56562880"
},
{
"raiFilteredReason": "56562880"
}
],
"deployedModelId": "MODEL_ID"
}
Filtered output using includeSafetyAttributes
If you set "includeSafetyAttributes": true, the response "predictions": []
array includes the RAI scores (rounded to one decimal place) of text safety
attributes of the positive prompt. The image safety
attributes are also added to each unfiltered output. If an output image is
filtered its safety attributes aren't returned. For example, the following is a
response to an unfiltered request, and one image is returned:
{
"predictions": [
{
"bytesBase64Encoded": "/9j/4AAQSkZJRgABA[...]bdsdgD2PLbZQfW96HEFE/9k=",
"mimeType": "image/png",
"safetyAttrbutes": {
"categories": [
"Porn",
"Violence"
],
"scores": [
0.1,
0.2
]
}
},
{
"contentType": "Positive Prompt",
"safetyAttrbutes": {
"categories": [
"Death, Harm & Tragedy",
"Firearms & Weapons",
"Hate",
"Health",
"Illicit Drugs",
"Politics",
"Porn",
"Religion & Belief",
"Toxic",
"Violence",
"Vulgarity",
"War & Conflict"
],
"scores": [
0,
0,
0,
0,
0,
0,
0.2,
0,
0.1,
0,
0.1,
0
]
}
},
],
"deployedModelId": "MODEL_ID"
}
Limitations
The following limits apply to different tasks:
Image generation limitations
- Bias amplification: While Imagen on Vertex AI can generate high-quality images, there may be potential biases in the generated content. Images generated rely on the product's training data, which can unintentionally include biases that may perpetuate stereotypes or discriminate against certain groups. Careful monitoring and evaluation are necessary to ensure the outputs align with Google's Acceptable Use Policy and your use case.
- Transparency and disclosure: It can be difficult for users to differentiate between AI generated Imagery and non AI generated imagery. When using AI-generated images within your use case, it is important to clearly disclose to users that the images have been generated by an AI system to ensure transparency and maintain trust in the process. We've applied metadata labeling to AI-generated images to help combat the risk of misinformation and as part of our responsible approach to AI.
- Insufficient context: Imagen on Vertex AI may lack the contextual understanding required to generate images that are appropriate for all situations or audiences within your use case. Be sure to check that your generated images align with your chosen context, purpose, and intended audience.
Image editing limitations
- Misrepresentation and authenticity: Editing images using Imagen on Vertex AI can result in misrepresentation or manipulation of images, potentially leading to the creation of deceptive or misleading content. It is important to ensure that the editing process is used responsibly, without compromising the authenticity and truthfulness of the images edited. We've applied metadata labeling to AI-edited images to help combat the risk of misinformation and as part of our responsible approach to AI.
Visual captioning limitations
- Accuracy and context sensitivity: Visual captioning may encounter challenges in accurately describing complex or ambiguous images. The generated descriptions may not always capture the complete context or nuances of the visual content. It is important to acknowledge that automated captioning systems have limitations in understanding images with varying levels of complexity, and their descriptions should be used with caution, particularly in critical or sensitive contexts.
- Ambiguity and subjective interpretations: Images can often be open to multiple interpretations, and the generated captions may not always align with human understanding or expectations. Different individuals may perceive and describe images differently based on their subjective experiences and cultural backgrounds. It is crucial to consider the potential for ambiguity and subjectivity in image descriptions and provide additional context or alternative interpretations where necessary.
- Accessibility considerations: While automated image captions can support accessibility by providing descriptions for visually impaired individuals, it is important to recognize that they may not fully replace human-generated alt-text or descriptions tailored to specific accessibility needs. Automated captions may lack the level of detail or contextual understanding necessary for certain accessibility use cases.
Visual Question Answering (VQA) limitations
- Overconfidence and uncertainty: VQA models may sometimes provide answers with unwarranted confidence, even when the correct answer is uncertain or ambiguous. It is essential to communicate the model's uncertainty and provide appropriate confidence scores or alternative answers when there is ambiguity, rather than conveying a false sense of certainty.
Recommended practices
To utilize this technology safely and responsibly, it is also important to consider other risks specific to your use case, users, and business context in addition to built-in technical safeguards.
We recommend taking the following steps:
- Assess your application's security risks.
- Consider adjustments to mitigate safety risks.
- Perform safety testing appropriate to your use case.
- Solicit user feedback and monitor content.
Additional resources
- Learn about Responsible AI for Large Language Models (LLMs).
- Learn more about Google's recommendations for Responsible AI practices.
- Read our blog, A shared agenda for responsible AI progress
Give feedback on Imagen on Vertex AI
If you receive an output or response that is inaccurate or that you feel is unsafe, you can let us know by submitting feedback. Your feedback can help improve Imagen on Vertex AI and broader Google efforts in AI.
Because feedback may be human readable, don't submit data that contains personal, confidential, or sensitive information.