Structured output for open models

Structured outputs enable a model to generate output that always adheres to a specific schema. For example, a model may be provided with a response schema to ensure that the response produces valid JSON. All open models available on the Vertex AI Model as a Service (MaaS) support structured outputs.

For more conceptual information about the structured output capability, see Introduction to structured output.

Use structured outputs

The following use case sets a response schema that ensures that the model output is a JSON object with the following properties: name, date, and participants. The Python code uses the OpenAI SDK and Pydantic objects to generate the JSON schema.

from pydantic import BaseModel
from openai import OpenAI

client = OpenAI()

class CalendarEvent(BaseModel):
    name: str
    date: str
    participants: list[str]

completion = client.beta.chat.completions.parse(
    model="MODEL_NAME",
    messages=[
        {"role": "system", "content": "Extract the event information."},
        {"role": "user", "content": "Alice and Bob are going to a science fair on Friday."},
    ],
    response_format=CalendarEvent,
)

print(completion.choices[0].message.parsed)

The model output will adhere to the following JSON schema:

{ "name": STRING, "date": STRING, "participants": [STRING] }

When provided the prompt, "Alice and Bob are going to a science fair on Friday", the model could produce the following response:

{
  "name": "science fair",
  "date": "Friday",
  "participants": [
    "Alice",
    "Bob"
  ]
}

Detailed example

The following code is an example of a recursive schema. The UI class contains a list of children, which can also be of the UI class.

from pydantic import BaseModel
from openai import OpenAI
from enum import Enum
from typing import List

client = OpenAI()

class UIType(str, Enum):
  div = "div"
  button = "button"
  header = "header"
  section = "section"
  field = "field"
  form = "form"

class Attribute(BaseModel):
  name: str
  value: str

class UI(BaseModel):
  type: UIType
  label: str
  children: List["UI"]
  attributes: List[Attribute]

UI.model_rebuild() # This is required to enable recursive types

class Response(BaseModel):
  ui: UI

completion = client.beta.chat.completions.parse(
  model="MODEL_NAME",
  messages=[
    {"role": "system", "content": "You are a UI generator AI. Convert the user input into a UI."},
    {"role": "user", "content": "Make a User Profile Form"}
  ],
  response_format=Response,
)

print(completion.choices[0].message.parsed)

The model output will adhere to the schema of the Pydantic object specified in the previous snippet. In this example, the model could generate the following UI form:

Form
  Input
    Name
    Email
    Age

A response could look like the following:

ui = UI(
    type=UIType.div,
    label='Form',
    children=[
        UI(
            type=UIType.div,
            label='Input',
            children=[],
            attributes=[
                Attribute(name='label', value='Name')
            ]
        ),
        UI(
            type=UIType.div,
            label='Input',
            children=[],
            attributes=[
                Attribute(name='label', value='Email')
            ]
        ),
        UI(
            type=UIType.div,
            label='Input',
            children=[],
            attributes=[
                Attribute(name='label', value='Age')
            ]
        )
    ],
    attributes=[
        Attribute(name='name', value='John Doe'),
        Attribute(name='email', value='john.doe@example.com'),
        Attribute(name='age', value='30')
    ]
)