Function calling for open models

This guide shows how to use function calling with open models in Vertex AI, covering the following topics:

  • Use function calling: Learn how to structure a request to the model that includes a function definition.
  • Detailed example: See a complete, multi-turn example of a conversation that uses function calling.

The following diagram summarizes the overall workflow:

Function calling enables a model to identify when to invoke an external tool or API and then generate structured data containing the necessary function and arguments for execution. All open models available on the Vertex AI Model as a Service (MaaS) support function calling.

For more conceptual information about the function calling capability, see Introduction to function calling.

Use function calling

To use function calling, you provide the model with a set of tool definitions in your request. The model uses these definitions to determine if it needs to call a function to answer your query.

The following example shows a request that asks the model to compare the weather in two cities. The request includes a tools array with a function declaration for get_current_weather.

{
  "model": "MODEL_NAME",
  "messages": [
    {
      "role": "user",
      "content": "Which city has a higher temperature, Boston or new Delhi, and by how much in F?"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g. San Francisco, CA"
            },
            "unit": {
              "type": "string",
              "enum": ["celsius", "fahrenheit"]
            }
          },
          "required": ["location"]
        }
      }
    }
  ],
  "tool_choice": "auto"
}

After receiving the request, the model determines that it needs to call the get_current_weather function for both cities. It returns a response containing a tool_calls array with the appropriate arguments for each function call.

The following snippets show the two function call objects that the model returns in the tool_calls array.

Boston

{"function":{"arguments":"{\"location\":\"Boston, MA\",\"unit\":\"fahrenheit\"}","name":"get_current_weather"} "id":"get_current_weather","type":"function"}

New Delhi

{"function":{"arguments":"{\"location\":\"New Delhi, India\",\"unit\":\"fahrenheit\"}","name":"get_current_weather"},"id":"get_current_weather","type":"function"}

Detailed example

This section continues the previous example and shows the complete multi-turn conversation.

After your application executes the functions based on the model's tool_calls, you send another request to the model. This new request must include the history of the conversation and the results of the function calls. The results are provided in messages with the tool role.

The following is an example of the request payload that includes the tool results:

{
  "model": "MODEL_NAME",
  "messages": [
    {
      "role": "user",
      "content": "Which city has a higher temperature, Boston or new Delhi and by how much in F?"
    },
    {
      "role": "assistant",
      "content": "I'll check the current temperatures for Boston and New Delhi in Fahrenheit and compare them. I'll call the weather function for both cities.",
      "tool_calls": [{"function":{"arguments":"{\"location\":\"Boston, MA\",\"unit\":\"fahrenheit\"}","name":"get_current_weather"},"id":"get_current_weather","type":"function"},{"function":{"arguments":"{\"location\":\"New Delhi, India\",\"unit\":\"fahrenheit\"}","name":"get_current_weather"},"id":"get_current_weather","type":"function"}]
    },
    {
      "role": "tool",
      "content": "The temperature in Boston is 75 degrees Fahrenheit.",
      "tool_call_id": "get_weather"
    },
    {
      "role": "tool",
      "content": "The temperature in New Delhi is 50 degrees Fahrenheit.",
      "tool_call_id": "get_weather"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g. San Francisco, CA"
            },
            "unit": {
              "type": "string",
              "enum": ["celsius", "fahrenheit"]
            }
          },
          "required": ["location"]
        }
      }
    }
  ],
  "tool_choice": "auto"
}

The model uses the function results to answer your original question. The following is an example of the final response from the model:

{
 "choices": [
  {
   "finish_reason": "stop",
   "index": 0,
   "logprobs": null,
   "message": {
    "content": "Based on the current weather data:\n\n- **Boston, MA**: 75°F 
    \n- **New Delhi, India**: 50°F  \n\n**Comparison**:
    \nBoston is **25°F warmer** than New Delhi.  \n\n**Answer**:
    \nBoston has a higher temperature than New Delhi by 25 degrees Fahrenheit.",
    "role": "assistant"
   }
  }
 ],
 "created": 1750450289,
 "id": "2025-06-20|13:11:29.240295-07|6.230.75.101|-987540014",
 "model": "MODEL_NAME",
 "object": "chat.completion",
 "system_fingerprint": "",
 "usage": {
  "completion_tokens": 66,
  "prompt_tokens": 217,
  "total_tokens": 283
 }
}