Use SAP function calling with Gemini

This document describes how to use SAP function calling with Gemini, by using the Vertex AI SDK for ABAP.

You can define custom functions and provide them to the Gemini models using the Function Calling feature. The models do not directly invoke the custom functions, but instead generate structured data output that specifies the function name and suggested arguments. This output lets you write applications that take the structured output and call external APIs. The resulting API output can then be incorporated into a further model prompt, allowing for more comprehensive query responses.

The Vertex AI SDK for ABAP simplifies function calling for ABAP developers by providing them opportunities to invoke custom logic written in SAP function modules by:

  • Passing SAP function module names to the model as function declarations, describing the name of the function, its purpose, and related parameters.
  • Setting auto-invocation of SAP function module implicitly while invoking the model.

The following is a typical ABAP developer's journey to invoke SAP function calling:

  1. User provides an input prompt.
  2. The SDK passes the input prompt and function declarations to the model.
  3. The model reviews the prompt and declared functions to derive the function to call and suggests the parameter values to call the function.
  4. If auto invocation is set, then the SDK calls the SAP function module.
  5. The SDK then invokes the model with the output of the called function.
  6. Model responds with a reliable answer for the final prompt enriched with the output of the called function.
  7. The SDK returns the response to the user.

SAP function calling with Gemini

If you choose not to auto invoke SAP function modules, then the SDK lets you use the function calling feature without any SAP function module invocation. In this case, you can follow the typical function calling workflow to use external tools such as APIs and functions.

Before you begin

Before using the Vertex AI SDK for ABAP for SAP function calling with Gemini, make sure that you or your administrators have completed the following prerequisites:

Enrich the Gemini AI model context with SAP data

This section explains how you can enrich the Gemini AI model context with SAP data by using the Vertex AI SDK for ABAP.

Instantiate the Gemini multimodal invoker class

To invoke function calling in SAP, you use the /GOOG/CL_GENERATIVE_MODEL class. You instantiate the class by passing the model key configured in the model generation parameters.

DATA(lo_model) = NEW /goog/cl_generative_model( iv_model_key = 'MODEL_KEY' ).

Replace MODEL_KEY with the model key name, which is configured in the model generation parameters.

Create an SAP function module

To create an SAP function module for auto-invocation by the SDK, follow the provided schema:

Category Parameter name Associated type
Importing IT_FUNCTION_PARAMETERS /GOOG/T_FUNCTION_PARAMETERS
Exporting EV_FUNCTION_RESPONSE STRING
Changing CV_PROMPT STRING
Exceptions /GOOG/CX_SDK Exception Class

Based on the importing parameters, write your custom logic within the function module, which can be either fetching SAP data through SELECT queries, or calling an external API or module to get the missing information.

Set the exporting parameter EV_FUNCTION_RESPONSE with the information to feedback to the LLM context. You can also change or modify the prompt text in CV_PROMPT based on the custom logic and your business requirement to further instruct the LLM based on different business scenarios.

Add function declaration

To add a function declaration to the LLM context, you can use the ADD_FUNCTION_DECLARATION method. Call the ADD_FUNCTION_DECLARATION method each time you need to add a function to the context.

DATA lt_parameters TYPE /goog/cl_generative_model=>tt_parameter_properties.
APPEND VALUE #( parameter_name = 'PARAMETER_NAME'
                type           = 'PARAMETER_TYPE'
                description    = 'PARAMETER_DESCRIPTION'
                is_required    = 'PARAMETER_IS_REQUIRED' ) TO lt_parameters.
lo_model->add_function_declaration( iv_name        = 'FUNCTION_MODULE_NAME'
                                    iv_description = 'FUNCTION_MODULE_DESCRIPTION'
                                    it_parameters  = lt_parameters ).

Replace the following:

  • PARAMETER_NAME: Name of the parameter.
  • PARAMETER_TYPE: The data type of the parameter, such as string, integer, or boolean.
  • PARAMETER_DESCRIPTION: A clear explanation of the parameter's purpose and expected format.
  • PARAMETER_IS_REQUIRED: If this parameter is mandatory for the function to operate, then set the value to ABAP_TRUE.
  • FUNCTION_MODULE_NAME: Name of the SAP function module.
  • FUNCTION_MODULE_DESCRIPTION: Description of the SAP function module.

Set auto invocation of SAP function module

To set auto invocation of the selected SAP function that is selected by the model, you can use the SET_AUTO_INVOKE_SAP_FUNCTION method. If ABAP_TRUE is passed in the importing parameter IV_AUTO_INVOKE, then the function module is invoked by the SDK and its response is included with the LLM context to generate the final response.

You must define your function module by following the schema described in the section Create SAP function module.

lo_model->set_auto_invoke_sap_function( abap_true ).

Generate content with function calling

To pass the prompt text to the Gemini model, you can use the GENERATE_CONTENT method. To get the response generated by Gemini with the additional context added from the SAP function module through function calling, use the GET_TEXT method.

DATA(lv_response) = lo_model->generate_content( iv_prompt_text ='PROMPT'
                           )->get_text( ).

Replace PROMPT with your text prompt.

Code sample

The following code sample illustrates how to use SAP function calling to receive a final response from the model.

DATA lt_parameters TYPE /goog/cl_generative_model=>tt_parameter_properties.
TRY.
  DATA(lo_model) = NEW /goog/cl_generative_model( iv_model_key = 'MODEL_KEY' ).
  APPEND VALUE #( parameter_name = 'PARAMETER_NAME'
                  type           = 'PARAMETER_TYPE'
                  description    = 'PARAMETER_DESCRIPTION'
                  is_required    = 'PARAMETER_IS_REQUIRED' ) TO lt_parameters.
  DATA(lv_response) = lo_model->add_function_declaration(
                                    iv_name        = 'FUNCTION_MODULE_NAME'
                                    iv_description = 'FUNCTION_MODULE_DESCRIPTION'
                                    it_parameters  = lt_parameters
                              )->set_auto_invoke_sap_function( abap_true
                              )->generate_content( iv_prompt_text ='PROMPT'
                              )->get_text( ).
  IF lv_response IS NOT INITIAL.
      cl_demo_output=>display( lv_response ).

  ENDIF.
 CATCH /goog/cx_sdk INTO DATA(lo_cx_sdk).
  cl_demo_output=>display( lo_cx_sdk->get_text( ) ).

ENDTRY.

Replace the following:

  • MODEL_KEY: The model key name, which is configured in the model generation parameters.
  • PARAMETER_NAME: Name of the parameter.
  • PARAMETER_TYPE: The data type of the parameter, such as string, integer, or boolean.
  • PARAMETER_DESCRIPTION: A clear explanation of the parameter's purpose and expected format.
  • PARAMETER_IS_REQUIRED: If this parameter is mandatory for the function to operate, then set the value to ABAP_TRUE.
  • FUNCTION_MODULE_NAME: Name of the SAP function module.
  • FUNCTION_MODULE_DESCRIPTION: Description of the SAP function module.
  • PROMPT: Your text prompt.

What's next