This page describes a preview that lets you experiment with registering an AI model endpoint
and invoking predictions with Model endpoint management. For using AI models in
production environments, see Build generative AI applications using
AlloyDB AI.
After the model endpoints are added and registered in the Model endpoint management, you can
reference them using the model ID to invoke predictions.
Use the google_ml.predict_row() SQL function to call a registered generic model endpoint to invoke
predictions. You can use google_ml.predict_row() function with any model type.
MODEL_ID: the model ID you defined when registering the model endpoint.
REQUEST_BODY: the parameters to the prediction function, in JSON format.
Examples
Some examples for invoking predictions using registered model endpoints are listed in this section.
To generate predictions for a registered gemini-pro model endpoint, run the following statement:
SELECTjson_array_elements(google_ml.predict_row(model_id=>'gemini-pro',request_body=>'{ "contents": [ { "role": "user", "parts": [ { "text": "For TPCH database schema as mentioned here https://www.tpc.org/TPC_Documents_Current_Versions/pdf/TPC-H_v3.0.1.pdf , generate a SQL query to find all supplier names which are located in the India nation." } ] } ] }'))->'candidates'->0->'content'->'parts'->0->'text';
To generate predictions for a registered facebook/bart-large-mnli model endpoint on Hugging Face, run the following statement:
SELECTgoogle_ml.predict_row(model_id=>'facebook/bart-large-mnli',request_body=>
'{ "inputs": "Hi, I recently bought a device from your company but it is not working as advertised and I would like to get reimbursed!", "parameters": {"candidate_labels": ["refund", "legal", "faq"]} }');
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-25 UTC."],[[["\u003cp\u003eModel endpoint management is a preview feature that allows you to experiment with registering AI model endpoints and invoking predictions, subject to Pre-GA Offerings Terms.\u003c/p\u003e\n"],["\u003cp\u003eRegistered model endpoints can be referenced by their model ID to invoke predictions.\u003c/p\u003e\n"],["\u003cp\u003eThe \u003ccode\u003egoogle_ml.predict_row()\u003c/code\u003e SQL function is used to call registered model endpoints for predictions and can be used with any model type.\u003c/p\u003e\n"],["\u003cp\u003eExamples are provided for invoking predictions using registered \u003ccode\u003egemini-pro\u003c/code\u003e and \u003ccode\u003efacebook/bart-large-mnli\u003c/code\u003e model endpoints.\u003c/p\u003e\n"]]],[],null,["# Invoke predictions with model endpoint management\n\nSelect a documentation version: 15.5.4keyboard_arrow_down\n\n- [15.5.5](/alloydb/omni/15.5.5/docs/model-endpoint-predictions)\n- [15.5.4](/alloydb/omni/15.5.4/docs/model-endpoint-predictions)\n- [15.5.2](/alloydb/omni/15.5.2/docs/model-endpoint-predictions)\n\n\u003cbr /\u003e\n\n|\n| **Preview\n| --- Model endpoint management**\n|\n|\n| This feature is subject to the \"Pre-GA Offerings Terms\" in the General Service Terms section\n| of the [Service Specific Terms](/terms/service-terms#1).\n|\n| Pre-GA features are available \"as is\" and might have limited support.\n|\n| For more information, see the\n| [launch stage descriptions](/products#product-launch-stages).\n\nThis page describes a preview that lets you experiment with registering an AI model endpoint\nand invoking predictions with Model endpoint management. For using AI models in\nproduction environments, see [Build generative AI applications using\nAlloyDB AI](/alloydb/docs/ai).\n\nAfter the model endpoints are added and registered in the Model endpoint management, you can\nreference them using the model ID to invoke predictions.\n\nBefore you begin\n----------------\n\nMake sure that you have registered your model endpoint with Model endpoint management. For more information, see [Register a model endpoint with model endpoint management](/alloydb/omni/15.5.4/docs/model-endpoint-register-model)\n\nInvoke predictions for generic models\n-------------------------------------\n\nUse the `google_ml.predict_row()` SQL function to call a registered generic model endpoint to invoke\npredictions. You can use `google_ml.predict_row()` function with any model type. \n\n SELECT\n google_ml.predict_row(\n model_id =\u003e '\u003cvar translate=\"no\"\u003eMODEL_ID\u003c/var\u003e',\n request_body =\u003e '\u003cvar translate=\"no\"\u003eREQUEST_BODY\u003c/var\u003e');\n\nReplace the following:\n\n- \u003cvar translate=\"no\"\u003eMODEL_ID\u003c/var\u003e: the model ID you defined when registering the model endpoint.\n- \u003cvar translate=\"no\"\u003eREQUEST_BODY\u003c/var\u003e: the parameters to the prediction function, in JSON format.\n\nExamples\n--------\n\nSome examples for invoking predictions using registered model endpoints are listed in this section.\n\nTo generate predictions for a registered `gemini-pro` model endpoint, run the following statement: \n\n SELECT\n json_array_elements(\n google_ml.predict_row(\n model_id =\u003e 'gemini-pro',\n request_body =\u003e '{\n \"contents\": [\n {\n \"role\": \"user\",\n \"parts\": [\n {\n \"text\": \"For TPCH database schema as mentioned here https://www.tpc.org/TPC_Documents_Current_Versions/pdf/TPC-H_v3.0.1.pdf , generate a SQL query to find all supplier names which are located in the India nation.\"\n }\n ]\n }\n ]\n }'))-\u003e 'candidates' -\u003e 0 -\u003e 'content' -\u003e 'parts' -\u003e 0 -\u003e 'text';\n\nTo generate predictions for a registered `facebook/bart-large-mnli` model endpoint on Hugging Face, run the following statement: \n\n SELECT\n google_ml.predict_row(\n model_id =\u003e 'facebook/bart-large-mnli',\n request_body =\u003e\n '{\n \"inputs\": \"Hi, I recently bought a device from your company but it is not working as advertised and I would like to get reimbursed!\",\n \"parameters\": {\"candidate_labels\": [\"refund\", \"legal\", \"faq\"]}\n }'\n );"]]