Connect to data from AlloyDB for PostgreSQL (Preview)

You can connect your AlloyDB for PostgreSQL instance to Agentspace Enterprise, enabling live access to your data.

Compared to importing from AlloyDB for PostgreSQL, connecting has the following advantages:

Avoids the need for importing, indexing, and duplicating data.
Maximal data freshness.
No additional indexing and storage costs.
No need to flatten structured data into uniform records.

Consider the following disadvantages of connecting compared to importing:

This approach supports only structured queries, not semantic or semi-semantic queries.
The setup is slightly more involved compared to importing the data.

Limitations

Connecting to AlloyDB for PostgreSQL has the following limitations:

Agentspace Enterprise supports access control on the level of database objects, such as tables and columns, but not on a per-user level. All users with access to your Agentspace Enterprise app can see the same data.
Creating an AlloyDB for PostgreSQL data store is supported through the API, not through the Google Cloud console.

Before you begin

Connecting to AlloyDB for PostgreSQL is in Preview with an allowlist. To use this feature, sign up with the following form: Opt-in/Opt-out to use Agentspace and connect to data from AlloyDB

After signing up, wait until you receive the enablement confirmation before following the instructions on this page.

Set up AlloyDB AI natural language

This connector relies on AlloyDB AI natural language in AlloyDB for PostgreSQL, which translates natural language queries into SQL queries.

Before connecting, set up AlloyDB AI natural language in your database using the steps in Generate SQL queries that answer natural language questions in the AlloyDB for PostgreSQL documentation.

Note that adding templates in AlloyDB AI natural language (a curated set of natural language questions with corresponding SQL queries) is particularly useful for quality. When adding templates, Google recommends the following:

Use aliases of the form "table_name.column_name" for physical columns (columns that map to columns in underlying tables and are not computed on the fly). This makes it easier to manage the schema in Agentspace Enterprise.
Use ORDER BY to avoid non-determinism. For example: SELECT customers.id AS "customers.id" FROM customers ORDER BY customers.id;.

Note: You can add templates and other context later. For example, when you notice low answer quality in Agentspace Enterprise for a specific question.

Set up authentication between Agentspace Enterprise and AlloyDB for PostgreSQL

Agentspace Enterprise accesses AlloyDB for PostgreSQL using a dedicated service account and a database user in your AlloyDB for PostgreSQL instance. With this approach, every Agentspace Enterprise user can see the same data, without user-specific access control.

A service account is used to connect and interact with your AlloyDB for PostgreSQL instance. To enable this, you need to configure the Identity and Access Management (IAM) settings in the Google Cloud project where your database instance resides.

To set up authentication, do the following:

Navigate to the IAM configuration page of the project that hosts your AlloyDB for PostgreSQL instance.
Grant the Cloud AlloyDB for PostgreSQL Database User role to the following principal: service-PROJECT_NUMBER@gcp-sa-discoveryengine.iam.gserviceaccount.com

Replace PROJECT_NUMBER with the Google Cloud project number where you use Agentspace Enterprise.
In your AlloyDB for PostgreSQL instance, create a DB user to be used by Agentspace Enterprise to execute SQL queries.

It is very important that you grant this user read-only access and only for tables, views and columns that should be accessible from Agentspace Enterprise. This ensures that:
- Queries from Agentspace Enterprise cannot change or delete your data. Agentspace Enterprise is designed to only read data from your database and not change anything, but Google can't fully exclude the possibility that the AI might generate unintended write-queries.
- Users in Agentspace Enterprise can only see data they are supposed to see.
  
  For example:
```
CREATE ROLE USER_NAME WITH LOGIN PASSWORD 'PASSWORD';
GRANT SELECT ON TABLE "public"."TABLE_NAME_1" TO USER_NAME;
GRANT SELECT ON TABLE "public"."TABLE_NAME_2" TO USER_NAME;
```
  When linking Agentspace Enterprise with AlloyDB for PostgreSQL, provide the username and password in the User and Password fields of alloydbConnectionConfig.

Link Agentspace Enterprise with AlloyDB for PostgreSQL

To connect your AlloyDB for PostgreSQL instance, follow these steps:

Create a data store in Agentspace Enterprise with your AlloyDB for PostgreSQL connection details using the API, including federatedSearchConfig.alloyDbConfig.

The following example:
```
curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  -H "X-Goog-User-Project: PROJECT_ID" \
  "https://discoveryengine.googleapis.com/v1alpha/projects/PROJECT_ID/locations/LOCATION/collections/default_collection/dataStores?dataStoreId=DATA_STORE_ID" -d '{
  "displayName": "DATA_STORE_NAME",
  "federatedSearchConfig": {
    "alloyDbConfig": {
      "alloydbConnectionConfig": {
        "instance": "INSTANCE_PATH",
        "database": "DATABASE_NAME",
        "user": "DATABASE_USER_NAME",
        "password": "DATABASE_USER_PASSWORD",
        "authMode": "AUTH_MODE_SERVICE_ACCOUNT"
      },
      "alloydb_ai_nl_config": { "nlConfigId": "NL_CONFIG_ID" }
    }
  },
  "industryVertical": "GENERIC",
  "solutionTypes": ["SOLUTION_TYPE_SEARCH"]
}'
```
Replace the following:
- PROJECT_ID: the ID of your project where you use Agentspace Enterprise.
- LOCATION: The location of your data store. This should be the same location as your Agentspace Enterprise instance.
  
  Important: For locations other than global, replace discoveryengine with the location-specific path. For more information, see Agentspace Enterprise locations.
- DATA_STORE_ID: the ID of the data store that you want to create. This ID can contain only lowercase letters, digits, underscores, and hyphens.
- DATA_STORE_NAME: the display name of the data store that you want to create.
- INSTANCE_PATH: The path of your AlloyDB for PostgreSQL instance, in the format projects/.../locations/.../clusters/.../instances/....
- DATABASE_NAME: the ID of the AlloyDB for PostgreSQL database.
- DATABASE_USER_NAME: the ID of the AlloyDB for PostgreSQL database user.
- DATABASE_USER_PASSWORD: the password of the AlloyDB for PostgreSQL database user.
- NL_CONFIG_ID: the ID of the AlloyDB for PostgreSQL AI natural language configuration.
Upload the schema information of your AlloyDB for PostgreSQL database. To do so, update the data store default schema with information about the tables and columns in the database.

The following command shows an example of updating the default schema with information about tables containing movie information. For your scenario replace table_movies and table_xyz with the tables and views you want to be accessible from Agentspace Enterprise. Add the columns you want to be accessible under properties to the corresponding tables and views. Add "keyPropertyMapping": "title" and "keyPropertyMapping": "description" to columns that should be used as title and description, respectively, of retrieved rows. This is important for correct display and ranking of search results in Agentspace Enterprise. Add "keyPropertyMapping": "uri" to URL columns, if any, to make search results clickable.
```
curl -X PATCH \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
"https://discoveryengine.googleapis.com/v1beta/projects/PROJECT_ID/locations/LOCATION/collections/default_collection/dataStores/DATA_STORE_ID/schemas/default_schema" \
-d '{
  "structSchema": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "table_movies": { // table name
        "type": "object",
        "properties": { // column names
          "movie_name": { "type": "string", "keyPropertyMapping": "title" },
          "movie_description": { "type": "string", "keyPropertyMapping": "description" },
          "movie_url": { "type": "string", "keyPropertyMapping": "uri" },
          "movie_year": { "type": "integer" }
        }
      },
      "table_xyz": { // another table
        ...
      }
    }
  }
}'
```
Replace the following:
- PROJECT_ID: the ID of your project where you use Agentspace Enterprise.
- LOCATION: the location of your data store. This should be the same location as your Agentspace Enterprise instance.
  
  Important: For locations other than global, replace discoveryengine with the location-specific path. For more information, see Agentspace Enterprise locations.
- DATA_STORE_ID: the ID of the data store.

Update the UI configuration so that search results from AlloyDB for PostgreSQL are displayed correctly in Agentspace Enterprise.

Run the following command.

curl -X PATCH \
-H "X-Goog-User-Project: PROJECT_ID" \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
https://discoveryengine.googleapis.com/v1alpha/projects/PROJECT_ID/locations/LOCATION/collections/default_collection/engines/AGENTSPACE_ID/widgetConfigs/default_search_widget_config?updateMask=uiSettings -d '{
  "uiSettings": {
    "dataStoreUiConfigs": [
      {
        "name": "projects/PROJECT_ID/locations/LOCATION/collections/default_collection/dataStores/DATA_STORE_ID",
        "id": "DATA_STORE_ID",
        "fieldsUiComponentsMap": {
          "title": {
            "field": "title",
            "displayTemplate": "{value}"
          },
          "text1": {
            "field": "description",
            "displayTemplate": "{value}"
          },
          "url": {
            "field": "url",
            "displayTemplate": "{value}"
          }
        }
      }
    ],
    "interactionType": "SEARCH_ONLY"
  }
}'

Replace the following:

PROJECT_ID: the ID of your project where you use Agentspace Enterprise.
LOCATION: The location of your data store. This should be the same location as your Agentspace Enterprise instance.

Important: For locations other than global, replace discoveryengine with the location-specific path. For more information, see Agentspace Enterprise locations.
DATA_STORE_ID: the ID of the data store.
AGENTSPACE_ID: the ID of your Agentspace Enterprise instance.

Next steps

To attach your data store to an app, create an app and select your data store following the steps in Create an app.
To preview how your search results appear after your app and data store are set up, see Preview search results.