Build LLM-powered applications using LlamaIndex

This page describes some use cases for building LLM-powered applications using LlamaIndex integrated with Cloud SQL for PostgreSQL. Links to notebooks on GitHub are provided to help you explore approaches or to help you develop your application.

LlamaIndex is a generative AI orchestration framework that lets you connect and integrate data sources with large language models (LLMs). You can use LlamaIndex to build applications that access and query information from private or domain-specific data using natural language queries.

LlamaIndex acts as a bridge between custom data and LLMs, facilitating the development of knowledge assistant applications with retrieval-augmented generation (RAG) capabilities.

LlamaIndex is well suited for document-centric applications because it emphasizes structured document management, which simplifies indexing and retrieval. This framework features optimized query mechanisms that enhance the speed and relevance of information access, along with robust metadata handling for nuanced filtering.

For more information about the LlamaIndex framework, see the LlamaIndex product documentation.

LlamaIndex components

Cloud SQL for PostgreSQL offers the following LlamaIndex interfaces:

  • Vector Store
  • Document Store
  • Index Stores
  • Chat Stores
  • Document Reader

Learn how to use LlamaIndex with the Quickstart for Cloud SQL for PostgreSQL.

Vector Store

This LlamaIndex integration lets you use the robust and scalable nature of Cloud SQL for PostgreSQL to store and manage your LlamaIndex data. By combining LlamaIndex's indexing and querying capabilities with Cloud SQL for PostgreSQL's high performance and reliability, you can build more efficient and scalable LLM-powered applications.

LlamaIndex breaks down a document —doc, text, and PDFs— into document components called nodes. The VectorStore can only contain the embedding vectors of ingested node contents and the text of nodes. A node, which is a first class concept, contains text content, vector embeddings, and metadata. You can apply filters on these metadata fields to restrict node retrieval to those that match specified metadata criteria.

To work with vector stores in Cloud SQL for PostgreSQL, use the PostgresVectorStore class. For more information, see LlamaIndex Vector Stores.

Store vector embeddings with the PostgresVectorStore class

The Cloud SQL for PostgreSQL for vector store shows you how to do the following:

  • Initialize a table to store vector embeddings
  • Create an embedding class instance using any Llama Index embeddings model.
  • Initialize a default PostgresVectorStore vector store.
  • Create and query an index from the vector store by using VectorStoreIndex.
  • Create a custom Vector Store to effectively store and filter metadata.
  • Add an ANN index to improve search latency.

Document and Index Stores

LlamaIndex Document Stores integration manages structured document storage and retrieval, optimizing for LlamaIndex document-centric capabilities. Document Store stores the content related to the vectors in the vector store.

For more information, see the LlamaIndex Document Stores product documentation.

Index Stores facilitate the management of indexes to enable rapid querying and data retrieval, for example, summary, keyword, and Tree index. Index in LlamaIndex is a lightweight storage only for the node metadata. Updates to node metadata don't require re-indexing (read embedding generation) of the full node or all nodes in a document.

For more information, see LlamaIndex Index Stores.

Store documents and indexes

The Cloud SQL for PostgreSQL notebook for Document Stores shows you how to use Cloud SQL for PostgreSQL to store documents and indexes using the PostgresDocumentStore and PostgresIndexStore classes. You learn how to do the following:

  • Create a PostgresEngine using PostgresEngine.from_instance().
  • Create tables for the DocumentStore and IndexStore.
  • Initialize a default PostgresDocumentStore.
  • Set up a PostgresIndexStore.
  • Add documents to the Docstore.
  • Use Document Stores with multiple indexes.
  • Load existing indexes.

Chat Stores

Chat Stores maintain conversation history and context for chat-based applications, enabling personalized interactions. Chat Stores provide a central repository that stores and retrieves chat messages within a conversation, allowing the LLM to maintain context and provide more relevant responses based on the ongoing dialog.

Large language models are stateless by default, which means that they don't retain previous inputs unless those inputs are explicitly provided each time. By using a chat store, you can preserve the context of the conversation, which lets the model generate more relevant and coherent responses over time.

The memory module in LlamaIndex enables efficient storage and retrieval of conversational context, allowing for more personalized and context-aware interactions in chat applications. You can integrate the memory module in LlamaIndex with a ChatStore and a ChatMemoryBuffer.
For more information, see LlamaIndex Chat Stores.

Store chat history

The Cloud SQL for PostgreSQL notebook for Chat Stores shows you how to use Cloud SQL for PostgreSQL to store chat history using the PostgresChatStore class. You learn how to do the following:

  • Create a PostgresEngine using PostgresEngine.from_instance().
  • Initialize a default PostgresChatStore.
  • Create a ChatMemoryBuffer.
  • Create an LLM class instance.
  • Use the PostgresChatStore without a storage context.
  • Use the PostgresChatStore with a storage context.
  • Create and use the Chat Engine.

Document Reader

Document Reader efficiently retrieves and transforms data from Cloud SQL for PostgreSQL into LlamaIndex-compatible formats for indexing. The Document Reader interface provides methods to load data from a source as Documents. Document is a class that stores a piece of text and associated metadata. You can use Document Readers to load documents that you want to store in Document Stores or used to create Indexes.

For more information, see LlamaIndex Document Reader.

Retrieve data as documents

The Cloud SQL for PostgreSQL notebook for Document Reader shows you how to use Cloud SQL for PostgreSQL to retrieve data as documents using the PostgresReader class. You learn how to do the following:

  • Create a PostgresEngine using PostgresEngine.from_instance().
  • Create PostgresReader.
  • Load Documents using the table_name argument.
  • Load Documents using a SQL query.
  • Set page content format.
  • Load the documents.

What's next