Data lineage displays the relations between your project's resources and the processes that created them.
You can view data lineage information in the Google Cloud console in the following forms:
- Lineage graph: shows the lineage that is upstream or downstream of a single root entry. For more information, see Lineage graph.
- Lineage path visualization (Preview): shows the lineage links between two selected resources. For more information, see Lineage path visualization.
- Lineage list view (Preview): shows detailed lineage information for resources in a single table that includes lineage information for resources with many connections. For more information, see Lineage list view.
You can also retrieve lineage information from the Data Lineage API in the form of JSON data.
Lineage is captured across projects. When you view lineage that is generated from multiple projects, you can view the aggregated lineage information in any of the relevant projects.
Roles and permissions
To view lineage information, ask your administrator to grant you viewer roles as described in Predefined roles for data lineage. You must have access in both the project where you view lineage, and the projects in which lineage is recorded.
Dataplex Universal Catalog tracks lineage information automatically when you enable the Data Lineage API. You don't need any administrator or editor roles to capture lineage for your data assets.
For more information about granting roles, see Manage access. You can assign a role at a higher folder or organization level (see Grant or revoke a single role).
Enable data lineage
Enable data lineage to begin automatically tracking lineage information for supported systems. You must enable the Data Lineage API in both the project where you view lineage, and the projects in which lineage is recorded. For more information, see Project types.
To capture lineage information, do the following:
-
In the Google Cloud console, on the Project selector page, select the project in which you want to record lineage.
Enable the Data Lineage API.
- Repeat the previous steps for each project in which you want to record lineage.
-
In the project where you view lineage, enable the Data Lineage API and the Dataplex API.
View lineage in Dataplex Universal Catalog
You can view data lineage information in the Dataplex Universal Catalog web interface.
To view the lineage, follow these instructions:
In the Google Cloud console, go to the Dataplex Universal Catalog Search page.
Select Dataplex Catalog as the search mode.
Search for the entry for which you want to view lineage information. For more information, see Search for resources in Dataplex Universal Catalog.
On the entry details page, select the Lineage tab.
To view the lineage graph, click Graph.
Select the process or data source buttons to display the details panel.
To view upstream or downstream lineage information for a resource, click Expand.
To view lineage in list view, click List.
To view the lineage path visualization, click List, and then select the target resource in the table of results. In the details panel, click Target, and then click Visualize lineage.
View lineage in BigQuery
You can view data lineage information in the BigQuery web interface.
To view the lineage, follow these instructions:
- In the Google Cloud console, go to the BigQuery page.
Open the table for which you want to see the data lineage.
Click the Lineage tab.
To view the lineage graph, click Graph.
Select the process or data source buttons to display the details panel.
To view upstream or downstream lineage information for a resource, click Expand.
To view lineage in list view, click List.
To view the lineage path visualization, click List, and then select the target resource in the table of results. In the details panel, click Target, and then click Visualize lineage.
View lineage in Vertex AI
Systems like Vertex AI Pipelines generate lineage data for Vertex AI models and datasets. You can view data lineage information in the Vertex AI UI web interface.
View lineage for a managed dataset in Vertex AI
To view the lineage for a dataset, follow these instructions:
- In the Google Cloud console, go to the Datasets page.
Click the dataset for which you want to see the data lineage.
Click the Lineage tab.
To view the lineage graph, click Graph.
Select the process or data source buttons to display the details panel.
To view upstream or downstream lineage information for a resource, click Expand.
To view lineage in list view, click List.
To view the lineage path visualization, click List, and then select the target resource in the table of results. In the details panel, click Target, and then click Visualize lineage.
View lineage for a model in Vertex AI
To view the lineage for a model, follow these instructions:
In the Google Cloud console, go to the Model Registry page.
Click the model for which you want to see the data lineage.
Click the Lineage tab.
To view the lineage graph, click Graph.
Select the process or data source buttons to display the details panel.
To view upstream or downstream lineage information for a resource, click Expand.
To view lineage in list view, click List.
To view the lineage path visualization, click List, and then select the target resource in the table of results. In the details panel, click Target, and then click Visualize lineage.
What's next
Track data lineage for a BigQuery table copy and query jobs.
Learn about Data lineage information model.