Tetap teratur dengan koleksi
Simpan dan kategorikan konten berdasarkan preferensi Anda.
Setiap operasi pipeline yang dibuat menggunakan Vertex AI Pipelines memiliki beberapa artefak dan parameter terkait, seperti model, set data, template pipeline, dan komponen. Silsilah artefak pipeline mencakup faktor-faktor yang
berkontribusi pada pembuatannya, serta artefak dan metadata yang berasal dari
artefak tersebut. Misalnya, silsilah model dapat mencakup hal berikut:
Data pelatihan, pengujian, dan evaluasi yang digunakan untuk membuat model.
Hyperparameter yang digunakan selama pelatihan model.
Metadata yang dikumpulkan dari proses pelatihan dan evaluasi, seperti akurasi model.
Artefak yang berasal dari model ini, seperti hasil prediksi batch.
Anda dapat menggunakan metadata ini untuk membantu menjawab pertanyaan seperti berikut:
Mengapa operasi pipeline tertentu menghasilkan model yang sangat akurat?
Operasi pipeline mana yang menghasilkan model paling akurat, dan hyperparameter apa yang digunakan untuk melatih model tersebut?
Bergantung pada langkah-langkah dalam pipeline, Anda mungkin dapat menjawab pertanyaan tata kelola sistem. Misalnya, Anda dapat menggunakan metadata untuk menentukan versi model mana yang berada dalam produksi pada titik waktu tertentu.
Untuk melihat dan menganalisis silsilah artefak pipeline, Anda dapat menggunakan Vertex ML Metadata atau Katalog Universal Dataplex.
Tabel berikut menguraikan perbedaan antara Vertex ML Metadata dan Dataplex Universal Catalog:
Fitur
Vertex ML Metadata
Katalog Universal Dataplex
Jenis metadata pipeline yang diambil
Semua artefak input dan output yang dihasilkan oleh proses pipeline.
Artefak input dan output yang dapat dipetakan ke nama yang sepenuhnya memenuhi syarat (FQNs) yang didukung oleh Dataplex Universal Catalog, umumnya dengan menggunakan Google Cloud Komponen Pipeline.
Geografi
Pembacaan satu region.
Pembacaan global, yaitu di beberapa region.
Project
Pembacaan project tunggal.
Pembacaan di seluruh organisasi di beberapa project.
Layanan terintegrasi
Terintegrasi dengan Vertex AI Pipelines, Vertex AI Experiments, Vertex AI Model Registry, dan Dataset.
Terintegrasi dengan beberapa produk, seperti Vertex AI, BigQuery, Cloud Composer, dan Dataproc. Google Cloud
Ikut serta?
Tidak, selalu aktif.
Ikut serta per project dengan mengaktifkan Data Lineage API.
Memetakan artefak Vertex ML Metadata ke Katalog Universal Dataplex
Untuk memetakan artefak Vertex ML Metadata ke FQN di Dataplex Universal Catalog,
Anda harus melakukan hal berikut:
Gunakan Google Cloud Komponen Pipeline saat membuat model Vertex AI dan
set data terkelola.
Gunakan judul skema kustom (google.VertexDataset atau google.VertexModel)
saat menentukan nama resource model atau set data terkelola di kolom metadata, seperti yang diilustrasikan dalam contoh berikut:
Menganalisis silsilah artefak pipeline menggunakan Vertex ML Metadata
Saat Anda menjalankan pipeline menggunakan Vertex AI Pipelines, artefak dan parameter dari pipeline yang dijalankan akan disimpan menggunakan Vertex ML Metadata.
Vertex ML Metadata memudahkan analisis silsilah artefak pipeline Anda, dengan memudahkan Anda melacak metadata pipeline Anda.
Halaman Metadata mencantumkan artefak yang telah dibuat di penyimpanan metadata default.
Di menu drop-down Region, pilih region tempat proses Anda dibuat.
Klik Display name artefak untuk melihat grafik silsilahnya.
Grafik statis yang menunjukkan artefak dan eksekusi yang merupakan bagian dari grafik silsilah ini akan muncul.
Klik artefak atau eksekusi untuk mempelajari lebih lanjut.
Menganalisis silsilah artefak pipeline menggunakan Katalog Universal Dataplex
Dataplex Universal Catalog menemukan metadata dari
Google Cloud resource, yang mencakup artefak Vertex AI Pipelines
seperti model Vertex AI, set data terkelola, dan
Google Cloud resource lain yang dapat ditemukan di Dataplex Universal Catalog. Anda dapat
menemukan artefak ini menggunakan kemampuan penelusuran metadata
Dataplex Universal Catalog dan melihat diagram silsilahnya.
Perhatikan bahwa Katalog Universal Dataplex mungkin tidak tersedia di semua region tempat Vertex AI Pipelines didukung. Jika Dataplex Universal Catalog tidak didukung di region Anda, gunakan Vertex ML Metadata.
Lihat daftar region yang didukung untuk Dataplex Universal Catalog.
Ikuti petunjuk berikut untuk melihat grafik silsilah untuk artefak pipeline di Katalog Universal Dataplex:
Untuk meluncurkan kueri penelusuran Dataplex Universal Catalog di konsol Google Cloud , buka halaman Penelusuran Dataplex Universal Catalog.
Gunakan filter untuk menelusuri artefak. Misalnya, Anda dapat menggunakan filter
Jenis data untuk menentukan jenis artefak, seperti model, set data,
atau tabel BigQuery. Untuk mengetahui informasi selengkapnya, lihat Menelusuri resource di Katalog Universal Dataplex.
Untuk melihat silsilah artefak, klik nama artefak, lalu klik tab Silsilah.
Pada grafik silsilah, proses Vertex AI didahului oleh
.
Hal ini mencakup artefak pipeline, komponen pipeline, dan template pipeline.
Untuk melihat detail proses, klik proses dalam grafik silsilah.
Untuk proses berdasarkan tugas pipeline dari proses pipeline, Anda dapat melakukan hal berikut:
Lihat operasi pipeline di Vertex AI dengan mengklik Open in Vertex AI di tab Details. Untuk melihat detail runtime dari operasi pipeline, seperti status, stempel waktu, dan atribut, klik Lainnya. Untuk melihat eksekusi pipeline di Vertex AI, klik Open in Vertex AI.
Untuk proses berdasarkan template pipeline, Anda dapat melakukan hal berikut:
Lihat detail template di Vertex AI dengan mengklik Open in Vertex AI di tab Details.
Lihat daftar tugas pipeline yang dibuat dalam operasi pipeline di tab Runs. Untuk melihat detail template pipeline di
Vertex AI, klik Lainnya, lalu klik
Buka di Vertex AI.
[[["Mudah dipahami","easyToUnderstand","thumb-up"],["Memecahkan masalah saya","solvedMyProblem","thumb-up"],["Lainnya","otherUp","thumb-up"]],[["Sulit dipahami","hardToUnderstand","thumb-down"],["Informasi atau kode contoh salah","incorrectInformationOrSampleCode","thumb-down"],["Informasi/contoh yang saya butuhkan tidak ada","missingTheInformationSamplesINeed","thumb-down"],["Masalah terjemahan","translationIssue","thumb-down"],["Lainnya","otherDown","thumb-down"]],["Terakhir diperbarui pada 2025-08-19 UTC."],[],[],null,["# Track the lineage of pipeline artifacts\n\nEach pipeline run created using Vertex AI Pipelines has several associated\nartifacts and parameters, such as models, datasets, pipeline templates, and\ncomponents. The lineage of a pipeline artifact includes the factors that\ncontributed to its creation, as well as artifacts and metadata derived from\nthe artifact. For example, a model's lineage can include the following:\n\n- The training, test, and evaluation data used to create the model.\n\n- The hyperparameters used during model training.\n\n- Metadata recorded from the training and evaluation process, such as the model's accuracy.\n\n- Artifacts that descend from this model, such as the results of batch predictions.\n\nYou can use this metadata to help answer questions like the following:\n\n- Why did a certain pipeline run produce an especially accurate model?\n\n- Which pipeline run produced the most accurate model, and what hyperparameters were used to train the model?\n\n- Depending on the steps in your pipeline, you might be able to answer system\n governance questions. For example, you could use metadata to determine which\n version of your model was in production at a given point in time.\n\nTo view and analyze the pipeline artifact lineage, you can use either Vertex ML Metadata or Dataplex Universal Catalog.\n\nThe following table outlines the differences between Vertex ML Metadata and Dataplex Universal Catalog:\n\nMap Vertex ML Metadata artifacts to Dataplex Universal Catalog\n--------------------------------------------------------------\n\nTo map Vertex ML Metadata artifacts to FQNs in Dataplex Universal Catalog,\nyou need to do the following:\n\n- Use Google Cloud Pipeline Components while creating Vertex AI models and\n managed datasets.\n\n- Use custom schema titles (`google.VertexDataset` or `google.VertexModel`)\n while specifying the model or managed dataset resource name in the `metadata`\n field, as illustrated in the following sample:\n\n {\n \"name\": \"projects/example-project/locations/us-central1/metadataStores/default/artifacts/example-artifact\",\n \"displayName\": \"My dataset\",\n \"uri\": \"https://us-central1-aiplatform.googleapis.com/v1/projects/example-project/locations/us-central1/datasets/example-dataset\",\n ...\n \"schemaTitle\": \"google.VertexDataset\",\n \"schemaVersion\": \"0.0.1\",\n \"metadata\": {\n \"resourceName\": \"projects/example-project/locations/us-central1/datasets/example-dataset\"\n }\n }\n\nAnalyze the lineage of pipeline artifacts using Vertex ML Metadata\n------------------------------------------------------------------\n\nWhen you run a pipeline using Vertex AI Pipelines, the artifacts and\nparameters of your pipeline run are stored using Vertex ML Metadata.\nVertex ML Metadata makes it easier to analyze the *lineage* of your\npipeline's artifacts, by saving you the difficulty of keeping track of your\npipeline's metadata.\n\nIf you're new to Vertex ML Metadata, read the [introduction to\nVertex ML Metadata](/vertex-ai/docs/ml-metadata/introduction).\n| For a step-by-step tutorial on analyzing artifacts and metadata generated across your Vertex AI Pipelines executions, see the [Using Vertex ML Metadata with Vertex AI Pipelines](https://codelabs.developers.google.com/vertex-mlmd-pipelines#0) codelab.\n\nFollow these instructions to view the lineage graph for a pipeline\nartifact using Vertex ML Metadata:\n\n1. In the Google Cloud console, in the Vertex AI section, go\n to the **Metadata** page.\n\n [Go to Metadata](https://console.cloud.google.com/vertex-ai/metadata)\n\n The Metadata page lists the artifacts that have been created in the\n **default** metadata store.\n2. In the **Region** drop-down list, select the region that your run was\n created in.\n\n3. Click the **Display name** of an artifact to see its lineage graph.\n\n A static graph showing the artifacts and executions that are a part of this\n lineage graph appears.\n4. Click an artifact or execution to learn more about it.\n\nAnalyze the lineage of pipeline artifacts using Dataplex Universal Catalog\n--------------------------------------------------------------------------\n\nDataplex Universal Catalog discovers metadata from\nGoogle Cloud resources, which include Vertex AI Pipelines\nartifacts like Vertex AI models, managed datasets, and other\nGoogle Cloud resources discoverable in Dataplex Universal Catalog. You can\ndiscover these artifacts using the metadata search capability of\nDataplex Universal Catalog and view their lineage graphs.\n\nFor more information about the Dataplex Universal Catalog metadata search capability,\nsee [Search for resources in Dataplex Universal Catalog](/dataplex/docs/search-assets).\n\nNote that Dataplex Universal Catalog might not be available in all regions where\nVertex AI Pipelines is supported. If Dataplex Universal Catalog is\nunsupported in your region, use Vertex ML Metadata.\n[View the list of supported regions for Dataplex Universal Catalog.](/dataplex/docs/locations)\n\nFollow these instructions to view the lineage graph for a pipeline artifact\non Dataplex Universal Catalog:\n\n1. To launch a Dataplex Universal Catalog search query in the Google Cloud console,\n go to the Dataplex Universal Catalog **Search** page.\n\n [Go to Search](https://console.cloud.google.com/dataplex/dp-search)\n2. Select **Dataplex Universal Catalog** as the search mode.\n\n3. Use the filters to search for the artifacts. For example, you can use the\n **Data types** filter to specify the type of artifact, such as model, dataset,\n or BigQuery table. For more information,\n see [Search for resources in Dataplex Universal Catalog](/dataplex/docs/search-assets).\n\n You can also [define your query in the search field](/dataplex/docs/search-syntax).\n4. To view the lineage of an artifact, click the name of the artifact, and then click the **Lineage** tab.\n\n On the lineage graph, Vertex AI processes are preceded by\n .\n These include pipeline artifacts, pipeline components, and pipeline templates.\n - To view the details of a process, click the process in the lineage graph.\n\n | **Note:** The process details are available only if the process has been catalogued in Dataplex Universal Catalog using a fully qualified name (FQN).\n - For processes based on pipeline tasks from pipeline runs, you can do the following:\n\n - View the pipeline run in Vertex AI by clicking **Open in Vertex AI** in the **Details tab** . To view the runtime details of a pipeline run, such as states, timestamps, and attributes, click **More** . To view the pipeline run in Vertex AI, click **Open in Vertex AI**.\n - For processes based on a pipeline template, you can do the following:\n\n - View the template details in Vertex AI by clicking **Open in Vertex AI** in the **Details tab**.\n\n - View the list of pipeline tasks created in pipeline runs in the\n **Runs** tab. To view the details of the pipeline template in\n Vertex AI, click **More** , and then click\n **Open in Vertex AI**.\n\nWhat's next\n-----------\n\n- Learn how to [run a pipeline](/vertex-ai/docs/pipelines/run-pipeline).\n- Get started [visualizing and analyzing pipeline\n results](/vertex-ai/docs/pipelines/visualize-pipeline).\n- Learn how to [build a machine learning pipeline](/vertex-ai/docs/pipelines/build-pipeline)."]]
.
Hal ini mencakup artefak pipeline, komponen pipeline, dan template pipeline.