Mantenha tudo organizado com as coleções
Salve e categorize o conteúdo com base nas suas preferências.
Neste documento, apresentamos os pipelines de conectividade gerenciada usados para importar metadados de fontes terceirizadas para o Dataplex Universal Catalog.
Com a conectividade gerenciada, é possível importar metadados para o Dataplex Universal Catalog em grande escala. Um pipeline de conectividade gerenciada extrai metadados de fontes de dados e os importa para o Dataplex Universal Catalog. Se necessário, o pipeline também cria grupos de entrada do Dataplex Universal Catalog no projeto doGoogle Cloud . É possível orquestrar os fluxos de trabalho e agendar os jobs de importação de acordo com os requisitos.
O pipeline de conectividade gerenciada faz o seguinte:
Cria um grupo de entrada de destino com base na configuração informada, se ele ainda não existir.
Executa o conector. O conector extrai os metadados da fonte de dados e gera um arquivo para importação no Dataplex Universal Catalog.
Monitora o progresso da extração de metadados.
Executa um job para importar os metadados para o Dataplex Universal Catalog.
Monitora o progresso do job de importação de metadados.
O pipeline de conectividade gerenciada usa o Dataproc sem servidor para executar o conector. Também usa os métodos da API de importação de metadados do Dataplex Universal Catalog para executar o job de importação.
Uma opção para importar metadados de fontes de terceiros é usar os conectores personalizados da comunidade. O arquivo README de cada conector contém as instruções de configuração e mais informações sobre ele.
[[["Fácil de entender","easyToUnderstand","thumb-up"],["Meu problema foi resolvido","solvedMyProblem","thumb-up"],["Outro","otherUp","thumb-up"]],[["Difícil de entender","hardToUnderstand","thumb-down"],["Informações incorretas ou exemplo de código","incorrectInformationOrSampleCode","thumb-down"],["Não contém as informações/amostras de que eu preciso","missingTheInformationSamplesINeed","thumb-down"],["Problema na tradução","translationIssue","thumb-down"],["Outro","otherDown","thumb-down"]],["Última atualização 2025-08-08 UTC."],[[["\u003cp\u003eManaged connectivity pipelines facilitate the import of metadata from various third-party sources into Dataplex at scale.\u003c/p\u003e\n"],["\u003cp\u003eUsers can create custom connectors to extract metadata from sources like MySQL, SQL Server, Oracle, Snowflake, and Databricks.\u003c/p\u003e\n"],["\u003cp\u003eThe managed connectivity pipeline uses Dataproc Serverless to run connectors and imports the metadata into Dataplex Catalog.\u003c/p\u003e\n"],["\u003cp\u003eThe pipeline automatically creates Dataplex Catalog entry groups if they don't exist, and then runs the connector to extract and import metadata.\u003c/p\u003e\n"],["\u003cp\u003eThe workflows used in the pipelines allow for the scheduling and orchestration of metadata import jobs based on user needs.\u003c/p\u003e\n"]]],[],null,["# Managed connectivity overview\n\nThis document provides an overview of the managed connectivity pipelines that\nyou can use to import metadata from third-party sources into Dataplex Universal Catalog.\n\nManaged connectivity lets you import metadata into Dataplex Universal Catalog at\nscale. A managed connectivity pipeline extracts metadata from your data sources\nand then imports the metadata into Dataplex Universal Catalog. If necessary, the pipeline\nalso creates Dataplex Universal Catalog entry groups in your\nGoogle Cloud project. You can orchestrate the workflows, and schedule the\nimport jobs based on your requirements.\n\nYou build your own custom connectors to extract metadata from third-party\nsources. For example, you can build a connector to extract metadata from sources\nlike MySQL, SQL Server, Oracle, Snowflake, Databricks, and others. For steps to build\na sample custom connector, see\n[Develop a custom connector for metadata import](/dataplex/docs/develop-custom-connector).\nYou can also use the\n[community-contributed custom connectors](#community-contributed-connectors)\nthat are available for a variety of third-party sources.\n\nFor steps to run a managed connectivity pipeline, see\n[Import metadata from a custom source using Workflows](/dataplex/docs/import-using-workflows-custom-source).\n\nHow managed connectivity works\n------------------------------\n\nThe following diagram shows a managed connectivity pipeline.\n\nAt a high level, here's how managed connectivity works:\n\n1. You\n [build a connector for your data source](/dataplex/docs/develop-custom-connector).\n\n The connector must be an Artifact Registry image that can run on\n Dataproc Serverless.\n2. You [run the managed connectivity pipeline](/dataplex/docs/import-using-workflows-custom-source)\n in Workflows, an orchestration platform.\n\n3. The managed connectivity pipeline does the following things:\n\n 1. Creates a target entry group based on your configuration, if the entry group doesn't exist yet.\n 2. Runs the connector. The connector extracts the metadata from your data source and generates a metadata import file that can be imported into Dataplex Universal Catalog.\n 3. Monitors the progress of the metadata extraction.\n 4. Runs a metadata import job to import the metadata into Dataplex Universal Catalog.\n 5. Monitors the progress of the metadata import job.\n\nThe managed connectivity pipeline uses Dataproc Serverless to run the\nconnector, and Dataplex Universal Catalog metadata import API methods\nto run the metadata import job.\n\nThe metadata that you import consists of Dataplex Universal Catalog\nentries and their aspects. For more information about\nDataplex Universal Catalog metadata, see\n[About metadata management in Dataplex Universal Catalog](/dataplex/docs/catalog-overview).\n\nCommunity-contributed custom connectors\n---------------------------------------\n\nTo import metadata from third-party sources, you can use custom connectors that\nare contributed by the community. See each connector's README file for setup\ninstructions and more information about the connector.\n| **Note:** These connectors are not officially supported by Google.\n\nWhat's next\n-----------\n\n- [Import metadata from a custom source using Workflows](/dataplex/docs/import-using-workflows-custom-source)\n- [Develop a custom connector for metadata import](/dataplex/docs/develop-custom-connector)\n- [Import metadata using a custom pipeline](/dataplex/docs/import-metadata)"]]
