Create notebooks

This document describes how to create Colab Enterprise notebooks in BigQuery. Notebooks are BigQuery Studio code assets powered by Dataform.

Before you begin

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Go to project selector

3. Make sure that billing is enabled for your Google Cloud project.

4. Enable the BigQuery API.

   Enable the API

5. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Go to project selector

6. Make sure that billing is enabled for your Google Cloud project.

7. Enable the BigQuery API.

   Enable the API

Enable BigQuery Studio

Follow the instructions at Enable BigQuery Studio for asset management to save, share, and manage versions of code assets such as notebooks.

Required permissions

Set the appropriate permissions to create, edit, or view notebooks.

All users with the Dataform Admin (roles/dataform.admin) role have owner access to all notebooks created in the project.

For more information about BigQuery Identity and Access Management (IAM), see Access control with IAM.

Permissions to create notebooks

To create and run notebooks, you need the following IAM permissions:

• resourcemanager.projects.get
• resourcemanager.projects.list
• bigquery.config.get
• bigquery.jobs.create
• bigquery.readsessions.create
• bigquery.readsessions.getData
• bigquery.readsessions.update
• resourcemanager.projects.get
• resourcemanager.projects.list
• dataform.locations.get
• dataform.locations.list

• dataform.repositories.create

• dataform.repositories.list

• dataform.collections.create

• dataform.collections.list

• aiplatform.notebookRuntimeTemplates.apply

• aiplatform.notebookRuntimeTemplates.get

• aiplatform.notebookRuntimeTemplates.list

• aiplatform.notebookRuntimeTemplates.getIamPolicy

• aiplatform.notebookRuntimes.assign

• aiplatform.notebookRuntimes.get

• aiplatform.notebookRuntimes.list

• aiplatform.operations.list

You can get these permissions from the following IAM roles:

• BigQuery Read Session User (roles/bigquery.readSessionUser)

• BigQuery Studio User (roles/bigquery.studioUser)

  The BigQuery Studio User role combines the following IAM roles:

  • BigQuery Job User (roles/bigquery.jobUser)
  • Notebook Runtime User (roles/aiplatform.notebookRuntimeUser)

  • Code Creator (roles/dataform.codeCreator)

Permissions to edit notebooks

To edit and run notebooks, you need the following IAM roles:

• BigQuery Job User (roles/bigquery.jobUser)
• BigQuery Read Session User (roles/bigquery.readSessionUser)
• Notebook Runtime User (roles/aiplatform.notebookRuntimeUser)
• Code Editor (roles/dataform.codeEditor)

Permissions to view notebooks

To view and run notebooks, you need the following IAM roles:

• BigQuery Job User (roles/bigquery.jobUser)
• BigQuery Read Session User (roles/bigquery.readSessionUser)
• Notebook Runtime User (roles/aiplatform.notebookRuntimeUser)
• Code Viewer (roles/dataform.codeViewer)

Create notebooks

Use the following sections to learn how to create a notebook.

Set the default region for code assets

If this is the first time you are creating a code asset, set the default region for code assets. You can't change the region for a code asset after it is created.

Create a notebook from the BigQuery editor

To create a notebook containing a default query for a given table, follow these steps:

1. Go to the BigQuery page.

   Go to BigQuery

2. In the tab bar of the editor pane, click the arrow_drop_down arrow drop down next to the + sign and then click Create Python notebook.

   The new notebook opens, containing cells that show example queries against the bigquery-public-data.ml_datasets.penguins public dataset.

Create a notebook from a table

To create a notebook containing a default query for a specific table, follow these steps:

1. Go to the BigQuery page.

   Go to BigQuery

2. In the Explorer pane, expand your project and find the table that you want to query.

3. Click more_vert View actions next to the table, click Query in > Python notebook.

   The new notebook opens, containing cells that show example queries against the selected table.

Create a notebook to explore the result set of a query

To create a notebook to explore the result set of a query, follow these steps:

1. Go to the BigQuery page.

   Go to BigQuery

2. In the editor pane, run a query that generates a query result.

3. In the Query results pane, click Explore Data and then click Explore with Python notebook.

   The new notebook opens, containing cells with code to return the query SQL and the query results.

Create a notebook from an existing notebook

To open any version of an existing notebook as a new notebook, follow these steps:

1. In the Google Cloud console, go to the BigQuery page.

   Go to BigQuery

2. In the Explorer pane, expand your project and the Notebooks folder, and if necessary, the Shared notebooks folder. Select a notebook.

3. Select the Activity pane.

4. Click more_vert View actions next to a notebook version and then click Open as new Python notebook.

   A copy of the notebook is opened as a new notebook.

Upload notebooks

You can upload a local notebook to use it in BigQuery Studio. The uploaded notebook is then visible in the BigQuery page of the Google Cloud console.

To upload a notebook, follow these steps:

1. In the Google Cloud console, go to the BigQuery page.

   Go to BigQuery

2. In the Explorer pane, expand your project, and then do one of the following:

   • Next to Notebooks, click more_vert View actions > Upload to Notebooks.
   • Next to the Google Cloud project name, click more_vert View actions > Upload to project > Python notebook.

3. In the Upload Notebook dialog, in the Notebook field, click Browse, and then select the notebook that you want to upload.

4. Optional: In the Notebook name field, edit the name of the notebook.

5. In the Region field, select the region where you want to upload your notebook.

6. Click Upload.

Your notebook appears in the Explorer pane.

Connect to a runtime

Use the following sections to learn how to connect a notebook to a Vertex AI runtime. A runtime is a compute resource that runs the code in your notebook.

For more information about runtimes, see Runtimes and runtime templates.

Connect to the default runtime

The default runtime is a preset runtime that requires minimal setup.

To connect to the default runtime, follow these steps:

1. In the Google Cloud console, go to the BigQuery page.

   Go to BigQuery

2. In the Explorer pane, expand your project and the Notebooks folder, and if necessary, the Shared notebooks folder. Click the name of a notebook to open it.

3. In the notebook, click Connect, or run any cell in the notebook.

   It might take several minutes to connect to the default runtime if you don't already have an active runtime.

Connect to a non-default runtime

If you want to use a runtime other than the default runtime, you must first create that additional runtime in Vertex AI.

To connect to non-default runtime, follow these steps:

1. In the Google Cloud console, go to the BigQuery page.

   Go to BigQuery

2. In the Explorer pane, expand your project and the Notebooks folder, and if necessary, the Shared notebooks folder. Click the name of a notebook to open it.

3. In the notebook, click the arrow_drop_down drop-down next to Connect and then click Connect to a runtime.

4. Click Connect to an existing runtime.

5. In Runtimes, select the runtime to use.

6. Click Connect.

Connect to a new runtime

To connect to a new runtime, follow these steps:

1. In the Google Cloud console, go to the BigQuery page.

   Go to BigQuery

2. In the Explorer pane, expand your project and the Notebooks folder, and if necessary, the Shared notebooks folder. Click the name of a notebook to open it.

3. In the notebook, click the arrow_drop_down drop-down next to Connect and then click Connect to a runtime.

4. Click Create new runtime.

5. In Runtime Template, select the Vertex AI runtime template to use.

6. In Runtime name, type a name for the runtime.

7. Click Connect.

Grant access to notebooks

To grant other users access to a notebook, add those users to an appropriate IAM role.

1. In the Google Cloud console, go to the BigQuery page.

   Go to BigQuery

2. In the Explorer pane, expand your project and the Notebooks folder, and if necessary, the Shared notebooks folder. Find the notebook that you want to grant access to.

3. Click more_vert View actions next to the notebook, and then click Share > Manage Permissions.

4. In the Manage permissions pane, click Add user/group.

5. In the New principals field, enter a principal.

6. In the Role list, select one of the following roles:

   • Code Owner: Can perform any action on the notebook, including deleting or sharing it.
   • Code Editor: Can edit the notebook.
   • Code Viewer: Can view the notebook.

7. Optional: To view a complete list of roles and advanced sharing settings, click Advanced sharing.

8. Click Save.

9. To return to the notebook information page, click Close.

Share notebooks

To share a notebook with other users, you can generate and share a link to the notebook. For other users to see the notebook you share, you must first grant access to the notebook.

To run a notebook, users must have access to the data that the notebook accesses. For more information, see Grant access to a dataset.

1. In the Google Cloud console, go to the BigQuery page.

   Go to BigQuery

2. In the Explorer pane, expand your project and the Notebooks folder, and if necessary, expand the Shared notebooks folder. Find the notebook that you want to share.

3. Click more_vert View actions next to the notebook, and then click Share > Copy link.

4. Share the link with other users.

Disable notebook output saving

You can prevent sharing saved notebook output with other users who have access to the notebook file by disabling notebook output saving.

When you disable output saving for a selected notebook, BigQuery deletes all output saved in the notebook file and doesn't save the output of subsequent runs.

However, users who have access to the notebook can still view its output in the following ways:

• Run the notebook to view its current output. This output is not saved.
• View an archival version of the notebook and its output in revision history.

To disable saving of output for a selected notebook, follow these steps:

1. In the Google Cloud console, go to the BigQuery page.

   Go to BigQuery

2. In the Explorer pane, expand your project and the Notebooks folder, and then select the notebook for which you want to disable saving output.

3. Click arrow_drop_down Toggle header visibility > Edit > Notebook settings.

4. In the Notebook settings window, select Omit code cell output when saving this notebook.

5. Click Save.

6. Click Reload.

Resolve conflicts

If you and another user make conflicting changes in a notebook, the service raises the error Automatic saving failed. This file was updated remotely or in another tab. and provides a Show diff link. To resolve the conflict, follow these steps:

1. Click the Show diff link. The Review remote changes dialog opens.
2. Optional: To compare the notebook source code, select the Raw source checkbox.
3. Optional: To compare the versions inline instead of in separate panes, select the Inline diff checkbox.
4. Review the changes and decide which to keep, revising your input if necessary.
5. Click Save your changes.

Rename notebooks

To rename a notebook, follow these steps:

1. In the Google Cloud console, go to the BigQuery page.

   Go to BigQuery

2. In the Explorer pane, expand your project and the Notebooks folder, and if necessary, the Shared notebooks folder. Find the notebook that you want to rename.

3. Click more_vert View actions next to the notebook, and then click Rename.

4. Type a name for the notebook, and then click Rename.

Troubleshooting

For more information, see Troubleshoot Colab Enterprise.

What's next

• Learn how to manage notebooks.
• Learn how to orchestrate notebooks.