Cloud Composer 1 is in the post-maintenance mode. Google does not release any further updates to Cloud Composer 1, including new versions of Airflow, bugfixes, and security updates. We recommend planning migration to Cloud Composer 2.

Troubleshoot PyPI package installation
Stay organized with collections Save and categorize content based on your preferences.

Cloud Composer 3 | Cloud Composer 2 | Cloud Composer 1

This page explains how to troubleshoot dependency conflicts when installing custom PyPI packages.

The most common problem with PyPI packages that you might encounter in Cloud Composer are dependency conflicts.

When you specify a new custom PyPI package to install, this package or its version might cause dependency conflicts with other custom PyPI packages or preinstalled packages in your environment. In this case, the environment update operation fails with an error.

Cloud Build and in-cluster build errors

If a problem with packages occurs, you get the error message about it in two ways, depending on the way your environment is deployed:

Cloud Build. An error message and a link to the Cloud Build log. For example:

UPDATE operation on this environment failed 25 minutes ago with the
following error message:
Failed to install PyPI packages.  Check the Cloud Build log at ...

In-cluster builds. An error message and the location of build logs. For example:

UPDATE operation on this environment failed 17 minutes ago with the
following error message:
Failed to install PyPI packages.  Check the in-cluster build logs for
details. They can be found in the Environment logs under the build-log-*
log name.

See the detailed error message

When a package installation fails, pip reports the detailed error message. You can find this error message in the build logs.

Locate the `pip` error in Cloud Build logs

You can follow the link from the Cloud Build error message, or locate build logs:

Locate build logs:
1. In Google Cloud console, go to the Build history page.
  
  Go to Build history
2. Select the failed build to see its logs.

In the build log, find the error message from pip. For example:

ERROR: apache-airflow-backport-providers-google 2021.2.5 has requirement
google-cloud-logging<3.0.0,>=2.1.1, but you'll have google-cloud-logging
1.15.0 which is incompatible.

Locate the `pip` error in in-cluster build logs

Locate build logs:
1. In Google Cloud console, go to the Environments page.
  
  Go to Environments
2. Select your environment.
3. Open the Logs tab.
4. Select All logs > Composer logs > Builds > Worker & Scheduler image.
5. In the Severity drop-down list, select Info.

In the displayed logs:

Locate the installer.sh error message. For example:

The command '/bin/sh -c bash installer.sh $COMPOSER_PYTHON_VERSION
fail' returned a non-zero code: 1

The preceding info messages have the detailed pip error. For example:

apache-airflow-backport-providers-google 2021.2.5 has requirement
google-cloud-logging<3.0.0,>=2.1.1, but you have
google-cloud-logging 1.15.0.

Timeouts during Python package installation

In recent versions, the pip utility changed its behavior when attempting to satisfy dependencies. If specified dependencies can't be satisfied, pip goes through all available versions of a package.

When you install a package, the following happens:

If the package is already preinstalled in your environment and Python dependencies that you specify don't require any changes, the package remains as it is.

Example: aiodebug==2.3.0 is installed and you specify aiodebug or aiodebug>=2.
If the package isn't already installed in your environment or it doesn't satisfy Python dependencies you specify, then pip tries to use the latest version that satisfies dependencies. This is either the latest version of the package if you didn't specify a version, or the latest version of the package that satisfies the specified restrictions.

Example: If you specify aiodebug, then the latest will be installed (2.*.*). If you specify aiodebug<2, then 1.*.* will be installed.

If this version causes dependency conflicts with preinstalled packages, pip goes through all available versions of the package to try to satisfy the specified dependency. When it happens, the following messages will be visible in the build log:
```
INFO: pip is looking at multiple versions of PYTHON_PACKAGE_NAME
to determine which version is compatible with other requirements.
This could take a while.
```
Before the change, pip failed with a dependency conflict without checking other versions.

Going through all available versions can take a significant amount of time and lead to a downgrade of packages or to a timeout during the Python package installation.

Preinstalled packages in Cloud Composer are regularly updated to the latest versions for security reasons. Therefore, a timeout can also occur when you upgrade an environment to a later Cloud Composer version.

Possible solutions:

Regularly update custom Python dependencies.
Use PythonVirtualenvOperator to isolate code snippets that require conflicting packages.

Conflicts with preinstalled PyPI packages

Some package conflicts occur between custom PyPI packages that you install and the preinstalled packages.

You can see the full list of preinstalled packages for your version of Cloud Composer on the Cloud Composer versions page.

To solve this problem, you can:

Install a different version of the custom PyPI package.
Install a different version of the preinstalled package. To do so, install a custom PyPI package with the name of the preinstalled package and specify the required version. We do not recommend downgrading preinstalled packages.
See if later Cloud Composer versions use a different version of the preinstalled package. You can check for potential PyPI package conflicts before you upgrade your environment to a later version of Cloud Composer.
Use PythonVirtualenvOperator to isolate code snippets that require conflicting packages.

Metadata server unavailable when installing PyPI packages

If your environment uses a custom pip.conf file, then you might face a problem when no new PyPI packages can be installed in your environment. In Cloud Build logs related to the package installation error, you can see the following warning message:

WARNING: Compute Engine Metadata server unavailable on attempt 3 of 3. Reason:
timed out
WARNING: Authentication failed using Compute Engine authentication due to
unavailable metadata server.

This problem is caused by a pip.conf file that does not allow access to the default package index at http://pypi.org/simple. For example, if your pip.conf file only permits installing packages from a custom Artifact Registry repository because of the redefined index-url parameter, installing packages from the default package index is not available.

To solve this problem, make sure that http://pypi.org/simple package index is added to your pip.conf file. For example, if your primary package index is defined in the index-url parameter, add the http://pypi.org/simple index in the extra-index-url parameter.