Forward Cloud Storage FUSE logs to Cloud Logging

This page provides instructions on how to forward Cloud Storage FUSE logs to Cloud Logging.

Overview

Cloud Storage FUSE logs help you understand how Cloud Storage FUSE operates and allow you to troubleshoot issues and monitor performance. When you use the --log-severity CLI option or the logging:severity configuration field to generate logs, logs are generated by Cloud Storage FUSE and can be viewed as standard output or local log files on the machine where Cloud Storage FUSE is running.

You can also forward Cloud Storage FUSE logs to Cloud Logging, which provides the following benefits:

  • View all Cloud Storage FUSE logs in a single location in the Google Cloud console, which provides a holistic view of Cloud Storage FUSE behavior and makes it easier to correlate events across different components.

  • Filter Cloud Storage FUSE logs by severity level, timestamp, resource type, and other fields, which can help you pinpoint errors or performance issues.

  • Create metrics-based alerts that are triggered by specific log events. For example, you can set up an alert to notify you whenever an error or warning log message from Cloud Storage FUSE is detected, allowing you to react to problems proactively.

  • Store logs in Cloud Logging for a configurable period, and export them to Cloud Storage for long-term storage, archival, or in-depth analysis.

Pricing

For pricing information about Cloud Logging, see Cloud Logging costs.

Get required roles

To get the permissions that you need to use Cloud Logging to view Cloud Storage FUSE logs, grant the Logs Writer (roles/logging.logWriter) IAM role on the project you want to collect metrics for.

This predefined role contains the permissions required to use Cloud Logging. For more information about the Logs Writer role, see the IAM documentation about roles/logging.logWriter.

For instructions on granting roles for projects, see Manage access to projects.

Before you begin

  1. Enable the Cloud Storage API.

    Go to Cloud Storage API

  2. Enable the Cloud Logging API.

    Go to Cloud Logging API

  3. Install the Cloud Ops Agent, which is Google Cloud's primary agent for collecting logs and metrics. Select an installation method based on the type of virtual machine you're using.

  4. Set the logging severity to trace by using either the --log-severity CLI option or the severity field to trace.

Forward Cloud Storage FUSE logs to Cloud Logging

  1. Specify /gcsfuse.log for your gcsfuse mount:

    logging:
  file-path: "/gcsfuse.log"
  severity: trace

  2. Configure the Cloud Ops Agent to forward logs from the /gcfuse.log file to Cloud Logging using the following configuration. In this configuration, gcsfuse_trace_logs is a receiver element which receives logs from the /gcsfuse.log file. The service element links the receiver element in the pipeline.

    logging:
 receivers:
  gcsfuse_trace_logs:
  type: files
  include_paths:
  - /gcsfuse.log
service:
 pipelines:
   gcsfuse_pipeline:
     receivers: [gcsfuse_trace_logs]

    To learn about additional logging configurations, see Cloud Logging configurations.

  3. Put the configuration for the Cloud Ops Agent in a path based on your operating system. To choose which path to put the configuration, see the Google Cloud Observability documentation for user-specified configurations.

  4. Restart the Cloud Ops Agent to pick up new configuration changes using the systemctl restart command:

    sudo systemctl restart google-cloud-ops-agent`

  5. Go to Cloud Logging in the Google Cloud console.

  6. Run the fuse_debug filtering query to validate the gcsfuse trace logs that are forwarded to Cloud Logging.

What's next