Cloud Storage FUSE metrics

This page provides an overview of metrics for Cloud Storage FUSE and how to collect, export, and analyze metrics data.

Overview

Cloud Storage provides in-depth metrics for Cloud Storage FUSE when you use Cloud Storage FUSE to mount your buckets to Cloud Storage as file systems. Metrics can help you continuously monitor performance, identify potential bottlenecks that can impact throughput and latency, and address issues quickly as they occur.

For diagnostic purposes, you can collect metrics in the background when you use Cloud Storage FUSE. Cloud Storage offers two ways for collecting, exporting, and analyzing metrics:

  • Cloud Monitoring exporter: metrics are collected and exported directly from Cloud Storage FUSE to Cloud Monitoring. Exported metrics can then be viewed in the Google Cloud console.

  • Prometheus port: metrics are exported in the Prometheus metrics text-based format at a specified Prometheus port such as http://Host:prometheus-port/metrics using the Prometheus open-source monitoring and alerting toolkit. Metrics can then be viewed as an output using a client URL such as curl http://host:port/metrics

Pricing

Cloud Storage FUSE metrics are charged based on your usage. For more information about pricing, refer to Google Cloud Observability pricing.

Use the Cloud Monitoring exporter

To use the Cloud Monitoring exporter to export, collect, and analyze your metrics from Cloud Storage FUSE to Cloud Monitoring, you must complete the steps outlined in the following sections.

Get required roles

To get the permissions that you need to use the Cloud Monitoring exporter, grant the Monitoring Metric Writer (roles/monitoring.metricWriter) IAM role on the project you want to collect metrics for.

This predefined role contains the permissions required to use the Cloud Monitoring exporter. For more information about the Monitoring Metric Writer role, see IAM documentation about roles/monitoring.metricWriter.

For instructions on granting roles for projects, see Manage access to projects.

Before you begin

  1. Enable the Cloud Monitoring API.

  2. Enable the Cloud Storage API.

    Go to Cloud Storage API

Set up the Cloud Monitoring exporter

To set up the Cloud Monitoring exporter to collect, export, and analyze Cloud Storage FUSE metrics, specify an integer value in seconds using the cloud-metrics-export-interval-secs configuration option using a Cloud Storage FUSE configuration file or the --cloud-metrics-export-interval-secs using the Cloud Storage FUSE CLI.

View metrics using the Google Cloud console

To view Cloud Storage FUSE metrics using the Google Cloud console, use the following instructions:

  1. In the Google Cloud console, go to the Metrics Explorer page.

    Go to Metrics Explorer

  2. Select the project you want to view metrics for.

  3. From the Metric drop-down menu, click Select a metric.

  4. Select VM Instance, then click Custom and enter the name of the metric you want to view. To see which metrics Cloud Storage offers for Cloud Storage FUSE, see Available metrics.

Use Prometheus to export, collect, and analyze metrics

The following sections describe how to configure and use Prometheus to export, collect, and analyze Cloud Storage FUSE metrics.

Configure Prometheus

To configure Prometheus, follow the instructions in Configuring Prometheus for the Ops Agent.

Use Prometheus

  1. Specify the HTTP port you want to export metrics to using either the prometheus-port configuration option in a Cloud Storage FUSE configuration file or --prometheus-port CLI option using the Cloud Storage FUSE CLI.

  2. Validate the Prometheus metrics endpoint using the following command:

     curl https://localhost:8080/metrics

    If successful, the command returns an output which contains text similar to the following:

     # HELP file_cache_read_bytes_count The cumulative number of bytes read from file cache along with read type - Sequential/Random
 # TYPE file_cache_read_bytes_count counter
 file_cache_read_bytes_count{read_type="Random"} 0
 file_cache_read_bytes_count{read_type="Sequential"} 80
 # HELP file_cache_read_count Specifies the number of read requests made via file cache along with type - Sequential/Random and cache hit - true/false
 # TYPE file_cache_read_count counter
 file_cache_read_count{cache_hit="false",read_type="Random"} 215
 file_cache_read_count{cache_hit="false",read_type="Sequential"} 5
 # HELP file_cache_read_latencies The cumulative distribution of the file cache read latencies along with cache hit - true/false
 # TYPE file_cache_read_latencies histogram
 file_cache_read_latencies_bucket{cache_hit="false",le="1"} 215
 file_cache_read_latencies_bucket{cache_hit="false",le="2"} 216
 file_cache_read_latencies_bucket{cache_hit="false",le="3"} 216
 file_cache_read_latencies_bucket{cache_hit="false",le="4"} 216
 file_cache_read_latencies_bucket{cache_hit="false",le="5"} 216
 ...
 file_cache_read_latencies_sum{cache_hit="false"} 483.62783500000023
 file_cache_read_latencies_count{cache_hit="false"} 220
 # HELP fs_ops_count The cumulative number of ops processed by the file system.
 # TYPE fs_ops_count counter
 fs_ops_count{fs_op="FlushFile"} 9
 fs_ops_count{fs_op="GetInodeAttributes"} 91
 fs_ops_count{fs_op="LookUpInode"} 584
 fs_ops_count{fs_op="OpenDir"} 122
 fs_ops_count{fs_op="OpenFile"} 9
 fs_ops_count{fs_op="ReadDir"} 184
 fs_ops_count{fs_op="ReadFile"} 220
 fs_ops_count{fs_op="ReleaseDirHandle"} 122
 fs_ops_count{fs_op="ReleaseFileHandle"} 9
 fs_ops_count{fs_op="StatFS"} 10
 # HELP fs_ops_error_count The cumulative number of errors generated by file system operations
 # TYPE fs_ops_error_count counter
 fs_ops_error_count{fs_error="function not implemented",fs_error_category="function not implemented",fs_op="GetXattr"} 1
 fs_ops_error_count{fs_error="function not implemented",fs_error_category="function not implemented",fs_op="ListXattr"} 1
 fs_ops_error_count{fs_error="interrupted system call",fs_error_category="interrupt errors",fs_op="LookUpInode"} 58
 fs_ops_error_count{fs_error="no such file or directory",fs_error_category="no such file or directory",fs_op="LookUpInode"} 6
 # HELP fs_ops_latency The cumulative distribution of file system operation latencies
 # TYPE fs_ops_latency histogram
 fs_ops_latency_bucket{fs_op="FlushFile",le="1"} 9
 fs_ops_latency_bucket{fs_op="FlushFile",le="2"} 9
 fs_ops_latency_bucket{fs_op="FlushFile",le="3"} 9
 fs_ops_latency_bucket{fs_op="FlushFile",le="4"} 9
 fs_ops_latency_bucket{fs_op="FlushFile",le="5"} 9
 ...
 fs_ops_latency_sum{fs_op="FlushFile"} 0.28800000000000003
 fs_ops_latency_count{fs_op="FlushFile"} 9
 # HELP gcs_download_bytes_count The cumulative number of bytes downloaded from GCS along with type - Sequential/Random
 # TYPE gcs_download_bytes_count counter
 gcs_download_bytes_count{read_type="Sequential"} 2.0971528e+08
 # HELP gcs_read_count Specifies the number of gcs reads made along with type - Sequential/Random
 # TYPE gcs_read_count counter
 gcs_read_count{read_type="Sequential"} 5
 ```

Available metrics

Cloud Storage FUSE provides the following metrics.

File system metrics

File system metrics provide insight into the operations performed on the mounted bucket such as latency and error rates.

Metric name Metric description
fs/ops_count The number of operations processed by the file system. This metric allows grouping by op_type to get counts for individual operations.
fs/ops_error_count The number of errors generated by file system operations. This metric can be grouped by op_type and error_category. Each error is mapped to an error_category in a many-to-one relationship.
fs/ops_latency The cumulative distribution of file system operation latencies. This metric can be grouped by op_type.

Cloud Storage metrics

Cloud Storage metrics provide insight into the interactions between Cloud Storage FUSE and Cloud Storage.

Metric name Metric description
gcs/download_bytes_count The number of bytes downloaded from Cloud Storage along with the read type. The read type specifies sequential, random, or parallel reads.
gcs/read_bytes_count The number of bytes read from Cloud Storage objects, which is different from download_bytes_count. For example, more data is fetched from Cloud Storage than is actually used by the application.
gcs/reader_count The number of Cloud Storage object readers that are opened or closed. This metric's data is grouped by the input/output (I/O) method type such as opened or closed.
gcs/request_count The number of processed Cloud Storage requests.
gcs/request_latencies The total distribution of Cloud Storage request latencies.

File cache metrics

File cache metrics provide insight into the performance and behavior of Cloud Storage FUSE file caching operations.

Metric name Metric description
file_cache/read_bytes_count The number of bytes read from the file cache and the read type, either sequential or random.
file_cache/read_latencies The total distribution of the file cache read latencies and the cache hit, either true or false.
file_cache/read_count The number of read requests made using the file cache, the type of request such as sequential or random, and the cache hit, either true or false.

What's next