Cloud BigTable - Package cloud.google.com/go/bigtable (v1.8.0)

Package bigtable is an API to Google Cloud Bigtable.

See https://cloud.google.com/bigtable/docs/ for general product documentation.

See https://godoc.org/cloud.google.com/go for authentication, timeouts, connection pooling and similar aspects of this package.

Reading

The principal way to read from a Bigtable is to use the ReadRows method on *Table. A RowRange specifies a contiguous portion of a table. A Filter may be provided through RowFilter to limit or transform the data that is returned.

tbl := client.Open("mytable")
// Read all the rows starting with "com.google.", but only fetch the columns
// in the "links" family.
rr := bigtable.PrefixRange("com.google.")
err := tbl.ReadRows(ctx, rr, func(r Row) bool {
    // TODO: do something with r.
    return true // Keep going.
}, bigtable.RowFilter(bigtable.FamilyFilter("links")))
if err != nil {
    // TODO: handle err.
}

To read a single row, use the ReadRow helper method:

r, err := tbl.ReadRow(ctx, "com.google.cloud") // "com.google.cloud" is the entire row key
if err != nil {
    // TODO: handle err.
}
// TODO: use r.

Writing

This API exposes two distinct forms of writing to a Bigtable: a Mutation and a ReadModifyWrite. The former expresses idempotent operations. The latter expresses non-idempotent operations and returns the new values of updated cells. These operations are performed by creating a Mutation or ReadModifyWrite (with NewMutation or NewReadModifyWrite), building up one or more operations on that, and then using the Apply or ApplyReadModifyWrite methods on a Table.

For instance, to set a couple of cells in a table:

tbl := client.Open("mytable")
mut := bigtable.NewMutation()
// To use numeric values that will later be incremented,
// they need to be big-endian encoded as 64-bit integers.
buf := new(bytes.Buffer)
initialLinkCount := 1 // The initial number of links.
if err := binary.Write(buf, binary.BigEndian, initialLinkCount); err != nil {
// TODO: handle err.
}
mut.Set("links", "maps.google.com", bigtable.Now(), buf.Bytes())
mut.Set("links", "golang.org", bigtable.Now(), buf.Bytes())
err := tbl.Apply(ctx, "com.google.cloud", mut)
if err != nil {
    // TODO: handle err.
}

To increment an encoded value in one cell:

tbl := client.Open("mytable")
rmw := bigtable.NewReadModifyWrite()
rmw.Increment("links", "golang.org", 12) // add 12 to the cell in column "links:golang.org"
r, err := tbl.ApplyReadModifyWrite(ctx, "com.google.cloud", rmw)
if err != nil {
    // TODO: handle err.
}
// TODO: use r.

Retries

If a read or write operation encounters a transient error it will be retried until a successful response, an unretryable error or the context deadline is reached. Non-idempotent writes (where the timestamp is set to ServerTime) will not be retried. In the case of ReadRows, retried calls will not re-scan rows that have already been processed.

Routing policies.

Scope constants for authentication credentials. These should be used when using credential creation functions such as oauth.NewServiceAccountFromFile.

DefaultSnapshotDuration is the default TTL for a snapshot.

GCRuleToString converts the given GcRule proto to a user-visible string.

AdminClient is a client type for performing admin operations within a specific instance.

NewAdminClient creates a new AdminClient for a given project and instance.

BackupIAM creates an IAM Handle specific to a given Cluster and Backup.

BackupInfo gets backup metadata.

Backups returns a BackupIterator for iterating over the backups in a cluster. To list backups across all of the clusters in the instance specify "-" as the cluster.

Close closes the AdminClient.

CreateBackup creates a new backup in the specified cluster from the specified source table with the user-provided expire time.

CreateColumnFamily creates a new column family in a table.

CreatePresplitTable creates a new table in the instance. The list of row keys will be used to initially split the table into multiple tablets. Given two split keys, "s1" and "s2", three tablets will be created, spanning the key ranges: [, s1), [s1, s2), [s2, ). This method may return before the table's creation is complete.

CreateTable creates a new table in the instance. This method may return before the table's creation is complete.

CreateTableFromConf creates a new table in the instance from the given configuration.

CreateTableFromSnapshot creates a table from snapshot. The table will be created in the same cluster as the snapshot.

This is a private alpha release of Cloud Bigtable snapshots. This feature is not currently available to most Cloud Bigtable customers. This feature might be changed in backward-incompatible ways and is not recommended for production use. It is not subject to any SLA or deprecation policy.

DeleteBackup deletes a backup in a cluster.

DeleteColumnFamily deletes a column family in a table and all of its data.

DeleteSnapshot deletes a snapshot in a cluster.

This is a private alpha release of Cloud Bigtable snapshots. This feature is not currently available to most Cloud Bigtable customers. This feature might be changed in backward-incompatible ways and is not recommended for production use. It is not subject to any SLA or deprecation policy.

DeleteTable deletes a table and all of its data.

DropAllRows permanently deletes all rows from the specified table.

DropRowRange permanently deletes a row range from the specified table.

RestoreTable creates a table from a backup. The table will be created in the same cluster as the backup.

SetGCPolicy specifies which cells in a column family should be garbage collected. GC executes opportunistically in the background; table reads may return data matching the GC policy.

SnapshotInfo gets snapshot metadata.

This is a private alpha release of Cloud Bigtable snapshots. This feature is not currently available to most Cloud Bigtable customers. This feature might be changed in backward-incompatible ways and is not recommended for production use. It is not subject to any SLA or deprecation policy.

SnapshotTable creates a new snapshot in the specified cluster from the specified source table. Setting the TTL to DefaultSnapshotDuration will use the server side default for the duration.

This is a private alpha release of Cloud Bigtable snapshots. This feature is not currently available to most Cloud Bigtable customers. This feature might be changed in backward-incompatible ways and is not recommended for production use. It is not subject to any SLA or deprecation policy.

Snapshots returns a SnapshotIterator for iterating over the snapshots in a cluster. To list snapshots across all of the clusters in the instance specify "-" as the cluster.

This is a private alpha release of Cloud Bigtable snapshots. This feature is not currently available to most Cloud Bigtable customers. This feature might be changed in backward-incompatible ways and is not recommended for production use. It is not subject to any SLA or deprecation policy.

TableIAM creates an IAM Handle specific to a given Instance and Table within the configured project.

TableInfo retrieves information about a table.

Tables returns a list of the tables in the instance.

UpdateBackup updates the backup metadata in a cluster. The API only supports updating expire time.

WaitForReplication waits until all the writes committed before the call started have been propagated to all the clusters in the instance via replication.

An ApplyOption is an optional argument to Apply.

GetCondMutationResult returns an ApplyOption that reports whether the conditional mutation's condition matched.

BackupInfo contains backup metadata. This struct is read-only.

BackupIterator is an EntryIterator that iterates over log entries.

Next returns the next result. Its second return value is iterator.Done (https://godoc.org/google.golang.org/api/iterator) if there are no more results. Once Next returns Done, all subsequent calls will return Done.

PageInfo supports pagination. See https://godoc.org/google.golang.org/api/iterator package for details.

Client is a client for reading and writing data to tables in an instance.

A Client is safe to use concurrently, except for its Close method.

NewClient creates a new Client for a given project and instance. The default ClientConfig will be used.

NewClientWithConfig creates a new client with the given config.

Close closes the Client.

Open opens a table.

ClientConfig has configurations for the client.

ClusterConfig contains the information necessary to create a cluster

ClusterInfo represents information about a cluster.

ErrPartiallyUnavailable is returned when some locations (clusters) are unavailable. Both partial results (retrieved from available locations) and the error are returned when this exception occurred.

FamilyInfo represents information about a column family.

A Filter represents a row filter.

BlockAllFilter returns a filter that matches nothing.

CellsPerRowLimitFilter returns a filter that matches only the first N cells of each row.

CellsPerRowOffsetFilter returns a filter that skips the first N cells of each row, matching all subsequent cells.

ChainFilters returns a filter that applies a sequence of filters.

ColumnFilter returns a filter that matches cells whose column name matches the provided RE2 pattern. See https://github.com/google/re2/wiki/Syntax for the accepted syntax.

ColumnRangeFilter returns a filter that matches a contiguous range of columns within a single family, as specified by an inclusive start qualifier and exclusive end qualifier.

ConditionFilter returns a filter that evaluates to one of two possible filters depending on whether or not the given predicate filter matches at least one cell. If the matched filter is nil then no results will be returned. IMPORTANT NOTE: The predicate filter does not execute atomically with the true and false filters, which may lead to inconsistent or unexpected results. Additionally, condition filters have poor performance, especially when filters are set for the false condition.

FamilyFilter returns a filter that matches cells whose family name matches the provided RE2 pattern. See https://github.com/google/re2/wiki/Syntax for the accepted syntax.

InterleaveFilters returns a filter that applies a set of filters in parallel and interleaves the results.

LabelFilter returns a filter that applies the given label to all cells in the output row.

LatestNFilter returns a filter that matches the most recent N cells in each column.

PassAllFilter returns a filter that matches everything.

RowKeyFilter returns a filter that matches cells from rows whose key matches the provided RE2 pattern. See https://github.com/google/re2/wiki/Syntax for the accepted syntax.

RowSampleFilter returns a filter that matches a row with a probability of p (must be in the interval (0, 1)).

StripValueFilter returns a filter that replaces each value with the empty string.

TimestampRangeFilter returns a filter that matches any cells whose timestamp is within the given time bounds. A zero time means no bound. The timestamp will be truncated to millisecond granularity.

TimestampRangeFilterMicros returns a filter that matches any cells whose timestamp is within the given time bounds, specified in units of microseconds since 1 January 1970. A zero value for the end time is interpreted as no bound. The timestamp will be truncated to millisecond granularity.

ValueFilter returns a filter that matches cells whose value matches the provided RE2 pattern. See https://github.com/google/re2/wiki/Syntax for the accepted syntax.

ValueRangeFilter returns a filter that matches cells with values that fall within the given range, as specified by an inclusive start value and exclusive end value.

A GCPolicy represents a rule that determines which cells are eligible for garbage collection.

IntersectionPolicy returns a GC policy that only applies when all its sub-policies apply.

MaxAgePolicy returns a GC policy that applies to all cells older than the given age.

MaxVersionsPolicy returns a GC policy that applies to all versions of a cell except for the most recent n.

NoGcPolicy applies to all cells setting maxage and maxversions to nil implies no gc policies

UnionPolicy returns a GC policy that applies when any of its sub-policies apply.

InstanceAdminClient is a client type for performing admin operations on instances. These operations can be substantially more dangerous than those provided by AdminClient.

NewInstanceAdminClient creates a new InstanceAdminClient for a given project.

Close closes the InstanceAdminClient.

Clusters lists the clusters in an instance. If any location (cluster) is unavailable due to some transient conditions, Clusters returns partial results and ErrPartiallyUnavailable error with unavailable locations list.

CreateAppProfile creates an app profile within an instance.

CreateCluster creates a new cluster in an instance. This method will return when the cluster has been created or when an error occurs.

CreateInstance creates a new instance in the project. This method will return when the instance has been created or when an error occurs.

CreateInstanceWithClusters creates a new instance with configured clusters in the project. This method will return when the instance has been created or when an error occurs.

DeleteAppProfile deletes an app profile from an instance.

DeleteCluster deletes a cluster from an instance.

DeleteInstance deletes an instance from the project.

GetAppProfile gets information about an app profile.

GetCluster fetches a cluster in an instance

InstanceIAM returns the instance's IAM handle.

InstanceInfo returns information about an instance.

Instances returns a list of instances in the project. If any location (cluster) is unavailable due to some transient conditions, Instances returns partial results and ErrPartiallyUnavailable error with unavailable locations list.

ListAppProfiles lists information about app profiles in an instance.

UpdateAppProfile updates an app profile within an instance. updateAttrs should be set. If unset, all fields will be replaced.

UpdateCluster updates attributes of a cluster

UpdateInstanceWithClusters updates an instance and its clusters. Updateable fields are instance display name, instance type and cluster size. The provided InstanceWithClustersConfig is used as follows:

• InstanceID is required
• DisplayName and InstanceType are updated only if they are not empty
• ClusterID is required for any provided cluster
• All other cluster fields are ignored except for NumNodes, which if set will be updated

This method may return an error after partially succeeding, for example if the instance is updated but a cluster update fails. If an error is returned, InstanceInfo and Clusters may be called to determine the current state.

InstanceConf contains the information necessary to create an Instance

InstanceInfo represents information about an instance

InstanceState is the state of the instance. This is output-only.

InstanceType is the type of the instance.

InstanceWithClustersConfig contains the information necessary to create an Instance

Mutation represents a set of changes for a single row of a table.

NewCondMutation returns a conditional mutation. The given row filter determines which mutation is applied: If the filter matches any cell in the row, mtrue is applied; otherwise, mfalse is applied. Either given mutation may be nil.

The application of a ReadModifyWrite is atomic; concurrent ReadModifyWrites will be executed serially by the server.

NewMutation returns a new mutation.

DeleteCellsInColumn will delete all the cells whose columns are family:column.

DeleteCellsInFamily will delete all the cells whose columns are family:*.

DeleteRow deletes the entire row.

DeleteTimestampRange deletes all cells whose columns are family:column and whose timestamps are in the half-open interval [start, end). If end is zero, it will be interpreted as infinity. The timestamps will be truncated to millisecond granularity.

Set sets a value in a specified column, with the given timestamp. The timestamp will be truncated to millisecond granularity. A timestamp of ServerTime means to use the server timestamp.

ProfileAttrsToUpdate define addrs to update during an Update call. If unset, no fields will be replaced.

GetFieldMaskPath returns the field mask path.

ProfileConf contains the information necessary to create an profile

ProfileIterator iterates over profiles.

Next returns the next result. Its second return value is iterator.Done (https://godoc.org/google.golang.org/api/iterator) if there are no more results. Once Next returns Done, all subsequent calls will return Done.

PageInfo supports pagination. See https://godoc.org/google.golang.org/api/iterator package for details.

A ReadItem is returned by Read. A ReadItem contains data from a specific row and column.

ReadModifyWrite represents a set of operations on a single row of a table. It is like Mutation but for non-idempotent changes. When applied, these operations operate on the latest values of the row's cells, and result in a new value being written to the relevant cell with a timestamp that is max(existing timestamp, current server time).

The application of a ReadModifyWrite is atomic; concurrent ReadModifyWrites will be executed serially by the server.

NewReadModifyWrite returns a new ReadModifyWrite.

AppendValue appends a value to a specific cell's value. If the cell is unset, it will be treated as an empty value.

Increment interprets the value in a specific cell as a 64-bit big-endian signed integer, and adds a value to it. If the cell is unset, it will be treated as zero. If the cell is set and is not an 8-byte value, the entire ApplyReadModifyWrite operation will fail.

A ReadOption is an optional argument to ReadRows.

LimitRows returns a ReadOption that will limit the number of rows to be read.

RowFilter returns a ReadOption that applies f to the contents of read rows.

If multiple RowFilters are provided, only the last is used. To combine filters, use ChainFilters or InterleaveFilters instead.

A Row is returned by ReadRows. The map is keyed by column family (the prefix of the column name before the colon). The values are the returned ReadItems for that column family in the order returned by Read.

Key returns the row's key, or "" if the row is empty.

RowList is a sequence of row keys.

A RowRange is a half-open interval [Start, Limit) encompassing all the rows with keys at least as large as Start, and less than Limit. (Bigtable string comparison is the same as Go's.) A RowRange can be unbounded, encompassing all keys at least as large as Start.

InfiniteRange returns the RowRange consisting of all keys at least as large as start.

NewRange returns the new RowRange [begin, end).

PrefixRange returns a RowRange consisting of all keys starting with the prefix.

Contains says whether the RowRange contains the key.

String provides a printable description of a RowRange.

Unbounded tests whether a RowRange is unbounded.

RowRangeList is a sequence of RowRanges representing the union of the ranges.

RowSet is a set of rows to be read. It is satisfied by RowList, RowRange and RowRangeList. The serialized size of the RowSet must be no larger than 1MiB.

SingleRow returns a RowSet for reading a single row.

SnapshotInfo contains snapshot metadata.

SnapshotIterator is an EntryIterator that iterates over log entries.

This is a private alpha release of Cloud Bigtable snapshots. This feature is not currently available to most Cloud Bigtable customers. This feature might be changed in backward-incompatible ways and is not recommended for production use. It is not subject to any SLA or deprecation policy.

Next returns the next result. Its second return value is iterator.Done (https://godoc.org/google.golang.org/api/iterator) if there are no more results. Once Next returns Done, all subsequent calls will return Done.

PageInfo supports pagination. See https://godoc.org/google.golang.org/api/iterator package for details.

StorageType is the type of storage used for all tables in an instance

A Table refers to a table.

A Table is safe to use concurrently.

Apply mutates a row atomically. A mutation must contain at least one operation and at most 100000 operations.

ApplyBulk applies multiple Mutations, up to a maximum of 100,000. Each mutation is individually applied atomically, but the set of mutations may be applied in any order.

Two types of failures may occur. If the entire process fails, (nil, err) will be returned. If specific mutations fail to apply, ([]err, nil) will be returned, and the errors will correspond to the relevant rowKeys/muts arguments.

Conditional mutations cannot be applied in bulk and providing one will result in an error.

ApplyReadModifyWrite applies a ReadModifyWrite to a specific row. It returns the newly written cells.

ReadRow is a convenience implementation of a single-row reader. A missing row will return a zero-length map and a nil error.

ReadRows reads rows from a table. f is called for each row. If f returns false, the stream is shut down and ReadRows returns. f owns its argument, and f is called serially in order by row key.

By default, the yielded rows will contain all values in all cells. Use RowFilter to limit the cells returned.

SampleRowKeys returns a sample of row keys in the table. The returned row keys will delimit contiguous sections of the table of approximately equal size, which can be used to break up the data for distributed tasks like mapreduces.

TableConf contains all of the information necessary to create a table with column families.

TableInfo represents information about a table.

Timestamp is in units of microseconds since 1 January 1970.

ServerTime is a specific Timestamp that may be passed to (*Mutation).Set. It indicates that the server's timestamp should be used.

Now returns the Timestamp representation of the current time on the client.

Time converts a Timestamp into a time.Time.

Time converts a Timestamp into a time.Time.

TruncateToMilliseconds truncates a Timestamp to millisecond granularity, which is currently the only granularity supported.

UpdateInstanceResults contains information about the changes made after invoking UpdateInstanceAndSyncClusters.

UpdateInstanceAndSyncClusters updates an instance and its clusters, and will synchronize the clusters in the instance with the provided clusters, creating and deleting them as necessary. The provided InstanceWithClustersConfig is used as follows:

• InstanceID is required
• DisplayName and InstanceType are updated only if they are not empty
• ClusterID is required for any provided cluster
• Any cluster present in conf.Clusters but not part of the instance will be created using CreateCluster and the given ClusterConfig.
• Any cluster missing from conf.Clusters but present in the instance will be removed from the instance using DeleteCluster.
• Any cluster in conf.Clusters that also exists in the instance will be updated to contain the provided number of nodes if set.

This method may return an error after partially succeeding, for example if the instance is updated but a cluster update fails. If an error is returned, InstanceInfo and Clusters may be called to determine the current state. The return UpdateInstanceResults will describe the work done by the method, whether partial or complete.