Stay organized with collections
Save and categorize content based on your preferences.
This document provides recommendations on how to shard your
Config Controller usage. Sharding is the process of splitting
Config Controller-managed Google Cloud resources across multiple namespaces,
clusters, or projects.
Sharding brings the following benefits:
Reduces the impact of changes: If a single shard stops functioning, the other
shards are not impacted.
Helps you manage security: Each shard can have dedicated IAM and RBAC
configurations. Malicious attackers who compromise one shard cannot access
other shards. Misconfiguration in one shard cannot affect other shards.
Better scalability: A single shard can have scalability bottlenecks such
as the number of managed objects, or API quotas. Having multiple shards
increases the overall scalability of your Config Controller usage.
Use sharding with Config Controller
There are a few different ways to implement sharding. The best approach for you
will depend on your specific needs and requirements.
Sharding models
There are two main sharding models:
By business lines or application teams: This model is typically used when
Config Controller is used by different teams. In this model, each
team has their own shard.
By environment: This model is typically used when
Config Controller is used in different environments. For example, you
might have a shard for your development environment, a shard for your QA
environment, and a shard for your production environment.
Minimize the need for cross-shard references
When you shard your Config Controller usage, you should minimize the
need for cross-shard references. Cross-shard references can make your
configuration more complex and difficult to manage. See
Resource references across instances
for more details.
By Config Controller instances: You can create multiple
Config Controller instances in one Google Cloud project.
By projects: You can create multiple Config Controller instances in
multiple Google Cloud projects. This mechanism helps address API quota issues if you
are hitting quota bottlenecks with one single project. See
Split your resources into multiple projects
for more details.
Caveats when implementing sharding
When implementing sharding for your Config Controller usage, there are some
potential issues you should be aware of and plan to mitigate.
Resource references across instances
One of the challenges of sharding Config Controller is dealing with resource
references across instances. For example, a platform team might create Projects
in one instance, and then app teams might create resources that refer to those
Projects in other instances. This can create problems such as:
Increased complexity: Managing resource references across clusters can
make your configuration more complex and difficult to manage.
Increased risk: If a resource is deleted in one shard, it can still be
referenced by resources in other shards. This can lead to unexpected
behavior and data loss.
Performance degradation: Resource references across clusters can
increase the latency of your configuration changes.
There are a few ways to work around the cross-reference challenge:
Sharding in a way that no reference across shards is needed. This
could be done by sharding by environments or by teams.
Using external references. This means that the object that is being
referenced is not actually managed by Config Controller. This can be a
good option if the object isn't changing frequently.
Having the same object available in all shards. This is a more complex
option, but it can be the best option if the object changes frequently.
The objects should share the same source of truth to avoid reconciliation
fights between these objects in different shards. You need to set the
conflict prevention policy
to none for these objects.
It is important to carefully consider the advantages and disadvantages of each approach
before choosing one.
API Quotas
Sharding could increase your API quotas. You should be aware of this and plan
accordingly. See
Manage API quota limits
for best practices on managing API quota limits.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-25 UTC."],[],[],null,["# Config Controller sharding guidelines\n\nThis document provides recommendations on how to shard your\nConfig Controller usage. Sharding is the process of splitting\nConfig Controller-managed Google Cloud resources across multiple namespaces,\nclusters, or projects.\n\nSharding brings the following benefits:\n\n- **Reduces the impact of changes**: If a single shard stops functioning, the other shards are not impacted.\n- **Helps you manage security**: Each shard can have dedicated IAM and RBAC configurations. Malicious attackers who compromise one shard cannot access other shards. Misconfiguration in one shard cannot affect other shards.\n- **Better scalability**: A single shard can have scalability bottlenecks such as the number of managed objects, or API quotas. Having multiple shards increases the overall scalability of your Config Controller usage.\n\nUse sharding with Config Controller\n-----------------------------------\n\nThere are a few different ways to implement sharding. The best approach for you\nwill depend on your specific needs and requirements.\n\n### Sharding models\n\nThere are two main sharding models:\n\n- **By business lines or application teams**: This model is typically used when Config Controller is used by different teams. In this model, each team has their own shard.\n- **By environment**: This model is typically used when Config Controller is used in different environments. For example, you might have a shard for your development environment, a shard for your QA environment, and a shard for your production environment.\n\n### Minimize the need for cross-shard references\n\nWhen you shard your Config Controller usage, you should minimize the\nneed for cross-shard references. Cross-shard references can make your\nconfiguration more complex and difficult to manage. See\n[Resource references across instances](#resource-references-across-instances)\nfor more details.\n\n### Sharding mechanisms\n\nThere are three main sharding mechanisms:\n\n- **By namespaces** : You can create additional namespaces and [configure Config Connector to manage resources in those namespaces](/config-connector/docs/how-to/advanced-install#project-namespace).\n- **By Config Controller instances**: You can create multiple Config Controller instances in one Google Cloud project.\n- **By projects** : You can create multiple Config Controller instances in multiple Google Cloud projects. This mechanism helps address API quota issues if you are hitting quota bottlenecks with one single project. See [Split your resources into multiple projects](/config-connector/docs/best-practices#quota-split) for more details.\n\nCaveats when implementing sharding\n----------------------------------\n\nWhen implementing sharding for your Config Controller usage, there are some\npotential issues you should be aware of and plan to mitigate.\n\n### Resource references across instances\n\nOne of the challenges of sharding Config Controller is dealing with resource\nreferences across instances. For example, a platform team might create Projects\nin one instance, and then app teams might create resources that refer to those\nProjects in other instances. This can create problems such as:\n\n- **Increased complexity**: Managing resource references across clusters can make your configuration more complex and difficult to manage.\n- **Increased risk**: If a resource is deleted in one shard, it can still be referenced by resources in other shards. This can lead to unexpected behavior and data loss.\n- **Performance degradation**: Resource references across clusters can increase the latency of your configuration changes.\n\nThere are a few ways to work around the cross-reference challenge:\n\n- Sharding in a way that no reference across shards is needed. This could be done by sharding by environments or by teams.\n- Using external references. This means that the object that is being referenced is not actually managed by Config Controller. This can be a good option if the object isn't changing frequently.\n- Having the same object available in all shards. This is a more complex option, but it can be the best option if the object changes frequently. The objects should share the same source of truth to avoid reconciliation fights between these objects in different shards. You need to set the [conflict prevention policy](/config-connector/docs/concepts/managing-conflicts#modifying_conflict_prevention) to `none` for these objects.\n\nIt is important to carefully consider the advantages and disadvantages of each approach\nbefore choosing one.\n\n### API Quotas\n\nSharding could increase your API quotas. You should be aware of this and plan\naccordingly. See\n[Manage API quota limits](/config-connector/docs/best-practices#quota-limits)\nfor best practices on managing API quota limits.\n\nWhat's next\n-----------\n\n- Learn more about [Config Controller scalability](/kubernetes-engine/enterprise/config-controller/docs/scalability)"]]