This page shows how to use an OpenID provider to configure authentication for GKE On-Prem user clusters. To learn how to use OIDC with ADFS, see Authenticating with OIDC and ADFS.
For an overview of the GKE On-Prem authentication flow, see Authentication.
Overview
GKE On-Prem supports OpenID Connect (OIDC) as one of the authentication mechanisms for interacting with a user cluster's Kubernetes API server. To make the authentication flow automatic for cluster users, GKE On-Prem provides the Kubectl Plugin for OIDC, a kubectl plugin.
In this exercise, you configure a relationship between your OpenID provider and
kubectl
.
Before you begin
This topic assumes you are familiar with OAuth 2.0 and OpenID Connect. This topic assumes you are familiar with OpenID scopes and claims.
Choosing an identity provider
- Use Active Directory Federated Services (ADFS) as your OpenID provider, and use an on-premises instance of AD as your employee database. See Authenticating with OIDC and ADFS.
- Use some other OpenID provider.
No matter which option you choose, you can use the Kubectl Plugin for OIDC, described in these instructions, to expedite configuring authentication and the authentication process itself. This topic explains how to use an OpenID provider other than ADFS.
Downloading the Kubectl Plugin for OIDC
Download the plugin and set access permissions:
Linux
gcloud storage cp gs://gke-on-prem-release/oidc-plugin/v1.1alpha/linux_amd64/kubectl-oidc . chmod +x kubectl-oidc
Windows
gcloud storage cp gs://gke-on-prem-release/oidc-plugin/v1.1alpha/windows_amd64/kubectl-oidc .
macOS
gcloud storage cp gs://gke-on-prem-release/oidc-plugin/v1.1alpha/darwin_amd64/kubectl-oidc . chmod +x kubectl-oidc
Installing the plugin
Install the plugin by moving the executable file to any location on your PATH
. The
executable file must be named kubectl-oidc
. To learn more, see
Installing kubectl plugins.
Creating a redirect URI
As part of establishing a relationship with your OpenID provider, you must specify a redirect URI that the provider can use to return ID tokens. The tokens go to the Kubectl Plugin for OIDC, which runs on each employee's local machine and listens on a port of your choice. Choose a port number greater than 1024 that is suitable for this purpose. Then, the redirect URI is:
http://localhost:[PORT]/callback
where [PORT] is your port number.
Registering the Kubectl Plugin for OIDC with your OpenID provider
Before you can use the Kubectl Plugin for OIDC with your OpenID provider, you need to register the plugin with the provider. Registration includes these steps:
Learn the provider's issuer URI. This is where the plugin sends authentication requests.
Give the provider your redirect URI.
Establish a client ID. This is the ID that the provider uses to identify the plugin.
Establish a client secret. The plugin uses this secret to authenticate to the OpenID provider.
Establish a custom scope that the plugin can use to request the user's security groups.
Establish a custom claim name that the provider will use to return the user's security groups.
How you perform these steps depend on your OpenID provider. To learn how to perform the registration steps with ADFS, see Authenticating with OIDC and ADFS.
Populating the oidc
specification in GKE On-Prem configuration file
During an installation, you generate a GKE On-Prem configuration
file using gkectl create-config
. The configuration includes the
following oidc
specification. You populate `oidc` with values
specific to your provider:
oidc: issuerurl: kubectlredirecturl: clientid: clientsecret: username: usernameprefix: group: groupprefix: scopes: extraparams: usehttpproxy: capath:
issuerurl
: URL of your OpenID provider, such ashttps://example.com/adfs
. Client applications, like the Kubectl Plugin for OIDC, send authorization requests to this URL. The Kubernetes API server uses this URL to discover public keys for verifying tokens. Must use HTTPS. This field is required.kubectlredirecturl
: localhost redirect URL for the Kubectl Plugin for OIDC. You need to register the redirect URL with your OpenID provider for use by the client ID assigned to this cluster. This field is required.clientid
: ID for the client application, such as the Kubectl Plugin for OIDC, that makes authentication requests to the OpenID provider. This field is required.clientsecret
: Secret for the client application. This field is required.usehttpproxy
: Choose whether to deploy a reverse-proxy in the cluster to allow Connect Agent access to the on-premises OIDC provider for authenticating users. Value must be a string:"true"
or"false"
. This field is required.username
: JWT claim to use as the username. Default issub
, which is expected to be a unique identifier of the end user. You can choose other claims, such asemail
orname
, depending on the OIDC provider. However, claims other thanemail
are prefixed with the issuer URL to prevent naming clashes with other plugins.usernameprefix
: Prefix prepended to username claims to prevent clashes with existing names. If this flag is not provided andusername
is a value other than email, the prefix defaults toissueruri#
. The value-
can be used to disable all prefixing.group
: JWT claim to use as the user's group. If the claim is present it must be an array of strings.groupprefix
: Prefix prepended to group claims to prevent clashes with existing names. For example, given a groupfoobar
and a prefixgid-
,gid-foobar
.scopes
: Additional scopes to send to the OpenID provider as a comment-delimited list.extraparams
: Additional key-value parameters to send to the OpenID provider.capath
: Path to certificate for the certificate authority (CA) that signed your identity provider's web certificate.GKE On-Prem clusters use TLS to secure communication between their components. For Kubernetes to automatically generate client certificates during installation and node bootstrapping, GKE On-Prem must be installed with a CA.
By default, GKE On-Prem creates a new CA during installation which generates TLS certificates. The CA and generated certificates are stored locally within the admin cluster.
Example: Authenticating and authorizing a group
Many providers encode user-identifying properties, such as email and user IDs, in a token. However, these properties have implicit risks for authentication policies:
- User IDs can make policies difficult to read and audit.
- Emails can create both an availability risk (if a user changes their primary email) and potentially a security risk (if an email can be re-assigned).
Therefore, it's a best practice to use group policies, as GID can be both persistent and easier to audit.
Suppose your provider creates OpenID tokens that include the following fields:
{ 'iss': 'https://server.example.com' 'sub': 'u98523-4509823' 'groupList: ['developers@example.corp', 'us-east1-cluster-admins@example.corp'] ... }
oidc
specification like so:
issueruri: 'https://server.example.com' username: 'sub' usernameprefix: 'uid-' group: 'groupList' groupprefix: 'gid-' ...
After you've created the user cluster, you could then use Kubernetes role-based access control (RBAC) to grant privileged access to the authenticated users. For example, you could create a ClusterRole that grants its users read-only access to the cluster's Secrets, and create a ClusterRoleBinding resource to bind the role to the authenticated group:
ClusterRole
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: secret-reader rules: - apiGroups: [""] # The resource type for which access is granted resources: ["secrets"] # The permissions granted by the ClusterRole verbs: ["get", "watch", "list"]
ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: read-secrets-admins subjects: # Allows anyone in the "us-east1-cluster-admins" group to # read Secrets in any namespace within this cluster. - kind: Group name: gid-us-east1-cluster-admins # Name is case sensitive apiGroup: rbac.authorization.k8s.io # Allows this specific user to read Secrets in any # namespace within this cluster - kind: User name: uid-u98523-4509823 apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: secret-reader apiGroup: rbac.authorization.k8s.io
Creating the server certificate authority (CA) certificate
Your user cluster's kubeconfig stores its host's CA data in its
certificate-authority-data
field. You need decode this value and store it in
a local file, like server-ca-cert
:
cat [USER_CLUSTER_KUBECONFIG] | grep certificate-authority-data | awk '{ print $2}' | base64 --decode > server-ca-cert
Generating the client authentication configuration file
After you configure your user cluster for OpenID and create it, a user can log
in to the cluster by passing a client authentication configuration file to
kubectl oidc login
. You generate a client authentication configuration file
by entering this command:
Linux
kubectl oidc client-config --issuer-uri [ISSUER_URI] \ --redirect-uri [REDIRECT_URI] \ --client-id [CLIENT_ID] \ --client-secret [CLIENT_SECRET] \ --scopes "[CUSTOM_SCOPES]" \ --cluster-name [USER_CLUSTER_NAME] \ --server [CLUSTER_URL] \ --server-ca-file server-ca-cert \ --issuer-ca-file [CA_CERT] \ --extra-params [KEY]=[VALUE] > client-config.yaml
- [ISSUER_URI] is your issuer URI.
- [REDIRECT_URI] is your redirect URI.
- [CLIENT_ID] is the client ID for the `kubectl` application.
- [CLIENT_SECRET] is the client secret that was generated for you.
- [USER_CLUSTER_NAME] is your user cluster's name.
- [CLUSTER_URL] is the URL of the user cluster's Kubernetes API server.
--server-ca-file
accepts the path to the CA file that you created in the previous section.- [CA_CERT] is the path to the issuer's public CA file.
- [CUSTOM_SCOPES] is the comma-separated list of your custom scopes for security groups.
--extra-param
sends a key-value pair with the authentication request to the OIDC provider.
PowerShell
kubectl oidc client-config --issuer-uri [ISSUER_URI] ` --redirect-uri [REDIRECT_URI] ` --client-id [CLIENT_ID] ` --client-secret [CLIENT_SECRET] ` --scopes "[CUSTOM_SCOPES]" ` --cluster-name [USER_CLUSTER_NAME] ` --server [CLUSTER_URL] ` --server-ca-file server-ca-cert ` --issuer-ca-file [CA_CERT] ` --extra-params [KEY]=[VALUE] > client-config.yaml
- [ISSUER_URI] is your issuer URI.
- [REDIRECT_URI] is your redirect URI.
- [CLIENT_ID] is the client ID for the `kubectl` application.
- [CLIENT_SECRET] is the client secret that was generated for you.
- [USER_CLUSTER_NAME] is your user cluster's name.
- [CLUSTER_URL] is the URL of the user cluster's Kubernetes API server.
--server-ca-file
accepts the path to the CA file that you created in the previous section.- [CA_CERT] is the path to the issuer's public CA file.
- [CUSTOM_SCOPES] is the comma-separated list of your custom scopes for security groups.
--extra-param
sends a key-value pair with the authentication request to the OIDC provider.
This command produces a client authentication file called client-config.yaml
.
Do not manually edit this file. Each employee that needs to authenticate against
the user cluster should be given client-config.yaml
.
About TLS certificates
GKE On-Prem user clusters use TLS to secure all communication between cluster components. For Kubernetes to automatically generate client certificates during installation and node bootstrapping, GKE On-Prem must be installed with a CA.
By default, GKE On-Prem creates a new CA during installation which generates TLS certificates. The CA and generated certificates are stored locally within the admin cluster.
Authenticating against a user cluster using the Kubectl Plugin for OIDC
To authenticate against a user cluster using the client authentication file, perform the following steps from your local machine or VM:
Initialize the plugin using the
client-config.yaml
file:kubectl oidc login --clientconfig-file=client-config.yaml --user [NAME] \ --kubeconfig [KUBECONFIG_OUTPUT_PATH]
where:
- [NAME] is your chosen username.
- [KUBECONFIG_OUTPUT_PATH] the output location for the kubeconfig file where credentials are stored.
kubectl oidc login
launches a browser where the user or employee can enter their credentials.The kubeconfig file provided now contains an ID token that kubectl can use to authenticate to the Kubernetes API server on the user cluster.
You should now be authenticated. To see if you have authenticated successfully, enter any
kubectl
command. For example:kubectl get nodes --kubeconfig [KUBECONFIG_OUTPUT_PATH]