This document describes how to configure Knative serving running on Google Kubernetes Engine (GKE) to use GKE Sandbox for improved workload isolation from the host kernel.
Objectives
- Learn how to enable GKE Sandbox on your Knative serving cluster.
- Deploy a service specified to run on sandboxed nodes.
Costs
In this document, you use the following billable components of Google Cloud:
To generate a cost estimate based on your projected usage,
use the pricing calculator.
Before you begin
- This tutorial assumes that you have Knative serving installed and configured on your cluster.
- Ensure that your command-line environment is set up and the tools are up-to-date:
Add GKE Sandbox node pool to an existing Knative serving Cluster
Follow the steps to add a node pool enabled to use GKE Sandbox to your existing Knative serving cluster. This is where workloads that you opt in to use GKE Sandbox run.
Deploy a workload on the GKE Sandbox node pool
Once you have a GKE cluster with a GKE Sandbox-enabled node pool, update your Knative serving configuration. Also, update each service specification to make use of the new nodes. These updates are required just one time.
Enable the runtime class feature flag
In this section, you enable the Kubernetes runtime class by using Knative
feature and extension flags in the config-features ConfigMap in the
knative-serving namespace. To enable the runtime class, you set the ConfigMap
flag kubernetes.podspec-runtimeclassname to enabled. For more information,
see Kubernetes runtime
class
and Configuring
flags
in the Knative Features and
extensions
documentation.
Open the
config-featuresConfigMap for editing:kubectl edit configmap config-features -n knative-servingAdd the
kubernetes.podspec-runtimeclassname: enabledfeature flag to thedatafield.As an alternative to the preceding steps, you can run the following command to patch this flag into your Knative features:
kubectl patch configmap config-features \ --type='json' \ -p="[{'op': 'replace', 'path': '/data/kubernetes.podspec-runtimeclassname', 'value': 'enabled'}]" \ -n knative-servingTo verify your edits, run the following command to view the ConfigMap:
kubectl get configmap config-features -n knative-serving -o yamlThe ConfigMap should resemble the following:
apiVersion: v1 kind: ConfigMap metadata: name: config-features namespace: knative-serving labels: serving.knative.dev/release: v0.22.0 data: kubernetes.podspec-runtimeclassname: enabled
Deploy a Service configured to run in GKE Sandbox
After you've enabled the runtime class feature flag, you can create Services
that specify a runtimeClassName in the spec. Set the runtimeClassName to
gvisor to indicate that a service should run on the GKE Sandbox node pool.
Deploy a "Hello World" Knative application by running the following command:
cat <<EOF | kubectl apply -f - apiVersion: serving.knative.dev/v1 kind: Service metadata: name: helloworld-go spec: template: spec: runtimeClassName: gvisor containers: - image: gcr.io/knative-samples/helloworld-go env: - name: TARGET value: "gVisor User" EOFIf the Kubernetes runtime class name feature is enabled, the command should finish without errors.
Get the pods for your service running and their configured runtime class by running:
kubectl get pods \ -o=custom-columns='NAME:.metadata.name,RUNTIME CLASS:.spec.runtimeClassName,STATUS:.status.phase'The output should resemble the following:
NAME RUNTIME CLASS STATUS helloworld-go-00001-deployment-55f58477f-fxm6m gvisor Running
What's next
After verifying that you can deploy untrusted apps on your newly created GKE Sandbox node pool, you're ready to add runtimeClassName: gvisor to the
Kubernetes specs for other untrusted services you would like to execute on
GKE Sandbox node pools.
Additional reading:
- GKE Sandbox documentation docs to understand limitations that may break compatibility with some services.
- gVisor docs on deploying Knative services for a more detailed guide to deploying Knative apps on gVisor (also called GKE Sandbox).