How to identify dangling registry images in docker registry

Recently I worked on a PKS upgrade, where the upgrade halted due to no space on the docker registry. On further analysis we identified that the issue is caused due to dangling registry images in the docker registry. In this blog post, I will discuss how to identify and remove dangling registry images from the docker registry.

Symptoms

Docker registry is filled with dangling images. Dangling images are layers that have no relationship to any tagged images. They do not server any purpose and consume disk space.

Resolution

Run the below command to identify dangling images in the docker registry
docker images -aq -f 'dangling=true'

To identify the dangling images in a kubernetes cluster, follow the below steps:

1. Create a script dangling.sh with the below content
   sudo -i /var/vcap/packages/docker/bin/docker images -aq -f 'dangling=true'|wc -l
2. Copy the script to all worker nodes
   bosh -d service-instance_xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx scp dangling.sh worker:/var/tmp
3. Run dangling.sh script to see the number of dangling images in each kubernetes worker nodes
   bosh -d service-instance_xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx ssh worker -c 'sudo sh /var/tmp/dangling.sh'
4. Run the below command to remove the dangling images
   docker images -aq -f 'dangling=true' | xargs docker rmi