Recently I worked on a PKS upgrade, where the upgrade halted due to no space on the docker registry. On further analysis we identified that the issue is caused due to dangling registry images in the docker registry. In this blog post, I will discuss how to identify and remove dangling registry images from the docker registry.
Docker registry is filled with dangling images. Dangling images are layers that have no relationship to any tagged images. They do not server any purpose and consume disk space.
Run the below command to identify dangling images in the docker registry
docker images -aq -f 'dangling=true'
To identify the dangling images in a kubernetes cluster, follow the below steps:
- Create a script dangling.sh with the below content
sudo -i /var/vcap/packages/docker/bin/docker images -aq -f 'dangling=true'|wc -l
- Copy the script to all worker nodes
bosh -d service-instance_xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx scp dangling.sh worker:/var/tmp
- Run dangling.sh script to see the number of dangling images in each kubernetes worker nodes
bosh -d service-instance_xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx ssh worker -c 'sudo sh /var/tmp/dangling.sh'
- Run the below command to remove the dangling images
docker images -aq -f 'dangling=true' | xargs docker rmi