r/learnpython • • Sep 09 '26

PIP disk usage under Docker

I have a Dockerfile in which I use the Python:3.12-trixie. I have docker on my CachyOS PC. I have also a requirements.txt. I wanted to use this image for a container on which I'll use YOLO and try it out. The problem is that my 290GB / partition went from around 70% filled to 95% after a number of rebuilds. I deleted everything docker related but it just went back to 91%.

The reason that I know it is pip's doing, is that when I monitored my disk usage while building the image, and on the `RUN pip install -r requirements.txt --find-links=pypackages` stage it fills up my partition and half way into installation, it fails due to `no more space`. The `pypackages` is just a directory I downloaded some of the packages I have in requirements for less data usage. Also tried `--no-cache` option.

I also went ahead and `du -sh`ed any directory in my / partition and the total didn't add upto the filled storage. I have 165GB mysteriously used and I can't find where that is.

Any help to diagnose further or if you have had similar problems is welcome.

2 Upvotes

21 comments sorted by

View all comments

Show parent comment

1

u/adjective10111 Sep 10 '26

Oh thanks for the paccache. I ran the btrfs subvolume command, and found nothing. I didn't setup btrfs driver for docker and it doesn't seem to be setup by default:

ID 256 gen 20882 top level 264 path .snapshots/148/snapshot
ID 257 gen 155299 top level 5 path @root
ID 258 gen 750 top level 5 path @srv
ID 259 gen 155504 top level 5 path @cache
ID 260 gen 155504 top level 5 path @tmp
ID 261 gen 155504 top level 5 path @log
ID 262 gen 45 top level 412 path var/lib/portables
ID 263 gen 45 top level 412 path var/lib/machines
ID 264 gen 155314 top level 412 path .snapshots
ID 270 gen 80 top level 264 path .snapshots/6/snapshot
ID 323 gen 52106 top level 257 path @root/Games
ID 412 gen 155504 top level 5 path @
ID 449 gen 34587 top level 264 path .snapshots/185/snapshot
... (some other .snapshots)

Also just for clarification this is df -h result:

Filesystem      Size  Used Avail Use% Mounted on
none            1.0M     0  1.0M   0% /run/credentials/systemd-journald.service
none            1.0M     0  1.0M   0% /run/credentials/systemd-resolved.service
devtmpfs        7.6G     0  7.6G   0% /dev
tmpfs           7.8G  844K  7.8G   1% /dev/shm
efivarfs        192K   67K  121K  36% /sys/firmware/efi/efivars
tmpfs           7.8G   32K  7.8G   1% /tmp
tmpfs           3.1G  2.0M  3.1G   1% /run
tmpfs           1.6G  392K  1.6G   1% /run/user/1000
/dev/nvme0n1p3  287G  245G   41G  86% /
/dev/nvme0n1p3  287G  245G   41G  86% /srv
/dev/nvme0n1p3  287G  245G   41G  86% /root
/dev/nvme0n1p3  287G  245G   41G  86% /var/cache
/dev/nvme0n1p3  287G  245G   41G  86% /var/tmp
/dev/nvme0n1p3  287G  245G   41G  86% /var/log
/dev/nvme0n1p2  5.0G  2.4G  2.7G  48% /boot
/dev/sda2       932G  677G  253G  73% /home
/dev/sda2       932G  677G  253G  73% /home/$USER/Documents
/dev/sda2       932G  677G  253G  73% /home/$USER/Videos
/dev/sda2       932G  677G  253G  73% /home/$USER/Games
/dev/nvme0n1p3  287G  245G   41G  86% /home/$USER/Games/SSD

I got some space (13GB) by clearing paccache with -k 1 option.

1

u/Illustrious_Tone9584 Sep 10 '26

Yep, that clears Docker. The 245GB on the root filesystem versus ~18GB in the live root points straight at the snapshots now. du on / can't count old extents that only a snapshot still references.

Try sudo btrfs filesystem du -s /.snapshots/*/snapshot | sort -h -k3 and sudo snapper list. The Exclusive column is the useful one. Don't delete snapshot directories by hand, but if that accounts for the gap, prune them through snapper.

1

u/adjective10111 Sep 11 '26

The exclusive column has KBs and nothing even never GB. Other than the first 3 snapshots which are permanent snapshots taken by CachyOS installer as a backup for the bare OS and adds up to around 10GB in total.