Monday, February 24, 2014

ZFS - Pool/Disk/Volume full - Can't rm!

So you realize your Open Indiana/Solaris/Whatever server is erroring on bootup because your storage is full. What do you do?

 You probably assume the same thing I did. Enter into recovery mode, delete a couple files, boot as normal. Well too bad since on ZFS that will not work! You'll most likely get an error message saying that you can't rm, the disk is full! Crap, what do you do now?

If you google you'll find a few solutions that people confirm as working. Unfortunately they did *not* work for me.

One is to use dd to zero out a file.
dd if=/dev/null of=some_file
rm some_file

I still got the same error when attempting to run dd that I got with rm.


Another popular solution is to pipe data over a large file
echo "1" > some_file
rm some_file
That also didn't work for me.


 The only solution that worked was to clear out old snapshots. Thankfully I had one available
zfs list -t snapshot
zfs destroy your_non_essential_snapshot
That freed up enough space to allow me to go clear out files I really didn't need (logs, packages, etc).


Hope that helps someone!