GRUB2, Ubuntu & rmdir os-prober mount error

Updated: February 24, 2017

The problem you are facing is as follows. You have decided to update your GRUB2 menu configuration. The execution of the command seems to hang or take a very long time, and optionally, you may decide to try to break it. At this point, you see a lot of errors on your screen, reading something like: rmdir: failed to remove '/var/lib/os-prober/mount': Device or resource busy. This results in a corrupt GRUB menu.

Your menu now has duplicate entries or it is missing entries. You are wondering if it's safe to reboot your host, and if you're going to have a valid bootloader configuration, which lets you safely use your system. We will demystify these worries in this guide.

Problem in more detail

I have encountered this problem when trying to update a Ubuntu 16.10 GRUB2 configuration from command line, after having added a new operating system into my multi-boot lot. The error did not seem life-threatening, but it did result in a menu that was missing several of my operating system entries.

I wanted to debug the issue and see what was hiding under the /var/lib/os-prober/mount directory, so I tried to cd there, and examine the contents. The intuition tells me this is the mount point onto which each detected partition is mounted, after various filesystem flags have been tested, and then various checks are executed to identify its type and label accordingly. I was unable to check the contents of the directory, and it shows to be in a weird state:

ls: cannot access 'mount': Transport endpoint is not connected
total 12
drwxr-xr-x  3 root root 4096 Oct 30 22:29 ./
drwxr-xr-x 73 root root 4096 Oct 22 19:06 ../
-rw-r--r--  1 root root   56 Oct 30 22:29 labels
d?????????  ? ?    ?       ?            ? mount/


What happens is - if you break the execution of the command, you may end up with a stale mount. Basically, an invalid file handle, hence this unknown state. This also means the OS prober will not have been able to detect all the entries, and you will only have a partial menu available.

The resolution is quite innocent - reboot. This should fix the issue in two ways. First, as the filesystem takes care of itself, the mount directory will now have proper attributes, and it will be accessible once again. This will allow the GRUB update command to run, examine all the partitions, and eventually if will delete the directory. Clean and simple and perfectly safe. You will never be in any real danger, as it only affects the other operating systems. But it is still not a robust solution, and there has to be a better integral, built-in GRUB recovery method should the command be interrupted.

total 16
drwxr-xr-x  3 root root 4096 Oct 30 22:29 ./
drwxr-xr-x 73 root root 4096 Oct 22 19:06 ../
-rw-r--r--  1 root root   56 Oct 30 22:29 labels
drwxr-xr-x  2 root root 4096 Oct 30 22:26 mount/

sudo update-grub
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.8.0-22-generic
Found initrd image: /boot/initrd.img-4.8.0-22-generic
Found Chapeau release 24 (Cancellara) on /dev/sda13
Found Fedora release 24 (Twenty Four) on /dev/sda14
Found Windows Boot Manager on /dev/sda2@/EFI/Microsoft/Boot/bootmgfw.efi
Found Ubuntu 16.04.1 LTS (16.04) on /dev/sda7
Found CentOS Linux release 7.2.1511 (Core)  on /dev/sda8
Found Linux Mint 17.3 Rosa (17.3) on /dev/sda9
Adding boot menu entry for EFI firmware configuration

Other tips and tricks - GRUB2 reinstall

In general, if you're wondering how you should go about reinstalling GRUB2, if it ever gets displaced, a common scenario in multi-boot setups like mine, you have a bit different considerations for if you're using ms-dos partitions + MBR and if you have EFI + GPT.

For the classic setup, you just install GRUB to MBR, job done. For EFI systems, you do not need to go the full recovery way as I've outlined in my recent guide on this matter. It's a simple matter of reinstalling the package and then updating the menu.

apt-get install --reinstall grub-efi

yum (or dnf) reinstall grub-efi


Here's another GRUB2 tutorial, and blimey am I racking them up recently. But that is what happens when you have a living and breathing estate of wild, untamed distros, and they all do their weird thing. Luckily, the problems are never as sinister as that first moment of panic, unless you're troubleshooting systemd, of course.

In our case, the rmdir errors are nothing more than stale mounts and a less than robust implementation of the OS prober. Reboot, let the filesystem cleanse itself, then run the command again to get a full and up-to-date multi-boot menu. And if you need to reinstall the GRUB, a simple command will do the trick, provided you are able to login into your box and you don't need a full recovery. There we are, the masters of our destiny, and we have just nailed another topic.


You may also like: