Broken apt, missing dependencies, what now?

Updated: May 11, 2020

The problem you're facing is as follows. You tried to run a regular update on your Linux box, most likely an Ubuntu- or Debian-based system that uses apt as its package manager. After you updated the repo contents, you ran the dist-upgrade command, and soon hit an error. Apt was complaining about missing or broken dependencies, and suggested you re-run the command with --fix-broken or alike. However, this does not seem to help.

I encountered this issue a few weeks back on KDE neon. The problem really annoyed me, because without some expert knowledge, there's no way out. You're left with an unusable system that cannot be reliably updated. This is so brittle, and we're in 2020, when one would expect some level of robustness from software. LELZ. I did praise neon in the past for its resilience, but then it went ahead and undid its own work. Anyway, let me show you how you can resolve this issue.

Problem in more detail

This is what you see on the command line:

dpkg: error processing archive /var/cache/apt/archives/calligraplan_1%3a3.2.2+p18.04+
git20191212.1837-0_amd64.deb (--unpack):
trying to overwrite '/usr/share/locale/ar/LC_MESSAGES/calligraplan.mo', which is also in package calligra-data 1:3.1.0+p18.04+git20191222.0027-0
dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)
Errors were encountered while processing:
/var/cache/apt/archives/calligraplan_1%3a3.2.2+p18.04+
git20191212.1837-0_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

In my case, the wonky package was Calligra. I'm not sure why it was there in the first place, but hey, we need to fix this, right. So the standard procedure is to run apt with the --fix-broken option. Or perhaps, you could try with --ignore-missing, another valid option. Except you hit a cyclic dependency hell, and there's no way out.

sudo apt --fix-broken install
Reading package lists... Done
Building dependency tree      
Reading state information... Done
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
calligra : Depends: calligraplan (>= 1:3.1.0+p18.04+git20191222.0027-0) but 1:3.1.0+p18.04+git20191119.0027-0 is installed
calligraplan : Depends: calligra-libs (= 1:3.1.0+p18.04+git20191119.0027-0) but 1:3.1.0+p18.04+git20191222.0027-0 is installed
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

At this point, you cannot really remove Calligra, because the package manager is already in an unstable state. The logical step would be to ignore this specific package, or any broken package, using the --ignore-missing option or alike, alas, this does not seem to work, either. So the system is borked.

Solution

The only way around this issue is to actually edit the status file of the package manager, and manually yank out any mention of the affected packages and their dependencies, so that apt can resume and install the healthy portion. Now, this is not a good thing, because you will be left with unpatched software - in some cases, this could be important, Internet-facing stuff, maybe even with a real vulnerability or two, or maybe software with bugs that manifest when you work (say one of the critical system libraries). But it does allow you to continue working, and maybe receive a healthy batch of updates sometime in the future.

What you need at this point is to open /var/lib/dpkg/status in a text editor as root. At this point, I hit another stupid issue, and that's the fact you cannot "supposedly" launch Kate as root, because of would-be security vulnerabilities. Total and utter nonsense.

sudo kate /var/lib/dpkg/status
Executing Kate with sudo is not possible due to unfixable security vulnerabilities.

But you can sudo su -, and then run Kate as root, no problem. This is one of those "let's find a problem to a solution" cases, because it has zero actual bearing on real work, and does not make any difference whatsoever, except to annoy users. As if you can't just open a terminal window and delete stuff with sudo if you like.

Open the file. This is just a long list of packages that are going to be installed. Each package comes with a description, version, and any dependencies. For me, the solution was to look for any mention of the word calligra, and then just delete those package blocks from the status file. In general, this file illustrates the mess quite well, with all the different depends, recommends, breaks and whatnot sections.

Package: libzxingcore1
Status: install ok installed
Priority: optional
...
Version: 1.0.7-1+18.04+bionic+build2
Depends: libc6 (>= 2.14), libgcc1 (>= 1:3.4), libstdc++6 (>= 5.2)
...

Package: mokutil
Status: install ok installed
Priority: optional
...
Version: 0.3.0+1538710437.fb6250f-0ubuntu2~18.04.1
Depends: libc6 (>= 2.14), libefivar1 (>= 34), libssl1.1 (>= 1.1.0)
...

Package: libkf5itemviews-dev
Status: install ok installed
Priority: optional
...
Version: 5.65.0+p18.04+git20191222.0152-0
Replaces: libkf5itemviews-doc (<< 5.61.90-0)
Depends: libkf5itemviews5 (= 5.65.0+p18.04+git20191222.0152-0), qtbase5-dev (>= 5.8.0~)
Recommends: libkf5itemviews-doc (= 5.65.0+p18.04+git20191222.0152-0)
Breaks: kio-dev (<< 5.28), libkf5iconthemes-dev (<< 5.51), libkf5kcmutils-dev (<< 5.51), libkf5kio-dev (<< 5.51), libkf5xmlgui-dev (<< 5.51)
...

But our goal is simple - to delete conflicting entries and allow our system to work. Indeed, once I've completed this, I was able to re-run the apt package manager and complete the updates. Job done.

Conclusion

I am surprised and dismayed that we're in a state where one badly packaged component can completely ruin the system state in Linux. And the thing is, there are many ways to prevent this, if only Linux was developed as a product and not as a series of fragmented functions by devs who don't see the bigger picture of actual usage. For example, check that all packages are sane BEFORE running the installation. Auto-remove bad packages if they are identified. Prompt the user. Blacklist the bad component. Anything, something.

But this was an issue in 2000, and it's still an issue in 2020, and this is why I'm mostly using Windows in my production setup, because suffering is not my middle name. If you find yourself apting on the command line, and things go south, then you may want to look at the status file. It will help you work around broken or missing dependencies, and allow you to update your box. On a sad but happy note, we're done here.

Cheers.