Firefox & cookies corruption problem

Updated: December 3, 2018

A strange problem befell one of my computers running Windows, with Firefox being the default browser, utilizing a profile that goes back a good decade or more. One blue Monday, I opened the browser, went to one of the sites I frequently visit and use, and noticed that I'd been logged out. Another site, same thing. It would appear all my login sessions were gone.

Since I keep multiple backups of everything, I restored the Firefox cookies database - cookies.sqlite file into the Firefox profile, and I was back to normal. Several days later, the issue happened again. Intrigued, I started exploring this somewhat obscure and not-well-documented problem. I believe I know why, and I have a solution.

Teaser

Note: Image taken from wikipedia.org, licensed under GFDL 1.2.

Problem in more detail

Online sources offered very little. I found a reference to a similar issue from around Firefox 6 time, and a lone reddit post regarding Quantum (Firefox 57 and above). Lots of random questions on how to restore the cookies, but not many details on what people were experiencing. In general, the underlying motif was that during an ordinary browsing session, the cookie database would get corrupted. People would not notice that until the next browser start, when the corrupt database is removed and a new one created. Both these files are located in the Firefox profile, named cookies.sqlite and cookies.sqlite.bak. If you keep backup copies, overwriting the database is the simplest way to restore old session logins. But that does not give us more insight into why.

Indeed why. Well, the one thing that was "unusual" was that in this particular case, I have done no temporary file and cookie cleaning in about 18 months. I would normally do that once a quarter or so, flush several GB worth of junk and clean most of the cookies (keep the important ones). Interested to see how Firefox and Chrome actually handle junk accumulation, I let things be.

Firefox options, data

On a side note, I noticed that Chrome took a bit longer showing favicons on loaded sites, but there were no slowness, errors, or any weird behavior. The only real issue was Firefox gimping on the cookies front, and this happened sporadically, without any real reason. I figured rinsing the crud could help.

Solution

I did clean the temporary data and cookies (except the needed logins). I noticed that Firefox now ran ever so slightly faster - a tiny but noticeable performance improvement. Chrome was also serving favicons instantly. What should be a deterministic behavior isn't really. The browsers ARE impacted by the slow buildup of temporary data, and it does affect them. This is interesting on its own.

Manage data

Firefox wise, I let the system run for a good two months before writing this article. After three instances of cookie database corruption within about two weeks, it's been quiet since the cleanup, and I'm 100% confident that this resolved the problem. And I even have a pretty good technical guess.

The Firefox cookies.sqlite file has a size limit. I noticed the value is not identical across various systems, but the prevalent size is 1,024 KB, and occasionally 2,048 KB. This also dictates how many cookies can be registered into the database. If you hit this limit, Firefox will consider your cookies.sqlite corrupt and rinse it. There is no intelligent mechanism that will remove oldest cookies or the least used ones or try to increase the size of the database.

This is a guess, but I'm pretty confident this is what's happening. And the solution is to cleanup your browser data once in a while. I'm a pretty heavy user, so there were hundreds if not thousands of cookies there, while the meaningful list runs into low tens. Once a year or every 8 months seems like a reasonable household cleaning schedule.

Conclusion

Voodoo articles are never my favorites, because I hate guesswork. Software problems should be deterministic, so you can easily recreate and showcase issues. But like race conditions, which manifest in their odd combos of timing and sequences of events, something like this is hard to replicate, even if the solution mechanism is obvious.

Anyway, if you're running Firefox and get your login sessions (cookies) deleted, my suggestion is to restore from a backup (keep one, yeah) and then perform an orderly cleanup, removing what you do not need and keeping only the minimum set of cookies for your common logins. You can clean manually or use a maintenance tool, although they can sometimes be tricky, but that's a different story altogether. Anyway, that would be all for today. Don't say Captain Obvious, because it ain't. See ya.

Cheers.