Linux fragmentation - The Sum of All Egos

Updated: May 18, 2018

If Tom Clancy had been a technophile of the software kind, he'd have used this title instead of the familiar one for one of his iconic blockbuster thrillers. The thing is, Linux accounts for a tiny percentage of the overall desktop market share. The perennial 1% has been around roughly since 2005, and even if the actual share is higher than that, it's still a small and largely insignificant fraction. And yet, there are hundreds of Linux distributions populating this narrow, crowded arena. Why? Well, ego, of course.

One might say: open source. Ah, well, the open-source nature of Linux has been the chief excuse to the colorful abundance of replication and duplication of the Linux desktop world, while at the same time serving as the main catalyst to the expansion of Linux in the commercial space, which makes for a dubious cause. I believe the reason is different. Let me tell you what it is.

Teaser

You can't fight nature

Let's begin with a bit of background, shall we.

Linux desktop distributions are mostly developed by non-professionals (the word amateur is wrong here), who do so mostly in their spare time, to the best of their available abilities (financial, technical, temporal). Sometimes, these distributions are a product of individuals, and sometimes relatively small groups of people, often with a very strong focus on pure development and almost no focus on other aspects of what one might call productization. Arguably, this model seems to be working well well, ever since Papa Linus created the kernel back in the early 90s. Only it's an illusion.

The kernel is a tightly controlled entity, with one person in charge (Linus). The kernel is mostly developed by big corporations like Intel, IBM, Google, Microsoft, Red Hat, and some others, with roughly 80% of all code contributions coming from the enterprise world - with people getting paid to do so, and with clear objectives from above. Lastly, the kernel is packaged with ten years of support and backward compatibility, in the case of the two big players - Red Hat and SUSE - and it's a non-user-facing technology.

In one sentence, it's everything a typical desktop distribution isn't.

I have argued this at OSS Europe 2017 in Prague, Czech Republic, in my talk titled The Empire Strikes Back - We Just Need an Emperor. You may also want to glimpse some fancy slides yonder, but for now, stay with me, that is, of course, if you actually find this article worth your time.

Linux distributions are meritocratic products. Actually, no. The word merit is wrong. Pluralistic is more like it. Developed by consensus, most of the time - unless there's bickering or disagreement, which means spins and forks of said projects - often with no clear objectives - there's no higher goal, no commercial incentive or responsibility for the product or user satisfaction. Take it or leave it.

As such, Linux distros go against the very nature of what products ought to be, and how people expect to consume them. Which means they cannot succeed.

Any attempt to go against the natural order of things is bound to fail. This is because everything in nature aims to minimize its energy, i.e. reach its optimal steady state, in which there be balance and harmony. If a system is not in this state, it requires a constant input of energy to maintain. Linux distributions work against the natural hierarchy, and so they collapse without sufficient input of effort into their existence.

The human society is a pyramid. Workplaces are pyramids. Some workplaces pretend to be nuovo-chic so they use words like open spaces and flat structure and similar corporate nonsense, but it's nothing more than a decoy to keep the employees happy while slaving away behind the keyboard with a feeling of freedom and independence that they don't really have.

If you're wondering if Dedo be talking bullshit, think about your workplace. Does it have a managing director? Does it have someone in charge of finances? Does someone call shots? The answer is, if it's more than an early startup or some embryonic experiment at some hi-tech campus, you bet your ass those people are there, and they are ABOVE you in the hierarchy, and that's how it's always been, since early Sumeria to medieval Japan to modern-day software companies.

This means that if a Linux distribution has no boss, i.e. leader - and that it is led by a committee, then it will not get much done. In fact, this is true of any software project. But let's focus on the distros. Committees actually force groups to normalize to the lowest common denominator whereas pyramids force people to normalize upwards (toward a higher denominator). Of course, just need the pyramids to be designed right, too, as there can't be too many people at the top, or too many at the bottom. But overall, if you need to get something done, doing it in teams is the best way to kill efficiency.

Leader

Even ducks like hierarchy, and they don't even code!

Note: Image taken from freeimages.com, courtesy of Pierre Amerlynck.

Again, if you need to relate, think of any "decision making" meetings at a work place. Team work works ONLY when there's a clear sense of direction and strategy. Someone decides what needs to be done. Then, there's an element of freedom and flexibility at a lower point in the hierarchy structure, where you can achieve more by grouping people to perform tasks, usually within a narrow scope. But there has never been a single great thing achieved by having everyone sit down and discuss company goals.

Because people are not equal - in skill, talent, ambitions, efficiency, or ability to abstractize. Some people are natural leaders. Some like to follow. Some work well with ambiguity. Others hate uncertainty. Some are great tacticians, others love micro-management. Focus and attention to detail, perseverance, orderliness, there are millions of qualities that we do not share equally with colleagues, and so, having work done by everyone means the top talent doesn't get to do what they should be doing, and the parasites at the bottom keep on leeching resources.

The word democracy comes to mind. The political system is supposedly a concept of equality - one person, one vote, and indeed, it provides stability, because changes and fluctuations are absorbed by the big average. Then, if you think more deeply, even in democracies, you have leaders - political leaders on various levels, and eventually, a small cabinet or forum or whatever of elected decision-makers.

Back to Linux. Pluralism is one of the main reasons why Linux distributions are amorphous entities. If no one calls the shots, the shots do not get called. And so, you have software projects that exist for no other reason than that they have been conceived.

I have also written on how to make Linux professional - and not surprisingly, my ideas mirror fairly closely the structure and imperatives of successful companies. Sadly, it's the way it is. There are reasons why time and again, after thousands of years of civilization, we come back to the same model. It's not because we're all collectively stupid, it's because that's the natural, hence optimal way of doing it. By having hierarchies in place, we manage chaos and success, and this has led us from herding gnu (almost a pun) on the plains of Africa to using UTF-8 encoded smileys on our touch phones. But it's also given us space flight, nuclear weapons, and vaccines.

But all this is just a side show for something much bigger. The ego game.

Come together

Linux be open-source, right. So, technically, you take some code and then you build off of it, and you make sure the code you write is accessible to others. In a nutshell, this is the basic idea behind most of the Linux-related work. And it is often given as a reason for the marvelous growth and success of Linux.

I think this is a sweet illusion. The open-source nature of Linux is mightily helpful in some areas, but then it sure has not helped Linux break through in the desktop space. On the contrary, closed-source solutions rule this space, and even in the mobile world, Android is just an enablement platform for a proprietary ecosystem dominated by Google, and far more importantly, millions of applications.

One thing that the open-source nature of Linux did allow is for the number of distributions to grow and proliferate. Any disagreement, be it technical or ideological, eventually becomes a fork in software code. Because, why not.

Over the years, we're seeing more and more Linux distributions getting created and entering the already saturated and stagnant 1% slice. Since we must assume there's a finite pool of intellectual resources, this means that every new distros is leaner on capabilities than before, and that it is also getting farther and farther from the pyramid model, as well as the minimum unit size needed to sustain itself. I'm not exactly sure what that number is, but you need at least 25-35 people to have a successful self-sustaining workspace. You can sometimes get away with less, but that's statistical anomaly and luck.

Society

Users searching for a bug. Ha ha.

Note: Image taken from freeimages.com, courtesy of lev olson.

Think about it. The one big breakthrough in the Linux world was - and still is - Ubuntu. Everything changed once Mark Shuttleworth decided to step into this world, business savvy, rich and focused. In a short span of years, he commercialized the distribution he created, he set up a fairly successful company, and he made the Linux desktop recognizable and synonymous with Ubuntu. We have the classic business model - a leader, a company, clear goals, things being run with real objectives. Has not been replicated since by any pluralistic model.

With the shifting of attention away from the desktop, and who can blame Canonical for that, right, the Linux desktop has gone back to what it has always been. A sort of a sandbox where people can develop code for their own enjoyment, at their leisure, without any big commercial goals. That alone would be wonderful.

Except - every single distro out there labels itself as the easiest, friendliest system in the world.

And every distro is trying to get bigger.

So we have a paradox.

Egos

What actually prompted me to write this article was that I had a moment of nostalgia and reminiscence while writing The Magnificent Seven unique Linux projects article. I went through the last decade of playing with Linux, and I found some real gems, things that might have been fantastic, except they came, and they went, and nothing happened.

I realized that the world of Linux development is very much like any corporate place, with an added bonus of pretending to be above the typical bickering and backstabbing. But the space is polluted with the same toxic attitude, the same passive-aggressive games, only it's done outside shiny offices.

Everyone wants Linux to succeed, but they want THEIR Linux to be the one that succeeds, right.

Distributions, hundreds, and even if you put all of them together, you have fewer developers than the number of just UI designers employed by Apple alone. Most of the projects have 3-4 people in the team in the best case, usually all of them developers, with zero skills in managing projects, customer focus, marketing skills, or anything you need for successful products, to say nothing of the finances. But let's not focus on the dev-led model, which is a different problem entirely.

From distributions, you can easily jump into any which other software category closely associated with the same development model. Why does Linux needs a dozen music players or PDF readers or anything? All of these are half-baked, incomplete solutions caught in a vicious Battle Royale - or Highlander - scenario. No one wants to yield. No one wants to cede an inch. And it's getting worse and worse.

This is not a new topic. I talked about Linux market back in 2009, I talked about the state of the Linux community five years ago. Many years back, I also wrote about the ego problem with small distros, and essentially, not much has changed since.

Like I said, it's getting worse.

You often hear about new distros forking, because A didn't get along with B - a privilege you don't have at the workplace, where you're forced to cooperate with others, for better or worse. But how many times do you hear of a distro voluntarily relinquishing its manpower and code in favor of another? How often do distros merge?

Imagine what happens if you ax distros 11-100 on DistroWatch list and distribute their resources among the top ten. You now have hundreds of people to work on the same project - lack of resources is always quoted as a big problem in the distro world, when it's almost entirely a self-imposed problem. But then, we need a pyramid, right. This means not everyone gets to work on what they feel like. Some people will have to do QA all the time. Boring, right? Someone will need to write documentation. Someone will always be there testing patches.

Narcissus

Distro, distro on the wall, who's the forkiest developer of them all?

But in the real world, we have hundreds of people who do not wish to conform and cooperate with others. Which is absolutely fine, don't get me wrong. But then, if you do not want to be a part of the bigger collective, why peddle your solution onto the masses, give it a shiny logo, and call it the easiest system in the world?

If you had to analyze similar scenarios at any typical workplace, you would use the phrase "snowflake syndrome" within minutes. But in the open-source world, the wild, rebellious, antagonistic nature of the typical developer is hailed as some kind of special gift for some reason. The really sad part is that most of the Linux folks are not really like that - those who work do NOT behave this way in their offices, and those who have friends sure do not treat them the way they blast away in forums and on IRC. It's Linux that gives platform to the anger and disdain. It's expected.

Yup, the open-source pluralistic nature of the distro-sphere actually enhances the negative qualities in the people involved. It breeds antagonism, and it promotes mistrust. Because people are ultimately selfish creatures, and when not bound by a common goal, they drift apart to form their own little tribes. If anything, the Linux distro world most resembles the proto-humanity before it discovered higher hierarchy orders and the benefits they bring to the species as a whole.

However, we cannot disregard the fun element of the whole deal.

Most people developing Linux are doing it for their own sake - to learn and explore new things, to enjoy themselves. Absolutely fine. The only, ONLY issue is when they try to sell their fun as the product. There's no end goal to one's fun, so there won't be any in their products, either. Which is why regressions come and go, desktops are created and forked and created some more, and nothing meaningful really happens.

Users have different expectations. They sure have zero sympathy for someone else's sense of fun.

Which is why, when you give Linux to an ordinary person, it's a big case of fail.

Conclusion

I do not think this will be solved soon, or ever. Linux allows a whole new dimension of creativity and character freedom that workplaces do not always allow. But the problems begin once people try to build business models they know from the strict corporate world in their little private ponds of chaos. That. Does. Not. Work.

I have always claimed that 90% of all distributions are unnecessary. I still believe that, and I think the actual percentage is even higher. Only a few systems should remain, and the superb, unique ideas from individual projects merged. Because yes, you do get occasional brilliance in Linux, like everywhere else in life. Grab the SUSE installer, grab the MX Tools, grab Ubuntu Dash, and cobble together a wonder. But that means defeating the ego. Joining the pyramid. And that means there are those who orchestrate, those who create, and those who oil the gears. In the end, for most people, it comes down to one question: Are you willing to be the lowly QA tester for the Great Success to the Benefit of Glorious Linuxstan? Food for thought.

Cheers.

You may also like: