Should Ubuntu Have Been Created?

Should Ubuntu Have Been Created?

I am both a Debian and an Ubuntu developer, and I’m sometimes amazed that Ubuntu discusses technical choices that were discussed (and solved) a few weeks earlier in Debian.

—Lucas Nussbaum
What many people don’t understand about Linux development is that it’s truly a team effort:
Red Hat develops the kernel,
Novell develops the applications,
Debian does the packaging,
and Ubuntu takes the credit!

—Joke found on a bathroom stall at LinuxWorld Boston 2005

The next chapter discusses challenges facing the free software community, but Debian/Ubuntu is a specific one to discuss here because it is a case study on the software vehicle that today is the most likely replacement for Windows and the Mac.

While the Linux community has benefited greatly from Ubuntu’s investments and focus on the deficiencies of Debian, it is not clear why Shuttleworth needed to fork Debian to improve Debian in the first place. Hiring volunteers to work full-time is a good way to speed up progress, but they could have done their work inside of Debian if Mark had told them to. If Ubuntu hadn’t been created, and instead those resources spent to improve Debian, things would be farther along today. In addition, there is an argument to be made that both Ubuntu and Debian are hurt by the split.

It is widely accepted in the free software community that Ubuntu and Debian have a special relationship. Ubuntu’s website says that Debian is “the rock” that Ubuntu is built upon. Given that Debian is installed on millions of machines, has been around for 15 years, and has 1,000 developers, this analogy is apt.

While everyone agrees that Ubuntu’s hurting Debian is bad for the free software movement, no one knows the extent of damage to Debian. There are no accepted and published metrics that geeks can use to help analyze the problem — like the number of Debian users who have switched to Ubuntu.⁵ The Debian developer community is growing linearly, while Ubuntu and other free software efforts are growing exponentially.⁶ There certainly must have been a slowdown at Debian around the time of Ubuntu’s creation in early 2004.

We know the changes in Ubuntu could have have been achieved by Debian because Shuttleworth hired ten of the best Debian volunteer developers, and they started work in a 100% Debian codebase. Debian is very highly respected within the Linux community, and the pre-Ubuntu consensus was that it was just missing a little polish and dynamism. This could have been easily fixed, especially with the shot in the arm of a few volunteers transitioning to full-time developers. Other computer companies have done their work directly in the Debian codebase, and Shuttleworth has never given justification as to why he couldn’t adopt a similar strategy.⁷

Geoffrey Moore’s recent book Dealing with Darwin talks about companies getting eaten up by “context”; irrelevant things not “core” or important to the business. With Ubuntu, Shuttleworth created yet another bug-tracking system, source control software, wikis, forums, and in general invested in a lot of infrastructure already in place at Debian. None of that work made Linux any more ready for human beings. Perhaps one-half of Shuttleworth’s early team focused on this non-core work. Even assuming the Debian infrastructure needed work to make it ready for human beings, it is cheaper and better to improve an existing system than to build a new one just to add a few features.

New Linux users are joining the Ubuntu team and contributing to the Ubuntu codebase and community because that is the one set up for them. By analogy, it’s as if someone took Wikipedia, made a few small changes, and this became the website people used rather than Wikipedia itself. If the changes were so good, why were they not made a part of Wikipedia, leveraging Wikipedia’s expertise and processes? Putting the changes directly into Wikipedia would also be better because it would improve Wikipedia, which is what everyone is already using. Forking a codebase is primarily social engineering, with widespread impacts, and goes against the spirit of cooperation inherent in science and free software. In the Linux kernel, the good ideas are incorporated and the bad ones weeded out, but this is all done within the context of one codebase and team. Anyone who wants to improve on Linus’s work can just e-mail him some code changes, there is no reason to create a separate, “competing” kernel.

There is evidence of the inefficiency in the separate organizations. Today, Debian developers and Ubuntu developers are working mostly on separate tracks. One of the best practices I learned at Microsoft was to give a developer full responsibility for a feature. The person adding the footnote code to Word would be responsible for the changes to the user interface, the file format, and the layout engine. This meant collaborating with other developers, but it also made just one person the feature expert, enabling him to make all decisions with a holistic view. Most importantly, it is efficient for this one person to fix any feature bug.

The hard part about doing something is learning how to do it. Making a gourmet dinner, landing the Space Shuttle, or performing open heart surgery is a few hours’ work if you know what you’re doing, but it is time-consuming and nerve-racking if you do not.

When a Ubuntu developer adds a feature, he designs and implements a feature in Ubuntu, but not in the Debian codebase. Now, the only person who understands the change is the Ubuntu developer who made it. Ubuntu publishes its source code on a website, but if a Debian developer grabs it, and runs into problems, he is not an expert in this code yet because it was the Ubuntu developer who first made the change. Therefore, he will need to spend time getting up to speed.

The time to get up to speed is comparable to the time to do the work in the first place. In fact, the Debian developer who integrated a huge set of “X.Org” patches from Ubuntu told me that they were just a “starting point,” unsurprisingly providing little more help than if he had done the work from scratch. I believe that if Shuttleworth understood this concept, he would not have created a separate Ubuntu.

If a different codebase had never been created and all the Debian and Ubuntu developers were working in the same one, they would automatically work more efficiently. They wouldn’t need to redo, and therefore re-learn, what someone else has just done. This would enforce a division of labor and would increase the pace of progress.

Having separate teams is inefficient, but it also hurts Ubuntu’s quality. Whenever a Debian developer is re-learning about a software change first made in Ubuntu, he isn’t using that time to move forward on new things.

Furthermore, Ubuntu isn’t the beneficiary of Debian’s greater expertise, which means their code is buggier than it could and should be. Debian and Ubuntu’s buglist is one of the best metrics today for the set of obstacles preventing world domination. Ubuntu’s user base has grown dramatically, but their small and young team has shown no ability to keep up with the new issues that have come piling in along with the new users. In May 2006, Ubuntu had 10,000 active bugs, and in January 2009, Ubuntu had 76,000.

I discuss more about the challenge of bugs in the next chapter, but the fact that Ubuntu has so many bugs means that there are unsatisfied users, and this is stunting Ubuntu’s growth. Debian’s much larger and more experienced team could provide great assistance, but because the team’s release cycles and bug list are separate, there is no unified effort being made to resolve this challenge.

Because the Ubuntu team is smaller than the Debian team, they argue that they are too busy to take ownership of their work inside Debian. This idea is flawed because if someone else is redoing your work, then you aren’t actually accomplishing anything. “Ya can’t change the laws of physics, Captain Kirk!” If you are not accomplishing anything, it doesn’t matter how busy you think you are. Smaller organizations should actually be more sensitive to wasted work because they have fewer employees.

Shuttleworth claims that Ubuntu and Debian are going after different markets, but he can give no examples of features Ubuntu wants that Debian doesn’t want. If you consider the areas in which Ubuntu has already made engineering investments: simple menus, 3-D graphics, faster startup, and educational software, it should be obvious that Debian wants all of these features as well.⁸

Many of the features that Ubuntu has added, like better suspend and resume for laptops, Debian is no longer motivated to add because almost any Debian user who wanted this feature is now using Ubuntu. Even if Debian does the work, they might not find the bugs because it doesn’t have that many users testing out the feature. Debian is being consigned to servers and embedded, which has always been their strength, however, these are areas now being targeted by Ubuntu.

In a recent blog post, Shuttleworth wrote that he admires Debian’s goal of building a universal operating system, but he also said in the same post that he believes its objectives are unrealistic. Mark should trust his idealistic side and realize that because software is infinitely malleable, his software innovations can be put into Debian. (It might have to be a separate package, but that is easily doable.) There are strongly unified teams building Wikipedia and the Linux kernel, and their success stories can be applied here.

There is understandably a fair amount of bitterness around, which itself decreases the morale and productivity of the community. Debian has spent over a decade doing foundational work, but Ubuntu has made just a few improvements and grabbed all the excitement and new volunteers. I can’t think of a better way of killing Debian than what Shuttleworth is attempting. I believe he is aware of this issue, but just ignores it.

A separate user community is inefficient, but this is dwarfed by the inefficiency of the separate developer community. The greatest long-term threat to Debian is that they stop accumulating institutional knowledge. The best way to prevent this is to encourage all Ubuntu users who wish to become contributors to directly join the Debian community. Debian is filled with many experienced programmers and is a great place to receive mentoring. And because changes are automatically propagated over to Ubuntu, work in Debian helps Ubuntu.⁹

I was honored to be a speaker and discuss some of these issues at the annual Debian developer conference in June 2007 in Scotland. A great thing about the free software community is how you can meet and talk with everyone. However, because Ubuntu is one of the most successful Linux distros and has been around for several years, the status quo is accepted. People who only look at the success of Ubuntu are ignoring the opportunity cost of doing things in a better way. One of the reasons that the progress of Linux is so slow is that the community is constantly shooting itself in the face. A big part of Microsoft’s success is the incompetence of its competitors.

If Ubuntu and Debian were to combine their resources, it would eliminate animosity and the end result would be more innovative and reliable. This organization would be in a very good position to replace Windows. The first free software distribution with a community of 10,000 developers wins.

If you’d like to learn more about my book (this is five pages of it) including how to get a free download, click here.

Mark Shuttleworth and I continue the conversation here.