Free download:

Software Wars, the Movie

Soundtrack for the book:


If you enjoyed the free download, a donation of the cost of a newspaper would be appreciated.



The best book explaining free market economics:

The best explanation of how we can build a space elevator in 10 years:


Book explaining a political solution to our economic problems:


To Mark Shuttleworth on Ubuntu / Debian

Version 1.03

Dear Mark,

Thank you for your feedback on my book excerpt about Ubuntu / Debian, and I appreciate your time.

The main concern I have about the current situation is this: when an Ubuntu person does some coding work and posts the diff to a website, it is now a workitem for someone else in Debian to dig into and understand. Of course, having that fix is helpful, but the time to do something is mostly the time to learn how to do it.

If you pick up a violin, it will be clumsy to you unless you know how to play one. It is the same with a patch if you haven’t seen it before. So therefore, whenever someone learns something, they should make sure that they do their work in both Debian and Ubuntu to save someone else time.

Neither you nor anyone else has ever acknowledged this problem or attempted to explain how this won’t happen all the time in the current situation.

With separate trees and buglists and such, people no longer work together simply as a matter of course of the tools they use. As a DD told me: this is social engineering. There are ways to mitigate this, but it is a problem that wouldn’t exist if it weren’t for the social engineering that preceded it.

Forks

In essence, I think forking is like war: it means that the discussions and diplomacy have failed. Any feature can be done in software, it is just a matter of technical and social work. So when you created Ubuntu, you were saying (before trying) that what you wanted to do couldn’t be done in Debian. This is an ironic statement given that you were starting with Debian and 10 of its best developers.

Note that forking is different from branching which happens all the time in DVCS. There are many branches of the Linux kernel out there, but the team is very unified. In the Ubuntu / Debian world you have separate documentation, incompatible packages, separate conference schedules, email lists, bug-tracking systems, and most importantly: separate teams maintaining a number of the same packages.

I know that some here might bring up a few good reasons for a fork:

  1. Non-free code. That affects only a few packages, and Debian already has mechanisms to handle non-free code. Furthermore, it is possible to create two versions of a Live CD with and without the questionable stuff. So for simplicity here, let’s leave out freeness issues and assume you agree with Debian policies.
  2. Having 6-month ship cycles. I have spent hours thinking about it and I concluded that this is doable with a separate tree in the Debian infrastructure. I’d rather not get into that here.
  3. You state that Debian’s greater generality is something you aren’t interested in. You brought up Debian’s larger number of supported architectures as an example. I address that in footnote 8, but I’d add that Ubuntu supports more architectures in every release so it seems even this is getting smaller. Furthermore, writing portable code is not that hard, especially if you write in a garbage collected programming language. In nearly all GC languages, you use the same “binary” on all platforms.

So when I subtract out those difference, I run out of reasons for the separation.

I realize that achieving consensus on things is work, especially in the distributed nature of a volunteer movement. But this is not your responsibility, it is the responsibility of the people doing the work. You say that you don’t think you could have steered Debian in the direction that you want, but every contributor to Debian steers it. You still have to achieve a consensus within Ubuntu, and as it gets bigger, it will also get somewhat harder. And as for whether it is hard to achieve consensus in Debian, I would just point out that there are hundreds of DDs and millions of users, so they’ve managed to make it pretty far.

Money

You assert that mixing money with volunteers would have been a problem for you. Of course, we can’t know. You reference Dunc-tank; this is something I do not address in my book because I felt it was too “inside baseball”. But it seemed like there were a number of problems with it, for example, the secrecy with the way the program was developed.

The failure of Dunc-tank is not proof that you can’t mix paid folks and volunteers. And isn’t that exactly what is happening in the Ubuntu community?! We can also look to the HP example. HP pays programmers to work on Debian, and makes money supporting Debian. So this situation, and others, have not turned into a problem. Likewise, Linus gets paid to work on the Linux kernel and yet this has not caused problems.

People should contribute to free software only when they want to. If you do something because you want to, it doesn’t matter what other motivations other people have to do what ever they are doing. I would love to get paid for taking pictures, and the fact that some do doesn’t make me enjoy taking pictures less.

You can also think about the money like this: someone is working on Debian their few hours a week, and if someone else shows up and starts working 40-50 hours a week, that is great. The more people the better! I presume you’d like to hire as many of the great Debian people as you could. The world needs more full-time programmers working on free software. Many of the other Debian volunteers are in school or using it in their company so they can’t work on Debian full-time.

I realize that mixing volunteer and paid work is a tricky issue. But the solution is not just to assume that humans are flawed and so create new institutions which has the same potential problem! Note that there is also tension in what you are doing today if you hire Debian people to do work that never makes it into Debian.

Fixes

I agree that there are people looking for conspiracy and malice. Part of your challenge comes with the territory of being a suddenly very important part of a movement filled with passionate geeks. I also think that you guys were slow to adopt the GPL / AGPL for the code that you write, for example. The urge to hoard is strong and maybe some in Ubuntu don’t understand yet that it was free software, and Debian, that made this all possible. Free software and all that it implies almost requires a bit of faith.

From my perspective, the toothpaste is out of the tube. I don’t know exactly what could be done, but I also don’t consider all the possibilities because I am far away. I feel that the relationship will be uneasy as long as it continues as it is today.

You say that many people in the Ubuntu community recognize the importance of Debian now. That is good to hear. I didn’t get that sense as much in 2007. At the Debconf you suggested that people should just work in whatever codebase they wanted to, and not necessarily both.

I think that a merge is not the big task it sounds because the entire Debian and Ubuntu would join in. But even today, I think there more things than can be done. Here is a thought experiment. Suppose you decided that Bzr is duplicate code and therefore the Launchpad repositories should be converted to Mercurial. This thought experiment doesn’t get Ubuntu much closer to Debian, but it is the kind of “out of the box” thinking one could consider. Maybe there is a way to fast-track Ubuntu people into the Debian team? What about cancelling MOTU and having them all join the Debian team? How can Debian’s debbugs and Alioth work better with Launchpad? What about having UDS at the next Debconf? What about sharing documentation and wikis?  I feel like one could come up with a long list of things big and small that would squeak greater efficiency and improve the situation.

You state that you don’t think Ubuntu can ever be loved by Debian developers. I hope you and your org don’t quit trying. No situation need ever be stagnant and the OS is such a critical piece of the free software movement.

One of Linux’s top challenges

I wrote this book for two reasons: explaining why free software is superior and what things can be done for faster world domination. (You can find a short version of the to-do list here.) It will not be easy to beat Microsoft.

Many, even in our movement, believe that free software will never achieve world domination on the desktop, and so I focused on that. One of the reasons why Unix never was much competition for Windows was because there were separate teams re-implementing each other’s features. This Ubuntu / Debian inefficiency is one of the big things I came up with. I think what you have done has been amazing for Linux, but Ubuntu is buggier and moving more slowly than it could be.

I presented a version of these ideas in Debconf 2007 to you and others, and listened carefully to all the feedback, but I didn’t feel that my main point — about the violin — was understood. So I’ve left it in for now. In fact, I met a number of people at that Debconf who agreed with me and gave me evidence of other inefficiencies. One DD was surprised that I understood the Debian and Ubuntu situation so well even though I am not really a part of either! And my Microsoft friends I’ve talked this over are easily convinced that it is an example of the FOSS community shooting itself in the foot.

This is not an issue I push much anymore, so I consider it mostly as food for thought as a case study on forks, and because if Ubuntu’s developers understand the importance of joining the Debian community, then the inefficiency can be minimized. It is cheaper for one person to do work twice than for two people to do the same thing.

Respectfully,

-Keith

10 comments to To Mark Shuttleworth on Ubuntu / Debian

  • […] This post was mentioned on Twitter by Yee Hon Choong and glurt, Victor Acioly. Victor Acioly said: RT @webupd8: To Mark Shuttleworth on Ubuntu / Debian http://bit.ly/bJpHFU #linux […]

  • Dulwithe

    Spectacular letter!!!

    D.

  • booker

    It’s all about the money. The whole argument for the existence of Ubuntu begins and ends there, whether it’s the referrer codes passed in amazon and ebay searches in Firefox, Ubuntu One, or what have you.

  • hitchcock

    Why call out just Ubuntu and Debian? I just stumbled onto these pieces and i havent followed your work or thinking in the past, but from what i see you are concentrating your focus on Ubuntu and Debian dynamic. Wouldn’t it solve a broader problem if you were to apply your theory across all distros? In other words, have 1 linux distro, not 1 million of them. Because maybe Red Hat has fixed something that Debian has had problems with for a while and now, according to you, if they wanted to take the same approach to fix it, it would take a lot of time for the debian dev to figure out the red hat devs code.

    Don’t get me wrong, I know something like what i suggested would be shot down as crazy by the community but, maybe it is actually in the best interest of linux to combine and strengthen as a whole and not by distros. I know that there is way to much money between Red Hat, Ubuntu and Novell that we could never get them all to play nice and combine, but think of the possibilities.

    • That is a good question. I sort of address it in the sections of the book before and after this. Many of the distros like Damn Small Linux and Mint are small efforts that are mostly subsetting and making minor tweaks that don’t create the problem that Ubuntu did.

      As Debian and Ubuntu are two of the biggest distros, what they do matters more. The market is big enough to support multiple commercial distros, but the “community” should work together.

  • jorge

    Maybe there is a way to fast-track Ubuntu people into the Debian team? How can Debian’s debbugs and Alioth work better with Launchpad? What about having UDS at the next Debconf? What about sharing documentation and wikis? I feel like one could come up with a long list of things big and small that would squeak greater efficiency and improve the situation.

    I know that we have Ubuntu developers who have gone on to apply to become DD’s. Lucas Nussbaum has been working on ways to better measure bug and patch contributions back to Debian.

    I am interested in discussing your ideas, are you planning on coming to Debconf in NYC?

    • Hi Jorge;

      It is great when Ubuntu people go on to become DDs. I am glad if the community stays close. The natural state of affairs is to drift away. I was quite concerned about that in 2007. Mark and others made it seem like putting work back in Debian wasn’t a priority and not worth it. (From his perspective, why bother creating Ubuntu if the most of work is also going to be done twice in Ubuntu, and Debian? It is a good question!)

      I wasn’t planning on going to NYC this year. People closer have better ideas. I’m glad to see the work Lucas is doing. If the community understands the importance of doing code changes in both trees and working together, the social engineering inefficiencies can be minimized.

      Warm regards,

      -Keith

  • Hello,
    At mid-July 2009 a long (and heated, until August) discussion thread started at debian-project list regarding release dates, freeze dates, cadence and collaboration between Debian and Ubuntu.
    http://lists.debian.org/debian-project/2009/07/threads.html
    Mr. Shuttleworth suggested modifying Debian dates in order to sync with Ubuntu LTS.
    http://lists.debian.org/debian-project/2009/08/msg00092.html
    The long discussion resulted in a suggestion of feasible collaboration approach from Debian
    http://lists.debian.org/debian-project/2009/08/msg00273.html
    And then Ubuntu / Canonical started a task
    https://blueprints.edge.launchpad.net/ubuntu/+spec/foundations-lucid-release-collaboration-with-debian
    that currently had achieved some advancements in coordination with Debian:

    Whiteboard

    Work items:
    kernel: DONE
    [vorlon]python: DONE
    [vorlon]gcc: DONE
    perl: DONE
    openoffice: DONE
    [kitterman]boost: DONE

    Work items for ubuntu-10.04-beta-1:
    [bryceharrington] X: DONE

    Work items for ubuntu-10.04-beta-2:
    [doko] java: DONE

    Status: Kernel=2.6.32, GCC=4.4, Python=2.6, OOo=3.2, Perl=5.10.1, Boost=1.40 (with the freeze delay, Debian looks like it’s going to 1.42, but it’s probably not worth the pain of us doing another transition for Lucid), Mesa=7.7, xserver=1.7, Java (IcedTea)=1.8

    I hope these first efforts, that are being observed by other teams at both sides, could result valuable enough to motivate other teams to participate, at BOTH sides.
    At a win-win agreement, all involved benefit from.

    Good luck for all.
    Andre Felipe Machado

  • Hello,
    It is worth to remember that Canonical is a for-profit company, and Debian Project is a community project:

    Scientific study about Debian Project governance and social organization
    http://www.techforce.com.br/news/linux_blog/scientific_study_about_debian_governance_and_organization

    So, it is unlikely that a “full merge” could be ever possible.

    Even so, there are the Debian Pure Blends
    http://wiki.debian.org/DebianPureBlends
    http://blends.alioth.debian.org/
    That may be a home for a “different” distro built upon Debian, but still Debian.

    Debian Project has the non-free repository section, but there are other sw that needs per-copy licenses.
    Well, these are complex business questions that could be discussed, as HP and IBM approaches to participate but to not own a distro.
    Regards.
    Andre Felipe Machado

    • Hi Andre;

      I will check out that study tho I’m somewhat familiar with the history of Debian.

      Ubuntu has governance structures as well, and is in fact developing the same ones and learning the same lessons in the process.

      When I talk about merging, I talk about people and processes. There are commercial companies that contribute to the Linux kernel. Where someone’s paycheck comes from is a different issue. Canonical doesn’t need to own Debian to have lots of influence. Its influence would be in proportion to the number of people working on it.

Leave a Reply

  

  

  

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>