Free download:

Software Wars, the Movie

Soundtrack for the book:


If you enjoyed the free download, a donation of the cost of a newspaper would be appreciated.



The best book explaining free market economics:

The best explanation of how we can build a space elevator in 10 years:


Book explaining a political solution to our economic problems:


To Mark Shuttleworth on Ubuntu / Debian

Version 1.03

Dear Mark,

Thank you for your feedback on my book excerpt about Ubuntu / Debian, and I appreciate your time.

The main concern I have about the current situation is this: when an Ubuntu person does some coding work and posts the diff to a website, it is now a workitem for someone else in Debian to dig into and understand. Of course, having that fix is helpful, but the time to do something is mostly the time to learn how to do it.

If you pick up a violin, it will be clumsy to you unless you know how to play one. It is the same with a patch if you haven’t seen it before. So therefore, whenever someone learns something, they should make sure that they do their work in both Debian and Ubuntu to save someone else time.

Neither you nor anyone else has ever acknowledged this problem or attempted to explain how this won’t happen all the time in the current situation.

With separate trees and buglists and such, people no longer work together simply as a matter of course of the tools they use. As a DD told me: this is social engineering. There are ways to mitigate this, but it is a problem that wouldn’t exist if it weren’t for the social engineering that preceded it.

Forks

In essence, I think forking is like war: it means that the discussions and diplomacy have failed. Any feature can be done in software, it is just a matter of technical and social work. So when you created Ubuntu, you were saying (before trying) that what you wanted to do couldn’t be done in Debian. This is an ironic statement given that you were starting with Debian and 10 of its best developers.

Note that forking is different from branching which happens all the time in DVCS. There are many branches of the Linux kernel out there, but the team is very unified. In the Ubuntu / Debian world you have separate documentation, incompatible packages, separate conference schedules, email lists, bug-tracking systems, and most importantly: separate teams maintaining a number of the same packages.

I know that some here might bring up a few good reasons for a fork:

  1. Non-free code. That affects only a few packages, and Debian already has mechanisms to handle non-free code. Furthermore, it is possible to create two versions of a Live CD with and without the questionable stuff. So for simplicity here, let’s leave out freeness issues and assume you agree with Debian policies.
  2. Having 6-month ship cycles. I have spent hours thinking about it and I concluded that this is doable with a separate tree in the Debian infrastructure. I’d rather not get into that here.
  3. You state that Debian’s greater generality is something you aren’t interested in. You brought up Debian’s larger number of supported architectures as an example. I address that in footnote 8, but I’d add that Ubuntu supports more architectures in every release so it seems even this is getting smaller. Furthermore, writing portable code is not that hard, especially if you write in a garbage collected programming language. In nearly all GC languages, you use the same “binary” on all platforms.

So when I subtract out those difference, I run out of reasons for the separation.

I realize that achieving consensus on things is work, especially in the distributed nature of a volunteer movement. But this is not your responsibility, it is the responsibility of the people doing the work. You say that you don’t think you could have steered Debian in the direction that you want, but every contributor to Debian steers it. You still have to achieve a consensus within Ubuntu, and as it gets bigger, it will also get somewhat harder. And as for whether it is hard to achieve consensus in Debian, I would just point out that there are hundreds of DDs and millions of users, so they’ve managed to make it pretty far.

Money

You assert that mixing money with volunteers would have been a problem for you. Of course, we can’t know. You reference Dunc-tank; this is something I do not address in my book because I felt it was too “inside baseball”. But it seemed like there were a number of problems with it, for example, the secrecy with the way the program was developed.

The failure of Dunc-tank is not proof that you can’t mix paid folks and volunteers. And isn’t that exactly what is happening in the Ubuntu community?! We can also look to the HP example. HP pays programmers to work on Debian, and makes money supporting Debian. So this situation, and others, have not turned into a problem. Likewise, Linus gets paid to work on the Linux kernel and yet this has not caused problems.

People should contribute to free software only when they want to. If you do something because you want to, it doesn’t matter what other motivations other people have to do what ever they are doing. I would love to get paid for taking pictures, and the fact that some do doesn’t make me enjoy taking pictures less.

You can also think about the money like this: someone is working on Debian their few hours a week, and if someone else shows up and starts working 40-50 hours a week, that is great. The more people the better! I presume you’d like to hire as many of the great Debian people as you could. The world needs more full-time programmers working on free software. Many of the other Debian volunteers are in school or using it in their company so they can’t work on Debian full-time.

I realize that mixing volunteer and paid work is a tricky issue. But the solution is not just to assume that humans are flawed and so create new institutions which has the same potential problem! Note that there is also tension in what you are doing today if you hire Debian people to do work that never makes it into Debian.

Fixes

I agree that there are people looking for conspiracy and malice. Part of your challenge comes with the territory of being a suddenly very important part of a movement filled with passionate geeks. I also think that you guys were slow to adopt the GPL / AGPL for the code that you write, for example. The urge to hoard is strong and maybe some in Ubuntu don’t understand yet that it was free software, and Debian, that made this all possible. Free software and all that it implies almost requires a bit of faith.

From my perspective, the toothpaste is out of the tube. I don’t know exactly what could be done, but I also don’t consider all the possibilities because I am far away. I feel that the relationship will be uneasy as long as it continues as it is today.

You say that many people in the Ubuntu community recognize the importance of Debian now. That is good to hear. I didn’t get that sense as much in 2007. At the Debconf you suggested that people should just work in whatever codebase they wanted to, and not necessarily both.

I think that a merge is not the big task it sounds because the entire Debian and Ubuntu would join in. But even today, I think there more things than can be done. Here is a thought experiment. Suppose you decided that Bzr is duplicate code and therefore the Launchpad repositories should be converted to Mercurial. This thought experiment doesn’t get Ubuntu much closer to Debian, but it is the kind of “out of the box” thinking one could consider. Maybe there is a way to fast-track Ubuntu people into the Debian team? What about cancelling MOTU and having them all join the Debian team? How can Debian’s debbugs and Alioth work better with Launchpad? What about having UDS at the next Debconf? What about sharing documentation and wikis?  I feel like one could come up with a long list of things big and small that would squeak greater efficiency and improve the situation.

You state that you don’t think Ubuntu can ever be loved by Debian developers. I hope you and your org don’t quit trying. No situation need ever be stagnant and the OS is such a critical piece of the free software movement.

One of Linux’s top challenges

I wrote this book for two reasons: explaining why free software is superior and what things can be done for faster world domination. (You can find a short version of the to-do list here.) It will not be easy to beat Microsoft.

Many, even in our movement, believe that free software will never achieve world domination on the desktop, and so I focused on that. One of the reasons why Unix never was much competition for Windows was because there were separate teams re-implementing each other’s features. This Ubuntu / Debian inefficiency is one of the big things I came up with. I think what you have done has been amazing for Linux, but Ubuntu is buggier and moving more slowly than it could be.

I presented a version of these ideas in Debconf 2007 to you and others, and listened carefully to all the feedback, but I didn’t feel that my main point — about the violin — was understood. So I’ve left it in for now. In fact, I met a number of people at that Debconf who agreed with me and gave me evidence of other inefficiencies. One DD was surprised that I understood the Debian and Ubuntu situation so well even though I am not really a part of either! And my Microsoft friends I’ve talked this over are easily convinced that it is an example of the FOSS community shooting itself in the foot.

This is not an issue I push much anymore, so I consider it mostly as food for thought as a case study on forks, and because if Ubuntu’s developers understand the importance of joining the Debian community, then the inefficiency can be minimized. It is cheaper for one person to do work twice than for two people to do the same thing.

Respectfully,

-Keith

Nanotrek

Interesting picture from:

http://www.neatorama.com/2010/03/17/nano-trek/

Open Data

There are a lot of people worried about “open data”, like Matt Asay and Tim O’Reilly.

Of course, they call free software “open source” and they think the lack of open data it is a big deal with companies like Google threatening to control what links we click on and companies like Facebook using free software but not letting you easily migrate your data to Myspace, etc.

But I disagree. The biggest problem with Google is their lack of support for free software. They’ve only started releasing stuff in the last few years, and a lot of it is the plumbing code, not cool the stuff that AI researchers of all kinds would be digging over. Perhaps it is because people have been talking about “open data” instead of free software. Even if you could export data from Google’s search servers, you wouldn’t necessarily be able to make sense of it if you didn’t have the accompanying software.

The free software battle is much more important than the open data battle. It is not really that hard to get your data out of Facebook or out of Amazon EC2. It is your data and you put it in there. And this can anyway be solved by a few standards or even some free import / export code.

The open data movement should be a (small) part of the free software movement.

Software War underway

Get your popcorn!

Microsoft is reviving itself right now. It has the money and the expertise. Windows 7 is a major improvement over Vista, even though it demonstrated the holes in Microsoft’s aging technology stack.

Windows 7’s additional 3 years of debugging, plus a new Windows Phone 7, Silverlight, Windows Azure, IE 9, Bing, Natal, Office 2010, etc. show how Microsoft is shoring up its weaknesses and taking the initiative. That is great as it makes the battle fun to watch.

But the seeds of Microsoft demise have already been sown. Microsoft will lose because it doesn’t understand that it is an “encyclopedia” of free software for every task that you can imagine that is the true power of the computer. Wikipedia is just one thing you can build using free software, and it is the million-man free software army that is working on this problem and gaining critical mass.

Hardware First

The first thing the free software community needs to do is convince every hardware company to fully support Linux: feature-complete GPL drivers in the Linux tree before the device ships. This should be done for all components, not just the ones inside PCs that ship with Linux today.

Hardware companies already know that it is in their self-interest to support free software, but they just aren’t executing. But as governments and companies and users continue to request Linux, things will improve. Note, it doesn’t matter that Windows 7 is much better than Vista because large organizations are now making the switch, and pushing PC vendors to better support Linux over time.

The Long Tail of Software

Perfect hardware support is something that Microsoft already promises, so this by itself will not be what convinces people to switch to Linux — it just makes this possible. What we need is to keep improving the various software programs for every task you can imagine in art and science. We also need to push people to write it in Python rather than C/C++. We have enough programmers today, but too many big and old codebases that need a coat of polish.

Do Not Mess With This Guy