Free download:

Software Wars, the Movie

Soundtrack for the book:


If you enjoyed the free download, a donation of the cost of a newspaper would be appreciated.



The best book explaining free market economics:

The best explanation of how we can build a space elevator in 10 years:


Book explaining a political solution to our economic problems:


Open Letter to the new Debian Project Leader

Dear DPL Anthony Towns,

I read your post-election comments about your mission as DPL to be increasing the tempo of Debian, and I think that is an excellent theme. This is the 21st century: Google existed only on paper 8 years ago, China and India are just coming online and will eventually contribute to OSS in a manner commensurate with their size, and maybe even in the Middle East and Africa they’ll quit killing each other or dying of pandemics and become productive members of the modern world. In 20 years, Linux will be running on 1 billion devices smaller than a PC and 1 billion PCs and Nicholas Negroponte will be giving us $100 to take one of his laptops. 1 billion PCs running Linux requires 20 years of 22% annual growth starting from ~20 million today. It needn’t happen that steadily and slowly as FireFox made it to 100 million in 1 year.

Linux, and the millions of lines of OSS on top and underneath it, is at an interesting inflection point: without fanfare, steadily making progress in the embedded, server, supercomputer, space, etc. space, but with little desktop marketshare and almost even smaller mindshare in the outside world. The lack of mindshare on the desktop is headwind for the server and embedded market and still allows for doubt of the legitimacy of the OSS development process.

OSS Picking up Steam
My initial experience with Debian was inauspicious. I was running Fedora Core 4, happily at the time, when I randomly bought a Linux magazine bundled with a Sarge (Debian 3.1) DVD, read a nice review of it, saw that it shipped with the 2.4 kernel, and promptly tossed it into the trash. As it turns out, 2.6 is supported in that release so my facts were wrong, but I stand by my sentiment. I have since discovered that 2.4 was only the default for Sarge and that now Debian is even transitioning away from the 2.4 kernel, and that is good. A Debian based on 2.4 might be a good reason for a derivative distribution, assuming there is still enough interest.

For comparison, FireFox is shipping once per year, Gnome is shipping twice per year and the Linux kernel is shipping 4 times per year. The Open Source movement is not only moving quickly but actually gaining momentum. The only constant is not simply change, but an increasing rate of change. This is in contrast to Debian, in which the last release took 3 years and the one before that took 2 years. I believe there is no technical reason why you cannot ship whenever you’d like. The Linux kernel merges in changes whenever they are ready without anything holding up the train. Your codebase is much bigger and your changes are more frequently more intrusive, but I believe there has to be a way to work things out even if it simply requires more resources.

Ubuntu & Debian
My second experience with Debian, running Ubuntu, has lasted much longer, and I am a happy customer and so is my father, who can’t really tell that he’s not running Windows anymore, though he does understand the geopolitical significance of it.

I have nothing against Ubuntu, and I love my operating system very much, but I believe that in principle, I should be running Debian. The Ubuntu guys took the efforts of your decade-old project, now containing 1,600 developers, added 10-30 of their own, threw in some new artwork, and have created a huge amount of excitement. Their success is a testament to their efforts, but to Debian’s even more. Of course, Debian’s success is a testament to many other people and so on, but Debian has built a large organization, focused itself on one big noble, common, goal and has built a superb system upon which there are 130 derivatives–one of which now threatens to dwarf Debian itself.

Everyone agrees that Ubuntu couldn’t exist without Debian, but I also believe that Debian is better setup to take Ubuntu where it needs to go. There are hints that the Ubuntu team feels like they brought a pork chop suit to a lion’s den. Ubuntu’s user base and development team is growing exponentially, but I believe they could get there much faster with more of Debian’s help. Its a simple fact of math that their many new bugs can be more easily divided between 1600 people than 10-30.

I am not a zero-sum person, and while I do believe that every time someone installs Ubuntu, it is good for Debian, I also see a downside. When he files bugs, the knowledge gained by the dev fixing the bug will not necessarily flow upstream to you guys. When someone writes software for Ubuntu, it may not run on a Debian system. I think benefits flow downstream better than upstream and I’m not sure if Canonical’s new tools will fix that.

Ubuntu is a relative and therefore a friend, but they want both desktop and server customers and therefore threaten the center of gravity of Debian and its 130 derivatives. And if Debian developers think the excitement is elsewhere, they might not necessarily move to Ubuntu. I wonder whether it would be better for Ubuntu to send most interested developers upstream to you guys, or whether you could share work on the desktop kernel which still needs a lot of work for laptops.

What can Debian learn from Ubuntu?
I believe an important lesson to learn from Ubuntu is that excitement is generated when you ship fast. Also, it is important to focus on the desktop and ease of use. Developers are users of your software, and users become future developers. We all start out as noobs. Eric S. Raymond wrote a great memo about the difficulty of getting printing working in Linux and if software is too hard for him to use, then it is just not completely ready for the outside world yet. All the code is there: the drivers, the network protocols and the management UI; they just require spit and polish. You can build a powerful system which is also easy to use and apt and Debian-installer are superb examples of that.

Is Debian Stagnant?
While Ubuntu doesn’t have the large, experienced team that you have, they do have excitement and momentum. When I look through the Debian website, I see some worrying metrics:

  • It is taking hundreds of days for new packages to be added, or for requests for help to be fulfilled.
  • There are 189 bugs assigned to the x86 kernel, and the average age of those bugs is 1 year
  • The Debian-apache e-mail alias is deathly quiet, other than spam (some cleverly containing bug numbers in the subject!), and mails wondering about Apache 2.2 go unanswered.
  • You’ve got churn on your bug database at the rate of 12,000 entries per month or 600 per workday. If half of those are by the dev and a full-time dev can work on 5 bugs per day, then you’ve got the equivalent of 60 full-time devs working on bug fixing, or that your 1,600 devs are doing 1.5 hours of bugfixing per week.
    I’m not going to judge anyone else’s volunteer contributions, but it is interesting to have some high-level measurements of the excitement of the team. I’d be curious to know what others think of these swags and how they compare to the time people spend hacking in other projects, maintaining their free blogs, myspace pages or other hobbies.
  • As Debian matures, your current set of volunteers might graduate from college, become satisfied or bored or otherwise move on.

A lot of what your devs do on a day to day base is hacking–integrating new versions of code, debugging it and keeping the system running. It is important stuff, but it is different from writing code and I believe it needs a constant stream of fresh blood to keep up with this somewhat tedious, labor-intensive task, and to allow your experienced people to solve the harder problems. You should be able to grow the team of contributors proportionally with the userbase if you want that. A lot of people would be excited to work on Debian and you should harness that.

What are the Big Issues?
In addition to a focus on desktop and polish, I think it is important to think big, long-term, and strategically. Things like apt and debian-installer play a big part in the definition of Debian. What future investments should your team be making? You could create a process whereby the community identifies the best big issues and then the DPL would provide the leadership to see that these efforts get off the ground. This can all happen in a meritocratic, volunteer process if there is buy-in. Here are a couple to think about:

  • Allow users to run new versions of code without upgrading the OS. It is an unsupported scenario on both Ubuntu and Debian to run FireFox 1.5 on your released OSes. Ironically, my mom would be okay with FireFox 1.0 for years on end, but your most passionate customers are not. There are workarounds, but they are time-consuming. The current policy is to only support fixes for security bugs, but FireFox 1.5 contains many security enhancements, some of which must not been backported, and I’ll bet the FireFox team considers FireFox 1.5 to be better in every way, including security. There is a backports mechanism, but it is not a first-class mechanism.
    It should even be possible to test pre-releases of FireFox and so be ready to sim-ship with the Mozilla folks–that would be exciting! Likewise, OpenOffice.org is going to start shipping every 3 months. Upgrading apps after install is not only a problem for the big apps, but also the small ones: GtkPod is my latest discovery of an app which requires many steps to install if not using the ‘official bits’. [Update: Backports being ‘unsupported’ may be a bigger problem in Ubuntu than Debian.]
  • Are there challenges which would limit your growth? If you had 10x the users, could you scale out your download mirrors and other infrastructure to handle it? I personally think a peer-to-peer APT would be cool and I know such a tool will get written if it makes sense, but developers don’t pay the bandwidth bills.
  • We can hate closed source, but we cannot wish it away. Closed source video, network, modem and biometric drivers, apps like Flash, Real, Skype and Adobe Reader, codecs, and Sun’s Java 5 VM are examples of closed code that are an important part of a modern computer and need to be supported, grudgingly.

I don’t even know if those are good problems, I’m just proposing that you guys find some and start chipping away at them. There are many challenges Linus will never get to, unless he gets pissed off and creates a ‘Linus Gnu/Linux’ distro.

Debian as Man of The Year
Many people in the outside world still do not get even the basics of why Open Source is viable and actually superior to closed source and your organization needs to continue to make explaining this part of the public message.

I also find it interesting that Debian has very little name recognition compared to MySQL, Redhat and IBM, yet the fact that Debian is such a large effort is newsworthy and should get you at the seat at the table with anyone you’d like. Google made it on the cover of Time, and Debian could win ‘Man of The Year.’

Developer Productivity
By far, the biggest thing which is slowing down both open and closed source software development is the use of old tools. C and C++ are fundamentally 30 year-old programming languages and they don’t have garbage collection, metadata, language support for multi-threading, modern class libraries, do not agree on what a string is or how to allocate memory, are not portable, and the compilers are huge, ugly and slow. Line for line, a Java app might be 20% slower than a C app, but algorithms define the speed of code and as Anders Hejlsberg said of .Net, a VM is the best use of Moore’s to come around in a long time. There are many huge benefits to modern languages but the biggest is that a developer is up to 10x more productive. I won’t argue that kernel mode code should be written in Java–yet–but the kernel is only a small portion of a distro and much of the other code could easily be.

I think Scott McNealy should be fired by Sun’s Board of Directors for his poor stewardship of Java (I run more C# apps than Java apps on Ubuntu!) but in the meanwhile, you should encourage code to be written in any managed language: Perl, Python, PHP, Java, C#, etc. Less VMs is definitely better, but the toothpaste is already out of the tube and multiple VMs aren’t a serious cost.

Fini
Some of this may be silly, obvious or other, but I offer it humbly as food for thought to an organization who has built something big and important, all of which prepares it well for the many exciting developments ahead.

Update: I also wrote an Open Letter to Ubuntu’s leader Mark Shuttleworth, talking to him about Ubuntu’s challenges.

12 comments to Open Letter to the new Debian Project Leader

  • Adam

    I can’t imagine anyone from Debian taking the time to seriously consider the content in this letter considering how you have chosen to phrase your issues from an Open Source perspective.

    Debian could care less about Open Source. Please take the time to study the difference between it and the Free Software Movement.

  • Boy oh boy, have you ever said a mouthful. Yes, I too have noticed much of what you mention when I tried Sarge last year. Talk about wearing a pork-chop suit to a lion’s den – try going in the Debian board after a new release and expressing that you are less than thrilled. Some of that must have been those 189 bugs I was experiencing when they were brand new.

    Debian fans, please try to understand, as you’re being defensive about your distro, we are expressing concern for it’s well-being. I have used and loved grml, Knoppix, and Damn Small; all of them based on Debian. Half of Linux depends on Debian, come to that. So we have dozens of distros getting to be the bride, and Debian brings up the rear as the bridesmaid. There’s something wrong with this picture, and pretending there isn’t doesn’t help.

  • Rick Handrich

    Not really a Debian specific rant … more of complaint about how Linux and Open Source software work generally … with the supposition that Debian, as the biggest and best, is responsible to change things … Not worth the effort to read the entire thing, although I did.

  • Quoted:”The Ubuntu guys took the efforts of your decade-old project, now containing 1,600 developers, added 10-30 of their own, threw in some new artwork, and have created a huge amount of excitement. Their success is a testament to their efforts, but to Debian’s even more.”

    I think it is a crime that all that Debian needed was a little love and GUI continuity. Your right, Debian is great, but it could be the greatest.

  • So here’s the debian community. Not enough people are available and/or willing to test packages, or managers to review these tests, and get everything approved and shipped in a new release. They have, perhaps, no incentive. It’s a community thing, not for cash. However, where are these communities? Are they over at Fedora? Or perhaps these communities have left for Novell, Slackware, Ubuntu, Mepis, Mandriva? The fragmentation is causing the freebies (free help) to dry up, perhaps? Debian is one of these communities where there are no bounties, no foundation paying huge sums of money (that I know of), and no corporate sponsorship. That’s a shame if I’m correct in my assumptions there.

    Many people are genning their distro off Debian, but not returning the benefit back. They may think, so what if it’s an older stable release? I can take the newer alpha release, approve a bunch of junk my way, swap packages and the like, and re-release it as my own distro. That’s what Ubuntu has done.

    To those who continue to do this to Debian, without returning enough back, I think it will backfire some day. The makers of Ubuntu will go back to the Debian site and it will say, “This site is down until further notice.” At that point, a lot of people will have to take a long, hard look at what has happened here.

    There needs to be an incentive at the Debian commune. The builders need payments to excite them enough to quit their day jobs and focus on this full-time. Since selling Linux software is bad form, they’ll need to take the Mozilla model. That means a foundation, a tech support center, great marketing, a fantastic website, partnerships with other communities and companies, and bounties. At that point, they would be looking a lot like Ubuntu and Canonical.

    Ubuntu and Canonical might as well just take over the Debian project, then.

  • Shuttleworth tried to take over Debian, but he couldn’t. Debian is rather chaotic, as an open project, and the Project Leader doesn’t have that much control after all…

  • Michael

    The Ubuntu project has different goals to Debian: Debian is about Free software, while Ubuntu is about being a useable system out-of-the-box for normal human beings.

    Lots of manufacturers will only have binary drivers for their hardware (at best), and that sort of thing goes against the very philosophy of Debian. That’s where other distros with a more “pragmatic” outlook come in to play. Also, the UI polishing and bugfixing that happens in distros like Ubuntu do return to Debian, because the patching occurs in the upstream application source.

    Also, if you want to play with the newest software on Debian, try using testing instead of stable. A constant stream of updates, and they don’t usually break your computer (that’s what unstable is for).

    Debian stable appeals to folks like me who use it to run servers, precisely /because/ the software isn’t constantly being updated. More frequent releases means more work for the security team; or, you force people to upgrade more often, which means more work for me.

  • zach

    I am a little unsure of the links between debian and the Free Software Foundation, but the FSF certainly has problems with Java and many other technologies because of the licencing. Their agenda is of a more “ethical” nature. Java for example will probably only get serious attention after the gnu version is complete. You are not going to get them to change their fundamental philosophies and many of the complaints in this letter relate directly to those.

    The fortunate thing for Microsoft is that the buck stops somewhere. Imagine if all the distributions’ resources were concentrated into one distribution, it would be amazing, a single force which could make apps and package managment systems all work so much easier. this seems to be what you’re asking from debian. But alas the linux community is too diversified in terms of philosphies.

    So if all religions were able to be integrated, which they are not on the basis of basic discrepancies, then how will it be done for the differing linux philosophies. One distro cannot be all to all, unfortunately. The upside is that if linux eventually conquers the desktop, it couldn’t be monopolized in the same manner as M$, because of this very thing and the gpl of course.

  • Generally i think you shouldn’t care about Debian, because you have a different mindset.

    Debian is about Free Software and will never ship non-free stuff by default.

    Debian is about doing the right thing. If you want the latest and greatest Firefox use klik or backport it. Mozilla does not play that well with Debian Security.

    Debian is not about a slick Desktop. Use Ubuntu!

  • psilo

    I shared your views a few years ago, but now I realise conquering the world simply isn’t Debian’s goal. There are plenty of other distro’s releasing cutting edge stuff, including non-free software and applying spit and polish.

    Debian is one of the few perhaps only distro putting the free software ideology, stability and portability above marketing, ease-of-use and ‘innovation’. Accept this and move on…

  • dregius

    I agree with the previous comment. I’ve see this issue so many times: swithing something new, changing to a more recent version, geekism, etc.
    Currently Debian is the only conservative distro, and that mentality is really rare today in every parts of the IT world. (And conservative just doesn’t mean refusing to change, but rather judiciousness)

    Look at the other areas, like software and hardware developing: bugs, incompatibility, integrity and consistency problems. Everything is developing with a so rapid pace, but are we ignorant enough to neglect our responsibility over it?

    Man, informatics is SCIENCE, not some casual stuff! Are you blind not to see how much depends on it? Such carelessness can result in a ten times greater catastrophy than Hiroshima. In 10-20 years almost all knowledge of humanity will be stored in digital format. We might risk losing the whole history because of fad or business interests.

    Yes. I’m talking about that you everyday people, home users, influence even the enterprise areas of IT by your computer habbits.

    “I want to use the newest Gnome destop” – someone might say, but he/she is not qualified to understand the deeper aspects what such mentality result in. I don’t say that everyday people shouldn’t use computers, I say that they shouldn’t be involved in, and shouldn’t affect the CORE of the IT development. But today that happens, when a stable/conservative project like Debian depends on the aknowledgement of an unqualified and fad community, who has the only expectation to use some new transparent taskbars.

    What if… I don’t know if there is any bank systems based on Debian, but let’s say there is. Now imagine that in the haste, project leaders yield to the pressure of the community and ship release that “too strict” beta testing. AND – thanks to a bug – YOUR family’s account vanishes.

    Now you might say that’s a very unlikely example, but I hope you get the picture. Debian was made to be a reliable freeware server OS in the first hand, and it may not fit the needs of a home user. You don’t need to use it for your dekstop.

    See, on the other hand, I wouldn’t be happy to see that my bank stores my account on a Windows based system.

  • […] Open letters seem to be all the rage these days.  How about we just go with "Craig, as benevolent dictator, says How It Is": […]