Dear DPL Anthony Towns,

I read your post-election comments about your mission as DPL to be increasing the tempo of Debian, and I think that is an excellent theme. This is the 21st century: Google existed only on paper 8 years ago, China and India are just coming online and will eventually contribute to OSS in a manner commensurate with their size, and maybe even in the Middle East and Africa they’ll quit killing each other or dying of pandemics and become productive members of the modern world. In 20 years, Linux will be running on 1 billion devices smaller than a PC and 1 billion PCs and Nicholas Negroponte will be giving us $100 to take one of his laptops. 1 billion PCs running Linux requires 20 years of 22% annual growth starting from ~20 million today. It needn’t happen that steadily and slowly as FireFox made it to 100 million in 1 year.

Linux, and the millions of lines of OSS on top and underneath it, is at an interesting inflection point: without fanfare, steadily making progress in the embedded, server, supercomputer, space, etc. space, but with little desktop marketshare and almost even smaller mindshare in the outside world. The lack of mindshare on the desktop is headwind for the server and embedded market and still allows for doubt of the legitimacy of the OSS development process.

OSS Picking up Steam
My initial experience with Debian was inauspicious. I was running Fedora Core 4, happily at the time, when I randomly bought a Linux magazine bundled with a Sarge (Debian 3.1) DVD, read a nice review of it, saw that it shipped with the 2.4 kernel, and promptly tossed it into the trash. As it turns out, 2.6 is supported in that release so my facts were wrong, but I stand by my sentiment. I have since discovered that 2.4 was only the default for Sarge and that now Debian is even transitioning away from the 2.4 kernel, and that is good. A Debian based on 2.4 might be a good reason for a derivative distribution, assuming there is still enough interest.

For comparison, FireFox is shipping once per year, Gnome is shipping twice per year and the Linux kernel is shipping 4 times per year. The Open Source movement is not only moving quickly but actually gaining momentum. The only constant is not simply change, but an increasing rate of change. This is in contrast to Debian, in which the last release took 3 years and the one before that took 2 years. I believe there is no technical reason why you cannot ship whenever you’d like. The Linux kernel merges in changes whenever they are ready without anything holding up the train. Your codebase is much bigger and your changes are more frequently more intrusive, but I believe there has to be a way to work things out even if it simply requires more resources.

Ubuntu & Debian
My second experience with Debian, running Ubuntu, has lasted much longer, and I am a happy customer and so is my father, who can’t really tell that he’s not running Windows anymore, though he does understand the geopolitical significance of it.

I have nothing against Ubuntu, and I love my operating system very much, but I believe that in principle, I should be running Debian. The Ubuntu guys took the efforts of your decade-old project, now containing 1,600 developers, added 10-30 of their own, threw in some new artwork, and have created a huge amount of excitement. Their success is a testament to their efforts, but to Debian’s even more. Of course, Debian’s success is a testament to many other people and so on, but Debian has built a large organization, focused itself on one big noble, common, goal and has built a superb system upon which there are 130 derivatives–one of which now threatens to dwarf Debian itself.

Everyone agrees that Ubuntu couldn’t exist without Debian, but I also believe that Debian is better setup to take Ubuntu where it needs to go. There are hints that the Ubuntu team feels like they brought a pork chop suit to a lion’s den. Ubuntu’s user base and development team is growing exponentially, but I believe they could get there much faster with more of Debian’s help. Its a simple fact of math that their many new bugs can be more easily divided between 1600 people than 10-30.

I am not a zero-sum person, and while I do believe that every time someone installs Ubuntu, it is good for Debian, I also see a downside. When he files bugs, the knowledge gained by the dev fixing the bug will not necessarily flow upstream to you guys. When someone writes software for Ubuntu, it may not run on a Debian system. I think benefits flow downstream better than upstream and I’m not sure if Canonical’s new tools will fix that.

Ubuntu is a relative and therefore a friend, but they want both desktop and server customers and therefore threaten the center of gravity of Debian and its 130 derivatives. And if Debian developers think the excitement is elsewhere, they might not necessarily move to Ubuntu. I wonder whether it would be better for Ubuntu to send most interested developers upstream to you guys, or whether you could share work on the desktop kernel which still needs a lot of work for laptops.

What can Debian learn from Ubuntu?
I believe an important lesson to learn from Ubuntu is that excitement is generated when you ship fast. Also, it is important to focus on the desktop and ease of use. Developers are users of your software, and users become future developers. We all start out as noobs. Eric S. Raymond wrote a great memo about the difficulty of getting printing working in Linux and if software is too hard for him to use, then it is just not completely ready for the outside world yet. All the code is there: the drivers, the network protocols and the management UI; they just require spit and polish. You can build a powerful system which is also easy to use and apt and Debian-installer are superb examples of that.

Is Debian Stagnant?
While Ubuntu doesn’t have the large, experienced team that you have, they do have excitement and momentum. When I look through the Debian website, I see some worrying metrics:

• It is taking hundreds of days for new packages to be added, or for requests for help to be fulfilled.
• There are 189 bugs assigned to the x86 kernel, and the average age of those bugs is 1 year
• The Debian-apache e-mail alias is deathly quiet, other than spam (some cleverly containing bug numbers in the subject!), and mails wondering about Apache 2.2 go unanswered.
• You’ve got churn on your bug database at the rate of 12,000 entries per month or 600 per workday. If half of those are by the dev and a full-time dev can work on 5 bugs per day, then you’ve got the equivalent of 60 full-time devs working on bug fixing, or that your 1,600 devs are doing 1.5 hours of bugfixing per week.
  I’m not going to judge anyone else’s volunteer contributions, but it is interesting to have some high-level measurements of the excitement of the team. I’d be curious to know what others think of these swags and how they compare to the time people spend hacking in other projects, maintaining their free blogs, myspace pages or other hobbies.
• As Debian matures, your current set of volunteers might graduate from college, become satisfied or bored or otherwise move on.

A lot of what your devs do on a day to day base is hacking–integrating new versions of code, debugging it and keeping the system running. It is important stuff, but it is different from writing code and I believe it needs a constant stream of fresh blood to keep up with this somewhat tedious, labor-intensive task, and to allow your experienced people to solve the harder problems. You should be able to grow the team of contributors proportionally with the userbase if you want that. A lot of people would be excited to work on Debian and you should harness that.

What are the Big Issues?
In addition to a focus on desktop and polish, I think it is important to think big, long-term, and strategically. Things like apt and debian-installer play a big part in the definition of Debian. What future investments should your team be making? You could create a process whereby the community identifies the best big issues and then the DPL would provide the leadership to see that these efforts get off the ground. This can all happen in a meritocratic, volunteer process if there is buy-in. Here are a couple to think about:

• Allow users to run new versions of code without upgrading the OS. It is an unsupported scenario on both Ubuntu and Debian to run FireFox 1.5 on your released OSes. Ironically, my mom would be okay with FireFox 1.0 for years on end, but your most passionate customers are not. There are workarounds, but they are time-consuming. The current policy is to only support fixes for security bugs, but FireFox 1.5 contains many security enhancements, some of which must not been backported, and I’ll bet the FireFox team considers FireFox 1.5 to be better in every way, including security. There is a backports mechanism, but it is not a first-class mechanism.
  It should even be possible to test pre-releases of FireFox and so be ready to sim-ship with the Mozilla folks–that would be exciting! Likewise, OpenOffice.org is going to start shipping every 3 months. Upgrading apps after install is not only a problem for the big apps, but also the small ones: GtkPod is my latest discovery of an app which requires many steps to install if not using the ‘official bits’. [Update: Backports being ‘unsupported’ may be a bigger problem in Ubuntu than Debian.]
• Are there challenges which would limit your growth? If you had 10x the users, could you scale out your download mirrors and other infrastructure to handle it? I personally think a peer-to-peer APT would be cool and I know such a tool will get written if it makes sense, but developers don’t pay the bandwidth bills.
• We can hate closed source, but we cannot wish it away. Closed source video, network, modem and biometric drivers, apps like Flash, Real, Skype and Adobe Reader, codecs, and Sun’s Java 5 VM are examples of closed code that are an important part of a modern computer and need to be supported, grudgingly.

I don’t even know if those are good problems, I’m just proposing that you guys find some and start chipping away at them. There are many challenges Linus will never get to, unless he gets pissed off and creates a ‘Linus Gnu/Linux’ distro.

Debian as Man of The Year
Many people in the outside world still do not get even the basics of why Open Source is viable and actually superior to closed source and your organization needs to continue to make explaining this part of the public message.

I also find it interesting that Debian has very little name recognition compared to MySQL, Redhat and IBM, yet the fact that Debian is such a large effort is newsworthy and should get you at the seat at the table with anyone you’d like. Google made it on the cover of Time, and Debian could win ‘Man of The Year.’

Developer Productivity
By far, the biggest thing which is slowing down both open and closed source software development is the use of old tools. C and C++ are fundamentally 30 year-old programming languages and they don’t have garbage collection, metadata, language support for multi-threading, modern class libraries, do not agree on what a string is or how to allocate memory, are not portable, and the compilers are huge, ugly and slow. Line for line, a Java app might be 20% slower than a C app, but algorithms define the speed of code and as Anders Hejlsberg said of .Net, a VM is the best use of Moore’s to come around in a long time. There are many huge benefits to modern languages but the biggest is that a developer is up to 10x more productive. I won’t argue that kernel mode code should be written in Java–yet–but the kernel is only a small portion of a distro and much of the other code could easily be.

I think Scott McNealy should be fired by Sun’s Board of Directors for his poor stewardship of Java (I run more C# apps than Java apps on Ubuntu!) but in the meanwhile, you should encourage code to be written in any managed language: Perl, Python, PHP, Java, C#, etc. Less VMs is definitely better, but the toothpaste is already out of the tube and multiple VMs aren’t a serious cost.

Fini
Some of this may be silly, obvious or other, but I offer it humbly as food for thought to an organization who has built something big and important, all of which prepares it well for the many exciting developments ahead.

Update: I also wrote an Open Letter to Ubuntu’s leader Mark Shuttleworth, talking to him about Ubuntu’s challenges.