Handhelds.org Experiences and Lessons Learned
Open source is about leveraging other software developers. By sharing the effort to produce an application, framework, or library, one can reduce the cost of that software component. Open source software flourishes when it satisfies a broad horizontal need and also when it fills niches too small for commercial developers.
Guiding Principals for Handhelds.org
We started handhelds.org in order to produce and share a solid, vendor-neutral, open source operating environment for handheld computers for use by research groups.
Despite being hosted by Compaq and HP, I think the community did view handhelds.org as vendor neutral. Handhelds.org included ports to several Toshiba handhelds, Dell handhelds, and other devices. Few of these were stable enough to run the whole distribution due to lack of corporate support from those vendors, but the Sharp Zaurus was well supported.
As the project progressed, we changed the charter to producing a user-friendly, high-choice, vendor-neutral, open source Linux distribution for handheld computers. While pursuing this new charter, we spent more time on QA of the software components and distribution releases. This new charter also influenced the design of the software.
Lessons Learned
One common open source hazard is picking up an open source component, bringing it in house, and modifying it without sharing the modifications. Once you do that, you take on the task of maintaining the modified component and the cost of updating it as new versions of the original component are released to incorporate new features, bug fixes and security updates. Therefore, it is very important to publish the modifications unless there is a business reason for taking on the cost of maintaining the modified software. In order to get the changes merged back into the original source, one must work according to the design, style, and conventions of the original source code. In addition, one must have good communication with the developers and maintainers of the original software. It is best to start the development of the new code while in consultation with the original project members. In the best case, other people use and help maintain your contribution to the original software.
For developers, open source is a social activity. If you're just using some open source software, there may be no reason to talk to the developers. But if you want to affect the bug fixes, features, or roadmap of an open source component, you'll have to interact with the developers of the software. The bigger the effect, the more the communication required. IRC and email are probably the most prevalent modes of communication, followed by teleconferences and face to face meetings.
There is a huge benefit to having company employees participating directly in the open source community using their corporate identities and email addresses. Stealth participation means that the company does not get the good will benefits. In the case of handhelds.org, if Linux on the iPAQ was not supported by anyone from Compaq/HP, it would not have taken off among the commercial users. In order for the software ecosystem to fully take off, there must be commercial users.
Things that went wrong
1. Hosting unconnected projects
There is really not much point hosting projects that are not coordinated with each other. Sourceforge and other hosting sites exist for this purpose and there is no reason to try to supplant them. What is useful is hosting related projects in order to help guide them towards achieving a common goal.
2. Not having enough time to work on it
Familiar was most stable when there were full time people working on it. A stable Linux distribution is not a part time activity.
3. Not having access to the design of the devices we were supporting
Even the product team did not know how they worked, so we had to reverse engineer our own products.
4. Not working closely enough with kernel.org
The biggest problem with handhelds.org is that little of the device support has made it into the kernel. After working on the kernel for so many years, I see how one would go about more successful kernel development in terms of uptake by kernel.org. And after working on it for so many years, I see how important this is.
When we started, Linux kernel development was making the transition from 2.3.x to 2.4.0. Unfortunately, the 2.4 kernel series lacked the device model to support suspend, resume and power management of a device lacking a BIOS. With 2.6, the kernel has the appropriate features, but none of us had enough time to work on updating iPAQ device drivers to fit the model and to be acceptable into the kernel tree. It is not enough just to create a working solution -- it must also fit design criteria of the chief kernel maintainers: Linux Torvalds, Greg Kroah-Hartman, etc. This is achievable, but it requires an evolutionary approach.
Things that went right
- IRC
Open source development is a social activity. It requires communication among the participants. Email serves its purpose, but real time communication is a big help. Real time communication via IRC, jabber, etc, is the equivalent of hallway and coffee break conversations. Mailing lists do not serve this purpose -- they require too much thought.
- ipkg
One of the best things that happened to handhelds.org early on was that Carl Worth developed ipkg, which is essentially a compact implementation of dpkg and apt-get. It started as a 9KB shell script but grew to 100KB C in order to handle all the packaging features required for robust distributions, especially regarding upgrades.
ipkg and ipkgfind enable Familiar users to install just the applications they want, pulling in the dependences as needed. Given a device with limited storage, you cannot install everything. Since not everyone wants the same applications, a distribution for handhelds needs to have a good way of enabling users to easily install the apps they are interested in.
The package upload system on handhelds.org made it easy for a registered developer to upload packages which could be downloaded and installed minutes later by other users all over the world. Early adopters and developers would hang out on the #handhelds.org IRC channel, and new packages would be announced there and then tested by everyone interested in it. This enables very quick iteration of bug fixes, feature development, and usability enhancements.
3. conference calls
We held weekly conference calls, with IRC channel, for the informal steering group that set the direction and managed the releases for handhelds.org. This was really important. The IRC channel we used was a way to keep notes of the meeting as well as to interject side comments into the conversation without disrupting the speaker.
3. developer conferences
Even though most of the work happened without face to face conversations, it was highly popular to have face to face developer conferences. We held three, hosting them at Cambridge Research Lab. Members of handhelds.org are still asking when we will hold the next one. We had a combination of talks, given by various members of the community, and intense development sessions. Sprints is a common term for that kind of development.
4. Working outside the corporate firewall
IRC, jabber don't work across Nokia's firewall and didn't work across Compaq or HP's firewall. We had to move our development outside the firewall to become a first class member of the open source community.
Similarly, the source code we worked on all resided outside the corporate firewall. If source repositories are split or mirrored across corporate boundaries, someone ends up being a second class citizen -- either the corporate developers or the rest of the community. Class boundaries only add friction and overhead.