Open Source Solutions for Small Business Problems

This week I’m at the Open Source Convention in Portland, aka OSCON. First impression, before showing up: it seems all focused on big business. Big ticket price. Lots of enterprise-related topics, and sponsors. Not really the meeting of geniuses and thought leaders as years past–or so I thought.

Second impression: Tim O’Reilly asking Brian Aker and Monty Widenius about the importance of various proprietary companies: Sun, Adobe, Microsoft. Their answer to Microsoft? Irrelevant. And Tim came back apologizing to the Microsoft sponsors. This just after the presentation that talked about the open source “tribe”, and introducing Tim as the leader of the cult. Was feeling a bit like I may have made a mistake, plopping down cold hard cash to support the cult of O’Reilly.

Fortunately, that thought was momentary. The rest of the event has been extremely rewarding, and very worthwhile.

At home, in my usual networks, I’m the token Linux guy. Beyond our company and some family I’ve converted to Linux, almost none of my friends use open source, and in business circles, I’m the resource for not just Linux, but web technologies, programming, system administration, and most anything computer related. I come to OSCON and I’m a mere end user, still damp behind the ears. There is true genius wandering the convention center. And suddenly, I’m one of the least technical people in the room, though still listened to for my experience trying to bring these projects to the small business world, identifying pitfalls and areas for improvement.

A few years back, I came to OSCON and it seemed that the worse dressed a person was, the higher up in the ranks of alpha geek he was. This year people looked much more presentable. That might be as far as the influence of corporate culture went.

I met R0ml at that previous OSCON I attended, at a BOF session led by Doc Searls. A “BOF”, if you’re not familiar with software conferences, is a “Birds of a Feather”, a mini, informal discussion about a particular topic led by anybody who puts an idea for one up on a bulletin board. I doubt R0ml remembers me, but it was an interesting discussion we had that evening that paralleled his talk today about elevating open source development from the realm of Techne to Praxis–from mere “making” of stuff to “doing” something to influence and lead people. He took issue with the assertion in an earlier keynote about Franklin and Jefferson being our founding geeks–mainly because while everybody needs a geek these days to make computers do their bidding, that’s a useful, technical, thing that puts us squarely in the realm of Techne. What Jefferson and Franklin really were, according to R0ml, were Polymaths, and that is also a better description of open source practitioners. We’re Renaissance people.

More later…

The Linux Link Tech Show (TLLTS) has a great segment dissecting the criticisms/wild flames put forth on a series of shows on the TWIT network. Wanted to add a couple comments missing from their discussion.

First of all, the Mac Break Weekly show apparently spends some time bashing the open source community, calling out Drupal, and how difficult it is to solve “simple” problems like uploading images for blog posts. In practically the same breath, the hosts claim that the open source community never has any innovation behind it. Irony drips:

• In the world of content management systems (CMSs), most of the innovation starts in open source projects these days, and Drupal is at the cutting edge of this with its powerful system of taxonomies, and hundreds of add-ons freely available.
• I can think of a grand total of 2 proprietary CMSs that have anywhere near as widespread use as most of the open source CMS systems. One has turned open source itself: Movable Type. The other is Sharepoint, and it’s widespread because Microsoft has shipped it out on lots of its server products.
• Complain about usability all you want, but name a proprietary product as powerful as Drupal, that’s easier to install, administer, and configure.
• The TWIT.tv site itself is running on Drupal.

So let’s talk a little about innovation. While Photoshop may still dominate the world of graphic design, but the lines aren’t so clear when it comes to animation. The Blender project recently released its second short animated film, Big Buck Bunny. While you might argue about the strength of its story, you cannot deny that the technical effects are as stunning as any major animated film coming out from the big studios. And it was created by 7 people in 9 months, using open source software. Even the big studios like Pixar, Dreamworks, and Industrial Light and Magic rely on open source software to deliver their magic, such as CinePaint, POVray, and several others.

On the subject of innovation, KDE4 is breaking new ground and stirring up controversy, laying a bedrock that promises the ability to do things beyond the standard “Desktop” paradigm that was invented over 30 years ago and we’ve all used ever since. Meanwhile, the GNOME team is working on creative ways to embed web applications into your desktop.

But the real innovations of the open source community are all a few layers deeper in the application stack, all the plumbing that powers the Internet. Microsoft itself borrowed its early networking stack from BSD, one of the earliest open source operating systems. Domain Name Service (DNS) and email were first implemented on the Internet using open source software (BIND and Sendmail).

The open source community tends to snicker whenever Apple claims to be innovative. Its core “innovations” were all invented somewhere else:

• The Mac Desktop interface borrowed heavily from Xerox PARC labs
• OS X uses BSD under the hood
• “Spaces” were in use in Unix systems for a decade before they arrived on the Mac
• The “Time Machine” functionality in Leopard is standard in many source code management tools

To its credit, Apple polishes these features better than anybody else, making them easier to find and use by normal people. But many, if not most of its innovations come from somebody else.

Read the previous post for more discussion about Linux support.

I started writing a response to a discussion in the latest “Linux Link Tech Show” episode, but ended up with something far too long, so I’ve split it up into 4 posts. The next post is about the TLLTS vs TWIT debate, and introduces this set of post. The previous two are about open source support–a true story of a support incident I had, and the unwritten rules of open source support. In this post, I analyze the fundamental differences between Windows, Apple, and Linux when it comes to support.

Dann and Linc had a quite spirited debate about the merits of having a company hire low-end tech support with scripts (Dann) versus having an experienced, savvy, tech professional able to really solve the user’s problem (Linc). Dann’s point was that it can easily be more cost-effective for both the support company and the end user to go straight to reinstalling a system if rebooting doesn’t solve the user’s problem, while Linc seemed to think a savvy tech person could get to the root of the issue much quicker, and brought up the point that there’s a cost to the frustration of users being put through the whole front-line support nightmare.

I’d suggest the situation is even more complex than that, but it also differs greatly between the open source projects and proprietary operating systems. First, let me make a bold statement:

Linux support is far better than Windows or Mac.

But it also has a completely different set of rules. Learn those rules, and you’ll be able to solve your problems more satisfactorily than it’s possible to do in the proprietary world. Let’s talk about these differences, looking specifically at a who, where, what, and how long.

Who can help with your problem?

Let’s take a look at who can help with your problem:

The bottom line here is that while you probably know fewer people personally who can help you with Linux, there are more options for commercial support, and you can reach people who can do more to solve your problem for free, than you can with either Mac or Windows.

The fundamental reason for this is that anybody can read the raw source code of any open source product out there, and with enough skill and talent, can solve your problem without needing to pay anybody for the right to do so. In the proprietary world, only one company can help beyond a certain point: for Windows problems, that’s Microsoft. For problems in an application, it’s the application developer.

So the next question is:

Where can you get help?
Again, let’s compare the options:

The main point of the table above is that in the proprietary world, the harder your problem is to solve, the fewer people can help you solve it. You quickly get down to one place to go, and if it’s in the operating system, it might be expensive or not possible to fix. In the open source world, it’s nearly the opposite case–it can be harder to find the simple quick answer to your question, but the harder and deeper your problem, the more places you can go to get help.

Where do you look for help first? This is the single stumbling block for most otherwise tech-savvy users new to Linux. To learn Windows or Mac you can take a class, talk to neighbors and friends, and find lots of very low-end help that way. For Linux, unless you’re friends with some hard-core geeks, you need to go online to find help. Once you’re there, it really depends on where your problem lies.

For just figuring out how to use a system, go to the forums for your distribution. The Ubuntu forums are a great place to ask general questions. Be specific about what you’re trying to do and you’re more likely to get help. Remember that this level of support is free, and people helping you are volunteering their time and knowledge. While you’re there, see if you can answer somebody else’s question–the more help you give, the more you’ll get in return.

The other fantastic place to go for help, especially for quick questions, is IRC. IRC is a system that provides chat rooms and instant messaging. Most open source projects with any community behind them have a chat room on irc.freenode.net. Install an IRC program like Chatzilla, Konversation, Pidgin, or any number of others, connect to irc.freenode.net, pick a nickname to use, and join the channel for the program or distribution you’re having trouble with. Ask your question nicely, and if you don’t get a response immediately, keep your chat program open for a few hours–not everybody is watching the channel every minute.

What kinds of problems can you solve?
Let’s look at the same types of problems as before, and look at the resolutions:

What’s the key thing in this section? Addressing actual problems is within your control, when you’re working with open source. In the proprietary world, you’re entirely at the mercy of a single software vendor. If your problem is in the interaction between two different applications, you’re really stuck — there’s nothing you can do.

But in the open source world, there’s always something you can do. You can hire anybody with the skills to solve your problem, and fix it in the software itself. You don’t need any blessing from any single software vendor.

Open formats are perhaps more valuable than open source software, for most businesses. Because this is such a compelling advantage of open source software, many proprietary programs are beginning to open up their formats to allow other software to read them–their customers are demanding it.

How long will it take to solve your problem?
I’m not going to bother with a table for this one. The answer is nearly always “too long,” regardless of the operating system.

Actually, that’s not quite always the case–it depends on whether somebody has already solved the problem or not, as well as whether the solution is a fix or a workaround.

Many, many problems in Windows are not really fixes, they’re just workarounds. The only real “fix” for a virus-infected machine is the workaround of reinstalling your operating system. The only fix for lots of other minor issues that cause your system to slow down over time is to reboot. These are not fixes, they’re workarounds.

Real fixes take a lot longer, and need acknowledgment that the problem is real. Workarounds are band aids to get you through until there’s a real fix. And there are some real differences between the entire approaches of each operating system around these fixes.

Windows is chock-full of workarounds. Because Microsoft has gone to great lengths to maintain backwards compatibility of just about everything that’s been released for Windows since Windows 95, it’s full of workarounds to keep the old behavior. Rather than fixing behavior that might really be undesirable, they’ve had to patch it with workarounds because too many existing applications turned out to depend on that bad behavior. That, fundamentally, is why Windows is so big, bloated, slow, and painful to work with.

Apple suffers a different problem: changing their closed libraries too quickly for external developers to keep up. Each release, they break lots of things, and don’t always tell 3rd party developers ahead of time. This means the number of third party developers of Apple software is shrinking–they’ve managed to alienate quite a few. So their polish and high quality comes at the price of having a healthy thriving developer community outside the walls of Apple. There’s little transparency in this process, so developers outside Apple are always playing catch-up, and having to work around these changes in behavior.

Linux has its share of problems, too. Linux does not attempt to maintain “binary” compatibility between versions, though it does its best to maintain compatibility in source code. There is endless debate about the “correct” way to fix a problem, and competition between fixes. The challenge of this is that it can be hard for application developers to keep up, especially if they want to keep their source code closed and only ship software in binary form.

But the process is completely, utterly transparent. Anybody can see the progress on any fixes to any part of the system, and can jump in with their own solution at any time. It’s a true meritocracy, with those doing the actual work and providing the best solutions winning out over time.

The whole open source ecosystem is nimble enough to provide real fixes to technical problems, rather than just simple workarounds. If the source of the problem is design, it can take a long time to get the right design in place and resolve all the issues that changing the design causes in other applications. But when a bug gets fixed, it’s really fixed and usually doesn’t appear again.

In my early Linux system administration days, when I was first trying to set up a mail server with spam filtering, I ran across a really puzzling bug in Dspam, the software I was trying to get working. While all the other users of the software were getting great results, with Dspam catching 99%+ of all their spam, it was only catching about 70% of my spam after quite a bit of training.

I posted my results, and confusion, to the Dspam mailing list. The original developer of this software (which has thousands of users), Jonathan Zdziarski, responded that that did not sound right. He asked if he could log into my server and see what was wrong.

I created a test account for him, and logged in to the same screen so I could watch what he was doing. As I watched, he put debugging messages into his code, ran several tests, and within 10 minutes had identified the problem: the 19-digit number he put into the database (MySQL) was different than the 19-digit number he got out. It was a storage error in somebody else’s software causing the problem, a rounding error. He filed a bug in MySQL, and it was fixed within a month or two. But he also had a workaround for me: change the data type to a set of characters instead of an integer. Problem solved.

What’s extraordinary about the open source community is that this level of support happens all the time, every day, without charge, in hundreds, thousands of projects out there. People that can get to the bottom of a problem and fix it at the source, not just provide a workaround, are directly reachable and motivated to see their software work as well as possible. They’re not hidden away from the public behind a large corporation, unreachable with layers of clueless support script readers stuffed between you and them. Here are some rules for getting open source support directly from the projects:

• Before asking anybody, do your homework. Use Google, read the project FAQ, make some attempt to learn the basics without pestering people with questions they’ve already answered hundreds of times. Nine times out of ten, your problem has already been encountered and somebody has figured out a workaround.
• Limit the scope of your question to the fundamental problem. Get to the point. I’m obviously guilty of being tremendously long-winded at times, but unless your question is right at the top and asked directly, you’ll probably get ignored. Developers are busy, they don’t want to read a novel, but they’re usually happy to answer a question.
• Provide supporting details after asking your question. Many programs will create a log with lots of information that can help somebody diagnose your problem. Find what looks relevant in the logs. Specify what version of operating system, distribution, application, etc. Specify what you were trying to do, what you expected, and what really happened. But provide this stuff after asking your initial question–people aren’t going to wade through a long email to find your question.
• Contribute something. The easiest way you can contribute is by answering other people’s questions. The whole thing works because people help each other, and this help goes both ways. If you always ask questions and never answer anybody else’s, or provide any other sort of contribution, you’ll eventually start getting ignored.
• Always, always, be positive, respectful, and polite when asking your questions. Developers have a lot invested in their project, and insulting it won’t gain you any favors. Developers are under no obligation to help you either–you haven’t paid for it. Common courtesy is valuable. Complements are welcome.
• Be patient. Sometimes the person who has the answer to your question is away from the computer. Usually you’ll get your problem solved quicker than you would calling some tech support line, but there are times it’s going to take a while. If you don’t get any response in a reasonable period of time (judged by how active the list or forum is, reasonable could be a couple hours or a couple days), there are several likely reasons: You haven’t been specific enough in your question; you’re in the wrong forum (e.g. users when it’s a developer question); nobody else is trying to do what you’re doing (in which case you may need to hire someone with the right skills); the project is dead (it happens sometimes–find another one); or the developers are swamped (give them more time, or come up with a new scenario that sheds light on your problem in a different way).

That list describes how we get open source support at Freelock. Aside from a couple unsupported hardware devices, or issues with proprietary programs, we have yet to get stumped, in over 6 years of extremely heavy Linux and open source use. We’ve never paid a dime for this support, though we have provided help to many others in return.