Open Source Solutions for Small Business Problems

At Freelock, we’re always trying to figure out ways to do things better. Recently I started digging into a developer tool that’s making, as Bryan over at the Linux Action Show would say, my head explode.

For a long time, we’ve managed our custom code projects and business documents in a central repository, called Subversion (also known as svn). Subversion is relatively easy to understand–it’s like having a library of files you can check a copy out of, do some work on it, and then check it back in. Subversion is the librarian that tracks who has copies of what, and when they checked it out. So if Erik checks in changes to a brochure, and then Jill goes to submit changes to the same document, Subversion will say “hey wait a minute, that document has already been changed–you need to make sure you put Erik’s changes in your document before I’ll let you put in your document.”

This is great for managing conflicts between people working on a single team, or for code that is being developed in relative isolation from the rest of the world.

The problem is, we’re doing more than that–we’re taking code from various open source projects and either customizing it or building new applications on top of it. And so when the outside projects get updated, we need to manually update anything we’ve written that depends on that code. There is no longer a single repository where we control our code–there is our code library, plus another one for every project we use.

This makes managing add-ons for projects like Joomla or ZenCart quite challenging, because our add-ons get scattered throughout the filesystem to be able to hook into the right place. And if we have to touch a core file, we’re going to end up needing to re-implement our change with any update to that core file.

There are other issues we run into, managing our code and hosting, all of which take fairly time-consuming, manual intervention. Here’s the list:

• Since we host and provide security updates for Joomla, Word Press, Zen Cart, Drupal, and others, we need to upgrade dozens of installations any time there’s a new release that has a fix for a security vulnerability. With Joomla this has happened quite a lot, and every Joomla installation needs to be upgraded individually–and tested. And since each installation is slightly different, we can’t manage them easily within a single repository, while updating the underlying code.
• Templates, modules, components, blocks, themes, plugins, and whatever. Developing these types of add-ons are our bread-and-butter. But code for these often get scattered across an installation, making it quite difficult to manage just our add-ons while we develop them, or roll back to earlier versions if there’s a problem.
• The Dojo Toolkit, and builds. We’re doing a lot of development with Dojo right now, to add desktop-like functionality such as trees, sortable tables, right-click menus, animations, and lots of other really cool things. However, if you don’t “build” the code after you write it, it’s painfully slow in a web browser. And due to the nature of how Subversion works, you can’t easily store a built Dojo tree if you ever want to change it again. Which means you’d need to build it every place you deploy it. And on some computers, it can take a long time to build–on our demo server, one of our projects currently takes 8 minutes.
• As we get more directly involved with open source projects like LedgerSMB, we’re finding the need to change core files while we hack away at some particular feature. To do this, you create a branch of the code, work on your feature, and then merge your changes back into the “trunk.” If you don’t have access to save directly to the project repository, doing this gets a lot more complicated.

Git to the rescue. Git solves all of these issues. Read on for a technical discussion of how.
(more…)

Unbelievable. Microsoft was one of the first places to support WebDAV, and after a little investigation, looks like they’ve completely changed how they support it–with security implications, and an amazing amount of brokenness…

At Freelock, we’ve used WebDAV to allow our clients to access to our servers since day 1. FTP is fundamentally unsecure, and as business level hosts, we refuse to allow that. SFTP is a really good option, but it does require us to set up local user accounts on the server and allow a higher level of access–something we would prefer not to do on our shared servers. WebDAV has long been the clear answer to this, supported by every major operating system with no extra add-ons, and also supported by most web development tools natively. That is, until last year, when Microsoft completely changed the way they do WebDAV. It even breaks compatibility with their own Sharepoint software!

Without testing this fully, this appears to be the situation, version by version:

* Windows 98, 2000, XP (does this still work in SP2/3? Dunno):
- Use Internet Explorer. Go to the File -> Open dialog, check the box to open as web folder, enter the WebDAV URL, and open.
This works really well, though if you bookmark it, it will open it as a web page, no longer a web folder.

* Windows XP SP2:
1. Apply a registry hack to enable basic authentication
2. Open Windows Explorer, and go to Tools -> Map Network Drive
3. Enter the path to the drive (you cannot use https unless you have Office installed) and a drive letter to map. Alternatively, use NET USE on the command line.

* Vista:
1. Apply registry hack to enable basic authentication (or set up server to use Digest authentication, and strip domain name out of user credentials)
2. Set up server to not reject requests to anywhere in the path for OPTIONS and PROPFIND requests

Read on for more details.
(more…)

Packtpub is running a sample from a developer’s guide for customizing SugarCRM. The author describes how to set up hooks for particular modules to build a custom workflow.

Custom workflows are a feature that is limited to the proprietary version of SugarCRM–they have not been available in the open source version. With custom development using techniques illustrated here, you can add your own workflows.

This looks to me like it’s written specifically for versions of SugarCRM before version 5. I haven’t had a chance to find out whether the same basic techniques would apply–SugarCRM 5 changes a lot of things from earlier versions, primarily with email handling and storing customizations in the database rather than scattered around files. The basic approach should work, however…

Developing a Simple Workflow within SugarCRM

Some handy tips for users of OpenOffice.org, looking to get away from the mouse…

Ten fantastic keyboard shortcuts in OpenOffice.org

1. You want to make sure nobody will be able to read your documents in 10 years
2. You want to help your buddy who works for Microsoft have enough income to buy a private island in the Carribean, because maybe he would invite you to come for a weekend
3. You feel sorry for the PC on the Mac commercials
4. Your buddy is buying it for you from the Microsoft company store, so you’re actually saving hundreds of dollars! You can’t get those types of deals on free software.
5. You hope that the extra emails it takes between you and your customers, partners, and vendors to get formats that they can open will improve your relationship with them
6. Having the newest software from Microsoft makes you cool
7. You want to extend Microsoft’s monopoly on the desktop, it’s just easier that way
8. You have a big technology budget, and can’t think of any better way to spend those dollars
9. You already spent the money on it, may as well force others to pay their Microsoft tax, too
10. You’re a big fan of Survivor, and like being dropped into an unfamiliar environment and having to figure out all over again how to do the things you need to survive
11. (Bonus!) You don’t know a better solution exists

If your reason for purchasing Office 2007 is #11, drop us an email and ask us how open source software can make your business run better.

I’ve been struggling to get Project Auriga to set HTTP Auth from a nice pretty login form, and think I have it working.

What follows is a very technical discussion–if you’re a business reader, you should probably skip this post…

HTTP Auth is a specific mechanism for handling authentication. HTTP Auth is built into Apache and IIS, and so the server can handle authentication purely through configuration, offering many different back ends for storing the data. Browsers also handle HTTP Auth natively, popping up a normal login box whenever it gets a Basic Authentication request from the server. But this login box is ugly, and doesn’t provide a friendly experience to allow people to create an account, get a password resent, or anything–it falls back to a basic error page. You can, of course, customize the error page, but not necessarily help people with the password login itself.

There are several benefits to using HTTP Auth, though. First of all, other applications on the same server can accept the same credentials, allowing you to sign in once and access multiple applications without having to log into each one. Secondly, you can set up stronger authentication methods, such as client-side certificates. Also, you can configure the server to protect large parts of a web site very easily, reducing exposure to information disclosure.

So how do you make a sign-in form on a web application set http auth? Browsers do not allow you to access these settings via script. You can use an XmlHttpRequest object to set authentication, but only after the proper challenge has been sent from the server. The biggest problem is, if the server sends this challenge twice in a row, your browser will intercept the second request and pop up the ugly password prompt. So designing a form that keeps this login prompt from popping up under most circumstances is quite the challenge.

The gist of the issue is that while you can open an XmlHttpRequest object with a user and password for http authentication, the browser will only actually use those credentials after the server has rejected a request. The process looks like this:

1. Your script creates and sends an XmlHttpRequest with http auth username and password.
2. The browser submits the request to the server, without sending the username and password.
3. The server responds with 401 requires authentication, and a WWW-Authenticate header specifying a realm.
4. The browser looks in its cache to see if it already has http auth set for that domain and realm. If it does, it sends those credentials, NOT THE ONES you specified in your XmlHttpRequest. If it does not have those credentials, only then will it set http auth to what your script asked for.
5. The server responds. Generally, if the username or password are incorrect, the server will repeat the 401 response, and WWW-Authenticate.
6. The browser gets its second 401 in a row, and pops up its password box. Your script never gets a chance to intercept this. So if the stored http auth credentials are wrong, or the user mistypes the password, their browser takes over and you get a password prompt.

How do you handle this situation? It turns out you need to engage in some trickery on both the client and the server.

Here’s a basic flow of how you need to handle this, from both the server and the client perspective:

1. First, collect the credentials from the user, and create your request as outlined above.
2. Browser sends request without credentials.
3. Server responds with 401 and WWW-Authenticate.
4. Browser sends cached credentials, if they exist, or your credentials if not.
5. If credentials are accepted, server allows log in and responds with 200. If credentials are not accepted, server returns an error code OTHER THAN 401, and does not send a WWW-Authenticate:
   1. We use 403 not authorized for a credential failure here. You might also use 400 Bad Request.
   2. Because the response was something other than 401, your browser caches the bad credentials.
   3. XmlHttpRequest status reflects the error condition.
   4. Your script checks the result for the error your server has returned. Now comes the crucial part:
   5. Your script submits a new request with different credentials to some server location that will return successfully. For example, we call a login method on our application, passing username “public” and password “?”.
   6. The browser sends the new credentials and submits the request.
   7. The server returns 200.
   8. The browser updates its http auth cached credentials with the new bogus ones.
6. Now you can present an error to the user, and ask for new credentials.

The key to the above process is that if the browser gets two 401 responses without having a 200 somewhere between, it will pop up its password box and there’s nothing you can do about it. So the key is to use a different error code to indicate bad credentials, and do an intervening request that will return 200 so that you can re-authenticate.

Logging Out
You cannot really log out of HTTP Auth. But you can change the credentials to a known bad user. That’s a key technique we use to effectively log out of an application, and we re-use this method to reset after bad credentials.

On the server
I’m very much still in development with this. You can see the server side code for Project Auriga logins here.

In this system, we do set a cookie after successful login, to keep from having to check credentials again. This script also allows for cookie-only logins without using http auth. The important bits:

• action=logout: if this is called, the script always returns successfully. This allows the client script to provide new bogus credentials. It passes a username of “public” to log out completely.
• action=httpauth: if this is called, and there are no http auth credentials or the http auth username is “public”, return a 401 and WWW-Authenticate. This is always the first request from a browser, and triggers the browser to re-request with the credentials.
• action=httpauth, with http auth username set, and it’s not “public”: The second or later requests, we never want to return a 401 or the browser will pop up its password prompt. So we return 403 (or 400) if the credentials are bad, or allow the script to continue processing if its good. In this case, our authenticate method returns true if credentials are good, false if the user is not found, and throws an exception if the credentials are bad.

That’s basically what you need to do on the server side. Now for the client.

Client-side logins
We’re using the Dojo Toolkit extensively in Project Auriga, so the login functions are using dojo.xhr* requests to wrap the XmlHttpRequest objects and provide convenient callback functions. You can see our login code here. Key items:

• auriga.login is called by the login form. Note that if this is the first time to this page, the dojo.xhrPost actually happens twice: first time with no credentials, and the second time with them. If the second post is accepted, auriga.login_complete is called. If the second post returns any kind of error, auriga.login_err is called.
• auriga.login_complete is easy… it just redirects to wherever the server response designates.
• auriga.login_err is the real trick here. If it detects the error code we’ve chosen for bad passwords, it immediately calls the server logout method, to get a good response so the next time the browser gets a 401, it won’t immediately pop up the password box.

You can see the code in action on our demo server.

Other notes

• Actually doing single sign-on is hard. We’re trying out different strategies for detecting whether a user already has http auth set, by calling our login method once on page load, but haven’t gotten that figured out. In our current script, just clicking Login with the form blank but authenticated elsewhere on the same domain and Realm, will log you in with your existing credentials.
• Because your browser stores credentials based on the domain and the realm together, all applications that you set to share these items must accept the same credentials. If you have a different password on a different system on the same server, you must set a different realm, or logging into one will log you out of the other.
• If you want to require http auth, but not Javascript, I suggest submitting something different to the server using Javascript to identify this type of request. Perhaps show your form only when Javascript is available, and when it’s not, have a link to a protected page to let your browser go ahead and show the password dialog.
• Using http auth can actually allow users to disable cookies, if your application is RESTful. In Project Auriga, the session login script supports either–the client pages and logins work with either a cookie or http auth. The login process attempts to set http auth and a session cookie. On subsequent attempts, it uses the cookie to avoid re-authenticating every request.
• Finally, a note on security: Basic Authentication provides no protection against passwords being sniffed over the network. If you need a secure login, be sure the server conversation uses SSL–otherwise neighbors on your wireless network can easily sniff out your password. HTTP Auth does not make your application more secure–it just makes it easier to share authentication with other resources on the same server.

LugRadio has a very interesting discussion in their current podcast about the role of a community manager, in creating a vibrant community around an open source project. They came to the conclusion that each project needs a leader that people trust to take the project in the right direction, someone to be a diplomat to resolve issues among people in the community and keep everyone rowing in the same direction, and a strong technical lead to solve the hard problems.

This sounds quite similar to the challenges a small business faces. “The E-Myth Revisited,” by Michael Gerber, is essential reading for anyone wanting to start a small business, and some of the same ideas apply to building a successful open source project.

Basically, Gerber says you need to have 3 personalities in your business:

• The entrepreneur, the person with the vision to drive the business/project forward, always a step ahead working on what’s next
• The manager, somebody to make sure all the work gets done on schedule, delivered on time, and that everybody is working in the same direction with the same priorities
• The technician, who actually does the work

And small businesses, like open source projects, are almost always started by technicians, people who just know they can do a better job than anybody else, so they set out to do it themselves. The reason why so many businesses fail is because the founders spend all their time building something without setting enough overall direction or managing their cash flow. Sound familiar?

The key to creating both a successful business and an open source project is balancing out these 3 personalities and making sure all are represented to an appropriate degree.

Patrick asks,

Why not OpenSuSE, instead of Ubuntu?

At Freelock, we provide a maintenance service contract to manage Linux servers. For a fixed monthly fee, we provide monitoring, system updates, application updates, and our help recovering anything that goes wrong with an upgrade. We’re looking at adding disaster recovery to the mix, raising the price to cover the cost of backing up all of the data and providing varying service level agreements on how soon we will recover your machine from a total loss. But for our base price, we only support Ubuntu and CentOS, with a preference for Ubuntu. So Patrick asks, why not OpenSuSE? Here’s my reply:

Hi, Patrick,

There are literally hundreds of different distributions of Linux, and 7 or 8 that are very widely deployed in server environments. The reasons Ubuntu is our preferred distribution include:

• No separate version of operating system–Red Hat and Novell/SuSE keep some of their stuff for their commercial version
• Commercial company backing it (Canonical, LTD), support contracts available
• Strong community support, lots of friendly help available
• “Long Term Support” releases, with a commitment by Canonical to maintain security releases for 5 years on designated versions
• Superior upgrade path when a version reaches end-of-life
• Nice balance between cutting-edge versions of new software, while making sure they’re stable
• Easy package management, well-built packages

OpenSuse has no commercial support available, and no commitment on the part of Novell to provide long term support to their free versions–you have to buy SuSE Enterprise Linux to get that level of stability/support, and that’s not the same software–they bundle older versions in their enterprise distributions that often don’t have features we’re coding on top of…

The upgrade path is another nice feature–we’ve had very good success upgrading Ubuntu boxes in place, without having to install an entirely new system and migrate the data over.

We support CentOS as well, which is a free version of Red Hat Enterprise Linux, because some of the software our clients run are only available for Red Hat, so we have to… CentOS doesn’t have as good an upgrade path as Ubuntu, so we don’t include system upgrades at software end-of-life with that. We have supported SuSE boxes in the past, but I think we’re down to a single one running some legacy software. The main reason we discourage it is to streamline our processes–it’s much easier to administer a bunch of the same operating system, than having every box be a one-off system.

We’ve been doing this long enough to have to upgrade several servers because the OS has reached end-of-life and there are no more security releases available. Of the free distributions, the only ones we can deploy with confidence knowing we won’t have to upgrade for at least 3 or 4 years are Ubuntu, CentOS, and Debian. While there are many other solid distribution choices you could make, none of the others quite stack up to meet our needs as well as Ubuntu and CentOS.

This is a new feature we’re starting: Ask Freelock! Have a question about using open source in business? Drop us a line, ask us a question. We’ll do our best to answer.