Open Source Solutions for Small Business Problems

One of my projects in the past few weeks has been to put together a SOAP server for a client. So suddenly I’ve had to learn a lot of the nitty gritty details about what works and what doesn’t…

While they’re fresh, let me jot them down here. WARNING: Extremely technical content ahead.
(more…)

At Freelock, we’re always trying to figure out ways to do things better. Recently I started digging into a developer tool that’s making, as Bryan over at the Linux Action Show would say, my head explode.

For a long time, we’ve managed our custom code projects and business documents in a central repository, called Subversion (also known as svn). Subversion is relatively easy to understand–it’s like having a library of files you can check a copy out of, do some work on it, and then check it back in. Subversion is the librarian that tracks who has copies of what, and when they checked it out. So if Erik checks in changes to a brochure, and then Jill goes to submit changes to the same document, Subversion will say “hey wait a minute, that document has already been changed–you need to make sure you put Erik’s changes in your document before I’ll let you put in your document.”

This is great for managing conflicts between people working on a single team, or for code that is being developed in relative isolation from the rest of the world.

The problem is, we’re doing more than that–we’re taking code from various open source projects and either customizing it or building new applications on top of it. And so when the outside projects get updated, we need to manually update anything we’ve written that depends on that code. There is no longer a single repository where we control our code–there is our code library, plus another one for every project we use.

This makes managing add-ons for projects like Joomla or ZenCart quite challenging, because our add-ons get scattered throughout the filesystem to be able to hook into the right place. And if we have to touch a core file, we’re going to end up needing to re-implement our change with any update to that core file.

There are other issues we run into, managing our code and hosting, all of which take fairly time-consuming, manual intervention. Here’s the list:

• Since we host and provide security updates for Joomla, Word Press, Zen Cart, Drupal, and others, we need to upgrade dozens of installations any time there’s a new release that has a fix for a security vulnerability. With Joomla this has happened quite a lot, and every Joomla installation needs to be upgraded individually–and tested. And since each installation is slightly different, we can’t manage them easily within a single repository, while updating the underlying code.
• Templates, modules, components, blocks, themes, plugins, and whatever. Developing these types of add-ons are our bread-and-butter. But code for these often get scattered across an installation, making it quite difficult to manage just our add-ons while we develop them, or roll back to earlier versions if there’s a problem.
• The Dojo Toolkit, and builds. We’re doing a lot of development with Dojo right now, to add desktop-like functionality such as trees, sortable tables, right-click menus, animations, and lots of other really cool things. However, if you don’t “build” the code after you write it, it’s painfully slow in a web browser. And due to the nature of how Subversion works, you can’t easily store a built Dojo tree if you ever want to change it again. Which means you’d need to build it every place you deploy it. And on some computers, it can take a long time to build–on our demo server, one of our projects currently takes 8 minutes.
• As we get more directly involved with open source projects like LedgerSMB, we’re finding the need to change core files while we hack away at some particular feature. To do this, you create a branch of the code, work on your feature, and then merge your changes back into the “trunk.” If you don’t have access to save directly to the project repository, doing this gets a lot more complicated.

Git to the rescue. Git solves all of these issues. Read on for a technical discussion of how.
(more…)

I’ve been struggling to get Project Auriga to set HTTP Auth from a nice pretty login form, and think I have it working.

What follows is a very technical discussion–if you’re a business reader, you should probably skip this post…

HTTP Auth is a specific mechanism for handling authentication. HTTP Auth is built into Apache and IIS, and so the server can handle authentication purely through configuration, offering many different back ends for storing the data. Browsers also handle HTTP Auth natively, popping up a normal login box whenever it gets a Basic Authentication request from the server. But this login box is ugly, and doesn’t provide a friendly experience to allow people to create an account, get a password resent, or anything–it falls back to a basic error page. You can, of course, customize the error page, but not necessarily help people with the password login itself.

There are several benefits to using HTTP Auth, though. First of all, other applications on the same server can accept the same credentials, allowing you to sign in once and access multiple applications without having to log into each one. Secondly, you can set up stronger authentication methods, such as client-side certificates. Also, you can configure the server to protect large parts of a web site very easily, reducing exposure to information disclosure.

So how do you make a sign-in form on a web application set http auth? Browsers do not allow you to access these settings via script. You can use an XmlHttpRequest object to set authentication, but only after the proper challenge has been sent from the server. The biggest problem is, if the server sends this challenge twice in a row, your browser will intercept the second request and pop up the ugly password prompt. So designing a form that keeps this login prompt from popping up under most circumstances is quite the challenge.

The gist of the issue is that while you can open an XmlHttpRequest object with a user and password for http authentication, the browser will only actually use those credentials after the server has rejected a request. The process looks like this:

1. Your script creates and sends an XmlHttpRequest with http auth username and password.
2. The browser submits the request to the server, without sending the username and password.
3. The server responds with 401 requires authentication, and a WWW-Authenticate header specifying a realm.
4. The browser looks in its cache to see if it already has http auth set for that domain and realm. If it does, it sends those credentials, NOT THE ONES you specified in your XmlHttpRequest. If it does not have those credentials, only then will it set http auth to what your script asked for.
5. The server responds. Generally, if the username or password are incorrect, the server will repeat the 401 response, and WWW-Authenticate.
6. The browser gets its second 401 in a row, and pops up its password box. Your script never gets a chance to intercept this. So if the stored http auth credentials are wrong, or the user mistypes the password, their browser takes over and you get a password prompt.

How do you handle this situation? It turns out you need to engage in some trickery on both the client and the server.

Here’s a basic flow of how you need to handle this, from both the server and the client perspective:

1. First, collect the credentials from the user, and create your request as outlined above.
2. Browser sends request without credentials.
3. Server responds with 401 and WWW-Authenticate.
4. Browser sends cached credentials, if they exist, or your credentials if not.
5. If credentials are accepted, server allows log in and responds with 200. If credentials are not accepted, server returns an error code OTHER THAN 401, and does not send a WWW-Authenticate:
   1. We use 403 not authorized for a credential failure here. You might also use 400 Bad Request.
   2. Because the response was something other than 401, your browser caches the bad credentials.
   3. XmlHttpRequest status reflects the error condition.
   4. Your script checks the result for the error your server has returned. Now comes the crucial part:
   5. Your script submits a new request with different credentials to some server location that will return successfully. For example, we call a login method on our application, passing username “public” and password “?”.
   6. The browser sends the new credentials and submits the request.
   7. The server returns 200.
   8. The browser updates its http auth cached credentials with the new bogus ones.
6. Now you can present an error to the user, and ask for new credentials.

The key to the above process is that if the browser gets two 401 responses without having a 200 somewhere between, it will pop up its password box and there’s nothing you can do about it. So the key is to use a different error code to indicate bad credentials, and do an intervening request that will return 200 so that you can re-authenticate.

Logging Out
You cannot really log out of HTTP Auth. But you can change the credentials to a known bad user. That’s a key technique we use to effectively log out of an application, and we re-use this method to reset after bad credentials.

On the server
I’m very much still in development with this. You can see the server side code for Project Auriga logins here.

In this system, we do set a cookie after successful login, to keep from having to check credentials again. This script also allows for cookie-only logins without using http auth. The important bits:

• action=logout: if this is called, the script always returns successfully. This allows the client script to provide new bogus credentials. It passes a username of “public” to log out completely.
• action=httpauth: if this is called, and there are no http auth credentials or the http auth username is “public”, return a 401 and WWW-Authenticate. This is always the first request from a browser, and triggers the browser to re-request with the credentials.
• action=httpauth, with http auth username set, and it’s not “public”: The second or later requests, we never want to return a 401 or the browser will pop up its password prompt. So we return 403 (or 400) if the credentials are bad, or allow the script to continue processing if its good. In this case, our authenticate method returns true if credentials are good, false if the user is not found, and throws an exception if the credentials are bad.

That’s basically what you need to do on the server side. Now for the client.

Client-side logins
We’re using the Dojo Toolkit extensively in Project Auriga, so the login functions are using dojo.xhr* requests to wrap the XmlHttpRequest objects and provide convenient callback functions. You can see our login code here. Key items:

• auriga.login is called by the login form. Note that if this is the first time to this page, the dojo.xhrPost actually happens twice: first time with no credentials, and the second time with them. If the second post is accepted, auriga.login_complete is called. If the second post returns any kind of error, auriga.login_err is called.
• auriga.login_complete is easy… it just redirects to wherever the server response designates.
• auriga.login_err is the real trick here. If it detects the error code we’ve chosen for bad passwords, it immediately calls the server logout method, to get a good response so the next time the browser gets a 401, it won’t immediately pop up the password box.

You can see the code in action on our demo server.

Other notes

• Actually doing single sign-on is hard. We’re trying out different strategies for detecting whether a user already has http auth set, by calling our login method once on page load, but haven’t gotten that figured out. In our current script, just clicking Login with the form blank but authenticated elsewhere on the same domain and Realm, will log you in with your existing credentials.
• Because your browser stores credentials based on the domain and the realm together, all applications that you set to share these items must accept the same credentials. If you have a different password on a different system on the same server, you must set a different realm, or logging into one will log you out of the other.
• If you want to require http auth, but not Javascript, I suggest submitting something different to the server using Javascript to identify this type of request. Perhaps show your form only when Javascript is available, and when it’s not, have a link to a protected page to let your browser go ahead and show the password dialog.
• Using http auth can actually allow users to disable cookies, if your application is RESTful. In Project Auriga, the session login script supports either–the client pages and logins work with either a cookie or http auth. The login process attempts to set http auth and a session cookie. On subsequent attempts, it uses the cookie to avoid re-authenticating every request.
• Finally, a note on security: Basic Authentication provides no protection against passwords being sniffed over the network. If you need a secure login, be sure the server conversation uses SSL–otherwise neighbors on your wireless network can easily sniff out your password. HTTP Auth does not make your application more secure–it just makes it easier to share authentication with other resources on the same server.