Although the Internet is global, and you may do business with people anywhere in the world, most people tend to look for people-networks close to home. Or do they? Should they? If the point of social networking is to connect with other people, ought it to matter where we are?

At Freelock, we have a handful of remote clients, but upwards of 90% of our clients are local. I founded the business on the assumption that people want to know who they’re doing business with, be able to see them face to face, and grow to trust them over time. Nothing breaks the ice like talking about a project in person, over a coffee or better yet, a margarita.

Good or bad, business gets done through personal relationships. How many deals have been cemented on the golf course? It’s a lot harder to say no in person, than it is with a quick dismissive email. So much communication happens non-verbally, through body language, tone of voice, and other channels that just aren’t available online. A video conference is a poor substitute for a face-to-face meeting.

I’ve gotten quite a bit of help from IRC. We have a private company jabber server so when we’re not in the same room, we can still have the feel of a team. We’ve had people helping us out from Bellingham, 120 miles north. The Internet enables some amazing things, and I definitely think it’s possible to work effectively at a distance. Many professions, including writing and coding, can be done quite effectively by individuals working by themselves, anywhere in the world.

But you can’t directly diagnose a connectivity issue in an office in Bellevue when you’re in India, or replace a hard drive. You can’t assemble a car from the other side of the world. And even for creative types who can work effectively on their own, relationships and trust only truly get cemented by meeting their editors, testers, or project managers in person.

Another founding principle of my business is that it’s much easier to ensure quality by having people work in person. If team members can do impromptu code reviews of each other’s work, quality goes up. The solitary developer working late at night may bang his head for hours against a problem that a colleague could solve in a 5 minute conversation. Having a team of people with complementary talents and different strengths working in one place leads to better results.

Once you’ve established that level of trust, remote work becomes more effective. You know when somebody’s cracking a joke, and it doesn’t sound so strange. You’re more likely to ask a quick question in a chat when you can preface it with a comment about an outside shared interest.

Yes, location matters. It’s not everything, and the Internet makes it possible to work together from a distance–but it still matters.

While they’re fresh, let me jot them down here. WARNING: Extremely technical content ahead.

First of all, SOAP is supposed to stand for “Simple Object Access Protocol.” It’s anything but simple. There is a lot of SOAP software out there, but subtle implementation gotchas that can be quite difficult to figure out.

We chose the native PHP SoapServer in PHP 5.2 to implement the project, mainly because we’re a PHP shop, and a little smoke testing revealed it was quite quick to get set up and going. It turns out that it’s quite hard to debug. For its good points, it can read in a WSDL and automatically map methods to methods on a class, and it converts arrays, simple objects, or complex objects to a valid response object, and request objects into simple or complex objects on the incoming side.

Problems with PHP’s SOAP Server:

• No validation of incoming or outgoing documents.
• No warnings, exceptions, or errors if it can’t convert a document to fit the schema–it just dies.
• No debugging information about what it’s doing.
• No ability to manage namespaces, especially if they need to be copied from the SOAP envelope into the payload.
• Difficult to test.
• No access to the raw XML of either the request or the response.

Using PHP’s SoapServer is quite simple, except when things aren’t perfect…

Here’s what the code looks like for the simple case:


< ?php
$xml = $GLOBALS['HTTP_RAW_POST_DATA']; // or file_get_contents('php://input');
// make sure you have something to process, throw an error if $xml is empty
$soap = new SoapServer('http://path/to/your.wsdl');
$soap->setClass(’mySoapClass’);
$soap->handle($xml);
?>

That’s basically it. You declare methods on ‘mySoapClass’ that correspond to the SOAP methods. These handler methods receive a simple object as a parameter, and you can do whatever you need to do with that data. Then it needs to return some data structure that can be serialized to the expected type defined in the WSDL. The return data structure can be an array, a simple object, or an object of a class you define that can serialize appropriately.

Great. With this much, it took me about a day to have a working web service with 8 methods and a bunch of complex data objects. The problems started when people connected with different SOAP software.

The web service I was implementing defines a specific SOAP Fault document, so if I did run across a problem, I could simply throw an exception of that type. My wrapper object kindly passed the custom fields defined in the WSDL.

Problem #1: Validation

As I said before, there is none. If the SoapServer gets anything it doesn’t like, it doesn’t send any response at all. And since all you get inside your method handlers is an already-converted object, you don’t have any way to validate the response without using a global variable or a call to a singleton.

In our case, the project specified that we strip out the payload from the SOAP header and store the payload XML on the disk as a document for several methods, and do processing on other methods. Processing a SOAP request was not a problem. Storing a valid XML document was. Several methods just stored the XML on the disk, with another method retrieving it and returning it to the caller. The problem we had was that the Soap Response was more picky than the Soap Request–so documents that we loaded from the disk and returned as the response would fail with no explanation.

Our solution was to load the raw XML into a DOM Document, and validate it against the schema. This mostly worked, until we had to deal with a request generated from Jitterbit. More on that later.

The question is, what to validate? We weren’t supposed to store the entire SOAP envelope–just the payload. So how to extract it? The simple way was to grab the first child of the Body element, append it to the DOMDocument itself, remove the original root, and call the normalize() method. This did generate a warning, but not a fatal error, and did the right thing. Furthermore, we could also access the raw libxml validation specifics, by calling libxml_use_internal_errors(true), and then when a document fails to validate, using libxml_get_errors() and libxml_get_error() to grab the details.

Problem #2: Returning valid XML responses

The root of all of our problems in this project has to do with where namespaces are defined. One limitation of libxml appears to be that you can only point it to one schema for validation at a time. We can validate against the SOAP Schema, or our custom schema, but not both at the same time, unless one includes the other. So our validation options consist of:

1. Include the SOAP Schema in the custom schema, and validate the entire SOAP body, or
2. Extract the payload from the SOAP body, and validate only that against our schema.

#2 is clearly the correct way–we really don’t care about the SOAP envelope once we have the message. But the problem is, many SOAP clients put the namespace declarations on the SOAP Envelope, and not the payload root element. In fact, the XML generated by the PHP SoapServer class does this itself.

So our first task was to generate the proper XML Namespace declarations on our generated payloads.

To do this, we could no longer rely on the PHP SoapServer’s automatic conversion of simple objects or arrays to XML–we had to generate our own XML, and tell the SOAP server to use that instead. This turned out to be difficult to track down, so here’s the answer:


< ?php
class dataClass{
/* Serialize XML as desired here.
Omit the XML declaration, start with the root element
You can also simply build the XML as a string from object properties
*/
function toXml(){
$this->myDom->documentElement->setAttribute(’xmlns’,'http://my.custom.namespace/version_1′);
$xml = $this->myDom->saveXML();
preg_match(’%< \?.*=\?>(.*)$%s’,$xml,$match); //strip XML declaration
return $match[1];
}
}

// snip to end of actual SOAP handler method:
$out = new SOAPVar($data->toXML(),XSD_ANYXML);
return $out;
}

The SOAPVar object allows you to control the output of the SoapServer a bit better, with support for namespaces, raw XML, or casting objects in a certain way.

Problem #3: Jitterbit

Validating a clean payload without namespace prefixes seems to work fine. Our technique of moving the nodes around with the DOM seemed to keep most of the namespaces intact, so even if there was no xmlns declaration on the payload root, we could still validate just the payload effectively, and serialize/de-serialize without issue.

Except for Jitterbit.

Now, I loaded up Jitterbit yesterday, and it didn’t do this–this might be a problem with an earlier version. The problem is, Jitterbit is extremely verbose, specifying not just a namespace prefix on every element, but also an xsi:type. And even that’s not enough to break it–except that the value for its xsi:type also contained a namespace declaration. And if this namespace declaration was not the root namespace, suddenly our validation broke.

It broke for us on types declared as ns:token, ns:string, ns:integer — the simple types specified by XSD itself, which Jitterbit put into a namespace prefixed with ns: and declared on the SOAP Envelope.

For example, here’s the start of a problem document:


< ?xml version="1.0" encoding="UTF-8" standalone="no" ?>
xmlns:ns="http://www.w3.org/2001/XMLSchema"
xmlns:oi="http://ws.outdoorindustry.org/v1_2/"
xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
xmlns:ws="http://ws.outdoorindustry.org/v1_2/ws/"
xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>

xmlns:tns1="http://ws.outdoorindustry.org/v1_2/"
xsi:type="tns1:PO">
06AB4EF9-6AF157E7-3F8441AB-AB499933
Preseason

999
Sample Vendor

The first validation error was on the VendorID, with xsi:type=”ns:token”. If I copied xmlns:ns=”http://www.w3.org/2001/XMLSchema” into the tns1:submitPO element, it validated fine. The PHP DOMDocument seems to be able to keep track of namespaces on elements and attribute names even after the envelope is gone. But not attribute values.

After hours of banging on this, we came up with 3 workarounds for this:

1. Completely regenerate the XML, after processing. To do this, we would need to create a custom data class for each incoming object, provide a classmap to the SoapServer, and then generate brand new XML out of the data object. This is perhaps the best approach, but I didn’t think of it until the project was over–I was thinking about writing out the data to the database and then loading our custom objects and serializing them as we do for our responses. The biggest drawback here is that we need to model the entire complexity of the request, as allowed in the schema. And this was a really complex object… lots of work to implement, when we’re only going to store this XML for passing to other systems.
2. Hack the XML to get the offending namespace into the stored document. This turned out to be easy to program, but uses lots of CPU resources–DOMDocuments are expensive to use. It’s also the most brittle approach, only catching this single case–if the namespace prefix changes, or a different required namespace is necessary, it’ll break. To do this, we created a new DOM Document, imported the root node of the payload, appended it to the document, and used setAttribute to set an “xmlns:ns” attribute on the root. This did not actually get the namespace recognized for validation, and normalizeDocument did not fix it–but creating a third DOMDocument, and doing $doc->loadXML($doc2->saveXML) did make the namespace recognized by the object so we could successfully validate.
3. Hack the XSD to validate the entire SOAP request. By including the SOAP schema in our custom schema (using xs:import), we could validate the raw SOAP request, then extract the payload and save it. The saved XML does not validate on its own, but we know it validates. So we can remove our validation check on the outgoing document, and as long as the other system does not explicitly validate the standalone XML, we’re okay.

Whew. Hope this helps somebody…

First, McCain has released his technology platform, which among other bits states his support for protecting the recording industry from piracy. Meanwhile, the Ohio Republican party used Jackson Browne’s “Running on Empty” in a commercial without permission.

John McCain 2008 - John McCain for President:

John McCain Will Protect The Creative Industries From Piracy. The entertainment industry is both a vital sector of the domestic economy and among the largest U.S. exporters. While the Internet has provided tremendous opportunity for the creators of copyrighted works, including music and movies, to distribute their works around the world at low cost, it has also given rise to a global epidemic of piracy. John McCain supports efforts to crack down on piracy, both on the Internet and off.

Wired Magazine - First Paris, Now This: Jackson Browne Sues John McCain

Indeed, some irony lies in a candidate who is running a law-and-order campaign being sued for intentional copyright infringing and appropriating someone else’s identity without their permission.

Apparently Browne is a well-known Obama supporter.

Second impression: Tim O’Reilly asking Brian Aker and Monty Widenius about the importance of various proprietary companies: Sun, Adobe, Microsoft. Their answer to Microsoft? Irrelevant. And Tim came back apologizing to the Microsoft sponsors. This just after the presentation that talked about the open source “tribe”, and introducing Tim as the leader of the cult. Was feeling a bit like I may have made a mistake, plopping down cold hard cash to support the cult of O’Reilly.

Fortunately, that thought was momentary. The rest of the event has been extremely rewarding, and very worthwhile.

At home, in my usual networks, I’m the token Linux guy. Beyond our company and some family I’ve converted to Linux, almost none of my friends use open source, and in business circles, I’m the resource for not just Linux, but web technologies, programming, system administration, and most anything computer related. I come to OSCON and I’m a mere end user, still damp behind the ears. There is true genius wandering the convention center. And suddenly, I’m one of the least technical people in the room, though still listened to for my experience trying to bring these projects to the small business world, identifying pitfalls and areas for improvement.

A few years back, I came to OSCON and it seemed that the worse dressed a person was, the higher up in the ranks of alpha geek he was. This year people looked much more presentable. That might be as far as the influence of corporate culture went.

I met R0ml at that previous OSCON I attended, at a BOF session led by Doc Searls. A “BOF”, if you’re not familiar with software conferences, is a “Birds of a Feather”, a mini, informal discussion about a particular topic led by anybody who puts an idea for one up on a bulletin board. I doubt R0ml remembers me, but it was an interesting discussion we had that evening that paralleled his talk today about elevating open source development from the realm of Techne to Praxis–from mere “making” of stuff to “doing” something to influence and lead people. He took issue with the assertion in an earlier keynote about Franklin and Jefferson being our founding geeks–mainly because while everybody needs a geek these days to make computers do their bidding, that’s a useful, technical, thing that puts us squarely in the realm of Techne. What Jefferson and Franklin really were, according to R0ml, were Polymaths, and that is also a better description of open source practitioners. We’re Renaissance people.

More later…

First of all, the Mac Break Weekly show apparently spends some time bashing the open source community, calling out Drupal, and how difficult it is to solve “simple” problems like uploading images for blog posts. In practically the same breath, the hosts claim that the open source community never has any innovation behind it. Irony drips:

• In the world of content management systems (CMSs), most of the innovation starts in open source projects these days, and Drupal is at the cutting edge of this with its powerful system of taxonomies, and hundreds of add-ons freely available.
• I can think of a grand total of 2 proprietary CMSs that have anywhere near as widespread use as most of the open source CMS systems. One has turned open source itself: Movable Type. The other is Sharepoint, and it’s widespread because Microsoft has shipped it out on lots of its server products.
• Complain about usability all you want, but name a proprietary product as powerful as Drupal, that’s easier to install, administer, and configure.
• The TWIT.tv site itself is running on Drupal.

So let’s talk a little about innovation. While Photoshop may still dominate the world of graphic design, but the lines aren’t so clear when it comes to animation. The Blender project recently released its second short animated film, Big Buck Bunny. While you might argue about the strength of its story, you cannot deny that the technical effects are as stunning as any major animated film coming out from the big studios. And it was created by 7 people in 9 months, using open source software. Even the big studios like Pixar, Dreamworks, and Industrial Light and Magic rely on open source software to deliver their magic, such as CinePaint, POVray, and several others.

On the subject of innovation, KDE4 is breaking new ground and stirring up controversy, laying a bedrock that promises the ability to do things beyond the standard “Desktop” paradigm that was invented over 30 years ago and we’ve all used ever since. Meanwhile, the GNOME team is working on creative ways to embed web applications into your desktop.

But the real innovations of the open source community are all a few layers deeper in the application stack, all the plumbing that powers the Internet. Microsoft itself borrowed its early networking stack from BSD, one of the earliest open source operating systems. Domain Name Service (DNS) and email were first implemented on the Internet using open source software (BIND and Sendmail).

The open source community tends to snicker whenever Apple claims to be innovative. Its core “innovations” were all invented somewhere else:

• The Mac Desktop interface borrowed heavily from Xerox PARC labs
• OS X uses BSD under the hood
• “Spaces” were in use in Unix systems for a decade before they arrived on the Mac
• The “Time Machine” functionality in Leopard is standard in many source code management tools

To its credit, Apple polishes these features better than anybody else, making them easier to find and use by normal people. But many, if not most of its innovations come from somebody else.

Read the previous post for more discussion about Linux support.

Dann and Linc had a quite spirited debate about the merits of having a company hire low-end tech support with scripts (Dann) versus having an experienced, savvy, tech professional able to really solve the user’s problem (Linc). Dann’s point was that it can easily be more cost-effective for both the support company and the end user to go straight to reinstalling a system if rebooting doesn’t solve the user’s problem, while Linc seemed to think a savvy tech person could get to the root of the issue much quicker, and brought up the point that there’s a cost to the frustration of users being put through the whole front-line support nightmare.

I’d suggest the situation is even more complex than that, but it also differs greatly between the open source projects and proprietary operating systems. First, let me make a bold statement:

Linux support is far better than Windows or Mac.

But it also has a completely different set of rules. Learn those rules, and you’ll be able to solve your problems more satisfactorily than it’s possible to do in the proprietary world. Let’s talk about these differences, looking specifically at a who, where, what, and how long.

Who can help with your problem?

Let’s take a look at who can help with your problem:

The bottom line here is that while you probably know fewer people personally who can help you with Linux, there are more options for commercial support, and you can reach people who can do more to solve your problem for free, than you can with either Mac or Windows.

The fundamental reason for this is that anybody can read the raw source code of any open source product out there, and with enough skill and talent, can solve your problem without needing to pay anybody for the right to do so. In the proprietary world, only one company can help beyond a certain point: for Windows problems, that’s Microsoft. For problems in an application, it’s the application developer.

So the next question is:

Where can you get help?
Again, let’s compare the options:

The main point of the table above is that in the proprietary world, the harder your problem is to solve, the fewer people can help you solve it. You quickly get down to one place to go, and if it’s in the operating system, it might be expensive or not possible to fix. In the open source world, it’s nearly the opposite case–it can be harder to find the simple quick answer to your question, but the harder and deeper your problem, the more places you can go to get help.

Where do you look for help first? This is the single stumbling block for most otherwise tech-savvy users new to Linux. To learn Windows or Mac you can take a class, talk to neighbors and friends, and find lots of very low-end help that way. For Linux, unless you’re friends with some hard-core geeks, you need to go online to find help. Once you’re there, it really depends on where your problem lies.

For just figuring out how to use a system, go to the forums for your distribution. The Ubuntu forums are a great place to ask general questions. Be specific about what you’re trying to do and you’re more likely to get help. Remember that this level of support is free, and people helping you are volunteering their time and knowledge. While you’re there, see if you can answer somebody else’s question–the more help you give, the more you’ll get in return.

The other fantastic place to go for help, especially for quick questions, is IRC. IRC is a system that provides chat rooms and instant messaging. Most open source projects with any community behind them have a chat room on irc.freenode.net. Install an IRC program like Chatzilla, Konversation, Pidgin, or any number of others, connect to irc.freenode.net, pick a nickname to use, and join the channel for the program or distribution you’re having trouble with. Ask your question nicely, and if you don’t get a response immediately, keep your chat program open for a few hours–not everybody is watching the channel every minute.

What kinds of problems can you solve?
Let’s look at the same types of problems as before, and look at the resolutions:

What’s the key thing in this section? Addressing actual problems is within your control, when you’re working with open source. In the proprietary world, you’re entirely at the mercy of a single software vendor. If your problem is in the interaction between two different applications, you’re really stuck — there’s nothing you can do.

But in the open source world, there’s always something you can do. You can hire anybody with the skills to solve your problem, and fix it in the software itself. You don’t need any blessing from any single software vendor.

Open formats are perhaps more valuable than open source software, for most businesses. Because this is such a compelling advantage of open source software, many proprietary programs are beginning to open up their formats to allow other software to read them–their customers are demanding it.

How long will it take to solve your problem?
I’m not going to bother with a table for this one. The answer is nearly always “too long,” regardless of the operating system.

Actually, that’s not quite always the case–it depends on whether somebody has already solved the problem or not, as well as whether the solution is a fix or a workaround.

Many, many problems in Windows are not really fixes, they’re just workarounds. The only real “fix” for a virus-infected machine is the workaround of reinstalling your operating system. The only fix for lots of other minor issues that cause your system to slow down over time is to reboot. These are not fixes, they’re workarounds.

Real fixes take a lot longer, and need acknowledgment that the problem is real. Workarounds are band aids to get you through until there’s a real fix. And there are some real differences between the entire approaches of each operating system around these fixes.

Windows is chock-full of workarounds. Because Microsoft has gone to great lengths to maintain backwards compatibility of just about everything that’s been released for Windows since Windows 95, it’s full of workarounds to keep the old behavior. Rather than fixing behavior that might really be undesirable, they’ve had to patch it with workarounds because too many existing applications turned out to depend on that bad behavior. That, fundamentally, is why Windows is so big, bloated, slow, and painful to work with.

Apple suffers a different problem: changing their closed libraries too quickly for external developers to keep up. Each release, they break lots of things, and don’t always tell 3rd party developers ahead of time. This means the number of third party developers of Apple software is shrinking–they’ve managed to alienate quite a few. So their polish and high quality comes at the price of having a healthy thriving developer community outside the walls of Apple. There’s little transparency in this process, so developers outside Apple are always playing catch-up, and having to work around these changes in behavior.

Linux has its share of problems, too. Linux does not attempt to maintain “binary” compatibility between versions, though it does its best to maintain compatibility in source code. There is endless debate about the “correct” way to fix a problem, and competition between fixes. The challenge of this is that it can be hard for application developers to keep up, especially if they want to keep their source code closed and only ship software in binary form.

But the process is completely, utterly transparent. Anybody can see the progress on any fixes to any part of the system, and can jump in with their own solution at any time. It’s a true meritocracy, with those doing the actual work and providing the best solutions winning out over time.

The whole open source ecosystem is nimble enough to provide real fixes to technical problems, rather than just simple workarounds. If the source of the problem is design, it can take a long time to get the right design in place and resolve all the issues that changing the design causes in other applications. But when a bug gets fixed, it’s really fixed and usually doesn’t appear again.

I posted my results, and confusion, to the Dspam mailing list. The original developer of this software (which has thousands of users), Jonathan Zdziarski, responded that that did not sound right. He asked if he could log into my server and see what was wrong.

I created a test account for him, and logged in to the same screen so I could watch what he was doing. As I watched, he put debugging messages into his code, ran several tests, and within 10 minutes had identified the problem: the 19-digit number he put into the database (MySQL) was different than the 19-digit number he got out. It was a storage error in somebody else’s software causing the problem, a rounding error. He filed a bug in MySQL, and it was fixed within a month or two. But he also had a workaround for me: change the data type to a set of characters instead of an integer. Problem solved.

• Before asking anybody, do your homework. Use Google, read the project FAQ, make some attempt to learn the basics without pestering people with questions they’ve already answered hundreds of times. Nine times out of ten, your problem has already been encountered and somebody has figured out a workaround.
• Limit the scope of your question to the fundamental problem. Get to the point. I’m obviously guilty of being tremendously long-winded at times, but unless your question is right at the top and asked directly, you’ll probably get ignored. Developers are busy, they don’t want to read a novel, but they’re usually happy to answer a question.
• Provide supporting details after asking your question. Many programs will create a log with lots of information that can help somebody diagnose your problem. Find what looks relevant in the logs. Specify what version of operating system, distribution, application, etc. Specify what you were trying to do, what you expected, and what really happened. But provide this stuff after asking your initial question–people aren’t going to wade through a long email to find your question.
• Contribute something. The easiest way you can contribute is by answering other people’s questions. The whole thing works because people help each other, and this help goes both ways. If you always ask questions and never answer anybody else’s, or provide any other sort of contribution, you’ll eventually start getting ignored.
• Always, always, be positive, respectful, and polite when asking your questions. Developers have a lot invested in their project, and insulting it won’t gain you any favors. Developers are under no obligation to help you either–you haven’t paid for it. Common courtesy is valuable. Complements are welcome.
• Be patient. Sometimes the person who has the answer to your question is away from the computer. Usually you’ll get your problem solved quicker than you would calling some tech support line, but there are times it’s going to take a while. If you don’t get any response in a reasonable period of time (judged by how active the list or forum is, reasonable could be a couple hours or a couple days), there are several likely reasons: You haven’t been specific enough in your question; you’re in the wrong forum (e.g. users when it’s a developer question); nobody else is trying to do what you’re doing (in which case you may need to hire someone with the right skills); the project is dead (it happens sometimes–find another one); or the developers are swamped (give them more time, or come up with a new scenario that sheds light on your problem in a different way).

That list describes how we get open source support at Freelock. Aside from a couple unsupported hardware devices, or issues with proprietary programs, we have yet to get stumped, in over 6 years of extremely heavy Linux and open source use. We’ve never paid a dime for this support, though we have provided help to many others in return.

For a long time, we’ve managed our custom code projects and business documents in a central repository, called Subversion (also known as svn). Subversion is relatively easy to understand–it’s like having a library of files you can check a copy out of, do some work on it, and then check it back in. Subversion is the librarian that tracks who has copies of what, and when they checked it out. So if Erik checks in changes to a brochure, and then Jill goes to submit changes to the same document, Subversion will say “hey wait a minute, that document has already been changed–you need to make sure you put Erik’s changes in your document before I’ll let you put in your document.”

This is great for managing conflicts between people working on a single team, or for code that is being developed in relative isolation from the rest of the world.

The problem is, we’re doing more than that–we’re taking code from various open source projects and either customizing it or building new applications on top of it. And so when the outside projects get updated, we need to manually update anything we’ve written that depends on that code. There is no longer a single repository where we control our code–there is our code library, plus another one for every project we use.

This makes managing add-ons for projects like Joomla or ZenCart quite challenging, because our add-ons get scattered throughout the filesystem to be able to hook into the right place. And if we have to touch a core file, we’re going to end up needing to re-implement our change with any update to that core file.

There are other issues we run into, managing our code and hosting, all of which take fairly time-consuming, manual intervention. Here’s the list:

• Since we host and provide security updates for Joomla, Word Press, Zen Cart, Drupal, and others, we need to upgrade dozens of installations any time there’s a new release that has a fix for a security vulnerability. With Joomla this has happened quite a lot, and every Joomla installation needs to be upgraded individually–and tested. And since each installation is slightly different, we can’t manage them easily within a single repository, while updating the underlying code.
• Templates, modules, components, blocks, themes, plugins, and whatever. Developing these types of add-ons are our bread-and-butter. But code for these often get scattered across an installation, making it quite difficult to manage just our add-ons while we develop them, or roll back to earlier versions if there’s a problem.
• The Dojo Toolkit, and builds. We’re doing a lot of development with Dojo right now, to add desktop-like functionality such as trees, sortable tables, right-click menus, animations, and lots of other really cool things. However, if you don’t “build” the code after you write it, it’s painfully slow in a web browser. And due to the nature of how Subversion works, you can’t easily store a built Dojo tree if you ever want to change it again. Which means you’d need to build it every place you deploy it. And on some computers, it can take a long time to build–on our demo server, one of our projects currently takes 8 minutes.
• As we get more directly involved with open source projects like LedgerSMB, we’re finding the need to change core files while we hack away at some particular feature. To do this, you create a branch of the code, work on your feature, and then merge your changes back into the “trunk.” If you don’t have access to save directly to the project repository, doing this gets a lot more complicated.

Git to the rescue. Git solves all of these issues. Read on for a technical discussion of how.

Managing lots of installations with Git
We haven’t yet started doing this, but for projects like Joomla and ZenCart, this is going to be really effective. It’s not quite as helpful for Drupal, which allows for a bunch of sites to be centrally managed out of the box–though even there it will help us recover quickly from a failed upgrade.

Here’s the basic approach:

1. Create a master git repository, either by pulling down the Subversion repository using git svn, or just unpacking tarballs of a release and importing into our git repository for the project.
2. Create a new git clone for each installation, and drop into each installation directory. Create a current branch for that repository, and rebase it to the current installed version.
3. Commit local modifications to the repository. Make sure the .git directory is getting backed up, and is not accessible through the web server.
4. When a new release is available, use git pull to update each local installation. Use git reset HEAD^ –hard to undo if there’s a problem.

Managing submodules, customizations
This we get pretty much for free by having dedicated repositories for particular problems. Managing add-ons we’d like to use elsewhere can still be challenging, but we can pull customizations from other repositories to assemble custom packages–it’s at least easier to manage these across installations.

The main thing to do is to make sure when you’re developing on a particular add-on, you’re always doing it in a branch dedicated to those changes. Then you can merge them into other projects more easily. It’s also possible to “cherry-pick” commits involving particular files, and pull those changes into a new branch if development ended up getting mixed in with different features.

Dojo Toolkit builds
We’re huge fans of the Dojo Toolkit. It makes developing really sophisticated browser-based applications much easier, not just providing a large set of active widgets but also a data abstraction layer and an encapsulation system for your own code.

Its biggest downside is that every feature you want to use means adding another file that the browser needs to load. And we use a lot of features… the base package loads about 20 files, and in our applications there are well over a hundred. This takes a long time to load, and isn’t very scalable.

The solution? Use the Dojo build process. A Java program can build all the features you want to use into a single compressed, stripped, javascript file. The difference in speed is incredible–applications suddenly load right up. And this build system can easily build your Javascript as well.

The problem is, this build system is not compatible with Subversion, because Subversion scatters its working copy data inside each directory, but the build system deletes the directories every time you build. So you can’t easily manage built code inside Subversion.

Git stores all of its repository information in the top level directory, so you can easily build and re-build Dojo and manage it without any issues.

Do local development on open source projects
Now here’s the traditional reason to use git: to allow you to manage changes to open source projects without needing write access to their code repositories. We simply set up git repositories to track their Subversion repository, and then we can use all the features of git to manage our customizations.

When the original project has new releases, we just pull them into our repository and git handles most of the merging for us. If we want to contribute back to the project, we can copy them into our Subversion tracking branch and commit them straight back to Subversion–or send a patch.

We’re starting to set up public git repositories for projects we work with extensively. You can browse them here: http://git.freelock.com. You can clone and pull updates from here: git://git.freelock.com/git/<projectname>.git. You can read more about how to manage non-standard Subversion repositories in git here.

At Freelock, we’ve used WebDAV to allow our clients to access to our servers since day 1. FTP is fundamentally unsecure, and as business level hosts, we refuse to allow that. SFTP is a really good option, but it does require us to set up local user accounts on the server and allow a higher level of access–something we would prefer not to do on our shared servers. WebDAV has long been the clear answer to this, supported by every major operating system with no extra add-ons, and also supported by most web development tools natively. That is, until last year, when Microsoft completely changed the way they do WebDAV. It even breaks compatibility with their own Sharepoint software!

Without testing this fully, this appears to be the situation, version by version:

* Windows 98, 2000, XP (does this still work in SP2/3? Dunno):
- Use Internet Explorer. Go to the File -> Open dialog, check the box to open as web folder, enter the WebDAV URL, and open.
This works really well, though if you bookmark it, it will open it as a web page, no longer a web folder.

* Windows XP SP2:
1. Apply a registry hack to enable basic authentication
2. Open Windows Explorer, and go to Tools -> Map Network Drive
3. Enter the path to the drive (you cannot use https unless you have Office installed) and a drive letter to map. Alternatively, use NET USE on the command line.

* Vista:
1. Apply registry hack to enable basic authentication (or set up server to use Digest authentication, and strip domain name out of user credentials)
2. Set up server to not reject requests to anywhere in the path for OPTIONS and PROPFIND requests

Read on for more details.

Microsoft disables basic authentication in Vista and now in Windows XP SP2. To turn it back on, it takes a registry hack:
http://support.microsoft.com/kb/841215
http://www.simple-groupware.de/cms/Main/WebDAV

First of all, it looks like there’s a Microsoft fix you can download that makes it work SOMETIMES.
http://kb.iu.edu/data/araf.html

Windows XP only supported SSL-secured webdav if MS Office is installed:
http://www.vistax64.com/vista-general/20684-webdav-windows-vista.html

Also in that thread: instructions for using “net use” to mount a webdav folder–maybe something we can create a batch file for?

Nice instructions, with screenshots:
http://kb.wisc.edu/luwmad/page.php?id=6280
http://www.yale.edu/its/software/win/configuration_instructions/WebDAV-Vista.pdf

Something to try: using backslashes in the URL:
http://www.wensh.net/archive.php/topic/1478.html

Lots of investigation in this thread:
http://www.nabble.com/WebDAV-with-Windows-Vista-to9454856.html
… along with the rather astonishing network trace showing how broken it is…

From the Subversion book: http://svnbook.red-bean.com/en/1.2/svn.webdav.clients.html

Microsoft Web Folders

Microsoft was one of the original backers of the WebDAV specification, and first started shipping a client in Windows 98, known as “Web Folders”. This client was also shipped in Windows NT4 and 2000.

The original Web Folders client was an extension to Explorer, the main GUI program used to browse filesystems. It works well enough. In Windows 98, the feature might need to be explicitly installed if Web Folders aren’t already visible inside “My Computer”. In Windows 2000, simply add a new “network place”, enter the URL, and the WebDAV share will pop up for browsing.

With the release of Windows XP, Microsoft started shipping a new implementation of Web Folders, known as the “WebDAV mini-redirector”. The new implementation is a filesystem-level client, allowing WebDAV shares to be mounted as drive letters. Unfortunately, this implementation is incredibly buggy. The client usually tries to convert http URLs (http://host/repos) into UNC share notation (\\host\repos); it also often tries to use Windows Domain authentication to respond to basic-auth HTTP challenges, sending usernames as HOST\username. These interoperability problems are severe and documented in numerous places around the web, to the frustration of many users. Even Greg Stein, the original author of Apache’s WebDAV module, recommends against trying to use XP Web Folders against an Apache server.

It turns out that the original “Explorer-only” Web Folders implementation isn’t dead in XP, it’s just buried. It’s still possible to find it by using this technique:

1. Go to ‘Network Places’.
2. Add a new network place.
3. When prompted, enter the URL of the repository, but include a port number in the URL. For example, http://host/repos would be entered as http://host:80/repos instead.
4. Respond to any authentication prompts.

There are a number of other rumored workarounds to the problems, but none of them seem to work on all versions and patchlevels of Windows XP. In our tests, only the previous algorithm seems to work consistently on every system. The general consensus of the WebDAV community is that you should avoid the new Web Folders implementation and use the old one instead, and that if you need real a real filesystem-level client for Windows XP, then use a third-party program like WebDrive or NetDrive.

A final tip: if you’re attempting to use XP Web Folders, make sure you have the absolute latest version from Microsoft. For example, Microsoft released a bug-fixed version in January 2005, available at http://support.microsoft.com/?kbid=892211. In particular, this release is known to fix a bug whereby browsing a DAV share shows an unexpected infinite recursion.

Unbelievable. Of course, this client is the one they dropped from Vista, so now you’re really stuck…

WebDrive, NetDrive

Both WebDrive and NetDrive are excellent commercial products which allows a WebDAV share to be attached as drive letters in Windows. We’ve had nothing but success with these products. At the time of writing, WebDrive can be purchased from South River Technologies (http://www.southrivertech.com). NetDrive ships with Netware, is free of charge, and can be found by searching the web for “netdrive.exe”. Though it is freely available online, users are required to have a Netware license. (If any of that sounds odd to you, you’re not alone. See this page on Novell’s website: http://www.novell.com/coolsolutions/qna/999.html)

Detailed list of versions and broken-ness:
http://www.greenbytes.de/tech/webdav/webdav-redirector-list.html
http://en.wikipedia.org/wiki/WebDAV

Possible clients to try:
EnginSite DataFreeway: http://www.enginsite.com/Download.htm#Freeware (Proprietary, Free)
BitKinex: http://www.sharewareconnection.com/download-bitkinex-ftp-client-from-sharecon.html (Shareware)
WebDrive: http://www.sharewareconnection.com/webdrive.htm (Free evaluation, $59/year)
NetDrive: http://www.sharewareconnection.com/netdrive.htm (Commercial but free)

Server configurations:
It sounds like there are two main new issues to consider when setting up a server for use by the native Vista WebDAV client (if you could call it that):
1. Need to allow WebDAV extensions at the site root for all users–specifically OPTIONS and PROPFIND.
http://forums.techarena.in/showthread.php?p=2661117
http://www.borlik.net/blog/archives/000157.html

2. Might need to rewrite authentication headers to remove the server or host name from the username. Found a link to a Java servlet configuration that did this–lost it now.