Let’s dissect this a bit.

• What is Network Neutrality?
• Why is Google paying less?
• What are the real costs of the Internet?

What is Network Neutrality?

First of all, Network Neutrality is a political issue related to the fundamental ways we’re charged to use the Internet. The EFF, most software and services companies, and most content providers are in favor of Net Neutrality.

People are charged for connecting to the Internet through a series of ISPs, and the ISPs pay telecommunications companies to connect to the wires and fiber optics that make up the Internet. On the Web, there are always two ends to Internet traffic–the web site you’re visiting, and you. The Internet routes all the traffic between you and the web site. You pay for your connection to the Internet, and the web site owner pays a web host or a data center or an ISP to be connected to the Internet. For every web site you visit, two people pay: you, and the web site owner.

Network Neutrality is an attempt to keep this model of paying for the Internet, essentially by putting a toll booth at the onramp and offramp. Proponents of Network Neutrality (which includes me) think this is enough, and do not want to see other revenue streams going to support the basic wires, because it opens up a huge conflict of interest. Charging for the pipe to your house, charging for the bandwidth you use, these are fair systems for paying for the Internet.

Opponents to Network Neutrality include mostly telecom companies like AT&T, Verizon, Sprint, and many others. What they want to change is they want to put surcharges on particular types of traffic. The specific underlying cause for this is VoIP–with the rise of services like Vonage, Skype, and dozens of other VoIP providers have cut into the revenue of the traditional phone companies. So instead of raising fees to ISPs and allowing this traffic to flow unimpeded, they want to preserve their old revenue model, and discriminate against voice traffic that goes over data.

If we looked at this as a highway, with tollbooths at all the on- and off-ramps, the telecom companies would argue that they just want to be able to charge more for semis hauling trailers, so that those activities that take up more space on the road would get charged more than individuals who are just driving a family car. Sounds good, right?

Except that’s not quite what they want to do. To extend our metaphor to fit, we’d need to add a high-speed train that the same companies also run. And then notice that they actually aren’t charging the trucks as much as they’re charging buses or electric cars or any other type of vehicle that might compete with their train service.

Network Neutrality does not prohibit telecom companies from charging whatever they want for traffic through their pipes. All it does is protect them from discriminating arbitrarily against different types of traffic. They could change from their current consumer all-you-can-eat for one monthly rate plan to charge by the GB of traffic, and none of the network neutrality folks would complain. Bigger bandwidth users will pay more for it. That’s neutral. But saying that VoIP, or Bittorrent, or video calling, or instant messaging, or email, should be charged at a different rate than web traffic only serves to protect monopolies and inhibit innovation.

Why is Google paying less?

Quite simply, they pay less per giga-byte of transfer because they’re buying it in huge bulk quantities. Regular home users do not get the best deal per giga-byte of traffic because most buy a flat unlimited plan, and never get close to the capacity they’re actually paying for. Back in the dial-up days, you used to pay more to get more hours online. Now you can be online 24×7, but you’re certainly not using all your bandwidth. That’s why you’re paying a higher rate.

Bandwidth does cost. Popular web sites pay high charges to get enough traffic to their servers, compared to home users. I’m sure Google’s Internet charges are well into the millions of dollars, probably tens of millions. Divided by the giga-byte they’re getting a great deal compared to home users. That is all this study is saying.

What are the real costs of the Internet?

Senator Ted Stevens did get it right in this respect: the Internet is a series of tubes. Think of the water system in any major city: Internet routers are like a pump station. An Internet backbone is like a water main between two pump stations. The copper wires or fiber-optic strands are quite like pipes, although data passes in both directions. As the pipes (the copper) comes out to your house, they get smaller and smaller, capable of carrying less data. Instead of water flowing over the pipes, it’s data, which is really just a tiny amount of precisely regulated pulses of electricity (or light, in the case of fiber). And instead of there being a big reservoir that is the source of the water, the data flows both ways, so all end points can also be the data source.

Creating the Internet took a huge amount of capital investment, to lay the wires and fiber across the country. The telecom companies largely built this system. But once it’s all built, there’s very little cost to operate it–just a relatively small amount of electricity and a bit of maintenance when things break. Extending the Internet to new communities, laying faster fiber to existing places, improving the technology along the way, these are the places your dollars go. That, and into the pockets of the companies that made the initial investment (mostly by piggybacking on the copper and profits of the previous phone system). These companies obviously should be rewarded for making the investment, but now that the infrastructure is largely in place, paying them more doesn’t necessarily result in further benefit to us.

The Internet should be managed like the rest of our infrastructure: our highways, our electrical systems, our water systems, etc. They should be managed like a utility, and not favor some interests over others. If their old business model of landlines is no longer viable, they shouldn’t be allowed to crush more innovative competitors by limiting access to the wires we all need to share.

One of my clients has suggested that phone companies should be split in two, with the part responsible for the actual infrastructure treated like a public utility, granted a monopoly on that infrastructure but not allowed to compete with services that depend on fair access to that infrastructure. It’s the very principle behind the idea of anti-trust, and I think it’s a great approach. What do you think?

So there I was, walking the dogs home from work. I crossed the Fremont bridge, then down the block to the one major intersection I need to cross, just in time for the walk sign.

But there are cars stopped right in the middle of the crosswalk, underneath the red light. And not just the crosswalk–there were fully 3 cars stopped on the intersection side of the the crosswalk, blocking cross traffic.

The guy parked squarely across the crosswalk shrugged and looked pretty embarrassed as our pack went around behind him. And then I had to walk up between the cars back to the crosswalk, along side some asswipe who had her head totally inside the car doing something or other that apparently was more important than driving or obeying traffic rules.

I rapped on her window as I walked by, and she let out a terrified shriek. Then she started yelliing at me. I was listening to the radio, and really didn’t care what she had to say, so I hollered “you’re in the middle of the intersection.” She rolled down her window to yell something more at me, so I hollered back at her, “you’re in the middle of the f**king intersection!” And continued on into the neighborhood.

Now, I’m usually a very calm, even-tempered guy. You never hear me say things like that. But assholes who think they need to make it through the next light at the expense of everybody who happens to be going the other direction really piss me off.

Don’t get me wrong–I’ve been caught unexpectedly in the intersection when the cars ahead didn’t move as much as I thought they would. And when this happens, I’m very embarrassed, and hyper-aware of everything around me, trying to get out of the way of any pedestrians or other cars I’ve unintentionally blocked. This woman received my moment of wrath because she was startled that I was even there, and couldn’t care less that her selfishness is the kind of thing that causes gridlock and endless traffic jams.

So how does that relate to our topics of open source, computers, business, and economics? A couple ways.

First, there’s the obvious parallel to network congestion. If you’re on a DSL connection, incoming and outgoing traffic have to alternate over the same pair of wires. If you flood the entire connection one way, the other way starts to suffer. For example, if you’re uploading a huge file over a network connection, it can greatly decrease your download speed, much more than you might expect. By keeping your traffic in either direction slightly under the maximum rate, the other direction can continue to flow freely. But if you max out one direction, the other direction suddenly can’t get through, just like lame drivers who block the intersection because they’re too impatient to wait for the next light.

The other point is about selfishness. Ayn Rand wrote some quite influential books. I read both Atlas Shrugged and The Fountainhead as a teenager, and they make a very persuasive case that people acting in their own self-interest can help the greater good. This is much of the justification for our current financial system, the whole idea of trickle-down economics that suggest if you do what’s in your interest, you’ll make lots of money and create jobs for others.

But gridlock provides a great counterpoint to this–the selfishness of a couple people who enter the intersection with no where to go can lead to stopping the cross-traffic, which could well be hundreds of other people. The actions of one person trying to get ahead can come at a cost to hundreds of others. The actions and profits of the big businesses that dominate our economy have come at the expense of the rest of us. One relevant economic phrase related to this is the tragedy of the commons.

The past

The assembly line was the key innovation of the auto industry, that famous invention of Henry Ford. By creating stations where each person did one thing, and moving the products past the people, it was possible to crank out thousands and then millions of mostly identical cars. As robotics came along, it was possible to remove people and reduce payroll. As globalization grew, it was possible to find places where the people you did need to employ cost less.

Who did this system benefit? For a while, it benefited auto workers, once the unions were in place to provide basic protections that limited repetitive stress injuries and guaranteed living wages. It definitely benefited owners of the car companies, at least the ones that survived to be bought by one of the big 3. It benefited the shareholders of the big 3, which for a time included in some small part much of the American middle class, along with many retirement funds more recently. And it brought cars and a whole car culture to several generations of Americans.

But there are many, many costs we’ve paid as a society for this. Many American cities had streetcar or rail systems in the 1920s and 1930s that were dismantled with the rise of the automobile. While some argue that streetcars were slow and inefficient compared to cars, many point to GM buying up the streetcar systems and dismantling them. And I have yet to see any accounting for the mass subsidization of the car system we’ve made over the years, the cost of paving the millions of miles of roads and parking lots, the gas distribution networks, or health issues related to air pollution. That’s not to mention the opportunity cost of dedicating all that land to our cars.

The number of people benefiting from the car companies is quickly declining through off-shoring and automation. Worse, for decades, the Big 3 have been buying up competitors and burying innovation. Rather than compete with its more lucrative mainstream cars, GM killed its early electric car program. Big car companies need to do a lot to maintain their monopoly. Big unions have arisen to provide some protection for the labor these companies needed. Big oil is perhaps the biggest benefiter of the auto industry, as it’s been for decades. But the effect of all of this bigness is funneling dollars from the mass middle-class to a tiny group of owners of these companies.

The present

The car companies are saying that if you account for all the parts suppliers and other companies that depend on the big 3 for revenue, they’re responsible for nearly 10% of our economy. But there’s some false logic here. First of all, if the big 3 go under, we’re not going to stop driving. It won’t take down the entire industry–we do love our cars. What it will do is create a big vacuum in the marketplace that will open it up to hundreds of innovative startups, giving them a much bigger opportunity to succeed.

If the big car companies fail, there will be a lot of talented people who know how to make cars looking for work, and many of them probably have some brilliant ideas to make them better. I think we’d see a renaissance in the car industry, with many companies going out of business but a lot more starting up to take their place. The old car industry has long been complacent, relying on heavy marketing to make their outdated business model keep running. And it worked, for a surprisingly long time.

The problem is, the assembly line is obsolete. It appeared during a period when talent was scarce, and raw materials plentiful. So big car companies got big by doing what they could to produce more with less people. Much of our business world has the idea that you make more money by replacing jobs that take skill with jobs that do not. Make the process smart, and make it so you can put a trained monkey (or robot) in a position so you can reduce your cost and make more money.

Unions stand in the way of this, protecting workers’ pay while their jobs become ever less. So these workers, their pensions, and their health-care benefits become a huge cost for the American car companies, compared to foreign companies that do not need to pay for these costs. Yet ironically, by paying decent wages, these companies also create people who have money to buy their products.

Here’s the real reason we’re in a recession and facing even worse: our entire business system that has been built on maximizing profits has worked so effectively in funneling capital to the wealthiest and away from our middle class, that the vast numbers of Americans can no longer afford to make the purchases that keep our economy running. We’re quickly becoming share-croppers in a neo-feudalistic world, slaves to our wages that effectively dwindle against rising health-care costs, mortgages that are 10 times as much as our parents, and fuel costs that have only diminished because nobody can afford to buy anything.

Who needs a bailout?

Now, I’m not suggesting we do nothing. If the big auto companies go out of business, there will be a lot of people out of work, with no safety net to keep them afloat. We pride ourselves on being a place where anybody can start up a business and potentially do really well–but the reality is, this is really hard to do in an environment where your potential customers aren’t buying anything.

I do think we need a bailout. I just think that rescuing the car companies is not who to bail out. We need to soften the landing of the hundreds of thousands of people who may find themselves suddenly out of work, and we need to figure out how to get capital in the hands of new entrepreneurs who can put these people back to work. The car companies have proven they’re not the ones with any sort of vision for the future, so they need to die now.

Right now, the barriers to creating a new business are steep. Very few people understand everything that’s involved in starting up a business, but most understand that the stakes are high if you have no other source of income. Many, if not most, successful businesses are started by people who have saved up a good amount of seed capital, or have spouses with a stable income, or by those who have done it before and have connections. Without money or connections, it’s extremely difficult to start a business.

I think we need a national program to help match engineers with business people and ideas, and provide seed money to get them launched.

There’s a few nascent events in Seattle along these lines: 6-hour startups, startup weekends, and the like put smart people in the same room and let them see if they can develop some sort of web site with business potential. We need this kind of thing expanded to include more than just software people, and also provide some financial backing to see if they can get running.

This is basically the role of venture capital, but the biggest problem with venture funds is that their timelines are too short–they expect a profitable exit event within a few years, and won’t necessarily fund something that isn’t designed to make a huge profit in a short amount of time.

Some of the work of the Small Business Administration and the Small Business Development Centers is admirable in this area–the problem is, not enough people know about these programs, and far too few engage with them. I think a large part of this is fear of losing health-care benefits, retirement plans, and other benefits that large companies provide at a huge advantage to smaller companies.

America has a huge mythos built on top of the entrepreneur, but very few people who actually become entrepreneurs. We would do far better to provide a social safety net for people out trying to start new ventures, making sure they have healthcare and other basic needs covered, along with the tools they need to figure out how to grow and thrive, than to bail out the decadent dinosaurs.

The Future

As bleak as the current financial system and business climate appears to be, there’s a ton of opportunity for real change. But we need to re-align our business value system to make this happen. Here are three specific mental adjustments the business world needs to make.

First of all, we need to recognize that people are our most valuable asset. Where businesses used to do everything possible to make people interchangeable cogs in a vast machine, to succeed in the future they’ll need to use perhaps our biggest untapped resource: our brainpower. To succeed, a business needs to find smart people who haven’t had their sense of initiative crushed by working in large businesses, and give them tools and encouragement to solve real problems. Rather than the cynical marketing to the lowest common denominator and appealing to “Joe Sixpack” as the average American, we need to recognize the genius in our neighbors and ourselves. We can be so much more.

Secondly, we must recognize that we are in a dire position, and need to work together to survive. It’s human nature to adopt an “us versus them” mentality. We’ve always separated people into our friends and our enemies. Nothing unites a people like a shared enemy. For a brief time, 9/11 brought Americans, and much of the world, together in shared suffering. In the large businesses of the world, success is measured by your sales compared to your competition. In the Cold War, nearly the entire planet was divided into the Free World versus the Communist world. But during all of this, we’ve been dumping waste into all the natural systems that support our very lives. We’ve been increasing the temperature of the planet by pumping greenhouse gasses into the atmosphere, causing galloping glaciers in Greenland and drowning cities. We’ve been poisoning our rivers and streams. We’ve been decimating entire fisheries at alarming rates, killing the great sources of our food. We’ve created rivers of garbage floating around in the middle of the oceans. And we’ve drawn down every natural resource available to us as if there was no tomorrow — soil levels, water levels, oil levels, all being consumed to give us an unreasonable standard of living. If we don’t start addressing these problems quickly, we won’t have a tomorrow.

We need to focus our energy on solving these problems, because they threaten our very existence. Our human enemies are mainly just the people who are unlucky enough to be faced with these problems first. It’s time to set aside our differences and get to work creating a place where our grandchildren can live.

Finally, we need to harness all the efficiency and productivity that technology provides, and focus that on solving real problems. Computers can automate many menial tasks. Large businesses have used technology to cut costs. In most cases, “costs” equals payroll–you save money by cutting jobs. But if our talent is our most valuable asset, it makes no sense to get rid of them. Instead, we need to take those assets and put them more directly to work on projects they’re best suited to handle.

The rise of agriculture meant that more people did not have to spend their lives finding sustenance–people who farmed created more food than they consumed, which led to the rise of towns. Industrialization further centralized our economy into big cities, and centralized wealth into the hands of a few. Now the Internet and open source is changing everything, providing a decentralized model that completely levels the playing field–individuals with no prior connections can become as influential as centuries-old institutions. We no longer need to be dependent on big economics, big corporations, big publishing houses–we now have the tools to build small local economies. Innovation is done by individuals. Small companies with smart people can displace the old dinosaurs. We need to teach people to be smart, and encourage innovation–not teach them to be dumb, staying mute in the face of rigid hierarchies of corporations.

Let the dinosaurs die, and let’s get on with solving our real problems.

A colleague who sounds a bit upset at last week’s results twittered a link to a blog post that accuses us of being “sheeple”, and going down the path of socialism, and apparently the author believes this will cause our country to collapse. The tail end of the McCain/Palin campaign hurled the Socialism epithet as well. Other colleagues have been sending cartoons of trick-or-treaters collecting candy on behalf of their friends who are “too lazy” to trick-or-treat.

There must be some fundamental difference in the way people see the world. I hear these things, and several thoughts come to mind: (warning: long, rambling unedited rant ahead)

• We’re facing some awfully large problems. Sub-par health-care system. Failing education system. Crumbling transportation infrastructure built around gas-guzzling cars as the cost of oil goes up. Failure of the free market to regulate itself. It strikes me that these are all problems we need to solve together, not leave to the laissez-faire system we’ve had that have rigged the playing field towards the rich, redistributing our country’s wealth to the very top.
• Why does the word Socialism have such negative connotations for the right wing? Is it just because it was part of the USSR’s name? Most of Europe has socialist programs in place, resulting in much better health coverage and a social safety net to help reduce homelessness.
• The word “lazy” is used as a broad brush to dismiss the efforts of anybody who hasn’t reached some level of success, often by people who are struggling themselves. I’d argue that people at the bottom of the pay scale, working 2 minimum-wage jobs and barely scraping by, are anything but lazy. Opportunity is capricious, and not evenly distributed.
• Open Source provides a proof-of-concept that illustrates that we’re better off working together to solve our problems, rather than keeping our solutions close to our chest and not sharing with our neighbors.

Us versus them

Our country was founded with a common enemy. For many people, it seems that patriotism depends upon having an external enemy. First the Brits. More recently, the Nazis and Japan, and then the Soviet Union. Now, it’s terrorism, along with illegal immigrants.

But Franklin Roosevelt, at his inauguration in the midst of the Great Depression, famously said “the only thing we have to fear is fear itself—nameless, unreasoning, unjustified terror which paralyzes needed efforts to convert retreat into advance.” And indeed, that is exactly what we fear today–terror. We’ve declared war on it, to the detriment of our civil rights and our moral standing in the world.

I think many people have transferred their fear and uncertainty of their own economic circumstances to this external threat of terrorism. But our real enemy, both here and abroad, is poverty and an unwillingness to work together in the face of much larger looming disasters, such as peak oil, climate change, water and energy shortages, overpopulation, mass extinctions. Instead of dealing with these issues, we’re blaming others for our problems.

I fail to see how the problems we face in the world are all because of terrorists, Democrats, or lazy people. I would argue we have terrorists because we have poverty, and hugely imbalanced distribution of wealth. If we don’t deal with those issues, we’ll surely see more terrorism in the future.

The individualist end game

Time is running out on many fronts. The sheer size of our population is putting a strain on the natural systems of our planet on which we’re completely dependent. Yet we squabble on how to divide up the planet, and those who play the dirtiest have long been winning.

In economics, there’s a distinction made between rivalrous and non-rivalrous resources. Rivalrous resources are those that can only be used by one person at any given time. Non-rivalrous resources are generally things that can be used by many at the same time. For example, my backpack is rivalrous–if you take it from me, I don’t have it anymore. A song is non-rivalrous–I can sing “Happy Birthday” without taking that away from you. Or at least, I should be able to…

In our current economic system, we have made more and more things rivalrous, at least in our legal system. We have carved up our land and made the vast majority of it private. We have granted patents on our very genes. We have allowed corporations to claim ownership of drinking water and charge people for access. We have companies whose sole purpose is to own a patent portfolio and make money off the use of ideas, whether or not other people came up with the same idea independently.

If you’re an individualist, you might see these things as capitalism at work, and the way it should be. But this is really a short-sighted view. Taken to the ultimate result of more and more people fighting over less and less, and you end up with situations like Rwanda in the 90s. According to Jared Diamond, one of the biggest factors leading up to the genocide was population-pressure–the population density had reached a level not seen anywhere else in Africa. Sons were fighting over postage-stamp lots of land to have something they could grow food on, as fathers divided the land into ever smaller bits. There were no shared spaces left, no mental buffers, not enough land to support the people. The rate we’re exhausting our planet’s resources, we risk a similar fate.

We’ve been steadily turning our common wealth into private wealth, redistributing wealth to a tiny few, leaving not enough for everybody else. We’ve been spending resources like oil that have taken millions of years to accumulate in a few generations, polluting our seas and soils, shipping our waste anywhere out of sight, and seeing all of this as somebody else’s problems.

Our country was founded on the principles of working together in the face of a common enemy. Ben Franklin is full of relevant quotes: “We must hang together, gentlemen…else, we shall most assuredly hang separately.” The biggest problem is that while these big companies take, take, take from our environment and people all to maximize their own profits, they’re stealing the resources everybody else will need in the long run, and socializing the true costs of them doing business.

Scarcity versus abundance

Ironically, the way out of this mess is to change perspectives and realize we really are blessed with tremendous resources. Among the largest, most under-utilized resource we’ve got are millions of smart people locked up doing menial work as cogs in the machinery of these big corporations. We’ve largely moved slavery out of agriculture into minimum-wage jobs across the country, and lower-wage jobs abroad. The term “wage slave” is not far off the mark. Prices of everything has been spiraling up while wages have flat-lined. In the past generation, the cost of buying a house has gone up by a factor of ten, while wages have doubled. Yet more people have bought houses than ever before, and we now see that it was mostly on borrowed money they couldn’t possibly pay back.

You hear the right wing complaining about big government, and how liberal policies reward people for not working. Our policies say a lot about what we value, and just about every decision congress makes benefits some people while penalizing others. We need to get a lot more strategic about our policy-making, and align those policies to address the big problems we’re facing–a health care crisis, global warming, deforestation, peak oil, the financial crisis, and more. If our most abundant under-utilized resource is brainpower, let’s make some policy changes that rewards using those resources, while penalizing the use of the natural resources we’re quickly exhausting.

We need to find ways of penalizing companies that consume natural resources and burden our systems with waste. We need to find ways of rewarding smart, responsible people and companies who help reduce waste in the system.

So how do we make this happen?

Let’s go back to the basics. What does everybody need? Food and shelter, obviously. Today, health-care needs to be a right–we’re a wealthy enough country that it’s absurd people have to choose between food and medication they need to treat and prevent a health issue. Beyond that, people need opportunity, and motivation. For far too long, our policy has been to motivate through fear. Make people afraid they won’t have enough food or have a place to live, and they’ll do drudgery work to avoid that. But it’s hard to give creativity free reign when you’re trying to scrape together the basic things you need to survive. Too many of our people are in exactly that position. And we need their help solving the bigger problems.

In fact, some of our biggest problems are caused by exactly this issue–people not having adequate food or shelter. Terrorists come from places where the future is bleak, where people aren’t secure that they’ll have a safe place to live and enough to eat, let alone make any positive contributions. But there’s always an opportunity to be destructive if your situation is dire.

Now, as much as I’ve been railing against big business, I do think business is the answer. We need to re-align our policies to favor small businesses, and lots of them, to power our way through this mess. Small businesses need to be part of their communities–they can’t just take their jobs to the lowest bidder. Small businesses are much more entrenched in location, and able to make contributions to help their communities. Small businesses provide jobs, solve multiple problems, and are able to think about things beyond merely satisfying their shareholders’ greed–they are able to optimize their business to fit in their part of the economic ecosystem, rather than doing whatever it takes to maximize profits. And small businesses succeed by recognizing that others need to succeed too.

The current barriers to people creating small entrepreneurial businesses that solve real problems are many:

• A workforce trained to be wage slaves, rather than thinking, and making decisions, for themselves.
• Crushing costs of doing business: payroll taxes, health care, business operational overhead that is a high cost of doing business.
• Lack of knowledge/experience in creating a business. Very few people who start a business have done it before, and until you’ve started a business, it’s hard to understand the magnitude of the task.
• Money. It takes money to get started. It takes money to hire people, provide training, lease space, buy equipment, get insurance, handle bookkeeping and taxes, and obtain licenses, let alone buy the raw materials that become your product.
• Time management. If you’re a service business, you need to have enough billable hours to cover your costs and make a profit. If you’re a product business, you need to create or manage your products. You need to devote some of your time to marketing and sales before you make any money at all. You need to divide your time effectively between doing the stuff you get paid for, letting the world know you exist, developing the key relationships with partners and vendors and customers, and handling all the little stuff that needs to get done to keep the business running.

The bottom line is, nobody can do all of this alone. To succeed in business, you need help, and lots of it. You need customers who will spread the good word. You need vendors to help with all the operational aspects of your business. You need employees to do the actual work of your business. You need competitors to prove to the marketplace that your product or service is valuable.

So that’s a very long-winded way to get to the point that the vast majority of businesses do not have a monopoly on their product, and do not need “Intellectual Property” (IP) protection to be successful.

Venture Capitalists are always looking for IP: patents, copyrights, trade secrets, or some element that gives a company an advantage over every other potential competitor in the market. They need this because the venture system is based on home runs–they expect that out of 10 businesses, 7 will fail, 2 may break even, and 1 will be successful enough to cover the losses of the other businesses. They don’t know which business will be that home run, so they’re going to invest in the businesses they think stand the best chance.

I’ve spoken with hundreds of people who think their idea is unique, and they don’t want to tell you about it because you might steal it and make a ton of money–or at least be a competitor. The problem is, there are lots of good ideas, and many people come up with very similar ideas independently–the real shortage is the talent to execute those ideas, turn them into a viable business.

One of my biggest challenges as an early open source solution provider was having my potential customers take my solutions seriously. After all, if the software is free, how can it be any good? And if it avoids vendor lock-in, why am I the only one proposing it while they’ve got a dozen vendors pitching Microsoft or Intuit solutions?

While the idea behind my business is wide open, and there’s no prohibitive barrier to competitors setting up shop and doing exactly the same thing, our business is succeeding on execution, on the talent of my employees. It doesn’t matter what your idea is–what matters is whether you can bring it to market.

Systems thinking

To wrap this long meandering post up, we can no longer afford to run businesses that maximize a single factor (profit) at the expense of everything else (people, environment, waste, the future). We need to start optimizing for all critical factors. Our products and businesses need to account for the full impact of what we’re doing, and if it doesn’t make the world a better place in some small way, it shouldn’t continue to exist.

Businesses don’t exist in a vacuum. Businesses can serve a highly constructive role in our society, and help address all of the major challenges we face. But we need to adopt the Unix architecture, create them as small pieces loosely joined, not big monolithic monsters that crush everything in their path.

Our Tax System, Explained in Beer

Suppose that every day, ten men go out for beer and the bill for all ten comes to $100. If they paid their bill the way we pay our taxes, it would go something like this:

The first four men (the poorest) would pay nothing.
The fifth would pay $1.
The sixth would pay $3.
The seventh would pay $7.
The eighth would pay $12.
The ninth would pay $18.
The tenth man (the richest) would pay $59.
So, that’s what they decided to do.

The ten men drank in the bar every day and seemed quite happy with the arrangement, until on day, the owner threw them a curve. “Since you are all such good customers,” he said, “I’m going to reduce the cost of your daily bee r by $20.” Drinks for the ten now cost just $80.

The group still wanted to pay their bill the way we pay our taxes so the first four men were unaffected. They would still drink for free. But what about the other six men – the paying customers? How could they divide the $20 windfall so that everyone would get his ‘fair share’?
They realized that $20 divided by six is $3.33. But if they subtracted that from everybody’s share, then the fifth man and the sixth man would each end up being paid to drink his beer.
So, the bar owner suggested that it would be fair to reduce each man’s bill by roughly the same percent, and he proceeded to work out the amounts each should pay.

And so:
The fifth man, like the first four, now paid nothing (100% savings).
The sixth now paid $2 instead of $3 (33%savings).
The seventh now pay $5 instead of $7 (28%savings).
The eighth now paid $9 instead of $12 (25% savings).
The n inth now paid $14 instead of $18 (22% savings).
The tenth now paid $49 instead of $59 (16% savings).

Each of the six was better off than before. And the first four continued to drink for free. But once outside the restaurant, the men began to compare their savings.

“I only got a dollar out of the $20,”declared the sixth man. He pointed to the tenth man,” but he got $10!”

“Yeah, that’s right,” exclaimed the fifth man. “I only saved a dollar, too. It’s unfair that he got TEN times more than I!”

“That’s true!!” shouted the seventh man. “Why should he get $10 back when I got only two? The wealthy get all the breaks!”

“Wait a minute,” yelled the first four men in unison. “We didn’t get anything at all. The system exploits the poor!”

The nine men surrounded the tenth and beat him up.

The next night the tenth man didn’t show up for drinks, so the nine sat down and had beers without him. But when it came time to pay the bill, they discovered something important. They didn’t have enough money between all of them for even half of the bill!

And that is how our tax system works. The people who pay the highest taxes get the most benefit from a tax reduction. Tax them too much, attack them for being wealthy, and they just may not show up anymore. In fact, they might start drinking overseas where the atmosphere is somewhat friendlier.

Ok. There are several problems with this.

The first problem, when you try to apply it to the national tax system, is that the bartender isn’t giving anybody a break. Instead, he’s increasing his rates. And we’re all demanding tax relief in the face of increased spending! So who’s paying for the extra $700Bn bailout, or the $600Bn+ we’ve spent in Iraq? If our government were indeed shrinking, this might be an apt metaphor–but it’s not. It’s growing. The question isn’t “how do we split the savings fairly,” it’s “who should pay for the increased cost?”

Secondly, where would the rich guy go drink? Of the developed countries, we have some of the most advantageous arrangements out there for the wealthy. Much of Europe has a more socialist system, and tax higher incomes much more than we do. China? You run the risk of being nationalized. I guess India might be a good place, if you really wanted to be a cheapskate and keep your relative wealth intact. Or Dubai.

A counterpoint

What we choose to tax says a lot about who we are and what we value. Our income tax system is progressive, because it taxes higher incomes at a higher rate, and below a certain level, there’s no tax at all. A Sales tax isn’t particularly progressive or regressive–it just taxes consumption, which in this age of waste probably isn’t a bad thing. A lottery is regressive, since lower income people tend to see them as their only path to success.

But there are other taxes that have a much bigger impact on our society than these. Payroll taxes are regressive, since they tax up to a certain level, and then no more. Business and Occupation taxes (which we have here in Washington) penalize small businesses that do not encompass their entire supply chain under a single corporate entity. Capital Gains taxes tax profits realized from changes in the value of something over the time you’ve owned it.

I’d argue that Washington State has a strong economy in spite of its tax policy, not because of it. Our strong economy is due to our abundance of natural resources such as hydro-power, agriculture, and natural beauty that makes it a place creative and entrepreneurial people want to live.

So what would I do? I’d look at the larger system, and the larger problems we face. We have a lot of unemployed/underemployed people. It’s very expensive to hire people, because of payroll taxes and the heavy burden of health insurance. Instead of taxing those things, we should eliminate the payroll tax and make it easier for businesses to hire good people. And provide health care to lower the burden on our businesses, and good education so that we have talent worthy of hiring. The current system is stacked in favor of large corporations who have the margins to support funding these things–but these same corporations have shown no sense of responsibility in doing so, and would sooner chase lower wages anywhere else in the world to improve their bottom line.

We also are facing major climate issues, national insecurity due to dependence on foreign oil, and congested traffic. All of these issues would be helped with a strong consumption tax on oil. We have a gas tax, but it’s nowhere near enough to encourage people to get out of their cars. Forget cap and trade policies–we need a carbon tax.

So I’d probably keep the current sales and income taxes, but change the tax policy away from penalizing those who provide good jobs here, and towards those who consume, exploit, and waste our natural resources.

I’ve got a list of niggling things that have been bothering me about the current Hardy Heron release. Since the biggest of these issues are related to hooking up a projector or external monitor, and I’m giving a guest lecture Wednesday evening, I decided to test-drive the Intrepid release candidate to see if they’ve resolved these issues.

It turns out, the answer is yes.

I’ve been running it for a few hours now, and I have to say, I’m finding a lot of nice little improvements. If you’re new to Linux, please forgive the technical nature of a few of these notes… just some random unedited thoughts. Here are some things I’m really liking right now:

• External monitors. Finally, after being promised for the past three releases, I can add an external monitor and configure it through a GUI, without needing to edit a configuration file. I can extend the desktop across both monitors, or clone the screen. For the past year, I haven’t been able to do this at all without the video display locking up. Now it’s working great. Fantastic!
• Broadband card support. My USB Verizon broadband card works in Network Manager. All I have to do is tell it to connect, and it does–no configuration necessary. Previously I was using gnome-ppp, which took a lot longer to make the connection, and took some configuration.
• Appointments show up in the panel with the right colors. I’ve used Evolution for appointments for years. I have a bunch of different calendars loaded–different ones for work and social events, and the calendars of my employees. In Evolution, I set each calendar to a color so I can easily see which calendar an event is on. However, until Intrepid, these colors did not get used in the Gnome display–that used some random set of colors. Now they match. Small, but much appreciated improvement.
• Suspend/Resume is much quicker. While suspending the machine has worked pretty well for quite a while, under Hardy there was a lot of load that kept you from getting to work right away. If you had a few applications running, it could take 10 minutes before it was usable again! Under Intrepid, you can start using the applications immediately after starting up. I see that the system is under high load for a similar period of time, but the user interface is no longer sluggish at all, and the load seems to drop much quicker.
• Avant Window Navigator plugins work. I’m hooked on this little launcher utility, but the one available in the Hardy repositories didn’t work with any plugins. With Intrepid, they’re all there and work great.
• Firefox with Flash doesn’t crash so much. Okay, this isn’t anything to do with Intrepid, so much as tracking down the nspluginwrapper package, which allows Flash to crash without taking the browser down with it. I found this based on a how-to on getting Flash sound to work.

Which brings me to the problems. I’ve hit two pretty substantial problems. Both of them are more of a nuisance than any sort of show-stopper, but they are the kind of nuisance problems that might keep some people in Windows:

1. Sound did not work correctly out of the gate. I first noticed this in a flash video. Here it is the second release with Pulse Audio, and it still doesn’t work correctly without some manual configuration. To get it working, I followed a how-to on the Ubuntu forums, basically installing libao, padevchooser, and some other libraries, removing previous config files for alsa and pulse from my profile, and making libao use pulse by default. This only took a couple minutes, but for somebody unfamiliar with Linux, this might be a big barrier.
2. Bluetooth. I have a bluetooth mouse, and hooking it up was a piece of cake. However, it doesn’t remember the connection. I have to delete the bluetooth profile, and re-pair it every time I shut off the mouse or suspend the computer. I’m sure there’s a pretty simple fix by editing a couple configs, but the point is, it should just remember that I’ve paired this device and not bother me again.

Those are the only two new issues I’ve seen appear in this release, so far. And I really only see one other major issue: I’m still seeing memory usage of Xorg creeping up as I use the system, especially after a suspend/resume cycle. It’s currently up to 835M, which seems like an obscene amount of RAM for the graphical environment. I saw this same type of memory leak under Hardy, under similar conditions, but to Intrepid’s credit, the system seems completely responsive and speedy. So it looks like I’m going to continue needing to log out and back in every couple days to free up the memory consumed by X.

Overall, I’m quite impressed, and not seeing any downsides to this release compared to Hardy, which was already pretty great.

That said, I’m as interested in getting a smart phone device as anyone. I carried a Palm V for years, until it would no longer hold a charge. I’d like to have instant access to my schedule, address book, and time management tools–which are now all online. I miss carrying a tide prediction tool in my pocket to know when there’s the right amount of sand to swim our Labs at the local beach.

The mobile landscape is a mess. There’s tons of one-off devices, each locked down with a small set of software, and most offering few options for adding new applications beyond what’s available through tightly controlled vendor channels. For its entire history, the mobile industry has done its best to control everything, charge for everything, and lock in their customers. But cracks are starting to appear in the armor.

Stephen J. Vaughan-Nickels misses the point in a recent article about the new Google Android phones coming out next month, The Android phone is here! So what?. He writes:

Now, what does Android have to offer that’s different [than the iPhone]? Well, it’s an open platform and open source so it will be easier for developers to write program for it. But will they?

After all, it all comes down to how many people will actually buy Android-powered phones. If you don’t have enough users, it’s not worth a developer’s time to make applications for them.

The point is that Android may be the first platform ready for prime-time, ready for general non-technical users, that is not locked to a single vendor on either the hardware or telecom level.

You can argue that the Apple iPhone has created a revolutionary platform, and with all the 3rd party applications now available through its store, that anything else is just a Johnny-come-lately to the party. But the Apple iPhone store is successful only because there’s no competitor–yet–that is ready for prime time. That changes with the release of Android, and again if/when the OpenMoko project gets its legs, and again if they add cell phone capabilities to the Maemo platform.

The problem with the iPhone is that it hasn’t changed a thing about vendor lock-in. There’s only one manufacturer for the iPhones, Apple. And there’s only one carrier (in each country) for the service–AT&T in the US. While people have hacked the iPhone to use it with other carriers, the new iPhone 3G makes you pay for AT&T anyway. While the iPhone may have provided better access for developers to create new applications than they’ve had in the past, that’s not saying much. And they haven’t done a thing to break down the vendor lock-in issues. And developers are chafing at Apple’s control over its iPhone store, with many useful applications getting rejected because they compete in some way with Apple’s own software, especially if they do a better job. This cartoon expresses the sentiments of many iPhone developers.

To paraphrase Jim Zemlin, Apple is like a 5-star hotel you can never leave.

Most interesting mobile platforms
So what are the alternatives? Skipping the one-off devices like the LG Dare, the T-Mobile Sidekick, the obsolete Palm Treos and the reviled, buggy, Pocket PC phones, there’s several platforms to consider, either now or on the horizon:

• Blackberry. Blackberries are still doing quite well, and have quite a bit of power. While they are offered through many carriers, there is still only one manufacturer. I don’t know that much about their technology, but if you’re writing an application you need to consider them. I do think they’re a solid choice for consumers looking for a do-it-all mobile device.
• Apple iPhone. In spite of its anti-developer stance and its extreme vendor lock-in, the iPhone is quite cool and very usable. I won’t buy one because of these reasons–I have no interest in supporting the monopolistic practices of Apple–but I can understand why people do, and if you’re developing an application, having a good story for the iPhone may get you a bunch of technophile users
• Google Android. While the name on this package is Google, what’s new about this effort is the number of vendors participating in the effort. Google created a consortium of a few dozen manufacturers and carriers to create a common standard they could all use. Since Google is behind it, it’s particularly well integrated into Google applications–Gmail, Google Calendar, Google Maps, etc. While not all of the stack is open, much more of it is open than the iPhone. Developers will be able to release their own applications. Users can install any application they want, without having to go through a single central store. And while the first device is coming out from T-Mobile, you can get an unlocked phone, and in a very short time, have choices of hardware and carriers all supporting the same platform. Much like the IBM PC overtook the Apple Mac in the 80s by democratizing manufacturer access to the design, I think the iPhone will quickly get eclipsed by a more open standard.
• OpenMoko. I’m an early follower of OpenMoko, watching them for nearly 2 years now. I purchased the first public release the day it was available. It has all the promise of Google, and more, because the entire stack is open. The problem is, they didn’t get a solid working stack created before shipping. So for the past 14 months I’ve had a pretty slick GPS that could sometimes make phone calls but never last longer than 4 hours without a charge. I think they’ll eventually eclipse the iPhone, too, though Android will be tougher competition with all of that industry backing. The OpenMoko folks have been making great strides lately, and I’m still hoping my device will be usable as an everyday phone before next year. If they can create a usable enough device for mainstream users, it will become a contender.
• Maemo. Maemo is another Linux-based platform, designed for Nokia Internet tablets and available for their N770/N800/N810 devices. These devices have Wi-Fi, a touch screen, and depending on which, GPS, FM radio, slide-out keyboard, and other niceties. They ship with Skype and standard VoIP clients, you can do video calls on them, and there’s a huge developer community making great applications for them. What they don’t have is a cell phone connection, though there are rumors they may release one soon.

So what should I buy?
Those are the exciting platforms on the mobile landscape these days. What do I recommend? Depends a bit on what your needs are, but for people who are looking for a cell phone with PDA functionality (appointments, calendar, tasks), mapping, web browsing, and email, here’s what I’d suggest today:

• Get a cheap phone and a Nokia N810, and carry two devices. Do you really need your cell phone to do everything? The Internet tablets are small, slick, powerful, and usable, and work great today.

In a month, when the first Android phones hit the market, that might be a great alternative, and slightly cheaper, though the first one won’t have a GPS and is missing some other features.

What about developing applications?
By far the most cost effective strategy for making an application to reach the most people, is to simply make it a web application. 4 or the 5 platforms above (all but Blackberry) ship with a web browser. In fact, those 4 all can use the same browser engine, WebKit, which also powers Apple’s Safari browser and the new Google Chrome.

Make a web application, and you don’t have to choose which device to support, you can support all of them.

What have I missed?
I’m no expert in the mobile space. These are just my observations, based on reading and hearing reviews of various devices and helping clients connect to our email services with them. If I’m overlooking some major area, or you disagree with something I’ve written, please add a comment!

Although the Internet is global, and you may do business with people anywhere in the world, most people tend to look for people-networks close to home. Or do they? Should they? If the point of social networking is to connect with other people, ought it to matter where we are?

At Freelock, we have a handful of remote clients, but upwards of 90% of our clients are local. I founded the business on the assumption that people want to know who they’re doing business with, be able to see them face to face, and grow to trust them over time. Nothing breaks the ice like talking about a project in person, over a coffee or better yet, a margarita.

Good or bad, business gets done through personal relationships. How many deals have been cemented on the golf course? It’s a lot harder to say no in person, than it is with a quick dismissive email. So much communication happens non-verbally, through body language, tone of voice, and other channels that just aren’t available online. A video conference is a poor substitute for a face-to-face meeting.

I’ve gotten quite a bit of help from IRC. We have a private company jabber server so when we’re not in the same room, we can still have the feel of a team. We’ve had people helping us out from Bellingham, 120 miles north. The Internet enables some amazing things, and I definitely think it’s possible to work effectively at a distance. Many professions, including writing and coding, can be done quite effectively by individuals working by themselves, anywhere in the world.

But you can’t directly diagnose a connectivity issue in an office in Bellevue when you’re in India, or replace a hard drive. You can’t assemble a car from the other side of the world. And even for creative types who can work effectively on their own, relationships and trust only truly get cemented by meeting their editors, testers, or project managers in person.

Another founding principle of my business is that it’s much easier to ensure quality by having people work in person. If team members can do impromptu code reviews of each other’s work, quality goes up. The solitary developer working late at night may bang his head for hours against a problem that a colleague could solve in a 5 minute conversation. Having a team of people with complementary talents and different strengths working in one place leads to better results.

Once you’ve established that level of trust, remote work becomes more effective. You know when somebody’s cracking a joke, and it doesn’t sound so strange. You’re more likely to ask a quick question in a chat when you can preface it with a comment about an outside shared interest.

Yes, location matters. It’s not everything, and the Internet makes it possible to work together from a distance–but it still matters.

While they’re fresh, let me jot them down here. WARNING: Extremely technical content ahead.

First of all, SOAP is supposed to stand for “Simple Object Access Protocol.” It’s anything but simple. There is a lot of SOAP software out there, but subtle implementation gotchas that can be quite difficult to figure out.

We chose the native PHP SoapServer in PHP 5.2 to implement the project, mainly because we’re a PHP shop, and a little smoke testing revealed it was quite quick to get set up and going. It turns out that it’s quite hard to debug. For its good points, it can read in a WSDL and automatically map methods to methods on a class, and it converts arrays, simple objects, or complex objects to a valid response object, and request objects into simple or complex objects on the incoming side.

Problems with PHP’s SOAP Server:

• No validation of incoming or outgoing documents.
• No warnings, exceptions, or errors if it can’t convert a document to fit the schema–it just dies.
• No debugging information about what it’s doing.
• No ability to manage namespaces, especially if they need to be copied from the SOAP envelope into the payload.
• Difficult to test.
• No access to the raw XML of either the request or the response.

Using PHP’s SoapServer is quite simple, except when things aren’t perfect…

Here’s what the code looks like for the simple case:


< ?php
$xml = $GLOBALS['HTTP_RAW_POST_DATA']; // or file_get_contents('php://input');
// make sure you have something to process, throw an error if $xml is empty
$soap = new SoapServer('http://path/to/your.wsdl');
$soap->setClass('mySoapClass');
$soap->handle($xml);
?>

That’s basically it. You declare methods on ‘mySoapClass’ that correspond to the SOAP methods. These handler methods receive a simple object as a parameter, and you can do whatever you need to do with that data. Then it needs to return some data structure that can be serialized to the expected type defined in the WSDL. The return data structure can be an array, a simple object, or an object of a class you define that can serialize appropriately.

Great. With this much, it took me about a day to have a working web service with 8 methods and a bunch of complex data objects. The problems started when people connected with different SOAP software.

The web service I was implementing defines a specific SOAP Fault document, so if I did run across a problem, I could simply throw an exception of that type. My wrapper object kindly passed the custom fields defined in the WSDL.

Problem #1: Validation

As I said before, there is none. If the SoapServer gets anything it doesn’t like, it doesn’t send any response at all. And since all you get inside your method handlers is an already-converted object, you don’t have any way to validate the response without using a global variable or a call to a singleton.

In our case, the project specified that we strip out the payload from the SOAP header and store the payload XML on the disk as a document for several methods, and do processing on other methods. Processing a SOAP request was not a problem. Storing a valid XML document was. Several methods just stored the XML on the disk, with another method retrieving it and returning it to the caller. The problem we had was that the Soap Response was more picky than the Soap Request–so documents that we loaded from the disk and returned as the response would fail with no explanation.

Our solution was to load the raw XML into a DOM Document, and validate it against the schema. This mostly worked, until we had to deal with a request generated from Jitterbit. More on that later.

The question is, what to validate? We weren’t supposed to store the entire SOAP envelope–just the payload. So how to extract it? The simple way was to grab the first child of the Body element, append it to the DOMDocument itself, remove the original root, and call the normalize() method. This did generate a warning, but not a fatal error, and did the right thing. Furthermore, we could also access the raw libxml validation specifics, by calling libxml_use_internal_errors(true), and then when a document fails to validate, using libxml_get_errors() and libxml_get_error() to grab the details.

Problem #2: Returning valid XML responses

The root of all of our problems in this project has to do with where namespaces are defined. One limitation of libxml appears to be that you can only point it to one schema for validation at a time. We can validate against the SOAP Schema, or our custom schema, but not both at the same time, unless one includes the other. So our validation options consist of:

1. Include the SOAP Schema in the custom schema, and validate the entire SOAP body, or
2. Extract the payload from the SOAP body, and validate only that against our schema.

#2 is clearly the correct way–we really don’t care about the SOAP envelope once we have the message. But the problem is, many SOAP clients put the namespace declarations on the SOAP Envelope, and not the payload root element. In fact, the XML generated by the PHP SoapServer class does this itself.

So our first task was to generate the proper XML Namespace declarations on our generated payloads.

To do this, we could no longer rely on the PHP SoapServer’s automatic conversion of simple objects or arrays to XML–we had to generate our own XML, and tell the SOAP server to use that instead. This turned out to be difficult to track down, so here’s the answer:


< ?php
class dataClass{
/* Serialize XML as desired here.
Omit the XML declaration, start with the root element
You can also simply build the XML as a string from object properties
*/
function toXml(){
$this->myDom->documentElement->setAttribute('xmlns','http://my.custom.namespace/version_1');
$xml = $this->myDom->saveXML();
preg_match('%< \?.*=\?>(.*)$%s',$xml,$match); //strip XML declaration
return $match[1];
}
}

// snip to end of actual SOAP handler method:
$out = new SOAPVar($data->toXML(),XSD_ANYXML);
return $out;
}

The SOAPVar object allows you to control the output of the SoapServer a bit better, with support for namespaces, raw XML, or casting objects in a certain way.

Problem #3: Jitterbit

Validating a clean payload without namespace prefixes seems to work fine. Our technique of moving the nodes around with the DOM seemed to keep most of the namespaces intact, so even if there was no xmlns declaration on the payload root, we could still validate just the payload effectively, and serialize/de-serialize without issue.

Except for Jitterbit.

Now, I loaded up Jitterbit yesterday, and it didn’t do this–this might be a problem with an earlier version. The problem is, Jitterbit is extremely verbose, specifying not just a namespace prefix on every element, but also an xsi:type. And even that’s not enough to break it–except that the value for its xsi:type also contained a namespace declaration. And if this namespace declaration was not the root namespace, suddenly our validation broke.

It broke for us on types declared as ns:token, ns:string, ns:integer — the simple types specified by XSD itself, which Jitterbit put into a namespace prefixed with ns: and declared on the SOAP Envelope.

For example, here’s the start of a problem document:


< ?xml version="1.0" encoding="UTF-8" standalone="no" ?>
xmlns:ns="http://www.w3.org/2001/XMLSchema"
xmlns:oi="http://ws.outdoorindustry.org/v1_2/"
xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
xmlns:ws="http://ws.outdoorindustry.org/v1_2/ws/"
xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>

xmlns:tns1="http://ws.outdoorindustry.org/v1_2/"
xsi:type="tns1:PO">
06AB4EF9-6AF157E7-3F8441AB-AB499933
Preseason

999
Sample Vendor

The first validation error was on the VendorID, with xsi:type=”ns:token”. If I copied xmlns:ns=”http://www.w3.org/2001/XMLSchema” into the tns1:submitPO element, it validated fine. The PHP DOMDocument seems to be able to keep track of namespaces on elements and attribute names even after the envelope is gone. But not attribute values.

After hours of banging on this, we came up with 3 workarounds for this:

1. Completely regenerate the XML, after processing. To do this, we would need to create a custom data class for each incoming object, provide a classmap to the SoapServer, and then generate brand new XML out of the data object. This is perhaps the best approach, but I didn’t think of it until the project was over–I was thinking about writing out the data to the database and then loading our custom objects and serializing them as we do for our responses. The biggest drawback here is that we need to model the entire complexity of the request, as allowed in the schema. And this was a really complex object… lots of work to implement, when we’re only going to store this XML for passing to other systems.
2. Hack the XML to get the offending namespace into the stored document. This turned out to be easy to program, but uses lots of CPU resources–DOMDocuments are expensive to use. It’s also the most brittle approach, only catching this single case–if the namespace prefix changes, or a different required namespace is necessary, it’ll break. To do this, we created a new DOM Document, imported the root node of the payload, appended it to the document, and used setAttribute to set an “xmlns:ns” attribute on the root. This did not actually get the namespace recognized for validation, and normalizeDocument did not fix it–but creating a third DOMDocument, and doing $doc->loadXML($doc2->saveXML) did make the namespace recognized by the object so we could successfully validate.
3. Hack the XSD to validate the entire SOAP request. By including the SOAP schema in our custom schema (using xs:import), we could validate the raw SOAP request, then extract the payload and save it. The saved XML does not validate on its own, but we know it validates. So we can remove our validation check on the outgoing document, and as long as the other system does not explicitly validate the standalone XML, we’re okay.

Whew. Hope this helps somebody…

First, McCain has released his technology platform, which among other bits states his support for protecting the recording industry from piracy. Meanwhile, the Ohio Republican party used Jackson Browne’s “Running on Empty” in a commercial without permission.

John McCain 2008 – John McCain for President:

John McCain Will Protect The Creative Industries From Piracy. The entertainment industry is both a vital sector of the domestic economy and among the largest U.S. exporters. While the Internet has provided tremendous opportunity for the creators of copyrighted works, including music and movies, to distribute their works around the world at low cost, it has also given rise to a global epidemic of piracy. John McCain supports efforts to crack down on piracy, both on the Internet and off.

Wired Magazine – First Paris, Now This: Jackson Browne Sues John McCain

Indeed, some irony lies in a candidate who is running a law-and-order campaign being sued for intentional copyright infringing and appropriating someone else’s identity without their permission.

Apparently Browne is a well-known Obama supporter.