Open Source and Open Standards

Jul 03 2009

Open standards are motherhood and apple pie – they ensure a level playing field in which many implementations can compete against each other, keep the barrier to participation low for newcomers, will outlive any given company, and ensure that systems can communicate with each other with a minimum of fuss. In other words, open standards create efficient and durable markets.

Open standards also keep costs low for buyers, who have many options and a minimum of friction when they want to switch from one implementation to another. Because the standard is open, there is no danger of being locked into a single vendor since anyone can create a new implementation against the standard. Since open standards will always exist, there’s no danger of the standard disappearing, becoming unsupported, or being later made proprietary. An open standard will encourage these efficient, durable markets for as long as the standard is useful.

Too often, plain-vanilla standards are conflated with open standards. It’s a shame, because it’s a very important distinction. Anyone can create a standard and collect a large ecosystem around it. Microsoft Word’s document format is a great example of this. It is a standard only inasmuch as it is ubiquitous. In fact, it was only documented formally in February 2008, before which anyone who wanted to use the format had to guess at its inner workings or pay a fee to Microsoft. So while the Word document format was a standard, albeit ad hoc, it created no market, discouraged competition, and represented a serious barrier to entry for anyone who wished to create a word processor that would compete with Microsoft Word.

By contrast, the Open Document Format was developed collaboratively and is freely available to anyone who wishes to implement it. The demand for a viable alternative to the Microsoft Office document formats had found an outlet, and there are now dozens of applications which support the format, including Open Office, Google Docs, and even Microsoft Word itself.

So just because something is a standard, that doesn’t mean it’s open.

What’s Different About Open Standards

In order for a standard to be open, it has some very specific characteristics. Not surprisingly, there are many opinions about what those characteristics are, and Wikipedia has an excellent collection of these. Though there are dozens of definitions available, there is broad agreement that users of an open standard must have a few specific characteristics, which I’ve described as freedoms:

  • The freedom to read and implement the standard at no cost.
  • The freedom from patent encumberances, either through royalty-free licenses or a promise of non-assertion by the patent holder.
  • The freedom to examine and participate in the development of the standard, through a consensus or majority decision-making process.

There are many other characteristics described by these various governments and international bodies, many of which are less definitional than they descriptive of a healthy and useful open standard. We will return to those shortly.

You could say that the freedom to obtain and implement a standard at no cost is the defining characteristic of an open standard. There is no fee, no contract, and everyone is using the same set of instructions. This freedom, though, only distinguishes open standards from closed, for-fee standards. Standards which are proprietary but available at no cost, like the Microsoft Office document formats, cannot be called open. This is at the heart of the confusion over open standards – many do not understand that truly open standards provide more freedoms than no-cost standards.

Freedom from patent encumbrances is crucial to making the first freedom practical. A prospective implementer needs to be assured that they won’t subject themselves to rent-seeking to gain compliance.

The ability to participate in the ongoing development and refinement of a standard is the most unique characteristic of open standards. It is how a standard stays open and viable over time. Open standards must have a facility for revision and correction that accommodates its community of implementers and users. If there is no facility for this, the standard is simply a mandate from one vendor or industry to others. This introduces the possibility of features that favor one implementation over another. Worse, a sudden and unexpected change in the standard could thrust existing implementations into non-compliance – essentially casting all existing implementations out of the ecosystem, which requires time, money, and effort to re-enter. This uncertainty decreases the likelihood that the standard will be useful and thus widely adopted, both now and over time. An open process removes this uncertainty and significantly improves the usefulness of the standard.

So describing a standard as open means much more than “no charge”. If a standard is meant to be a neutral set of rules by which implementations compete for users, simply being free (though useful) is insufficient. An open standard cannot be controlled by a single party, even if it is available at no cost. Without a transparent governance, the standard is simply free.

The Role of Open Source Software

I’ve framed these characteristics of open standards as freedoms to invite comparison with the four freedoms of open source software:

  • The freedom to run the program, for any purpose (freedom 0).
  • The freedom to study how the program works, and change it to make it do what you wish (freedom 1). Access to the source code is a precondition for this.
  • The freedom to redistribute copies so you can help your neighbor (freedom 2).
  • The freedom to improve the program, and release your improvements (and modified versions in general) to the public, so that the whole community benefits (freedom 3). Access to the source code is a precondition for this.

Open source and open standards are the same process, applied to two different problems: open source is for computer code, and open standards are for data. The details vary, but when we talk about the principles of open standards or the benefits of open source, we’re describing the same collaborative innovation process.

Returning to the list of open standards definitions at Wikipedia, you’ll see that a few of these definitions encourage or mandate multiple implementations of the standard. This is wildly important, but I don’t think it’s definitional for open standards. Multiple implementations ensure that the standard is practical, which is not something to take for granted when complex technical standards are developed by committee. Having multiple implementations also means choice for the user, which is central to the value of a standard. So multiple implementations are great.

Now imagine if at least one implementation of this open standard is done with open source software. We not only have a freely available standard that anyone may use, but a freely available implementation of that standard that can be examined and re-used by other implementers. An open source implementation is also an “implementation of last resort” should every implementation disappear or go out of business. Those who rely on the standard have not lost their ability to interoperate. An open source implementation inoculates an open standard against the future, and ensures that there is always at least one competitor in the market. This is good for the standard, and good for the users.

Contrariwise, imagine an open standard with only proprietary implementations. This is scenario works, but substantially undermines the stated goals of an open standard: choice, freedom, and durability.

We may have many proprietary options before us, but without an open source implementation, it is significantly more difficult to create an alternative. You’re starting from scratch each time, so the bar is unnecessarily high for newcomers to the market. Likewise, our freedom to change implementations with a minimum of friction is diminished.

Ideally, every implementation would provide some assurance of interoperability that facilitates my ability to freely choose among implementations. This is not always the case, though, and it’s possible for vendors to “embrace and extend” a standard, which is code for creating an implementation that adds vendor-specific hooks which may interfere with my ability to switch to another implementation. If all the implementations are proprietary, they could all be compromised in this way, leaving me with no true implementation of my otherwise open standard.

If I have no open source implementation, I am reliant on the continued success of the proprietary implementations. If a company goes out of business or leaves the market, I may be left with no viable options. Of course, other competitors could appear – I could even write an implementation myself, though that’s often so expensive as to be impractical. This is all unnecessary if there is an open source implementation available. Again, it is the implementation of last resort.

So while I think it’s excessive zeal to say that an open standard must have an open source implementation, it’s wise to examine claims about “openness”. If there is an open standard that lacks a transparent governance, and has no open source implementation, you have to seriously consider the long-term viability of that standard, and its ability to provide a properly functioning market.

One response so far

  • Patrick

    NASA World Wind seems a classic if not perfect example of open standards done with open source, your “implementation of last resort”