Although I am a believer in free markets and the benefits of competition, industry has a responsibility to work together on the foundational layers to build security, quality and reliability from the ground up to advance the professionalism of the field. In essence, the information technology industry must emulate other engineering disciplines, or technological disasters and cybersecurity holes will worsen.

Daconta is uneasy with the number of platforms and methods available to software developers, and sees ever-more options and disruptions in the near future; IPv6 and 64-bit computing seem to trouble him particularly. We’re already balkanized and disorganized, how can we possibly expect to produce reliable and useful software with all this messy innovation happening?

The answer, of course, is control. Lots of it. Specifically, three proposals:

• Licenses for software developers
• A new, reliable, layered software platform developed by the NSF and DARPA
• Treat software like engineering, not art.

Gracious. I barely know where to start. Let’s try to imagine the software development world in five years, with these proposals in place.

Software development is now a licensed activity. Like an architect or a mechanical engineer, you have to pass an exam and perhaps post a bond to practice the discipline. There’s probably a professional association, like the America Bar Association or the American Medical Association, to administer the credentials.

This licensing regime is actually a pretty good idea, because all software has to be developed according to some very specific methods, with plenty of testing and documentation to back it up. So rather than letting any fool with a compiler write software, they’ll have to spend a year or two learning the right way to write code. The process is cumbersome, but every piece of software that gets compiled is perfect. At least, as perfect as we know how.

The licensing and formal methods are only possible, of course, because we have a government-directed platform that we must build on. Anyone who wants to run software in the government must do so on this stack.

It sounds as though Daconta would like a broad mandate for NASA-style code development. You can probably see where I’m going with this. Another way to tell the story might be:

There is now a government-owned platform that every government program is mandated to use, from clouds to mobile phones. Any software built on that platform must submit to rigorous, independent testing before it is deployed. Imagine ISO 9000 and Common Criteria having a baby with teeth. Anyone writing software must be licensed to do so in the United States. As a result, the pool of available programming talent is decimated. The costs of developing software for government rise, naturally.

The pool is further diminshed because developers who want to work with the latest hardware or software no longer work for agencies or contractors — they’ve wandered back to the private sector, where they can enjoy the fruits of the free market. The government software platform quickly begins to show its age, since the only developers on the platform are those that are paid to use it. Platforms that people truly want to use are in the open market, innovating at their leisure, out of the reach of government agencies.

As the government is no longer able to consume most commercially available software, it is now back in the business of writing software itself. More accurately, it is back in the business of hiring system integrators to write that software on its behalf. It’s the 1970s all over again. Budgets explode, and innovation grinds to a halt. It’s all the agencies can do just to tread water. System integrators, of course, are delighted. They’re now commanding outrageous salaries for the few programmers trained and willing to work on this mandated government platform.

Let’s hope there’s a better way.

This doesn’t diminish the problem that Daconta is hinting at, of course. Software reliability is certainly something to worry about. But there is no single solution or set of policy prescriptions that will solve the problem. I don’t think that imposing additional controls on the development of software makes sense, certainly not for all cases. There are already robust certification regimes and methods for software that does very important work: fly an airplane, control a nuclear reactor, and so forth. We don’t need that kind of scrutiny on my game console or desktop.

It’s important to note that these robust certifications only evaluate the software itself, not the people who make it. This is what’s great about software: we can examine the final product before it’s distributed. I don’t really mind if my software is written by a clever 7 year old. If it’s doing the job it’s supposed to, that’s fine with me.

This focus on ends, rather than the means, is something you can’t do with a building or an airplane. With software, we can change our minds with far fewer consequences. We can thoroughly test and scrutinize before it’s in a customer’s hands. When we do find a flaw, it’s easier to patch software than it is a 777 or a skyscraper. We should take advantage of that fact, not try to make software rigid and inflexible just because we know how to manage rigid and inflexible things. Because software has these unique properties, we have the freedom to bring more or less scrutiny to bear, depending on the circumstances. Which is what we’re doing already, through programs like the federally-funded Software Engineering Institute at Carnegie Mellon.

So where Daconta sees mayhem and chaos, I see creativity and innovation. There are many opportunities to improve the reliability of our software, but none of them have to do with the process by which we arrive at a particular piece of code. Projects like David Wheeler’s OpenProofs, for example,  can provide the tools we need to be mathematically sure software is doing what we intended. The Linux Test Project does this for Linux, and it was inspired in no small part by governments’ Common Criteria mandates.

This is, in fact, how the process should work. The government should set the requirements for reliability and assurance, and allow the private sector to innovate its way toward those requirements. If we only create software that we can understand perfectly, we lose the ability to be creative, to innovate, and to take advantage of the collective intelligence and cleverness of millions of software developers. We will never eliminate risk in software, but we can manage that risk, not through more stringent controls, but by encouraging as many smart people as possible to address the problem.

[This was also published at opensource.com and GovFresh]

A few weeks ago, the Obama administration announced a $12 billion investment in community colleges, and $500 million of that is allocated to sponsoring the creation of open courseware. As described by Inside Higher Ed:

The Departments of Defense, Education and Labor will work with one or more community colleges and the Pentagon’s learning network to find ways to award credit “based on achievement rather than class hours,” and to “rigorously evaluate the results,” according to the White House briefing materials on the program.

The material would be made available to anyone, since it’s open, but it will be specifically focused on community colleges. The plan is to create a set of resources for educators that they can then customize for their own classes. Frederick Hess of AEI (“Reinventing the Wheel” at NRO and Inside Higher Ed) has taken issue with this. First, he asserts that this violates the Department of Education’s charter:

Federal law has long buttressed academic freedom and intellectual pluralism by prohibiting the U.S. Department of Education from exercising control over “curriculum, program of instruction … text books, or other educational materials by any educational institution.” The administration would suddenly have the department funding the creation and dissemination of entire courses.

Implementation is important. I’m certainly skeptical of any attempt to Federalize educational content, but I don’t think that’s what’s happening here. If the courseware is open, and properly licensed, it’s not “owned” by the U.S. Department of Education. They may have paid for its creation, but (if properly nurtured) it will have a life of its own, outside the purview of the Department. This is a significant feature of open content, which causes Hess some additional misunderstandings:

The proposal is both short-sighted and destructive. It’s one thing to encourage providers to develop ”open source” wares and to promote measures that encourage publishers, colleges and universities to reduce costs and save students money. But it’s another thing entirely for the federal government to use taxpayer dollars to provide services that will undercut those offered by self-sustaining private enterprises.

Again, I believe he’s missing the point. It’s not at all clear that funding the development of freely available content will crowd out the existing publishers. There is certainly a threat to content publishers if their business model is predicated on controlling access to content. But whether they realize it or not, vendors of textbooks and distance learning programs are selling much more than just content. They’re adding value through packaging, supplementary materials, and either an implicit or explicit certification process. They’re already competing, after a fashion, with programs like MIT’s OpenCourseWare and Open Yale Courses. Unless I misunderstand the market completely, the Administration’s program should be about as threatening as Wikipedia. If anything, content vendors should be delighted that the government will be providing a baseline of content that they can subsequently ignore or improve upon. You could almost understand it as a subsidy. Hess continues:

Today, the chokepoint is often not the lack of existing online courses or materials but the fact that colleges and universities offer them at prices that approximate those charged to students enrolled in more costly traditional instruction. Of course, this stickiness in price has been due to credentialing and regulatory practices that impede the emergence of low-cost entrants; state-funded institutions that use new e-learning students to cross-subsidize other units; and proprietary operators that have happily responded to this cozy arrangement by competing on convenience rather than price.

Hess is making a valuable point. While it’s certainly good to have free content available, it will not necessarily lower the cost of delivering that content to students. The costs are getting higher for a number of reasons, and it’s not because content is suddenly more expensive to produce. Yet another reason content providers shouldn’t feel threatened by the measure.

Here’s where he misunderstands the economics of freely redistributable content:

This is sensible only if one assumes that federal contracting and oversight ensure better outcomes than market transactions. But this is the same administration that explains that the “public option” is desirable in health care precisely because it believes in market competition.

I have to believe he’s being disingenuous in that second sentence. The difference between open content (which cannot be made scarce) and a “public option” in health care is profound. The first point is valid, however. If the government is going to be producing content, and retains control over the maintenance of that content, this program will be a waste of money. If, on the other hand, the content created is licensed under a  permissive Creative Commons license — say, the “CC-BY“, which allows for both commercial and non-commercial use with attribution, we worry less about the DoD, DoL, and DoE creating valuable content. If they do create useful content, it’s easy to imagine a community of educators, publishers, and institutions that would self-organize around their respective areas of interest and carry the content forward. Open source software communities do this all the time. If they cannot, it was an expensive failure.

So now we can create an understanding of what it would take to make this initiative work. There are three main objections, which I’ll summarize as “It’s wrong for the federal government to create content, and if it does, it won’t do it well, and even if it does create good content, it will crowd out an existing industry.” I don’t believe there’s any real threat to those delivering content today. That industry is growing just fine in the face of the open courseware efforts already underway. I believe we should be concerned about the open courseware program being intentionally or unintentionally used to assert some kind of control over the curricula in community colleges and elsewhere. Properly licensed content, though, regardless of its origin, reduces this threat. The threat can be further reduced by ensuring that in implementing this program, the DoE is supporting a community, not creating customers.

I am skeptical of this $500 million number, and suspect that this program is best structured as a series of small bets that fund a collaboration between DoD, DoL, DoE, and the open courseware community that already exists. With the economy the way it is, there are already concerns that open courseware cannot sustain itself. This seems like a simple way to ensure that the raw materials for higher education are inexpensive and of high quality — and it can be done for significantly less than the headline-grabbing half-a-billion the administration’s proposed.

Executed in this way, there’s little threat of centralized control, no chance of crowding out existing vendors, and freely redistributable content is available to anyone, significantly lowering the entry cost for those who want to enter the distance learning, community college, and other educational markets.

[HT: Glyn Moody, Paul Coen and Timothy B. Lee for getting my mind working on this.]