Having looked at music formats and instant messaging protocols, this final installment of a short series on open formats covers what may be the most ubiquitous of digital file formats: office documents. Spreadsheets, presentations, desktop databases, and the common text document hold most of the business information of our age.

In North America, at least, most of this information lives inside a set of patent-protected, binary (which makes them difficult to reverse-engineer), and undocumented file formats. The Microsoft Office formats, the most well known of which is the Microsoft Word format, are used to store millions (billions?) of documents, from personal journals to government legislation.

For those creating these documents, the problem is inherently disguised. If you create a Microsoft Word document, then you must have access to Microsoft Office and can therefore open, read, and modify the document. The problem arises when you don’t have access to a copy of Microsoft Office. This may be due to financial limitations, or it may be because you are running on a platform that is not supported by Microsoft. No matter how much money you have, you can’t buy a copy of Microsoft Office for Linux.

The frustration of receiving Microsoft Word documents as email attachments led Richard Stallman, founder of the Free Software Foundation, to write a brief manifesto covering the perils of this proprietary format.

The essential problem with a proprietary document format like Microsoft Word is that a private corporation owns the ability to access the works you have created. While it’s not likely that Microsoft is going to deny you access to your Microsoft Word-formatted love letters and chili recipes tomorrow, they do theoretically hold that right.

Confusion and Optimism

The closed binary format in Microsoft Office has been enormously broad in its reach. However, the life of this format is limited. Microsoft recently announced plans to move to a documented format that could potentially be accessed through non-Microsoft means.

The meaning of this announcement has yet to be truly understood. Some see this as the end of the proprietary Microsoft format and a great victory for freedom and openness, as millions of documents will be created in an openly documented format. Others are more cynical, citing licensing issues that will limit what people can do with the formats.

It seems clear, though, that while the legal issues around the new Microsoft formats remain disputed, their technical architecture (basically XML in Zip files) will be much more easily accessible regardless of whether access is endorsed by Microsoft or not.

I don’t clearly understand the issues around this yet myself. The Microsoft community/weblog site, Channel 9, posted a video interview about the new Office formats with Jean Paoli. Watching this video shows the Microsoft engineer’s obvious enthusiasm for openness. However, the video ironically requires proprietary Windows Media technologies for playback.

Alternatives and Workarounds

As Ogg Vorbis is to MP3, and as Jabber is to MSN/ICQ/AIM, so OpenDocument is to Microsoft Office formats. OpenDocument is a new set of standard office file formats for text documents, spreadsheets, presentations, and charts. This open and standard format is the default format in the forthcoming OpenOffice.org 2.0 office suite, but could theoretically be implemented by other applications as well.

Saving your documents in the OpenDocument format means that no one owns the ability to access your works. While the specifications aren’t perfect (I was dismayed to here complaints about the spreadsheet component), it remains a critical standard.

As is the case with instant messaging protocols, the move from proprietary to open office file formats can be eased with the help of transitional software. The OpenOffie.org suite (both the 1.x and upcoming 2.0 versions) can open, edit, and save the main Microsoft Office formats quite well. Using OpenOffice.org, I can easily open any Microsoft Word attachments I might get in my email.

For those that are still stuck with Microsoft Office as an overall platform in their organization, but are looking to move away from Microsoft Windows, there are more promising options. The Wine project is a compatibility layer for running Windows applications on Linux. Especially when packaged in the Codeweavers CrossOver Office product, it is surprisingly easy to actually run Microsoft Office on Linux. This is obviously only a transitional aid, and not a long term solution, but it is helpful.

Conclusion: Freedom Should Be On By Default

The core idea behind this series on open formats and protocols is that you should not be limited in access to what you have created yourself, regardless of the tools you used to create. No one would buy a pen that produced writing that could only be read through special glasses sold by the same company. Even more so, no one would allow their governments to publish documents created by this crippled pen.

Being locked out of content that should be free or that you have legitimately purchased is bad enough. I have to use illegal software to watch DVDs (that I have bought and paid for) on my laptop. However, it is even worse when you are locked out of content that you have created yourself.

If your mom buys a computer, writes you a letter, and emails it to you in the Microsoft Word format, you have to pay Microsoft to read the letter. Of course, your mom doesn’t have to use Microsoft Office, but if it is the default word processor on her new computer, she may not realize the issue.

If you have your wedding video-recorded and it is given to you by the production company in DVD format, you can’t make copies for your family or as a backup. Again, the production company doesn’t have to use the proprietary DVD format, but it is the only one that will play in everyone’s home DVD player. [UPDATE: Several people have corretly pointed out that the proprietary DVD encryption (CSS) is optional and need not be used on personal DVDs – good point.]

For these reasons, it is not good enough that freedom be available as an option. Freedom must be on by default.

The Catch-22 of Open Formats mini-series

1. The Catch-22 of Open Format Adoption, Part 1: Music
2. The Catch-22 of Open Format Adoption, Part 2: Instant Messaging
3. The Catch-22 of Open Format Adoption, Part 3: Office Documents (you are here)

I wrote last week about the catch-22 of open audio formats. Online music isn’t the only domain in which open formats are emerging, nor is it the most significant. The world of instant messaging (IM) is another case where open protocols have emerged to compete with their proprietary predecessors.

Chances are if you’re online to read this article, you use one of the major commercial IM services. MSN, AIM (which now includes iChat users and ICQ), and Yahoo’s IM services are all enormously popular.

In this case, Jabber is the free/open-source alternative. While the end-user experience is basically the same, Jabber has a different significantly architecture from the dominant IM.

I’m not an expert in this area, so I’ll keep my description as basic as possible. The Jabber protocol is similar to the email infrastructure, where anyone can setup a server that clients connect to. The server then relays messages to other clients on that same server, or to other servers to reach other clients. Like with email, then, you can setup your own server – though this isn’t practical for most people. The more likely scenario involves companies or internet services providers (ISPs) setting up Jabber servers much like they do with email. This differs from MSN, Yahoo, and AIM in that each of these services have their own central server systems that are controlled by the company that owns each network.

As with music file formats, most people don’t care about instant messaging protocols. They just want to chat with their friends and co-workers. You have to use the same protocol/service as your friends, or you’ve got no-one to talk to.

There are some key software applications that can help bridge the gap between proprietary protocols and the open Jabber protocol. Several instant messaging client applications, including Gaim, Trillian, and Adium, allow you to connect to all of the major IM networks. You can have contacts from MSN, AIM, and others, all on the same contact list as your Jabber contacts.

These multiple-protocol clients help ease the transition to open protocols. If I were to switch entirely to Jabber today, I would no longer be able to talk to many of my friends. However, using Gaim, I can use Jabber whenever possible, but still maintain contact with those of my friends still using proprietary protocols.

Of the 20 to 30 contacts I have in my instant messaging client, a little more than half of those are using Jabber. The remain contacts are either AIM, ICQ, or IRC (I’ve managed to drop any MSN contacts).

This is likely a higher ratio of Jabber-to-proprietary contacts than many. This is because at the small business where I work we have our own Jabber sever setup that allows us to have secure (and free) instant messaging (both for one-on-one chat and for group chats). The open-source nature of Jabber allows our company to easily control and manage our own instant messaging server. I would encourage other businesses to do the same. It has been a great tool for us.

The Catch-22 of Open Formats mini-series

1. The Catch-22 of Open Format Adoption, Part 1: Music
2. The Catch-22 of Open Format Adoption, Part 2: Instant Messaging (you are here)
3. Part 3: Coming soon

It started innocently. I started using a free/open-source web browser, then called Phoenix (now Firefox). It seemed to me to be faster, easier, and generally better than Internet Explorer on my Windows XP powered laptop. Soon after, having been unhappy with the performance of Microsoft Outlook with my large volume of archived mail and unwilling to pay for or pirate the promising new (2003) version, I switched from Microsoft’s Outlook to the Mozilla Thunderbird mail application.

Soon after, I made the switch from Trillian to the free/open-source instant messaging client, Gaim. It then dawned on me that the three applications I use most, my web browser, email client, and instant messaging client, were all free/open-source software. Not only were these applications free software, they are also available across multiple platforms (Windows, Linux, and some for Mac OS X).

Armed with the realization that I was close to platform independence, I intentionally sought to weed out the few remaining Windows-only applications in my arsenal. I made the switch from Microsoft Office to OpenOffice.org. This left some games, and Photoshop/Illustrator as the last key proprietary/closed-souce applications in my regular use.

Driven primarily by curiosity and technical/professional interest (particularly in alternative user-interface design), I decided to exercise my newfound platform/operating-system independence and switch my primary laptop to Linux. This was over a year ago and I’ve been relatively happy, given some challenges, since.

All along, these decisions and similar decisions at the company-level where I work were pragmatic. Open-source software is good, but it’s really about using the best tool for the job, be it free/open or proprietary. While the free/open systems were generally beating out the proprietary systems, the decisions were still driven primarily by a (relatively) qualitative comparison of features/quality, rather than any philosophical best about licensing.

I had always been turned off by Microsoft-hating Linux-zealots that play into my stereotype like a bad gay sitcom character. Microsoft has many problems, but they also develop some remarkably good software (I’m a fan of the Office suite and love Microsoft Streets and Trips). What bothered me more than the last-computer-conference-I-was-at-Tshirt attire was the fundamental belief that free/open-source software was better than proprietary alternatives.

However, having lived in a mostly free/open-source software environment for over a year now, I am starting to drink the kool-aid.

I had been running Mac OS X on a old iMac for browser testing purposes and following the development of OS X for a while too. It dawned on me that one of the primary reasons keeping from using OS X as my primary operating system (in addition to not wanting to replace my working hardware), was due the licensing, control, and ownership of the software and platform.

Mac OS X, since it is built largely upon open-source components and protocols, carries with it many of the pragmatic benefits of open-source software. However, since a significant amount of the Mac OS X system is not free/open-source, you do not have all of the freedom you would have on a completely free/open-source system. You are, in the long term, still at the whim of a private corporation (Apple Computer).

To many people, quite understandably, this will not matter – especially since Apple is producing some great software lately. That’s fine with me. I’m not at a point where where I want to force everyone to use free/open-source software. The idea of forcing something in the name of “freedom” just doesn’t sit well with me.

That said, I am starting to think that governments and other public institutions might have some kind of moral and ethical (though not, at this point, legal) obligation to share software it develops or has developed on its behalf.

While I’m not about to start burning the Windows XP license that came with my laptop in the street, I’m starting to take into account licensing and freedom to use and control software in the decisions and recommendations I make for myself and those around me.