Equals Drummond

At the Glue Conference this week I’m enjoying a great set of speakers lined up by Eric Norlin on the topic of how everything in the networked universe gets glued together using Web 2.0 tools and beyond. (The talk Mitch Kapor gave this morning was worth the trip all by itself.)

In a few minutes I’ll be on a panel called Implementing the Open Web. In chatting with Lloyd Hilaiel of Yahoo, Kevin Mullins of MIT, and Phil Windley of Kynetx about this topic last night, we hit on one key point that Phil articulated this way: “People tend to conflate ‘open’ with ‘public domain’, i.e.,  that anything that qualifies as open must be freely available to all.”

It struck me how true this is. It reminds me of the Richard Stallman quote describing open source (cited in the Wikipedia Gratis versus Libre article): “Think free as in free speech, not free beer.”

In terms of data on the Open Web, what this means that even though a particular pool of data may be available via an open standard, publicly-accessible interface, it does NOT mean this data must be publicly available to anyone. If that were true, the whole concept of a personal data store — a key premise of VRM (Vendor Relationship Management) — would not be possible.

So what makes any system or node participating in the Web “open” is not that its data is public, but that the metadata and services for accessing it are available via a publicly discoverable, open-standard interface. The public discovery portion of this is the goal of the XRD work now underway at the XRI Technical Committee at OASIS (based on the original XRDS work – see this blog post by Eran Hammer-Lahav of Yahoo to understand the differences). The open standard portion is the output of IETF, W3C, OASIS, and all the other SSOs (standards-setting organizations) for the net. (The potential of the Open Web Foundation, once it finishes its bootstrap stage, is to make this process of creating open standards even more lightweight and distributed.)

This combination – open discovery of open interfaces accessible over open protocols – is the DNA of the Open Web. And it applies equally to both public and private data. In fact it can finally open up what might be called the Permissioned Web - the Web of all all data that any one party has permission from other parties to access.

That would lead us to the need for integrating identity and permissions with the data, which brings us to the motivations for XDI as a semantic data sharing format/protocol – but my panel is about to start so that will have to be another post.

Something else so good I just have to blog it: Eran Hammer-Lahav’s Discovery Coordination Report on the new metadata-discovery list he set up. Eran’s turning into a one-man hub of all things discovery as he drives forward together with the rest of the OASIS XRI TC towards the pushing out the new XRD 1.0 spec.

I have high hopes for this spec and Eran is one of the key reasons (plus the efforts of his co-editor Nat Sakimura of NRI, who is working OpenID miracles in Japan, and other new TC members who have joined to finally make simple, safe, uniform metadata discovery a reality on the Web).

For most people, watching the evolution of technical specifications is like watching a glacier move. To those of us living the process, though, there can be a great deal of drama to it — in fact it’s much more like climbing an icefall inside the glacier (anyone doubting how much adrenaline that takes should read John Krakauer’s description in Into Thin Air of climbing Mt. Everest’s Khumbu Icefall). For example, the failure of the OASIS Standard vote on the XRI 2.0 specifications last May — the first ever in 40+ OASIS Standard votes — was a watershed in the interaction of two standards bodies (W3C and OASIS).

The repercussions from that event have been equally unpredictable. Who would have thought that just four months later the XRI TC and W3C TAG would have rough consensus on how to resolve their differences? Or that the discussions would spill over to the much larger topic of uniform metadata discovery on the Web? Or that discovery could turn out to be the key to building identity into the browser? Or that interest in the XRDS discovery format would boil up enough to beget a new spec intended for uniform metadata discovery for any type of URI or XRI?

But that’s just what has happened. Two weeks ago at the Internet Identity Workshop, Eran Hammer-Lahav, author of the OAuth Discovery spec and founder of the XRDS-Simple list, led a marathon session on a new uniform metadata discovery specification to be called XRD 1.0. With 20 to 40 people in attendance all afternoon, Eran first ran through his exhaustively-researched blog post on HTTP and discovery, then through the proposed simplifications to the current XRDS/XRD schema. By the end there was rough consensus on XRD as a mechanism for uniform metadata discovery across all the different Internet identity and data sharing specs that need it (XRI, OpenID, OAuth, OpenSocial, XDI, Data Portability, etc.)

The name “XRD” is itself quite revealing of the evolutionary path to this point. When the OASIS XRI TC first developed the XML-based metadata discovery format we needed for XRI resolution back in 2003, we called it XRID (XRI Descriptor). We made it as simple and generalized as we could simply because any resource could have an XRI, so there was no telling what type of metadata might be needed over time. We focused primarily on one clear requirement: given input identifier x and service type y, define how to discover service endpoint URI z.

By 2005, when OpenID grew to the point of needing a discovery format, the authors of the Yadis (Yet Another Discovery spec) authors looked at XRID and saw something very close to what they needed. But XRID assumed you needed a sequence of descriptors corresponding to an XRI resolution chain. With OpenID a sequence wasn’t needed because an http(s) URI would have just one descriptor. So the XRI TC renamed the metadata format to XRD (Extensible Resource Descriptor) and created a separate XML wrapper element called XRDS (XRD Sequence) for cases like XRI resolution where you needed to wrap a sequence of XRDs.

However for cross-compatibility between XRI and OpenID, OpenID discovery just assumed the outer XRDS wrapper element even if it contained only one XRD. So the discovery format became widely known by the wrapper element, XRDS.

It wasn’t until Eran’s deep-dive on uniform metadata discovery that he recognized that the base case should be the other way around, i.e., for most URIs the the base discovery document should be an XRD, and only in cases like XRI resolution do you need the XRDS wrapper element.

Since the XRI TC had already made the decision in our next round of specs to split off XRDS from XRI Resolution, it was easy to just call this new specification XRD 1.0 (”1.0″ reflecting that it is the first standalone specification for XRD). However what we didn’t realize until the XRI TC F2F meeting the day after IIW was that XRD as both a metadata discovery format and protocol would be comprehensive enough that XRI 3.0 Resolution could become simply a “profile” of XRD 1.0 — and thus dramatically shorter.

We also didn’t realize how badly many different stakeholders want a Web-wide metadata discovery mechanism. Within a week after IIW we had six new people join the XRI TC to be part of the XRD work, and as of this writing nine more are in the queue.

So the roadmap of the next generation of XRI TC outputs is clear now. We will produce two OASIS Standard-track specifications:

• XRI 3.0 (including Syntax, Resolution, and Bindings) as a uniform syntax and resolution protocol for shared semantics across hierarchical URI schemes.
• XRD 1.0 for uniform metadata discovery for any URI or XRI.

Stay tuned for updates – hopefully this set of specs will set a glacier speed record.