What is so advanced in AMQP?

As a matter of fact I was looking at posting on AMQP, but Nolan has beaten me to it. Nevertheless, I believe there is more that hope to see XMPP used as a mainstream application messaging protocol before 2011. It only need to addresses a few additional features found in AMQP to be perceived as a valid contender. Let me explain my position by analysing the claims made in the announcement. Starting with the marketing blurb, the announcement explains in preamble that AMQP was developed

In response to internal requirements, market demand and partners' electronic trading needs, the contributors are collaborating on specifications for defining and building messaging infrastructure that is broadly applicable for enterprise use, totally open, platform agnostic, inter-operable, and which aims to provide developers with simpler and more powerful ways of constructing messaging dependent applications.

I doubt anyone would ever use the world “retarded” in a new protocol name. Similarly it would be very inappropriate to admit that a new protocol is not driven by a real world requirement, does not address a market need, would only be implemented in a narrow vertical community, as a closed protocol, running on dedicated hardware and providing a more obscure API to developers. In the end, AMQP is just another message queueing protocol incarnation, for a very specific vertical: the finance industry. Is that enough for being advanced? 

Let’s now compare the two protocols models. XMPP and AMQP use a similar model that defines a server's semantics explicitly. AMQP explains its approach by “interoperability” which as been part of XMPP from the very beginning. Both models specify a modular set of components and standard rules for connecting these. AMQP defines three main types of components, connected into processing chains in a server to create the required functionality:

• The “exchange” component receives messages from applications and routes these to message queues, based on various criteria, usually message properties or content;
• The “message queue” component stores messages until a consumer client application can process them;
• The “binding” component defines the relationship between a message queue and an exchange and provides the message routing rules.

This is known territory for anyone familiar with message middle ware architecture. An AMQP server is nothing but a specialised email server, with each exchange acting as a message transfer agent, and each message queue as a mailbox. Bindings define the routing tables in each transfer agent. Applications “publish” messages to individual transfer agents, which deposit the messages into the appropriate mailboxes. Other applications “consume” message by taking them out of the mailboxes. Again just another expression of the classic concepts of store-and-forward queues and topic subscriptions, with a zest of on-demand message queues and message queue forking.

XMPP pubsub extensions already address many of these features. Persistent nodes play the role of AMQP store-and-forward queues, instant nodes can be used for on demand AMQP queues. But AMQP also makes way for content-based routing, explicitly using message attributes. XMPP does not provide a similar explicit way to define content-based routing, as the pubsub extension leaves this decision to the implementation.

If we now look at the transport protocol, AMQP is defined as “a binary protocol with modern features: it is multi-channel, negotiated, asynchronous, secure, portable, neutral, and efficient.” This definition alone let me believe that XMPP was very “advanced” and has possessed many modern features for a long time now. From this phrase, the only word that can lead to possible counter arguments is “efficient”. And from my perspective, this is even subject to appreciation.

The AMQP protocol defines a transport layer and a functional layer. The functional layer comprises a set of commands which can be mapped to the XMPP pubsub extension without too many difficulties. The transport layer “handles channel multiplexing, framing, content encoding, heart-beating, data representation, and error handling”. Well, apart from “heart-beating” which is sometimes the cause of heated debates on the JSF mailing list, the rest of the functions is present in the current XMPP standard. What is not yet in XMPP, though, is an appropriate reliability mechanism. But the subject gives rise to similarly heated threads at the JSF. 

In the end, paraphrasing Nolan, I totally agree that:

AMQP offers point-to-point, publish/subscribe, and many-to-many messaging. It's a binary protocol which could provide a slight increase in speed over XMPP, but it lacks a standard message body format, presence, and everything else the JSF has pushed out. Not to mention that AMPQ still hasn't reached version 1.0 yet.

In short AMQP arrives too late, but is by no mean advanced.

Beyond the technical aspect, I also believe AMQP’s announcement is the implicit recognition of the larger space XMPP is bound to take as the real-time, presence enabled messaging protocol. I am basing my analysis on these very simple facts. AMQP is a tentative answer to the threat imposed on incumbent messaging protocols developed for the financial industry by open and extensible protocols such as XMPP. The advent of cheap, standard based, alternatives to proprietary messaging systems opens large breaches in the positions of first tier vendors such as Tibco or IBM. It threatens even more the second tier vendors such as those involved in AMQP. In effect, I believe the risk is greater for the second tier, as displacing the first tier will certainly take longer.

In term of business solutions, 80% of the financial message based communication can be handled through “unreliable” messaging protocols. This has been a strong driver behind the very early adoption of IM by the financial industry to conduct many communications leading to money transactions. I believe XMPP could easily add the required reliability to cater for a great part of the remaining 20% in communications. And I am not mentioning the other advantages that can be derived from XMPP’s embedded presence support.

As a matter of fact, I do not believe AMQP will ever achieve its dream of dominance, simply because it is promoted by a software consortium. Despite known procedural problems, the IT industry has a strong tendency to rely on consortia to produce technology. Web services, middleware’s current silver bullet, represents the archetypal example with its internal fighting, fragmentation, lack of architectural coherence, design by committee, and feature overload. It seems inevitable that Web services’ history will resemble its CORBA ancestor. An alternative exists with open source standards. At the heart of these alternative practices are two essential prerequisites: cooperation and trust. Without cooperation, an evolutionary process cannot exist. Without trust, no common experts group can act as an ultimate arbiter. This is precisely where software consortia find their limitations. Putting customers and competing vendors into a consortium and expecting them to come up with a high-quality product is naïve. Commercial realities ensure that cooperation and trust are the last things any participant will care about. Naturally, software consortia contribute to an evolutionary process just as much as open source projects do. But the final direction will always be set by the customers in the consortium with their wallets. For those acquainted with the financial industry practices, this role is usually not benevolent...

In the end, I believe the AMQP announcement has been driven by the same “Fear, Uncertainty and Doubt” (FUD) practices described in “Accidental Empires”. In chapter 14 of his book, Robert Cringely describes techniques one can use to control the market:

• Announce a direction, not a product, and people will stop buying competing products because you’ve just told them they’re the “retarded” way of doing things.
• Don’t support anybody else’s standards and make your own.

When analysed in context, AMQP represents a strong endorsement of XMPP as a viable technology. In my opinion it also highlight the need for application specific extended specifications build on top of existing XMPP extensions. In particular, instead of defining a “generic” reliability mechanism for XMPP, it might be more efficient to define context related extensions. Don’t you think there is a great probability for “reliability” in the financial industry to bare a different meaning than the same word in another industries?

Finally, again paraphrasing Nolan, “we could learn a few things from them, like their requirements.” I entirely agree users’ requirements are always worth reading. We could probably also get these requirements directly through the main JSF’s sponsor. After all, isn’t JP Morgan Chase one of their blue chip customers? This to emphasised Nolan’s plea for advocacy in this space. But in my opinion, advocacy must move well above the John O'Hara or John Davies level. To a level that understands you can get the same feature set as offered by proprietary messaging protocols for 20% of the cost… This is a message the financial industry usually receives well without a “reliable” protocol.

Technorati Tags: XMPP, Jabber, Interoperability, AMQP, Antecipate