Saturday, December 16, 2006

Basic instincts

What an interesting statement issued here after yet another superficial interpretation of the well known and long identified dual model for IT applications distribution.

At present, there is no one company covering these issues comprehensively. Microsoft must build scalable presence, Google must build trust. At Jabber, Inc. we're watching this looming battle with the inherent interest of an arms dealer.

The author is by the way Jabber Inc's public relations person. The actual issues referred to in the post are

  • The inherent need for a high trust level that must come associated with any application service provider.
  • The scalability of the distribution system's architecture to support the service.

I somewhat agree that presence is called to play an increasing role in many upcoming applications, be they used in an ASP or edge model. But this is certainly not the unique constraint that one needs to consider for building a successful ASP infrastructure. Generic reliability, graceful degraded functions, high availability and self healing are just a few of the basic needs for a proper scaling architecture. Real time oriented data services will certainly also play a role. In that respect, Google probably has an edge.
On the other front, I am not certain any of the two contenders can pretend to inspire a sufficient trust level. But, contrary to infrastructure, this is not a technical matter.

To conclude, although I have unfortunately observed several times that many in the PR world often associate “public” with “dummies”, I find rather curious to compare carrots (the illusive trust level of a corporation) and apples (the forbidden fruit of seamless scalability). More interestingly, I am wondering if the author is not falling into that category of people spending 5 months of their year's time in front of a television or a computer. Beyond the natural mental mimetism which results from trying to penetrate DoD contractors defenses and sell them presence, too much exposure to WoW, "GI Jane", [insert here any other B TV series, video game or reality show] is the only rational explanation I could find to why the inherent interests of this company should be those of an "arms dealer" when the context is "live" services…

Technorati Tags: , , ,

Labels: ,

Friday, December 15, 2006

Google Mail and Talk outage

Even Google is not king of 5 nines reliability...

Around 10:15 am (Central Europe time) GMail started spitting out "Server Error We're sorry, but Gmail is temporarily unavailable. We're currently working to fix the problem -- please try logging in to your account in a few minutes." POP access was also timing out from standard email client. At the same time logging onto GTalk would lead to contact lists in error

<iq type="error" id="aacca" to="jlseguineau@gmail.com/psi100C6E2E"></iq>
<query xmlns="jabber:iq:roster">
<error type="wait" code="500">
<internal-server-error xmlns="urn:ietf:params:xml:ns:xmpp-stanzas"/>
</error>
</query>
</iq>

Around 10:30 the contact list access was re-established, and normal operations resumed on GTalk. But GMail was still unaccessible. Only towards 10:42 did it come back to a pallid life, but even at that time some function of the interface were spitting operational errors. UI functionality came back to normal around 10:45, with SMTP out re-established around 10:47, but at the time SMTP in was still not restored. In addition, the content of my "sent Mail" folder had disappeared... At 10:52 SMTP in was restored and I was able to receive some earlier test messages. Still, not all of them, as they were held back on their originating server following the outage, waiting to be retried later. At 10:57 the content of my my "sent Mail" folder re-appeared. And at around 11:02 the service was restored to its normal working state. In the end, the recovery was well orchestrated, and Google has certainly put some thought behind its procedure. Every external access was degraded gracefully, be it POP/IMAP, web UI or XMPP. Similarly, the service came back to life gradually, without spike, with different functions being restored independently. This incident brings to light some of the architectural options they took, and show how their messaging infrastructure is integrated.

In particular, contact lists are definitively shared between mail and instant messaging, which is why GTalk showed partial service outage when the contact list service was not operational. It also mean that Google could very easily come up with a presence enabled address book where end users could aggregate their contacts.

In the end, although I experienced a mere 40 minutes of inconvenience, the service was restored to its full capacity through what look like an entirely automated process. Lesson to be learned by some would be web 2.0 entrepreneurs…

Thursday, December 14, 2006

Translating between otherwise incompatible networks

This is the most common definition given of a gateway in the telecommunication context:

A gateway is node that translates between two otherwise incompatible networks or network segments. Gateways perform code and protocol conversion to facilitate traffic between different architectures.

On the ground of this definition, the application that IBM has built on its Websphere J2EE server for Sametime is effectively a gateway. For that exact same reason, I believe that a very substantial leap of faith is required to envisage that it could lead to a multi-protocol presence server.
Fom the information available, the gateway is an application build on the JSR-000116 SIP Servlet API. It is in its way following a similar architecture of what IBM tout as the Websphere Presence Server. The SIP Servlet API is an attempt at leveraging developers' knowledge of HTTP web containers and extending it to SIP. On paper this is a sensible idea. In practice it has a few drawbacks.

  • First, it is entirely based on the SIP protocol, which means that applications built on top of this technology will be SIP applications. As such, the SIP servlets are designed to process SIP requests and return SIP responses.
    That somewhat restrict the scope, as each network protocol has very specific characteristics which are rarely though of as "universal". Moreover, not every presence protocol uses a request/response paradigm in the way SIP does. XMPP for example uses simple notification packets to signal modification in presence sates, without requiring these packets to be answered.  In that respect, it becomes more difficult to mold XMPP into a servlet paradigm in the way it was possible with SIP. Consequently, the advantages expected from a common programming paradigm become somewhat blurred.
  • Second, it relies on a J2EE server to run. One can argue that it is a proven robust architecture. I would simply say that J2EE servers were not invented to provide solutions to real-time communications problems, but rather to bring solution to transactional application problems in the enterprise. The table herewith summarizes the differences between the two contexts.

All in all, if the IBM gateway had been implemented using a more appropriate application server such as a JSR 22 JAIN SLEE API implementation, I would have been tempted to take the leap of faith. Unfortunately, this gateway architecture prohibits any use beyond bridging between SIP and other protocols. To be fair, the same inability applies to the equally shortsighted approach taken by Jabber Inc. A solution that only sees the world through the eyes of a particular protocol will always leave us far from what is necessary to build a multi-protocol presence server. For the simple reason presence encompasses more than just protocols…

Technorati Tags: , , ,

Labels: , ,

Wednesday, December 13, 2006

Ready for personal presence solutions?

Mike Gotta had a dream: a presence aggregator service on every Windows workstation. If Microsoft could execute on this excellent idea, this would bring them back to the forefront of the individual applications scene.

As in most disruptive changes, the issue will certainly not come from the technology side. What Mike calls a "headless presence client aggregator" is nothing more than a publish/subscribe broker where address handles are used as topics... I think that technically, it can be implemented in no time. If this service is aggregating presence information at a client level, the required processing power and memory footprint can be kept very low. A documented API would allow both applications and "watcher" clients to subscribe or publish information to the common presence aggregator, ending up in a fully layered architecture that could benefit any application running on the workstation. The secret here would be to keep it simple.

The unknown lies in the incontrollable urge Microsoft always exhibit to "control the world" when adding feature to its flagship products. For this aggregator to succeed, it would have to be open…

  • If this presence aggregator imposes a particular communication protocol, such as the RTC flavor of SIP/SIMPLE, then the adoption would be severely impaired. On the contrary, if the aggregator is open to other communication protocols, and provide by default both an RTC and an XMPP transport, Microsoft would be sending a very strong signal to the industry.
  • If the rivalries between different Microsoft line of products were ironed out and they agreed to use this service rather than creating their own version of a presence aggregator, they would certainly be sending a strong trust signal to their user base. And as the API would be publicly documented they would not be accused of creating an unfair advantage for themselves.

In the end, there are many "if" for a somewhat obvious and rational solution. This period of the year is always full of dreams and whishes. In this case, I am certain the model would work, but would Micosoft be rational and open?

Technorati Tags: ,

Labels: ,

Speaking of cluetards...

Have you notice the drums roll and trumpeting around the latest Sametime interoperability announcement? I always find it entertaining to witness how incumbents can turn a failure to respond to their customers wishes into a world saving progress without even a blush!

At the end of the summer of 2004, I had the opportunity to see IBM in action at a workshop held by the Financial Service Instant Messaging Association on the subject of IM interoperability. FIMA had been worried very early on by the disparity of instant messaging protocols, both in the public and the enterprise IM space. So they set to give their least common denominator functional definition of interoperability, which when you think of it is not asking for the moon…
At the workshop, vendor presented their views, and it all went smoothly until the chair of the Jabber Software Foundation asked IBM why they wanted to support the still unfinished SIMPLE protocol as a federation protocol rather than XMPP which was already stable and widely used alongside Sametime in the financial industry.
Anyone either business savvy or a little familiar with Sametime's technical architecture would have though along the same lines. From an architecture standpoint, the multiplexers, community hubs and community server applications of sametime, with their permanent TCP links, can be functionally mapped one to one with XMPP client connectors, session managers and service components. From a business perspective, federating the existing large financial industry's Sametime islands with the expanding XMPP footprint would have help IBM keep an edge against the nascent Microsoft LCS expansion.
Instead, we were granted a flame from the IBM representative in favor of SIMPLE which was more typical of a Gorilla's behavior than a business representative:

If challenged by a younger or even by an outsider male, a silverback [Silverbacks are the strong, dominant troop leaders] will scream, beat his chest, break branches, bare his teeth, then charge forward.

In short, SIMPLE was the way to go, and everyone not endorsing it was stupid and doomed to fail. Well, two years after, IBM announced interoperability with XMPP, which instantaneously guaranties secure and controlled communication with any XMPP server, including those at Google. They also announced interoperability with AOL, but funnily enough they do not specify what protocol they use. You maybe interested to know that AOL offers both an LCS flavor of SIMPLE and XMPP to enterprises for federation with AIM. As there is no mention of interoperability with LCS in the Sametime announcement, it would be interesting to have a clearer insight on what they IBM is using for interconnecting with AOL. As to Yahoo!, well it is always a little difficult to have anything but a fuzzy statement from these guys. Anyway, the announcement said federation between Sametime and Yahoo messenger is on the "road map".

Ken Camp came up with a nice and illustrative contraction to apply to people unable to spot the appropriate answer to end-users' aspirations, cluetards… In this case, IBM is only different from the average by the size, just a large cluetard. If I were a financial Sametime customer, I would claim compensation for having been treated this way and left longing for so many years.
More importantly, this episode illustrates once again how large incumbent IT vendors are blind to real users' needs, when they are not in line with their core business. IBM business is in selling servers, not in providing communication between peoples. I find it amazing that a pure technical choice made by some ex-Ubique geek ended up depriving an important business community of an easy and beneficial way to conduct their business with small firms or other institutions.

Beyond this, Giacomo is in my opinion asking the right question: should we forget SIMPLE? In a perfect world, I would certainly answer positively. In the real world, it will take some time in view of the interests at stake. Having been involved in designing applications federating both protocols, I certainly have a unique perspective on the strength and weaknesses of each of them. From a technical standpoint, my only observation is that SIMPLE simply lacks the simplicity needed for a wide adoption.

  • Except the Microsoft LCS there is no SIMPLE based instant messaging and presence (IMP) application in service outside the telecom industry. Anybody having looked closely at the actual protocol used by LCS will agree that it's actual compliance with SIMPLE stops short after the first SERVICE packet. Microsoft has leaned the hard way that SIMPLE was not ready for IMP, and has been obliged to supplement the lack of the protocol with LCS own extensions. Such things as multi client instances, message sessions, contact lists management, preferences, multi user conferencing are notably absent from the specification and have partially been implemented by Microsoft using non-standard extensions.
  • From a user agent perspective, the closest to a "standard" SIMPLE client implementation is the Counterpath' eyebeam softphone and it XLite derivative. Their development team deserves the greatest respect for having incessantly been trying to accommodate the ever changing scope of XCAP, and the various contact lists management options that must be part of a "workable" SIMPLE solution. But while doing this, they had no time to automate the status change to "on the phone" when a SIP call was taking place…
  • From a standard stand point, those who have red the 2500 plus pages from the 3GPP describing the use of SIMPLE for presence are well aware that, for example, list services, XCAP and a stable definition of rich PIDF are on the critical path for delivering the service as expected by the IMS architecture. XCAP is probably the most blatant example of wheel re-invention in this industry, where the oversized ego of the author is holding back an entire industry. More importantly, if one look at the SIMPLE RFCs and drafts authors, one can quickly infer that the standardization process is done by a vendor led consortium. As I have said before, almost every vendor led consortium has failed to impose any long lived interoperability standard on the Internet. I am afraid SIMPLE is another one of them, and it is time to realize it…
  • Finally, there is the complexity of SIMPLE. It makes it difficult to grasp by the average application developer outside those versed in "voice over…". When adding together the rather rigid framework provided by SIMPLE and the vast differences in interpretations by various implementers, it makes this protocol look "elitist" when compared to XMPP. As I mentioned above, having designed and implemented converged systems supporting those two protocols simultaneously, I can vouch that any application programmer can grasp how XMPP works; the same is not true from SIMPLE.

As Ken put it, simplicity and elegance alway make winners and losers. Simplicity is unfortunately not the first quality of SIMPLE. As a result, it has not seen mainstream adoption by web application developers which are instead turning to XMPP. That said, SIMPLE will not go away easily in view of the heavy investments made by many cluetards. Unfortunately this state of affairs will help maintain the silo divide between the telecom and the internet worlds.

The term "elegance" finds its origin in the Latin "eligere" which also gave the derived term "election". In fashion as in real life, elegance implies making a choice. In the Sametime case, it was between remaining Incredibly Blind and Mediocre, or working at solving real end-users pain points. You can guess what choice was made…

Technorati Tags: , , , ,

Labels: ,

Saturday, December 02, 2006

Maps of every country unite!

Some time ago, I was researching XMPP presence indicators that could be embedded into a web or blog page. This is how I came across Jobble, a Google maps and XMPP presence mashup. I knew of other similar services, such as Jabber World Map, Jabber Google Map and Talk Maps. They all provide the same kind of information: a map with the presence state representation of online XMPP users. For some reason, I prefer the way the Jobble site presents the information. But it is only a matter of taste.

As I was toying with these different services, it appeared to me that they were used by different demographics, which were themselves geographically concentrated. For example, Jobble, which is a Polish project, has a vast majority of subscribers from north-eastern Europe. The same applies to Jabber World Map which is coming from the Netherlands. Talk Maps has a slight majority of users in Europe, but overall users are thinly spread. Curiously, north-american users don't seem to fancy this kind of service…
My second observation was that those of my contacts using these services were scattered on several separate maps. I would have liked to have them on the same map instead. This is how I thought of federating XMPP maps.

Putting people on a map is a way to make them feel closer, and somehow participate in a feeling of community. But when these communities do not intersect, the chances to communicate and establish new relationships remain low. I have discussed earlier why a closed community would ultimately be less sticky than an open one, and looking carefully at these services shows they somehow act as closed communities. I think that the choice of registering to a particular map service must only be governed by the features that can be derived from the user presence, and not from the presence collection itself. For example, someone may prefer the way Jabber World Map show online users when the mouse hovers above a symbol, other the traditional point and click approach of Google Maps found in Talk Maps. Furthermore, someone may want to develop a similar service using Yahoo flash based maps, to be able to add some overlays.

Unfortunately, the ways these services are implemented today only lead to a walled garden result. Each service is built around its own data store and does not communicate with the rest of the world. Furthermore, they only allow static geo-location mapping, leaving out mobile devices or traveling users. Thinking of it, it is not that difficult to modify these services and turn them into distributed geo-location providers. The base features I would expect of such services are summarized bellow:

  • Single registration and management point. The options chosen for my geo-location information using a particular service may become available to other similar services without forcing me to register with each of them individually.
  • Static and dynamic geo-location data support. I want to be able to specify either static longitude and latitude values for a fixed workstation, or have the service pick up this information dynamically from a particular client.
  • Privacy and information propagation control. It may look strange to mention privacy amongst the expected features, as putting one's presence on a map is already revealing a lot. But I would expect the service to provide a way to control the propagation of these data outside the service through an appropriate combination of white and black lists.

Looking at possible implementation options, I believe we have to keep the current approach which exposes the service through a presence enabled bot. It has the immense advantage of being well understood by the average user.
As a first step, the bot would need to be modified to understand the Personal Eventing (PEP) extension notifications. This will cater for dynamic geo-location updates. As PEP implementation progress, the service will automatically get access to dynamic geo-location information as they are set on the end user device, and will be able to display it on the map along with the user's static data.  From an end user standpoint, this would also simplify the fine grain tuning of which piece of information is made available to the service directly at the PEP level.
The bot could also be adapted to support PEP subscriptions. This would be appropriate in case the user's home server or the user profile does not allow an automatic subscription to PEP when someone subscribes to the user's presence information.
Finally, the service itself would have to expose a user geo-location node as specified by the publish/subscribe XMPP extension.  It would not be necessary to implement a full fledged publish/subscribe service, but rather make the service understand only the appropriate subset dealing with the http://jabber.org/protocol/geoloc namespace. Another map service interested to get geo-location information would then subscribe to this node and receive any published modification.

<message from= "map.montague.net" to="map.capulet.com">
  <event xmlns="http://jabber.org/protocol/pubsub#event">
    <items node="n48ad4fj78zn38st734">
      <item id="a1s2d3f4g5h6bjeh936">
        <geoloc xmlns="http://jabber.org/protocol/geoloc" xml:lang="en">
          <country>Italy</country>
          <locality>Verona</locality>
          <lat>45.44</lat>
          <lon>10.996</lon>
        </geoloc>
      </item>
    </items>
  </event>
  <addresses xmlns="http://jabber.org/protocol/address">
    <address type="replyto" jid="romeo@montague.net/orchard"/>
  </addresses>
</message>

Another possible approach would be to make the service itself support presence subscriptions and have the geo-location information implemented in a PEP node.  This way the relationship between two map services would be entirely presence driven, enabling added control through presence states. For example, there would not be notification toward a service which does not appear available. This is important in period of maintenance, or can be used as a way to throttle down heavy traffic. Presence would also be used to propagate end user's availability states.

<presence from= "map.montague.net" to="map.capulet.com">
  <show>away</show>
  <addresses xmlns='http://jabber.org/protocol/address'>
    <address type="replyto" jid="romeo@montague.net/orchard">
  </addresses>
</presence>

The end result would be a distributed geo-location information aggregation that could be leveraged through a standard and open protocol. Several local aggregators could be built by geographically close communities, but see their reach extended to the Internet as a whole because of the distributed design. From an end user stand point, there is not lock in, as it is possible to choose a service on the feature set offered and move to another service implementation without being forced to use yet another address handle. The combination of service based and PEP based filters would ensure a good level of privacy control.  And in the end my presence and geo-location information as seen by Jobble will be reflected on as Jabber World Map, Jabber Google Map and Talk Maps without forcing me to add a bot in my contact list for each different service.

Technorati Tags: , , , , , ,

Labels: , , ,