Friday, August 26, 2005

The socio-politics of SOA

For years I have earned my living in creating P2P solutions for integration problems. P2P stands for both point-to-point and peer-to-peer. The technical architecture used back then was point to point; the responsibility for the solution was peer-to-peer.

Point-to-point integration has a strong tactical essence. Two programmers from two different departments find themselves in an ad-hoc team that has to realize ASAP (it's always ASAP) a business process that spans across the two departments' information systems. P#1 and P#2 figure out what has to be done: P#1 has to code 1-2 functions and P#2 has to introduce in his/her application some calls to those newly added functions by P#1. Probably the DBA gets involved too, as some data replication might be needed. Once they're through, QA goes in.

The QA team got many questions, and they might encounter many issues during their testing. But P#1 & P#2 are always around; they assume responsibility for their bilateral ad-hoc solution and are always helping when needed. When the code goes into production, our two good Ps are still there as 3rd level support. In general, production problems are not politically beneficial. P#1 and P#2, being constantly aware of their public image, are doing the best they can to support their solution. That's the peer-to-peer, or programmer-to-programmer nature of the point-to-point architecture.

P2P architecture is an Enterprise Architecture nightmare. Links of all kinds are created in an ad-hoc and uncontrolled fashion among pieces of codes. These links grow organically and prosper, just like URLs, without central control. Hubs of links are created around major functions and eventually Enterprise IT applications turn to be a single organic, complex system, as A.L. Barabasi interestingly describes in his book LINKED: The New Science of Networks. And there are no bots to perform a post-mortem link analysis like there's over the internet, so no one actually knows what is linked to what.

Still, in this mess, each one is aware of his/her 1st degree of links (just like in LinkedIn) – and that has its moral and business merits as previously described.

Service Oriented Architecture helps a lot in the removal of disordered links and the reestablishment of Enterprise Order. (Are p2p links equal the disordered tags, while Enterprise SOA represents yet another ontological desire to have one, decisive, in-control view of the world (Enterprise)? Are we seeing again the same patterns we've discussed in previous posts? I'd say we do. I'd say that management and control in a distributed world – and IT has become highly distributed in the past 5 years– will always face the same issues of chaos vs. order).

But one thing no one tells you about SOA is that it removes the chaotic links, along with the simple, intuitive, human (bottom-up) responsibility. Programmers are no longer peers. The direct and human relations that used to be between two point-to-pointers have gone. Once a service gets published in the services' registry, anyone with the right permissions can get a description of the service and invoke it in a standard, well-known manner.

But see what is happening to our faithful, passionate and responsible P#1 & P#2. Previously P#1 knew whoever touches his code; he was, most probably, part of the project as well. He was the one that explained, trained and gave permissions to use his "precious". But now, the SOA wall is erected between P#1 and the rest of the world. Under an Enterprise SOA, P#1 is becoming insignificant, losing his political basis. A service that is actually hiding his code can now be used without any of his intervention, as the registry is holding whatever information needed for the world to know how to utilize the service. And once P#1 and P#2 understand their new political situation they automatically become careless. As if the service mediates, not just the implementation behind its WSDL, but also the humans resposnsible for it.

Suddenly, testing and production support of composite applications (i.e. applications that orchestrate existing [web] services) become highly difficult. The invisibility introduced by the service façade makes the QA team less capable of pin-pointing the exact code that's doing the problem. When they discover a bug, P#1 and P#2 are really hard to trace down. Sometimes they discover that P#1 and P#2 are no longer in the company. Same goes for production issues.

I am a great believer of SOA; I have designed an SOA Hub that serves 250 applications with hundreds of services and millions of service invocations a day. I am telling my stories, so Enterprise SOA acceptance will become smoother and easier. I believe the difficult times SOA is facing in Enterprises is partially because of all those things vendors are not telling their customers.

What I've described here is something we have greatly suffered from in the first year of SOA implementation. It took us some time to figure out why the QA teams are complaining about SOA; why operation teams are highly dissatisfied with their problem tracking tools. But once we understood that P2P is radically different from SOA – not just in its technical aspects, but also in its socio-political aspects - we have developed special training packages that touched specifically these issues and we provided a new set of life-cycle administration tools, that incprporated knowledge we would normally ask P#1 & P#2 for, thus allowing QA and operation teams to accomplish their own tasks, without chasing down our good old Ps.


Blogger gonen said...

hi muli . Nice blog . I will make time to read it , I'm sure i will be able to learn a lot .

amihay gonen

8:51 AM  
Anonymous Andrew S. Townley said...

Hi Muli,

I've been reading several of your posts after stumbling across your blog (from where, I haven't a clue at this stage). Anyway, I'm also involved in an SOA project at the moment, so I know where you're coming from. Ours isn't as big as yours, but the potential's there.

What I wanted to point out is you're dead right about the social dynamics of SOA not being the same as they were in the past. However, I think it's not too surprising when you think about the driving force behind SOA: the service.

Services cannot be written like "internal integration" applications anymore. In reality, they're "shrinkwrapped software" (see Joel on Software's Five Worlds essay: This implies exactly the sort of changes you're talking about, but most people haven't realized this yet. However, it affects us (SI) and it affects organizations building and deploying services internally into an SOA.

What I think we'll find is that eventually, more and more people are going to realize services are shrinkwrapped, not add-hoc. When this happens, everyone's life is going to be better, but it'll mean big changes to the way P1 & P2 do their jobs--exactly as you've described in your article.

Anyway, I've enjoyed reading your blog.



10:00 PM  
Blogger Muli Koppel said...

Hi Andrew

Thanks for your comment


4:54 PM  

Post a Comment

<< Home