Other posts in this trilogie:
- ESB's don't scale, which shouldn't be a problem because that is by design
- The demise of the ESB in a world of webservices
There are many reasons not to have an ESB in your environment. The ESB as we know it has less and less a reason to be implemented in today's world of webservices. There's a full post based on the dimise of the ESB in a world full of webservices.
But this post is about why you should have an ESB and why it does make sense to implement it. Or rather, when it makes sense to implement it.
Remember what the ESB was good for?
So what was the routing about? Well, it was all about having a logical name resolve to a physical location, where the ESB would determine where to send the message based on an address that one can remember, or that is not tied to a physical location. In the world of HTTP(s), which is dominated by the world of TCP/IP, this is actually handled by the DNS (Domain Naming Service). The DNS can perfectly handle this translation and do the routing. It's what is was designed for. Almost its raison d'etre. Maybe not even almost.
So in a world where we do SOAP based communications, there really is no reason to have an ESB.
What do we keep the ESB around for?
You keep the ESB around for the many-to-many messaging. You keep it around for those cases where you have one or more sender of (similar) messages and one or more recipients of these messages. Because the ESB is a great message dispatcher, it's the message routing part that kicks in. It's something the DNS can't do. In those cases where you want to send the same message to an unspecified amount of recipients where the recipients can register or deregister to receive messages at their leisure you'll want the ESB because managing this at the client side, maintaining a list of recipients and sending each message to each of the registered recipients is bad for performance, susceptible to errors and complicated.
- ESB's don't scale, which shouldn't be a problem because that is by design
- The demise of the ESB in a world of webservices
There are many reasons not to have an ESB in your environment. The ESB as we know it has less and less a reason to be implemented in today's world of webservices. There's a full post based on the dimise of the ESB in a world full of webservices.
But this post is about why you should have an ESB and why it does make sense to implement it. Or rather, when it makes sense to implement it.
This is a post in a series about the Enterprise Service Bus, ESB, an acronym so easily to love or to hate. Parts of this post are also found in another post on the dimise of the ESB in a world of webservices.
Remember what the ESB was good for?
- Data Exchange
- Data Transformation
- Data Routing
So what was the routing about? Well, it was all about having a logical name resolve to a physical location, where the ESB would determine where to send the message based on an address that one can remember, or that is not tied to a physical location. In the world of HTTP(s), which is dominated by the world of TCP/IP, this is actually handled by the DNS (Domain Naming Service). The DNS can perfectly handle this translation and do the routing. It's what is was designed for. Almost its raison d'etre. Maybe not even almost.
So in a world where we do SOAP based communications, there really is no reason to have an ESB.
What do we keep the ESB around for?
You keep the ESB around for the many-to-many messaging. You keep it around for those cases where you have one or more sender of (similar) messages and one or more recipients of these messages. Because the ESB is a great message dispatcher, it's the message routing part that kicks in. It's something the DNS can't do. In those cases where you want to send the same message to an unspecified amount of recipients where the recipients can register or deregister to receive messages at their leisure you'll want the ESB because managing this at the client side, maintaining a list of recipients and sending each message to each of the registered recipients is bad for performance, susceptible to errors and complicated.
The ESB is great at dispatching the same message to multiple recipients, doing transformations of the data on the fly. So duplicating a production load of messages into a test environment through the ESB, where the ESB is transforming the data just in time before sending it to the test systems (e.g. anonymizing the data, translating from one spoken language to another, transposing id's such that they align with keys in a database etc) is a common practice.
I have worked at a financial institution where we fenced the core banking systems with an ESB, catching every transaction send to the corebanking system and duplicating it to a standby system as well as a test system in a slightly altered form to be able at one side to have resilience at an extremely low price and production load with relevant data in a test environment.
Basically this means that you should keep the ESB around for being able to follow the concept of publish/subscribe. In my opinion the only right concept when it comes to asynchronous communciations. And since this can only be done by either using lower level network protocols like UDP (TIBCO's TIB/Rendezvous was built on UDP and the first real pub/sub message bus), or by having a central dispatcher sending a message to the registered recipients, or by handling it all at the sender side it makes most sense from a security as well as a manageability perspective to do publish/subscribe through a centralized dispatcher. Although for the really lean and mean high performance situations, you want to resolve to lower level network implementations.
Understanding pub/sub and implementing it in the enterprise allows for maximumenablement of the business with minimal IT investments. The afore mentioned use-cases of the monitoring systems where all systems publish monitoring messages (on business level) and various systems subscribe to them, for example a dashboard, and you'll get a very rich BAM (Business Activity Monitoring) solution, which without much additional effort can be used as an application level monitoring solution as well.
The other use-case where production feeds were used to implement improved availability as well as the capability of testing a new release with relevant production loads is of course pretty awesome as it didn't need a lot of additional work once the ESB fence was set up.
So dispite popular believe, there is a reason why you should consider implementing an ESB in your enterprise. But don't adopt it as the default means for communications between applications, instead apply the ESB as a niche product and not as mainstream.
Basically this means that you should keep the ESB around for being able to follow the concept of publish/subscribe. In my opinion the only right concept when it comes to asynchronous communciations. And since this can only be done by either using lower level network protocols like UDP (TIBCO's TIB/Rendezvous was built on UDP and the first real pub/sub message bus), or by having a central dispatcher sending a message to the registered recipients, or by handling it all at the sender side it makes most sense from a security as well as a manageability perspective to do publish/subscribe through a centralized dispatcher. Although for the really lean and mean high performance situations, you want to resolve to lower level network implementations.
Understanding pub/sub and implementing it in the enterprise allows for maximumenablement of the business with minimal IT investments. The afore mentioned use-cases of the monitoring systems where all systems publish monitoring messages (on business level) and various systems subscribe to them, for example a dashboard, and you'll get a very rich BAM (Business Activity Monitoring) solution, which without much additional effort can be used as an application level monitoring solution as well.
The other use-case where production feeds were used to implement improved availability as well as the capability of testing a new release with relevant production loads is of course pretty awesome as it didn't need a lot of additional work once the ESB fence was set up.
So dispite popular believe, there is a reason why you should consider implementing an ESB in your enterprise. But don't adopt it as the default means for communications between applications, instead apply the ESB as a niche product and not as mainstream.
As always, I'm really interested in your views and ideas. More perspectives make for a richer understanding of the topic. So please share your thoughts, stay respectful and be argumentative.
Iwan