API Management in Azure using Microsoft's solution - Resources and Templates [2/2]

This is a series of posts regarding the topic of API's and API Management on Microsoft Azure. Not all posts in this series are directly related to Microsoft Azure and their implementation of API Management, that is intentional. The series also explains about API's, about creating API's and about what it construes to in order to manage them, conceptually and in reality. The reason for this series is that over the past 12 months I've come across many articles on the web, have been in many discussions and advised various clients of mine on this topic. Sometimes discussing with fellow architects, other times with vendors, still other discussions where with developers and managers. But the overall situation has always been that none of the people at the other side of the table had a full grasp of what developing API's in an organisation means, what it entails to manage them or what should be worried about when deciding to go API. I hope you enjoy reading these articles, and when you feel like it, comment on the articles. I always take serious comments serious and will reply to my best effort.

This post is the last post in of a series on API Management using Microsoft's solution.

API Management is not the same as an API Manager. In short, API Management is what you need to do with your API's, and an API Manager is something with which you can do this. There are a lot of different solutions that allow you to implement API Management, one of these is Microsoft's solution on Azure. The tricky part is that when you start reading their documentation on API Management you'll read little about how to actually implement API Management in your organization using their solution. It's all very conceptual. This shouldn't be a problem since the concept behind API Management are more important than the actual implementation… until you want to actually implement it.

Read the previous posts on the topic to learn more about the concept, continue reading this post to learn more about how to use Microsoft's solution in Azure, their cloud platform.

Resources and Templates

Finally, resources and templates, the bricks and mortar of the cloud. In the cloud you're typically dealing with resources. An infinite amount of resources, or at least that's how it feels. Everything in the cloud is, or should be a resource. Some are very basic like computing power, storage and networking. Others are more comprehensive like database, firewalls and message queues. And then there are a quite a few that are truly complex and very useful on a high level, for example directory services.

The cloud, being what it is, is like a normal infrastructure on which you host something that generates value for your business. Hence everything you need to run business applications in a traditional hosting environment, you also need in a cloud environment. Obviously there are significant differences between traditional hosting platforms and the cloud, but when you don't take too close a look, you're likely not to see these differences.

So in the cloud you also need to define systems, attach storage, put a firewall in front, put connectivity in place etc. You can do this by hand every time you need an application's infrastructure. Typically through a portal by clicking your way around and assemble the infrastructure. But more sophisticated and a way better practice, is to define the infrastructure in a text file, typically JSON for most cloud platforms, and use the cloud vendor's tooling to create the infrastructure based on this file. As such, the file becomes a template for a specific infrastructure setup you need. By providing a parameter-file you can externalize specifics of the infrastructure. For example the URL's to locate a web-service can be defined in this parameter-file to distinguish between an infrastructure intended for testing and the same infrastructure intended for production runs.

The particular template is called a resource template, it defines which resources are needed and how are they specified in order to run a business application.

One of these resources that you can use is an API manager, just like you can specify databases and virtual machines as resources. And here's your challenge.

The challenge is in that an API Manager consists of three parts;

1. Developer portal, used by developers to find your API's and their documentation.
2. Publisher portal, used by API developers and the likes to manage the API's.
3. Gateway, used by applications developed by those mentioned above in 1 to access API's managed by those as mentioned above in 2.

Each of these have their own context and is used by a different group of 'users'. The real interesting part of the API Manager is the API Gateway as it is the component that exposes the API's you've been developing. This is your product. It is the resource that is part of your software. And the thing is; it can be shared or limited in scope to just the product you're developing.

Ideally you would have one gateway per product, because the gateway and particularly the API's it exposes, are part of your product and as your product evolves, the API's that come with it will evolve as well. And of course you would want a consistent life cycle across all components that relate to your product. Since the API gateway is just like any other resource in Azure, the above is perfectly doable. In fact, it is possible to include the API gateway as part of your product's resource template and when you provision the relevant infrastructure and deploy your product on it, the API gateway is provisioned as well.
Pretty awesome, when you're willing to forget that the costs of an API gateway are pretty steep. We're talking about close to €2,5k / month. There's not really a price based on usage. Microsoft is really weird in that when it comes to pricing in the cloud. That whole pay-per-use is not really everywhere in their pricing schemes. I like Amazon better in that regard.

So an API gateway per product is not really an option in most cases I would argue. Instead, I would advise to have a gateway per product suite. In case you have teams that handle multiple products, scope the gateway to such teams, or otherwise scope the gateways to department. Use it as a rule of thumb though and not as the law.

The point here is that you want to be able to have your API's evolve with your products and that you want teams to be as independent of each other as possible. But in addition you want your API's to be operated independent of each other. And this is important. In Azure, API's don't scale, it's the gateway that scales. And you want to be able to be smart about that. Especially when it comes to policies and usage tracking or rather generating value from API's being used. When a team is responsible for the success of its products and therefore the value that is being generated, it becomes obvious that that team would want to be able in control of what is affecting their success.

The alternative would be to work with an SRE approach, where you have a team that's responsible for your platform, including your cloud platform(s). This team would then realize the API gateway for the other teams as a service. The catch here is that this platform team decides where your API's are 'hosted', or rather whether or not you share the API gateway between teams or not. Unless your platform team is really committed and more importantly has a thorough understanding of what API's really are and I mean really understand this, I would advice against this approach. I would oppose it for the sole reason that your API's are the window into the soul of your organization. When the API is not performing well, your organization is not performing well. And especially when you're going API first, and thus build a platform, you're screwed without proper API management.

In case you do decide to go the platform team route, make sure that your processes are completely automated. Including the deployment of new API's as well as new versions of existing API's. My preposition here is that you'll be working agile as can be, deploy to production as soon as you're confident that your software is up to it. Meaning that new software needs most likely new (versions of) API's. Don't make the platform team a bottleneck, so make sure that you're working with them to deploy the changes API's consistently, repeatable and consistently. Better to abide by their rules then put your own in place. Drop the whole platform team approach when they're not providing a 100% automated deployment process for your API's.

Then there's the portals. The developer portal is a tricky one because it provides access to your API's from a developer perspective. You should be really nervous when you're nervous about potential unwanted developers nosing into your API registry. Because it means your security is way, way, way below par. Remember, API's are different from regular services in that they are build such that they make no assumptions as to who accesses them. And unless you've build them that way, you'll be in for some really serious security challenges. That said, there's no reason why not to have different portals for developers within your organisation and developers from outside your organisation. And have API's exposed only to internal teams and publicly exposed API's. Just make sure that this is an exposure aspect and not, I repeat, not an API implementation aspect.

So develop API's as if they're possible accessed by just anybody and their mother. Expose API's to a subset of this group.

Then there's the operational aspect of your API captured in the publisher portal. Here you should take an approach that only the team responsible for an API should have access to the API from a management perspective in an operational environment. In other words, access to an API's policies is for the team that 'owns' the API only. You'll need to take care of that. period.

Mind that Microsoft is rapidly changing their API Management service on Azure. Most likely. as I type this, they're making life easier for you on using the service in your organization. The concepts as I've described still hold though. And as Microsoft will hopefully come to realize that also for API Management a pay-per-use model is the way to go, you'll be able to treat API Management as part of your product instead of as part of your platform.

This concludes my series on API Management using Microsoft's solution. I hope you did enjoy it.

The complete series:

• API-first design
• API's and Architecture, match made in heaven 
• Why API's should be managed and by whom
• Resources and Templates [1/2] 
• Resources and Templates [2/2]

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link to my blog to all your Whatsapp friends and everybody in your contactlist.But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

The Arc-E-Tect's Predictions for 2017 - API's and Webservices [2/10]

The Arc-E-Tect's Prediction on API's

It's 2017, meaning 2016 is a done deal, and most of my predictions for 2016 (I predicted them about a year ago, but never got around documenting them in any form) have proven to be absolutely bogus. Which leads me to come up with my predictions for 2017 and properly document them. So continue your effort reading this post and hopefully you'll enjoy reading them. Unfortunately we'll have to wait for about a year to find out to what extent I got it all right, but for now, ... API's!

Why API's? Well, API's are all the rage and everybody and their mother is working on platforms, and as we all know; Without API's there is no platform.

API's in, Webservices out

Okay, in 2017 we'll feel ashamed when we talk about web-services and SOA. Instead we'll talk about API's. This is closely related to my first prediction on Microservices, which you can read here.

API's are basically another word for webservice to many people. So there's not much different here. Just as with the Microservices, we'll see mentioning of API's more and more where in the past we talked about webservices. Until people are referring to platforms, something that is really picking traction. Still I'm not talking about platforms, but about API's instead.

The reason for this is their strong relationship with Microservices. Delivery of a platform is a strategic decision that defines the direction in which an organisation is thinking about products. API's, which expose the functionality of a platform, are products. You can read all about this in my series on API Management on Azure, which you can find here. Other than web-services, which are pieces of functionality and/or data exposed via a well defined interface, API's are always targeted at an external consumer of the service. In other words, an API never knows who's calling nor does it make any assumptions about who's calling. Web-services on the other hand might very well be limited to a known set of consumers and make assumptions about their consumers.

The promise of web-services, predominantly a decoupling between functionalities in an IT landscape, is limited to those case where the web-service, or rather it's interface is treated as an API. API's, almost by definition, are bound to deliver on the promise of decoupling. When developed properly, taken care of and fronting independent pieces of software, API's are the closest thing to silver bullets in software we've come across so far. And since we love silver bullets, API's are in, web-services turned out to be just plain old regular bullets, so they're out.

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link to my blog to all your Whatsapp friends and everybody in your contactlist.But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

The Arc-E-Tect's Predictions for 2017 - Microservices and SOA [1/10]

The Arc-E-Tect's Prediction on Microservices

It's 2017, meaning 2016 is a done deal, and most of my predictions for 2016 (I predicted them about a year ago, but never got around documenting them in any form) have proven to be absolutely bogus. Which leads me to come up with my predictions for 2017 and properly document them. So continue your effort reading this post and hopefully you'll enjoy reading them. Unfortunately we'll have to wait for about a year to find out to what extent I got it all right, but for now, ... Microservices!

Why Microservices? Because they're not just a hype anymore, but are moving into the realm where people are applying them because they're useful and not just cool. So my prediction is:

Microservice in, SOA out

That's right. In 2017 people will start looking at Microservices as something that is useful and way better to have in your architecture than services. So a Microservices Architecture will replace Service Oriented Architectures in 2017.

What does this mean?

Well not that much at first, until one realizes that an important reason for SOA, the DRY principle is no longer valid. DRY, Don't Repeat Yourself, is an important aspect of procedural programming that got more attention in object oriented programming and became one of the even more important aspects of SOA.

In procedural programming developers introduce procedures to have a single implementation of an algorithm or procedure. By calling that procedure over and over again instead of programming it over and over again, the developer reduces the chance for errors in the code.

Then came object oriented programming and procedures, methods, were clustered together in classes to ensure that concepts and their behaviour only had to be programmed once and used often. It's a flaky definition, but covers my intention. So re-use was immensely important for object oriented programming and this is where DRY became an important architecture principle, even a pattern.

After some time, developers started moving towards services, web services that is. In OO, Object Oriented, there's a great practice to separate the interface to a class' methods from their implementation. Depending on the programming language you use, you'll use a variety of techniques to accomplish this. In an SOA, and yes, I'm a bit short on the elaboration here, we separate the implementation of a service from the interface to access it. But the whole idea is that you implement certain functionality only once, the service, and use it from around the world. By sticking with the interface you can change the implementation without affecting the callers of the service.

So in SOA, Service Oriented Architectures, re-use is huge and DRY is a pattern.

Here come the Microservices. Microservices are not tiny services, instead they are completely self contained pieces of business concepts. They're independent of each other. They can use each other, they can rely on one another, but they don't depend on each other. That's a key principle of Microservices. It also means that if both use the same algorithm or other functionality, they both implement it. Meaning that the code is not reused! Because if one would reuse parts of the other, it would depend on it. So, in a Microservices architecture, DRY is an anti-pattern.

In 2016 everybody started doing Microservices, but actually they implemented an SOA and called it a Microservices architecture because SOA was, well, old school. An SOA has become something tainted because for one most enterprises confuse an SOA with an architecture build around an ESB and they're strengthened in their belief this is correct by the ESB vendors. Which is of course just marketing horse dung as you can read here. In addition, SOA's have hardly ever delivered what vendors and architects promised: Agility, lower costs of change and independence of clients (consumers) and servers (producers). Why? Because ESB's and by association SOA's make architectures more complex and harder to change although they hide their complexity inside the products, making them look less complex. But ask yourself: Did you ever try to migrate from one ESB (version) to another? 

So we drop the SOA and brought in the Microservice to attract new developers.

Fortunately, Microservices are hard and costly to develop and maintain. From day one. Therefore all adopters of this new and shiny architecture had to reconsider their choice to move to this new architecture at a very early stage. Realising that understanding is more important than knowing and none of the vendors had a ready out-of-the-box Microservices solutions that they could sell, the level of knowledge and understanding is already in 2017 such that Microservices will start to replace SOA's.

In 2017, companies will drop service oriented architectures in favour of Microservices architectures instead of implementing SOA's with Microservice look-a-likes.

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link to my blog to all your Whatsapp friends and everybody in your contactlist.
But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Iwan

When Scrum needs to mature in the enterprise, architecture comes to the rescue.

This post is the consequence of a comment on my previous post that startled me as it stated that in Scrum there's no place for the architect. Scrum has no future were it not for the architect.

First of all, I am not a fan of Scrum, mainly because it sounds too much like an acronym and I hate acronyms. The other reason why I don't like Scrum is because it entices too many religious zealots to start a jihad against all non-believers. And then there's the third reason, the reason why I don't like Scrum is because too many so-called Scrum practitioners use it as an excuse for not going for longevity and quality of their deliverables.

So, with that of my chest, let me tell you that I really love what Scrum stands for. That whole thing about delivering something useful to who ever pays you for building it as soon as possible. Allowing for the customer to change his mind constantly about what is important and what not. And the very notion that sometimes you do something and than you have to redo it but differently is part of the deal is excellent.
Scrum is awesome and it addresses a lot of very significant aspects of old-school project management. Especially when it comes to long analysis phases, and even longer design phases and excruciatingly long development phases, Scrum has introduced a lot of benefits.

One of the main reasons why Scrum is such a success from an adoption perspective is the fact that it has been introduced in the limited scope of development project teams. And then in most cases those teams that had to face a lot of changes in requirements and priorities, namely front-end, user facing systems. Systems that directly addresses the needs of an end user.

As the common enemy called "customer" united all developers, Scrum thrived.

The rebellious attitude introduced with Scrum appeals to many developers, and I don't mean any disrespect to developers. I am a developer and often I am my own customer, or in Scrum terminology, I am my own product owner and I suffer from changing priorities and requirements all the time. It's part of usable software.

In many organizations these days, development projects are done with the Scrum manifesto in one hand and a sincere lack of documentation in the other.Here lies already a big problem, namely the fact that in these organizations people, even Scrum zealots, still talk about projects. In and by itself it is impossible to do projects using Scrum. The very notion of projects is preposterous, but that'll be covered in another post sometime.
These enterprises have introduced Scrum in their software development projects and those organizations that are a bit more mature have introduced DevOps, which is the second stage in Scrum-olution if you ask me.

Havoc is wrecked when a Scrum team is developing software, functionality that depends on the deliverable of another Scrum team. How are sprints aligned? How are interfaces co-developed, how are release dates, even in continuous delivery situations, coordinated? Scrum, nor DevOps, can answer these questions. The reason for this is that the scope of Scrum was never intended to go beyond the activities of the Scrum team. Inter-scrum-communication is, well not existent.
Of course it exists, because it happens all the time. Initiatives like meta-scrums, meta-sprints and meta-other-Scrum-buzzwords are popping up all over the place.
The fact is that once you look beyond the scope of sprints and epics and what the Product Owner is looking at from a back log perspective, there's not a lot that you can't can do without the quality and talents of an architect. It is always the architect that delivers the bigger picture.

Scrum is, I believe, something that stems from the sport rugby. And if not, than still for the purpose of this post, it does. And where the Scrum teams play the game, it is the architects that define the playing field as well as the rules by which to play. Mind you, Scrum does not equal developer's anarchy. It allows for the freedom of the team, if suitable, to apply a pragmatic approach to realizing the product owner's products. But the freedom is within the limitations set forth by the architect in terms of policies, principles and standards. Because of the architect, features integrate and systems can communicate. But more importantly, it is architecture that allows for cohesion and consistency between different applications and services, thus ensuring for example compliance to laws and regulations. Especially in environments that are predominantly based around SOA (WS* and REST both), it is imperative that policies and standards are defined and adhered to in a consistent and cohesive manner in order to be and stay compliant.
This becomes very apparent when principles like:

• "One Version of the Truth"
• "Re-use of business logic"
• "Role based Access Control"

Are to be followed. Without an architecture framework, it becomes extremely expensive from a governance point of view for an organization to enforce or even ensure adherence.

Do you need an architect in a Scrum team? Probably not as long as you've got one or two members in your team that are taking care of the relevant design work, whether or not implicit to the Sprints deliverables. But in the bigger picture, there is definite need for an architect, especially when you not only want your development teams to be agile, and capable of adjusting to the whims of the product owner but also your whole business to be agile and capable of adjusting to market demands.

Thus, unless you want to keep Scrum to be Waterfall's baby brother for ever and ever, you need to mature your agility and that is only possible by applying architecture and involve architects. But not so much within Scrum teams, but across these teams.

A Bottom-Up approach to Maintenance and Support doesn't make any sense

First of all, please note that my view on IT is that it services the user and when used in an enterprise, it services the enterprise. Typically we refer to the enterprise part that IT is servicing as 'The Business'. There is no other raison d'etre for IT other than to service.
With that being said, it should come as no surprise when I say that I consiser IT to be a tool. It's a tool that is supposed to be utilized.

With that out of the way, let's move on to the topic of Maintenance and Support and why it makes no sense to address this bottom up.

First of all, what is bottom-up in this context. For that we define the stack that an IT system typically consist of. At the very bottom of the stack we have the communications infrastructure. This is the Network. It is somewhat an odd-ball in this discussion, as the Network is not limited to a few systems. But for argument's sake, we start with the Network, the communications layer.
Next we have the hardware layer, this can be physical or virtual. But in case we're talking virtual, it does make sense to separate this layer into two sub-layers. For the purpose of this post, the division is not relevant.
On top of the hardware we have an operating system, and with that the rest of the system infrastructure. Think about anti-virus, system firewalls, etc.
Next we have the middleware, or rather we have software engines. Think in this regard about application servers, database servers, webservers, messaging servers etc. These are not all considered middleware, but they are engines. Generic pieces of software that provide specific services to application specific software.
Which brings me to the next layer, the application software. This is in fact the software that provides the services that the business benefits from.
Finally, there're the business processes in which the applications play a part. As with the communications layer, this is sort of an odd-ball. But within this post it is in fact relevant.

Now looking back at the start of this post, IT services the business. And the business is in fact defined by its processes. Without processes there is no business. Mind that the processes do not necessarily need to be formalized or even repeatable. I'm just saying that the business is defined by how the various actors interact, these interactions are processes, the business processes.
Consequently, when the business can't execute its processes, it doesn't function. Nothing gets doen. When it is impossible to execute the business critical processes, the business seizes to exist.
Thus, from an architect's perspective it becomes critical to understand these processes, or at least their criticality, and the demands regarding the processes to be executable. Hence, and now we're getting close to where I want to be, the architect needs to understand which steps, which actions in the various processes are automated. This is where IT kicks in!

Now you understand that an architect should worry about the availability of the capability to execute a business process and an IT architect, should worry about the automated parts. These are the business services, or in fact the applications. (Yes, I know that I'm simplifying this a bit, but within the context of this post, that is fine.)
So, (parts of) applications need to be available and the data used in these applications need to be secured. I'm saying parts of applications because more and more we see applications to be composed of reasonably disparate parts. What is 'available'? It means that a business service, an application's functionality, is accessible and once is started it is concluded within an acceptable timeframe. This is important, because when the system does execute but not fast enough, it should be considered unavailable.
What does it mean to secure data? Well it means that the data has to be available, can't be tampered with and can only be accessed by those that are supposed to have access.
We typically define these by using KPI's, typically RTO (Recovery Time Objective, i.e. how long till a service is available again), RPO (Recovery Point Objective, how much committed data can be missed) and an up-time defined in an amount of time the service can be unavailable over a period of time.
These are all business requirements, all to be defined by those that can understand what it means when a process can't be executed.

We're getting close to where we need to be, or rather where I want to be with this post. The topic of Maintenance and Support. Once a process is being used, it needs to be supported and maintained. For IT systems, it means the same. These systems need to be supported and maintained as well.
The roles that need to maintain and support are dividable in three areas: Functional, Application and Technical. Basically it's about the people that understand what the system should do, which services it should provide. Functional. The people that understand how the application works and how it can be kept working. Application. The people that understand how it all actually runs on the systems. Technical.

Let's bring in the animal kingdom, shall we; There are different ways to skin a cat. By this I mean that there are different ways to keep a process executing. And that's the whole point! The process needs to be kept executing. This is what maintenance and support is about. Full Stop.
By realizing this, it becomes clear that from a functional perspective it must be defined in what circumstances what needs to be done to keep the process executing. And in the cases IT is needed, it must be defined what needs to be done.
But that's not the point, the point is that those KPI's that are defined, the RTO, RPO, etc are regarding the business service. Explicitly not, and I emphasize this, the application or the infrastructure or the network. This is what needs to be managed. So in order to provide in the requirements regarding this, it must be implemented at the top of the stack. And then you go down the stack realizing these requirements. Just like any other business requirement.

Talking contractual issues, the SLA and OLA, is the exact same issue. The SLA is defined on a business process layer, the SLA defines to what extend the process can execute and then trickles down. Down the stack, to ensure that it can be done. And the OLA's are there to ensure that everybody is on the same page and commits to help meeting the SLA with everything at their disposal.
It should be clear that it is rather pointless to define and agree an SLA on a lower level in the stack when that doesn't support the requirements higher up in the stack. It would be a waste of money as the enterprise will still falter when poop hits the fan.
Again, it's the same as with all other requirements, you don't build something that doesn't help doing business.

Agreed, my post title should've been "A Top-Down approach to Maintenance and Support is the right way" as this is what I'm discussing here. But I choose to be a bit controversial here. Why? Because too often I see that the bottom-up approach is taken. Enterprises consistently fix availability at the infrastructure level. And genuinly believe that this is cutting it. Not realizing that it doesn't. Furthermore, they invest extensively in expensive IT solutions that are typically complex to maintain and support. And because of this, there's rigid standardization enforced to keep costs down.
Consistently, enterprises are solving business problems that are not conceived as business requirements, we actually call them non-functionals most of the time, using technology. Preferably hardware, virtualized.
This is counter-intuitive because any other business requirement is actually addressed top-down.

One of the reasons for this behavior is that 'the business' doesn't think of how important a business process is, how important the automated activities are, how valuable the data is. When asked for the RPO and RTO it is typically conceived to be a technical issue. Typically the answers to RPO and RTO are that they need to be '0', i.e. no downtime and no data loss. Of course this is incorrect in all circumstances, because the RPO and RTO are to be defined on a business service level and in pretty much all cases it needs a significant amount of analysis to come up with the real numbers.

So, yes, the title is correct. Why? Because we keep on doing it the wrong way. We keep on messing up. We keep on spending money where it doesn't need to be spend. And we keep on not delivering what is actually required. Why? Because thinking about it and get to the real answer is actually hard and just throwing more kit at the solution will seem that it will fix the problem.

BYOD - Bring Your Own Disaster

Just to be clear, it's not a typo.
First of all, let me explain something. Over the past year or three I have heard a lot about BYOD and in most cases the acronym stood for Bring Your Own Device. And although I've been bringing my own device since very long, since about 2003 at several customers it has always been a problem to actually use my own device.
So yes, we've been able to bring our own devices, but actually using the device at our job doesn't really go beyond reading your email and accessing your calendar. Granted, sharing contacts between your device and the corporate is there as well.
I've had discussions with many subject matter experts from the leading software vendors regarding this topic (IBM, HP, CISCO, Microsoft), and interestingly enough the struck me that although they all have their tools ready to be sold to their customers to facilitate a BYOD policy, within their own ranks their policy was in all case more a matter of CYOD, or Choose Your Own Device.
Okay, so let's see what we're talking about here. Within the corporate environment there are basically 3 models possible (mind these are models):

1. Everything is provided by the enterprise. So you get your IT equipment from your employer and he takes care of everything, including payment of the hardware and its maintenance. You work in the office and that's where you keep your stuff. When your employer decides that you should be able to work from home, you get a laptop and you can work offline from home. Or in fact you can log into the corporate network using a VPN so you can access your files and maybe even your applications, although they're in most cases installed on your laptop. 


2. You get all the computer equipment you need and your employer handles everything. But because laptops are more expensive than desktops and because laptops are also harder to maintain, because you never know when they're available for maintenance, you're allowed to work from home on your own personal desktop and in case you've got a laptop... well that's even more practical as you can work from any location from which you can setup a VPN connection to the corporate network. Most likely you've installed some software yourself and by means of a USB drive or some other external storage. Hey, maybe even dropbox! You just have to make sure that you're running some OS that is supported by the companies admins and meanwhile you keep the anti-virus and other such software on your system.


3. Then as a last model, there's the option to use whatever device you have to connect to the corporate network and do whatever it is you do to accomplish your tasks. You connect to your employer's network through the internet, either via a wired, a wireless or a mobile connection. (Yes, I know that a mobile connection is also wireless). In this last model, you bring your own device and not so much your own computer. There's a big difference.

So now what's the relevance of all this. Well mainly it's got to do with risk. There are a number of different risks involved here that are less relevant in model 1, a bit more so in model 2 and significant in model 3.
So let's cut to the chase and do some business here. As we want to get onto the Disaster part of BYOD.

One of the more obvious risks of course in these models is the risk of data leakage. Both intended and unintended. With the topic of data leakage you're also bound to include leakage prevention and that is widely considered a security aspect of an IT environment. Apart from technology that can be applied, there is also a lot of organizationals aspects to this. I'm not getting into this. Not in this post.

The thing about Data Leakage Prevention from a technology perspective is all about control. Mainly control over where the data is stored and where data is communicated from and to. Basically, if you control where it is stored and you have full control over this location, you can just prevent anybody from accessing it and there's no risk of leakage... in an ideal world, I know. It also means that you control when it is accessed and if you control the communication lines, you also control what is communicated. Basically this is the strength of model 1. In model 2, you have less control over where it is stored and when and how data is communicated, but there are still plenty technologies and solutions available to apply sufficient control at an acceptable cost to prevent, to a degree, data leakage. A weak link in this chain is the fact that it is fairly simple to loose a USB drive and with that data. Whether intentionally or not.
It gets a bit more interesting when working on your own device. Yes you can loose that tablet or smartphone, but there's another thread that's more significant on your own device than it is on your own computer and that's the thread of malware.

The interesting part of devices is that they are technically not really mature. New devices are popping up every few months, new operating systems, new capabilities etc etc. And in addition to this, their storage is almost limitless, by that it is very limited. So most people that use devices also use cloud solutions for storing their data, in many cases the corporate's data.

But this is all just a matter of potential compromises and risk calculations. Yu accept the risk, there's no problem. But most companies can't handle certain data to get out on the street, or at least that's what they think.

Like I said, it's all risk based and security threads should be looked at from a risk perspective. Looked at from the perspective of what are the chances that it happens and what is lost when it happens. This requires a certain level of organizational maturity. A movement that has just started, hence not many organizations are ready to handle security in this way. Oh yes, they may have completely adopted the risk-base approach to security, but they're far from having classified all the threats and defined the proper counter measures to avert the threats or mitigate the risks. Thus 'Lock down' is still what happens, until we're ready, is what's being said.

Yup data leakage is a problem and with your own devices this is even more of a problem but predominantly because we want to lock our data down because we can't act on the security risks from a threat-classification perspective. So we put a lock on the data, which means we put a lock on the devices. And since it's your own device, your boss is putting a lock on your device... in vain in most cases as either the technology is not adequate or is not yet developed for your device. Ah, let's not forget about the variety of all devices. All the different PC operating systems, the various mobile platforms, the various versions of each platform, the capabilities of devices and the variety of ways they can connect.

The disaster kicks in when the CSO (Chief Security Officer) and her underlings are confident they're on top of it. Because they never are. And at the first breach of their defenses, typically moments after a device is brought into the organization, you've Brought Your Own Disaster. Mind that the breach doesn't have to be a disaster in itself, but a breach of security in an environment where threats are dealt with in throwing technology at them and lock things down, ... well you get the drift.

Data Leakage is interesting, but there's more to the disasters that can happen by Bringing Your Own Device.

Think about compliance. Who's going to pay for all the licenses of the software installed on your devices? And who is going to ensure that you work with the right versions or even applications? Even though there are open formats doesn't mean that they're completely embraced by enterprises. In fact, I know way more organizations that keep all files in proprietary formats like Microsoft Office formats, the old ones, instead of the open XML based formats. And this is just an example of programs for which there are open formats that are supported by free (as in free beer) programs.
Since it's your own device, you're supposed to get the licenses yourself in many cases. And why would you pay for them where you can get them for free (as in illegal downloading free) from the Piratebay? Btw, I don't condone piracy.
Yet, since you use those programs for your work and when in the office even within the corporate's network, there's a compliance issue.

Again, people responsible for the various security aspects in the enterprise, including compliance aspects want to take the risk-based approach to this, but until this is in place, they revert to technology to ensure that you won't have any non-compliannt, read pirated, software on your device. There's an interesting aspect to this; Typically there's a policy in place that's supposed to prevent you from putting pirated software on your device. This policy is defined in terms of technical counter measures based on some tool that will probe your device and delete everything it deems not in accordance with the policy. Of course this can only work when you device supports this, in every aspect, including you allowing it to support this.

Again, by putting pirated software in the office by means of bringing your own device into the office and use it to do your job, disaster has been brought to the office. True, this is also not always resulting in your employer's bankruptcy or something even more serious.
Actually, to be honest, the Disaster in Bring Your Own Disaster is not referring to the problems you're causing with your own device, it's the disaster of managing these devices, which is something that is just impossible at this point in time with technology, considering the multitude of devices, types of devices, operating systems, platforms, capabilities etc. It can only be handled by having implemented risked-based security. Which is something that requires a lot of organizational changes, and change in mindset as well.

This is why most enterprises are actually reverting to a Choose Your Own Device strategy. Out of a pre-selected and manageable set of devices ranging from phones to tablets to desktops and laptops, you're allowed to pick the ones you want to use, as an extension of the office.

Thanks for reading, and as always, please let me know your thoughts about this topic.

Iwan

The Awesomeness of the Process

Hi,

It's been a while, but here's another blog post on IT Architecture and anything that comes close.
I remembered today while waiting on the train to go home, after a meeting in which we, a bunch of architects, came to the exact same conclusion as we've done several times in the past two or three weeks. Namely, it's all about the process if you want to do it right. Check my earlier posts and you'll find out that I actually think that the process is significant.

Anyways, I was waiting for the train and while reiterating what was said during that meeting and afterwards made me remember what one of my professors told us in either the 2nd or 3rd class on databases, or rather on Information Analysis:

"Always remember that you should only store in a database that what you want to get out".

Because we used NIAM as the modeling technology and this was in the earlier nineties, this made perfect sense because in those days reporting systems were the most important IT systems with SAP and BAAN dominating the ERP arena (I know I'm not doing any justice to either one of them) and NIAM is extremely about reporting. (Again, I know I'm not doing it any justice by stating it like this.)
Okay, so lets get back to our meeting today, well actually, let's get back to my opinion of processes. I'm honestly a big fan of Business Process Management, mainly because they define a business and they make IT worth our while... unless you love playing games because IT always makes computer games possible.

The main part about BPM are the processes. The interesting thing with processes is that they, when defined correctly do exactly define what the sequence of activities is to get something done, who is responsible for each single activity and who is accountable. And, and this is where it gets interesting in the context of this post, what is needed to start and do the activity and what the outcome is.
The beauty of this is that it allows you, if you keep this in mind, to ensure that any actor for any activity only gets the information she needs to perform her task. Especially in those activities where somebody needs to review, accept or verify something this is extremely valuable as it ensures that she is not overwhelmed with large complex documents.

I have worked in many organizations around the globe in a multitude of different verticals and markets. Always there was some sort of process in which one or more entities within the organization had to accept either a document, an architecture or some code if not an application that was to be put in production. The hardest part in these processes was to convince people that it was perfectly fine for them to receive more information than they needed and that you were relying on their expertise to filter out the information they were looking for. This because the document they were handed was a living document and as the process went on, it would grow and mature.

Of course this is complete bullshit, living documents don't exist and since they're written, or supposed to be written by a variety of people with different views and backgrounds the quality is poor at the best in most cases. But there're many reasons why we, including myself, are settling for the one big document. For one, it always does seem to be convenient. Just one document to be maintained, one template to be developed, one artifact to be communicated.

Then there is the reason that we're all very poor in defining requirements, even requirement engineers, so we find it very hard to define what we exactly need to do our job, how we need that delivered and in what format. So the document that holds everything is very suitable.

Working on a process and defining all activities,mother responsible and accountable actors as well as the input and output for every activity can be very boring or at least seem to be very academic. A paper exercise, and a since we all love the tangible, well all the sane people I know, we are easy to digress and move to the concrete with documents and diagrams. Letting go of the process and concentrate on the artifacts. Remember, we tend to fix problems in the infrastructure instead of the application, mainly because we can touch it and hold it and it is easy to budget and plan. Not because that's the best place to fix the problem.

This is more so for the managers of the departments that are impacted by such a process. When they're not aligned and don't define the areas of their responsibility and accountability, preferably without overlap and without gaps. Yup there's a challenge. It will be very hard to point out who is acting in what capacity during which activity.
Management buy-in is a prerequisite even to start working on the process. And with proper alignment of the management, it will become a simple as life exercise to define the process.

There is the problem of the man documents to be managed, or rather the many products to be managed. Some are input and some are output. But when you keep in mind that output always has to be input for some activity, you can state that there is always only input to be considered. But that brings with it, that you need a rock solid system for managing all these products. And no, a file-server or email system doesn't cut it. What you need, at the very very least is a collaboration platform that has document management in it, but preferably you have a Business Process engine that manages your process. In this context I don't mean a document management system that allows you to define a document flow. No sir, I mean a process engine that allows to attach products to the process.

Now forget about the bureaucratic formalities of a business process and documents and templates and responsibilities and accountability and actors and all that stuff. Instead think about your job. Your role in the organization you work for and the activities you're supposed to do. Wouldn't it be awesome if you would be provided with all the information and all as soon as somebody would ask you to do something? A situation in which you would know exactly what information you're supposed to dig up yourself to do your job and what information you need handed to you? At all times?
Well if your answer is "YES" than you want to experience the awesomeness of the process, because this is exactly what it is about.

And it's not just a hollow promise. No, it's something that comes out of the box when you define the process not based on the information that can be provided, but what has to be provided. When you ask the responsible person what he needs to complete an activity instead of just throw all you have at him and have her figuring it out herself.

One last thought on this topic before I conclude; You never define a process that is perfect, you perfect a process that is defined. So it is correct to think seriously about what the process should be and than adjust it as you apply it. Basically the idea behind Lean.

I hope you enjoyed reading this post and think you can benefit from it. Don't hesitate to comment. As always, different views paint a clearer picture.

Iwan

Is there a market for the Cloud in the world of Corporates?

(Disclaimer: This is by no means a "definitive guide")

Abstract

I really think that most corporates are on an enterprise level not ready or even suitable to embrace the Cloud. It's the SMB that should embrace the Cloud as it would benefit significantly from Cloud offerings. Why? Because the cloud cannot be part of a policy, it should not be a strategy. Furthermore, corporates tend to only centralize infrastructure architecture and decentralize application architecture and the Cloud is driven by application architecture and business architecture and not (so much) by infrastructure architecture. In fact, it requires you to abandon control over your infrastructure and hand it over to your legal department.

Okay, I admit it, I've been an Amazon fan since I was ordering books and CD's for the whole department from Amazon.com, had them shipped to the office in the Netherlands at least on a weekly basis.
My hero in the corporate market? Jeff Bezos, the man behind Amazon.com. My biggest complaint on Amazon? That they haven't yet opened an amazon.nl store so I can purchase a Kindle Paperwhite for my wife and two sons and save some trees by getting them books in Dutch all digital. I sooooo badly want a Paperwhite myself, to replace my Kindle-with-keyboard, I sent my kids to English class so they could read books in English and I could give them my Kindle and buy the Paperwhite, with 3G of course.
Amazon has always been this very customer centric for-profit company, disrupting the market, any market by a single strategy: improve the ways consumers can spend their money at Amazon.

But this post is not about how awesome Amazon is, it is about whether or not the Cloud is for corporates.

First of all, depending on who you are, your response by now could be a whole heartedly and fully sincere "YES!". In which case you're probably a sales manager or similar person of one of the many companies that sell cloud for the enterprise services.

Second of all, well let me first cover the SMB market;
The SMB market is probably the single one market that benefits most of SaaS, Software as a Service, which is sort of a rebranding of the almost complete failure ASP offerings were. Being a company in the SMB market, you're too small to host many of the software you can get as a service yourself and by choosing the SaaS offering you can benefit from their economy of scale. It totally makes sense.
The interesting part of SaaS is that it is way less a matter of elasticity, which is commonly associated with the cloud (and rightfully so), but the emphasis is on self provisioning. As an SMB you would get services like Office365, Salesforce.com or the Google Apps offerings. All SaaS offerings. Things like IaaS and PaaS are not for you, unless your raison d'etre is a service based on online accessible services. Something called in the early days as a dotcom.
The Cloud is, for an SMB, interesting because the cost model is one based on use and not ownership. Your cloud provider meters your consumption and you pay what you consume. Your costs increase when your use increases and typically that means when you're growing as a business. The model is very fair.
So back to the second of all...

Second of all, the cloud doesn't really exist, it's a substitute word for different offerings, so actually the cloud might be there for corporates but what is the cloud anyway. Generally acceptable as a definition is the one provided by NIST. It also has a nice definition on IaaS, PaaS and SaaS.

SaaS

First SaaS as I already covered some SaaS when discussing SMB's just a few sentences earlier. SaaS is more about economies of scales to reduce costs related to the use of standard(izable) software offerings. For SMB's this is usually a small step as their processes are not that formalized, or at least they're typically rather flexible when it comes to their processes. Because in many ways, SaaS offerings are based on usage based on a best-practice.
Corporates are notorious when it comes to processes and adherence to them. From experience I know that the larger the corporate the more effort it takes to change a process and more importantly, the smaller the chance that their processes are according to best practices. So in this perspective, SaaS can be very cumbersome to implement because it would require a change in the processes of the corporate. Something about teaching old dogs new tricks springs to mind. Although it can be done, read "Who Says Elephants Can't Dance?" by Louis V. Gerstner, not dogs but elephants.

IaaS

So let's continue at the bottom, IaaS, Infrastructure as a Service. Basically what you get is a server, nothing on it, or maybe an OS. You deal with it. There's not a lot more to it and as a corporate you should already be familiar with this. It's what you've done for millennia, you bought new hardware with or without an OS installed and you took it from there. Whole IT departments are in place and ready to support this server. Typically the server is one of a really lot and over time your IT Operations staff has a number of very smart system admins.
The big benefit of IaaS is that you don't have to wait for the delivery of the hardware provided you drink sufficient glasses of water as the server can and should be provisioned during the period that you get up from behind your desk, walk over to the water cooler and get yourself a glass of water. From experience I know that it can take slightly longer when you're not yet a customer with the Cloud provider because you'll need to sign up as a customer. This is not a joke, with the current leading Cloud Service providers, this is the process and the timeframe during which the new server is provided. Will it be a physical or a virtual server? Well you don't know and you shouldn't care. It is the server that you want, with as many CPU cores and memory and bandwidth as you need. It's all provisioned from a seemingly unlimited pool of servers and CPU's and memory and bandwidth. Yup, there's some real magic going on, because the Cloud provider just keeps on providing servers to you, never asking you to stop asking for more.

Once you've got your sever, you're left at your own devices. The Cloud provider will not maintain that server at all. He will just ensure that the server is provided to you at all times and when it crashes that it will be available to you within the time you agreed in the SLA.
Did I mention that you need to take care of everything yourself and that the Cloud provider will not manage your server? It's not the service he provides when talking about IaaS, you are getting Infrastructure as a Service.

So why would you go for IaaS? Why would you go for infrastructure as a service? Well, just like you would go for any other technology; it solves your problem or it can sufficiently contribute to the solution to your problem. Most of the time you don't want to go for IaaS if there's a PaaS solution for you. Read on for the stuff about PaaS.

Onwards, with the things you can get as a service. PaaS, Platform as a Service, takes you up the software stack.

PaaS

So on top of the infrastructure, there is an Operating System and then some. Basically the 'Platform' in PaaS refers to the software layer on top of which an application runs. A lot of people I know are thinking 'Application Server' or 'Middleware' when talking about Platforms, but actually some applications run on the Operating System itself, so the OS might be the platform in PaaS.
Here's a rule of thumb that can help you: a Platform as meant in PaaS is any piece of software that is not a business application (take the term very broad) and is managed by the Cloud provider. Please take not of that last bit!
So in PaaS, the platform is managed by the Cloud provider, just like in IaaS the infrastructure is managed by the Cloud provider. Infrastructure is (virtualized) hardware, Platform is software. Is this arguably the correct definition? Yes, we can argue about it, but it fits the NIST definition, which is convenient.

Oh yeah, don't forget that the Platform in PaaS is a software layer on top of which an application runs. Typically it provides technical services to the business application. This means that typically a DBMS is not a Platform.
An Application Server like a JavaEE container, is a Platform.

Interestingly, the application server in the previous paragraph can be part of the PaaS service, so it is managed by the Cloud provider, or the OS on which it is running is the Platform and managed by the Cloud provider but the application server is not. The key here is what is and what is not managed and it always moves up the stack. So you can't have a PaaS contract where the Platform is an application server that is managed, but the OS is not managed by the Cloud provider. Anyone tells you this is possible, well question their expertise on Cloud, like really hard.

Typically you would want to go for PaaS, because that means that there's a lot managed by the Cloud provider and you only need to manage the contract with him including the SLA's you agreed upon. The Cloud provider will ensure that you've got your Platform and that it complies with the capacity you agreed upon, that it stays available and in case it becomes unavailable, that it will become available within the timeframe agreed upon in the relevant SLA.
You also want to move to PaaS if possible because it means that you'll be drawing on the fact that the Cloud provider has staff that can actually support the platform 24x7 so you won't need that expertise in your organization in the amounts of being able to support it 24x7.

Like with IaaS, a PaaS environment is provisioned quickly. A matter of minutes more likely than days or even weeks and months. And with a shear unlimited amount of resources at your disposal, so click away and get provisioned.

IaaS instead of PaaS

So why would you go for IaaS instead of PaaS, or maybe go for PaaS but only for the OS and not the application server, say? Well that is because you need something that's not standard.
Especially with PaaS, you will have to comply with standards. And these are not your standards at all, these are the Cloud provider's standards. And you're required to stick with them and not deviate from them because that's just not possible because you have nothing to say about the service except for the quality you're getting. The platform is managed by the Cloud provider and not you, remember.
So when you're in need, let's say, for a clustered WebSphere Application Server environment, you're likely out of luck getting that as a Platform, because it's too cumbersome to maintain such an environment. You're likely going to get the OS as a Platform and install you're own cluster of application servers on top of it.
The Cloud provider will provide you with separate, independent servers, or instances. All perfectly managed. This means that you'll need to ensure that the applications running on it support this. That they can deliver the availability requirements by themselves, not relying on the infrastructure.

What? I can't rely on the infrastructure for ensuring I meet my availability targets and RPO and RTO and stuff like that? That's correct. You cannot.
The separate, individual servers will be available and when not, they'll be available shortly. It's the managed part that will be made and kept available by the Cloud provider. The Cloud provider is not going to do whatever he can to keep the application available. Not at all.
So there's a strict divide between what the Cloud provider will take care of and what you'll have to take care of and there's little collaboration to be expected.
So if you need that business service to be available at all times so your customers can get access to critical information at all times, your application needs to be robust and resilient in a way that it does not rely in any way on the infrastructure it's running on, it cannot make any assumptions as to how the infrastructure works and whether or not there's anything specific running to keep the infrastructure resilient.
Well, that's a lie, there is something you can assume. Something you actually have to assume: The Infrastructure is a piece of crap and it will crash over and over and over and you have no control over it.

Why are IaaS and PaaS tricky for the corporates?

That's a good question, because they are actually very well suited to move to the cloud, the problem is in perception. The cloud is a technology, it's a means, it's a tool to solve a problem. It's a tool that has an impact on the organization of the corporate when employed. It also significantly changes the way a corporate will deal with it's IT resources. Control is achieved by means of contracts because the handson option is gone to the extent of the service provided to the corporate. So all of a sudden the legal department is the one that does Systems Operations, those men and women that have no clue what IT is, are required to draw up contracts and SLA's that are the equivalent of what the IT department used to do by means of actually doing stuff. This is for many corporates a huge paradigm shift, that requires a significant amount of maturity not only in its IT department but across the board. Something that is kind of not existent in traditional corporates.

Another issue is that from an (enterprise) architecture level, the Cloud is a strategy. It's something that should be used across the enterprise, but never put as dogma(*). It's not a generally applicable tool and it requires a very thorough understanding about what Cloud is, when it is to be applied and what the restrictions are. Interestingly it is quite different from for example stating that all systems should be on either Linux or Windows, or that all in-house developed applications are to be developed in C# using Visual Studio. Using the Cloud or however you want to call it, should never be considered an architecture principle. Because it isn't. This is very weird for large companies. I haven't seen many companies that understand this.

Something to take with you

One important aspect of IaaS and PaaS is that both are revolving around infrastructure, almost solely around infrastructure. But the decision to go for IaaS or PaaS or actually going cloud has nothing, well hardly nothing, to do with infrastructure, it is an application driven decision. There is absolutely no point in considering the cloud when your application is not ready. When it is not designed in a way that it can assume that the infrastructure is a piece of junk, when it is not designed in a way that it takes care of its own resilience instead of relying on the infrastructure you shouldn't consider the cloud.
The Cloud provider will not go beyond the service he provides, so he will not take care of your applications that are running in the cloud. He will simply just not do it. This is a good thing, because its not his area of expertise. In fact, application support is very labor intensive, it needs humans to do proper. And the Cloud provider is always looking for eliminating the human factor and do as much in an automated manner as possible. So the humans you would need for support are eliminated by the Cloud provider, in a non-felony way of course, in most cases.

Hope you enjoyed this post,

Iwan

*: Dogma is typically for those that have no clue what they're doing and just do because it's dogma. As most people are happy if decisions are made by others, because it would mean they don't have to think and they don't have to assume responsibility, dogma as easily embraced.

Communicating while working - Message in a bottle... kind of

Hi fellow architects and other readers,

Over the coming period I will make an effort to post weekly an article on communication in the digital realm and its implication on architecture and the role of an architect. The articles will be diverse in terms of tone, topic and angle. This was my intention at the time I wrote the first article in a series on January 17th, 2013. This has been weeks, well months, ago and this is only the second installment of a multi-post item on communication. Shame on me.

Last time I have been discussing my experiences with online chatting from a personal level. It was a historic overview of me using various programs and media to digitally stay in touch with friends and family.

This time I'm taking it into the office and will discuss the the digital means of communication in the workspace. I talked about VoIP last time as for the consumer it was revolutionary, but in this post I will not touch upon it at all as VoIP in the office is a mere technology to have a phone system in place. Although digital, it is still the traditional way. (Leave your opinions in the comments because there's a lot to say about VoIP in the office not being just a traditional means of communication like the regular phone).

Ever since I was first connected to a computer network, there has been a form of email system. The digital equivalent of a regular letter. Back in the late eighties early nineties these were proprietary systems, confined to the network you were on. Although there was an open system implemented on UNIX and we used it in university.

Email has been around forever and it is the primary means of communication between people in the digital realm, well in most cases. Email is the predominant identification of a person on the Internet, it is reasonable to state that everybody with an Internet connection has at least one email address. Many, like myself have multiple addresses. Email is faster than postal mail so it is very convenient to most people. With the advent of broadband, emails have become richer in content as well, and the Internet email protocols have been adopted by all email systems and there are hardly any proprietary systems in use any more, but for very specific situations. Most of these are dealing with specific security circumstances. Email is not secure by any means, it is arguably significantly less secure than regular mail as the email can be opened and read by anybody with sender and receiver ever being aware of this. This is hard to accomplish with an "analog" letter as you would notice the envelope being opened when you receive the letter.

This security aspect of email has been a concern for many, especially companies that deal with confidential data are extremely aware of this.

Another important security issue is that it is very simple to send an email pretending to be somebody else.Since the email protocol works on clear text, anybody that can intercept an email can change the email before sending it along its way. And since the premise of the internet is that it is a highly redundant network that can withstand a nuclear attack, anybody can sit in between any two parties that send an email. But more importantly, the internet is a mesh network of point-to-point connections. It's a graph where every connected computer is a node. The connections are know because every node has an address, its IP. And these IP's are structured in a hierarchy.
On top of that, pretty much all connections are wired connections, because these are reasonably reliable and cheap as well. This also means that continents are connected by, literally, just a few wires. Put your computer on one of these wires and you're in the middle of all continental communications. Including email. Although very simplified, this is actually scary accurate.
So it's easy to pretend you're somebody else when sending an email. Just as simple as to write another name at the bottom of a letter. And the solution to prevent this is analogous to the analog letter; Signing the letter with a signature that is hard to fake. And here's another analogy with the analog world, how do you know what signature belongs to whom? In the physical world, this is handled by big books with names and signatures and when you get a letter that's signed, you open the book and compare signatures. And this is not a joke, this is how it's done. And in the digital world, we do the same. We have digital books (registries) with the names and the digital signatures that belong to these names and this is how we validate the authenticity of a signature. It's that simple... and really complicated. Because bits are only of the value 0 or 1 and therefore very easy to recreate. Do it in the right order and you can make a perfect copy of a signature. So we've got all kinds of mathematical schemes to ensure that it is as hard as possible to recreate the order in which the bits are written. And we distribute the signatures using a key infrastructure.
The tricky part of this is that you need to trust the person sending you his signature to be the person he claims to be based on the signature. Consequently this is not a solution to be used on a large scale where nobody knows anybody.

The key with email is, that its use and its validity is completely based on trust. But there is always plausible denial as an option for the "sender" when he inadvertently sends an email he never wanted to send in first place.

By the way, this previous part of this post is about half of what non-repudiation is all about. You just can't get non-repudiation without diverting to a small group of people you want to exchange emails with that should not be able to deny to have ever send an email.
The other half, denying you ever received and opened an email is the other half. This is like registered email (delivery receipt) with or without a signed receipt. Typically only enterprise grade products like Lotus Notes and Microsoft Exchange to name the two biggest email systems for the corporate have the option to automatically reply to the sender that the email was delivered to the recipient's inbox and again when the email was openen. Systems like Hotmail and GMail don't support this. So it's of little use to be honest in today's email eco-system.

But there's still a valid use of email in the enterprise. It's one of the most efficient ways to inform large groups of people about something that concerns them all. Because everybody is familiar with emails, it's adoption as a means to convey a message is massive. The analogy with mail helps of course.
The ability to add attachments to an email is of course also of huge benefit. One can send a large document or documents, which could be anything ranging from a text file to pictures or schematics or videos or music tracks, as an attachment where the email body is just an introduction to the real goodies in the attachment.

The problem with email is actually its wide adoption and its low threshold usability. Resulting in spam, unsolicited marketing garbage that clutters the corporate email inboxes with irrelevant emails that prevent people from doing their jobs. And then there are all the jokes and department party pics that keep people away from their work as well. Due to the little effort it takes to write an email to somebody, it's asynchronous nature, the improper use of any means to make an email more urgent, to raise its importance has caused over the last 5 years or so a transition of the enterprise from email based communications to something else.
Many enterprises are still searching for a good replacement. With a lack of alternatives to email that have all the good stuff and none of the bad stuff, email is still the prime means of collaboration in enterprises.

In the late nineties I was working at companies of all sizes where their email systems were limited to the extend of the enterprise and then there was the personal email. With systems like Compuserv and MSN (both were at that time a proprietary alternative to the web) one could send emails to other users outside your own organization. This changed when the internet-bubble started to grow around 1999. Hotmail was the big advocate for internet based email and with websites popping up like corn grains in a popcorn machine wanting to send you information, email grew rapidly and enterprises started to understand the importance of email to communicate with possible customers.
Interestingly enough, email turned into the most prominent and important way to communicate with the rest of the world along side with websites, but internally neither the intranet nor the corporate email systems took over this role from internal circulations, flyers handed at the door and the surprise brochures you stumbled upon in the morning when getting at your desk. For some reason we still don't see email as a viable means to communicate internal stuff and we still rely on the hard copy of the same message.
I noted myself that I am more likely to read a piece of paper left at my desk the night before than an email containing the same information left in my inbox around the same time. The reason behind this I don't know. The piece of paper is more intrusive, no question about it. It's typically placed on my keyboard so it prevents me from doing my work. The email in my inbox is easily ignored. It just sits there being unread. Maybe this is why I read the piece of paper, I have to pick it up and put it somewhere else before I can do my job. But still I can move it aside without reading it. So I guess it's more a matter of habit, a piece of paper is to be read. This is what I was brought up with. Books, papers, magazines, flyers, brochures, pamphlets. They are all pieces of papers with words on them, picked up by me to be read. I have to read it, it's the natural course of things. Email is not like that. I am more likely to think an email is too long to be read than a double sided printed memo about the same.

Based on this, I don't think that email will ever be as effective as paper. Not in the corporate, not to inform people. Yes you can use it very effectively to get your point across, but nothing more than your point. We, the working people, are not yet ready to use an all digital format to inform each other. We're still too analog. And when we're ready, email will not be that format. Why? Because it's too much like mail without an 'e'. What that other format will be? Intranet, document management systems, social media for the enterprise, chat programs? Well, I'll venture into those areas in the next installments of my blog, and I seriously will make it an effort to not wait this long again for my next post.

Untill my next post...

Iwan

Find me on LinkedIn or Twitter

IT is irrelevant when it comes to business criticality!

I hope I got your attention, because that is the sole reason for the title of this post. Of course IT is not irrelevant, it is very relevant. But it is not that important as many architects would like you to believe. Well IT architects that is. I would like to believe that your Enterprise Architect and your Business Architect would actually subscribe to this notion.

So what was I thinking when I decided that IT has no relevance when it comes to business criticality? You may actually ask yourself what I was smoking or sniffing when I started this post. Well the sheer fact that businesses where there long before the arrival of IT and most of our business processes today are still very well possible to be executed without IT. Probably not as efficient, after all that is why we have IT, but they can be done nevertheless.

Communication on the other hand is very crucial and having a means of communication is critical for business continuity. The more efficient we communicate, the more efficient we can do our work. This is why you always see that the most effort in IT has always been in the area of getting the right information at the right time at its destination.

And this is where my post starts to make sense, well it's the start. When we talk about information getting at the right time at the right place, we are talking about defining the business process, after all this is what a business process is all about; Moving information from one person to another, from one system to another. With every person, every system doing its magic with that information. Transforming it, enriching it, creating new information from it.

Back in the days we didn't have computers, we did have processes. And we had technologies to move information from one person to another. By couriers, Pony-Express, tube-systems, etc. Than with the advent of machines and later computers we started to make some steps in the process more efficient by automating them. Most of the processes stayed, in essence, the same, but we made them faster so we could produce more of the same. Make more profit and keep shareholders happier. But since the processes are still the same, we can replace the automated steps with the original manual steps again. Without compromising our business other than not being able to produce as much. So IT is irrelevant for the operation of the business unless you want to make money that is.

But here's the deal, when it comes to the operation of a business, the continuation of the business, the success of the business. IT is only a tool. It is the constellation of processes that defines the business, that ensures business operation, allows for continuity and determines the success of the business. Probably IT has a strong impact on the success of the business, but unless the process is clearly defined, the costs of performing each step in the process is known (to a degree) and we know what the cost of an automated variant of that step would cost, there's no reason to automated. In development countries, the emerging markets, manual labour is relatively cheap. This is why you'll see 10 men digging a canal instead of one man using a bulldozer. Because its cheaper, more cost efficient. Even in the western world, where manual labour is relatively expensive, automated solutions are often more expensive. Work is being automated or machine aided because time is a constraint, scalability is an issue or the labour is too hard and machines more reliable. But automation is and should always be a conscious decision.

With today's state of technology, we can automated pretty much everything and more importantly we can automate the governance of our business processes. This means that we can control that very essence of our enterprise, the process and keep tabs of what is happening when where by who and why, all in real time. Allowing us to make split second decisions when they are needed but more importantly ensuring that the the right information will arrive at the right time with the right person or system. Alerting us when this is not the case, telling us why it is not the case, keeping track of how often this is not the case. We call this Business Process Orchestration and in my humble opinion it is the going to be the area in IT that will garner the biggest cloud.

An important aspect op Business Process Orchestration (BPO) is that it allows us to unambiguously define business processes in a way that is almost technology agnostic, meaning that we can define he process without too much information as to how steps in the process are implemented. Of course to make the process track and traceable, we need to know the implementation and in a true BPO governed environment we have to go to that level of detail, but think about this; when we define our process at a high level such that it is almost irrelevant how the steps in the process are executed we can use this definition in various contexts.

Which is important why?

Remember I stated that the essence of an enterprise are it's processes, this means that in order to make the enterprise more profitable it is required to make the processes more efficient. And in that same train of thought, when the health of the enterprise is in jeopardy, it is important to make the failing processes more healthy. This can only be done when you control the processes and are able to manage them. This is exactly what BPO intends to do for us.

Now think of a scenario in which the health of the enterprise is pretty awesome but all of a sudden the enterprise finds itself close to dying. A serious disaster has struck the enterprise, for example an earth quake hitting the data center, a tsunami has taken its toll or a revolution has overthrown the stability of the country. In this case you're faced with a situation in which the business continuity of the enterprise is at risk. You're required to ensure that those parts of the business that are crucial for its continuity are taken care of and running as always. Which from an IT perspective means those systems that are crucial need to be up and running, most likely at an alternative site. And here's the multi-million dollar question, literally, what are the crucial systems?

In the old days this was simple, the was only one system, the mainframe, you needed 2 of them to be ready for whatever disaster. Nowadays this is not that simple. Systems depend on each other, have different designations and different technologies. More uses and a variety of uses as well. So how do you determine what's crucial and what's not? Business processes to the rescue.

Look at the processes that are crucial, the systems used in those are crucial. Look at the costs of making these redundant and see if the 'manual' option is viable as well. For those systems where there can not be a manual alternative you spend the dollars to make them disaster recoverable. While doing so, don't forget to keep in mind the RTO ( Recovery Time Objective) and RPO (Recovery Point Objective) of these systems and obviously also the Confidentiality, Integrity and Availability (CIA) rating of the required information.

As you should understand by now, the business process is key in all of this. And there is no point in spending time and money on IT without knowing what processes are being served and how crucial these are.

As always, I'm more than happy to read your comments and learn from them. Happy to feedback as well on any of your questions and maybe other may answer them as well.

Iwan