Cloud Native Enterprises - Rapid elasticity

Which don't have a lot to do with Cloud Native Apps but everything with truly embracing the paradigm shifts the Cloud has brought IT within the realm of businesses.

Read the Introduction first.

After reading the introduction to these posts you know what a cloud infrastructure is and what cloud native applications are, what about cloud native enterprises. Well these are enterprises that adhere to these same 5 characteristics. These enterprises, or organisations in general, cannot be modelled according to traditional enterprise models because of their specific market, competition, growth-stage, etc. These enterprises need to be, for all accounts, be cloud native in order to grow, succeed and be sustainable. Interestingly, but not surprisingly they require The Cloud and Cloud Native Applications.
In coming posts I will address every essential characteristic of The Cloud as defined by NIST from a perspective of the Enterprise. Unlike most cases, I will post these within the next 7 days and I certainly do hope before coming weekend.

• On-demand self-service. When online services and core systems really seamlessly integrate.
• Broad network access. When customers, partners and users are distinct groups treated equal.
• Resource pooling. When synergy across value chains makes the difference.
• Measured services. When business resources are limited.

Rapid elasticity. When business is extremely unpredictable.

Very few organisations can say that year round they experience the same amount of business. Almost every organisation experiences something like a 'season'. There are always highs and lows in an organisation's load. This can be 'Black Friday/Cyber Monday' for retailers, tax-season for the IRS or its equivalent in your country, or for example hotels during the summer holidays.
But not only in sales, there are highs and lows, often very predictable, but in production companies there are peaks in order fulfilment.

In order to deal with these fluctuations in 'business load', organisations need to be elastic. The more elastic an organisation is, the better it will be able to handle the fluctuations. Mind, this is not the same as being agile. Being agile is being able to change direction as needed, in a timely manner. Being elastic is being able to change scale as needed. Also in a timely manner. Arguably, being elastic is more difficult to accomplish than being agile.

Elasticity is a matter of scaling up, and down. The more elastic an organisation can be, the more efficient it can do its business. Efficiency here is a matter of spending just enough. Another difference between elasticity and agility. The former being about efficient in using resources, the latter is about being effective in resource utilisation.

From the Cloud we know that elastic means scaling up, or down of the IT infrastructure in order to manage fluctuating load. Thus not having to worry about under-utilisation of IT infrastructure, thus paying for infrastructure that is not being used. And at the same time, not having to worry when load increases and the stability of the environment isn't compromised due to over-utilisation of that same infrastructure.

In the Cloud Native Enterprise, we project these traits onto the organisation itself. Onto the business.

Traditionally, scaling organisations are hard to accomplish. Extremely hard in fact. For one there is always the challenge of resource availability. Where resources are of course human resources. The most common way of addressing this is through contractors. The hiring and firing processes around contractors are far more flexible than for permanent employees. Thus bring some elasticity to the organisation. But getting the 'bodies in' is only part of the challenge. Getting the 'right bodies in' is another aspect. Adding personnel with the right skills is possibly even harder. First you need to find them, next you need to validate that you found them. The hiring process is cumbersome and the further you need to stretch the organisational elastic band, the harder it becomes. The amount of effort needed is not increasing linear, but almost exponential.
Organisations need to be able to scale up quickly. Which often also means that the HR department needs to be elastic as well. Again, the same challenge apply.
In an economy with an up-beat, there are more companies that are looking for the same scalability and resources become even more of a challenge to find.
I haven't mentioned the strain on the organisation itself due to growing too large. More management layers need to be introduced in order to be able to grow and manage this. And this is where scaling down becomes a problem. Adding more management layers in the hierarchy to handle the size of the workforce, is relatively easy compared to removing these layers again.

The more seasonal a market is, the more of a challenge elasticity becomes for an organisation. Tax-offices will be overloaded with calls to the help-desk as the deadline for submitting your tax-forms nears.

At one point I was involved in a court-case as one of the expert witnesses where my client succumbed at its own success. The client was in a business where 98% (!!!) of its revenue was generated in 5 consecutive days of the year. It couldn't handle the increased business as it couldn't predict what that load would be. Especially since in the months prior to their 'peak' they took over their largest competitor.

Organisations can't scale their workforce to the point where we can talk of elasticity. Scaling up and down is a matter of weeks. Where elasticity is the ability to scale up and down in a matter of days, hours or even minutes and less.
This is where the Cloud comes in. It is not so much the elasticity of Cloud infrastructure that plays a role, but the ability of an organisation to utilise the Cloud such that it can scale its business efficiently. Cloud infrastructure is an important part, where it comes to handle IT load. But the true business elasticity comes from handling at a business level the fluctuating load.
Extreme automation of business processes allows an organisation to reduce its dependence on specific business knowledge of its employees. Everybody can press a button or enter data when the 'system' does the heavy lifting in applying business rules and execute repetitive tasks. Understand that by following this paradigm, the reliance of the organisation on less-educated employees is reduced to a bare minimum, and it can focus its efforts towards attracting higher educated employees. Employees that require less managerial guidance which allows for a more flattened hierarchy. Flat hierarchies are more elastic.
Automated processes can in addition, benefit from the technological elasticity of the Cloud. Thus having a double edged sword.

Elastic organisations are not trivial. Far from it in fact. Main reason being that traditionally, organisations scale with their business by either increasing its workforce through short term contracts, i.e. contractors, which can be hired and fired as needed. Or by attracting permanent employees and assume a positive attitude towards the future. Or laying off personnel in advance when the future is perceived less positive. It is how organisations are used to manage the seasons.

The Cloud Native Enterprise on the other hand is far from traditional. At its core it will embrace technology resources not to replace human resources, but to complement them. It invests in senior, higher educated experts, to reduce the reliance on management. In effect, pay more to less, in order to reduce the need to scale the workforce.

Concluding

And so the Cloud Native Enterprise is the enterprise where IT is part of the business when it comes to delivering business products. Allowing a business to grow and shrink as needed, just ahead of time and with the flexibility of a rubber band.

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect


The text very explicitly communicates my own personal views, experiences and practices. Any similarities with the views, experiences and practices of any of my previous or current clients, customers or employers are strictly coincidental. This post is therefore my own, and I am the sole author of it and am the sole copyright holder of it.

Cloud Native Enterprises - Broad network access

Which don't have a lot to do with Cloud Native Apps but everything with truely embracing the paradigm shifts the Cloud has brought IT within the realm of businesses.

Read the Introduction first.

After reading the introduction to these posts you know what a cloud infrastructure is and what cloud native applications are, what about cloud native enterprises. Well these are enterprises that adhere to these same 5 characteristics. These enterprises, or organisations in general, cannot be modelled according to traditional enterprise models because of their specific market, competition, growth-stage, etc. These enterprises need to be, for all accounts, be cloud native in order to grow, succeed and be sustainable. Interestingly, but not surprisingly they require The Cloud and Cloud Native Applications.
In coming posts I will address every essential characteristic of The Cloud as defined by NIST from a perspective of the Enterprise. Unlike most cases, I will post these within the next 7 days and I certainly do hope before coming weekend.

• On-demand self-service. When online services and core systems really seamlessly integrate.
• Resource pooling. When synergy across value chains makes the difference.
• Rapid elasticity. When business is extremely unpredictable.
• Measured services. When business resources are limited.

Broad network access. When your business hours are truly 24x7.

This is an interesting aspect of the Cloud Native Enterprise. Because many organisations are already 24x7 businesses. Especially in the online world, being always on is a requirement to stay in business. 

Network here is not referring to the the computer network on a technical level as we know it. According to NIST, one of the characteristics of the cloud is broad network access, which from the definition, this means that the cloud is always accessible through the internet. The computer communications network.
Within the context of the Cloud Native Enterprise I am referring to the communications network of the business. On the one side, this encompasses all parties a business has dealings with. Think customers, users. partner, employees etc. I will get back to this later on in this post. But it also encompasses all means through which this communication takes place. Think devices and associated channels.

So, when we look at the business that is truly cloud native, when we talk about broad network access, we mean that the business is accessible by its complete business network, via a variety of devices and channels.

The Cloud Native Enterprise exposes its services, all of its services, in the same way to customers as it will to users and partners, as well as employees. Every member of every group enjoys the same level of service and the same constraints. Distinctions are made using the concept of role-based access to services. Still all services are always accessible to all members of the network through the same channels. In addition, it will provide the same level of service through all channels on all devices. Ideally. Of course contextual limitations are to be taken into account.

Traditional Focus - Cost vs Value

Consider the more traditional enterprise, where customers are assigned a specific account manager. The account manager maintains the relationship with the customer. She has specific KPI's that need to be met and she is more or less free in determining how to achieve this. Users of the customer are not aware of the account manager, instead they interface through a help-desk with the enterprise which will consists of self-service functionality for the more mundane support, automated systems like chatbots and more complex interaction through a help-desk agent.
Partners of the enterprise on the other hand, will work with peers within the enterprise. Informal contacts are more prevalent and accepted. Where customers and users will need to follow the formal processes in order for the enterprise to be more efficient, partners will follow the informal communication lines in order to be more effective. We see in traditional enterprises that with customers and users, processes are cost driven. Partner oriented processes are value driven.

Cloud Native Focus - Revenue

In the Cloud Native Enterprise, all interactions are focusing on revenue. Key aspect here is the homogeneous approach towards interaction with stakeholders. Customers, employees, users, partners are all considered stakeholder of the enterprise. Access to the enterprise is homogeneous across the full network, and processes are optimised for revenue. Sometimes resulting in focus on efficiency, other times on effectiveness.

What you will see is that business scalability, both temporal and geographical, is addressed explicitly. Every stakeholder in the enterprise's network can access the enterprise 24x7 and from any location.

For every organisation this is a challenge to manage. But for the Cloud Native Enterprise the mere premise of the Cloud as an IT facilitator, it becomes a matter of survival.

Consistent User Experience

The Cloud is great for realising products and services that scale with your business. Not talking about elasticity here, but about the ability to have a single approach towards addressing your stakeholders' needs and demands.

Leveraging the characteristics of the Cloud, including its technical aspects of broad network access, means that it is possible to provide all users of all services and products with a consistent user experience. This entails the same experience provided to customers, partners, employees etc. This consistent user experience across all services will allow for a seamless transition for a user from one role to another.

Concluding

As with its technical counterpart, (broad) network access is a key characteristic for the Cloud Native Enterprise, as it will provide a homogeneous approach towards accessibility of products and services across the full breadth of the business network

(Special thanks to my colleague Mandeep for pointing out some much needed clarifications in the original post)

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect


The text very explicitly communicates my own personal views, experiences and practices. Any similarities with the views, experiences and practices of any of my previous or current clients, customers or employers are strictly coincidental. This post is therefore my own, and I am the sole author of it and am the sole copyright holder of it.

Cloud Native Enterprises - On-demand self-service

Which don't have a lot to do with Cloud Native Apps but everything with truely embracing the paradigm shifts the Cloud has brought IT within the realm of businesses.

Read the Introduction first.

After reading the introduction to these posts you know what a cloud infrastructure is and what cloud native applications are, what about cloud native enterprises. Well these are enterprises that adhere to these same 5 characteristics. These enterprises, or organisations in general, cannot be modelled according to traditional enterprise models because of their specific market, competition, growth-stage, etc. These enterprises need to be, for all accounts, be cloud native in order to grow, succeed and be sustainable. Interestingly, but not surprisingly they require The Cloud and Cloud Native Applications.
In coming posts I will address every essential characteristic of The Cloud as defined by NIST from a perspective of the Enterprise. Unlike most cases, I will post these within the next 7 days and I certainly do hope before coming weekend.

• Broad network access. When customers, partners and users are distinct groups treated equal.
• Resource pooling. When synergy across value chains makes the difference.
• Rapid elasticity. When business is extremely unpredictable.
• Measured services. When business resources are limited.

On-demand self-service. When online services and core systems really seamlessly integrate.

This is the post where I'll address the aspect of the On-demand Self-service. An aspect which for many organisations is rather challenging. Just like the aspect of Measured services, this aspect is addressing internal governance structures. Although the governance implications will touch on the financial governance of an enterprise, the challenge is more of an organisational nature and in fact  the financial parts are not always relevant.

In any case. When we talk about on-demand self-service within the context of Cloud Native Enterprises we talk about an enterprise where tools and technologies needed to explore or exploit business models can be obtained when needed, by the person that needs it. Often we talk about IT resources, software and or hardware. But think in terms of services, which are often IT services.

In a traditional environment, IT services are obtained by the IT department. This is where the IT experts are and this is where the required frameworks for 'proper IT' are defined and used. Business departments will request, based on functional specifications, certain IT solutions, experts within the IT department will make sure that the best solution for the lowest price will be procured, installed and configured. Hassle free operation for the business people as a result.

Traditionally the reason for an IT department is centralisation of these resources in order to make as efficient use of these experts as possible. Actually, what I most often see is that within IT departments there are separate teams for specific expertise, so called Competence Centres or Centres of Excellence. Efficiency has been a key objective for many years within organisations, often because IT resources (computers as well as personnel) were relative expensive compared to other resources.

As efficiency has been a key objective in these traditional environments, the scheme above played out nicely. Scarce personnel is being utilised optimally in terms of keeping them busy and in the meantime, their experience and knowledge ensured the best value for money possible. Time was not a factor until recently. Of course, it always took too long to provide the solutions to the business, but offset against costs, this has always been a small price to pay. Relatively speaking, and yes, pun intended.
But things have changed and the old timelines are no longer acceptable. Timing becomes more and more an issue. Business ideas can no longer wait to find their way into the hands of users. Not only because lean, agile and nimble Start-Ups will chip away the market shares of enterprises, but because enterprises between themselves are leveraging technology to get the competitive edge.
We see that the introduction of The Cloud in the IT departments of enterprises has resulted in extremely short timelines when it comes to delivering solutions to the business by their IT departments. Enterprises with an IT department that can't leverage the characteristics of The Cloud as defined by NIST will perish. You can't move fast enough? You're shark food. It's a dog-eat-dog-world nowadays.

The CIO v.s. the CDO

We see that CIO's are being complemented by CDO's, Chief Digital Officers, in those enterprises where the CIO has not been able to transform the IT department from a traditional silo'd business enabler into a heterogeneous multi-disciplinary flat business partner. CDO's are introduced to drive the digital transformation, a transformation that has been going on since the advent of computers in enterprises. But where it focused on efficiency and controlling cost in the past, now the transformation is focusing on effectiveness and creating value. Often the CIO hasn't been informed about this new direction as often the CIO is not in the boardroom. Understand that in many of the organisations I've been, the CIO reports to the Chief Financial Officer, instead of the CEO, COO or the Chief Marketing Officer. IT is associated with cost in these enterprises and not with revenue.

Product/Platform Paradigm

But in the Cloud Native Enterprise, it isn't the IT department that delivers IT solutions, not even when they can do this with the speed of The Cloud. In these enterprises, the business services it self.
When IT solutions are needed, the business will obtain them by themselves. Here the IT department is no longer delivering the solutions, but the platform onto which the solutions will run and integrate with the core IT systems. Systems like Identity and Access Management, Monitoring and Metering, Resilience and Disaster Recovery. Integration of solutions in the platform is what IT is about. Products and business solutions is what the business is about.
The IT departments in these enterprises are basically nothing more than in-house system integrators. But extremely sophisticated at that since integrations are subjected to profound automation.

It is with Cloud Native Enterprises where we see the true value of the Product/Platform paradigm. IT delivers the platform, the platform is its business product. Business delivers business products. This is where the concept shines as those that understand how to valuate the product from a business perspective are accountable for the product that creates the value to the business.

Concluding

And so the Cloud Native Enterprise is the enterprise where IT is part of the business when it comes to delivering business products. Allowing business solution to be created using IT on-demand, through self-service. This also means that the traditional IT department has become a business department as well, run as a revenue catalyst and not as a cost centre.

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect


The text very explicitly communicates my own personal views, experiences and practices. Any similarities with the views, experiences and practices of any of my previous or current clients, customers or employers are strictly coincidental. This post is therefore my own, and I am the sole author of it and am the sole copyright holder of it.

Cloud Native Enterprises - Measured service

Which don't have a lot to do with Cloud Native Apps but everything with truely embracing the paradigm shifts the Cloud has brought IT within the realm of businesses.

Read the Introduction first.

After reading the introduction to these posts you know what a cloud infrastructure is and what cloud native applications are, what about cloud native enterprises. Well these are enterprises that adhere to these same 5 characteristics. These enterprises, or organisations in general, cannot be modelled according to traditional enterprise models because of their specific market, competition, growth-stage, etc. These enterprises need to be, for all accounts, be cloud native in order to grow, succeed and be sustainable. Interestingly, but not surprisingly they require The Cloud and Cloud Native Applications.
In coming posts I will address every essential characteristic of The Cloud as defined by NIST from a perspective of the Enterprise. Unlike most cases, I will post these within the next 7 days and I certainly do hope before coming weekend.

• On-demand self-service. When online services and core systems really seamlessly integrate.
• Broad network access. When customers, partners and users are distinct groups treated equal.
• Resource pooling. When synergy across value chains makes the difference.
• Rapid elasticity. When business is extremely unpredictable.

Measured service, when business resources are limited.

This is the post where I'll address the aspect of the Measured service. It is an interesting aspect of the cloud native enterprise because it addresses the financial governance model of the enterprise.
Let me first start with what I witness in most organisations regarding the financial governance model. Great and small, old and new. And even in start-ups. The model is in about all cases one in which there is an annual budget allocated for pretty much everything that costs money. Often times, the budget is based on past experiences. It's a copy of last years budget corrected, or extrapolated, based on last year's developments, revenue, target met or not, etc. What I often times see, and I'm actually pretty sure I have seen this in all but one or two cases, is that once a year there is a prediction as to what the business will look like in the coming year. This is not the strategic plan, but next year's plan. There's a project portfolio, or more recently a product portfolio, that covers the coming year. Predictions are made about next year's resource needs etc.
The most common thing here is that the more resources an organisation has at its disposal, the more anal the enterprise is about these plans.

Being ambitious as a business

What I hardly ever see is a prediction of what the enterprise's ambition is with respect to its market position. Its business ambition. Do not mistake this for a strategic ambition, or a mission statement. What I am talking about is an 'objectives portfolio', an overview of the enterprise's objectives. Each of it's value chains, lines of business, P&L areas, however the enterprise's business is divided should have objectives, and ideally a roadmap littered with objectives. This is the objective-portfolio.

Why this is so important is that an enterprise should be working to meet these objectives in any way it can. And the reason why this is so important, is that these objectives are in fact the key drivers for business sustainability. From a strategy perspective, the business objectives should be completely in line to support the strategy. By working towards these objectives, the strategy is implemented, which in turn will increase sustainability.
An important key take-away is that objectives can be met in any number of ways. There is hardly ever only one way to do so. But in general, out of all the different ways to achieve an objective, there are only one or two that are the best. The best within the context of the enterprise at a specific moment. Which means that what is the best way now, might not be the best way in the next quarter. There could be a missed opportunity, new regulation, or a shortage of resources. So re-evaluating the the proper course of action continuously, should be a given.

For enterprises resources in terms of money are often not considered a bottleneck, instead I see often that the availability of the right people to do the job to be a problem and in even more cases the precious resource we call time is a problem of its own magnitude.
For smaller companies I see that often money is the issue and people and time not so much. Reason being that smaller companies tend to focus more on a limited amount of business solutions. Where enterprises are straying all over the place, small companies are equipped with laser sharp focus.

"Now where's the Measured service coming into play?", you ask.

Measuring achievements not performance

Working with a roadmap based on objectives, with a portfolio of objectives, allows for embracing the concept of validated learning, and metered funding.
Like I stated earlier, objectives are met by choosing one of many implementations, where the choice is based on the context of the moment the choice is made. For example the timing to introduce a new consumer product just ahead of the holidays. Or the introduction of a key competitor's product that will pave to way to broad market acceptance of something controversial. Or the abundance of developers that can code in specific programming language. By consciously choosing the seemingly best implementation to meet the objective, you know what to look for in order to re-evaluate that decision. Staying on track with the calendar to meet the holiday boom, monitoring market acceptance of that product, keeping an eye on Craigslist for job postings for those coders, etc.
Obviously this requires monitoring of the results of your efforts, but that is exactly what it means to apply the concept of 'validated learning'. You determine based on what your efforts are contributing to the bottom-line (whatever that is to you) or not or is even counter productive, this is your hypothesis. You undertake whatever action you've got up your sleeve and once you're done, you check your hypothesis and determine what your next move should do. It's actually pretty close to just being agile.

Turning variable into fixed

Now the trick is, and this is where it becomes extremely complicated for enterprises, to allocated resources in such a way that you can continuously work on meeting the objectives, but change the way you do so, throughout the year. From a financial planning perspective, this is rather different from most enterprises I've seen over the years. Like I stated earlier, budgets are allocated annually, based on plans. The the bigger the budget request, the more assurances need to be given to get the budgets allocated. Business Cases are written, plans are drafted, dreams are... well often the most realistic and accurate of these three.
The way I often see it addressed is by introducing pre-funded teams. Which basically means that one of the larger portions in the budget is fixed, namely human resources. In fact, this only addresses the problem when an enterprises HR strategy is relying heavily on out-sourcing, contractors or a combination of these two. When most of the work is done by own staff, HR costs are already fixed, and working with pre-funded teams is merely an HR-planning issue. I would prefer this situation, but it is often just not feasible. With pre-funded teams, a large part of the budget is fixed, which is an accountant's dream, or so I've been told on various occasions. How to fix all other variable costs? That's a challenge, but this is where a cloud environment helps, as it will not fix IT costs, but it will allow these to be limited to the bare minimum as needed, so costs are not fixed, but risk is contained and limited. Another dream coming true.
Your measured service at play here is now that for one a large chunk in the budget is no longer variable, so planning is straight forward, no measuring is really needed as the teams are based on objectives to be met and not work to be done. I understand this is the first time I phrase it this way but it is crucial to understand the different. I leave you to think about it for a second.

Concluding

Measuring is needed for the remaining chunks in the budget. Although still variable, this chunk, the IT resources costs are contained and limited and investments are only needed when the IT is required. And more importantly, revenue is generated shortly after investments are done, or else further investments are stopped. So we measure the revenue, off-setting it to the limited costs.

I hope you understand the paradigm shift in the financial governance from a mainly cost driven concept where revenue is mainly a matter of wishful thinking, educated guessing at best, to mainly realised revenue based.

Re-iterating. Measured services relate to the fact that the in the cloud native enterprise, the enterprise is governed such that costs are considered investments. But only concerning those costs that can't be fixed. In other words, variable costs are considered investments. Investments are related to business objectives to be met and not work to be done. This eliminates in a large part the risk that the focus is on work already done instead of objectives still to be met when it comes to new investments.




Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect


The text very explicitly communicates my own personal views, experiences and practices. Any similarities with the views, experiences and practices of any of my previous or current clients, customers or employers are strictly coincidental. This post is therefore my own, and I am the sole author of it and am the sole copyright holder of it.

Cloud Native Enterprises - Introduction

Which don't have a lot to do with Cloud Native Apps but everything with truely embracing the paradigm shifts the Cloud has brought IT within the realm of businesses.

A couple of weeks ago I was asked to do a presentation to a number of mainly (project) managers on the topic of "The Cloud". I think one of the reasons I was asked had to do with the fact that I usually tend to be (a bit) iconoclastic and another reason might be that I have been a huge proponent of The Cloud. Look at previous posts of Arc-E-Tect and you should be able to find a number of posts on the topic. In any case, I obliged.

While preparing the slides, digging into my archive of presentations as I remembered a presentation I did on the topic a couple of years ago to a group of mainly lawyers and auditors I decided that it was the perfect time to not talk about The Cloud as an IT phenomenon but as a business phenomenon. As you might have gathered when reading my posts on this blog, I do tend to see IT from a business perspective. Pure simple logic dictates that we apply IT in organisations to strengthen our businesses. The decision was made, I was going to talk about Cloud Native Enterprises.

The Cloud According to NIST

What are Cloud Native Enterprises? First of all you need to know what The Cloud is, and I always use the NIST definition of Cloud. NIST made it easy to define The Cloud by naming 5 essential characteristics of cloud:

• On-demand self-service. A consumer can unilaterally provision computing capabilities, such as server time and network storage, as needed automatically without requiring human interaction with each service provider.
• Broad network access. Capabilities are available over the network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e.g., mobile phones, tablets, laptops, and workstations). 
• Resource pooling. The provider’s computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to consumer demand. There is a sense of location independence in that the customer generally has no control or knowledge over the exact location of the provided resources but may be able to specify location at a higher level of abstraction (e.g., country, state, or datacenter). Examples of resources include storage, processing, memory, and network bandwidth.
• Rapid elasticity. Capabilities can be elastically provisioned and released, in some cases automatically, to scale rapidly outward and inward commensurate with demand. To the consumer, the capabilities available for provisioning often appear to be unlimited and can be appropriated in any quantity at any time.
• Measured service. Cloud systems automatically control and optimize resource use by leveraging a metering capability at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency for both the provider and consumer of the utilized service.

(source: The NIST Definition of Cloud Computing)

There's not really an order in these characteristics, but they are essential. You have to keep in mind that these are related to IT systems, typically infrastructure hosting services.
I'm not going to re-iterate these characteristics and explain what The Cloud is, instead I'll go over these essential characteristics to explain what a Cloud Native Enterprise is.

Cloud Native Applications

But before I do this, let me explain what a cloud native application is, because it is important to understand this. First of all; Every cloud native application can run in a traditional data center. Second of all; From a NIST characteristics perspective there is no difference between public and private and hybrid clouds. As long as all 5 essential characteristics are met, we can talk about a cloud environment.

Back to the cloud native application. This is an application that thrives on the cloud. It is one that uses the 5 characteristics to the fullest and doesn't compromise. Let me go over them one by one.

• On-demand self-service. A cloud native application is accessible by a user, which could be another application, but often it is a person. But it is not only accessible by a user, but everything pertaining that user can be done by the user, whenever the user pleases. So the user can sign-up herself, configure her environments herself and do whatever she needs to do, all by herself. Irrespective of the role of the user. So an administrator of the application can do whatever needs to be done, as can a business user. This is hugely different from traditional applications, where you would need to request an account at some service-desk in order to gain access.
• Broad network access. Cloud native applications are network accessible and not only that, they're accessible through standard mechanisms and through common devices. Typically we see that these applications are accessible via the internet, or at least through internet protocols. Often we use http or https to access the applications. Originally through a browser, but with the advent of mobile devices we see that the generic web-browser is replaced by a specialised browser, the client application, that accesses the application's logic via web- and internet-protocols and presents a user experience fully embracing the client device's capabilities.
• Resource pooling. This for applications means that the applications are multi-user applications and the users are not aware nor impacted by each other, unless of course, business logic demands interaction.
• Rapid elasticity. Probably the most unique selling point of the cloud is elasticity. The amount of resources needed to perform a specific task consistently independent of the load on the systems is expanding or retracting as needed. Elastic. No wonder Amazon calls their virtual server environment Elastic Cloud Compute (EC2). For cloud native applications the same goes. The application scales up and down as needed by the load on it. Typically this is done using the underlying infrastructure's elasticity, but mind that the cloud 'nativeness' of the application is that we're talking about functional elasticity. The responsiveness of the application is consistent across many workloads. This can mean that the application's functionality scales down when load increases (e.g. loading only the first 3 review comments for an item in a web-shop instead of 10) and scales up when the load decreases.
• Measured service. Here we run into the situation where the usage of the application is metered. At any time it is known what the application's utilisation is. Which user is using what functionality at what time with which guarantees. The idea here is, obviously, that the user will be billed based on her usage (of resources) of the application in contrast to some flat fee or a compute-power based fee. An example is that a free-tier account can access a certain function only once every two seconds, where a pro-tier account can access the function 5 times a second. The key here is metering and billing.

Every time I use the word 'user' above, I want to reiterate, it can be a person, but also another system. Especially in the current API economy, the user mentioned will very likely be another application.

Applications that are cloud native adhere to these 5 characteristics, and they do this from a business functional perspective. There are all kinds of technical implications like a transition from ACID transactions we typically see in traditional applications, towards BASE transactions we see in cloud native applications. Same goes for in depth security in the cloud and more of a focus on perimeter security in traditional data center hosted applications.

Cloud Native Enterprises

Now that you know what a cloud infrastructure is and what cloud native applications are, what about cloud native enterprises. Well these are enterprises that adhere to these same 5 characteristics. These enterprises, or organisations in general, cannot be modelled according to traditional enterprise models because of their specific market, competition, growth-stage, etc. These enterprises need to be, for all accounts, be cloud native in order to grow, succeed and be sustainable. Interestingly, but not surprisingly they require The Cloud and Cloud Native Applications.

In coming posts I will address every essential characteristic of The Cloud as defined by NIST from a perspective of the Enterprise. Unlike most cases, I will post these within the next 7 days and I certainly do hope before coming weekend.

• On-demand self-service. When online services and core systems really seamlessly integrate.
• Broad network access. When customers, partners and users are distinct groups treated equal.
• Resource pooling. When synergy across value chains makes the difference.
• Rapid elasticity. When business is extremely unpredictable.
• Measured service. When business resources are limited.

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect


The text very explicitly communicates my own personal views, experiences and practices. Any similarities with the views, experiences and practices of any of my previous or current clients, customers or employers are strictly coincidental. This post is therefore my own, and I am the sole author of it and am the sole copyright holder of it.

The natural beauty of organic convergence

... in an agile world.

Summarising

I am a big proponent of the concept of 'Organic Convergence'. Something I have come up with and what goes on from the assumption that within a social group norms and values ​​become a standard, a common ground. The degree of interaction between the members within the group is decisive for the speed with which this happens and the benefits that can be reaped from it. More interaction means faster and more valuable convergence. And in doing so, I foresee that when the different teams work together, forming a social group, uniformity will result. With the added benefit that the result is broadly supported but not imposed.

The world is organic and evolutionary. At times it is revolutionary, then we are dealing with disruption. Disruption is beautiful because that turns you into a market leader and increases your chances of survival. This, of course, only when you are the disrupting force.

So where is this coming from? Well, it's coming from a discussion I had several times in the past weeks about Canonical Data Models (CMD), Enterprise Resource Models (ERM) and centralisation of governance in an organisation that just decided to execute a full-blown agile transformation over the coming years.

This organisation's legacy is, like so many other organisations with a challenging bureaucracy, is one of centralisation of capabilities (see my view on this in Perish or Survive, or being Efficient vs being Effective and an duo of posts on Competence Centres in huge organisations here and here) and a fairly directive management style. Although, as in many organisations, this one has come a long way, but still the overall cultural view is that teams are not autonomous nor self-proficient and Enterprise or Domain Architects are to define standards across the enterprise. Mind that these standards are not on enterprise level topics like principles, frameworks, etc. No in many cases the standards are about what technology to use, what architectural framework to use, or what model to implement. Down to the level of programming language and style.

Some context on the CDM

In this section I am elaborating on the concept of a CDM and why it is a remnant of centralisation in IT organisations. You can skip to the next section if you would like so.
The discussions I referred to at the beginning of this post were about whether or not an enterprise wide model had to be enforced top-down. Or possibly should emerge bottom-up. The discussion was about CMD (Canonical Data Model) and ERM (Enterprise Resource Model). Now the thing here is that the 'M' in the naming is not helping. We're not talking about just a model, but also its implementation. The CMD is a remnant from the days in which Enterprise Application Integration (EAI) was something new. In the late 1990's and early 2000's integration of software systems and thus creating synergy was key in many IT organisations. Different systems used different information models. What one system considered a Customer would be different from another system's definition. Typically across business domains these differences were huge. Then there is the problem where two departments in an organisation have different names for the same concept. Understandably, the CDM is going to be extremely helpful. In many ways, a CMD resembles the synthetic language Esperanto.
But unlike Esperanto, the CDM is a model. It's a definition and not an implementation. It becomes all the worrisome once the CDM gets implemented using one single implementation. The problem here is that definition and implementation all of a sudden become one. This is a problem because implementations require technical specifications like data-types and formats, which diverts the discussion from semantics towards syntax. Or in other words, the CDM becomes an IT concern instead of a business concern.
Typically the CMD gets implemented in a centralised part of an architecture. The main database, the ERP system or as part of the ESB. And historically this sort of made sense. When the CDM was invented, it had to also be implemented and preferably in a single location. Considering the focus on efficiency in IT, centralisation was paramount. Definition and implementation would go hand in hand, developed in parallel. Because of the challenges at hand, mainly the integration challenges, the CDM evolved into the 'language' on the ESB. Everything on the bus would speak the same language, when needed a connector would translate from and to the common language.

In those early days of EAI, the challenges were merely integrating the organisation's own systems. All within the organisation itself. Considering the context, a CDM made perfect sense and I would argue that based on the peculiarities of these environments, having a CDM and use it as the common language on the bus is the right choice. Think about the CDM not being Esperanto, but Hindi in India. Within a very neatly scoped context it was a common language.
In many occasions I have struggled with inconsistencies in information models for different enterprise systems and a CDM would have been or was the best solution. In my experience, the best way to deal with a CDM is on the ESB. Selecting the ESB as the integration model of choice allows for the best leverage of a CDM implementing the integration.
When you add a centralised department that will govern the CDM to the mix, preferably with the organisation's recognised authorities, like domain architects and enterprise architects, and you're in for a treat. Mind that a CDM needs tight governance and is high maintenance, so centralised governance would allow for relatively low costs and huge gains. Even in environments where the number of to be integrated systems is small or even tiny, like in the 1990's and the first 5 years of the 2000's, it is important to have just a single definition of each item in the model.

The Directive Culture

If you assume that you know what is necessary from a business concept perspective, then it is obvious to centralise and decide for yourself what to do next and how. This is how traditionally CDM's came to be around and currently that translates into Enterprise Resource Models. You'll find a group of (self-proclaimed) experts that define an enterprise wide business concept model (ERM) and by decree require everybody to use this model for all inter-operability between business applications. Within the context of a small business scope, like a business unit, this is sometimes doable...

However, in today's complex organisational environments this is no longer feasible. For all sorts of reasons, but within the context of today's enterprises, the complexity, also in vocabulary, of cross domain businesses and the movement to work more and more on short-term goals with ever shorter cycle times, make this approach practically unfeasible. For the simple reason that centralisation automatically also means that you introduce a bottleneck. After all, everything has to go through the same funnel in the organisation. More teams that invoke this centralised department more often generate a higher burden, which can only be solved through organisationally scaling-up, i.e. more people. And that is costly if at all possible. Automation is not a solution either, because this central department has to invent and deliver new things. Think of it as a team of linguists inventing a new language, enforcing everybody to use that language, but not teaching them to speak the language. And at the same time, requiring this team to come up with translations of new words from a multitude of languages into that new language. It's just not doable.

It get worse when it is expected of this centralised team to not only come up with translations, but to actually invent new concepts. And keep in mind that it is often these teams themselves that have this expectation of being able to prescribe the ERM and/or CDM.

The answer to this problem is to decentralise this task. Something that is not unheard of, because crowd-sourcing has been the solution for organisational scalability issues for years. The central role is then one of governance. This requires two major efforts.

1. Clear definition of what the rules are. 
2. Clear processes to maintain the rules.

Mind that both are critical for success and extremely hard to accomplish. Granted, defining a workable set of rules by which to live is hard by itself and requires not only the proper mindset and experience, but also intrinsic authority. But unless there is a workable process to maintain these laws in order to keep them relevant and actually act accordingly is one that requires quite a bit of perseverance.

The Meta Problem

The interesting thing about this is that it is almost a meta problem. Because with the introduction of another centralised function there is again a bottleneck. Scaling-up will again become a problem. However, now there can be some automation. After all, good rules are ideal for automating verification of compliance. When the work is automated, decentralisation of the execution of that work, bringing it closer to the source, nothing stands in the way. From the LEAN idea, that is also a specific form of waste to eliminate.
And this is where it becomes interesting. After all, is it not necessary to check that these automated processes are being carried out? And of course the answer is a well sounding and heartfelt 'Yes'. And if we set off from a Directive Culture, we will of course centralise that control again. But in a Learning Organisation that is of course a no-go, because centralisation does not solve the problem and ultimately results in decentralisation. Much more valuable for the organisation is that on the basis of accountability and responsibility the culture is such that there is a clear sense of ownership in which quality assurance is intrinsic. This is not possible if your starting point is scepticism regarding the ability to take ownership, or in other words to succumb to the tendency to create a controlling authority. The basic assumption must be that quality assurance becomes a common practice that can be realised with (intensive) guidance and positive incentives.

The above therefore argues for and will result in an environment in which trust predominates and is therefore a good breeding ground for a safe working environment.

The Decentralised Models

First of all I want to establish the notion that for an ERM or a CDM to work, we need to remember that these are merely models. There should never be an implementation aspect involved with these models.

For these models to work, it is required that they are broadly supported within the user-base. The model will only be adopted by a user when it is applicable in her problem domain. In other words, the model needs to be relevant. And relevance can only be established when there is practicality.
If you think about this, you'll understand that in general there, initially, not be a single resource model that can be used across the whole organisation, probably not even one that can be used across value-chains. Which of course makes perfect sense, because two separate value-chains implement two separate business products, which in turn will address two separate business needs. Hence, different business concepts will be at play.
It would be futile to assume that one can come up, out of the blue, with a resource model that can be applied in both of the two value-chains. And any attempt to do so, should be stopped, immediately. Instead, two different models should be created and a third as well when a third business product is developed.
Meanwhile the involved architects, both domain and product architects, should start identifying the commonalities and differences in all resource models and formulate a shared model across these value-chains. (Read my post on agile architecting to understand how architects provide a huge benefit to organisations.)
This interaction between architects, especially when you join domain and product architects in this interaction, will result in a core of related business concepts that form a model on which the organisation defines its business products. Generates business value if you will. In a large organisation with several business units, operating in different verticals, it is not uncommon to have several of these core models. It would be the wrong activity to combine these into a single model. Doing so would result in an artificially created model which is not applicable other than in academic discussions and of no benefit to the organisation.

Organic Convergence

The model that stems from this, be it a resource model, a data model, an information model, is one that emerged from practical use. By definition it is therefore applicable and usable. In addition, the model consists of aspects of several models that converged such that commonalities and differences are considered and leveraged to add value.

I call this 'Organic Convergence'.

It is something came up with during my discussions and what logically results from the assumption that within a social group norms and values ​​become a standard, a common ground. The degree of interaction between the members within the group is decisive for the speed with which this happens and the benefits that can be reaped from it. More interaction means faster and more valuable convergence. And in doing so, I foresee that when the different teams work together, forming a social group, uniformity will result. With the provision that the result is broadly supported but not imposed.

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

The bright future of the CC in the agile world

This is the second in a two part blog post on Competence Centers. Read the first post here.

Summarising

The role of Competence Centers in large organisations is diminishing. In many organisations there actually is a notion forming that these centers are obsolete. Considered being a hindrance in an ever more agile environment, the roles of the engineers in these CC's are no longer found in job descriptions. But on close examination, we see that where Competence Centers are fading a way, platform teams are arising as the new home for these experts.

Recap

In my previous post I elaborated on the reasons why in today's day and age of business agility the Competence Centers are no longer wanted. The main driver is most likely the notion that we no longer see IT as a way of doing business more efficiently but more effectively. Take this in for a bit and try to see the significance of this difference in looking at IT.

In my post you could read why the CC is obsolete and why maintaining a CC in an organization that is transitioning towards (business) agility is bad practice.

With that said, I want to explain why I think that the CC still has a place in today’s agile organisations, but in an extremely different way than we’re accustomed to. In my post I hinted to this already in the final part of the post.

Competence Center Classification

Typically Competence Centers consist of experts in a particular functional but more often than not, a technical domain. In my experience, large organisations that benefit from a CC have CC’s addressing aspects like application integration (teams of specialist for a specific ESB product), database systems, infrastructure and networking, mobile app development and often it is the case that there is a team that is very proficient in collaboration software like Lotus Notes, Sharepoint etc. What you’ll most likely also encounter is a group of experts organized as a CC for capabilities like testing, project management and coaching on specific topics (like Agile, Scrum and other new hip buzzwords).

I think that coaching is something that is really of extreme benefit to an organization and having a group of coaches in your organization makes perfect sense. I see them more as grease to the software development machinery than a way to make products more efficiently. Good coaches are the conscience of a team, a department, an organization. They continuously and relentlessly challenge an organization on a variety of aspects.

Capabilities and roles

Read my post on ProjectManagers to read more about how I feel about their role in today’s product development environments. Competence Centers around testing and other activities are to some degree, a large degree, a waste of time and effort. Don’t get me wrong, you really need to test, but that should be part of the team’s efforts and not, definitely not part of an external, at least to the Product Team, team. Testing is part of the product development cycle and should be done by the Product Team as an integral part of that cycle. It should also be automated as much as possible. The same goes for Requirements Engineering, who are often smart enough not to call themselves a Competence Center, but still act as one.

About Testing and Requirements Engineering a departments alike, there’s only one thing that makes sense, which is that they should be dissolved, their members become part of product development teams, considering they do have an engineering mindset. And the practice of testing and the further development of a testing practice should be instigated by a Test Engineering Guild. Same obviously goes for Requirements Engineering etc. (Guild: [...] an organization of persons with related interests, goals, etc., especially one formed for mutual aid or protection [...]). I really don’t think that it makes any sense to have a testing department, and it is counter-productive to have such a department and have testing done outside of the Product Team.

Architecture Centers of Excellence

There’s also the typical architecture group in many large organisations. They are often working in isolation of the developers. There’re a multitude of reasons why there is a distance between developer and architect and often it is due to management expectations. This also results in a reputation that is far from good when it comes to architects. Read my post on the role of architecture and architects in an agile environment. I really am convinced that architecture is a key aspect of real business agility, real meaning measurable instead of gut-feeling and goose-bumps. Especially when an organization needs to be agile in order to work on their long term strategy, i.e. adopt business agility as a strategy, they need to have architecture at the core of their product development. Architects should not be in an Architecture Competence Center, often called Architecture Center of Excellence. Instead, they should be extremely close to their user-base and form a guild. Senior management should abolish any formal group of architects and facilitate an architecture guild, up to the point where active architects in such a guild should be openly credited for their practical contributions to their users.

Back to the more technical CC’s.

The Engineering profession

As I mentioned before, CC’s are often consisting of an organisation’s experts. Which makes sense because the whole of the organization should be able to benefit from their expertise. Mind that this notion and the creation of a CC means that the organization understands the problem of scaling. Experts can’t be scaled up, i.e. cloned, so their time is booked as efficient as possible.

Because the CC will become a bottleneck, we need to find something different to solve this problem. And there is a very effective way of doing so.

Technical Competence Centers should be transformed into Platform teams. Platform teams are teams that develop a product, just like Product Teams, but instead of building business products, they create a platform onto which products can be run. The clearest example would be a hosting platform, developed by a traditional infrastructure team. Business products need to be hosted and instead of every Product Team catering for its own infrastructure to host their products, the hosting platform is leveraged. This happens automatically when products are hosted in the Cloud. The likes of Amazon, Microsoft and Google to name three behemoths, understood early on that infrastructure should be a commodity. Since hardware has been a commodity since the advent of the PC, and with the advent of virtualization technologies server hardware joined these ranks, infrastructure as a service (IaaS) has become the norm. Mind that there is still a lot of proprietary hosting platforms out there, we call it on-premise, but what it entails is a sub-standard platform.

Too often I hear that SysAdmins are reluctant to create a true platform of their on-premise systems because of job security. Cynics stating that once they’ve created a user-friendly experience in their data centers as found in the cloud, the SysAdmins are out of a job, are in my opinion wrong. I think that unless infrastructure in the data center is made as accessible to developers like the Cloud is, the SysAdmins will be out of a job. The fact that all around me I hear people nowadays referring to infrastructure as a ‘hosting platform’ instead of server infrastructure as was the case several years ago, means at least to me, that SysAdmins understand this.

But infrastructure is not the only platform to be considered. Take any of the examples of CC’s I gave earlier on in this post. For example the database experts should be working on a database platform instead of creating and maintaining databases. The database team would create a platform that allows for Product Teams to host their database on. It would ensure all compliance aspects, performance, security, resilience etc. The database platform would be an extension of the hosting platform, building on top of it. Consider the situation where the supported databases are SQL Server and MySQL. Product Teams would use these when needing a relational database. But when requiring a NoSQL database, say Mongo, they would do it by themselves on the hosting platform. Who would be better equipped to create a database platform then the database experts that where in the Database CC?

A similar approach would be taken with the former Application Integration CC. They would develop an Integration Platform that other teams can leverage for their integration needs.

From required-to to required-by

The point with a platform is that it is a product as well. So there are users of the platform and it is the Platform Team’s task to ensure that their product is adding value to the business. This means that the platform they create supporting a more agile business, that the platform is user friendly and addressing the user’s needs as much as possible. The user here is the Product Team or the Product Teams building their products using these platforms.

There is an immense shift in mindset here that you should pay considerable attention to. What I mean is that the expert in the CC was required to be invoked in the product development process as part of the process. The product had to be integrated with other products, so the Integration Expert had to be involved. If not only because the means of integration are so complex that mere mortals shouldn’t be allowed to access them. The same goes for databases. Nowadays almost everything needs a database, so in almost every situation the database experts had to be invoked to create one for the Product Team and maintain it.

The CC’s, being the result of rigorous cost-cutting and an efficiency-fetish, have become integral parts of the product development process. Policies dictating their involvement. But platform teams are different. They deliver a product that can be leveraged by Product Teams, but there is always an alternative if not only to do it yourself. The sense of autonomy in agile environments, dictates that Product Teams decide themselves how to develop their products. This holds true for every product, including platforms. So our Competence Experts need to shift from a ‘you have to run it by me’ attitude to a ‘you will want to run it by me’ attitude. And that’s a huge shift.

Many of the people I know that are part of a Competence Center, a Center of Excellence or similar organisational body are really having a hard time wrapping their heads around this. When explaining to them that they’re now supposed to create a product that people will have to want to use instead of just have to use, they have a hard time to fathom what this entails. In my experience, platform teams are among the teams that are the most urgently in need of intensive coaching on concepts like agile and lean.

Not Platform but Product

These teams are often called Competence Centers, but they are merely Product Teams grouped around a certain business aspect using specific development technology. The classification of ‘Competence Center’ is not warranted and these teams should avoid becoming a platform team and instead focus on obtaining business knowledge. Since they are in fact developing business products. Interestingly enough, these teams are more and more converted into Competence Centers although they are probably the teams that are actually in close contact with the user. Being a specialist in a certain technology, or being a team that uses a specific technology to create value doesn’t make you a Competence Center, and certainly not a Platform Team.

Concluding

The role of the Competence Center in large organisations is diminishing as agile development methods are adopted. The shift from efficiency towards effectiveness is emphasising this even further and CC’s will be obsolete in most if not all cases. Does this mean that all efforts exerted in the past in honing the skills of the experts corralled in these centers were in vain?
Absolutely not. Guilds, Platform Teams and new Product Teams are taking hold and further enhance the agility of the business. Provided organisations are adopting as part of their Agile Transformation strategies also concepts around Autonomy, Facilitating Leadership and User-Centric Design, the future of those that once formed Competence Centers will be bright.

Thanks once again for reading my blog. Please don't be hesitate to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Please subscribe to my blog to show your support.

Arc-E-Tect

The 'obsoletion' of the CC in an agile world...

... or 'The incompetence of the CC in an agile world'. This is the first in a two part blog post on Competence Centers.

Summarising

Competence Centers have no place in agile environments. They're a remnant from the past in which IT was considered to be something to reduce costs instead of creating value. In the new world, CC's are incompetent, which means they should be dissolved and the freed brains of the specialists are available to work on other (business) challenges.

They still exist, you see them everywhere, the larger the organisation the more prevalent they are. The CC and I don't mean Carbon Copies or those long lists of recipients in emails that the sender wants to be aware of the message she's delivering. I am talking about Competence Centers.

What is a Competence Center?

Competence Centers, once upon a time, made sense. Not always as clearly as one would think, but with an overzealous tendency of making IT more efficient, centralization of capabilities seemed to make perfect sense. You can find these centers in 'normal' factories as well, specialised areas in the factory that is optimised to do something special. More on the topic on efficiency and effectiveness can be found in this post. But obviously every decent book on Lean Manufacturing would be fine as well. And while you're at it, take a look at the classic 'The Goal' by Eliyahu M. Goldratt.
There's a huge benefit of having Competence Centers, CC's, in organisations where you want to improve IT's efficiency, especially when the aim is to improve cost-efficiency. In other words, when you want to reduce IT costs to the max, you're best of by introducing Competence Centers. Mind that CC's are to reduce cost, not improve production. This is a misconception I see a lot.

Before I start explaining the benefit of having CC's in your organisation. Let me explain what a Competence Center is. Here's what Gartner has to say about it:

competency center

An organizational structure used to coordinate IT skills with an enterprise. Competency centers provide expertise for project or program support, acting both as repositories of knowledge and resource pools for multiple business areas. Skills-based competency centers, the most common type in an information services organization, are used for application development, software language skills, data management, Internet development and network design. Within the enterprise, it is increasingly common to find competency centers (or shared services) for travel, finance and human resources. Repository-based competencies act exclusively as sources of information.

As things with Gartner go, ... they're typically right on target, but so easy to misinterpret. And then there's the problem with dates. Most people that read Gartner reports forget to check the date. Every article Gartner publishes is dated. And as you know, in today's world, a year or two is a lifetime.

Anyhow, Gartner's definition of a CC is spot on for most enterprises. But just to be sure, within the context of this post, a Competence Center is a centralised organisation body in which a specific capability or function is concentrated to be used and reused by as many teams, persons, departments within the organisation.
The benefit of having a CC is obvious. By concentrating an organisation's expertise into a single department, the organisation is in a position to leverage that expertise to the fullest... in terms of efficiency.

Within a traditional organisation this makes perfect sense. IT is enabling the organisation to be as efficient in conducting its business as possible. This can only be achieved when IT itself is extremely efficient. Resources within IT are therefore to be utilised as close as possible to 100%. By introducing CC's, the organisation can achieve this goal. Especially because the work done by the CC is specialised work. By putting all experts in a single department or even a single location, the experts can optimise their processes and standardise their work.

Why Conway is relevant?

This works perfectly in environments where business products are developed in relatively long lasting projects with separate phases for analyses, design, implementation, testing and deployments; Waterfall projects. The CC's work really well because they are required to act for a specific project on preset milestones in which they are asked to deliver one or more of their specific standardised solutions. And they are invoked only once or twice during the project, everything is optimised around the 100% utilisation target.
Scaling is either not necessary or required within limitations. Conway's law is at help here as well. In short, what Conway says is that the social structure of an organisation, the organisational structure, reflects the architecture of the software that is developed/used within the organisation. Look around and you see this holds true for pretty much all organisations.

So, with Conway on our side, we can state that CC's and their economic justification (i.e. resource optimisation) warrant their existence and strengthens their position as well. The important aspect here is that in a stable system, being an organisation that evolves gradually yet constantly, the requirement for a CC is a self fulfilling prophesy. The Competence Center is therefore a necessity by design.
Which should sound pretty scary to you once you think about it. But there's nothing to worry since it makes perfect sense to have these centers in environment where waterfall is applied to do projects. More over, when governing waterfall projects using PRINCE2 or alike methodologies, it becomes even more likely that the best solution is the use of CC's throughout the organisation.

Disrupting the CC

So, where's the incompetence of the Competence Center you're asking yourself. Well, it lies in the fact that recently there have been a number of extremely disruptive developments in how we apply IT in our organisations.

Scrum

For one there is Scrum. I deliberately don't call it agile. Scrum typically revolves around developers being able to discuss what they're developing with their users in (extremely) short timespans. This means that once a developer is working on something and actually believes that something is finished, the user comes into play to verify, or rather validate that view of the developer. Typically and preferably, the developer will start working on the functionality once it's clear that so far everything is perfect, or at least good enough for the user. What's that got to do with the CC? Well, if the CC comes into the equation and between the developer and the user, there's a CC that needs to work its magic... well it will take longer for the developer to hear that user is either happy with or needs some adjustments to the work of the developer. Because the CC is involved several times over the course of product development and at seemingly randomly chosen times.

Cloud

Then there's the Cloud. A little piece in cyberspace where apparently anything goes as long as you have a credit card. Of course this is initially always the case, but very quickly Cloud usage in any organisation matures. The disruptive part of the cloud with regard to CC's is that the business model of the Cloud vendors like Amazon, Google and Microsoft, is to get as many solutions in the Cloud as fast as possible. Which entails two very important aspects. For one, everything is automated or can be automated and as consequently can be expressed in code. And secondly, everything is abstracted to the point where intrinsic complexity of IT solution is removed by wizards and mouse clicks. Where the likes of Amazon, Google and Microsoft actually make the transition from wizards in a browser to coding in an editor to be almost zero-effort. Point being that the CC can be circumvented easily in Cloud environments, not because it's the Cloud, but because the Cloud is removing the raison d'etre of the CC. It's unique position within the organisation's processes to get things done is not existing in the Cloud.

Continuous Delivery/Deployment

Continuous Delivery/Deployment practices are a third disruptor I want to mention. Here the disruption is caused by the fact that effort is put into reducing the cycle time between idea - product - insights (the build-measure-learn loop from a business perspective) as much as possible. Cycle time reduction is achieved predominantly by automation. CD Pipelines, sets of integrated tools used by teams responsible for getting an idea as a product into the hands of a user and getting feedback from the user are team specific. Addressing the specifics of the product developed and the markets using those products to allow for fine tuning in order to get product/market fit. Together with Lean practices, the Continuous Delivery or even Deployment paradigms build on concepts that help further reduce the cycle time. Which means that there is within a given timeframe more opportunity to go to production. Since the process requires the involvement of the CC, the CC is required more often to act.

What about other disruptors?

Obviously there're a lot more aspects and disruptors that are putting the CC to shame, but the above are the ones I see most often causing a problem with Competence Centers. And the reason why they are so problematic in organisations that rely on Competence Centers is the fact that these are implemented from a mindset that centralisation results in more efficient resource utilisation. Thus resulting in lower cost. And there's your problem, because the CC's are bottlenecks in the process. It's fine when it is possible to plan for their involvement in the process, because one can do some proper resource planning. But when the CC is called to action whenever needed, they're causing delays. The problem is caused by the lack of elasticity, which obviously is needed. And we all know how challenging it is to scale a department up and down. Especially one that was created because of its specialised nature.

Scaling is always a problem

So scaling is a problem and it is needed to be able to support all product developments Just In Time an in such a way that you don't need to plan for their services or make reservations.
The only way to handle this, as you might have guessed is by automating the process steps the CC would perform. Machines are excellent when it comes to scaling. So the CC can only be competent within the agile enterprise when it automates its own activities.
Far fetched? I think not! Think about IT infrastructure teams that are extremely specialised. Provisioning servers, connectivity and storage is not a trivial task. You need specialists to do this, especially when it needs to be done within quality margins. And infrastructure was one of the first areas where automation was used to be able to support the JIT characteristics of agile product developments. Automation and virtualisation resulted in... tada... The Cloud.

Automation has been the culprit for the demise of the infrastructure teams, sort of. Because the automation functionality was given to the product developers, who started provisioning infrastructure from a self-service portal.

The trend is that CC's are becoming bottlenecks in processes. The problem is solved by widening the bottleneck through scaling of the CC by adding resources. Which is costly if at all possible, and automation is used to fix this. Automated processes are fool-proof in many cases, and therefore development teams can access the automated processes all by themselves. The CC is rendered obsolete.

Efficient or Effective?

Is this a problem? Not from an enterprise perspective, because less CC's means less costs. So more (cost) efficient. And on top of that, or even more importantly, the CC's are obsolete because their tasks are automated, therefore more consistent in execution and they can be executed at will. Or if you will, just in time. So more effective as well as more efficient.
From a human resources perspective, there is a challenge. Because the experts in the CC are no longer required since the machine made them obsolete. Or did it? In fact, the specialist made herself obsolete. As an organisation you want to cherish these specialists because they understand how to fix process bottlenecks, often have an engineering mindset and are most likely able to adjust and become a specialist in other areas as well. Or even more desirable, they become platform engineers that will maintain and further develop the platform that was once maintained by the CC. Not in a manner of keeping the lights on but further developing the platform. Just like the infrastructure engineers created the Cloud platform and continuously develop it. Not as a Competence Center, but as a platform team.

Hence, Competence Centers are incompetent in agile environments, but that's an opportunity as it means that perfectly automatable functions no longer keep the engineers and specialist busy with repetitive tasks and allows them to work on not automatable functions we tend to call platforms.

Thanks once again for reading my blog. Please don't be hesitate to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Please subscribe to my blog to show your support.

Arc-E-Tect