Translate

April 6, 2018

Metrics Matter - An Introduction to a series on Measurements in an Agile world.

This will be a series of posts on metrics, KPI's and measurements as they relate to agile concepts, teams, organisations, etc.
I started on these posts some time ago after numerous discussions with several of my clients on the topic. My initial post started to become really long and after I discussed some of the concepts with peers at the Lean Startup Summit Amsterdam 2018 I decided to have several posts on the topic of metrics, starting with measuring agile teams.

Before I dive into the topic of metrics and agile teams I want to highlight a couple of observations related to this.

First of all is the notion of agile transformations. In the last decade there have been a number of large organisations that made the transition from a traditional hierarchic organisations embracing a risk averse policy when it came to business development towards a more innovative and iterative stance where experimentation and feedback cycles were employed to limit risk. Companies like ING Bank, G.E., Intuit and several more are well known and well documented. One of the most heard reasons for doing this transformation relates to an ever faster changing environment in which these organisations operate. In many occasion I have heard that the democratization of IT infrastructure like the cloud has been a key player in setting the urgency to transform for these organisations. Where as in the past competitors faced the same problems as these organisations sharing the same risks, new players in an ever more crowded market (any market) have adopted a new model of operation in which these challenges and risks are no longer a factor.

New and foremost nimble organisations commonly referred to as startups have surfaced and without the lack of legacy both in investments as well as organisation and culture are more capable of addressing the market's needs than their large(r) counterparts. Often the Chinese proverb 'death by a thousand cuts' is used when referring to large corporations loosing market share to small companies.
The realisation that the new kids on the block are formidable and sometimes fierce competitors, plays a key role in these 'agile transformations'. This all happens in a market that is more and more addressed by regulators. Not so much to control and protect national investments, but to open up internationally allowing foreign players into domestic markets and to support competition.
What I see happening all around me with clients of every size is the realisation that without innovation, business fear for their sustainability. An important but often missing realisation is that the lack of innovation in an organisation is the result of a rigorous focus on costs instead of value. The larger the company, the more focus on efficiency in the past. I have written about this before, read the post [Perish or Survive, or being Efficient vs being Effective] for example.

What I see happening a lot is that organisations are trying to redefine their focus but maintaining their means of measurement, i.e. the metrics used to validate the organisation's progress are left unchanged while transforming the organisation from one that's efficient into one that's effective. Or in different terminology: The Agile Transformation.

Caveat emptor: Unless your objective is to become more effective and are willing to move your drive for efficiency to the background, an agile transformation is a lost cause and a waste of resources.

Assuming you're doing an agile transformation for all the right reasons, you will get to a point where you'll realise that you will need to change your KPI's and more importantly, you will need a set of metrics that allows you to validate the effectiveness of your transformation.
Just a little terminology definition is warranted here. I know the theory behind metrics, KPI's, measurements, lagging and leading, etc. But I am maintaining a rather simplified view on things:
  • KPI's, Key Performance Indicators, are typically found in reports. 'Past perfect tense' is their use and they indicate whether or not you've reached your goal.
  • Metrics are typically found on dashboards. 'Future time' is their use and they indicate whether or not you're on the right track or are in need of a change.
Forget about all the other theory around KPI's and metrics and all. To understand these posts, this is all you need to know. In all fairness, I think KPI's are something of the past when we are talking about agile transformations. Well at least when it were up to me. And this is not because KPI is an acronym but because I think the better measurement would be the KVI, the Key Value Indicator. The difference is that instead of performance we measure value. We're not measuring how efficient we are, but how effective we are instead. Cost vs. Value.
So I'll also talk about KVI's instead of KPI's in this and the following posts. It is also more in line with autonomy, self-steering teams and the more recently hipster term OKR's (Objective and Key Result) which will be addressed in one of the posts in this series as well. And in particular why you should be very careful with applying them in your organisation.

In this series of posts I will cover the following high-level topics:

  • Measuring Agile Team performance. In this post I will also address the challenge of maintaining the integrity of the Team-accountability while looking at individuals as well as teams when measuring performance.
  • Agile Maturity. This post will cover the aspects of maturity scans, their value and why you shouldn't care.
  • Agility Metrics. A post that will address how to implement metrics that measure your agility. Especially valuable when in the midst of an agile transformation.
  • Business value and Metric Harmonisation. An important aspect of keeping stock of your KPI's (and KVI's) is making sure that between teams they are not conflicting.
  • Visualisation and Dashboarding. I'll explain why there is no single one dashboard. Why visualisation is extremely important when using metrics and how dashboard, metrics and proper visualisation ensure a lack of surprises in your KPI's. Which will highlight the necessity of using KVI's instead.
So that's at least 5 more posts on the topic for you. Depending on the time I have to spare in the coming period, you'll be able to find the posts sooner or later. Please keep in mind that I will most likely post some articles on other topics as well. I'm dying to shed some light on the challenge of BizDevOps in large organisations. The real challenge I mean.


Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect


The text very explicitly communicates my own personal views, experiences and practices. Any similarities with the views, experiences and practices of any of my previous or current clients, customers or employers are strictly coincidental. This post is therefore my own, and I am the sole author of it and am the sole copyright holder of it.

April 5, 2018

Metrics Matter - Measuring Agile Teams

This will be a series of posts on metrics, KPI's and measurements as they relate to agile concepts, teams, organisations, etc. Sounds familiar? If so, better skip to the section Measuring Agile Team performance otherwise it might get rather boring.
I started on these posts some time ago after numerous discussions with several of my clients on the topic. My initial post started to become really long and after I discussed some of the concepts with peers at the Lean Startup Summit Amsterdam 2018 I decided to have several posts on the topic of metrics, starting with measuring agile teams.

In this series of posts I will cover the following high-level topics:
  • An introduction to this series.
  • Measuring Agile Team performance. This post.
  • Agile Maturity.
  • Agility Metrics.
  • Business value and Metric Harmonisation.
  • Visualisation and Dashboarding.
So that's at least 5 more posts on the topic for you to read. Depending on the time I have to spare in the coming period, you'll be able to find the posts sooner or later. Please keep in mind that I will most likely post some articles on other topics as well. I'm dying to shed some light on the challenge of BizDevOps in large organisations. The real challenge I mean.

Measuring Agile Team performance

In this and the other posts in this series, I will use a metric-analogy that has proven very helpful in the past. The analogy is one of driving a car. As you will see, driving a car is very much like managing a team, especially an agile team. And don't be alarmed, even when you don't have a driver's license or when you've never driven a car yourself, the analogy helps.

While reading this post, you might think that the post is about agile teams related to software development. This is not intended and I refer to the teams in this post as product development teams. These are teams that create a product that is to be used by somebody, the user. This could very well be a computer program, but just as easily a security policy, a reference architecture or even a toilet-bowl.

Key Performance Indicators


When we talk about agile teams, we typically talk about Scrum teams. But just to be clear, I want to make sure that you understand that when I talk about agile teams, I talk about Product Teams. Product Teams are teams that are responsible for the delivery of a product to a user of the product. And often that is a Scrum team since most of the organisations I know that are in the process of an agile transformation, or already have performed this transformation. But this is besides the point, for now.

Measuring an agile team's performance is basically the same as measuring anything else. Although, as we do in IT, we don't treat it the same.
When we measuring a team, we typically want to be able one of two things.
  1. Know how the team is performing.
  2. Know how the team has been performing.
And in most cases I've come across in the past couple of years, we don't want to know anything about the team, but about the team's members. The reason for this, is that we have performance agreements defined on the level of the individual employee of an organisation. Usually in the form of SMART objectives. Most, if not all organisations will in fact have these objectives defined in terms such that the employee's efficiency within the organisation is measured. Which makes perfect sense when you understand that the performance of anything is in most cases associated with its utilisation.
Key Performance Indicators, KPIs, therefore are almost always associated with utilisation and thus with efficiency. Too bad that doesn't make sense in an agile environment since agile is not about efficiency but about effectiveness.
Measuring the performance of an individual team member or even a team is not helping in determining how the individual or the team is performing. And in my experience it is actually counterproductive and not supporting the agile transformation when your organisation is in the middle of one. We will see that we need to define agile performance either in terms of value creation (preferably business value) or in terms of an agile team's behaviour. This post is about the latter, where the first is all about Key Value Indicators, KVIs.

Back to metric basics


Going back to the basics of metrics leads us to reconsider why we do take measurements. And the main reasons I can think of are that you want to be able:

  1. To verify that a change has led to meeting the desired objectives.
  2. To ensure that you are on the right track to meet the desired objectives.
  3. To validate that you're more capable of meeting desired objectives.
As you can see, all these are around objectives, achievements and capabilities. There is nothing there about performance.

Metrics in an organisation


One of the main reasons I advise clients to implement metrics and take measurements is to ensure that it is possible to direct the course of action of an organisation. When it comes to measuring a team you want to make sure that the team is meeting the desired objectives, or at least is on the right track or that it is more capable to meet these objectives. And more importantly, the objectives are business objectives.
The more challenging part is that you want to be able to combine this with autonomy. In fact, from a leadership point of view, you want to facilitate the team to have as much autonomy as humanly possible. Which boils down to a trust issue. The amount of autonomy is based on the amount of trust. The more trustworthy a team is, the more autonomy you're willing to give the team. But I am getting ahead of myself.

One of the perks of a team in an agile environment is autonomy. The reason being that the fewer times a team needs to ask and wait for permission, the faster it can react on feedback. (It isn't for nothing that in many organisations that have embraced an agile way of working, there is bound to be found a poster that says something in the lines of "Ask for forgiveness, not for permission".) Agility is all about being able to react on a changing world in a timely manner. This is all there is to it.
Logically, the more time is needed to react, for example because certain signatures are required, the less agile a team can be. Consequently, when a team can make its own decisions, it will be able to react faster on feedback and hence the team is more agile. So the team needs to be able to process the feedback loop, whatever the feedback loop is.

There is a catch in all this, especially when you do come across that poster I mentioned before, the one about forgiveness and permission. The catch being that there needs to be a trust-relation with the team. And not only within the team, but with the rest of the organisation and the team as well. Trust in this perspective is what makes or breaks the agile ecosystem.

Trust


I think the first part of this trust relationship is between the product team and the Product Owner. There is always a discussion whether the Product Owner is part of the team or not. In my opinion the Product Owner should not be part of the team, but instead is the link between all stakeholders and the team. The PO is accountable for the products the team creates and is accountable for the business value of the products. (Mind that I am talking about accountability, because I am genuinely convinced that metrics and measurements are directly linked with accountability.) Where the PO is accountable, it is the team that is responsible for the product being delivered. It should be clear that there needs to be a relation of trust between PO and PT, obviously. Because when you’re held accountable for something that somebody else is responsible for, you need to be able to trust that other person. And when that trust is not there, you want to apply as much control as possible. The normal reaction of anybody that is given a task to do, but is also told how to do it, would be a clear “Don’t you trust me?”. For many managers, especially in the more established organisations, it will be a challenge to tell their direct-reports to do something without telling them how to do it. They will have this attitude only towards those that are identified as the manager’s protégé or right hand.

Now take it a step further, which I think is required in an agile environment, if not only because otherwise things wouldn’t scale. Taking this step will mean that the team is not told what to do, let alone how to do it, but instead the team is told to meet an objective. The PO is giving the team an objective instead of a list of tasks. Basically this means that the focus is on meeting objectives instead of executing tasks. The team is responsible for meeting the objective and is allowed to use any means necessary. Albeit within limits. Trust is now about “trusting the team not to overstep set boundaries”, instead of “trusting the team to do as told”. What is interesting with this paradigm shift is that the team is now responsible for getting the job done instead of for executing the tasks at hand. Consequently, trust is gained when jobs are done… and this means that trust is gained when value is delivered. And that is exactly what the PO is held accountable for. The loop is closed.

Read my posts [You know you're the Product Owner...] and [The Power of Accountability] on the topic of PO's and their accountability as well.

Now when it comes to trust in an agile world, we're bound to consider how things work in product oriented organisations. Ideally these organisations work with Product Owners that are held accountable for the product(s) they own. And the Product Owners work closely with product teams to develop those products. The PO has the mandate to change the priority of the features, the functionality, that is developed by the team, which is a (reasonably) stable team throughout the lifetime of the product they develop.

The Accounting Dream


Interestingly enough this should be the financial controller's wet-dream since it takes away one variable from the P&L equation. In a traditional project oriented organisation, year over year there are two variables that determine the P&L of a product development department (e.g. an IT department). For one there is the project portfolio which is different every year and may very well change during the year as well. And secondly there's the variable of the resources needed to do these projects, the people working in the projects. Year-over-year the staffing requirements will be different, resulting in budget headaches as well as staffing headaches.
So on the cost side there is a lot of uncertainty, but also on the revenue site. Because revenue is always expected revenue when it comes to budgets.
Do consider the case where the team working on products is stable throughout the year, and all of a sudden a key variable has become a constant. Mind that typically staff is a major cost aspect of any organisation. Of course the other variable, delivered product or rather delivered value, will remain variable. But that is fine.
I am always very curious why organisations have such a hard time with using stable teams. Especially since in a lot of large(r) organisations the CIO reports to the CFO instead of the CEO or even the COO. Wouldn't the CFO love to have to deal with one less variable in his books? I guess that's why I almost flunked economics classes in high school, I don't get this. My guess is that most of the time, organisations are focusing on efficiency instead of effectiveness, and that means that people need to be kept busy as much as possible, which means that they can't work on a new business proposition when they're already working on another product. So new people are hired instead.

Trust and Agile - I


Having stable teams does not only help the CFO, it also means that a team can be assigned to work on a product and gain business knowledge about the product instead of only technical knowledge needed to develop the product. The value of the team itself in terms of the organisation is increasing as well.
With a PO on the product defining what the team is working on in what order with the proper mandate, should ideally result in on the one hand stable so-called 'pre-financed' teams. On the other hand it should result in objectives to be met by the PO from a business perspective.
Remember that the PO is held accountable for the (lack of) success of the product that is owned. From that perspective, the PO will be extremely motivated to understand the product(s) that are owned and especially the value created with or by the product. Unless the PO does understand this, the product will lack success and that will be on the PO's plate. That motivation will very likely be intrinsic as well. Since a lot of the value will be in pretty much all cases be depending on timing of the release of the (new) product (version) to the users, the PO will need to be able to control this as well. Which is also a major benefit of being in charge of feature prioritisation.
Referring back to some earlier paragraphs in this post; the PO is accountable, but the team is responsible for the development of the product and its features. And with quality already covered as being a matter of trust. It is now time to consider timing a key aspect of the trust-relation between the PO and the Product Team. In project oriented organisations the quality aspect is also an important trust relation between PM and the Project Team. But in an agile organisation, the PO is depending on the timeliness of his product as agile means being able to respond in a timely manner to a changing market. Hence the PO will need to be able to also trust the time aspect along with the quality aspect of the Product Team as both directly relate to the success for which the PO is held accountable.

Just as a side note. In almost none of the organisations I have worked with, a project manager was held accountable for either the budget or the timelines. In fact the PM is typically receiving both as constraints or requirements to be dealt with. The PM is responsible for meeting timelines and staying within budget, but since there is no mandate for either, the PM cannot be held accountable. You simply can't be held accountable for something you have no control over. Even legally this is the case.

Gaining trust


I think that the general theme of this post is about trust. I am convinced that the relationship between PO and team is one primarily determined by trust and respect, but typically respect is coming with trust. Automatically.
Even between the PO and the customer there's a trust-relation. But since this post is about measuring agile teams and not the PO, I'll refrain myself from that aspect in this post.

The general question is: How does a team gain the trust of the PO? The answer is quite simple: The PO must be able to depend on the team. Which does make perfect sense, since the PO depends on the team when it comes to the PO's accountability.
As a consequence, a product team's performance is closely related to its dependability. Which you'll likely consider counter intuitive. Reason being that performance is typically associated with speed. Adding to this that traditionally we add the cost factor to it, and we associate performance with efficiency as well. So good performance is the highest speed at the lowest cost. Which in an agile environment doesn't make sense.
First of all, agility is about effectiveness instead of efficiency, so forget about lowest cost start thinking about highest revenue. Agile is about being able to seize the opportunity. Respond in a timely manner, remember. The speed factor is about being able to change direction while still delivering value.
With this, gaining trust means showing consistency in delivering value and ability to respond.

Measuring performance in terms of Agile Traits


With the above in mind, we now know that we need to measure performance of a team in terms of agile traits. Because when we talk about agile, we talk value driven. And that is business value. So when we talk about performance, we talk about business performance and not about performance in a technical sense. And business performance is all we should care about in the first place.

How do we measure business performance in terms of agile traits? That's easy, we need to make sure that we have metrics in place that focus on what we want to know, which is: How dependable is the team? And the one (and only?) person that should worry about is, is the one that depends on the team: The Product Owner.
In my experience there are only two aspects here that make sense to have metrics on. First of all we need to accept that trust to a high degree is emotional, subjective, so we need to make it objective. And that can be done by measuring a team's commitment. Secondly, we have to take quality into consideration. This is because the PO depends on the quality of the product, because only something that is actually used can deliver value.

Commitment, in my experience, is best measured in keeping track of what a team commits to working on during a sprint. Since the team will tell the PO what they can do in the allotted time and a good Scrum Master will see to this, the team will commit itself to deliver those features, that functionality. And we also need to keep track of what the team actually delivers at the end of the sprint, because that gives us a clear understanding of not only what is being delivered, but by looking at the difference between what the team was committed to deliver and what was actually delivered we know whether a team over- or under-commits. Over time you want this difference to be 0.
And since there always is a subjective part on the trust side, the PO will have to take the burn-down chart in consideration as well. As long as the PO is available to review features that are reported to be finished as soon as they are. And as long as size of the work being done is small enough, the burn-down chart will be more or less a straight diagonal downwards, starting at the top on day one of the sprint and all the way down at the bottom by the end of the sprint. Any deviation will require investigation.

As a side-note: Mind that this also requires small work items. My experience is that the best way to do this is when the team takes as a rule of thumb that a work item needs to be 'finish-able' within a day, max 2 days, but preferably one day. This has a number of benefits. One being that at the team's stand-up ritual the PO can listen in and have an overview of what will be made available to be reviewed by the end of the day, so time can be reserved to do that review and feedback from the PO can be input to next day's stand-up (via the Scrum Master or the developer). In addition, anything that wasn't finished most likely will need help from another team member, which can already be signalled after a day. Another benefit is that, especially when working with Behaviour Driven Development (or Test Driven Development), every story addresses a single behaviour.

What you want to see from a team is that the difference between committed and realised work items is close to zero with a steady burn-down chart. When this is what you get as a PO, you know that your team is dependable when it comes to promises.
Then there's the quality of the work being done. This is a little more involving, but shouldn't be too hard. In general it means that it is being tracked how often the product doesn't work/perform as advertised. The challenge here is more often than not that there needs to be a feedback-loop in place in order to be able to collect this metric. A truly mandated PO will have the loop in place, but I see more often than not that the loop is not there, typically because the PO is not truly mandated. The fall-back scenario is counting incidents. But ideally you want to keep track of the user's perception as more often than not the main quality problems are related to user-experience.
Nevertheless, regressions need to be tracked as well. Maybe this is even more important than first-time issues. Because regressions mean that the team's product development lack quality, which is far more serious than a product lacking quality.

Conclusion


Concluding there are only three metrics required to get a good grasp of the (business) performance of a team:
  • Difference between committed and delivered by the team.
  • Number of 'incidents' where the product doesn't behave as advertised.
  • Number of times regressions occur.
Based on these metrics, the PO will be able to judge whether or not the Product Team is dependable and thus can be trusted with the responsibility to develop a product with which the PO can create business value for which the Product Owner is accountable.

To-be-kept-in-minds 


There are some things to be kept in mind on this topic. For one, there's the metric velocity. The velocity of a team is a team internal metric and I really want to advice to not ask for it at all. Velocity is based on extremely subjective metrics, typically story points. These can be gamed in order to maintain a constant velocity. Which defeats the purpose of story points, as they are a team internal indicator regarding what the team will find on its path during the sprint. It can refer to complexity, dependencies, etc. Anytime story points are related to time to be spend you know you're using them wrongly.
Secondly, although it is very tempting to compare teams, you should try to refrain yourself from it. Instead you should look at how teams compare considering in progression based on the metrics. It is way more important to know whether a team is evolving, improving, or not. Teams that do not improve or not as fast as other teams are most likely facing an impediment for their growth, one that the PO cannot control. This is where intervention is required from management.
Thirdly, in general, and especially in agile environments, metrics should be based on objectives and achievements and not on tasks and activities. There is only one reason why one would want to know about what a team is doing, or rather how a team is doing things, and that is because you want to distil practices and identify best-practices.

But what about Team Happiness?


That is an interesting question. Team happiness is an important metric to measure a team's performance as well. This is not so much a metric from an agile perspective. I would say it is a leadership metric as leadership should be held accountable for the happiness of the teams. I will cover this in another post. As other metrics that are important.



Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect


The text very explicitly communicates my own personal views, experiences and practices. Any similarities with the views, experiences and practices of any of my previous or current clients, customers or employers are strictly coincidental. This post is therefore my own, and I am the sole author of it and am the sole copyright holder of it.

March 18, 2018

The natural beauty of organic convergence

... in an agile world.


Summarising

I am a big proponent of the concept of 'Organic Convergence'. Something I have come up with and what goes on from the assumption that within a social group norms and values ​​become a standard, a common ground. The degree of interaction between the members within the group is decisive for the speed with which this happens and the benefits that can be reaped from it. More interaction means faster and more valuable convergence. And in doing so, I foresee that when the different teams work together, forming a social group, uniformity will result. With the added benefit that the result is broadly supported but not imposed.


The world is organic and evolutionary. At times it is revolutionary, then we are dealing with disruption. Disruption is beautiful because that turns you into a market leader and increases your chances of survival. This, of course, only when you are the disrupting force.

So where is this coming from? Well, it's coming from a discussion I had several times in the past weeks about Canonical Data Models (CMD), Enterprise Resource Models (ERM) and centralisation of governance in an organisation that just decided to execute a full-blown agile transformation over the coming years.

This organisation's legacy is, like so many other organisations with a challenging bureaucracy, is one of centralisation of capabilities (see my view on this in Perish or Survive, or being Efficient vs being Effective and an duo of posts on Competence Centres in huge organisations here and here) and a fairly directive management style. Although, as in many organisations, this one has come a long way, but still the overall cultural view is that teams are not autonomous nor self-proficient and Enterprise or Domain Architects are to define standards across the enterprise. Mind that these standards are not on enterprise level topics like principles, frameworks, etc. No in many cases the standards are about what technology to use, what architectural framework to use, or what model to implement. Down to the level of programming language and style.

Some context on the CDM

In this section I am elaborating on the concept of a CDM and why it is a remnant of centralisation in IT organisations. You can skip to the next section if you would like so.
The discussions I referred to at the beginning of this post were about whether or not an enterprise wide model had to be enforced top-down. Or possibly should emerge bottom-up. The discussion was about CMD (Canonical Data Model) and ERM (Enterprise Resource Model). Now the thing here is that the 'M' in the naming is not helping. We're not talking about just a model, but also its implementation. The CMD is a remnant from the days in which Enterprise Application Integration (EAI) was something new. In the late 1990's and early 2000's integration of software systems and thus creating synergy was key in many IT organisations. Different systems used different information models. What one system considered a Customer would be different from another system's definition. Typically across business domains these differences were huge. Then there is the problem where two departments in an organisation have different names for the same concept. Understandably, the CDM is going to be extremely helpful. In many ways, a CMD resembles the synthetic language Esperanto.
But unlike Esperanto, the CDM is a model. It's a definition and not an implementation. It becomes all the worrisome once the CDM gets implemented using one single implementation. The problem here is that definition and implementation all of a sudden become one. This is a problem because implementations require technical specifications like data-types and formats, which diverts the discussion from semantics towards syntax. Or in other words, the CDM becomes an IT concern instead of a business concern.
Typically the CMD gets implemented in a centralised part of an architecture. The main database, the ERP system or as part of the ESB. And historically this sort of made sense. When the CDM was invented, it had to also be implemented and preferably in a single location. Considering the focus on efficiency in IT, centralisation was paramount. Definition and implementation would go hand in hand, developed in parallel. Because of the challenges at hand, mainly the integration challenges, the CDM evolved into the 'language' on the ESB. Everything on the bus would speak the same language, when needed a connector would translate from and to the common language.

In those early days of EAI, the challenges were merely integrating the organisation's own systems. All within the organisation itself. Considering the context, a CDM made perfect sense and I would argue that based on the peculiarities of these environments, having a CDM and use it as the common language on the bus is the right choice. Think about the CDM not being Esperanto, but Hindi in India. Within a very neatly scoped context it was a common language.
In many occasions I have struggled with inconsistencies in information models for different enterprise systems and a CDM would have been or was the best solution. In my experience, the best way to deal with a CDM is on the ESB. Selecting the ESB as the integration model of choice allows for the best leverage of a CDM implementing the integration.
When you add a centralised department that will govern the CDM to the mix, preferably with the organisation's recognised authorities, like domain architects and enterprise architects, and you're in for a treat. Mind that a CDM needs tight governance and is high maintenance, so centralised governance would allow for relatively low costs and huge gains. Even in environments where the number of to be integrated systems is small or even tiny, like in the 1990's and the first 5 years of the 2000's, it is important to have just a single definition of each item in the model.

The Directive Culture

If you assume that you know what is necessary from a business concept perspective, then it is obvious to centralise and decide for yourself what to do next and how. This is how traditionally CDM's came to be around and currently that translates into Enterprise Resource Models. You'll find a group of (self-proclaimed) experts that define an enterprise wide business concept model (ERM) and by decree require everybody to use this model for all inter-operability between business applications. Within the context of a small business scope, like a business unit, this is sometimes doable...

However, in today's complex organisational environments this is no longer feasible. For all sorts of reasons, but within the context of today's enterprises, the complexity, also in vocabulary, of cross domain businesses and the movement to work more and more on short-term goals with ever shorter cycle times, make this approach practically unfeasible. For the simple reason that centralisation automatically also means that you introduce a bottleneck. After all, everything has to go through the same funnel in the organisation. More teams that invoke this centralised department more often generate a higher burden, which can only be solved through organisationally scaling-up, i.e. more people. And that is costly if at all possible. Automation is not a solution either, because this central department has to invent and deliver new things. Think of it as a team of linguists inventing a new language, enforcing everybody to use that language, but not teaching them to speak the language. And at the same time, requiring this team to come up with translations of new words from a multitude of languages into that new language. It's just not doable.

It get worse when it is expected of this centralised team to not only come up with translations, but to actually invent new concepts. And keep in mind that it is often these teams themselves that have this expectation of being able to prescribe the ERM and/or CDM.

The answer to this problem is to decentralise this task. Something that is not unheard of, because crowd-sourcing has been the solution for organisational scalability issues for years. The central role is then one of governance. This requires two major efforts.

  1. Clear definition of what the rules are. 
  2. Clear processes to maintain the rules.

Mind that both are critical for success and extremely hard to accomplish. Granted, defining a workable set of rules by which to live is hard by itself and requires not only the proper mindset and experience, but also intrinsic authority. But unless there is a workable process to maintain these laws in order to keep them relevant and actually act accordingly is one that requires quite a bit of perseverance.

The Meta Problem

The interesting thing about this is that it is almost a meta problem. Because with the introduction of another centralised function there is again a bottleneck. Scaling-up will again become a problem. However, now there can be some automation. After all, good rules are ideal for automating verification of compliance. When the work is automated, decentralisation of the execution of that work, bringing it closer to the source, nothing stands in the way. From the LEAN idea, that is also a specific form of waste to eliminate.
And this is where it becomes interesting. After all, is it not necessary to check that these automated processes are being carried out? And of course the answer is a well sounding and heartfelt 'Yes'. And if we set off from a Directive Culture, we will of course centralise that control again. But in a Learning Organisation that is of course a no-go, because centralisation does not solve the problem and ultimately results in decentralisation. Much more valuable for the organisation is that on the basis of accountability and responsibility the culture is such that there is a clear sense of ownership in which quality assurance is intrinsic. This is not possible if your starting point is scepticism regarding the ability to take ownership, or in other words to succumb to the tendency to create a controlling authority. The basic assumption must be that quality assurance becomes a common practice that can be realised with (intensive) guidance and positive incentives.

The above therefore argues for and will result in an environment in which trust predominates and is therefore a good breeding ground for a safe working environment.

The Decentralised Models

First of all I want to establish the notion that for an ERM or a CDM to work, we need to remember that these are merely models. There should never be an implementation aspect involved with these models.

For these models to work, it is required that they are broadly supported within the user-base. The model will only be adopted by a user when it is applicable in her problem domain. In other words, the model needs to be relevant. And relevance can only be established when there is practicality.
If you think about this, you'll understand that in general there, initially, not be a single resource model that can be used across the whole organisation, probably not even one that can be used across value-chains. Which of course makes perfect sense, because two separate value-chains implement two separate business products, which in turn will address two separate business needs. Hence, different business concepts will be at play.
It would be futile to assume that one can come up, out of the blue, with a resource model that can be applied in both of the two value-chains. And any attempt to do so, should be stopped, immediately. Instead, two different models should be created and a third as well when a third business product is developed.
Meanwhile the involved architects, both domain and product architects, should start identifying the commonalities and differences in all resource models and formulate a shared model across these value-chains. (Read my post on agile architecting to understand how architects provide a huge benefit to organisations.)
This interaction between architects, especially when you join domain and product architects in this interaction, will result in a core of related business concepts that form a model on which the organisation defines its business products. Generates business value if you will. In a large organisation with several business units, operating in different verticals, it is not uncommon to have several of these core models. It would be the wrong activity to combine these into a single model. Doing so would result in an artificially created model which is not applicable other than in academic discussions and of no benefit to the organisation.

Organic Convergence

The model that stems from this, be it a resource model, a data model, an information model, is one that emerged from practical use. By definition it is therefore applicable and usable. In addition, the model consists of aspects of several models that converged such that commonalities and differences are considered and leveraged to add value.

I call this 'Organic Convergence'.

It is something came up with during my discussions and what logically results from the assumption that within a social group norms and values ​​become a standard, a common ground. The degree of interaction between the members within the group is decisive for the speed with which this happens and the benefits that can be reaped from it. More interaction means faster and more valuable convergence. And in doing so, I foresee that when the different teams work together, forming a social group, uniformity will result. With the provision that the result is broadly supported but not imposed.


Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect