The bright future of the CC in the agile world

This is the second in a two part blog post on Competence Centers. Read the first post here.

Summarising

The role of Competence Centers in large organisations is diminishing. In many organisations there actually is a notion forming that these centers are obsolete. Considered being a hindrance in an ever more agile environment, the roles of the engineers in these CC's are no longer found in job descriptions. But on close examination, we see that where Competence Centers are fading a way, platform teams are arising as the new home for these experts.

Recap

In my previous post I elaborated on the reasons why in today's day and age of business agility the Competence Centers are no longer wanted. The main driver is most likely the notion that we no longer see IT as a way of doing business more efficiently but more effectively. Take this in for a bit and try to see the significance of this difference in looking at IT.

In my post you could read why the CC is obsolete and why maintaining a CC in an organization that is transitioning towards (business) agility is bad practice.

With that said, I want to explain why I think that the CC still has a place in today’s agile organisations, but in an extremely different way than we’re accustomed to. In my post I hinted to this already in the final part of the post.

Competence Center Classification

Typically Competence Centers consist of experts in a particular functional but more often than not, a technical domain. In my experience, large organisations that benefit from a CC have CC’s addressing aspects like application integration (teams of specialist for a specific ESB product), database systems, infrastructure and networking, mobile app development and often it is the case that there is a team that is very proficient in collaboration software like Lotus Notes, Sharepoint etc. What you’ll most likely also encounter is a group of experts organized as a CC for capabilities like testing, project management and coaching on specific topics (like Agile, Scrum and other new hip buzzwords).

I think that coaching is something that is really of extreme benefit to an organization and having a group of coaches in your organization makes perfect sense. I see them more as grease to the software development machinery than a way to make products more efficiently. Good coaches are the conscience of a team, a department, an organization. They continuously and relentlessly challenge an organization on a variety of aspects.

Capabilities and roles

Read my post on ProjectManagers to read more about how I feel about their role in today’s product development environments. Competence Centers around testing and other activities are to some degree, a large degree, a waste of time and effort. Don’t get me wrong, you really need to test, but that should be part of the team’s efforts and not, definitely not part of an external, at least to the Product Team, team. Testing is part of the product development cycle and should be done by the Product Team as an integral part of that cycle. It should also be automated as much as possible. The same goes for Requirements Engineering, who are often smart enough not to call themselves a Competence Center, but still act as one.

About Testing and Requirements Engineering a departments alike, there’s only one thing that makes sense, which is that they should be dissolved, their members become part of product development teams, considering they do have an engineering mindset. And the practice of testing and the further development of a testing practice should be instigated by a Test Engineering Guild. Same obviously goes for Requirements Engineering etc. (Guild: [...] an organization of persons with related interests, goals, etc., especially one formed for mutual aid or protection [...]). I really don’t think that it makes any sense to have a testing department, and it is counter-productive to have such a department and have testing done outside of the Product Team.

Architecture Centers of Excellence

There’s also the typical architecture group in many large organisations. They are often working in isolation of the developers. There’re a multitude of reasons why there is a distance between developer and architect and often it is due to management expectations. This also results in a reputation that is far from good when it comes to architects. Read my post on the role of architecture and architects in an agile environment. I really am convinced that architecture is a key aspect of real business agility, real meaning measurable instead of gut-feeling and goose-bumps. Especially when an organization needs to be agile in order to work on their long term strategy, i.e. adopt business agility as a strategy, they need to have architecture at the core of their product development. Architects should not be in an Architecture Competence Center, often called Architecture Center of Excellence. Instead, they should be extremely close to their user-base and form a guild. Senior management should abolish any formal group of architects and facilitate an architecture guild, up to the point where active architects in such a guild should be openly credited for their practical contributions to their users.

Back to the more technical CC’s.

The Engineering profession

As I mentioned before, CC’s are often consisting of an organisation’s experts. Which makes sense because the whole of the organization should be able to benefit from their expertise. Mind that this notion and the creation of a CC means that the organization understands the problem of scaling. Experts can’t be scaled up, i.e. cloned, so their time is booked as efficient as possible.

Because the CC will become a bottleneck, we need to find something different to solve this problem. And there is a very effective way of doing so.

Technical Competence Centers should be transformed into Platform teams. Platform teams are teams that develop a product, just like Product Teams, but instead of building business products, they create a platform onto which products can be run. The clearest example would be a hosting platform, developed by a traditional infrastructure team. Business products need to be hosted and instead of every Product Team catering for its own infrastructure to host their products, the hosting platform is leveraged. This happens automatically when products are hosted in the Cloud. The likes of Amazon, Microsoft and Google to name three behemoths, understood early on that infrastructure should be a commodity. Since hardware has been a commodity since the advent of the PC, and with the advent of virtualization technologies server hardware joined these ranks, infrastructure as a service (IaaS) has become the norm. Mind that there is still a lot of proprietary hosting platforms out there, we call it on-premise, but what it entails is a sub-standard platform.

Too often I hear that SysAdmins are reluctant to create a true platform of their on-premise systems because of job security. Cynics stating that once they’ve created a user-friendly experience in their data centers as found in the cloud, the SysAdmins are out of a job, are in my opinion wrong. I think that unless infrastructure in the data center is made as accessible to developers like the Cloud is, the SysAdmins will be out of a job. The fact that all around me I hear people nowadays referring to infrastructure as a ‘hosting platform’ instead of server infrastructure as was the case several years ago, means at least to me, that SysAdmins understand this.

But infrastructure is not the only platform to be considered. Take any of the examples of CC’s I gave earlier on in this post. For example the database experts should be working on a database platform instead of creating and maintaining databases. The database team would create a platform that allows for Product Teams to host their database on. It would ensure all compliance aspects, performance, security, resilience etc. The database platform would be an extension of the hosting platform, building on top of it. Consider the situation where the supported databases are SQL Server and MySQL. Product Teams would use these when needing a relational database. But when requiring a NoSQL database, say Mongo, they would do it by themselves on the hosting platform. Who would be better equipped to create a database platform then the database experts that where in the Database CC?

A similar approach would be taken with the former Application Integration CC. They would develop an Integration Platform that other teams can leverage for their integration needs.

From required-to to required-by

The point with a platform is that it is a product as well. So there are users of the platform and it is the Platform Team’s task to ensure that their product is adding value to the business. This means that the platform they create supporting a more agile business, that the platform is user friendly and addressing the user’s needs as much as possible. The user here is the Product Team or the Product Teams building their products using these platforms.

There is an immense shift in mindset here that you should pay considerable attention to. What I mean is that the expert in the CC was required to be invoked in the product development process as part of the process. The product had to be integrated with other products, so the Integration Expert had to be involved. If not only because the means of integration are so complex that mere mortals shouldn’t be allowed to access them. The same goes for databases. Nowadays almost everything needs a database, so in almost every situation the database experts had to be invoked to create one for the Product Team and maintain it.

The CC’s, being the result of rigorous cost-cutting and an efficiency-fetish, have become integral parts of the product development process. Policies dictating their involvement. But platform teams are different. They deliver a product that can be leveraged by Product Teams, but there is always an alternative if not only to do it yourself. The sense of autonomy in agile environments, dictates that Product Teams decide themselves how to develop their products. This holds true for every product, including platforms. So our Competence Experts need to shift from a ‘you have to run it by me’ attitude to a ‘you will want to run it by me’ attitude. And that’s a huge shift.

Many of the people I know that are part of a Competence Center, a Center of Excellence or similar organisational body are really having a hard time wrapping their heads around this. When explaining to them that they’re now supposed to create a product that people will have to want to use instead of just have to use, they have a hard time to fathom what this entails. In my experience, platform teams are among the teams that are the most urgently in need of intensive coaching on concepts like agile and lean.

Not Platform but Product

These teams are often called Competence Centers, but they are merely Product Teams grouped around a certain business aspect using specific development technology. The classification of ‘Competence Center’ is not warranted and these teams should avoid becoming a platform team and instead focus on obtaining business knowledge. Since they are in fact developing business products. Interestingly enough, these teams are more and more converted into Competence Centers although they are probably the teams that are actually in close contact with the user. Being a specialist in a certain technology, or being a team that uses a specific technology to create value doesn’t make you a Competence Center, and certainly not a Platform Team.

Concluding

The role of the Competence Center in large organisations is diminishing as agile development methods are adopted. The shift from efficiency towards effectiveness is emphasising this even further and CC’s will be obsolete in most if not all cases. Does this mean that all efforts exerted in the past in honing the skills of the experts corralled in these centers were in vain?
Absolutely not. Guilds, Platform Teams and new Product Teams are taking hold and further enhance the agility of the business. Provided organisations are adopting as part of their Agile Transformation strategies also concepts around Autonomy, Facilitating Leadership and User-Centric Design, the future of those that once formed Competence Centers will be bright.

Thanks once again for reading my blog. Please don't be hesitate to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Please subscribe to my blog to show your support.

Arc-E-Tect

The 'obsoletion' of the CC in an agile world...

... or 'The incompetence of the CC in an agile world'. This is the first in a two part blog post on Competence Centers.

Summarising

Competence Centers have no place in agile environments. They're a remnant from the past in which IT was considered to be something to reduce costs instead of creating value. In the new world, CC's are incompetent, which means they should be dissolved and the freed brains of the specialists are available to work on other (business) challenges.

They still exist, you see them everywhere, the larger the organisation the more prevalent they are. The CC and I don't mean Carbon Copies or those long lists of recipients in emails that the sender wants to be aware of the message she's delivering. I am talking about Competence Centers.

What is a Competence Center?

Competence Centers, once upon a time, made sense. Not always as clearly as one would think, but with an overzealous tendency of making IT more efficient, centralization of capabilities seemed to make perfect sense. You can find these centers in 'normal' factories as well, specialised areas in the factory that is optimised to do something special. More on the topic on efficiency and effectiveness can be found in this post. But obviously every decent book on Lean Manufacturing would be fine as well. And while you're at it, take a look at the classic 'The Goal' by Eliyahu M. Goldratt.
There's a huge benefit of having Competence Centers, CC's, in organisations where you want to improve IT's efficiency, especially when the aim is to improve cost-efficiency. In other words, when you want to reduce IT costs to the max, you're best of by introducing Competence Centers. Mind that CC's are to reduce cost, not improve production. This is a misconception I see a lot.

Before I start explaining the benefit of having CC's in your organisation. Let me explain what a Competence Center is. Here's what Gartner has to say about it:

competency center

An organizational structure used to coordinate IT skills with an enterprise. Competency centers provide expertise for project or program support, acting both as repositories of knowledge and resource pools for multiple business areas. Skills-based competency centers, the most common type in an information services organization, are used for application development, software language skills, data management, Internet development and network design. Within the enterprise, it is increasingly common to find competency centers (or shared services) for travel, finance and human resources. Repository-based competencies act exclusively as sources of information.

As things with Gartner go, ... they're typically right on target, but so easy to misinterpret. And then there's the problem with dates. Most people that read Gartner reports forget to check the date. Every article Gartner publishes is dated. And as you know, in today's world, a year or two is a lifetime.

Anyhow, Gartner's definition of a CC is spot on for most enterprises. But just to be sure, within the context of this post, a Competence Center is a centralised organisation body in which a specific capability or function is concentrated to be used and reused by as many teams, persons, departments within the organisation.
The benefit of having a CC is obvious. By concentrating an organisation's expertise into a single department, the organisation is in a position to leverage that expertise to the fullest... in terms of efficiency.

Within a traditional organisation this makes perfect sense. IT is enabling the organisation to be as efficient in conducting its business as possible. This can only be achieved when IT itself is extremely efficient. Resources within IT are therefore to be utilised as close as possible to 100%. By introducing CC's, the organisation can achieve this goal. Especially because the work done by the CC is specialised work. By putting all experts in a single department or even a single location, the experts can optimise their processes and standardise their work.

Why Conway is relevant?

This works perfectly in environments where business products are developed in relatively long lasting projects with separate phases for analyses, design, implementation, testing and deployments; Waterfall projects. The CC's work really well because they are required to act for a specific project on preset milestones in which they are asked to deliver one or more of their specific standardised solutions. And they are invoked only once or twice during the project, everything is optimised around the 100% utilisation target.
Scaling is either not necessary or required within limitations. Conway's law is at help here as well. In short, what Conway says is that the social structure of an organisation, the organisational structure, reflects the architecture of the software that is developed/used within the organisation. Look around and you see this holds true for pretty much all organisations.

So, with Conway on our side, we can state that CC's and their economic justification (i.e. resource optimisation) warrant their existence and strengthens their position as well. The important aspect here is that in a stable system, being an organisation that evolves gradually yet constantly, the requirement for a CC is a self fulfilling prophesy. The Competence Center is therefore a necessity by design.
Which should sound pretty scary to you once you think about it. But there's nothing to worry since it makes perfect sense to have these centers in environment where waterfall is applied to do projects. More over, when governing waterfall projects using PRINCE2 or alike methodologies, it becomes even more likely that the best solution is the use of CC's throughout the organisation.

Disrupting the CC

So, where's the incompetence of the Competence Center you're asking yourself. Well, it lies in the fact that recently there have been a number of extremely disruptive developments in how we apply IT in our organisations.

Scrum

For one there is Scrum. I deliberately don't call it agile. Scrum typically revolves around developers being able to discuss what they're developing with their users in (extremely) short timespans. This means that once a developer is working on something and actually believes that something is finished, the user comes into play to verify, or rather validate that view of the developer. Typically and preferably, the developer will start working on the functionality once it's clear that so far everything is perfect, or at least good enough for the user. What's that got to do with the CC? Well, if the CC comes into the equation and between the developer and the user, there's a CC that needs to work its magic... well it will take longer for the developer to hear that user is either happy with or needs some adjustments to the work of the developer. Because the CC is involved several times over the course of product development and at seemingly randomly chosen times.

Cloud

Then there's the Cloud. A little piece in cyberspace where apparently anything goes as long as you have a credit card. Of course this is initially always the case, but very quickly Cloud usage in any organisation matures. The disruptive part of the cloud with regard to CC's is that the business model of the Cloud vendors like Amazon, Google and Microsoft, is to get as many solutions in the Cloud as fast as possible. Which entails two very important aspects. For one, everything is automated or can be automated and as consequently can be expressed in code. And secondly, everything is abstracted to the point where intrinsic complexity of IT solution is removed by wizards and mouse clicks. Where the likes of Amazon, Google and Microsoft actually make the transition from wizards in a browser to coding in an editor to be almost zero-effort. Point being that the CC can be circumvented easily in Cloud environments, not because it's the Cloud, but because the Cloud is removing the raison d'etre of the CC. It's unique position within the organisation's processes to get things done is not existing in the Cloud.

Continuous Delivery/Deployment

Continuous Delivery/Deployment practices are a third disruptor I want to mention. Here the disruption is caused by the fact that effort is put into reducing the cycle time between idea - product - insights (the build-measure-learn loop from a business perspective) as much as possible. Cycle time reduction is achieved predominantly by automation. CD Pipelines, sets of integrated tools used by teams responsible for getting an idea as a product into the hands of a user and getting feedback from the user are team specific. Addressing the specifics of the product developed and the markets using those products to allow for fine tuning in order to get product/market fit. Together with Lean practices, the Continuous Delivery or even Deployment paradigms build on concepts that help further reduce the cycle time. Which means that there is within a given timeframe more opportunity to go to production. Since the process requires the involvement of the CC, the CC is required more often to act.

What about other disruptors?

Obviously there're a lot more aspects and disruptors that are putting the CC to shame, but the above are the ones I see most often causing a problem with Competence Centers. And the reason why they are so problematic in organisations that rely on Competence Centers is the fact that these are implemented from a mindset that centralisation results in more efficient resource utilisation. Thus resulting in lower cost. And there's your problem, because the CC's are bottlenecks in the process. It's fine when it is possible to plan for their involvement in the process, because one can do some proper resource planning. But when the CC is called to action whenever needed, they're causing delays. The problem is caused by the lack of elasticity, which obviously is needed. And we all know how challenging it is to scale a department up and down. Especially one that was created because of its specialised nature.

Scaling is always a problem

So scaling is a problem and it is needed to be able to support all product developments Just In Time an in such a way that you don't need to plan for their services or make reservations.
The only way to handle this, as you might have guessed is by automating the process steps the CC would perform. Machines are excellent when it comes to scaling. So the CC can only be competent within the agile enterprise when it automates its own activities.
Far fetched? I think not! Think about IT infrastructure teams that are extremely specialised. Provisioning servers, connectivity and storage is not a trivial task. You need specialists to do this, especially when it needs to be done within quality margins. And infrastructure was one of the first areas where automation was used to be able to support the JIT characteristics of agile product developments. Automation and virtualisation resulted in... tada... The Cloud.

Automation has been the culprit for the demise of the infrastructure teams, sort of. Because the automation functionality was given to the product developers, who started provisioning infrastructure from a self-service portal.

The trend is that CC's are becoming bottlenecks in processes. The problem is solved by widening the bottleneck through scaling of the CC by adding resources. Which is costly if at all possible, and automation is used to fix this. Automated processes are fool-proof in many cases, and therefore development teams can access the automated processes all by themselves. The CC is rendered obsolete.

Efficient or Effective?

Is this a problem? Not from an enterprise perspective, because less CC's means less costs. So more (cost) efficient. And on top of that, or even more importantly, the CC's are obsolete because their tasks are automated, therefore more consistent in execution and they can be executed at will. Or if you will, just in time. So more effective as well as more efficient.
From a human resources perspective, there is a challenge. Because the experts in the CC are no longer required since the machine made them obsolete. Or did it? In fact, the specialist made herself obsolete. As an organisation you want to cherish these specialists because they understand how to fix process bottlenecks, often have an engineering mindset and are most likely able to adjust and become a specialist in other areas as well. Or even more desirable, they become platform engineers that will maintain and further develop the platform that was once maintained by the CC. Not in a manner of keeping the lights on but further developing the platform. Just like the infrastructure engineers created the Cloud platform and continuously develop it. Not as a Competence Center, but as a platform team.

Hence, Competence Centers are incompetent in agile environments, but that's an opportunity as it means that perfectly automatable functions no longer keep the engineers and specialist busy with repetitive tasks and allows them to work on not automatable functions we tend to call platforms.

Thanks once again for reading my blog. Please don't be hesitate to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Please subscribe to my blog to show your support.

Arc-E-Tect