What makes an ideal agile team?

Summarising

In many occasions I am being asked by clients to share my thoughts on how they are (planning to) form their Scrum teams. My recurring comments always are to have the team as small as possible, stable and consisting of full-time individuals, maxing out at 9 persons. Have one or more coaches linked to the teams and make sure that there is a mandated Product Owner.
Make sure that relationships are based on trust and ensure that accountability and responsibility are two aspects addressed organisationally with full support of upper management.

The other day I was asked by a manager of one of my clients to comment on a proposal of one of his vendors. A key vendor that has been under delivering since a very long time, but still considered to be a strategic partner in most of my client's business. The vendor is currently experimenting with a different way of addressing its customers by better aligning its services to the needs of its customers. The vendor is proposing to setup a cross-functional team that will be tasked to deliver relevant products in short cycles.

My client is a little sceptical but willing to give the vendor the benefit of the doubt. As such he's agreeing to become one of the first customers to work according to the new proposed process and is now asking me to see what to look for in the proposal. Being the first customer and thus investing time and effort in amending the current mode of operations with this vendor, there's some head room to 'ask' for specific adjustments to the proposal.

The vendor is going to deliver new products and services in a variety of categories to my client and is proposing to setup a Scrum team. My client now wants to know if there are specific aspects of this team that need to be addressed. Also the role of my client with regard to this team is a point of interest in the negotiations. And obviously the delivery of products and services to my client.

Basically the question is: How should my vendor organise its team in order to better meet our requirements? The answer is reasonably straight forward.

In principle you want to keep a Scrum team as small as possible, that is actually a basic principle. The maximum size for a well-functioning team is 9 people. This has already been discovered by the ancient Romans, no kidding.

I would like to argue to start with 6 or 7 people. Rather 6 than 7 by the way. The reason for this is that otherwise you will get specialisations within the team that come to lie with one person, that will immediately become the SPOC / F, who can not get sick or go on holiday.
The strength of a well-collaborating team lies mainly in the knowledge spread throughout the team as well as complementing personalities of the team. This allows members to challenge, supplement and assist each other. It is a common mistake to see what kind of techniques and technologies you want to use and to find the experts for that. It is a Team and not a Group we're trying to forge. Understand the difference, it's important.

I find the use of FTEs in resource planning of Scrum teams confusing and dangerous. I've seen this plenty of time and it never resulted in anything good. One FTE can be filled in by 1 or more people, and that is what you want to prevent. You want the team to be stable, not only within a sprint but also over sprints. Almost at any cost. Again, I'm not talking about a group but about a Team. And, they must all have an engineering mindset. People who dislike doing things themselves and want to automate everything by definition.

Recently I was asked on the subject within the context of an initiative to start treating an operating environment as a platform, which was going to be treated like a product. With a Product Owner at the helm. And a group of my client's experts was assembled to form the 'Scrum team'. I was asked to provide my thoughts on this setup.

We're talking about server provisioning, networking, identity and access management, firewalls, certificates etc. This team is also going to be responsible for operating the platform they're developing. My initial thought is to have therefore functional expertise (provisioning, networking, identity management, loadbalancer, firewalls) and an engineering mindset (automating, monitoring, everything as code), if possible either a senior in one or more functional areas and medior in engineer and vice versa. Depending on the team composition, there may also be a medior / medior combination when there's enough seniority in the overal team. Attitude is more important than knowledge in my opinion. I would therefore always prefer someone who wants to automate, will always test and always asks for insight into production.

I do not see why one would need a full-time Scrum Master, that's probably overkill. Having said that, it seems wise to let the team choose their Scrum Master from the team itself. And with experienced teams that have been working together for a considerable time this is likely a viable option. But when you're just starting with Scrum in your organisation or when the team is just starting with Scrum and Agile concepts, I always prefer a Scrum Master from outside the team. Since the Scrum Master is the team's conscience, and at times will have to use strong words to get focus back on the team's goals and principles... it will be challenging for a Scrum Master from within the team.
Add a coach for roughly the first 6 sprints to coach the Scrum Master, even though the Scrum Master might be experienced, it is still good to have a coach. The team chooses the Scrum Master because it will ensure that it's their choice, a choice supported by the team. You want good chemistry between team and conscience.
The role of the Scrum Master is all about addressing people to the standards and values ​​of the team, but also facilitating them in doing their work. I have seen in various situations that it can work when the Scrum Master is also facilitating in another supportive role, e.g. as Tool integrator in Continuous Deployment environments. I'm not a proponent of this though.

I told the client that I was missing the Product Owner (PO) in his approach. The PO is relevant because the PO determines what the team is going to work on. That would be the person who talks to the customers about what is needed, etc. And to the users about what was delivered. Therefore the PO is accountable for what the team will deliver. The team is responsible for what it creates, the PO is accountable. These are the two most important aspects.

Keep in mind that responsibility can be delegated and shared, accountability cannot be delegated nor shared. So a delegated PO doesn't exist, as it would mean delegated accountability.

So a stable team and a PO (outside the team) defining the team's priorities. No one else but the PO is mandated (i.e. empowered) to determine what the team is working on, because the PO defines the priorities of the products and features to be developed. The team plans the work. So there has to be, by definition, a lot of trust between the PO and the Team, because the PO must be able to rely on the team's commitment to what they are going to do in a sprint. Here comes the Scrum Master in the picture again. Because the Scrum Master has to make sure that the team does not overcommit.

Here's an interesting aspect, the trust aspect. An aspect I will address in a future post, where I will cover more on metrics and KPI's and the trouble of the user story in this regard.

In my opinion, the role of customers should be not only one of the customer, but also one of the user. This will allow a user-centric approach on developing the product, and at the same time be very customer aware.

You should look for a future post on stakeholder management and agile processes to get some more insights on this topic.

I told in another case with another client of mine that it makes perfect sense to start treating their Scrum teams as internal startups. Basically consider the PO of these teams as the CEO of the startup and the management team would be the Venture Capitalists. By doing so, there's ample opportunity to experiment and evolve into a value driven minded organisation.
In this particular case I suggested to see if it would be feasible to go along these same lines.

The PO will need to determine what an MVP would be. Something small delivered in short increments, so to quickly find out if something is usable. Start cultivating a mindset where 'Done' means 'In use at a customer!' (Done = Live).
Agile means being able to deal with change in a timely manner. So here comes the point of view that the PO is a person who has a strong personality and is met with a lot of respect. Somebody that knows what the product should look like, with a product vision. And the PO will have to be mandated/empowered. It should never be the case that the customer can circumvent the PO to get things prioritised. This is for internal and external customers. And managers.

Again, I told my client to get a coach involved here as well. Even if it is an experienced PO, it is important to be coached. In the beginning intensive, but perhaps later (even after 6 sprints) it may be a bit less. It might be an opportunity to invest in an agile-coach, who will take care of all the coaching work (team, Scrum Master and PO).
Just like the PO is mandated to define what is and what is nog going to be in the product, the Team and the PO should be granted autonomy, independence and self-reliance. The PO is positioned external to the Team and in case of scaling up the team to two teams I would like to argue that both teams get the same PO.

In this particular setting, one could even consider starting with three small teams of 3 to 4 people who all work on their own part within the platform, three products if you will. A team for provisioning servers, one that works on network and connectivity (including DNS, firewalls, loadbalancers) and a team that focuses on IAM Automation (Directory services for example) and certificates. Combine these teams with a single Scrum Master and a PO with three Teams. Obviously an Agile coach for the whole bunch.

So, concluding: A maximum of 9 individuals, all full-time, in the Team. One of which will assume  the role of Scrum Master. Alternatively, form 3 teams of 3 to 4 full-time individuals with an overarching Scrum Master. In addition, a PO with knowledge of the subject matter and full mandate and an agile-coach.

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

API Management in Azure using Microsoft's solution - Resources and Templates [2/2]

This is a series of posts regarding the topic of API's and API Management on Microsoft Azure. Not all posts in this series are directly related to Microsoft Azure and their implementation of API Management, that is intentional. The series also explains about API's, about creating API's and about what it construes to in order to manage them, conceptually and in reality. The reason for this series is that over the past 12 months I've come across many articles on the web, have been in many discussions and advised various clients of mine on this topic. Sometimes discussing with fellow architects, other times with vendors, still other discussions where with developers and managers. But the overall situation has always been that none of the people at the other side of the table had a full grasp of what developing API's in an organisation means, what it entails to manage them or what should be worried about when deciding to go API. I hope you enjoy reading these articles, and when you feel like it, comment on the articles. I always take serious comments serious and will reply to my best effort.

This post is the last post in of a series on API Management using Microsoft's solution.

API Management is not the same as an API Manager. In short, API Management is what you need to do with your API's, and an API Manager is something with which you can do this. There are a lot of different solutions that allow you to implement API Management, one of these is Microsoft's solution on Azure. The tricky part is that when you start reading their documentation on API Management you'll read little about how to actually implement API Management in your organization using their solution. It's all very conceptual. This shouldn't be a problem since the concept behind API Management are more important than the actual implementation… until you want to actually implement it.

Read the previous posts on the topic to learn more about the concept, continue reading this post to learn more about how to use Microsoft's solution in Azure, their cloud platform.

Resources and Templates

Finally, resources and templates, the bricks and mortar of the cloud. In the cloud you're typically dealing with resources. An infinite amount of resources, or at least that's how it feels. Everything in the cloud is, or should be a resource. Some are very basic like computing power, storage and networking. Others are more comprehensive like database, firewalls and message queues. And then there are a quite a few that are truly complex and very useful on a high level, for example directory services.

The cloud, being what it is, is like a normal infrastructure on which you host something that generates value for your business. Hence everything you need to run business applications in a traditional hosting environment, you also need in a cloud environment. Obviously there are significant differences between traditional hosting platforms and the cloud, but when you don't take too close a look, you're likely not to see these differences.

So in the cloud you also need to define systems, attach storage, put a firewall in front, put connectivity in place etc. You can do this by hand every time you need an application's infrastructure. Typically through a portal by clicking your way around and assemble the infrastructure. But more sophisticated and a way better practice, is to define the infrastructure in a text file, typically JSON for most cloud platforms, and use the cloud vendor's tooling to create the infrastructure based on this file. As such, the file becomes a template for a specific infrastructure setup you need. By providing a parameter-file you can externalize specifics of the infrastructure. For example the URL's to locate a web-service can be defined in this parameter-file to distinguish between an infrastructure intended for testing and the same infrastructure intended for production runs.

The particular template is called a resource template, it defines which resources are needed and how are they specified in order to run a business application.

One of these resources that you can use is an API manager, just like you can specify databases and virtual machines as resources. And here's your challenge.

The challenge is in that an API Manager consists of three parts;

1. Developer portal, used by developers to find your API's and their documentation.
2. Publisher portal, used by API developers and the likes to manage the API's.
3. Gateway, used by applications developed by those mentioned above in 1 to access API's managed by those as mentioned above in 2.

Each of these have their own context and is used by a different group of 'users'. The real interesting part of the API Manager is the API Gateway as it is the component that exposes the API's you've been developing. This is your product. It is the resource that is part of your software. And the thing is; it can be shared or limited in scope to just the product you're developing.

Ideally you would have one gateway per product, because the gateway and particularly the API's it exposes, are part of your product and as your product evolves, the API's that come with it will evolve as well. And of course you would want a consistent life cycle across all components that relate to your product. Since the API gateway is just like any other resource in Azure, the above is perfectly doable. In fact, it is possible to include the API gateway as part of your product's resource template and when you provision the relevant infrastructure and deploy your product on it, the API gateway is provisioned as well.
Pretty awesome, when you're willing to forget that the costs of an API gateway are pretty steep. We're talking about close to €2,5k / month. There's not really a price based on usage. Microsoft is really weird in that when it comes to pricing in the cloud. That whole pay-per-use is not really everywhere in their pricing schemes. I like Amazon better in that regard.

So an API gateway per product is not really an option in most cases I would argue. Instead, I would advise to have a gateway per product suite. In case you have teams that handle multiple products, scope the gateway to such teams, or otherwise scope the gateways to department. Use it as a rule of thumb though and not as the law.

The point here is that you want to be able to have your API's evolve with your products and that you want teams to be as independent of each other as possible. But in addition you want your API's to be operated independent of each other. And this is important. In Azure, API's don't scale, it's the gateway that scales. And you want to be able to be smart about that. Especially when it comes to policies and usage tracking or rather generating value from API's being used. When a team is responsible for the success of its products and therefore the value that is being generated, it becomes obvious that that team would want to be able in control of what is affecting their success.

The alternative would be to work with an SRE approach, where you have a team that's responsible for your platform, including your cloud platform(s). This team would then realize the API gateway for the other teams as a service. The catch here is that this platform team decides where your API's are 'hosted', or rather whether or not you share the API gateway between teams or not. Unless your platform team is really committed and more importantly has a thorough understanding of what API's really are and I mean really understand this, I would advice against this approach. I would oppose it for the sole reason that your API's are the window into the soul of your organization. When the API is not performing well, your organization is not performing well. And especially when you're going API first, and thus build a platform, you're screwed without proper API management.

In case you do decide to go the platform team route, make sure that your processes are completely automated. Including the deployment of new API's as well as new versions of existing API's. My preposition here is that you'll be working agile as can be, deploy to production as soon as you're confident that your software is up to it. Meaning that new software needs most likely new (versions of) API's. Don't make the platform team a bottleneck, so make sure that you're working with them to deploy the changes API's consistently, repeatable and consistently. Better to abide by their rules then put your own in place. Drop the whole platform team approach when they're not providing a 100% automated deployment process for your API's.

Then there's the portals. The developer portal is a tricky one because it provides access to your API's from a developer perspective. You should be really nervous when you're nervous about potential unwanted developers nosing into your API registry. Because it means your security is way, way, way below par. Remember, API's are different from regular services in that they are build such that they make no assumptions as to who accesses them. And unless you've build them that way, you'll be in for some really serious security challenges. That said, there's no reason why not to have different portals for developers within your organisation and developers from outside your organisation. And have API's exposed only to internal teams and publicly exposed API's. Just make sure that this is an exposure aspect and not, I repeat, not an API implementation aspect.

So develop API's as if they're possible accessed by just anybody and their mother. Expose API's to a subset of this group.

Then there's the operational aspect of your API captured in the publisher portal. Here you should take an approach that only the team responsible for an API should have access to the API from a management perspective in an operational environment. In other words, access to an API's policies is for the team that 'owns' the API only. You'll need to take care of that. period.

Mind that Microsoft is rapidly changing their API Management service on Azure. Most likely. as I type this, they're making life easier for you on using the service in your organization. The concepts as I've described still hold though. And as Microsoft will hopefully come to realize that also for API Management a pay-per-use model is the way to go, you'll be able to treat API Management as part of your product instead of as part of your platform.

This concludes my series on API Management using Microsoft's solution. I hope you did enjoy it.

The complete series:

• API-first design
• API's and Architecture, match made in heaven 
• Why API's should be managed and by whom
• Resources and Templates [1/2] 
• Resources and Templates [2/2]

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link to my blog to all your Whatsapp friends and everybody in your contactlist.But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

Roadmaps are why you should ditch your architects

Summary

Architects that are not with you along the way are of no use. You should not pay too much attention to them, they're not worth the frustration they cause as nothing good will come from it.
But if you find yourself in the company of an architect that is trying to be with you all the way, you should cherish that architect and consider yourself extremely lucky. Not only because your life will be better, but more importantly because this is a very rare breed.

Classification of Architecture Irritation

In case you're interested in finally getting that excuse to get rid of your architects, you're most likely frustrated with them, their architecture and probably you are not able to see any added value of the architect in your organisation. Therefore, I think you'll most likely fit into one of these groups:

• Your architect is too technical and strict, and that the architecture is too restrictive and technical.
• Your architect is just the right person for the job, and a great job he's doing with that just right architecture.
• Your architect is too high-level and vague to be helpful, and that the architecture is too high-level and vague to be useful

And in case you are an architect, you will fit in one of these three categories and your surroundings classify you as either too strict, awesome or just not helpful at all.

Now think about it really carefully... if you're the just right architect or you're dealing with that architect. Since how long do you feel that way? Has it been forever? Well, then this post isn't for you, but continue reading anyway. In case your answer is something like 'just recently' or 'since not so long', then this post is for you because there's a reason why it's been not since forever and it's bound to change in the near future. Well, it'll be very likely you'll going to move to one of the other groups.

The roadmaps are what make architects roadkill

When your architect already is one of those that you can't really consider useful, and is creating architectures that are not really helping you. Or when you're one of those architects. Well, you're in for a treat, because this is a cause of possibly the main frustration within your IT department. Or even within your whole organisation.

Now why is this?

This is because these architects only add time and costs to your projects and nothing else. They too often behave like auditors, telling you what's wrong and not how to fix it. But other than auditors, the architects refer to something they created themselves in the past, using the RTFM reply.
Auditors are a pain in the buns because they point you at how you're not playing by your own rules. Based on what your own definitions of good and bad are and what is and what's not acceptable, they'll verify to what extent you play by those rules. Architects, too often, will work on a reference architecture, architecture principles or even guidelines and that's it. You use their output to work on your own products. And you'll notice that as soon as you start using their stuff, it's either too restrictive (we develop software in either Java or .Net), too outdated (we develop our software based on JavaEE 5) or too vague (we develop our software based on the multi-tier architecture). Whatever it is, it's not helping and as soon as you ask your architect to do a scan of your products, it will be noticed that you're not using Java or .Net but NodeJS instead, that it's based on Java 8 or that it's an architecture based on SOA principles utilising a serverless infrastructure using Amazon Lamda.
So by either abiding the rules written or checking to what extent your compliant, you're loosing time and wasting money.

In many cases this is because people think architecture is a noun, while it is in fact a verb. Let me repeat that;

Architecture is a verb, and definitely not a noun!

So when you document an architecture, you're documenting a living thing. Something that is changing constantly.  So understand that when you come across a document with an architecture, look at the date of the document, because it's a snapshot of what the architecture looked like at that date. Or rather, it's likely to be the case.

But it doesn't end there, architecture isn't just some thing that changes constantly. It's an opinion of somebody who is mandated to be opinionated. What this means is that the architecture is also showing where you want to go as an organisation with your IT landscape. It's the as-is, the to-be and the route to get there. It's like a sat-nav system, it determines based on GPS coordinates where you are on the map, you enter your destination and it will calculate the route to follow to get at your destination based on your location. This is architecture. Well, part of it. The part that's relevant for this post.

That reference to driving and navigation is something I've done before. Read about it here. It's how I explain to my kids what my job is. I see where my client is, I negotiate with him where he would like to be and then come up with a plan how to get there. Including a clear definition of the impact on his situation along the way. Goods and the bads of the trip.
When you look at how many architects work, it's no surprise that they're so frustrating. The analogy with the SatNav is very clear in this;
There're those that will just hand you a (road)map and you just figure it out. See where you are, find where you need to be, plan your trip and go. And oh, btw, the date on the map is likely not very comforting. Or you'll get a turn-by-turn overview of how to get from A to B, based on the outset of when you started to plan your trip. It's based on some possibly old view of the world and any changes to the roads, terrain or otherwise are hopefully taken into consideration. But any traffic jams or other obstructions along the way are not in scope.
And then there's our trusty connected SatNav system. And the analogy holds. Because this thing is connected to the world. It not only keeps track of where you are during the trip itself, but it will keep an eye out to where you will be shortly. That won't be your final destination, but in case there's some roadwork up ahead, it will guide you past it. You'll be informed about relevant information along the way about gas stations and gas prices, hotels, restaurants etc.
The interesting part here is that those that have ever driven with a map, remember the fights between the driver and the person reading the map. Or those near death experiences when you were driving and reading the map yourself. And in case you printed that yahoo page with the turn-by-turn route to wherever you thought you were going to end up, you'll remember that all of a sudden there's a lot of places with the same name, roads are always under construction and the turn-by-turns are always irrelevant after the 5th turn or so.
These are the architects that think architecture is a noun.

The SatNav is constantly with you, and the better the system the more it anticipates on the road ahead, keeping you informed about your surroundings, distance to destination as well as the arrival time. It will keep in mind that weather conditions might be cause to divert your trip or maybe start the AC to make sure that the climate in the car matches the weather outside. This is when architecture is a verb.

Architects that are not with you along the way are of no use. You should not pay too much attention to them, they're not worth the frustration they cause as nothing good will come from it.
But if you find yourself in the company of an architect that is trying to be with you all the way, you should cherish that architect and consider yourself extremely lucky. Not only because your life will be better, but more importantly because this is a very rare breed.

Conclusion

I did already post about project managers and their demise here. Well, it's not a lot better when it comes to architects. Actually it's worse, because these architects, the ditch-able ones, are useless even in waterfall projects, but more so in agile projects, I give you that.
But where you can still find some use for most of the architects in waterfall projects, because as you know, these projects consider documentation as a key deliverable and are run under the premise that after creating the documentation for the next 12 to 24 months they don't need to change. In Agile projects this is different and this is why many scrum teams have said their goodby's to architects and even organisations have stated that architects are no longer need. And yes, this is the best thing that could have happened to our profession as IT architects. It's a matter of weeding out the turn-by-turn style and map style architects.

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link to my blog to all your Whatsapp friends and everybody in your contactlist.
But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Iwan

API Management in Azure using Microsoft's solution - API's and Architecture, match made in heaven

This is a series of posts regarding the topic of API's and API Management on Microsoft Azure. Not all posts in this series are directly related to Microsoft Azure and their implementation of API Management, that is intentional. The series also explains about API's, about creating API's and about what it construes to in order to manage them, conceptually and in reality. The reason for this series is that over the past 12 months I've come across many articles on the web, have been in many discussions and advised various clients of mine on this topic. Sometimes discussing with fellow architects, other times with vendors, still other discussions where with developers and managers. But the overall situation has always been that none of the people at the other side of the table had a full grasp of what developing API's in an organisation means, what it entails to manage them or what should be worried about when deciding to go API. I hope you enjoy reading these articles, and when you feel like it, comment on the articles. I always take serious comments serious and will reply to my best effort.

Summary

The architect is arguably the one most concerned with API's as it is determined by the architecture where the API's reside, what should be API'afied. This has nothing to do with application architecture or integration architecture. The architectural structure of software has no relevance as to when and what to expose through an API. It is the business or domain architect that decides what will be exposed through an API, when and only when the Product Owner wants to reap the benefits of an API to his product. API development may not be controlled by rules, instead it should be governed by principles and guidelines. Although why something should be exposed as an API is a business (architect) call, the how it is exposed is a technical (architect) call.

API's need empowered teams to develop them and more importantly to maintain them. This is not an small thing. API's being extremely hard to develop, they are also extremely strainuous to an organisation because they typically upset the whole structure of the organisation. Even when that organisation is already a product organisation with product teams and product owners.

API's and Architecture

API's are among the hardest things to develop. Why this is, you can read in my previous post here. I think that the role of the architect is extremely important when it comes to API's because they provide access to functionality and information without compromising the underlying systems. And it's not access to just some information, but to information that has meaning on a business level. It's about disclosing your business to the unknown consumer. Well, it's the consumer of your API. So this means that your API's need to fit into the grand scheme of all things IT in your organisation. This is where the architect comes in, because the architect is the city-planner of your IT landscape. See my post on the role of the architect in your organisation, and how that architect may fail miserably here. 

Basically it would be the architect that has a strong say in what the boundaries of your IT landscape are and therefore where the API's are to be found. Mind that this is the IT component of your business architecture.

Say what? Well, it's not that difficult to understand. Consider your business architecture, in a world where IT is considered in terms of products that the organisation is making money off. (As opposite to seeing IT as an enabler or unique selling point.) So basically the organisation defines its business in terms of products. These products and the relationships between them are a way to see the business architecture. It's a very complex exercise to do this and on top of that, many organisations don't even want to make money perse. GO's and NGO's are examples. Still, consider that you do stuff that is of benefit to others, than what you do and what you'll do it with are your products.

These products are at times realised using IT. Maybe the full product is IT or parts of it are. These IT products and their relationships are the IT components of the business architecture I was referring to.

Every connection between any two products is a prime candidate of becoming an API, or becoming API'afied, if that were a word.

Did you notice that I am referring to the business architecture of the organisation and not to an application architecture that is adopted by an organisation. For example, I'm not referring to the 4-tier architecture by Forrester, or the traditional 3-tier (presentation, business, data) architecture, or an SOA or anything like that. Like I discussed in the API-first design post, these architectures have no relationship to your decision what should and what should not be API's. Your business architecture is. Sometimes, especially in larger organisations, that business architecture is decomposed into multiple domain specific architectures, again on a business level. Here you might argue that the relations between domains are even more prime candidates to become API's. And again, how your application architecture is defined has no effect on that.

The Platform

So this is where the architect has value in the API driven universe. Consider this; API's thrive in a platform economy, who decides whether or not there will be a platform and how the platform is shaped? It's the architect and no one else. And this is because the architect is the only one that views things beyond product boundaries, that's the architect's scope. The Product Owner and his Product Team are only concerned with the product. A true Product Owner will also understand the value of API's, because it means more users of his product and more users means more value, but it's the architect that decides a platform is to be developed, raising the value of API's several levels upwards. You should take a look at 'The Age of the Platform' by Phil Simon to understand the value of a platform and why it is such a key to success for many fast growing organisations. Mind that platforms are not only valuable in relation to external customers but also for internal customers.

So basically, the architect is the key role when it comes to API's. The architect defines the business architectures, models the IT products in it, considers where interfacing between domains and products is done through API's. The architect also decides whether or not to work on a platform and that kind of implies going for API's as well.

Organisational Impact

It should also be the architect to determine whether or not the organisation is fit for developing API's. Like I stated in API-first design, developing API's is one of the hardest things to do in software development. Which not only implies, but makes it rather explicit that your organisation needs to be ready in various areas to be able to develop and maintain API's. Continuous Delivery, and preferably Continuous Deployment needs to be in place. You would like to strive for a DevOps IT organisation and you need a very capable Product Owner. On top of that the developers working on API's need to be able to understand the relevant business concepts and need to have extensive experience in the pitfalls of software development. The architect's responsibility is to inform management, including HR management, the impact of adopting a platform strategy, or an API strategy on the organisation. Since architects need to be able to quantify the impact of a change to the IT landscape across all aspects of the enterprise, the architect will need to be able to quantify the impact on staff, staffing and processes alike.

Think about the opportunity to start small, have a single team mature enough to develop and maintain API's. That team should be mandated and thus empowered to develop API's and put them in production, with full autonomy. If the team needs to deploy in production, it should be their call. They should be held accountable and they're responsible for the success of the API. Once that works take all lessons learned and scale your organisation to more teams to work on API's for their products. The architect will know what domain and what product will be best to provide API's to interface with in order to create the most business value.

Governance vs. Control

And then there's the other thing, which is governance. There's no room for control in an API economy. A Product Team that develops API's to access the product's information and functionality will need to have full autonomy. And that means that it can't be controlled. Control means that there's a committee that decides whether or not the team has been abiding by the restrictive rules defined by some other committee. This assumes that it is possible to define rules that API's need to follow without knowing the API and therefore the context within the API is being used. Utterly impossible because when you start developing an API you have no clue about your customer and by monitoring very closely you get an understanding of who your users are and why they want to use your API.

So should there be no control at all? Of course there needs to be control, but it's of the scalable kind. Scalable control is called governance. Governance is done by first define a series of principles. A principle is a rule or belief that governs ones personal behaviour. It's intention is that because one believes that the principle makes sense and one is better off by applying the principle, life is better.

A principle is a rule or belief that governs ones personal behaviour.

By their very nature, principles are used for governance and not for control. It's is one of the most formidable ways to get self organising and autonomous teams work in concert and by the same playbook. The interesting part here is also that you can choose not to stick with a principle in a certain situation. A clear motivation is available implicitly because one abides by the principles, unless there's a good reason. It also allows in your governance structure room for classifying principles in those that can be dropped when the team chooses so and which need an explicit approval from some authoritative body, like an Architecture Board.

Besides the principles, which are set by the architect, the architect needs to define a set of guidelines that inform the Product Team the best course of action to develop an API. These can be related to processes, like how to publish an API. But also technical, for example naming conventions, versioning strategies, etc. Good guidelines are no-brainers but converge various alternative solutions to a problem into a single solution that is the default selectable solution to the problem. It's different from a principle because it's not appealing to a developer's common belief system but to the developer's common sense.

Governance is an architect's dominion. It's not related to a specific team, but it's there to ensure the integrity of the complete IT landscape. This is where the architect shines. The Product Team is responsible for its own principles and guidelines and they should never conflict with the overall, architect driven principles and guidelines. One could argue that the team only makes architecture decisions, based on the principles and guidelines.

Mind that principles are hard to define, almost as hard as API's. Always think of an API as something that sounds like a rule, that has a clear rationale and has an impact which on the one hand details the impact on the existing landscape, the external impact. But also has an internal impact, which details limitations, prerequisites etc on how the software subjected to the principle is developed.

Quite often I see principles that hint towards solutions. For example: High available systems are deployed on a cluster. The rationale is clearly a technical one and the impact restricts the technical aspects of the solution. But an architecture principle's rationale should be a business rationale instead. The impact should allow for as much freedom to the team developing the software without compromising the principle. For API's one needs to be extra careful when defining principles. Stating that API's are always exposed as SOAP webservices seems to be a good principle. SOAP is a great way to do b-2-b integration and it allows for a standardised approach to security, validation, etc. But what if the better way to expose the API is as a RESTful service using JSON? Well in that case the Product Team can motivate why the principle is dropped. But a principle like API's are always exposed based on internet standards and web-based protocols achieves the same, it also allows the product team to select the best solution for their situation and still hold on to the principle. Furthermore, the architect could define a guideline, or guiding principle, stating that for b-2-b connections SOAP is preferred and for b-2-c REST/JSON is preferred.

Empowerment

I've stated before, the Product Team should have a high degree of autonomy when it comes to API development. You want governance in place to make sure that anarchy won't reign. This means that the Product Team should be empowered to decide for itself when principles are selling them short, when a new not yet anticipated situation surfaces or the API's 'market' changes over night due to new insights. In addition, you want the team to be empowered to decide how they develop API's in the most efficient way. Meaning that thinking of an API and having consumers calling the API should be an efficient as possible process. This is an important statement, because it means that the team can't just say 'screw the process or the organisation for that matter' because they have autonomy. Because doing so, would not benefit the complete process from idea to used product.

The complete series:

• API-first design
• API's and Architecture, match made in heaven 
• Why API's should be managed and by whom
• Resources and Templates [1/2] 
• Resources and Templates [2/2]

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link to my blog to all your Whatsapp friends and everybody in your contactlist.
But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Iwan

The Bimodal misconception, why Gartner has to realize the ignorance of its customers

Okay, first of all, I'm talking about Gartner's Bimodal IT thing here which is defined by them, that is Gartner as follows: "Bimodal is the practice of managing two separate but coherent styles of work: one focused on predictability; the other on exploration.", which is taken from here. And it makes no sense, because Gartner omitted to mention that this is an application architecture thing.

The problem I have with Gartner's definition of Bimodal IT is that it implies rather explicitly that predictability and exploration don't go together, which is of course totally correct. You explore because you can't predict. But the reason why Bimodal IT was introduced is because we face a stand-off between waterfall and agile, between Silo'd IT and DevOps, between client/server and service oriented systems, between legacy and the new world.

Reading the rest of Gartner's definition (Mode 1 is optimized for areas that are more predictable and well-understood. It focuses on exploiting what is known, while renovating the legacy environment into a state that is fit for a digital world. Mode 2 is exploratory, experimenting to solve new problems and  optimized for areas of uncertainty. These initiatives often begin with a hypothesis that is tested and adapted during a process involving short iterations, potentially adopting a minimum viable product (MVP)  approach. Both modes are essential to create substantial value and drive significant organizational change, and neither is static. Marrying a more predictable evolution of products and technologies (Mode 1) with the new and innovative (Mode 2) is the essence of an enterprise bimodal capability.) you understand the flaw in their reasoning. The flaw is in that the definition assumes that an organization can be divided in two ways of addressing its business, or rather that it can adopt a way of working in that it addresses its business simultaneously in a 'we know what the customer wants' and a 'we want to know what a customer wants' way of working. Which is bogus, of course. Totally bogus.

At one of my current clients, Bimodal IT is a hot topic. Here it is used such that the back-end systems are considered Mode 1 systems and the front-end systems are considered Mode 2 systems. Furthermore, the epic mistake is made to consider everything running in the cloud to be a front-end system and everything in the on-premise environment, which is a traditional data-centre, to be a back-end system. Makes no sense to me, and I can guarantee that within the next 6 months there will be back-end systems running in the cloud.

To understand Bimodal IT, we need to understand why the term was coined. Basically, you need to re-read the definition and understand what is being said. You need to put it in the context of time, and address it from an architectural perspective. Because that is what Bimodal IT is, well that is its raison d'etre.

So, let's start with 'one focused on predictability' and forget about this. It's an ambiguous term. Ambiguous because for one we want our software delivery cycle to be predictable, we want the quality of our solutions to be predictable and we want the costs of our solutions to be predictable. At all times, no difference between front-end and back-end, no difference between cloud and data centre and no difference between old and new. At the same time, we can't predict what the best feature set of our solution is. We can't ever predict it, no difference between fron
t-end and back-end, no difference between cloud and data centre and no difference between old and new.
So predictability only applies to our solution delivery process and not to our solution.
And then there's 'the other on exploration' which we should forget as well. It's an ambiguous term as well. It's ambiguous because we need to always explore to find the right feature set for our solution. This is because we can't predict the right set. It has nothing to do with where we're hosting, whether it's front-end or back-end or anything else.
So Bimodal IT by Gartner's definition makes no sense, because from the outset it's wrong.

But here's the catch, at the time Gartner coined the term, it made perfect sense. It's just that Gartner forgot to mention that Bimodal IT actually hinges on application architecture.

Back to this customer. Bimodal IT is used as an excuse not to comply with the ancient and really seriously out dated processes in IT that are (to be) followed within this organization and I assume unwillingness to try to modernise the processes. By just simply referring to Gartner and their definition of Bimodal IT and stating that this is the exact situation at their organisation, some of my clients architects and project managers are implying that the processes only apply to what's running in the back-end and what they're working on, which is the front-end.(*)

Let's see why thinking in terms of Bimodal IT is making sense. It makes sense because at the time Gartner coined the term, we developed our systems predominantly being Waterfally and the hipsters were being Agile. There was nothing exploratory about it, and read the Agile manifest and it says nothing about exploration, but the more about predictability.
Mode 1 and 2 are all about delivery cycles. Mode 1 is long, Mode 2 is short. One could say Mode 1 is about being ignorant and Mode 2 about being smart.

So, ignorant vs smart, what's that about?

Well, in Mode 1, we tend to think that it makes sense to first define the complete solution, design it and build it before we deliver it to our users. And because for some reason, release into production always results into (often major) headaches, we also tend to think it makes sense to not release that often, so we limit the occurrences of headaches. Hey, if you only bang your head against a wall twice a year, you only run the risk to crack your skull twice a year. So makes perfect sense, doesn't it?
Then there's Mode 2, where we think that is makes sense to just grab the most important, or valuable features and develop these and take them to our users, and then do it again with the next most valuable solutions and put them out to our users. And we tend to think that because we only release a tiny thingy, it's very unlikely that it'll go wrong and will only cause (if at all) a dull feeling in the back of our head. Hey, if you nudge your head against a wall every few days, at the most you know that the wall is still there, but that's about all to be felt.

So Bimodal IT is where one group of people thinks that doing something really hard is less risky when you make it even more complex but do it fewer times and the other group of people thinks quite the opposite, that doing something really hard is less risky when you simplify it and do it more often. Kind of practice makes perfect.

And yes, years ago when Gartner came up with Bimodal IT, we really had this Mode 1 and Mode 2 division as described above. But in this day and age, the Mode 1 people are a rare breed. Mind that we're discussing this in an era where major banks (ING) are firing pretty much everybody that is Mode 1, where major airlines (KLM) are spending millions of €'s to transform in to a full Mode 2 organisation and where it's hip to be Mode 2 at IT departments of local and national governments.

But Bimodal IT is there to stay for a while. Not because of predictability, nor because of exploration. Not even because of Waterfallians (Mode 1 people). Nope, it's because of Monoliths. Those ancient systems that were delivered in a day and age when we thought Client/Server was awesome and everything should be consolidated into one system so we could guarantee data consistency and we could really vouch for abiding to the DRY principle (Don't Repeat Yourself).

These systems can't be changed, they are architected to be unchangeable. We don't want them to be that way, but they are. It's because our application architectures of a couple of years ago resulted in unchangeable systems, we can only change them by changing the complete system. The monoliths are preventing us from becoming post-Bimodal IT organisations.

Back to the misconception. Bimodal IT is something that has to do with the monoliths in our IT landscape and nothing with how we perceive our IT or the systems in our IT landscape. The misconception is that we need to adopt a Bimodal IT approach to how we do IT because we don't have to. We have to get rid of the monoliths and we need to start by isolating them behind a layer of changeable software. Hide them like a skeleton in a closet of services.

By adopting Bimodal IT today, or continue with Bimodal IT today, means you either have a totally wrong understanding about how to manage your IT department, or you are still in the previous millennium when it comes to IT architecture, or you're just using Gartner's reputation as an excuse for not having to modernise your IT processes.

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link to my blog to all your Whatsapp friends and everybody in your contactlist.
But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Iwan


Although this post is my personal opinion, there're quite a few articles to be found on the internet about this topic that I've read. So here're some interesting reads:

- Gartner Glossary: Bimodal IT
- Gartner's bimodal IT mistake: DevOps can deliver velocity and quality
- Bimodal IT: Gartner's Recipe For Disaster
- Beware the Dangers of Bimodal IT
- Gartner’s Evolution of Bimodal IT
- The Flaw at the Heart of Bimodal IT




*: Interestingly enough, they forget to mention that the back-end is running on-premise and they're in the cloud and yes, the front-ends hosted on-premise are just as much under the same regime as the back-ends.