Where's the Agile Architect hiding?

Nowadays everybody and their mother is turning, transforming into agile beings. But where is the Agile Architect? In October of 2017 I did a presentation at ASAS2017 about the Agile Architect. Ever since I have been contacted about the presentation and specifically about what the architect's role is in agile organisations. Because of this, I think it is time to shed some light on this. Thus... Find out in this post.

Summarising

The Agile Architect is this Product Architect that is taking care of the organisation's software product landscape and maintaining its integrity. The Product Architect ensures that based on the input of the Business Architect and the Product Owner the product landscape is sound, adaptable and delivers overall architectural agility.

The difference between doing agile and being agile

The Agile Architect is a rare beast in today's agile organisations. But before I go down this rabbit hole I need to emphasise the fact that I am talking here about organisations that are agile and not those that are doing agile.
The main difference is that when you do agile, you're living by the book, like a religious zealot. You follow the rules, have your rituals and so on. It typically means that you're organised according to the school of SCRUM. Small teams of max 9 individuals, one Scrum Master, a Product Owner. working in Sprints that take typically two weeks, have your daily stand-ups, your retro's and demo's and a backlog of user stories and epics. In contrast, when you are agile, you're taking the lead from the agile manifesto, do the hard things as often as possible, prioritise your activities according to value, have a build-measure-learn loop in place and determine your next actions based on feedback from your users. This is the short version... Obviously.

Independence and autonomy are prerequisite for Agile Architecture

So having established this, I can continue on the topic of the agile architect. Well, just before I venture into that direction, let me explain that in an agile organisation, you also find a strong longing for independence and autonomy. Mind that it is not anarchy that is sought after. The challenge here is that different teams want to move with different speeds. Often this is called bi-modal or multi-modal (you can read about this for example in my post "Let me spell out why Bimodal IT doesn't work"). It doesn't matter how you want to call it, what it boils down to is that different teams work on different products and the organisation's market dictates that each product needs to adopt to changed circumstances in a timely manner, where 'timely' has different meanings for different products. It's extremely logical and also extremely unsexy. These different requirements regarding development speeds requires independence between products and therefore teams. Obviously, whenever there is a dependence, the speeds are linked and in fact agility is lost. Something I'll cover in another post some time soon.
So independence is required here. Autonomy is also important, because if a team cannot decide by itself how to solve problems, how to implement features, how to enable proper operational management, and instead have this dictated by a third party. There is a dependency with a bottleneck and hence agility is at risk. Look for the upcoming post "The Incompetence of the CC..." that will feel enlightening in case you don't understand this right now. But again, it is in fact quite logical how this all works.

So teams should be able to act independently of each other and should have a high level of autonomy, in fact the organisation should strive for the highest level of independence and autonomy that is possible within the context of that organisation. Note the subtlety of the last part of that sentence; that is possible within the context of that organisation. I am explicitly not saying that all control should be dropped and no governance should be implemented.

With the teams being autonomous and independent, the question about the role of the architect arises. Typically the first question I get is whether the architect should be part of the Product Team or not. And my answer is at all times that the architect is not part of the Product Team, where the Product Team is the team that develops and supports a (business) product. Just like the Product Owner is not part of the team, the Product Architect (that's what I call the architect) is not part of the team. This doesn't mean that the architecture of the software is to be neglected, what I am saying is that the team itself is responsible for the application architecture, referring to the internal structure of the software. There are typically a lot of important impromptu decisions to be made during the development of the software where the right decision is determined by the specific context in which the decision is key. The team knows best. That is not to say that anything goes. I'll get to what I mean by this in a second.

The Product Architect is not so much responsible for the software architecture, but for the positioning of the product in the organisation's landscape.The Product Architect is accountable for the integrity of this landscape. The meaning of this is best described through a use-case.
[Read this post on the importance of accountability and responsibility.]

Agile Architecting explained

Consider an organisation's product landscape to consist of several business domains, each consisting of several business products. A business product is a product that is delivered to a user, often a customer of some sorts. The customer typically accesses the business product through an interface. This could be a specialised App, a webpage or maybe a customer-success-agent. This is sometimes referred to as the business front-end, or the customer interface. What is important here is that the customer is not interacting with a piece of software perse, but instead could be interfacing with another human being. Demonstrating that a business product can consist of man and machine. In addition to this, business products can be related to each other. The final step of using one can lead to using another product. There can be a sequence, just like when you go to the movies. You buy a ticket, you get some drinks and snacks at the concession stand and then see the movie. These are three different business products and clearly the buying a ticket and watching a movie are closely related. But independent since the person buying the ticket doesn't have to be the person watching the movie. Think about gift-tickets.
So you see there are several (related) business products. How they relate to each other is part of the responsibilities of the Business Architect(s), sometimes called Business Domain Architect or just Domain Architect. It is the Business Architects that will typically decide to which domain a business product belongs. Buying things could be the Sales domain and watching a movie in the Media domain, or buying the ticket and watching the movie can belong the the Movie domain and the drinks and snacks in the Concession Sales domain. It's arbitrary but needs to be decided upon, this is what the Business Architect does.
Once it is decided to which domain a (new) business product belongs, it is the (new) Product Owner that determines how the business product is taking shape. Think about the MVP. Often the MVP is a first incarnation of a product, but it can be in fact be a completely different product that is used to verify that there is in fact a need, a requirement, for the new product. An MVP could be a simple teaser webpage announcing the new product or poster with a box attached that potential customers can put a card in saying how excited they are to get hold of the new product.
When the Product Owner ties the knot that software is to be developed, something typically decided after consulting the Business Architect and the Product Architect(s) for the products in a business domain, it is the Product Architect that comes into play. This is where the Product Architect is taking the task to maintain the landscape's integrity. First of all, the Product Architect will have to determine whether or not a new product is to be developed or an existing product is to be enhanced or adapted. In case of a new product development, it has to be determined what the best technology is to do this. Here it is important for the Product Architect to understand the product. The Product Owner is a significant source of knowledge and information here. Who'll be the user of the product and how will the user access the product? What are the security requirements and what about compliance aspects? Transactional integrity is important as well. Also the intended life span of the product is important. And obviously the product's relations and integrations with other products is key and for this the Business Architect needs to be consulted. Obviously all architectural principles are taken into account and the Product Architect will define what principles are such that specific practices are to be maintained. Consider a product that does financial transactions, that could mean that specific frameworks and products need to be used for the implementation. Or the fact that the product is going to be used by students after marketing campaigns, resulting in a deployment in Amazon AWS instead of the organisation's own data centres.
The Product Architect, in response to the Product Owner's requirements for the product, will also be responsible for example to define an API that is published as part of the product offering.

Product Architect vs Product Team

By deciding all this, the Product Architect is to some extent dictating the products architecture, and still it is the Product Team that is responsible for the software architecture and not the Product Architect. So how does this work? Remember I promised earlier in this post that I would elaborate on this.
It is actually very simple and all based on the role of Architecture Principles. The organisation will have its organisation wide Architecture Principles, principles that are to be followed at all times. Possibly depending on context, though. I always like to mention that KISS (Keep It Simple, Stupid) is the prime-directive of everybody. By stating so, everybody introducing complexity is violating this principle. As a consequence, the principle enforces the policy to deliver always the minimum that is required, hence helps implementing a LEAN mindset.
Apart from principles that are organisation wide and principles that are applied in a specific context, emphasising that the context needs to be specific, a Product Architect can also define so called 'local architecture principles' that are local to the teams within the circle of influence of the Product Architect. Think about these as local extensions on the existing principles. Maybe because the domain is subject to some specific regulatory context or maybe because the domain is specific for a specific market.
The Product Teams are always to abide by these principles. So although they have autonomy on how to do things, they need to stay within the framework of these principles. They can't just decide to make an overly complex solution, because that would violate KISS.
In addition, the Product Architect can also limit the acceptable answers to implementation questions. For example the Product Architect can state as a principle that all API's are based on Open Web Standards, but in addition decide that all API's that are exposed on the internet are to be based on REST/JSON and API's that are with business partners to be based on WS-*.

Consequential impact

All the above decisions also lead to other decisions. A decision for a specific deployment platform may mean that only certain teams are capable of doing the development. Or deciding that the product needs an API as well as a web-UI may result in the situation that two teams will work on the product, one on the product with the API and the other working on the web-UI based on the API.
Also, envision the situation where timelines are not really up to the needs of the Product Owner, it is then for the Product Owner and the Product Architect to determine the architectural shortcuts to be taken. Using different technologies to leverage the availability of another team for example.

Concluding

As you can see, the Product Architect's role is very challenging, just like the role of Product Owner. But although it is very challenging, it is also very much scoped towards the external aspects of the products. The scope is about the product in its surroundings. Think about the layout of a street with houses instead of the layout of the individual houses. Of course, the type of house, or rather building is a Product Architect's call. Villa's, brown stones or apartments is up to the Product Architect, not the amount of bathrooms, the type of tiles or the kind of sofa's. That's up to the Product Team in response to what the Product Owner can turn into value.

The Agile Architect is this Product Architect that is taking care of the organisation's software product landscape and maintaining its integrity. The Product Architect ensures that based on the input of the Business Architect and the Product Owner the product landscape is sound, adaptable and delivers overall architectural agility.

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

Let me spell out why Bimodal IT doesn't work

Summary

In short, in a Bimodal IT environment, both modes need to move at the same pace in order to optimise productivity of the organisation. Value is created by products in the hand of the user, not by parts on a shelf or piles of almost ready products. This means that unless all relevant information is exposed by the Mode 1 systems, the Mode 2 systems will adjust their pace.

There's no excuse to use Bimodal IT as an excuse

Bimodal IT is an excuse for those that just don't want to accept the fact that we live in a world where there's no one size that fits all and that's also a world where we need to live together instead of separately. So thinking that Bimodal IT might work is totally ignoring the fact that the old and the new together result in synergy, that the short term and the long term go hand in hand and that classic IT models like the ancient Change/Run division can co-exist with brand new DevOps way of working. We have to realise that in IT only in the binary system it's black and white, everywhere else in IT there're more than 50 shades of grey.

If you're a wondering about my view on Gartner's Bimodal IT model, you should read my post on it. It's available here.

Why this post? Because in the last week, although I've stayed home for a day and a half caught by the flu, I had several discussions with colleagues about Bimodal IT and unfortunately, they considered in an opportunity. Not an opportunity to bridge the gap between systems that have a short life cycle and those that have a long life cycle. Or rather, short and long delivery cycles. And to be completely precise it's about the time it takes to think of a new feature and deliver it to a user. One mode is considering long cycles, the other is considering short cycles. Or rather, one is assuming many features to be released over time the other just a few. Think about a core accounting system and its front end. Accounting doesn't change that much over time, possibly some compliance reporting, front ends change all the time. Browser changes, mobile devices, etc.

Back to my discussions. The most symptomatic was about "Bimodal IT is used as an excuse not to comply with the ancient and really seriously out dated processes in IT that are (to be) followed within this organization and I assume unwillingness to try to modernise the processes. By just simply referring to Gartner and their definition of Bimodal IT and stating that this is the exact situation at their organisation, some of my clients architects and project managers are implying that the processes only apply to what's running in the back-end and what they're working on, which is the front-end." [from my previous post]

Understanding the frustration

It's the frustration about ancient processes and the requirement to comply with them that is causing this excuse to rear its ugly head. Fine, I can see this and since the organisation doesn't want to look into this and adapt its processes to become a little bit more 21st century, I can only agree to look for every possible reason not to stick with the old ways.

But here's the catch. Well, let's wait for a second here. I had this other discussion on the same topic. Now the discussion was about that Mode 1 systems need to be stable because they're the core systems, forming the backbone of the organisation. Keeping in mind the adagium "Never change a working system" they should be kept stable at all times.
Unfortunately, these systems exist in an ever changing world, so maybe they don't change, their context changes. And because they are so important, after all they're the backbone of the organisation, when troubles arise, they need to be fixed as soon as possible. High speed and high quality. In addition, you want to address issues as soon as they arise instead of queueing them up and apply in large batches.
It's the general misconception of the dinosaur; When you need quality, you spend a lot of time testing. I'll blog on that some other time, for now suffice to state that time and quality are hardly related.

The catch

Than there was the catch. The catch being that Mode 2 systems are those systems that are changed very frequently. Possibly because a first iteration is brought to the user as soon as possible and new functionality added regularly. Possibly because a new idea is tested and dropped when failing or productised when successful. What most of these systems have in common is that they rely on Mode 1 systems, since that's where the organisation's biggest asset resides: Information. You don't want to change an organisation's information model too much, especially not when the information has been build up in years. Information from these systems are exposed, to be used by other systems, typically Mode 2 systems.
Mode 2 systems are new, by definition. They're implementations of never-thought-of-ideas, and therefore it is very likely that the Mode 1 systems do not (yet) expose the relevant information. Hence with the advent of the Mode 2 systems, the Mode 1 systems all of a sudden need to be changed, regularly, often, quickly, consistently. All the time maintaining a high level of quality.
There is no chance that Mode 2 people will allow the Mode 1 people to move at their own speed because that means they'll be moving just as slow. 'It's the slowest boyscout in the line' case.

In short, in a Bimodal IT environment, both modes need to move at the same pace in order to optimise productivity of the organisation. Value is created by products in the hand of the user, not by parts on a shelf or piles of almost ready products. This means that unless all relevant information is exposed by the Mode 1 systems, the Mode 2 systems will adjust their pace.

Decoupling not to the rescue this time

Mind that decoupling techniques like API's, ESB's, etc are not solving this problem. Interfaces are owned by the systems that implement them. No matter what method is used to define these interfaces. So thinking that there's an ESB, or an API manager or other decoupling technology will prevent the Mode 2 people to move as slow as the Mode 1 people is foolish.
Also, introducing a Canonical Data Model or some other data defined insulation layer will not help you either. In fact, that might, or rather will, introduce more complexity so will slow things down even more.
And let's be honest; Why would we need to insulate ourselves from working together. Understanding each other's contexts and limitations? Agile and DevOps is all about breaking down silos. Understanding that collaboration gets us further. And in the end we need to deliver products in the hands of the user. Having said that, there's no point in not changing archaic processes and apply some LEAN on them.

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link to my blog to all your Whatsapp friends and everybody in your contactlist.
But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

The Bimodal misconception, why Gartner has to realize the ignorance of its customers

Okay, first of all, I'm talking about Gartner's Bimodal IT thing here which is defined by them, that is Gartner as follows: "Bimodal is the practice of managing two separate but coherent styles of work: one focused on predictability; the other on exploration.", which is taken from here. And it makes no sense, because Gartner omitted to mention that this is an application architecture thing.

The problem I have with Gartner's definition of Bimodal IT is that it implies rather explicitly that predictability and exploration don't go together, which is of course totally correct. You explore because you can't predict. But the reason why Bimodal IT was introduced is because we face a stand-off between waterfall and agile, between Silo'd IT and DevOps, between client/server and service oriented systems, between legacy and the new world.

Reading the rest of Gartner's definition (Mode 1 is optimized for areas that are more predictable and well-understood. It focuses on exploiting what is known, while renovating the legacy environment into a state that is fit for a digital world. Mode 2 is exploratory, experimenting to solve new problems and  optimized for areas of uncertainty. These initiatives often begin with a hypothesis that is tested and adapted during a process involving short iterations, potentially adopting a minimum viable product (MVP)  approach. Both modes are essential to create substantial value and drive significant organizational change, and neither is static. Marrying a more predictable evolution of products and technologies (Mode 1) with the new and innovative (Mode 2) is the essence of an enterprise bimodal capability.) you understand the flaw in their reasoning. The flaw is in that the definition assumes that an organization can be divided in two ways of addressing its business, or rather that it can adopt a way of working in that it addresses its business simultaneously in a 'we know what the customer wants' and a 'we want to know what a customer wants' way of working. Which is bogus, of course. Totally bogus.

At one of my current clients, Bimodal IT is a hot topic. Here it is used such that the back-end systems are considered Mode 1 systems and the front-end systems are considered Mode 2 systems. Furthermore, the epic mistake is made to consider everything running in the cloud to be a front-end system and everything in the on-premise environment, which is a traditional data-centre, to be a back-end system. Makes no sense to me, and I can guarantee that within the next 6 months there will be back-end systems running in the cloud.

To understand Bimodal IT, we need to understand why the term was coined. Basically, you need to re-read the definition and understand what is being said. You need to put it in the context of time, and address it from an architectural perspective. Because that is what Bimodal IT is, well that is its raison d'etre.

So, let's start with 'one focused on predictability' and forget about this. It's an ambiguous term. Ambiguous because for one we want our software delivery cycle to be predictable, we want the quality of our solutions to be predictable and we want the costs of our solutions to be predictable. At all times, no difference between front-end and back-end, no difference between cloud and data centre and no difference between old and new. At the same time, we can't predict what the best feature set of our solution is. We can't ever predict it, no difference between fron
t-end and back-end, no difference between cloud and data centre and no difference between old and new.
So predictability only applies to our solution delivery process and not to our solution.
And then there's 'the other on exploration' which we should forget as well. It's an ambiguous term as well. It's ambiguous because we need to always explore to find the right feature set for our solution. This is because we can't predict the right set. It has nothing to do with where we're hosting, whether it's front-end or back-end or anything else.
So Bimodal IT by Gartner's definition makes no sense, because from the outset it's wrong.

But here's the catch, at the time Gartner coined the term, it made perfect sense. It's just that Gartner forgot to mention that Bimodal IT actually hinges on application architecture.

Back to this customer. Bimodal IT is used as an excuse not to comply with the ancient and really seriously out dated processes in IT that are (to be) followed within this organization and I assume unwillingness to try to modernise the processes. By just simply referring to Gartner and their definition of Bimodal IT and stating that this is the exact situation at their organisation, some of my clients architects and project managers are implying that the processes only apply to what's running in the back-end and what they're working on, which is the front-end.(*)

Let's see why thinking in terms of Bimodal IT is making sense. It makes sense because at the time Gartner coined the term, we developed our systems predominantly being Waterfally and the hipsters were being Agile. There was nothing exploratory about it, and read the Agile manifest and it says nothing about exploration, but the more about predictability.
Mode 1 and 2 are all about delivery cycles. Mode 1 is long, Mode 2 is short. One could say Mode 1 is about being ignorant and Mode 2 about being smart.

So, ignorant vs smart, what's that about?

Well, in Mode 1, we tend to think that it makes sense to first define the complete solution, design it and build it before we deliver it to our users. And because for some reason, release into production always results into (often major) headaches, we also tend to think it makes sense to not release that often, so we limit the occurrences of headaches. Hey, if you only bang your head against a wall twice a year, you only run the risk to crack your skull twice a year. So makes perfect sense, doesn't it?
Then there's Mode 2, where we think that is makes sense to just grab the most important, or valuable features and develop these and take them to our users, and then do it again with the next most valuable solutions and put them out to our users. And we tend to think that because we only release a tiny thingy, it's very unlikely that it'll go wrong and will only cause (if at all) a dull feeling in the back of our head. Hey, if you nudge your head against a wall every few days, at the most you know that the wall is still there, but that's about all to be felt.

So Bimodal IT is where one group of people thinks that doing something really hard is less risky when you make it even more complex but do it fewer times and the other group of people thinks quite the opposite, that doing something really hard is less risky when you simplify it and do it more often. Kind of practice makes perfect.

And yes, years ago when Gartner came up with Bimodal IT, we really had this Mode 1 and Mode 2 division as described above. But in this day and age, the Mode 1 people are a rare breed. Mind that we're discussing this in an era where major banks (ING) are firing pretty much everybody that is Mode 1, where major airlines (KLM) are spending millions of €'s to transform in to a full Mode 2 organisation and where it's hip to be Mode 2 at IT departments of local and national governments.

But Bimodal IT is there to stay for a while. Not because of predictability, nor because of exploration. Not even because of Waterfallians (Mode 1 people). Nope, it's because of Monoliths. Those ancient systems that were delivered in a day and age when we thought Client/Server was awesome and everything should be consolidated into one system so we could guarantee data consistency and we could really vouch for abiding to the DRY principle (Don't Repeat Yourself).

These systems can't be changed, they are architected to be unchangeable. We don't want them to be that way, but they are. It's because our application architectures of a couple of years ago resulted in unchangeable systems, we can only change them by changing the complete system. The monoliths are preventing us from becoming post-Bimodal IT organisations.

Back to the misconception. Bimodal IT is something that has to do with the monoliths in our IT landscape and nothing with how we perceive our IT or the systems in our IT landscape. The misconception is that we need to adopt a Bimodal IT approach to how we do IT because we don't have to. We have to get rid of the monoliths and we need to start by isolating them behind a layer of changeable software. Hide them like a skeleton in a closet of services.

By adopting Bimodal IT today, or continue with Bimodal IT today, means you either have a totally wrong understanding about how to manage your IT department, or you are still in the previous millennium when it comes to IT architecture, or you're just using Gartner's reputation as an excuse for not having to modernise your IT processes.

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link to my blog to all your Whatsapp friends and everybody in your contactlist.
But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Iwan


Although this post is my personal opinion, there're quite a few articles to be found on the internet about this topic that I've read. So here're some interesting reads:

- Gartner Glossary: Bimodal IT
- Gartner's bimodal IT mistake: DevOps can deliver velocity and quality
- Bimodal IT: Gartner's Recipe For Disaster
- Beware the Dangers of Bimodal IT
- Gartner’s Evolution of Bimodal IT
- The Flaw at the Heart of Bimodal IT




*: Interestingly enough, they forget to mention that the back-end is running on-premise and they're in the cloud and yes, the front-ends hosted on-premise are just as much under the same regime as the back-ends.