The Power of Accountability

Some months ago, I had a discussion with a friend and colleague of mine and ever since I’m planning to write this post. This post is about responsibility and accountability.

Summarising

Unless people feel accountability and responsibility instead of just writing it down, RACI tables are a playbook for the finger-pointing game and not a tool for continuous improvement. Accountability and responsibility are the keys to successful products. For architectures this means that they need to be owned, maintained and challenged. Constantly. Continuously.

The reason for the discussion I mentioned was that he was asked to write a short report on the state of the IT department of one of his clients. The issue at hand was that there were a number of serious problems with the IT systems and although outages were fixed fairly quickly, the root-causes were hardly ever addressed. Instead people would blame each other. Fervently. After a couple of interviews, my friend concluded that there was a serious lack in governance in all areas of the organisation. And over dinner we were musing over this whole governance thing. And while enjoying a good meal and afterwards a really nice cognac, we more or less stumbled upon two words that I have been using more and more ever since: Responsibility and Accountability.

So that is what this post is about. This time I’m writing about responsibility and accountability and their role in architecture.

RACI explained

I think that you’re familiar with the concept of RACI tables. Tables in which on a variety of interrelated topics it is noted who is R(esponsible), A(ccountable), C(onsulted) and I(nformed). In my experience, the RACI table is typically very much discussed and eventually agreed upon. But hardly ever relevant in practice. Reason for this is that the RACI table is listed in some document and only looked at when there is trouble.

Let’s dissect the RACI for a moment. I’ll do this back-to-front as that is serving my purpose best. Starting with Informed. This depicts all people and parties that are to be informed about something. Could be an design decision, a contract, a change in a process, anything for that matter. For simplicity I’ll stick with ‘decision’. Then there’s Consulted. Where the Informed are merely told about a decision that has been made, the Consulted are asked their opinion about that decision. Their input is weighing into the decision. The main difference between the Informed and the Consulted is the fact that the Consulted have influence. Informed listen, Consulted talk.

Thirdly, there’s Responsible. I know, it should be Accountable, but I’ll address them last. The Responsible are about making the decision based on their own knowledge and understanding, taking into account what the Consulted said about the decision as well. The Responsible are the people that are all about that the decision is a sound decision. They (should) care about quality. In addition, the Responsible are in many cases also responsible for consulting the Consulted and informing the Informed. I agree with this. Then lastly there is the Accountable. Note the plural in Responsible, Consulted and Informed and the fact that the Accountable is not plural. Accountability is not a team-sport! The Accountable is held accountable for the results, the effects, the impact of the decision.

When we look at this in a Product context, then the RACI is as follows: Responsible is about the production of the Product, the quality of the product and the quantity of the product. The Accountable is about the money made (or lost) with the (sales of) product. Now when you factor in the Product Owner, it should be clear that the Product Owner is all about creating value with the Product she owns. Because of the accountability aspect, the Product Owner needs to find the perfect balance between quality, quantity and cost in order to generate the most value... but the Product Team is responsible for achieving this quality, produce the quantity and keep the cost level stable.
It goes without saying that the relationship between Product Owner and Product Team is to be very tight and based completely on trust.

RACI for the Architecture

You're probably asking yourself where architecture comes into play. I hope you've already read my post on the Agile Architect. In that post I am explaining the role of the architect in an agile environment. Just read it if you haven't already done so.

With architecture it becomes tricky, because architecture can be implicit as well as explicit. Implicit architecture is architecture that emerges. It is the Product Team that is formulating the product's architecture while creating and changing the product. As the product is developed, the team will find, define and adopt practices. Some will become best practices within the organisation as teams share their experiences. But as the team is identifying practices that they find supportive to their product development, they will become an integral part of the way the team works operates and there you go; Implicit Architecture. Mind that architecture is not some soft of diagram with blocks and arrows, instead it is a set of rules by which the product is developed. There's a team understanding that once you live by these rules, the world will be a nice place to live. The architecture is a framework of norms and values. Everybody in the team is held responsible for working within this framework. At best, the Scrum Master can be held accountable for the fact that everybody is following the rules of the team. But with respect to the architecture, this is a stretch.
Once these principles are formalised they become explicit. You'll be able to read them somewhere. The Ten Commandments of the team according to which they develop their products. At this point the team is still responsible for complying with the architecture principles, but since they're written down they need to be maintained. Mind that an architecture is something that needs to be nurtured. It needs to be constantly challenged, validated and maintained. And that should be somebody's responsibility. And that somebody should be held accountable for the relevance of the architecture.

Who da man?

Think about it, once you define an architecture, once you write it down, you create the architecture for somebody to be used. You have created a product. And there should be somebody owning that product and somebody developing that product. Somebody being accountable for the value architecture and somebody responsible for the quality of the architecture. Give it some thought and write down who is within your organisation accountable and who is responsible for any of the explicit architectures. And here's another challenge; ask them whether they feel the same way. For argument's sake, let's call the person accountable 'Chief Architect' and the person or persons responsible 'Product Architect'. And now ask them who they think should be consulted and who should be informed, just to make the RACI complete.

Yeah, that last one is tricky as can be. Traditionally and be aware that tradition is still prevalent, users are informed about products. And traditionally business is consulted about the product. Which of course is all backwards. Which product has the best product/market fit? Exactly, it's the product that meets the needs of the users best. And who better to define what product is meeting those needs best than the user. So users need to be consulted, not informed. I repeat, users should be consulted and not informed. That also holds true for the architecture.
Since the Chief Architect is accountable for the architecture, she will have to make sure that the architecture is relevant. That it is continuously validated for its applicability. Thus, the Chief Architect will need to consult with the developers whether their needs for an architecture, for the architecture are met, and that they can apply the architecture while developing the products. Consequently, the Product Architects are responsible for continuously consult the developers about what they need, how they need it, what works and what doesn't, etc.
In simple terms, the Chief Architect is accountable for the product/market fit of the architecture, and the Product Architect is responsible for making sure that the architecture addresses the needs of the developers. Obviously, when there are domain architects or other more over-arching architects, they befall the same faith.

RACI to solve problems

Now back to the discussion with my friend. The whole accountability and responsibility thing. I hope that you understand that unless the RACI is not only written down, but actually implemented, it doesn't mean a thing and can only be used to 'properly point fingers' when things go wrong. But it will not help you prevent problems. This is because unless you feel ownership, you feel accountable, you are not motivated to pro-actively work on improving your product.
So unless people feel accountable and responsible, they will not consult the right people, inform others in a timely matter and will work on preventing incidents. Finger-pointing is the result. And this is also the case for less obvious products like architectures, contracts, privacy policies and for example employee contracts.

Accept the Challenge

Again I challenge you to name the person(s) accountable for the explicit architectures in your organisation as well as those responsible for those architectures. And I dare you to hold them accountable and responsible. And I double dare you to make them consult their users and not just inform their users. Well, actually, start by asking who their users are, and you are likely to be in for a surprise.

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

Where's the Agile Architect hiding?

Nowadays everybody and their mother is turning, transforming into agile beings. But where is the Agile Architect? In October of 2017 I did a presentation at ASAS2017 about the Agile Architect. Ever since I have been contacted about the presentation and specifically about what the architect's role is in agile organisations. Because of this, I think it is time to shed some light on this. Thus... Find out in this post.

Summarising

The Agile Architect is this Product Architect that is taking care of the organisation's software product landscape and maintaining its integrity. The Product Architect ensures that based on the input of the Business Architect and the Product Owner the product landscape is sound, adaptable and delivers overall architectural agility.

The difference between doing agile and being agile

The Agile Architect is a rare beast in today's agile organisations. But before I go down this rabbit hole I need to emphasise the fact that I am talking here about organisations that are agile and not those that are doing agile.
The main difference is that when you do agile, you're living by the book, like a religious zealot. You follow the rules, have your rituals and so on. It typically means that you're organised according to the school of SCRUM. Small teams of max 9 individuals, one Scrum Master, a Product Owner. working in Sprints that take typically two weeks, have your daily stand-ups, your retro's and demo's and a backlog of user stories and epics. In contrast, when you are agile, you're taking the lead from the agile manifesto, do the hard things as often as possible, prioritise your activities according to value, have a build-measure-learn loop in place and determine your next actions based on feedback from your users. This is the short version... Obviously.

Independence and autonomy are prerequisite for Agile Architecture

So having established this, I can continue on the topic of the agile architect. Well, just before I venture into that direction, let me explain that in an agile organisation, you also find a strong longing for independence and autonomy. Mind that it is not anarchy that is sought after. The challenge here is that different teams want to move with different speeds. Often this is called bi-modal or multi-modal (you can read about this for example in my post "Let me spell out why Bimodal IT doesn't work"). It doesn't matter how you want to call it, what it boils down to is that different teams work on different products and the organisation's market dictates that each product needs to adopt to changed circumstances in a timely manner, where 'timely' has different meanings for different products. It's extremely logical and also extremely unsexy. These different requirements regarding development speeds requires independence between products and therefore teams. Obviously, whenever there is a dependence, the speeds are linked and in fact agility is lost. Something I'll cover in another post some time soon.
So independence is required here. Autonomy is also important, because if a team cannot decide by itself how to solve problems, how to implement features, how to enable proper operational management, and instead have this dictated by a third party. There is a dependency with a bottleneck and hence agility is at risk. Look for the upcoming post "The Incompetence of the CC..." that will feel enlightening in case you don't understand this right now. But again, it is in fact quite logical how this all works.

So teams should be able to act independently of each other and should have a high level of autonomy, in fact the organisation should strive for the highest level of independence and autonomy that is possible within the context of that organisation. Note the subtlety of the last part of that sentence; that is possible within the context of that organisation. I am explicitly not saying that all control should be dropped and no governance should be implemented.

With the teams being autonomous and independent, the question about the role of the architect arises. Typically the first question I get is whether the architect should be part of the Product Team or not. And my answer is at all times that the architect is not part of the Product Team, where the Product Team is the team that develops and supports a (business) product. Just like the Product Owner is not part of the team, the Product Architect (that's what I call the architect) is not part of the team. This doesn't mean that the architecture of the software is to be neglected, what I am saying is that the team itself is responsible for the application architecture, referring to the internal structure of the software. There are typically a lot of important impromptu decisions to be made during the development of the software where the right decision is determined by the specific context in which the decision is key. The team knows best. That is not to say that anything goes. I'll get to what I mean by this in a second.

The Product Architect is not so much responsible for the software architecture, but for the positioning of the product in the organisation's landscape.The Product Architect is accountable for the integrity of this landscape. The meaning of this is best described through a use-case.
[Read this post on the importance of accountability and responsibility.]

Agile Architecting explained

Consider an organisation's product landscape to consist of several business domains, each consisting of several business products. A business product is a product that is delivered to a user, often a customer of some sorts. The customer typically accesses the business product through an interface. This could be a specialised App, a webpage or maybe a customer-success-agent. This is sometimes referred to as the business front-end, or the customer interface. What is important here is that the customer is not interacting with a piece of software perse, but instead could be interfacing with another human being. Demonstrating that a business product can consist of man and machine. In addition to this, business products can be related to each other. The final step of using one can lead to using another product. There can be a sequence, just like when you go to the movies. You buy a ticket, you get some drinks and snacks at the concession stand and then see the movie. These are three different business products and clearly the buying a ticket and watching a movie are closely related. But independent since the person buying the ticket doesn't have to be the person watching the movie. Think about gift-tickets.
So you see there are several (related) business products. How they relate to each other is part of the responsibilities of the Business Architect(s), sometimes called Business Domain Architect or just Domain Architect. It is the Business Architects that will typically decide to which domain a business product belongs. Buying things could be the Sales domain and watching a movie in the Media domain, or buying the ticket and watching the movie can belong the the Movie domain and the drinks and snacks in the Concession Sales domain. It's arbitrary but needs to be decided upon, this is what the Business Architect does.
Once it is decided to which domain a (new) business product belongs, it is the (new) Product Owner that determines how the business product is taking shape. Think about the MVP. Often the MVP is a first incarnation of a product, but it can be in fact be a completely different product that is used to verify that there is in fact a need, a requirement, for the new product. An MVP could be a simple teaser webpage announcing the new product or poster with a box attached that potential customers can put a card in saying how excited they are to get hold of the new product.
When the Product Owner ties the knot that software is to be developed, something typically decided after consulting the Business Architect and the Product Architect(s) for the products in a business domain, it is the Product Architect that comes into play. This is where the Product Architect is taking the task to maintain the landscape's integrity. First of all, the Product Architect will have to determine whether or not a new product is to be developed or an existing product is to be enhanced or adapted. In case of a new product development, it has to be determined what the best technology is to do this. Here it is important for the Product Architect to understand the product. The Product Owner is a significant source of knowledge and information here. Who'll be the user of the product and how will the user access the product? What are the security requirements and what about compliance aspects? Transactional integrity is important as well. Also the intended life span of the product is important. And obviously the product's relations and integrations with other products is key and for this the Business Architect needs to be consulted. Obviously all architectural principles are taken into account and the Product Architect will define what principles are such that specific practices are to be maintained. Consider a product that does financial transactions, that could mean that specific frameworks and products need to be used for the implementation. Or the fact that the product is going to be used by students after marketing campaigns, resulting in a deployment in Amazon AWS instead of the organisation's own data centres.
The Product Architect, in response to the Product Owner's requirements for the product, will also be responsible for example to define an API that is published as part of the product offering.

Product Architect vs Product Team

By deciding all this, the Product Architect is to some extent dictating the products architecture, and still it is the Product Team that is responsible for the software architecture and not the Product Architect. So how does this work? Remember I promised earlier in this post that I would elaborate on this.
It is actually very simple and all based on the role of Architecture Principles. The organisation will have its organisation wide Architecture Principles, principles that are to be followed at all times. Possibly depending on context, though. I always like to mention that KISS (Keep It Simple, Stupid) is the prime-directive of everybody. By stating so, everybody introducing complexity is violating this principle. As a consequence, the principle enforces the policy to deliver always the minimum that is required, hence helps implementing a LEAN mindset.
Apart from principles that are organisation wide and principles that are applied in a specific context, emphasising that the context needs to be specific, a Product Architect can also define so called 'local architecture principles' that are local to the teams within the circle of influence of the Product Architect. Think about these as local extensions on the existing principles. Maybe because the domain is subject to some specific regulatory context or maybe because the domain is specific for a specific market.
The Product Teams are always to abide by these principles. So although they have autonomy on how to do things, they need to stay within the framework of these principles. They can't just decide to make an overly complex solution, because that would violate KISS.
In addition, the Product Architect can also limit the acceptable answers to implementation questions. For example the Product Architect can state as a principle that all API's are based on Open Web Standards, but in addition decide that all API's that are exposed on the internet are to be based on REST/JSON and API's that are with business partners to be based on WS-*.

Consequential impact

All the above decisions also lead to other decisions. A decision for a specific deployment platform may mean that only certain teams are capable of doing the development. Or deciding that the product needs an API as well as a web-UI may result in the situation that two teams will work on the product, one on the product with the API and the other working on the web-UI based on the API.
Also, envision the situation where timelines are not really up to the needs of the Product Owner, it is then for the Product Owner and the Product Architect to determine the architectural shortcuts to be taken. Using different technologies to leverage the availability of another team for example.

Concluding

As you can see, the Product Architect's role is very challenging, just like the role of Product Owner. But although it is very challenging, it is also very much scoped towards the external aspects of the products. The scope is about the product in its surroundings. Think about the layout of a street with houses instead of the layout of the individual houses. Of course, the type of house, or rather building is a Product Architect's call. Villa's, brown stones or apartments is up to the Product Architect, not the amount of bathrooms, the type of tiles or the kind of sofa's. That's up to the Product Team in response to what the Product Owner can turn into value.

The Agile Architect is this Product Architect that is taking care of the organisation's software product landscape and maintaining its integrity. The Product Architect ensures that based on the input of the Business Architect and the Product Owner the product landscape is sound, adaptable and delivers overall architectural agility.

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

The Arc-E-Tect predictions for 2017 - In hindsight [2/2]

Last year, like every year, I did some predictions on what would be in and what would be out in 2017. But unlike other years, last year I actually posted those predictions on the internet.
Before I start with my predictions for 2018, I will go back to my predictions for 2017 and see how things turned out.
This is part two, and part one you can find here.

#6: KVI in, KPI out

"Forget about performance. Performance, in the end, means nothing when it comes to an organisation’s bottomline. What matters is value. However you want to cut it, unless value is created, it’s not worth the effort. And by value being created I mean that the difference between cost and benefit increases."

First prediction that I am looking at in this post is a bust. Although more and more teams and organisations are transforming into agile adopters, the value driven aspects of agility is still undervalued by most. I hardly come across organisations, departments or even just teams where success is measured in terms of realised value. Vanity metrics are pretty much still the norm. It's a shame because it also means that the promise of applying agile concepts are still a long way from being realised.

#7: Products in, Projects out

"It shouldn't surprise you, but I'm not a big proponent of projects and instead love to see it when organisations switch to a product focused approach. But in 2017 it will turn out that I'm not the only one."

This is happening big time in a lot of environments I've been in. The main reason why organisations transition from a project perspective towards a product perspective is because of CI/CD (Continuous Integration/Continuous Delivery). With reduced cycle times as a result of automation of the software delivery process, it is almost impossible to not release a product early and keep on working on it. Hence, delivery to production does not result in the end of a team.
My main concern in these situations is the lack of a Product Owner who has mandate over scope. The Project Manager typically does not have that mandate. It is the next step.

#8: Heterogeneous in, Homogeneous out

"In 2017 we’ll truly face the uprising of new and more technologies, concepts, architectures, models, etc. And in order to be able to manage this we will finally understand that we need to embrace the fact that our environments consist of a multitude of everything. In many smaller organisations that are at the forefront of technology and that are working in agile environment it is a given, but now that large organisations have also set out to adopt the ‘Spotify’ concept and thus teams have a huge amount of autonomy, polyglot is key."

Yes! Most organisations have dropped their need for huge standardisation efforts. Instead I see that architecture principles are becoming highly popular. With that and the gradual move towards autonomous teams I do see a shift in mindset where homogeneous environments is no longer considered the answer to all IT problems. This is also a mindset shift from efficient towards effective.

#9: Activities in, Roles out

"The thing is, we’re moving, as an industry, in the direction where we want be able to get feedback as early in the process as possible, which means that every person concerned with creating and delivering a products will be involved in everything needed to create that product and ensure that it works as intended and more importantly as needed. In this setup, everybody is what we in 2016 called a full-stack developer."

In 2017 this didn't happen. The T-Shaped employee and the Full-stack developer are found in small organisations. Large enterprises still have a culture based on decades of functional hierarchies. Contracts are still based on roles where T-Shaped and Full-stack have yet to find their spot. Unless agile transformations are no longer considered to be merely an IT and even just a software development thing, it will become very hard to get into cultures where teams are considered to be the atomic entity in product development and instead of roles and responsibilities, tasked are performed as activities.

#10: Agile in, Waterfall uhm... also in

"Well, agile is finally in and is going to replace waterfall projects in those organisations where there is an active movement towards agile. Which nowadays are the majority of enterprises. These organisations are heavily invested in dropping the traditional practices and adopting new, more business value oriented practices."

In 2017 I saw more and more organisations realising that the typical waterfall projects can actually be done in agile ways. This notion is actually causing the existence of waterfall to be questioned. Do we still need waterfall? No, not at all. But we still need large projects. In 2017 I saw a realisation by many managers as well as architects that large project and waterfall are not different words for the same behemoth, instead there is no a clear tendency to actually do large projects by applying agile practices and waterfall seems to be relegated to only tiny projects. Ironic, but pretty awesome.

This was part two of a two part on a quick glance on my predictions of 2017. Yesterday, I have posted part one of the series and see about how the first 5 predictions turned out. Next week will be about my predictions for 2018.

I hope you enjoyed this post. Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

The Arc-E-Tect predictions for 2017 - In hindsight [1/2]

Last year, like every year, I did some predictions on what would be in and what would be out in 2017. But unlike other years, last year I actually posted those predictions on the internet.
Before I start with my predictions for 2018, I will go back to my predictions for 2017 and see how things turned out.

#1: Microservice in, SOA out

"In 2017 people will start looking at Microservices as something that is useful and way better to have in your architecture than services. So a Microservices Architecture will replace Service Oriented Architectures in 2017."

With a massive transition towards agile practices and organisations embracing scaled agile frameworks, it has been inevitable, the Microservice Architecture (MSA) has been broadly embraced. Or has it?

In 2017 I've seen that those organisations that require true agile concepts in practice in order to be(come) sustainable also embraced MSA as the architecture of choice. The change in mindset that is required for MSA to thrive in an IT landscape and an organisation itself for that matter turns out to be more encompassing than mostly thought. I've seen it fail in those organisations that merely do agile, and succeed in those situations that are agile. Yes, MSA and Agile are going hand in hand.

#2: API's in, Webservices out

"Okay, in 2017 we'll feel ashamed when we talk about web-services and SOA. Instead we'll talk about API's. This is closely related to my first prediction on Microservices, which you can read here."

Here I can be short: There's hardly any talk about web-services anymore. It's all about API's nowadays and that has been the case for the better part of 2017. Over the course of 2017 the notion of API's also shifted from merely glorified web-services towards true business services.

#3: Application Architecture in, Application Model out

"Yes, in 2016 I've been confronted with application models. Again and again I have been slapped with models of applications and yes, I've been on the other end of the slapin' stick as well. Shoving application models into other people's faces. Stuffing it down their throats, making them, no forcing them to understand."

Unfortunately this prediction didn't come true at all. Although it depends on how you look at it. In 2017 I've been in more discussions than before about Application Architectures, although in most cases people were actually talking about models. I guess the terminology is out of vogue, but a lot of architects still have a hard time to use the correct terminology. Still, to me the Application Model isn't out and the Application Architecture isn't in. Just yet. Probably with a more widespread adoption of MSA, we're bound to ditch the model and embrace the architecture.

#4: Internet in, Intranet out

"So the internet will be in, and no longer will we consider the intranet as the context in which our software is running. Talk with any cyber security firm and they will tell you that security has become a real issue since computers got connected. Networks are the root of all evil when it comes to viruses and the likes. The larger the networks, the bigger the problems. And with heterogeneity the number of threats only grew, probably exponentially."

This so turned out to be a correct prediction, and like I envisioned, one of the main drivers has been security. And the lack of it, in many cases.

In most environments I've been working in and with over the course of 2017 there was a real notion that no longer was it affordable to not consider security on an application level and assume that applications could be accessed from the internet. Even when that wasn't supposed to happen. Finally we know that assuming the network to be secure is an assumption that really does make an ass of u and me (assume -> ass-u-me)

The good if not best aspect of this is a security-by-design mindset in most if not all people involved in product development.

#5: DevOps in, Scrum out

"I can be very short about this. Business has finally come to understand that IT is not something that enables them to deliver new products to their customers but instead IT is what they deliver to their customers. IT has become a product, and therefore an immediate business concern."

In 2017 it turned out to be not that short, unfortunately. What I've seen happening is that unless agility is a true business concern, a matter of business sustainability, DevOps is not something organisations want to embrace. Although this is primarily a matter of large enterprises, those with seemingly enough money in the bank to linger a while longer before feeling the need of being able to wart of the threads of start-ups and their agility.

This was part one of a two part on a quick glance on my predictions of 2017. In the next couple of days, possibly tomorrow, I will post part two of the series and see about how the remaining 5 predictions turned out. Next week will be about my predictions for 2018.

I hope you enjoyed this post. Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect