How Charming is Laziness?

Long time ago, while I was still a student at the PolyTechnic in Enschede, The Netherlands, I would justify my choice to study computer science, by proclaiming that laziness leads to efficiency. Which of course makes no sense I now know, because it leads to effectiveness. Penny-pinching leads to efficiency. You can read all about it in this post Perish or Survive, or being Efficient vs being Effective. But there's this little concept that results in more efficiency and laziness as well. It's therefore a charm. It is called automation and closely related to that third time.

Third Time Is Automated Principle

One of the key principles at Amazon AWS is that everything must be automated. It's not just that everything should be automatable, but it should be automated.

Whether or not it's an urban legend, but word has it that when you create something that requires manual action, you're out looking for a new job. For a company like Amazon AWS it is clear why automation is such a huge thing. And it is clear as well why they have such focus on API's. API's are how automation across the board is facilitated. But most companies are not Amazon or any of the other cloud providers. Most of you my dear readers are more likely to run your systems on Amazon AWS, Google Cloud or Microsoft Azure, then run those applications for your customers. Probably, your IT landscape is in size not even close to Amazon. Probably comparing to Amazon AWS as the Netherlands compares to the rest of the EU. In most if not all of the dimensions you can think of.

Arguably, the principle of "Automate Everything" doesn't apply to you. I'll leave it up to you to think of one or more arguments why automation is not something you should hold dearly.

Challenge to you: put in the comments a good reason why you think automation is not necessarily needed. I'll make an effort to counter your argumentation as a reply to your comment.
But read on first.

The benefits of automating processes are many. Irrespective of the kind of processes. An automated process is infinitely more likely to be repeatable than any manual process. This results in higher quality since errors will either be made consistently, and can be fixed, or will consistently not occur at all. How compelling is that?
Although the automated and manual version of a process might take the same amount of time to be executed, the automated process allows a person to work concurrently on something else that cannot be automated. And automated processes do not rely on the availability of a specialist to execute the process. So automation makes you and your organisation more scalable.

Not automating processes, even IT processes doesn't make sense. Still, when it comes to IT, we hardly do this. Why?

The situation at Amazon

I've come across many situations where things weren't automated. Worse yet, they could be automated. They were not automatable.

For me this was always an interesting fact to find out. For one, we're in IT and IT is all about automation. In fact, in Dutch we refer to IT as Automation (Automatisering - Dutch). The paradox is that we apply IT all over the enterprise to automate business processes, but when it comes to the IT processes themselves, automation is very likely the last thing on our minds. And when you think of it, that doesn't make sense at all. Walk into a room full of IT people, and just pose the statement that it is hard to understand why we're so good at automating business processes, yet we don't have automation in our own processes. And you'll see at least 90% of in-agreement-nodding heads, the remaining 10% are too flabbergasted with the realisation that this is a true statement. Same statement in a room full of non IT people and the first thing you find yourself doing is explaining why this is.

The fact that IT people are not automating their own processes is tough to explain, and I for one do not have such an explanation. There is an explanation though, for why Amazon AWS's processes are all automated. It's because one of the core principles by which they do IT is "Automate Everything". At a dinner party with Werner Vogels (Amazon's CTO) I was invited to, being the Chief Architect of a FinTech startup, I asked Werner (all attendees were told that we were on first-name basis), how it was possible for a huge company like Amazon AWS to live by these rigorous principles. Everything is an API, Automate Everything, and a few more. His reply was that there were two main reasons why it works. 

1. Senior management all the way up to Jeff Bezos, were openly behind these principles. In fact many of them were mandated by Jeff Bezos himself. 
2. Everybody in the organisation experienced for themselves the validity of these principles.

I asked him to clarify that second reason. According to Werner Vogels, Amazon is dedicating a lot if not all of its time to make its processes impacting customer experience as efficient as possible without impacting customer satisfaction. 'A satisfied customer is a returning customer'. The effects of changing a process is made visible to the whole company, at all times. So compliance to the principles will result in changes to the processes and the effects would be visible. Therefore, the validation of the principles would be continuous. And according to Werner, nothing is as motivating to change your processes than to see the effects of your efforts.

Rest of the dinner I was milling this over, enjoying the food and talk to the other guests.

The situation in the 'real world'

Unfortunately, most of us work at real enterprises. And those two reasons Werner Vogels had given me why IT process automation worked for Amazon aren't obvious in the situations I have found myself in.

For example, the amount of automation in a process within IT is not a metric that anybody is held accountable for anywhere I've worked or consulted. Neither are the benefits of automation part of somebody's accountability.

IT process cost reduction (IPCR) is as far as I know not a KPI within enterprises. Nor is the time-to-market (TTM). The latter often does find itself in another incarnation on reports and dashboards, namely as MTTR, the Mean Time To Resolution. When it comes to MTTR, we do see them on reports, but the MTTR is in hardly any case part of somebody's accountability. Same goes for time-to-market. Although it is always mentioned for any improvement project, it is hardly measured or reported on. It's a project issue in most cases, meaning that an organisation is doing projects instead of delivering products.

The lack of real metrics and if you will KPI's means that from an accountability perspective, there is not a clear person that has an incentive to push automation. And if you read my blogs on a regular basis, you know that I'm big on accountability.
Visibility of the effects of changes to the IT processes is another challenge for organisations. We often find ourselves in organisations that don't have a culture to measure our IT processes. This is especially true for organisations that have been around for decades. Since IT process automation is not pushed, there is no incentive to find bottlenecks in processes or pinpoint areas that are up for improvement. Resulting in the situation where the effect of improvements are not visible in most cases.

The lack of accountability and the rather big hurdle to be taken by IT departments in order to automate result in a situation in which we are just not automating our processes, because it gets no priority on our backlogs or funding in our PRINCE2 budgets. Process automation is collateral. And it's not a matter of not being as large a company as Amazon. It's a matter of not being aware of how automation affects the bottomline. And of not being held accountable for impacting the bottomline.

Third Time Principle

Three times is a charm, or at least should be automated.

In organisations I worked previously, the principle of automation as in play at Amazon was a little bit more pragmatic. One that has worked well for me, is the principle of "Everything done a third time, will be automated", reasoning behind this is that if you do the same thing a third time, it is extremely likely you will do it a 4th, 5th and even more times. The time required to create the automation will be saved by executing the task over and over again.

Time invested is never gained, but can only lead to savings later on.

The Third Time Principle is a compromise, although one might argue that it is a Troyan horse. By adopting this principle, the argument that it creates too much overhead for mundane actions is off the table. Only those actions that are performed repeatedly are automated. Built in justification for the investment needed.

The Third Time Principle is easy to adopt. Since it is a compromise, it can be applied when the push to automation is bottom-up. We often see that engineers see the need for automation since that is where automation will solve a problem and effects are noticed. To develop the automation is often a matter of getting the time to do so. Automation is now competing with other requirements for development time, for priority on the backlog. We all know where the priorities will be. Applying The Third Time Principle will remove this obstacle. Engineers can justify the need for automation and the Product Owner can justify the priority of the automation stories on the backlog.

Continuous Delivery

When striving for Continuous Delivery (CD) and more so for Continuous Deployment (also CD), there is no other way than to automate everything. CD requires a rigorous regime to move every manual task to the left in a process defined western style (i.e. left to right notation). Product development follows the DTAP model. Development is followed by testing is followed by accepting is followed by production → DTAP. In CD we hold true to the paradigm that everything manual is done in D, and TAP are fully automated. Any manual action in T, A and P needs to be automated and the scripts for this are developed in D, because otherwise it can't be done.
So when striving for Continuous Delivery, everything in the product development cycle is to be automated.

Accountability

When process automation metrics are part of somebody's accountability, that person will also be mandated to automate as much as possible. This is for the simple reason that you can't hold somebody accountable for something they cannot influence. Therefore, when process automation is measured through some metric for which somebody is accountable, it is that person's prerogative to drop The Third Time Principle and instead adopt the Automate Everything Principle.

Accountability is a matter of top-down enforcement. It is also a key aspect of culture change. When we want to adopt a culture in which we allow ourselves to reap all benefits of automation, especially our IT processes, we can't get around the fact that we need to revisit the metrics by which we hold ourselves accountable.

Scalability

One aspect that I haven't really touched upon is scalability, although I did mention it briefly. Automation is a key aspect of scalability, not so much scalability on a technical level but organisational scalability. Manual actions always need a person to execute them. The more complex the activities become, the more experienced or knowledgeable the person needs to be. And before you know it, it requires a specific person within the organisation. Because you rely on this person, that person will become the person with all required knowledge, it's a vicious circle. Think about it for a second, and I'm sure that you can think of a process or an activity in a process where you rely on a specific person and you know that person by name. In fact, if you want the activity to be done, you will even call that person because she's among the busiest persons in the company.

When you want to keep activities simple, you need to automate them. The more you automate, the easier it is to keep the activity simple. Hardly anything is more simpler than to have a command like 'do_complex_activity.py' provided that the automation is done using Python. Anybody can run this command provided they have the rights to do so, and when done really properly, anybody can do it at any time, because all the complexity of who can do what when is taken care of by the automation code as well.

Concluding

In the end, you'll agree that automation is what we need to apply to all our processes. At least when it's the third time you're doing the same job. It will improve quality and consistency. It also means that we as a company can scale our organisation without the need to increase headcount. It will be important to monitor the benefits of automation, without metrics it will remain a matter of good faith on the short term, and it will not justify the investments on a longer term. By understanding the benefits of automation and getting insights on where automation has most impact.

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect


The text very explicitly communicates my own personal views, experiences and practices. Any similarities with the views, experiences and practices of any of my previous or current clients, customers or employers are strictly coincidental. This post is therefore my own, and I am the sole author of it and am the sole copyright holder of it.

Why CDM resonates with Waterfallians and not so much with Agilisto's

Architecture often is a matter of perception.

Architects that consider that architecture to be a noun, often consider, for example, that a CDM (Canonical Data Model) is a solution to a problem. Architects that consider architecture to be a verb, are very likely not considering a CDM at all. And although I have very strong personal feelings against architectural artefacts like a CDM, I'll try to explain in this post, why they can be perceived as an addition to an IT landscape. I think that is also very much an issue with a historic touch.

The School of Waterfallians

When you're a Wterfallian, when you come from the Waterfall school, then architecture is part of the analysis and design phase. This phase ends with 'The Architecture'. And that "picture with the 1000 words compendium" will have to last for the rest of the project. So you have to design something, come up with an architecture that will remain unchanged for months if not years. Your data model is obviously part of your architecture and will be included. Soon you'll draw the conclusion that a durable architecture requires for CDM, because how else can you ensure that your design fits within the landscape of existing and future designs?
If you have been raised with the standards and values ​​of a waterfall organisation, then you also know that deviating from previous decisions is out of the question. That only causes delays and budget overruns. Waterfallic architects will often focus on edge-cases, because they have the experience that these are the reasons to double back on previously made decisions. Logical behaviour is therefore also that you want to have it, your list of edge-cases, as complete as possible before you get started. Waterfall is a self-reinforcing approach when it comes to architecture and culture.

The Agilista School of Architecture

If you come from the agile school of architecture, then architecture is part of the development phase. Architecture emerges. The architect is thus a developer, or better said, the developers create the architecture. The agile architect therefore only design the rules-of-engagement, they merely create the play-book. A comply-or-explain concept ensures that the architecture can emerge and the best architecture at that point of time can be defined by the developers. The best architecture within the current context emerges. So you do architecture (verb), and you do that by adhering to the rules. This is a continuous process, you have to play by the rules continuously. When we look at a CDM, we find that the CDM itself is not that exciting in an agile world, instead the rules that a (C) DM has to comply to are. They are the grammar of the language.
When you were raised with the norms and values ​​of an agile organisation, then you also know that sticking to earlier made decisions is out of the question. At least, if this is against better judgment. That only causes a huge waste of resources instead. The agile architects will focus on the most common cases, because they have the experience that these have to be realised first in order to create value, validate hypotheses and provide insight into the next steps to be taken. Logical behaviour is therefore also that you want to have the first use-case defined as soon as possible in order to get feedback. Agile is also a self-reinforcing approach when it comes to architecture and culture.

Culture and Values

The crux is in culture and therefore in standards and values ​​and therefore in accountability. When the performance of an architect is measured based on the number of times that decisions have to be reconsidered, then an ever decreasing number of this metric is, for a waterfallian, an indication that things are improving. For the agile architect the opposite applies and an ever increasing number of reconsiderations will be better... or not.
Because the agile architect does not double back to earlier decisions, as the agile architect tries to find the best decision for a situation every single time again.

Hmmmm, then how would you evaluate the performance of the agile architect? That question is in fact not that hard at all to answer. The agile architect is not about the architecture but about the rules. The more stable they are, the better. At the same time, the rules have to be sufficiently complete to be able to develop products that offer the organisation a bright future.

The Architecturally Challenged

Who has the more challenging task? The Agilista or the Waterfallian?

That can not be answered, because they can not coexist. But it does make it clear that both have a big challenge. Each in their own system.

The waterfallic architect is mainly concerned with analysing and specifying everything in advance. It is someone who is good at overseeing many concrete aspects. She is someone who knows a lot, someone who thinks in as-is and to-be. That is to be expected, because in a waterfall organisation the architect is often the person with the most experience with a product or product group. The focus is mainly on 'what' and 'how' in the 'why, what, how' system.

The agile architect is mainly concerned with understanding the dynamics of the organisation. She is someone who is good at working at the abstract level and thinks in concepts. It is someone who understands a lot, someone who thinks in from as-is towards to-be. That is to be expected, because in an agile organisation is often the architect who has the most experience in how the organisation has developed. The focus is mainly on 'why' and 'what' in the 'why, what, how' system.



Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect


The text very explicitly communicates my own personal views, experiences and practices. Any similarities with the views, experiences and practices of any of my previous or current clients, customers or employers are strictly coincidental. This post is therefore my own, and I am the sole author of it and am the sole copyright holder of it.

Product Owner and Architect, agile tag-team

Picture yourself in an environment where agile practices are adopted as the standard way of developing solutions. Now if you're really ambitious, you're picturing yourself in an environment where there are no projects, only products. Consider yourself to be part of a team working on a product, a product team. The team is comprised of all the necessary expertises to create the product it's developing. The team you're part of is responsible for the product, cradle to grave.
But wait a minute, are you really part of the team? Or are you external to the team, but do consider yourself heavily involved with the team? You might be the Product Owner or an architect.

Is the PO not part of the team? Maybe, maybe not. It's not that important actually. And what about the architect? Is the architect not part of the team? Well, the architect, I would think and this is just my very humble opinion, is not part of the team. The architect is concerned with the team, but not this team, but all the teams that are in the architect's domain. The architect is there to ensure that the products form a cohesive landscape that, by synergy, creates the most value for the organisation as is thinkable. Back to the PO. The PO might own many products in a portfolio...

Your picture. Did you picture yourself in this organisation as part of the team, of a team, or as somebody outside the team? It doesn't really matter too much. Really.

The interesting part of such an organisation, be it a for-profit or a not-for-profit or even something governmental, such an organisation is catering for its business, for the users of its products. And the products are there to generate value, pure and simple, life's better with every new version of the products the organisation releases. The product teams are there to work on these products, to make sure that the users are actually able to use these products and once they have those products at their disposal, they can continue to use them.

Other than in more traditional organisations where products are developed in projects, where there is a project organisation and a operations department to keep the solutions running. In these organisations teams, the project teams, are only concerned with getting the products out. And in many organisations it doesn't even goes as far as that and the project teams are primarily concerned with ensuring the products are accepted by the operations department. This is how these organisations work, and this is how teams operate. That's not to say that the individual members of the teams are not concerned about the quality of what they create or whether or not users actually like their products and use them. And that's not to say that the people working operations department are not concerned with the development of the software and that the quality is great etc. Organisationally, there's a divide and more importantly there's no one single entity that is responsible cradle to grave for the products.

What about this tag-team, this agile tag-team?

The effectiveness of the PO-Architect team, this tag-team, is the greatest in this product oriented organisation. The more 'agile' is considered a business trait instead of just an IT trait or even solely a software development trait, the bigger the impact the tag-team can and will have on the bottomline of the organisation, in a positive way.

Here's why. The PO, when peeled like an onion is at the core only caring about creating business value. And the PO has two currencies that he can use to invest in order to create even more value. For one there's the crude oil of product development: €'s. Here we have the simple fact that the the business needs to be able to earn more money with the PO's products than the PO had to invest in creating the products. It's a simple matter of cost/benefit and ROI. The less the PO needs to spend, the more the business makes and the shorter the time for the business to earn back the investment the better.
But there's also the second currency, which is time. Time taking to create the product, or a new value creating feature of an existing product. Time has no ROI. You have to invest time, knowing you'll never get it back. You might save time with your product, but the time you invest, you can only invest once. This is important because you want to invest as little of that time in something that generates the most value. Think about this for a second... what it means is that you want to create a new product (or feature) that will generate the maximum value in a minimum timeframe. The crude oil, your €'s are secondary to this, as a PO, because you have a product team available that you need to pay for anyway. Remember, you're working in a product oriented organisation not a project oriented organisation. Your teams are pre-funded! This is also important to note.

So the priority of the PO should be generating as much business value as possible. Therefore the PO needs to prioritise the various products and their features such that the most value creating features are worked on by the product team first. And this is extremely tough for the PO, especially when the PO is not a business person because the PO needs to know what's good for the gander as that is good for the goose.
And in many cases the PO will need to experiment in order to find out whether or not a feature is really needed. Whether it is really going to be used. Whether the form it will be delivered in is the most optimal form. Experiments cost time as well, so the balance between experimenting and applying is critical.

This is where the architect comes in. Without the architect the PO is lost, or at least the PO is susceptible to overspending. First of all, the architect's view is more than just a product and is therefore prone know about solutions to partial problems available elsewhere in the organisation. And leverage them. The architect is the right person to lay the foundation for future features. In part by ensuring that there is a generic (application) infrastructure that can be leveraged throughout the lifecycle of the product. When the PO has the insight to define a product roadmap or a business direction for the product(s) to evolve in, and shares this with the architect. The architect will be even better equipped to pro-actively define the foundations of the products, where a little investment upfront pays itself back at a later stage. See the parallels with the PO, the architect is there to understand where and how best to invest product development time to save more product development time at a later stage.
But there's more, obviously. The architect, and this is the 'secondly' belonging to the 'first of all' from above, is best equipped to define where what corners can be cut for most impact and what experiments to run in order to get the most out of new or specific capabilities. The architect will be the single one person that understands how to scale the organisation's technologies. How to move from a simple, crude yet effective archaic technology to a more complex, elegant and sophisticated version and when to do so. The architect therefore can and will save the most valuable currency of the PO, feature development time, at every step of the way.

Remember that PO and business value creation? Value should be created as quickly as possible, so short development times are important. As much as possible value should be created, so the most desired product features with the biggest impact on the bottomline should be worked on by the team. But which are these features? Well the architect doesn't know these answers, but is perfectly suited to limit the amount of time it takes to either develop a feature or to find out what feature should be developed by devising an experiment. The PO still needs to run the experiment though.

Ah, here comes the hardest part of the post. How to set the right priorities? Which is a question the PO will need to answer. The common way to do this, the wrong way indeed, is to work on the simplest feature to develop. Or the one that can be delivered the fastest. Often these are the same features by the way. This will only result in the PO and the team to feel really good about themselves because they added many features to the product. Something that will in most cases be far from adding the most business value to the product.

Again the architect is a great team member for the PO in this as architecture is not about simple to develop features or quickly realisable solutions. Architecture is about ensuring durability, sustainability. It's about creating a playing field in which complex solutions can be developed quickly, because the groundwork has been done already and therefore a solid foundation is present.

The challenge is with the PO still as business value should be the key to the priority of the feature backlog. So with every feature the PO will need to define how much value will be created, preferably in terms of what will be changed, and what will be the benefit of the change. From a user's perspective, always from a user's perspective. The PO will also need to establish a baseline for each feature he's planning to invest development time in. This is for the simple reason that there's no way to determine whether or not you've improved life as you know it, without knowing what life is before you started your improvements. And consequently, the PO will need to start gathering the right metrics in order to know those improvements. Defining the reason why you want to change something, i.e. defining what the benefits will be, will more or less define what metrics will show progress and therefore what needs to be done to establish a baseline.
Leaves the PO with defining the top priority of all the features that need working on. And this is encoded in the metric used to measured progress and the reason for the change to be developed. From these a projected generated business value can be established.
And this is where the architect can shine again. Not with this little piece of math, but with the part about metrics and measurements. The architect will be the instigator of the existence of a set of architecture principles that allow for metrics and measurements. Principles that lead to the existence of comprehensive measurements, to profound standardisation of those aspects that need no specialisations, to extreme automation of setting up environments. This can be left with the product team, but when there are more than one team, there will be a need and at the least a benefit, of coherence across teams, which is where the architect is acting.

Concluding, the PO and the Architect make a wonderful team that jointly work on the best prioritisation of the feature backlog for the products of the PO. By doing so, they'll minimise the required investments in feature development time for the product team to develop a feature, all the while maximising the created business value. And as icing on the cake, they are in the process ensuring that the hypothesised improvements are validated. You want a cherry on top? With this awesome tag-team you have a perfect opportunity to define a ubiquitous language which is business domain specific and spoken across the whole of the organisation.


Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contactlist. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

"We develop our frontends in Angular" is not an Architecture Principle

Did you ever hear that story about that architect who was telling the developers how to develop their solution? Badabing, badaboom!!!

Yeah, I know it’s a bad joke, there’s no laughter there. I better not aspire a job as a standup comedian. Thing is though that there’re still plenty architects out there that are not only defining architectures, but also tell the developers how to develop that architecture. It amazes me why this is the case.

I know that not that long ago I told you to ditch your architects when they are busy creating roadmaps. But actually you shouldn’t because there’s a real legitimate reason for defining roadmaps. The reason being that they’re an excellent tool. So ditch the architect when roadmaps are what they deliver, because those are the ones you want to get rid of.

Now there’s another breed of architects out there that I have a hard time to understand. I really think that architects should be architecting. That’s what their roles are, just like developers should develop. People should be doing what they’re supposed to do, and preferably spend some time outside their boxes while thinking about what their job really is.

For an architect, the sole raison d’etre is to have answers when asked questions and at the same time be enigmatic. Well, maybe that last part should translate into being helpful to whoever is asking the questions to get the answers themselves.

The architect, as I see it, is the one that defines the boundaries within which the developers can act freely while being confident that there’s no harm to the overall integrity of the enterprise or the organization for that matter. The strength of the architect, the real power, is that this can be done by defining Architecture Principles. These are extremely powerful because you can link them directly to your organisation’s mission, vision and strategy. Well if you can’t than you should reconsider your principles. But if you can, you’ve got your business justification for your principles and you won’t need to define the business value of those principles either. But just like any power-tool, they’re not trivial and to formulate the right principles you really need to be ready to spend some serious time. Furthermore, you need to limit the amount or principles to a maximum of 10, just like the 10 commandments. More will be hard to remember by heart and live by, which is what you should expect from your developers. Less is better, but beware of the fact that you should cover all the important bases in your environment.

There’s another tool that’s really powerful, which is the reference architecture. Which in fact is a model, one of the good kind as you can read here. A reference architecture is a model of an application that meets all the architecture principles. But beware; It’s a model, not an architecture! It’s a visualization of what the architect had in mind when he defined the set of architecture principles. And more importantly, it’s one of many possible models.

So as the architect defining the reference architecture you should be aware that everything in it, can, should and will be taken with a grain of salt. And as the developer you should understand that the reference architecture is just a tool for your architect to illustrate what was meant by the architecture principles, and definitely should not be taken literally.

Ah, so what about the joke, the bad one?

Well, there’s this group of architects that don’t understand the difference between architectures and models and instead of defining architecture principles and reference architectures, they tell the developers what and how to develop such that they comply with the principles.

There’re some serious problems here. For one, the architect omits the definition of architecture principles for whatever reason. Likely because it’s too hard, or they don’t see the benefit. Shame on them! Then there’s the case where the architect ignores the fact that the developers are very likely more proficient in developing than the architect. Oh, and let’s not forget about the fact that architecture principles are almost timeless, or at least stick around quite a bit longer than the typical technology, tool or framework. Yet, this architect dismisses this simple fact of IT and at the same time doesn’t keep the construction manual, because that’s what it is, in line with the current state of IT affairs.

For example, I discussed a couple of years ago, where ‘discussed’ is a euphemism for ‘verbal fighting’ with some fellow architects in some kind of architecture board that it made no sense to dictate that a multi-tier architecture should always be deployed such that each tier should be on separate infrastructure. Instead the application should be complying to the fact that data access logic not to be mixed with business logic nor with presentation software. Sometime ago I had a discussion, which was a real discussion, about whether or not we should define in reference architecture that either Eclipse or Visual Studio should be used for coding. Uhm, I think not, thank you very much. And yes, this was a real discussion.

There’s the discussion about what part of an application is developed in what programming language? We had the discussion at a startup I was Chief Architect at. Guess what, the developers made those decisions, I just stated the principles that I wanted them to comply with.

So what it actually boils down to is that architects need to govern, and that should take up most of their time. They should safeguard the integrity of an organisation’s IT landscape and make sure that in all cases the applications or rather the products in that landscape can comply with the architecture principles. The architect should promote and facilitate an unambiguous way of working among teams. The architect should also be concerned about an ubiquitous vocabulary within the organization when it comes to business and that it is translated by IT consistently.

The architect should not dictate that a particular programming language to be used, that a particular infrastructure is to be deployed to or that specific tooling is to be used. And in those cases where there’s a good reason for the architect to do this anyway, than it should be implied instead of explicitly dictated. In case all development is to be done in Java, the architect should ensure that only Java developers to be hired. In case everything should be hosted on Amazon AWS, the architect will have to pave the way for the teams to use Amazon AWS, making it the favorable option for the teams of the many hosting alternatives they can choose from.

As you might have guessed here, is that the architect I’m referring at is the enterprise architect or domain architect. Definitely not the application architect or the solution architect. The latter are both very much there to device a solution for a problem. They’re more like designers than architects in that sense. Which brings me to another little piece of personal amazement. Why are there so many large organisations that have domain architects on enterprise level whose domain has nothing to do with the business, but are strictly technical? Probably this is some remnant from the time that we considered IT to be a cost center and centralization of IT was the holy grail for many a CIO. Nowadays where we see IT as a product, something that is not just setting us apart from the competition but an actual product, there’s hardly any room for the these architects, at least not on an enterprise level. They should be working closely with products teams from the same product portfolio. Become Product Portfolio Architects, safeguarding the integrity of the IT landscape of the portfolio. Is it a demotion? Definitely not, it just means that they’re relevant again.

To paint a little context here, I’ve been that architect I’m complaining about. Emphasis on ‘been’. I enjoyed the technology too much to let it go, but as the domain architect I wasn’t allowed to work on the software, so all I had was telling the developers what to type. Initially I got away with it, because I was at least one of those architects that actually could still code. But really soon thereafter it became apparent that I hardly had enough knowledge or experience to really do the work. To me that was a real eye opener and I had to good fortune I was working with people that didn't hide the truth from me. They expertly conveyed to me that yes I was more than a decent architect, but as a programmer I really wasn't up to it anymore. And as much as that hurt at the time, it allowed me to focus on architecting. I still code, because I still think I need to be able to understand and to know that things might work. But as an architect, especially in an agile world I find myself more and more facilitating, communicating and coaching. Working 3 sprints ahead of the developers, not because I'm that a fast thinker, but because being an architect I need to be able to set directions. Let the team worry about how to get to our destination, be their own satnav system, and me the architect am more and more the person controlling the system's settings. Controlling whether or not to allow ferries, toll roads, to prefer back roads, go for the scenic route or the fastest or most economic route.

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contactlist.But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

Principles of the API-First paradigm

API's are weird puppies. They've been around since forever and whole battles have been fought in courts to gain access to API's. This was in the old days when having a platform with an API for 3rd parties to develop for your platform meant that you had a lot of leverage over those parties. Not providing early access to them meant that they would be late to the party and all the sweets where already shared between the early guests. Having an API meant having power. Not providing access to your API resulted in lawsuits because you were a monopolist, using your position as the platform owner to bully your competition into obliteration.

Nowadays the situation is completely different. Having an API means having a future. Nowadays to survive as a business you need to have a platform, which in turn means you need to have an API. Unless you have an API, one that suits the needs of 3rd parties and that is easy to use, you're doomed. You'll be just an Application Vendor, and toast before you can say "I wish I had built a platform". An excellent book on the topic is "The Age of the Platform" by Phil Simon.

As you can read in my previous post, API's are the hardest thing to develop. It always helps to have some guidance when you're up for a difficult task, and often principles help you getting this guidance. There're quite a few other guidelines out there on the interweb that are more than helpful. Below are 5 principles of the API-First paradigm. Followed by 5 derived principles and at the end of the post, I'm listing a couple of more than interesting resources on API design.

One final note before I go into the principles. An API is not much more than an interface into the depths of your platform. It discloses your platform and makes it possible for others to leverage it for their needs. It is therefore critical to have comprehensive documentation for your API. Now comprehensive doesn't mean that you need a big fat document describing everything you can think of that might be remotely related to your API. Typically you will need to document the technical interface of the API, i.e. the call signature of the API, including the input and output parameters of the API.

In addition you will want to document it's behavior exhaustively, i.e. only develop the behavior your documentation mentions. It makes perfect sense to use BDD (Behavior Driven Development, see my post on the topic) for this purpose. By doing so, your users know exactly what to expect from the API and it allows them to create stubs that behave the same as your API without the need to have access to your API's implementation. Finally it helps to have a simple example on how to use the API without making any assumptions as to why one would use the API. This could be through showing the relevant cURL statements.

Enjoy your reading…

Principles of the API-First paradigm

The API is the product.

It is owned by a product owner, and the responsibility of a product team. A team is responsible for it, from cradle to grave. This team is responsible for managing and ensuring its proper operation. Don't consider an API as part of another product, something that's a by-product, because it means that you won't treat it with the proper dignity and soon you'll find yourself alienating from your users.

The API makes no assumptions about its Consumers.

It is robust and resilient enough to thwart of any consumer with bad intentions as well as ignorant ones that have no clue about what they're doing. All this without compromising the service level to all well intending consumers. Mind that in most cases you're working on an API while making assumption as to who and why they'll use your API, only to find yourself in a situation where your users are all but the ones you envisioned, and all using your API for completely different reasons than why the API was created in the first place.

The API is suiting the needs of the Consumer

The API is developed only when there is at least one consumer and it is tailored to suit this consumer, by this the API is creating business value through its consumer. Don't be so arrogant to develop something that's supposed to be an API without having at least one consumer lined up to actually use it and tell you how great it is. Apart from the fact that working on something that is not being used is a complete waste of time, it also means that you're not really understanding the fact that you should not make any assumptions about the API's consumers. No assumptions at all.

The API is built to last

An API is never breaking compatibility with its consumers, although it may rely on the paradigm of 'Ignore what is unknown' to allow for extending the result. API's, other than the humans that build them, never break a contract. Period! This is to say that they will respect contracts, will have addendums to the existing contract, or in those cases when the API, which is after all a product, is no longer adding value, they will void the contract, but obviously respecting the contractual notice period. On a side note, it is fair to assume that its consumers abide by the convention that they'll ignore what they don't understand or know, hence the fact that you can add addendums to existing contracts without actually breaking the contract.

The API is idempotent.

At all times, in any situation, always. It doesn't matter what you do, how you do it or when you do it, when you call an API it always behaves exactly the same. Meaning that its result is always the same and therefore perfectly predictable. The order in which API's are called doesn't matter, as long as you put in the same values, you will get the same results.

Architecture principles for the API

The API is clear in its error reporting

Errors are either caused by the API implementation, the API consumer or something unexpected. It makes perfect sense to stick with the conventions of HTTP errors, why invent something new, when the existing is fitting your purposes perfectly. But whatever you do, make a clear distinction between these three causes for errors.

The API is always managed.

An API can be managed independently from;
Technical level, i.e. it can be accessed securely and consistently throughout its lifespan -> Scalability, Availability, etc

Application level, i.e. consumers can use the API based on a published interface that is behaving per this interface -> Interfaces, authentication/authorization, etc.

Functional level, i.e. users of the consumer can be provided access to the API based on the agreements made -> Throttling, metering, monitoring and reporting, etc.

The API can be implemented independently of its interface;

They are truly decoupled. Multiple implementations can co-exist and determining which implementation to choose for each individual call can be context driven. Mind that the interface should be technology agnostic, or at least as much as possible, and in no way should the interface disclose anything about the implementation other than that it should explain how it behaves.

The API is technology agnostic.

The interface can be called by a consumer of any technology without any assumptions of the technology used in the API implementation, and vice versa. This is pretty much an extension of the previous principle, but important enough to mention explicitly.

API's are like dogfood.

The API is used by the team responsible for it in the same use cases as the consumers for which the API was intended; "Eat your own dogfood" paradigm. Always make sure that in case you need functionality that one of your API's is providing, you're calling that API yourself and making yourself dependent on the API. Only this way you understand your API, and more importantly you'll be very cautious about how you treat your API and therefore how you treat its users. After all, you'll be one of them.

Interesting Reading

• Google's API design guidelines
• 18F API guidelines
• W3C Web API Design Cookbook
• CISCO API design guidelines
• Haufe API style guide
• Heroku API guidelines
• Paypal API guidelines

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contactlist.But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect