January 14, 2017

API Management in Azure using Microsoft's solution - Why API's should be managed and by whom

This is a series of posts regarding the topic of API's and API Management on Microsoft Azure. Not all posts in this series are directly related to Microsoft Azure and their implementation of API Management, that is intentional. The series also explains about API's, about creating API's and about what it construes to in order to manage them, conceptually and in reality. The reason for this series is that over the past 12 months I've come across many articles on the web, have been in many discussions and advised various clients of mine on this topic. Sometimes discussing with fellow architects, other times with vendors, still other discussions where with developers and managers. But the overall situation has always been that none of the people at the other side of the table had a full grasp of what developing API's in an organisation means, what it entails to manage them or what should be worried about when deciding to go API. I hope you enjoy reading these articles, and when you feel like it, comment on the articles. I always take serious comments serious and will reply to my best effort.

I hope you've read my previous post on the topic of API Management, if not I strongly advise you to click here and read it.


Although managing API's has some resemblence with Looijen's model for IT management, the typical segregation between business and IT that stems from Looijen's model is disastrous for API management. API's need to be managed by a single team, the Product Team. Any other way of managing an API doesn't make sense. API's are products and should be treated that way.

API Management

API's are interfaces to (business) functionality that are accessed by, in theory, just about anybody. As can be read in my previous post, the idea of having API's and one of the main reasons why developing them is so hard, is the fact that you don't know up front who is going to call the API. The point in fact is that in a world of API's there is no room for the arrogance of the developer knowing exactly who is going to use the software and why. Instead, API's are inherently developed to support the case that the developer was wrong.

Because of this, when you expose an API, you'll need to make sure that you can support the API once it is deployed and used by somebody. Supporting means that you need to make sure the API is running consistently over time, the API must be usable and stay usable over time. Hence the API must be managed.
In order to do so, API's need to be managed on three different levels that to some extent relate to the support levels documented by Maarten Looijen, of which a description in Dutch can be found here. Basically he defines Functional, Application and System Management

The Looijen Model

(Skip this section in case you're not interested in the different kinds of support types.)
The model is not shocking as it states that there're three types of support that is required to support an IT system:

  1. Functional Management. [Original text translated by Google:] Functional management is the management form that includes all administrative tasks that are required in the context of the use of information systems. As usage focuses on functionality, such as entering, handling, procurement, transportation and storage of data refers to information management.
  2. Application Management. [Original text translated by Google:] Application management involves developing and maintaining applications. In the words of Parts and Looijen: "There may be many situations in which changes should be made in the original application software or database structures (...) Whenever such a situation occurs, application maintenance is performed but also.. when there is no direct reason is to perform application-maintenance, it is because of the continuity may still be required in order to maintain a complete management shape surrounding this executive task area application maintenance, including management and executive tasks. " Application management is like form, as it were about the task area applicate maintenance built around. It is to recognize application is essential that once developed information not evident also delivers the tools to easily support the change process (eg. Documentation). In addition, persons other than the original developers may be involved in the execution of modifications. Therefore it is important to maintain conditions which an application must meet so that they can be maintained 'easy'.
  3. Systems Management. [Original text translated by Google:] Technical management includes the management form all tasks necessary to install, accept and operationalize and maintain information systems and technical infrastructures. Under technical management also includes optimizing processes and making changes in the technical infrastructure as a result of errors, extension or replacement. The technical management is focused on the technical platform, consisting of equipment associated with basic software, and implementation of information systems built on it.

Management of API's the Looijen way

Now that you understand what I mean by the Looijen model, it's time to inform you how API's should be managed.

API's are to be managed on the same levels as other software, that is the functional, the application and the system level.

System Level: On the system level, API's need to be managed such that they are made available. At the very minimum there should be an error stating the API cannot be accessed. At system level you manage the infrastructure on which the API is running. This might include some middleware as well. Since API's are to be called a lot, system management also covers scaling, i.e. elasticity, as well as all kinds of resilience aspects. Also security aspects like DDOS prevention, (web application) firewalls, etc. are part of the system management aspects of API management.

Application Level: On the application level, API's need to be managed such that they can be accessed, that the API documentation, (how to call the API?), is available, etc. Basically what it means is that there is an API to be called and that the technical aspects of the API itself are managed. For example the fact that an API accepts both JSON and XML parameters, that there is an API specific certificate defined such that the caller can verify the API call is served by a trusted system etc.

Functional Level: On the functional level, API's need to be managed such that they provide some value to somebody, preferably to both the consumer of the API as the team behind the API. API's are excellent for creating win-win situations, and these are business relevant situations. Basically what managing API's on a functional level means, is that you manage the relationship between the API and the consumer of the API. Think about a consumer having to pay for access to the API and setting different thresholds to control this.

When you look at the above different levels of API management, you may have noticed that on a system and an application level, managing the API is rather technical, but on a functional level it's really a business concern. Switching back to Looijen's model and you see that he makes a clear distinction between the two as well. System and application management are technical concerns, functional management is a business concern.

What if API's are not managed?

So what if you don't manage API's? The answer is actually rather simple. It becomes irrelevant, and in the end will seize to exist. For one thing, an API needs to be kept available. When it's not available, no consumer can access it. So on a system level it needs to be managed. This is no different from any other software application. Not managing an API on a system level results in the same chaos as not managing any other software, with the side note that API's are called by any(kind of) consumer, so the 'blast radius' cannot be predicted. From this, there's no reason not to manage the API on a system level as any software needs to be managed. There's one other point to be made here, and that is that an API needs to be kept available, meaning that it needs to scale, preferably horizontally and it needs to be resilient. Both are aspects that are not traditionally system management aspects as with API's this needs to be addressed on a software level.
On an application level things are getting a little convoluted with the system level management. In fact you might already have gotten the opinion that system level management is very much like application management and in fact you're correct. But system level management is more of a 'keep the shop open' issue, and application level management is more of a 'retain customers and grow your market' issue. When customer are not attended to, they'll go elsewhere. API's need to be managed on an application level as they are called because the consumer wants to. When the consumer needs to, eventually he'll leave for a more convenient API that provides sufficient added value to migrate to it. Hence the API needs to be amended to support other protocols, other data formats, more security etc. All aspects of application level management. See this as a matter of keeping the interface of the API consistent, but the implementation behind it evolving. So the API might be providing a consistent end-point to a consumer but evolves from a low-cost inefficient implementation to a costly but efficient implementation. This is API management on application level; replacing implementations behind an interface. If an API is not managed like this, it will become obsolete at some point. Obsolete API's are useless but cost a lot.

The system level and application level management concerns of an API are clearly required in case an API to some system or information is provided. When the API is not managed on both levels, the API will become irrelevant. But what about the functional level?

Functional management of an API is addressing an API on a business level. Read the post on API first design here to understand why API development is costly and if not anything else, the most difficult thing you can do in software development. Because it's so tricky, it's important that there's a business reason to actually do API development in the first place. And this is where functional management of an API comes to the stage. This is where it is managed that an API is monitored on a business level, for example keeping track of how many different consumers are using the API, how many of the consumers are 'returning customers' etc. Functional management of an API is all about managing the metrics about consumers of the API and act on them. Consumers that do a lot of calls may be willing to pay for each call to the API, consumers from specific industries will be more willing to sacrifice some performance in order to gain security etc.
When API's are not managed on a functional level, there is no point in developing an API. API development is so complex that there needs to be a business justification to do so. And it needs to be validated that the initial justification which is always based on assumptions, is correct. Not managing API's on a functional level will ensure that funding in the broadest sense of the word, will stop eventually.

Who manages API's?

API's are managed by both business and IT. Refer to Looijen again and you see that for software solutions, managing these solutions requires both IT and business. The same goes for API's. In a traditional organisation, one that has a clear separation between business on one side and IT on the other, API's are technically managed by IT and the developer of the API needs to provide an interface to the business to manage the API on a functional level.
Further complicating the scene, refer to organisations where IT is divided into a department that is more concerned about applications and the other department more about infrastructure. In this situation system level management is done by the infrastructure people and application level management is done by the application people.

Followers of the Looijen model are typically of the opinion that Application Support can be done by system managers. The reasoning often is that system managers are technically skilled people that understand applications as they understand the infrastructure on which the applications are running. And since they manage the technical aspects of the application and not the functional parts, they're more than suitable to manage the application as well as the underlying infrastructure. This view is incorrect (click here to read why) and when it comes to API's it's disastrous.
When you have an API based platform, and there really is no other way to have a platform, you need to consider the software implementing the API as well as the infrastructure on which it is running as a single system. It makes hardly any sense to see them as two separate pieces of the API puzzle. The reasoning here is that scaling, resilience and other aspects of the system level management of the API, i.e. managing the availability of the API, is an application or software concern that leverages infrastructure which in fact could be to a great extent be software as well. Therefore API's are preferably managed on both system and application level by the same team. Consider the API as a product and we're talking a Product Team here.
The importance of an API's availability and the complexity of developing an API leave no room for any other approach than a very tight cohesion between the API implementation and the full stack on which the API is running. Both the API and the underlying stack need to be developed in conjunction.
By the way, software developers shouldn't be concerned with system level management of the API and system managers shouldn't be concerned with application level management. Either both are in the same team, or the team consists of members that understand both levels of management, a rare breed indeed.

So what about functional managers of API's? These are business people, people that understand the business behind API's as well as the API they're managing. They're not technical people at all, they understand platforms, but from a business perspective. They can see IT as a product, and they discuss IT in terms of solutions instead of technologies. They're not Product Owners, they might be the most important stakeholders of the PO on some level.
API's have an implementation and they have consumers, but there're also people that make money of the API. It's the holy trinity of the API: Producer, Consumer and Beneficiary.
Because managing an API on functional level will have its repercussions on both the application level and system level of the API, turning the functional dials needs to be done in concert with the application and system level management efforts. Therefore, functional management should be done preferably by the same team as is taking care of the application and system management of the API. Resulting in a true Product Team.


API's need to be managed because otherwise they will become irrelevant and obsolete. In this API's are not other than other software solutions. The complex nature of API's are an additional reason to manage them carefully.
API management can be discussed using the model by Looijen on IT management as all key aspects of his model apply and the clear distinction between business and IT exist with API's as well.
On the other hand, Looijen's idea of a clean segregation between business and IT and even within IT is not a logical segregation for API management. In fact, API's need to be managed by a single team on all levels of API management; The Product Team.

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link to my blog to all your Whatsapp friends and everybody in your contactlist.
But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.


No comments:

Post a Comment

Note: Only a member of this blog may post a comment.