Architecture Principle - Everything is PorN Principle

Architecture principles are ground rules by which we develop our software. This ranges from ways of working to styles of coding, from software decomposition to business compliance, from infrastructure choices to security implementations. As you can see, it covers pretty much everything related to the development of business services, online, offline and internal.

Description

This principle assumes that in the software life-cycle different versions of a software component can exist concurrently. Traditionally this is implemented in organizations by adopting the DTAP model. A software component matures through the Development, Test, Acceptance and Production phases, where each phase has its own primary stakeholders and in many cases different SLA's, RACI matrices and ownership are assigned. In D, the developers control all facets of the component, in T no development can take place, test engineers test the component. In A, a user panel tests the correctness of the software by verifying all requirements are met and when the component is accepted it is taken into Production. This works well in environments where software components are relatively monolithic in nature, in these environments there will be more than one developer specific environments but only one T, A and P will exist.

In Service Oriented Architectures, and especially those situations where service orchestrations are the preferred means of composing applications, the DTAP model is too rigid to be usable. Within an agile environment we therefore follow the Everything is Production or Not (PorN) Principle, which means that everything is considered to be a production version, albeit that the there is a difference between the various maturity phases of the software, which manifests itself in different SLA's and different user authorizations.
In essence this means that we always deploy to a production environment, where components, services, that are still in development phase will be covered by a lower SLA than services that are already in the production phase. Additionally, those services that are in the development phase can only be accessed by 'users' that have the role 'DEV' attached to them. This is according to RBAC is everywhere.

Rationale

Within agile environments we typically assemble our applications through Service Orchestration. By 'stringing' services in a particular order we add or change new behavior of our (online) services. In reality this means that when new behavior is developed, many of the software components, services, will already exist, new components will be developed and will be in some state available to be used. As such, the new service XYZ can depend on the service implements ABC as deployed in production, and service QRS that is currently in the process of being accepted by a user panel. By dropping the DTAP model and adopting the PorN model, we ensure that new services are always developed in a scenario where either a service is being developed, or has been developed, in which case the latter scenario means that the service is of sufficient quality, Production quality.
With this setup, we can then always deploy any constellation of services in the same environment as our Production environment. The only difference between what is already production quality and what is not yet production quality is related to the services that are currently under development and those that are not. Or, in other words, the consumer of a service can be either a production quality grade component or not. Since consumers can be users, we differentiate between 'unforgiving users', for example paying customers, and 'forgiving users', for example developers or testers. Also, consumers can be other services, which either behave correctly or might behave erratic.
Thus, we have consumers that have high expectations or behave correct, resulting in a service constellation that will need to provide and can provide a particular SLA. Or we have a service constellation that has consumers that either expect little or behave badly, resulting in a service constellation that cannot provide the same SLA but doesn't have to either. Hence, it is the consumer that dictates the environment with its SLA and not the maturity of the software. Therefore we need to control which users can access a particular instance of a service.

Implication

Dropping DTAP and adopting PorN, we can no longer rely on infrastructure setup to safeguard the various maturity phases in the software development life-cycle. Instead we can do this by implementing the RBAC is everywhere principle, thus limiting the users that can access a (version of a) service that is still in development to users that have for example the role Developer. Unless the user has the role, she can't access that (particular version of that) service. Or in case the consumer is not a user but another service, that service needs to have that particular role.

In principle, it's all architecture, unless it's not... or something like that.

Welcome to the wonderful world of architecture principles. A world where everything is a has to unless you've got a good justification not to.

This is an introductory for a multi-piece on a variety of Architecture Principles that I've come across in the last 5 years while working on Cloud, SOA and POA architectures.

Architecture principles are ground rules by which we develop our software. This ranges from ways of working to styles of coding, from software decomposition to business compliance, from infrastructure choices to security implementations. As you can see, it covers pretty much everything related to the development of business services, online, offline and internal.

Architecture principles are setup such that you just comply with them, unless there's a very good reason not to, and then you don't. A good example might be that everything is hosted in the public cloud, unless for compliance reasons we need to host on-premise.

All principles should lead back to our strategic goals, business goals preferably.

As a refresher, the way architecture within many companies works is as follows: As a company we have a mission, and a vision on how to accomplish the mission we've set out. There's a strategy on how to implement that vision, which consists of strategic goals that when met, we know we've implemented our vision. Turning the whole thing into a road-map from where we are to where we want to be with milestones to be reached, and steps to be taken in order to get to the next milestone. Our strategic architecture shows us the shape we're in at the various milestones, the tactical architecture shows us where we are after each step we have to take. Architecture is therefore a journey. It's not a noun but a verb instead.

Architecture is just like a satnav, you get your bearings and determine where you are, you set the location of where you want to go and the satnav calculates the best route depending on your settings regarding toll-roads, ferries and time our distance settings. And with every turn to be taken, the satnav recalculates the best route based on the most current information regarding traffic, road works, etc.

Architecture principles tell us the rules by which we live while on our journey, there's a justification for these rules explaining why obeying that rule takes us closer to our goal(s). Traffic law if you will. There's a speed limit on the road we're traveling. Obeying the law and sticking with the speed limit will ensure a save trip to our destination, but sometimes we have a passenger that's bleeding out, if we don't hurry and get to an ER, forgetting about the speed limit, caring for the passenger only. When we attest the traffic fine associated with the ticket for speeding the judge will waive the fine as a person's life is deemed invaluable. But if we just go speeding because it's so much more fun to go 200 km/h instead of the allowed 130 km/h there's no lenience from the judge as 'fun' is not considered a proper justification. Taking into account, and taking stock off the fact that every violation, even when justified, increases our architectural debt.

Architecture principles are therefore to be considered the default action to be taken, the default direction to head for. It's a matter of "This is how we do it, unless...". There are sometimes obvious 'unlesses', as in "We use Linux, unless Linux is not an option". But in that same vain, the principle could be: "We use Linux, unless Linux is not an option, then we use Windows". In which case there is no unless, there's no excuse to not use Linux or Windows. Every principle of this kind is clearly formulated as such. When there's no room to maneuver, there's just no room to maneuver.

Lastly, regarding architecture principles, we have to understand that principles will change, they will be revisited over time. As our mission, vision, strategy or goals change, our principles may have to change as well. As our world changes, our principles will change as well. They're not carved into stone, they're written using Permanent Markers. And we all now that permanent markers on a whiteboard can be erased with whiteboard markers.

Changing our principles or violating them, results in architectural debt. And as with financial debt, there is an interest to be paid. The more debt we have, the more our software costs. This is very much like technical debt that accumulates during software development, but where the interest paid on technical debt is typically analog to the interest on a mortgage or even a personal loan with a bank, architectural debt is close to the interest a loan-shark issues. And not paying of those debts, well the analogy continues there as well.

Without any further ado, let's start our journey...

Iwan

API Versioning, another religious war

Summary: When you ask your best friend, e.g. Google, for an answer as to what is really the best way to handle different versions of the same interface, you'll find out that there is no straight answer. Actually it is a religious war and many of the  arguments given for either of the various ways of versioning interfaces are actually just opinions on aesthetics. Below you'll read my reasoning for what my views are on the various options you'll have to version your service interfaces. Probably you'll find other ways of skinning this cat as well.


One of the core aspects that we need to decide upon when working on our web-services is decide on how we will handle different versions of the services.

There are two dimensions to this:

1. Production vs. Non-Production (We're not doing DTAP but PorN) See the relevant principle here: Everything is Production or Not Principle. 
2. Current version of the web service and the next version of the web service.

Although these dimensions seem to be different, in fact we're talking about one and the same challenge we'll have to overcome.

Basically we have a version of a service in production, and one version in development (in case we need to change the service). Once that development version reaches production, we have two versions in production according to this principle Service Interface Compatibility Principle. As long as no customer can access the version that is in development, this version is not in production, according to principle RBAC is everywhere, every service can only be used by a user with the right role. Somebody not being a developer cannot use a service that is in development. This is an important notion because it removes the necessity of a separate infrastructure to separate production from non-production, although it still might be a good idea to have some separation.

As a consequence, we can deploy a service and limit access to the service through Role Based Access Control and since development of a service always results in a new version of that service, we can see the solution to dimension 2 as a solution for dimension 1, essentially allowing us to see both to be the same.

Looking at service versioning, we have to distinguish between service interface and service implementation. Since both have their own life-cycle, we can treat them separately. In addition, when we talk about service versioning we are mainly concerned about the impact of a new version of a service onto the consumer of that service, so we're concerned about the service interface and not so much with the service implementation. So within the scope of this post, we're discussing the version of the interface and not its implementation.
Furthermore, when we talk about services, we talk about RESTful services. And finally, we talk about the version of the published interface which is different from the version of the source code implementing that interface or the service attached to it.

When the interface of a service changes, we either deal with a change that breaks the consumer, or one that doesn't break the consumer. At all times we strive for the latter, since not breaking the consumer means no impact on the consumer. Changes that extend the resource, are changes that have no impact, since consumers of RESTful services are to ignore what they don't know. So any new fields in the resource representation will be ignored. Consumers that ignore these fields will not be able to benefit from the change. These are syntactical changes we want, and versioning has negligible impact on the consumer, the consumer does not need to be aware of the change.
Changes to the syntax of the resource representation we don't want but are inevitable, are those that change existing fields. For example they change the type of a field from an integer to a float. In this case we need to make sure that the consumer is not confronted with the change without it changing as well. Hence either the service should no longer be accessible, resulting in an error at the consumer, limiting damage. Or by being backwards compatible and thus returning the old representation to that consumer.

As a third change, we have semantic changes, which are changes to the actual resources. For example a generic resource turns out to have become more specific and we start limiting representations to just those specific resources. For example we have the concept of Customer which is a Company with which we have a Contract, and then we also have Customers that are not companies but Consumers, although still Customers, they're not the same. As the semantics of the resource has changed, we can't talk about the resource in relation to the changing interface as if it were the same resource. The actual resource has changed, so has its representation and therefore it impacts the consumer, which will break. In this case, again, we need to make the old service to become unavailable or we keep on supporting it.

There are in essence three major philosophies when it comes to API or service versioning:

1. Include a version in the URI that identifies the resource, e.g. https://www.foobar.com/api/<version>/<resource>
2. Include a version in the URI that identifies the resource, using a request parameter, e.g. https://www.foobar.com/api/<resource>?version=<version>
3. Include a version in the Accept-header of the request, thus using http Content Negotiation, Accept: application/vnd.<resource>.<version>+json

Of these three cases of changing interfaces, the last one is the easiest to address. Since the change results in a new resource type, the identifier of the resources, the URI, must be changed. This results in a URI in accordance with: http://www.foobar.com/api/<version>/<resource>. Although the location of the version indicator is arbitrary, the best practice is to have it as close to the domain as possible. Typically right before or right after the resource. The philosophy here is that we're talking about a new resource and not a new version of an existing resource. Hence the version indicator in front of the resources. Alternatively, the version is omitted and a new resource name is used. This is preferred when the resource should have a new name. In the example above the URI could change from http://www.foobar.com/api/customer to http://www.foobar.com/api/customer/consumer and http://www.foobar.com/api/customer/company.
Note that it remains a challenge to decide what the old URI will return.

The first and second change types are actually similar and should indicate a new version of an existing resource. When we encode the version in the URI, it makes most sense to do it after the resource indicator; http://www.foobar.com/api/<resource>/<version>. The problem here becomes immediately clear as it very much resembles the version in the previous scenario. Keeping these the same will be confusing. Alternatively, this would be a good candidate for the Accept-header option, keeping the URI the same and only use the accept-header for content negotiation. The resource stays exactly the same, so the identifier of the resource as well. The version of it's representation changes though, i.e. the indicator in the request depicting what format is accepted by the consumer, is explicit. Accept: application/vnd.<resource>.<version>+json.

There is a rather major drawback with this approach in that it becomes rather complicated to quickly test the solution, since from a browser, the http-header is almost never open for manipulation. So this solution doesn't allow for usage from the browser address-bar. This is typically solved by supporting both versioning strategies simultaneously. Which allows for two major advantages, besides support for programmable consumers and browsers;

1. The URI approach for versioning will return the relevant documentation of the API for that particular version, if that version exists or an http error when it doesn't.
2. The baseline URI can remain and stay usable. Access to the resource is through http://www.foobar.com/api/<resource> and the acceptable version by convention when none is specified in the header is the first version of the resource, or alternatively, the oldest supported version in case versions are at some point dropped.

By following the above, the intentions of services and therefore API's remain.

When you ask your best friend, e.g. Google, for an answer as to what is really the best way to handle different versions of the same interface, you'll find out that there is no straight answer. Actually it is a religious war and many of the  arguments given for either of the various ways of versioning interfaces are actually just opinions on aesthetics.
Above you're reading my reasoning for what my views are on the various options you'll have to version your service interfaces. Probably you'll find other ways of skinning this cat as well.
My advice to you is to closely look at your specific situation, what you want from your interfaces, the need for (backwards) compatibility and then choose wisely.