Oh Behave! How JIRA turned into a Veggie Salad

Hi,

Previous I posted about Why Uber is so applicable in the agile delivery of softwarer. It is only part of the story about how agile delivery is doing stuff differently from many other development practices.

Provided you have access to the JIRA and Confluence sites of one of the companies I worked for recently, which is the FinTech startup dopay, and you're following the various tickets on JIRA and read the relevant wiki site every now and then, you will come across terminology like Cucumber and Gherkin. Probably you've been wondering when you would come across Tomato and Lettuce as well. Chances are you will, because JIRA is turning into a Veggie Salad. This is, by the way, a trend you'll come across throughout the agile world.

All jokes aside, what's all this vegetarian mumbo-jumbo about.

Cucumber is a tool in use at many agile teams to verify that the developers build the right swing. Swing? Yes, swing, agile development is all about building the right swing .

You'll hear me refer often referring to the swing, and most of the times proclaiming that I want to make sure we create the right swing. The point in building the right swing is making sure that what ever is being build is behaving in the right way, is delivered at the right time and costs just about the right amount. Getting all three right is extremely tricky, and pretty close to impossible. So we go for the system that behaves the right way first and get as close as possible to our two other goals, making it on time for a decent price.

I'm using the term 'behave' deliberately. We don't want a system that has the right functionality but isn't usable, instead we want a system that is usable and as such, it will deliver the right functionality. There's a small difference in wording which is crucial. Here's an example:

Say you need to enter the details of an employee for one of the customers in a HR services system. Now the system would provide the function to enter employee information in a nice slick wizard interface, taking you through the whole process and when you're on page 6 and press the submit button all details are stored in the database, which is when you find out that the format of the mobile number of the employee had to be international. Good thing the faulty data wasn't stored, because it would corrupt the database. And now you can enter all data again, as the helpful system did a reset of the screens so no corrupt data can flood into the database and you're getting a second chance to enter the data correctly.

The functionality is exactly as it should be, employee details can be entered, the system validates all data and ensures that only correct data is entering the database. Awesome. Or not, as in fact this is what we want, yet is not really what we wanted. What we wanted, or what what I would've wanted, is that the system would have behaved in such a way that it would have me entering the data and as I go from field to field it would validate the information entered and inform me straight away that the data was incorrect and let me fix it without resetting the fields already entered correctly. It's the same functionality I'm getting of the system, but the behavior is what mattered. And I hope you saw the way I expressed that functionality in terms of what I want the system to do, and not to provide.

I want behavior.

I want behavior? Yeah, and good behavior, I want this from my kids as well as from the systems I use. Mind that I want my kids to get good grades in school, but not by cheating, but by doing their best.

We describe the behavior of our systems in such a way that we can not only document the behavior we expect from our systems, but we can test that it really behaves that way. The documentation is done using the language Gherkin and the testing is done using the tool Cucumber. And no, these are not the tools for the developers, these are the tools for the entire enterprise. And I can hear the screaming starting;

 Ohhh no, now he's asking me to start programming in some weird programming language,  AAAARGH!!!

Actually, you couldn't be more right. I do want everybody to start programming, and use that weird language Gherkin to do so. But unlike those obscure programming languages that are used by those developer types you see wandering around in office, Gherkin is really simple to be used. In fact, you've already been programming in Gherkin for the larger part of your life. Gherkin is just plain English (or any of about 40 other languages for that matter). Gherkin is really easy to write, read and well, read and write. But first let me show you a little piece of behavior description that is more formal than a typical requirement: 

Given that our sales agent Johnny Dough already registered ACME Inc as a customer, when Alice is accessing her Sales Management Dashboard to access the latest sales reports, then she will need to log into the customer portal first. 

I hope the line above makes sense and it is perfectly clear how we want the system to behave when a user wants to access her data. And guess what, that was all perfect Gherkin. Yup, Gherkin is human readable and not just developer readable. The cool part, this is also 100% the test script that will be executed to test the system.

SAY WHAT!!!!

Yes, you read it right. While you document the features of your system, i.e. the way you want your system to behave, you create at the same time the test scripts to verify that the systems really behaves that way. No more excuse that the wrong swing was build. And oh, there's another little interesting piece that you might have guessed from that little Gherkin script, it has an actual example of real life data in it. It's Johnny Dough that registered the customer, ACME Inc, and it's Alice who will have to log in. So we explain the behavior we want by means of concrete examples, not some vague abstract wording.

For you, my dear reader, this means that you'll be able to read and understand how the system will behave and you can comment on it, you can make adjustments. And more over, when you are the one that requested some feature, you'll be the one that can verify whether or not the behavior is documented correctly, maybe even document it yourself. It has become irrelevant who you are, as you understand what the system is doing and in which cases.

Having said this; when the user stories in JIRA state you as the stakeholder,

I as the <STAKEHOLDER> want <CERTAIN BEHAVIOR> so that I can <GET BUSINESS VALUE>

I as the Cards Operations Manager, want the last updated customer at the top of the list of customers in the dashboard so that I can continue working on that customer when I wasn't yet finished entering all data

In the sample user story above, the Cards Operations Manager will be the right person to determine what the exact behavior is that he needs and verify that the Gherkin scripts do specify that behavior.

I hope you enjoyed reading this post, and that you'll be writing a lot of Gherkin, because we tell our system: Oh Behave! 

: The swing cartoon originally referred to the problems around the waterfall method, and was a case in point for moving to the iterative product development approach. But the problem is still there, the impact is just less significant. The bottom line, as shown in the cartoon, is that everybody has their own perception of what is needed, hence we need to all work together to get the right perception because it is the combination of all the views on the product that creates the most value. We do this by describing the wanted behavior in terms of scenarios that reflect the input of everybody within dopay. Because we're all unique, and together we're AWESOME.

Architecture Principle - Everything is PorN Principle

Architecture principles are ground rules by which we develop our software. This ranges from ways of working to styles of coding, from software decomposition to business compliance, from infrastructure choices to security implementations. As you can see, it covers pretty much everything related to the development of business services, online, offline and internal.

Description

This principle assumes that in the software life-cycle different versions of a software component can exist concurrently. Traditionally this is implemented in organizations by adopting the DTAP model. A software component matures through the Development, Test, Acceptance and Production phases, where each phase has its own primary stakeholders and in many cases different SLA's, RACI matrices and ownership are assigned. In D, the developers control all facets of the component, in T no development can take place, test engineers test the component. In A, a user panel tests the correctness of the software by verifying all requirements are met and when the component is accepted it is taken into Production. This works well in environments where software components are relatively monolithic in nature, in these environments there will be more than one developer specific environments but only one T, A and P will exist.

In Service Oriented Architectures, and especially those situations where service orchestrations are the preferred means of composing applications, the DTAP model is too rigid to be usable. Within an agile environment we therefore follow the Everything is Production or Not (PorN) Principle, which means that everything is considered to be a production version, albeit that the there is a difference between the various maturity phases of the software, which manifests itself in different SLA's and different user authorizations.
In essence this means that we always deploy to a production environment, where components, services, that are still in development phase will be covered by a lower SLA than services that are already in the production phase. Additionally, those services that are in the development phase can only be accessed by 'users' that have the role 'DEV' attached to them. This is according to RBAC is everywhere.

Rationale

Within agile environments we typically assemble our applications through Service Orchestration. By 'stringing' services in a particular order we add or change new behavior of our (online) services. In reality this means that when new behavior is developed, many of the software components, services, will already exist, new components will be developed and will be in some state available to be used. As such, the new service XYZ can depend on the service implements ABC as deployed in production, and service QRS that is currently in the process of being accepted by a user panel. By dropping the DTAP model and adopting the PorN model, we ensure that new services are always developed in a scenario where either a service is being developed, or has been developed, in which case the latter scenario means that the service is of sufficient quality, Production quality.
With this setup, we can then always deploy any constellation of services in the same environment as our Production environment. The only difference between what is already production quality and what is not yet production quality is related to the services that are currently under development and those that are not. Or, in other words, the consumer of a service can be either a production quality grade component or not. Since consumers can be users, we differentiate between 'unforgiving users', for example paying customers, and 'forgiving users', for example developers or testers. Also, consumers can be other services, which either behave correctly or might behave erratic.
Thus, we have consumers that have high expectations or behave correct, resulting in a service constellation that will need to provide and can provide a particular SLA. Or we have a service constellation that has consumers that either expect little or behave badly, resulting in a service constellation that cannot provide the same SLA but doesn't have to either. Hence, it is the consumer that dictates the environment with its SLA and not the maturity of the software. Therefore we need to control which users can access a particular instance of a service.

Implication

Dropping DTAP and adopting PorN, we can no longer rely on infrastructure setup to safeguard the various maturity phases in the software development life-cycle. Instead we can do this by implementing the RBAC is everywhere principle, thus limiting the users that can access a (version of a) service that is still in development to users that have for example the role Developer. Unless the user has the role, she can't access that (particular version of that) service. Or in case the consumer is not a user but another service, that service needs to have that particular role.

In principle, it's all architecture, unless it's not... or something like that.

Welcome to the wonderful world of architecture principles. A world where everything is a has to unless you've got a good justification not to.

This is an introductory for a multi-piece on a variety of Architecture Principles that I've come across in the last 5 years while working on Cloud, SOA and POA architectures.

Architecture principles are ground rules by which we develop our software. This ranges from ways of working to styles of coding, from software decomposition to business compliance, from infrastructure choices to security implementations. As you can see, it covers pretty much everything related to the development of business services, online, offline and internal.

Architecture principles are setup such that you just comply with them, unless there's a very good reason not to, and then you don't. A good example might be that everything is hosted in the public cloud, unless for compliance reasons we need to host on-premise.

All principles should lead back to our strategic goals, business goals preferably.

As a refresher, the way architecture within many companies works is as follows: As a company we have a mission, and a vision on how to accomplish the mission we've set out. There's a strategy on how to implement that vision, which consists of strategic goals that when met, we know we've implemented our vision. Turning the whole thing into a road-map from where we are to where we want to be with milestones to be reached, and steps to be taken in order to get to the next milestone. Our strategic architecture shows us the shape we're in at the various milestones, the tactical architecture shows us where we are after each step we have to take. Architecture is therefore a journey. It's not a noun but a verb instead.

Architecture is just like a satnav, you get your bearings and determine where you are, you set the location of where you want to go and the satnav calculates the best route depending on your settings regarding toll-roads, ferries and time our distance settings. And with every turn to be taken, the satnav recalculates the best route based on the most current information regarding traffic, road works, etc.

Architecture principles tell us the rules by which we live while on our journey, there's a justification for these rules explaining why obeying that rule takes us closer to our goal(s). Traffic law if you will. There's a speed limit on the road we're traveling. Obeying the law and sticking with the speed limit will ensure a save trip to our destination, but sometimes we have a passenger that's bleeding out, if we don't hurry and get to an ER, forgetting about the speed limit, caring for the passenger only. When we attest the traffic fine associated with the ticket for speeding the judge will waive the fine as a person's life is deemed invaluable. But if we just go speeding because it's so much more fun to go 200 km/h instead of the allowed 130 km/h there's no lenience from the judge as 'fun' is not considered a proper justification. Taking into account, and taking stock off the fact that every violation, even when justified, increases our architectural debt.

Architecture principles are therefore to be considered the default action to be taken, the default direction to head for. It's a matter of "This is how we do it, unless...". There are sometimes obvious 'unlesses', as in "We use Linux, unless Linux is not an option". But in that same vain, the principle could be: "We use Linux, unless Linux is not an option, then we use Windows". In which case there is no unless, there's no excuse to not use Linux or Windows. Every principle of this kind is clearly formulated as such. When there's no room to maneuver, there's just no room to maneuver.

Lastly, regarding architecture principles, we have to understand that principles will change, they will be revisited over time. As our mission, vision, strategy or goals change, our principles may have to change as well. As our world changes, our principles will change as well. They're not carved into stone, they're written using Permanent Markers. And we all now that permanent markers on a whiteboard can be erased with whiteboard markers.

Changing our principles or violating them, results in architectural debt. And as with financial debt, there is an interest to be paid. The more debt we have, the more our software costs. This is very much like technical debt that accumulates during software development, but where the interest paid on technical debt is typically analog to the interest on a mortgage or even a personal loan with a bank, architectural debt is close to the interest a loan-shark issues. And not paying of those debts, well the analogy continues there as well.

Without any further ado, let's start our journey...

Iwan