August 17, 2016

Oh Behave! How JIRA turned into a Veggie Salad

Previous I posted about Why Uber is so applicable in the agile delivery of softwarer. It is only part of the story about how agile delivery is doing stuff differently from many other development practices.
Provided you have access to the JIRA and Confluence sites of one of the companies I worked for recently, which is the FinTech startup dopay, and you're following the various tickets on JIRA and read the relevant wiki site every now and then, you will come across terminology like Cucumber and Gherkin. Probably you've been wondering when you would come across Tomato and Lettuce as well. Chances are you will, because JIRA is turning into a Veggie Salad. This is, by the way, a trend you'll come across throughout the agile world.
All jokes aside, what's all this vegetarian mumbo-jumbo about.
Cucumber is a tool in use at many agile teams to verify that the developers build the right swing. Swing? Yes, swing, agile development is all about building the right swing (star).
You'll hear me refer often referring to the swing, and most of the times proclaiming that I want to make sure we create the right swing. The point in building the right swing is making sure that what ever is being build is behaving in the right way, is delivered at the right time and costs just about the right amount. Getting all three right is extremely tricky, and pretty close to impossible. So we go for the system that behaves the right way first and get as close as possible to our two other goals, making it on time for a decent price.
I'm using the term 'behave' deliberately. We don't want a system that has the right functionality but isn't usable, instead we want a system that is usable and as such, it will deliver the right functionality. There's a small difference in wording which is crucial. Here's an example:
Say you need to enter the details of an employee for one of the customers in a HR services system. Now the system would provide the function to enter employee information in a nice slick wizard interface, taking you through the whole process and when you're on page 6 and press the submit button all details are stored in the database, which is when you find out that the format of the mobile number of the employee had to be international. Good thing the faulty data wasn't stored, because it would corrupt the database. And now you can enter all data again, as the helpful system did a reset of the screens so no corrupt data can flood into the database and you're getting a second chance to enter the data correctly.
The functionality is exactly as it should be, employee details can be entered, the system validates all data and ensures that only correct data is entering the database. Awesome. Or not, as in fact this is what we want, yet is not really what we wanted. What we wanted, or what what I would've wanted, is that the system would have behaved in such a way that it would have me entering the data and as I go from field to field it would validate the information entered and inform me straight away that the data was incorrect and let me fix it without resetting the fields already entered correctly. It's the same functionality I'm getting of the system, but the behavior is what mattered. And I hope you saw the way I expressed that functionality in terms of what I want the system to do, and not to provide.

I want behavior.

I want behavior? Yeah, and good behavior, I want this from my kids as well as from the systems I use. Mind that I want my kids to get good grades in school, but not by cheating, but by doing their best.
We describe the behavior of our systems in such a way that we can not only document the behavior we expect from our systems, but we can test that it really behaves that way. The documentation is done using the language Gherkin and the testing is done using the tool Cucumber. And no, these are not the tools for the developers, these are the tools for the entire enterprise. And I can hear the screaming starting;
 Ohhh no, now he's asking me to start programming in some weird programming language,  AAAARGH!!!

Actually, you couldn't be more right. I do want everybody to start programming, and use that weird language Gherkin to do so. But unlike those obscure programming languages that are used by those developer types you see wandering around in office, Gherkin is really simple to be used. In fact, you've already been programming in Gherkin for the larger part of your life. Gherkin is just plain English (or any of about 40 other languages for that matter). Gherkin is really easy to write, read and well, read and write. But first let me show you a little piece of behavior description that is more formal than a typical requirement: 
Given that our sales agent Johnny Dough already registered ACME Inc as a customer, when Alice is accessing her Sales Management Dashboard to access the latest sales reports, then she will need to log into the customer portal first. 
I hope the line above makes sense and it is perfectly clear how we want the system to behave when a user wants to access her data. And guess what, that was all perfect Gherkin. Yup, Gherkin is human readable and not just developer readable. The cool part, this is also 100% the test script that will be executed to test the system.
Yes, you read it right. While you document the features of your system, i.e. the way you want your system to behave, you create at the same time the test scripts to verify that the systems really behaves that way. No more excuse that the wrong swing was build. And oh, there's another little interesting piece that you might have guessed from that little Gherkin script, it has an actual example of real life data in it. It's Johnny Dough that registered the customer, ACME Inc, and it's Alice who will have to log in. So we explain the behavior we want by means of concrete examples, not some vague abstract wording.
For you, my dear reader, this means that you'll be able to read and understand how the system will behave and you can comment on it, you can make adjustments. And more over, when you are the one that requested some feature, you'll be the one that can verify whether or not the behavior is documented correctly, maybe even document it yourself. It has become irrelevant who you are, as you understand what the system is doing and in which cases.
Having said this; when the user stories in JIRA state you as the stakeholder,

I as the Cards Operations Manager, want the last updated customer at the top of the list of customers in the dashboard so that I can continue working on that customer when I wasn't yet finished entering all data

In the sample user story above, the Cards Operations Manager will be the right person to determine what the exact behavior is that he needs and verify that the Gherkin scripts do specify that behavior.
I hope you enjoyed reading this post, and that you'll be writing a lot of Gherkin, because we tell our system: Oh Behave! 
(star): The swing cartoon originally referred to the problems around the waterfall method, and was a case in point for moving to the iterative product development approach. But the problem is still there, the impact is just less significant. The bottom line, as shown in the cartoon, is that everybody has their own perception of what is needed, hence we need to all work together to get the right perception because it is the combination of all the views on the product that creates the most value. We do this by describing the wanted behavior in terms of scenarios that reflect the input of everybody within dopay. Because we're all unique, and together we're AWESOME.

August 15, 2016

Architecture Principle - Everything is PorN Principle

Architecture principles are ground rules by which we develop our software. This ranges from ways of working to styles of coding, from software decomposition to business compliance, from infrastructure choices to security implementations. As you can see, it covers pretty much everything related to the development of business services, online, offline and internal.


This principle assumes that in the software life-cycle different versions of a software component can exist concurrently. Traditionally this is implemented in organizations by adopting the DTAP model. A software component matures through the Development, Test, Acceptance and Production phases, where each phase has its own primary stakeholders and in many cases different SLA's, RACI matrices and ownership are assigned. In D, the developers control all facets of the component, in T no development can take place, test engineers test the component. In A, a user panel tests the correctness of the software by verifying all requirements are met and when the component is accepted it is taken into Production. This works well in environments where software components are relatively monolithic in nature, in these environments there will be more than one developer specific environments but only one T, A and P will exist.
In Service Oriented Architectures, and especially those situations where service orchestrations are the preferred means of composing applications, the DTAP model is too rigid to be usable. Within an agile environment we therefore follow the Everything is Production or Not (PorN) Principle, which means that everything is considered to be a production version, albeit that the there is a difference between the various maturity phases of the software, which manifests itself in different SLA's and different user authorizations.
In essence this means that we always deploy to a production environment, where components, services, that are still in development phase will be covered by a lower SLA than services that are already in the production phase. Additionally, those services that are in the development phase can only be accessed by 'users' that have the role 'DEV' attached to them. This is according to RBAC is everywhere.


Within agile environments we typically assemble our applications through Service Orchestration. By 'stringing' services in a particular order we add or change new behavior of our (online) services. In reality this means that when new behavior is developed, many of the software components, services, will already exist, new components will be developed and will be in some state available to be used. As such, the new service XYZ can depend on the service implements ABC as deployed in production, and service QRS that is currently in the process of being accepted by a user panel. By dropping the DTAP model and adopting the PorN model, we ensure that new services are always developed in a scenario where either a service is being developed, or has been developed, in which case the latter scenario means that the service is of sufficient quality, Production quality.
With this setup, we can then always deploy any constellation of services in the same environment as our Production environment. The only difference between what is already production quality and what is not yet production quality is related to the services that are currently under development and those that are not. Or, in other words, the consumer of a service can be either a production quality grade component or not. Since consumers can be users, we differentiate between 'unforgiving users', for example paying customers, and 'forgiving users', for example developers or testers. Also, consumers can be other services, which either behave correctly or might behave erratic.
Thus, we have consumers that have high expectations or behave correct, resulting in a service constellation that will need to provide and can provide a particular SLA. Or we have a service constellation that has consumers that either expect little or behave badly, resulting in a service constellation that cannot provide the same SLA but doesn't have to either. Hence, it is the consumer that dictates the environment with its SLA and not the maturity of the software. Therefore we need to control which users can access a particular instance of a service.


Dropping DTAP and adopting PorN, we can no longer rely on infrastructure setup to safeguard the various maturity phases in the software development life-cycle. Instead we can do this by implementing the RBAC is everywhere principle, thus limiting the users that can access a (version of a) service that is still in development to users that have for example the role Developer. Unless the user has the role, she can't access that (particular version of that) service. Or in case the consumer is not a user but another service, that service needs to have that particular role.