Translate

November 17, 2017

In depth security... what when the tables are turned?

With many new initiatives at governments and organisations in the area of security, I see that tables are being turned... and initiatives are lagging. Again.


Summarising

We all do our best keeping the baddies out of our systems, away of our data. But what if the baddies try to do the same? Trying to keep us out of our systems, away from our data? There's a new threat in town, it's our own systems.

Lately I have been talking a lot with one of the Information Risk Managers of one of my clients. Let's call him Frank, it makes writing this post a lot simpler.
So Frank is a different kind of Information Risk Manager. He's very much involved and active in product development. Frank's not policing, he's establishing policies in stead. Mind that I'm specifically not saying that he's writing policies, instead he's establishing them. Together with those around him that are supposed to live according to these policies, he is ensuring that every step of the way, they're involved. It's pretty cool. Then again, Frank is pretty cool.

Frank and I have been doing a couple of security awareness sessions at the client's site. These sessions revolve around Secure Software Development and the intention of these sessions is making sure that those that are developing the products know that they need to build in security from the start. Yes, we are only looking at IT peeps, even though security is obviously a business concern. Well, one step at a time.

I take it that you're well aware of OWASP's top 10. And likely you've at least heard somebody mentioning the GDPA, General Data Protection Act. And considering you're reading this post, you haven't lived under a rock in the past couple of years, so data breaches like the one at Equifax or Yahoo for that matter should be known facts by you.
I love it when these things are being discussed. Stolen passwords, accounts hacked, social engineering to get into systems. Examples galore to justify all those millions of Euro's and Dollars spend on security. Implementing all kinds of programs at enterprises and organisations. At one client I actually told the ISO (Information Security Officer) that his presentation on the topic leaned towards plain old terrorism considering how she was scaring the CFO to approve her budget request.
Well Frank is not like that. Instead Frank is explaining why awareness is so important. He's addressing it from a business benefit to do in-depth security, privacy by design and so on. I actually had to talk firmly to Frank and have a slide in his deck to convey the amount of vulnerabilities found in our systems in the past couple of months. Just to show that not doing anything is not an option. Reluctantly Frank complied.

At this client, we do threat modeling. We do Privacy Impact Analysis, Business Impact Analysis, etc. We analyse a lot. And we're moving into a way of doing all this such that the developers can keep on becoming more and more agile. Sprint based, epic-by-epic, one story at a time. Ensuring that nobody gets into the systems, privacy sensitive information is encrypted and transaction integrity is ensured.
Nobody gets in, and who's in can't see any data and those that can, won't be able to change is. It's awesome, it's secure. And it's lagging behind today's and more importantly tomorrow's threats.

We're spending so much time to lock things down, in depth, on all levels, by design. Making sure that there're no backdoors, systems are patched. But what about the threat of being locked out yourself?

Data hostage is the new trend. With the advent of biometrics, access to systems by certain individuals becomes more prevalent. Unless it's your iris, you won't get into the server room. Unless it's your fingerprint, you won't be able to unlock the device. With face recognition systems build into operating systems like Windows 10, iOS and Android you only have to look at your screen to be granted access to all that data.

Frank knows about all this stuff as well. Together with a couple of really smart developers he's working on utilizing this new tech to make security more convenient for employees that need to access on a regular basis, sensitive information from their devices. Multi-factor, with biometrics as a second factor is around the corner. It'll be all about what you know (PIN or password) and who the system recognises.

This makes it more and more interesting for cyber criminals to no longer try to get access to your data and instead prevent you to get access to your data. When we rely heavily on the uniqueness of our bodies. By storing precise information about a person's iris, face-metics, fingerprints and in the (near) future a person's DNA, we can ensure that people are uniquely identified. But what is going to happen when that data is compromised? 'Access Denied'.
In today's systems where identification is based on what you know and what you, we are nog focussing on corruption of passwords, PINs or authenticator apps. We're focussing on people not getting in, but we forget about people keeping us out. And when you can't get in, your systems are held hostage. Not only your data, which is typically the target of today's Ransom-ware.

We need to think of new architectures that also consider this aspect of security. It's the availability aspect of security. Data and systems need to be available, but consequently also accessible. Availability is typically associated with redundancy and clustering. Cloning and mirroring. On a more functional level it is associated with the GDPA views on a person's right to always access her data and the right to be forgotten. But what if access is denied? What if you're locked out?

Another interesting feat today is that once security is breached, users need to be informed. But what about the reverse. Your security wasn't breach in the sense that people gained access, but instead people removed your access?

This is new territory for many of us and I don't have a suitable solution for these kind of threats. What I do know is that you'll need to address this from your architecture. And you'll need to involve your Information Risk Manager from day one. And make her tag along full time. Constantly. Just like I talk with Frank on a steady basis. Sharing thoughts, discuss the philosophy of security and challenge each other on everything but mainly on how we can make everybody involved in the process of product development aware of these at times very interesting nuggets of architecture.


Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

October 20, 2017

What's the story about those other user stories?

User stories expressing what the Product Owner or her Team want or need are bad practice. The PO delivers, never requests!


Summarising

Backlogs containing user stories that are catering for the needs and wishes of a Product Owner or Product Team are faulty. These kind of stories are unable to compete with stories that clearly add value to the business. By putting these stories on the PO's backlog compromise the ability of the PO to be successful. Instead these stories should be listed on a second backlog owned by the Team. This will improve the team's sense of autonomy, the Team's relationship with the PO.

In the past 6 months or so I've been actively working with agile teams on their backlogs. Specifically the Product Owners. As you might already know, I consider the Product Owner role as one of the most demanding and challenging roles within an agile environment. You might want to read my post on it, which you can find here. One of the major challenges I'm facing in working on backlogs is getting the Product Owner, and also the Product Team to understand why internal user stories are a bad-practice.

Internal user stories are those stories the PO wants, or the team itself. Most of the time, these are stories where an upgrade of some technology is needed, where technical debt is taken care of, or for example where the Product Owner wants some reporting from the tooling. There're any number of stories on backlogs where it is the Team or the PO in the 'I as a ...' clause of the story. Look at your own backlog and see for yourself.

You might have read my post "Perish or Survive, or being Efficient vs being Effective" on what agile encompasses in terms of motivation, or drive. When you commit to agile practices, you commit to effectiveness and in pretty much all cases you commit to a value-driven approach towards what has the highest priority.

Considering the above, than you realize that in most cases, internal user stories are not really creating value. Mind that you will have to consider the story and not the motivation for the story. Be pure in thought.
This is because of the simpel fact that the user, or customer requires and the PO (and her Team) delivers. That's all there is to it.

All of a sudden it is clear why a story like "We, the Product Team, want to upgrade to the next version of technology such and so, in order to reduce our technical debt" is a no-no on the backlog. Why? Because the user, nor the customer typically doesn't care and shouldn't care about the technology used.
Don't believe me? Well, have the user dictate the version of Ruby on Rails to be used in the development of an iPhone app and listen to what the Team has to say about this. I rest my case.
Having said that, it should also be clear that there is typically a more than valid reason why a Team wants to get rid of technical debt, even suggesting how to get rid of it as well. And in most cases it is related to the request of the user, demanding better stability, performance or security. The Team's story is an implementation aspect and nothing more.

Think about this a bit and then you're bound to see that when the PO or the Team will create their own stories and put them on the backlog, they will need to compete with the stories of the other stakeholders. Stories that add value, for the simpel reason that the stakeholders will ensure a clear value-add because that is what gets the story at the top of the backlog. (Provided the PO will actually prioritize the backlog based on business value and not things like vanity or volume of a voice.)
The last thing anyone should want is have the team's stories be relegated to the black-hole called "Tail-of-the-Backlog".

One of the most common ways of getting the things done, the Team wants done is by allowing the Team to spend some time on other things than stories. Sort of a budget expressed in time, the Team can spend on their own stories. I like to maintain the 80-20 rule here, as it can be applied in pretty much everything related to IT. So 80% of the time is spend on creating value, and 20% on something that is not necessarily adding value. How that 20% is allotted is up to the PO and her Team. Could be a day a week, could be two days during a two week sprint, or possible a full sprint once every five sprints.

I call it budget, because the PO's budget is expressed in time. She has a team available to her, a stable team paid for, for the lifespan of the Product she's owning. Consequently all she has is time to spend. Allowing the Team to work on their own stories, makes perfect sense. This is because the Team itself knows best on what stories are the most valuable. Whether it's the technology upgrade, the refactoring of old code or investigating some new concepts and their applicability.
Once the Team maintains its own backlog of its own stories, there is no need to compete with the stories of other stakeholders, the stories bound more clearly add stories. Moreover, the Team will more so be empowered to autonomously decide how stories are realized. Deciding when it is best to cut corners and when not. All of a sudden, the Team is not only responsible for the product's quality, but also accountable for the technical quality of the product.

The PO's backlog is one expressed in value creation, with the highest priority set to the story (potentially) creating the most value. Allowing the PO to be value-driven. The PO therefore can maintain a backlog consisting of user stories with clear stakeholders. People, not roles, that can be called upon to verify the product's ability to deliver value. And with the opportunity to ask these people beforehand how value is created, what metrics are needed to validate the product's benefits, etc. And of course, the story will contain the name(s) of the people to explicitly invite to the Team's demo of the product.
The Team's backlog at the same time can be prioritized on the account of what will reduce the product's costs the most. Accommodate for future developments as well as operational aspects, etc.

Maintaining both backlogs also strengthens the relationship between the PO and the Team, putting the emphasis on trust, accountability, facilitation and empowerment. The corner stones of agile teams.



Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

October 9, 2017

ASAS2017 in hindsight

ASAS2017 in hindsight

A couple of days ago, Thursday October 5th to be exact, I attended ASAS2017. It was my fifth attendance of the Agile Software Architecture Symposium.

Summarising

This year, as part of keeping a promise, I did a presentation at ASAS 2017 about Agile and Architecting. Adapting the presentation up to the last minute based on feedback I garnered while attending other presentations, I felt really agile.
One of the key take aways for me was that doing a Kahoot! to know the audience instead of raising hands makes a lot of sense when you want to know who your audience was. But you need to keep those stats. And it's almost imperative to have a clear means of being contacted shown on a slide. At the end.

A couple of days ago, Thursday October 5th to be exact, I attended ASAS2017. It was my fifth attendance of the Agile Software Architecture Symposium. This time around I was one of the speakers. It was a promise I made last year to the organizers. The promise was that if 2017 would have an ASAS, I would submit a proposal for a talk and if it would happen as such, I would be one of the speakers at ASAS 2017. Promises are there to keep, so I kept my promise.

Funny part, although at the time I really didn't think it funny, is that just days before my family and I would drive to Croatia for our summer vacation I checked when in November ASAS would be this year. Yup, I was mistaken by a month and the date was instead of somewhere in November, some time early October. The 5th to be exact. And apart from my submission in early April this year, I had nothing prepared at that time.

The topic of my talk would be something about architects being able to survive in an agile world. Something I am talking about almost on a daily basis with clients and especially their architects and agile coaches. But since April I had changed my story continuously... based on the feedback I got during those discussions. Understanding what works and what doesn't is important and crucial when you're coaching. So it was more or less imperative that the contents of the talk would be significantly different when compared to my initial view on the matter in March and the presentation I would give on the 5th of October.
Just to give you a hint, this is what I proposed in April:

How architects can be successfully agile.

Summary: Architects are known for their talent to ensure that projects run out of budget or out of time. The best of architects are successful in ensuring that projects run out of both time and budget. But with the advent of agile practices and the agile manifest, architects are no longer needed, they have no longer a role in continuously postponing a project and ensuring that tons of money are wasted. With waterfall becoming water under the bridge, there is no longer a need for procrastination by the architect. Unfortunately, as evolution goes, architects are genetically inspired to build ivory towers. Doomed to follow the path of the dinosaur. Unless... unless they manage to become agile. This session is about how architects can become agile and thrive again, gaining the respect of... well pretty much everybody and their mother.

When I set off on this journey, I figured I would talk about architects and how they need to change their attitude and way of working. But over the course of the months since April I realized more and more that it's not the architect that needs to change, so I adapted and changed my presentation.
In fact I changed my presentation even 10 minutes before I had to go on stage and give it. Truth be said that in those final minutes the changes weren't significant, yet they were adaptations to feedback I got while attending other presentations that day.

I started of with a Kahoot! to get to know my audience. Instant feedback in terms of laughs meant to me that it was a good way to break the ice... but since I forgot to save the results and in the first place to test the quiz myself, meant that I don't have the right metrics to judge the appropriateness of my slides.

I ended with the opportunity for my audience to ask questions. And that was when the room stayed quiet. Silence. No questions. Not a captivating story that was raising questions. Until a question was asked. By one of the Arc-E-Tect blog followers. There was the answer, and that's when the questions came. And people stayed afterwards, wanting to know more. And then there was the after-party and more people wanted to discuss the presentation.

So in hindsight, the presentation went well. It went well considering the metrics I had set for myself up front. How many people would be in the room, how many would leave during the presentation, how many would join the Kahoot and how many questions would be asked during the presentation. And finally how many would want to talk about the topic after the presentation was done and they had ample opportunity to talk about other things to other people. In all cases it was beyond expectations.
Considering my expectations; the Kahoot! was about getting around 75% of the audience to join, which I think was achieved. I don't have numbers about the audience. Considering people leaving the room during the presentation, I only saw 2 persons leave in a room of about 50 people. Considering questions asked, 3 questions was for me the minimum, based on the fact that I thought I would have time to spare for questions and the story was compelling. 4 questions were asked with 4 different persons asking more questions.

All in all I feel that my presentation at ASAS 2017 was a success, also as an experiment to see if you can consider a presentation to be a product that adds value and can be done in an agile way. 

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

August 4, 2017

The "Eat your own dog-food" fallacy

Why having people eat their own dog-food doesn't accomplish anything sustainable.


Summarising

Having somebody eat their own dog-food doesn't necessarily make the improve its taste but might make them get used to the taste instead. Awareness of the (lack off) quality in one's work is important when transitioning from a Dev/Ops organisation towards a DevOps organisation. But experiencing a lack of quality first hand is often not the way to do so, or even an option. Agile transformations are only possible by means of cooperation.

There's a common understanding in the world of organisations that are transforming from Dev/Ops towards DevOps organisation that this works best when the devs are forced to eat their own dog-food. It's a reaction from the Ops people towards the Dev people.

Basically it means that once you've got the Devs supporting their own products, they'll make sure that those products are of a high quality. Common believe is that since they, those Devs, don't want to get called Sunday night at 3 AM because their software crashed. Which makes perfect sense, who does want to get called Sunday night at 3 AM because something they created crashed? I wouldn't, would you?

So, if you want those Devs to focus more on quality, on less crashing software, you have to make them support that same software themselves. In other words, turn those Devs into Ops people and that'll show them.

About a year ago, I joined a team at one of my customers to help them transform selected development teams into more agile teams utilising Continuous Delivery mechanics and move towards, lord forbid, DevOps. One of the slogans we used to get the necessary buy-in was:

"Eat Your Own Dog-Food"

And later we added "And Clean Your Own Shit!". Totally convinced that this is how it works. Make people feel the pain they're causing and they'll become better persons. If you would do a little time-travel and rewind about 2 years, you'll hear me saying that "one should feel the pain they're inflicting".

I think you'll recognise this when you've been in that situation where you want to move from Dev/Ops towards DevOps. Or become more agile. Or maybe you need to convince people that agile or DevOps is the way to go.

It makes total sense. People with kids know this. You want them to stay away from the fire, let them burn their fingers. Or those people with dogs shitting all over the place... let them clean that shit every time their dogs drop their poop in the playground. It works, really does.

But there's a huge problem in this, I'll get to that. First I want to ask you if you've noticed that slightly grumpy undertone in me mentioning of the "Eat your own dog-food" slogan and everything associated with it.

Agile and SCRUM in particular are developer driven ways of working. It's the developers that want to change things. Reason, obviously, is that developers want to develop software that people are using, so they want to get create something that is as close to what is usable as possible. Meaning that they want so put something in the hands of the user as quickly as possible and then make adjustments and put that into those same hands. And again. And again. And again.
Our organisations are such that specialised Ops people are managing those applications when in the hands of those users. And they need to keep up with all those adjustments... You understand the predicament those Ops people are in.
So when you tell these Ops people, that you want on your side while transforming into agile organisations that those Devs should eat their own dog-food. Ever tasted dog-food? So how do you think that this resonates with those Ops people? Pretty awesome, don't you think?
That connotation of dog-food is getting the nay-sayers called Ops on your hand, shouting "Yay" instead of "Nay". Mission accomplished. You're done.

Well forget it. That's the "Eat your own dog-food" fallacy. It's a fallacy because it doesn't solve anything and it certainly won't help you in your agile transformations. Considering your organisation has separate Dev and Ops teams, and considering that the reason for this is a more efficient Ops team because they will support many products. Because supporting a quality product is not a full-time job. Read my post on this topic. There's no way that having the Devs eat their own dog food will improve quality. Which was the premise in the first place, remember? And now you should say that in fact it doesn't make sense at all. Because there's an Ops team and for a good reason. So what makes you think that anybody in the organisation will just allow the Devs take over the Ops jobs? Ain't gonna happen. No way. Meaning that whatever you're trying, the Devs won't even get the opportunity to eat their own dog-food, even if they would want to. You can fix it by job-rotation... in certain situations, in certain organisations. Fallacy explained.

Considering the above, how would you need to address the agile transformation? How to move towards a DevOps setting? The answer is quite simple and probably extremely hard to implement. The more alluring the "eat your own dog-food" approach is, the harder it will be to do the correct thing. The sustainable approach, let's call it.

If you want the developers to develop software of a higher quality, you need to make them aware of the problems they are causing because of the lack of quality in the software. And you do this, by introducing them to their operations colleagues. Let them work closely together. Geographically close. As in side by side. Not pair programming close, but across the desk close.
What will happen is that the colleague from Ops will complain to his Dev colleague about a problem instead of to his Ops colleague. Feedback loop is tiny and closed. And most likely, it's friendly constructive feedback, because it's directed to the immediate colleague from across the desk. That person with whom lunch is shared. Not bottled up feedback. It becomes a feedback loop in which the Dev can immediately ask the Ops person why it's such a huge problem, this tiny thingy.
Getting the Dev and the Ops to work at the same desk, allows them to become aware of each other's work. And generates understanding. Understanding towards their respective worlds. It creates an atmosphere where the Dev will improve the quality of the product, because otherwise the Ops colleague is called Sunday morning 3 AM, and that's not something the Dev wants.

As a parting gift, another tip: Make sure that Devs and Ops don't huddle together when you put them in the same room. Instead put the Dev next to the Ops, side by side, hand in hand. When you allow them to huddle together, you should put them in separate rooms. Just to make sure that their respective complaining is not effecting them during work hours.
By allowing those to blood-types in your organisation to become aware of each other and befriend each other, you'll pave the way to become a true agile organisation with a smooth transition towards a DevOps mindset.

Here's an exercise for you: See how it works the other way around. Feel free to use the comments to discuss.

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

July 6, 2017

You know you're the Product Owner...

...when your product's users are complaining and you have to worry about your bonus.


Summarising

The Product Owner's bonus is on the line when users complain! You're not worried about the upcoming appraisals although the users are complaining? And you're not looking for that boat you fancied for so long because the users are giving you great reviews all the time? You're not a Product Owner.

In this day and age of DevOps and Agile, the most coveted job in the world seems to be the one of Product Owner. Never have I seen so many co-workers turn into a PO as recently is the case. Operators turning into Operations PO, testers turning into the Testing PO, security experts fulfilling the role of Security PO. It's amazing to see all these Product Owners mushrooming in organisations.
Understandably, because their original jobs are nearing their expiration date, or so they're let to believe.

The other day I had an interesting discussion with one of the architects of a client of mine. We're discussing a lot these days about architecture, API's, services, data warehouses and other interesting stuff. But this time around he challenged me. Seriously.
This particular client is a typical Project oriented organisation. Projects develop something and once it goes into production and becomes at times business critical, a very efficient department takes over. (Just in case you're wondering why I put efficient in italics, read this post). This architect is part of a department that is making the transition from a Project oriented towards a Product oriented way of working. It's a significant move and absolutely not trivial.
What's interesting is that the general understanding of the necessity of a mandated Product Owner has caught on with this client of mine. What hasn't caught on is that the PO is supposed to be somebody from the business. Take this with a tiny grain of salt thought, as by stating that the PO needs to be a business person, I mean that the PO needs to understands the business in which his products make a difference and generate value. Do you need to be an MBA? Nope, but you do need to understand the relevance of the product for your user.
And this is exactly the issue at hand. All the different so called PO's my friend the architect is dealing with do talk with the user, but do not necessarily understand the relevance of the products they're using. The Operations PO discusses the stability of the product, which makes perfect sense because the focus of an operations person is ensuring that the product is not crashing. One could argue that the relevance of an operations person is the fact that products will crash, which is a bit ironic. The Testing PO is of course focusing on whether or not the product is conforming to the requirements and specifications. This is what testing is all about: Is what has been build, delivering what was intended in the first place. And with all the security incidents, global incidents at that. And with all the new laws and regulations around privacy and what not, the role of the Security PO is cut out, it's focusing on limiting the risks for the organisation by having the products being used by, well the users.

Since all these PO's are doing their job extremely well, the products are up to par and are in fact creating value for the organisation. They surely do. But that is not to the credit of these PO's. The reason for this, is that none of the PO's are concerned with the best product, i.e. the product that is helping the user to conduct business. They are all focused on the product delivering what was intended, namely stability, requirements and security.
I hear your brains churning on this, so let's make an assumption here to illustrate: What if the product crashes all the time, but when it doesn't it removes the hassle of manual steps in a complex process? And although it crashes, data integrity is guaranteed? The operations person won't like it and will very likely take the product out of commission. Why? Because operations is affected when stability is an issue.
So now it turns out that the product is fully according to spec, tests are 100% green but the user will not stop complaining because the product is still not helping to drive business? The tester will not look into this as a testing issue, but as a specification issue. The fact that the tests didn't reveal this major flaw, namely usability. And even when it did, usability being a testable requirement is a novelty. It is with my client's organisation and it is in many others.
Guess you can fill in the problem with the Security officer acting as PO. Consider it a small exercise to flood your brain with some endorphin.

The issue here is that none of these so called PO's is accountable for the success (or failure) of the product. None of them is. And this is what sets the PO apart from everybody else in the organisation:

The PO's bonus is on the line when users complain!

This means that when a accountability of a product's success, i.e. the level if complaints from users about the product, is not with you, you're not the Product Owner. It also means that unless you get a full mandate to make a success out of a product, you're not the Product Owner either.
Don't accept the role of PO unless you get full mandate, which includes discretionary say about the product team, its road-map, funding, etc.

Back to our wannabe PO's, because that's the correct word for them. Their bonus is not on the line, they're not responsible for the product's success. They're definitely not accountable. But that doesn't mean that they're not responsible for making the product a success. Their knowledge, insights, experience and general professional view on the product is invaluable input to the PO to create a success out of a product. The PO shouldn't ask for their input, but when the input is not provided, the questions should be raised. They're not impacted by bad reviews, the PO is. They do have to worry about their jobs, because if the PO can't use them...


Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

June 29, 2017

Perish or Survive, or being Efficient vs being Effective



Summarising

In IT we are not dealing with commodities, although it may seem to be that way, it isn't. Software development is a matter of engineering and not producing. Hence efficiency is not the focus you want to have as a business, instead you want to be more effective. Effectiveness is what is required to be adaptable to a market where changing one solution for another is becoming increasingly trivial. Open standards and the democratisation of IT resources because of the cloud ensure users that the risk of vendor lock-in is negligible. This requires an organisation to be able to adapt to the wishes and needs of its users, not being able to churn out loads and loads of software. Therefore in order not to perish in today's world, effectiveness is needed not efficiency. To thrive in such a world you'll need to be efficient at being effective.

Over the past couple of weeks I had some discussions with a colleague of mine. He's an architect as well and we're in similar situations where we are asked to coach teams and organisations to transition from a traditional setup into an agile setup.

Last week or I was asked by this colleague if I could co-review a report one of his clients wrote that was all about a transition from a legacy waterfall organised project into an agile project. What struck me, and fortunately my colleague concurred, is that the main motivation for this transition was to become a more efficient organisation. Which in fact is an ill-chosen motive.


Let's back-up a bit and consider two similar words that are fundamentally different in meaning: Efficient vs Effective. Traditionally, in process engineering we're striving to become more efficient. The whole idea is that by becoming more efficient, you can produce more and hence benefit from economies of scale and the likes. It's a process improvement adagio that's been around since long. It is also a motive for improvement that leads to silo's, specialised silo's. And here you already see the first sign of why efficiency is wrong when it comes to agile methods. In an agile world we want to get rid of silo's not create them.
So where's the effectiveness coming into play? Well, that's actually rather evident. In order to be agile, you need to be able to turn on a dime at a moment's notice. Which means that whatever you do, you need to be very effective when you do it.

The point here is, that Efficiency focuses on minimising cost by spending as little as possible on the creation of a product on a per product basis. By doing so, the cost of the product reduces and the profit margin per product increases. Typically this is achieved by leveraging specific capacity for specialised tasks. Effectiveness on the other hand focuses on maximising revenue, by spending as much time on value creation by doing what is needed. By doing so, the costs of the product increases but the relevance of the product for the consumer and therefore its value increases more and this has a positive effect on profit. Typically communication lines between dependent parties in a process are shortened by introducing multi-disciplinary teams.

It makes sense to focus on efficiency when you need to produce large quantities of some product, and you know that there's no to hardly any need for diversification. For example when you produce nuts and matching bolts, it makes sense to produce them at the lowest cost possible. Efficiency is for growing your market share with a commodity product. Instead, when you need to grow your business by growing your market instead of your market share. Or where your product is anything but a commodity, efficiency is killing. You'll perish, eventually.

Considering you're in IT, that's most likely why you're reading my blog, your product is anything but a commodity, even when it's a commodity. And growth, especially sustainable growth, is accomplished by growing your market, not your market share. So drop the urge to be more efficient and become more effective.

Point is that you need to be able to adapt to your market. Your user, not even your customer, will initially not have a clue what she needs. Hey, that's why you've adopted agile principles. But once she is up to speed on what her demands are, she'll be more and more demanding. Hence you need to be able to adapt, continuously. And no, it's not adaptation in the IT department either, but your business needs to be able to adapt. And there's the catch, or rather your answer. Because by becoming more efficient in your production line, i.e. your IT department, your business will become less agile. This is because you've optimised the production process and software development is an engineering process. And before you ask, software development is a case of engineering and not producing. That, by the way, is the reason why off-shoring and out-sourcing is so cumbersome.

So you want to be able to adapt your product, you being the Product Owner, as the one being accountable for the company's profit (or loss). Or at least partially. So you want to be able to adapt your product, so it complies with the wishes and definitely the needs of your users. This requires a team that's effective, not a team that's efficient. Meaning that you want a team that can do pretty much everything needed to adapt the product autonomously. Not a several teams that can do specific jobs very efficiently.

This is why you need to focus on effectiveness instead of efficiency when you want to make the move to agile. And I'm convinced that you need to make the move to agile ways in order to survive and not to perish in this world that is changing faster every day. Organisations that are lean, nimble and agile are the ones that will survive in the long run, where the length of long is becoming shorter every day.

So where does this leave the architect in all of this? At the centre of agility. The architect is the one that is perfectly positioned to define what kind of competencies, qualities and personalities are needed to make a team into an effective team. The architect is also the person that is in a position to ensure that a product is adaptable. A product's adaptability and therefore a business' agility is determined by its architecture and the product team's perfectly equipped to make it so. More importantly though, the architect is in a rather unique position to not only ensure that product teams are effective and business becomes agile, but also be very efficient at this. Only when you architecture is in order and your team is effective will you be ready to improve on your efficiency, allowing you to not only survive but actually thrive.


Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contact-list. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

June 12, 2017

Microservices on steroids, getting from just agile to business agile [part 2 of 2]

Summary

This is how you get Microservices on steroids; apply them in a Policy Oriented Architecture. Meaning that you need to put a policy handler in between a service's interface and its implementations. The reason why you want to do this, is because you want to provide agility to your business and not just have an IT department that's agile. Or in fact have an IT department at all for that matter.

It’s time for part 2 of a two part series on Microservices on steroids. I already had part of the post ready but today I was in a presentation on the topic of Continuous Delivery and Architecture and one of the aspects that came up was about ‘Product definition’ and I figured that I needed to put that into the mix as well.

The conclusion of the previous post (Microservices on steroids [1/2]) was that adopting a Microservices based architecture, which pretty much is an SOA the way it was meant to be, will only get you technical agility. As you can read in my post on Continuous Delivery not being something for the IT department only (The Continuous Delivery IT Team Fallacy), technical agility is only part of the story and in fact has limited value if you’re not targeting business agility.

But let’s first consider what Business Agility exactly means. It is in fact exactly what it literally means, agility of the business. The main difference with the ‘normal’ agility is that it allows the business to change its course rapidly without a direct need of involving IT to make this change. So without needing to worry about whether or not IT can handle these changes.
And it allows the business to respond to changes in its marketplace when and where necessary with the help of IT. So business agility is an extension of technical or IT agility.

An example of business agility is for example bank XYZ that issues debit cards to its customers age 16 and up. One of its competitors, bank ABC, is in a marketing campaign targeting teenagers and issues debit cards to 12 to 16 year olds when a letter of consent is signed by their parents or legal guardians. Our bank XYZ will lose out on a large customer segment when not also addressing these teenagers. So it will need to issue cards to them as well. Business agility is when bank XYZ can do so, without major activities that need to take place in order to change course. Think about being able to make this business change in 1 two-week sprint.

From experience I can say that an agile business requires an agile IT that has its architecture on track. An SOA, preferably based on Microservices, is the best basis for this. If you ask me, I would stay away from centralized components like single ESB installations for an organization, shared database clusters or API management solutions that are centrally governed. It’s not so much a matter of me disliking these technologies, which I really don’t. It’s about the centralized governance issue, which results in shared resources, which seriously hampers independence. Centralization also means, artificial, restrictions in autonomy. In effect, it means that you reduce your ability for agility.
Now, don’t get me wrong, centralization doesn’t have to be a bad thing. In general it allows you to limit your costs by benefitting economies of scale. It’s a way of improving profitability by reducing costs. There are many situations in which this is the best way to address profitability. Especially in areas where business functionality is established and there is not really a need any more to figure out your product-market-fit, focusing on cost reduction is good. Within the same organization you also want to be able to improve profitability by increasing revenue. This is especially true for those situations or products for example where product-market-fit is a challenge or when the business scope of a product is changing or increasing still.
 
Think back a couple of posts where I explained my stance on Gartner’s Bi-model IT (The BI-Modal Misconception...). The two situations I mention are the closest thing that comes to Bi-model IT that actually makes sense. It’s where the (business) need for change and agility has diminished and functionality is stable vs. where the need for change and agility is absolutely there and business survival depends on it. There’s not a single thing that relates to risk or quality or ability. It’s about the need to be agile or not that defines the need to become agile or not. And typically, Mode 2 is revenue increase focused where Mode 1 is costs focused. Gartner’s emphasis on legacy transforming to the digital world is not relevant at all either.

Back to business agility. This is where the business is agile as established before. And like I stated earlier, this requires an architecture in which the different components are independent and can be independently deployed. Where ‘components’ are ‘products’. Independently deployable components are almost the same as Microservices. And although Microservices is a buzz word, it does have significant merit to base your architecture on Microservices in order to deliver agility to your business. It’s the perfect foundation for business agility.
Which gets me to the point of my presentation on the topic of Continuous Delivery and Architecture I mentioned earlier.
It is a common misconception that Agile and Architecture don’t play well together. Which is over course utter BS, and if you don’t believe me, I posted about this not that long ago (Product Owner and Architect, Agile Tag Team). Agile and Architecture play extremely well together as long as the Product Owner and the Architect(s) play well together. Especially when it comes to Microservices you need the domain architect or the business architect or both when you have that luxury. It is the architect’s responsibility to understand the business domain and together with the PO define what products make up that domain. Every product is a very likely candidate of either become a Microservice in its own right or becomes a composition (or constellation) of Microservices. And the Product Architect, which you might know as being the Application Architect, Project Architect or Solution Architect, is the single one person that together with the PO defines where the Product ends and the Platform on which it runs starts. The Product Architect therefore is the person that defines the (technical) boundaries of the Product and the domain architect defines the product’s place in the IT landscape. And like I said, a product is a Microservice or constellation of Microservices. Ideally, of course. Domain architect and Product architect should also work closely together in defining what API’s to consume and API’s to provide. Again together with the PO since it’s all about the PO’s product.
In this situation, we have a nice decomposition of a product or several products in interfaces and implementations. And that is exactly what we want from an SOA, especially one that is comprised of independently deployable components, services if you will. Or even Microservices.
Once all interactions between products and within products are based on interfaces, we can talk about a true SOA.

Now here’s an interesting detail. Interfaces are nothing but documentation. Be it fairly intelligent documentation or rather very usable documentation, but documentation nonetheless. And everybody that thinks an interface is something else is wrong. We don’t put an interface in an architectural layer because that doesn’t make sense. The same goes for API’s. An API is also nothing more than documentation that conforms to specific standards. Of course it’s a bit more than just a document, but really, just a little bit more. An API without an implementation is nothing and in fact you can’t actually deploy an API, nor can you an interface for that matter.

Having established that, it’s time to start the confusion and get on with it big time.

So, since an interface is nothing but documentation and it’s something you can’t really deploy. In order to provide business agility through an SOA, we need to put something in between interfaces and their implementations… But before I venture into that area, let me explain how you get business agility through an SOA.
We do this by not implementing an SOA but a POA. Where an SOA is a Service Oriented Architecture, a POA is a Policy Oriented Architecture. Basically this is an architecture in which you have services separated into an interface and one or more implementations of the interface. All implementations are available concurrently and based on policies, one of the implementations is selected. The policies are business policies and governed by the business. Business policies are a fancy word for business rules.

An example of such a business rule is that when transferring money between a customer’s own accounts there is no need for additional validation of the transaction once the customer is signed into her internet banking. Another rule is that when transferring money from a customer’s own account to an account of another customer, an additional signature is needed. Both are financial transactions, but based on the context, different business rules apply.
Policies are context driven, which amounts to the notion of under what circumstances a service is consumed as that determines the implementation of the interface to be used. Contexts can be everything imaginable. Within the example of the transaction as mentioned above, it can be the amount of the transaction, the age of the customer, the time of day, the geographical location of either customer in the transaction, the type of accounts, the client device used and the way a customer was authenticated. Or any combination of these contextual parameters or something completely different altogether.
In a POA, there is a clear distinction between interfaces, policies and implementations. The policy sits between the interface and the implementation. And it is business definable, meaning that the business rules can be changed without changing the interface and existing implementations. Either resulting in implementing policy compliance, i.e. some software is developed that implements the functionality to comply with the policy or by implementing another implementation of the service.
Some 10 years ago, Aspect Oriented Programming (AOP) was hot in software development country and although this way of software development is more or less forgotten, it is exactly what POA (see the similarity in acronyms?) wants to establish. By injecting new code, or rather new aspects of an implementation policies can be changed.
The problem with AOP was that was not trivial to do and it was hard to debug once there was an error in the functionality. AOP is also at the class level instead of the service level, and it is far from accessible to non-developers. Still the paradigms of AOP are what POA intends to deliver. A more flexible way of changing nuances in an interface’s implementation.

As you can understand is that the role of the architect and the PO is very significant in POA. The Product Owner needs to understand where policies are to be in place from a business perspective. Not every interface needs policies, and sometimes you don’t even want to have the ability to change a policy without strict governance. The architect, and typically the Product Architect, needs to define how policies are implemented. Since there are as many ways of implementing policies as there are use cases for policies, it is important to at least within the context of a product the same mechanism is used. This is where the architect comes in.

The power of an API, or a Microservice or even a normal service, is so much more when POA aspects are applied as the Product is enabling or rather facilitating business agility. Especially when the used framework for policy definition and implementation is one that allows for doing so by a non-developer. Unfortunately there are not that many frameworks available, most solutions are based on Case Management solutions or consist of glorified Business Process or Business Rules engines. Centralized solutions that require specific expertise to operate and maintain them. Consequently, the autonomity and therefore the agility of the teams and business users are limited. This is where this post more or less started off from. Centralized solutions are limiting agility, and when you want to extend agility to the business, you should limit it in the technology.

So this is how you get Microservices on steroids; apply them in a Policy Oriented Architecture. I hope you liked this post and feel free to comment and discuss. Although the POA is not new, you don’t see it applied that often if at all. I don’t think it’s a matter of complexity or a technology issue. The challenge is typically in that business and IT need to be considered as one and not two separate departments.

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contactlist. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

June 8, 2017

‘The Continuous Delivery IT Team’ fallacy


Summarizing

Considering Continuous Delivery something for your IT department is throwing your money out the window when doing an Agile Transformation program. If you want to do so, make sure you throw it into my direction. IT is a business concern, Continuous Delivery is a business concern.

Over the past couple of months I did a series of awareness sessions on Agile, Continuous Delivery and DevOps at a large client of mine. As is rather common, also at this organization the initiative to move towards Agile ways of working, Continuous Delivery and a longing for DevOps is with the software developers. This makes perfect sense when you think of it, because it's the developers that want or rather need to make changes and the pace by which they have to deliver changes is only increasing. But I guess I don't need to tell you this.
But Continuous Delivery (or Deployment for that matter) is only possible when you don't consider it a software development thing. There really is no point in spending any time to move to Continuous Delivery when you are not planning to do it broadly. I'll get to that in a second.
During these awareness sessions, which I do with colleagues from the same team, we discuss with non-developers like risk officers, sys admins, project managers, architects, test consultants, etc. we outline what Continuous Delivery and DevOps mean within the context of my client's organization. It's a common story and I won't bore you with the details, but obviously we touch upon the benefits of small increments, feedback loops and so on. And the fact of the matter is that our story actually does sound like it's the perfect thing. And we consistently get the question: "So, really, it can't be all awesomeness, so what's the catch?". And in the rare occasions we don't get this question, we still answer it.

The biggest problem with Continuous Delivery and everything following from it, is that it is not a software development thing, or even an IT thing. When you think so and still go down that road, spare yourself the frustration and disappointment, drop me an email asking for my IBAN number and transfer the budget for your transformation project into my account. At least one person will be happy with you spending that money.
And yes, even when you think it's an IT thing, you're better of giving me that money. This is what I call:

'The Continuous Delivery IT Team' fallacy

Let me explain. First of all, let's make sure you understand what I consider Continuous Delivery. It's the process that produces a product that results in business value in order to be able to sustain the organization up to the point that it can be delivered to a user and it will be delivered to a user as soon as possible. It's not a perfect definition or even a formal definition, much because there's so much more to it. But what's important is that it defines work on a product as being done, when it is ready to be delivered to a user. The actual delivery is an explicit manual step which is decided by the Product Owner to be taken as where the rest of the process is preferably fully automated. This as opposed to Continuous Deployment, where the delivery to the user is also automated and hence triggered by the developer when committing his code to the source code repository. Again, this is a definition close enough to what it is and fitting the purpose of this post.

With Continuous Delivery and agile working in general you want to receive feedback for what you've been doing as soon as possible. And preferably you want this feedback to be such that for one you know that you've done well and secondly you're actually contributed to the bottom line of the organization you work for. This is why we like to work at the granularity of user stories and epics and delivery on a per user story or at least on a per epic the changes to a user.
As you now understand, for every single user story or at least epic, you need to do everything that needs to be done for a release, because you're delivering to a user. Somebody is going to actually use that little piece of software you've worked on with such a passion. Fact may be that the complexity is limited because increments are small, still you need to release new or changed software. And there's the catch.
With software delivery, or product delivery in general, it's not just the product development team that's involved, or more specifically the software development team. It's other teams and people involved as well. Think about marketing, legal and compliance, worker's associations, security and risk management. These are all teams that are not part of the IT department. And no, security officers are not part of an IT department, and in case they are, they most definitely shouldn't be. And the biggest catch of all, the 'business' needs to be involved from day -1. Unless all of these different roles, teams, people, stakeholders, however you want to call them are on board and work in those same small increments, not becoming a bottleneck and automate as much as possible, your Continuous Delivery efforts are a waste of everybody's time and your organization's money.
Back to your 'business', it's them that request for features and not the user. It's them that pay for the development of the product not using it per se. When that 'business' is not capable or willing to define the product's features such that they can be delivered in tiny chunks, than you're out of luck and not much will come from Continuous Delivery in your organization.

The Product Owner is key in all of this. Being the hinge between the Product Team and the rest of the world, the PO is the single one person that can and must ensure that the product is delivered incrementally, with business value visibly added with each increment. PO can't do this, you've got yourself some trouble. The PO, at all times, must be able to relate every single feature, one way or another, to an improved life for the user of your product and therefore a positive effect on the organization's bottom line.

So, Continuous Delivery is not an IT thing, it's a business thing. And don't let anybody convince you otherwise. Being as it is, moving towards an Agile way of working and Continuous Delivery or Deployment, in fact would mean that you no longer consider your IT as being delivered by a separate IT department, but as an integral part of your business.
This does make perfect sense, considering your IT as part of your business I mean. It makes perfect sense because more and more business are all about information and are run based on information. IT is no longer a tool, like a glorified typewriter, it is in fact what is producing business value. No IT no business.

Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contactlist. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

June 7, 2017

Microservices on steroids, getting from just agile to business agile [part 1 of 2]

Summary

Agility is important, and business agility even more important. With this, the architect has to play a crucial role by defining the right principles, commandments if you will, and seeing to it that the product development teams live by them. Principles like KISS and Independent Deployability are two principles not to mess with. MIcroservices are an important and extremely helpful aspect of this, so is continuous delivery. But you need to get those microservices on steroids in order to be able to move from just agile to business agile. In this first of a two piece post, I am laying the ground works for explaining how you can facilitate business agility in your architecture.



Since you’re reading this post, it’s likely you’re an IT architect or have to deal with one every once in a while. Meaning that you’ve been confronted one way or the other with Service Oriented Architectures (SOA). Maybe you’re even a proponent of this architectural style.
I’m not going to explain what an SOA is, there’s quite a few very good sources available online explaining what it is and why it’s a good way of building your IT landscape, or not. As always, there are just as many proponents as there are people against and SOA. Over the years I’ve grown to like the premise of an IT landscape where fairly small pieces of functionality can be strung together to realize business functions. And with the advent of RESTful web-services, the premise of a true SOA has become more prevalent as ever.

Nowadays all you see and hear around you are Microservices, even though there doesn't seem to be a clear definition that everybody feels comfortable with, everybody is doing Microservices. One of the most interesting definitions I heard is that it is a really small service. But not as small as a nano service. But just to make sure that you know what I refer to when discussing Microservices, here're two definitions I like to use. The first is from Wikipedia, which in and by itself is no guarantee for it to be correct, but at least it's a common definition:

"Microservices is a variant of the service-oriented architecture (SOA) architectural style that structures an application as a collection of loosely coupled services. In a microservices architecture, services should be fine-grained and the protocols should be lightweight. The benefit of decomposing an application into different smaller services is that it improves modularity and makes the application easier to understand, develop and test. It also parallelizes development by enabling small autonomous teams to develop, deploy and scale their respective services independently. It also allows the architecture of an individual service to emerge through continuous refactoring. Microservices-based architectures enable continuous delivery and deployment."

The second definition is form Martin Fowler, which is in essence the same:

"In short, the microservice architectural style [1] is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies."


Microservices are not easy to develop, much like API's, they require a rather typical level of maturity of a development team to develop them. The hard part of Microservices is typically the "independently deployable" aspect of the services and then in particular maintaining that independence of time. Let's call it sustainable independence.
What I usually see is that over time the architecture becomes convoluted, resources entangled (the whole HATEOAS thingy is not trivial at all) and adversity against rearchitecting the IT landscape to get back to independent microservices.
To develop them,  the Microservices, you need to invest. In general, a Microservice is way more expensive to develop than a regular service. Much like an API being a lot more expensive than a simple service interface. So I would argue that there needs to be a business case to capture some functionality or resource in a Microservice.

Still, Microservices are a good thing to look into, if not only to be able to dismiss them. But more importantly, they are very helpful in becoming agile. I would almost argue that in order to be agile, you need an SOA based on Microservices and API's. Having said this, I need to emphasise that we're still talking about agility of the product development team, the IT people.
Mind that having an architecture with highly independent units of deployment is a blessing for all of us that want to do Continuous Delivery or Deployment for that matter. It makes life so much easier and as an architect you should really make sure that independence of components, services or units of deployments, however you want to call it, is a prime-directive to all your developers, great and small. It should be up there on the wall with the rest of your architecture principles. First of course you mention KISS and second the declaration of independence for your components.

But as I said, agility so far is technical agility. You've designed your software, your product, for change. Which allows you to be agile. Your PO (Product Owner) wants something different, you can change your code easy as can be. But when you think about it, it won't make your business more agile, it will make your IT more agile.

There's quite a bit more to be done before your business gets its agility... more on that in the next part.



Thanks once again for reading my blog. Please don't be reluctant to Tweet about it, put a link on Facebook or recommend this blog to your network on LinkedIn. Heck, send the link of my blog to all your Whatsapp friends and everybody in your contactlist. But if you really want to show your appreciation, drop a comment with your opinion on the topic, your experiences or anything else that is relevant.

Arc-E-Tect

Special thanks to Sytse for pointing out one or two textual errors and helping me correcting them.