November 10, 2011

The Relevance of Data Governance; Why the 'I' is more Important than the 'T'

It is always interesting to see that in most enterprises the T in IT is considered most important or at least it gets the better part of the IT department’s budget. This is probably due to the fact that the T is always changing. In fact in order to keep the competitive edge, enterprises need to adopt new technologies, they need to embrace new technologies. Competitive edge becomes cutting edge becomes bleeding edge.
In addition, enterprise need to keep investing in the T because their T vendors have clearly defined end-of-life dates and when the T is not supported, the enterprise is risking its continuity.
The T is a moving target and requires a constant stream of budget in order for the enterprise to keep up with the competition. In order to control the constant stream of budget and being able to manage the changes in the enterprise due to new and improved T’s we define projects to migrate from one T to another, to stay up to date with our T and to assess our maturity with respect to the T.

I think most people understand the T more than the I as well.

In many enterprises the T is treated as an asset, it is considered a business’ capital, where it is only a tool. But we forget about this. Because it is so dynamic, so hard to capture, to restrain and harness it keeps us preoccupied and we forget about the I in IT. Whereas the I is the asset. Without the I there would be no IT.
More interestingly, the I should typically not change at all, there is no end-of-life for the I. Where the T will have to change constantly, the I cannot be allowed to change. Once it is fixed and considered correct, we need to keep it fixed in order to keep it being correct. And where we do whatever we can to change the T, where we spend as much as possible to change the T to keep up with the competition, we spend close to nothing to maintain the I, to keep the I stable, fixed and correct. All the new T we introduce in our enterprise, is directly or indirectly harming the integrity of the I, yet we fail to prevent the T from impacting the I. The T is no longer there to improve the I, to enrich the I, to make the I the added value of the enterprise. In many enterprises the T transcended from the means to the end.
We have become so dependent on the T that every change in the T is controlled, the T is governed by processes, forms, people and standards in varying orders of importance depending on the maturity of the enterprise.

It is ironic that the T was once introduced for the I to benefit from. Enterprises were all about the I in those days, but the T took over, it’s dynamic nature made it more interesting, less boring than the I. Its dynamic nature made us govern its changes. It became our object of expenditure. The I was forgotten, a second class citizen in the enterprise. But it is the dynamic nature of the T that makes it less of an asset in the enterprise, every investment in the T is by its very nature not a long term investment. All of a sudden a short ROI is important for every change in the T, because it is not an asset. The T is an opportunity, one that depreciates more and more rapidly as we are more and more dependent on the T to keep the competitive edge.
The I is hardly changing at all. It is growing in size and forms, but once it’s here, it’s here to stay. The I never changes, it’s location may change but the I itself doesn’t change, the I therefore is an asset must be considered an asset. The I is long term planning and as with all long term planning in order to make sense, in order to be fruitful it requires to be well thought through, to be looked at from any angle and it needs to be handled with great care. It needs to be governed.

Back to reality.

I've been most of my professional life contracted or employed by financial institutions and I think that over the last 2000 years the way banks operate hasn’t drastically changed. The advancements in technologies has made banking more efficient and has provided us with more and more ways to get in touch with customers but the essence of banking hasn’t changed. People trust banks with their money and they trust others with this money. Banks have invented interest and interest rates in order to implement a viable business.
When one looks closely at the different banks all over the globe they mainly differentiate by the level of service offered to their customers and where it is relevant the reputation of the bank in the market it operates in, be it a geographical market or a business market.
The advancements in technology in the last decade have turned financial systems into commodities, where software companies can develop generic solutions for financial markets in a far more cost effective way than these financial institutions can. Due to this commoditization of technology in the financial sector, it becomes more and more difficult for enterprises in this market to differentiate by the level of service without exclusivity. Nowadays every bank has an online presence and the maturity of the bank defines the diversity of its online presence (internet, phone, mobile, smart-phone apps, etc). Because of this, the technology they have at their disposal does no longer provide a means to differentiate them from their competition. It no longer can provide them with a competitive edge, it can merely streamline their processes and polish their image.
These days they have to differentiate from competitors by the amount of information they have about their customers in the context of the customer’s world. The customer wants to know his complete financial situation. He wants to know his exact position, but also the bandwidth of his credit. And he also wants to know this in the context of his current situation. For example, he is abroad and needs to pay a hotel bill, he will not be interested in the amount of money in his savings account, but he will be interested in the limits of his credit cards. But when this same customer is back from vacation he will be less interested in his credit limits, but will be more interested in his expenses for each of his methods of payments. He will be interested in what is paid by which credit card, what is paid for by debit card, when and where does he use an ATM and how much did he take from the ATM. He will want to know what bills are being paid and how much each bill was. What are recurring payments and is there in trend in the amounts of these payments.
It is not technology that he wants from his bank, but information. As a bank, people traditionally trust them with their money and in an ever more digitized world, they trust banks with their data about their financial situation. It is up to banks to turn this data into relevant information for their customers. In order to do this efficiently and to provide the services to their customers efficiently they need technology. The competitive edge is in the fact that a bank can turn this data into information for their customers that is relevant in different situations.
Meanwhile, the same data is used to streamline the internal processes and allow for better analysis of the financial markets and the general business a bank is in. Customer data is the micro level information that is important in understanding macro level economies as long as it is in abundance. After all the financial business is more than anything else a business of people and their trust. And accurate information is the cornerstone of this trust.

September 23, 2011

Flexibility in the 21st century: Agility


I've been a developer for pretty much ever since I got introduced to computers. First I developed for my Sinclair ZX81, then I moved on to MSX, made the transition to DOS in 1989 and moved along with Microsoft to Windows. I've developed on OS2 and various flavors of Unix and I've been using a multitude of programming languages ranging from BASIC to Modula2 and Pascal to C, C++ and Java.

I've been to university studying Computer Science in the 20th century and back then we were thought to design and develop for flexibility. The reason behind it has always been that you should be prepared for the fact that once you're done and you've delivered there will be new releases, new requirements, changed requirements, etc. Possibly even while you were working on the system you would be confronted with new insights, new ideas, new and more important requirements. Re-use was all the rage and you should prepare yourself for changes by being able to re-use 80% of the code, of the features, of the functionality already created and only add an additional 20% of design, code, testing and documentation for the new stuff.
Even in the wonderful world of OO, CBD and Modularity, the beat of the drum was about being flexible. Create a flexible solution, one that can bend and twist all over the place to accommodate for every change in the requirements. Be it functional or non functional. The whole idea was and with many of us currently still is, that when you design or develop something it should be so flexible that a new requirement would be already implemented by 80% of the existing code.

I remember having reviewed a system that was almost entirely written in database tables. An engine would simply read a set of tables and an object model was instantiated and executed. Which is not all that unfamiliar when it concerns for example workflow engines or rules engines. But this little piece of software concerned an actual programming language in database tables. I could write a simple "Hello World" application in the database. The reasoning behind this little piece of ingenuity was that there wouldn't be any deployment stress with testing of the software etc once the application was deployed. Just updating the tables would allow for new functionality. Of course until after the first update of the tables wrecked the system because there was no syntactical or semantical checking of the 'new program'.

Flexibility is something that needs to be provided by frameworks. This is why you would create frameworks. But the thing with frameworks is, is that it is extremely hard to actually write a framework that is actually usable as a frameworks. They tend to be too restricted to a specific purpose or too generic. Most of the popular frameworks started out as a specific solution that was repeatedly implemented, based on a well known enterprise design pattern. After the 3rd time of implementing the pattern the designer or developer recognizes the parts that are consistently changing and those that remain and he'll be able to start developing the framework. This even holds true for design-time frameworks.

With the advent of eXtreme Programming and Agile project management (any flavor) the world of software development also got an overhaul. No more lengthy analysis and design phases prior to development and instead short cycles to get the product out. In the old days (which in IT means 'not that long ago') analysis and design was considered extremely important because one wouldn't want to miss a requirements, be able to prepare for the next version, build in hooks and nooks to facilitate additions without corrupting the overall architecture. But the longer the time between conceiving an idea and realizing, the further apart the idea and the realization of the idea are. Hence the short cycles of today, which still have an analysis and design phase, but shorter. And no, one can't do without analysis, design in agile or even without architecture, but that in another post.

Now, what is so dramatically different in terms of flexibility in the 21st century compared to the previous millennium? Well, the difference is in mindset of the developer. The developer and even the designer creates for replacement not for changing. By this I mean that when designing or developing nowadays, one needs to keep in mind that pieces can be ripped out and replaced by something completely else. Without heart feelings. And yes, letting go of your baby while still in its infancy is a change in mindset. Today you create something that is not supposed to last forever, but something that lasts until the next big thing. No more stone carved COBOL systems that will survive humanity.

The big difference here is that every application is a business functional framework, one that targets a specific area of business functionality but caters only for that functionality that is required. New functionality is added in a consistent manner, existing functionality is removed when no longer valid or replaced by a new piece of code when required to be changed. Mind that the key here is 'replaced'. Code is not amended to implement the change. Because it is way to cumbersome to change the code, it is better, faster and more convenient to build the new functionality from scratch and add it to this business framework. This is flexibility through agility allows IT to build business applications that are consistently 'correct' in design and easy to support and maintain.
The whole premise of agile development is to scope an iteration purely to those requirements that are immediately necessary and developable within the constraints of the iteration (timelines, budget, resourcing, etc), while not taking (too much) into consideration what might be required in the next iteration. Which basically means that whatever you create may be thrown away in the next iteration because it conflicts with the requirements of that next iteration.
The idea that in agile development there is no place for analysis and design and certainly not architecture that many agile adepts have embraced is of course a consequence of architects, designers and analysts not being able to analyze, design and architect keeping in mind that every single component in the solution should be throwawayable. They're quite often still create flexible systems instead of agile systems, hence the conflicts with the developers.

Agile systems are designed by keeping in mind that the components in the systems are decoupled, on a functional level. On a requirement level that is, which boils down to the fact that the analyst needs to define the requirements and their impact on the technology such that the requirements themselves are decoupled from each other. Dependencies between requirements result in tightly coupled implementations. When two requirements depend on each other, they should be treated as one. Consequently the designer should design the solution for each requirement independent of the other requirement. Re-use of functionality already designed for one requirement to be met should not be in the design... or at least not in the first iteration of the design. Preferably, re-use should only be taken into account in the technical design of the solution if at all in the design. Why not leave this decision to the developers?
This will allow for the assumption that what is designed today will not be there tomorrow.

So where does this leave the architect. It is clear that every iteration will have its analysis and its design phase. You cannot get around it. It is the rule of problem solving: understand the problem (analysis), think of what the solution(s) might be (design), implement the solution that is most feasible if feasible at all (development). You just cannot solve a problem you don't understand.
But where is the architect in all this. Surely you can't architect a system for which the requirements are not clear at all, they're made up and prioritized as the project goes. Or rather as time moves on. The architect defines the architectural principles that the solution with every iterations needs to comply to, to safeguard the consistency of the solution, the project's deliverables, in the grand scheme of all systems and solutions in the environment. The architect will define that at all times user interface and business logic need to be separated in design and in implementation to run on different (logical) servers. The architect will define that users of the system that are external to the company need to explicitly be authenticated using a secure token, where as users that are accessing the system through the local LAN can be authenticated using userid/password only. The architect will define that the allowed technologies for implementing the solution should be platform agnostic, allowing the solution to be deployed on Windows, Unix, Linux and OSX systems. And the architect will verify compliancy with the architectural principles prior to deployment in production.
The architect is or heads the design authority.

Without architectural principles that guarantee the integrity and consistency of the IT landscape yet support or at least allow agility in this same landscape, agile methodologies will create chaos and therefore significantly increase the TCO of the landscape.

As always, I welcome any comments. Whether you agree or disagree or just find this post informative. Please let me know where you think I am wrong or right and moreover why you think this. It will be very educational for me as well as the other readers of this post.

Thank you for reading this far.


September 7, 2011

The Role of the Enterprise Architect


Every once in a while I have to explain to people what the role of the Enterprise Architect is. And although this question isn't all that trivial at all, I still answer it with reasonable ease.

Disclaimer: I consider myself an Enterprise Architect, but with an IT perspective. This is important to note because it significantly limits my scope.

Another disclaimer, this post is from September 2011, since then, a lot has changed and although I've made some minor corrections in December 2014, you'll have to bear with me that the main gist of the post is 3 years old.

With that out of the way I can provide you with my answer to this question and explain it to you within the proper context.

Spoiler alert: The Enterprise Architect is responsible for formulating the vision for the mid and long term from a technology point of view, the IT Vision. A vision that is in-line with the business vision. He is responsible for defining a strategy that is to be followed in order to realize this vision, the IT Strategy. A strategy that is in-line with the business strategy. And consequently the Enterprise Architect is responsible for creating a roadmap that when followed implements the strategy, the IT Roadmap. The IT Roadmap consists of strategic business projects that implement key milestones and are fully in-line with the IT Strategy as well as strategic IT projects that facilitate in the implementation of the IT Strategy.

First of all I think that the Enterprise Architect's role differs from a Solution Architect's role in one very significant way; Where the Solution Architect is confronted with a problem and request to create a solution for it that complies with a given set of rules, laws, regulations, standards, guidelines and what not. The Enterprise Architect is confronted with the
current state of the world and an idea of how the world might turn out to be in the foreseeable future. And then is requested to define all problems to be expected to finally come up with solutions to all of them, such that they all comply with a set of rules, laws, regulations, standards, guidelines, etc. Some of which are unknown at the time the solution are defined.
Basically, the Solution Architect is given a problem and architects the solution, the Enterprise Architect has to come up with the problem first and then architect the solution.

With this out of the way, I can finally start answering the question: What is the role of the Enterprise Architect? His (or her for that matter) role is to create, govern and maintain the architecture of the enterprise from a technology perspective. Which more or less means that the Enterprise Architect is responsible for defining how the enterprise architecture as a whole is mapped onto technology. In the previous sentence I refer to the enterprise wide business architecture. And yes, I realize that this doesn't really has any meaning. Until, of course, I explain to you what I mean by enterprise wide business architecture. What I mean by this is the complete set of business decisions, visions, objectives, processes etc that have been formulated. It means the essence of the enterprise. The primary reason and motives of running the enterprise, its raison d'etre, if you would pardon my French.
The Enterprise Architect (from an IT point of view) is responsible for defining the IT perspective of this business architecture. He is responsible for defining which technology, principles, concepts, systems, applications etc are interrelated and how they are interrelated in order to enable the enterprise to meet its goals. And in addition he is responsible for maintaining this architecture, amending it as business goals evolve, and govern changes to it.

At this point I typically get the first blank stares from people. Rightfully so, because it is very abstract and Ivory Tower'ish. But there is a simpler answer which explains the role of the Enterprise Architect more clearly, in my opinion.

The Enterprise Architect is responsible for formulating the vision for the mid and long term from a technology point of view, the IT Vision. A vision that is in-line with the business vision. He is responsible for defining a strategy that is to be followed in order to realize this vision, the IT Strategy. A strategy that is in-line with the business strategy. And consequently the Enterprise Architect is responsible for creating a roadmap that when followed implements the strategy, the IT Roadmap. The IT Roadmap consists of strategic business projects that implement key milestones and are fully in-line with the IT Strategy as well as strategic IT projects that facilitate in the implementation of the IT Strategy.

Although the Enterprise Architect has to be constantly aware of the ever changing environment and adopt his architecture to remain in-line with the business architecture. This in part as a result of a changing business vision and consequently changing IT Vision. The Enterprise Architect will also have to reconsider his IT Strategy as well as his IT Roadmap based on a changing market. Typically in organizations this is a yearly recurring effort.
This caters for the creating and maintaining the IT Vision, IT Strategy and IT Roadmap in order to create and maintain the Enterprise Architecture. It does not cover the governance of the Enterprise Architecture. In order to implement governance, the Enterprise Architect is also responsible for defining and enforcing all policies and procedures that required for ensuring that project executed within the enterprise (both IT and business projects) are according to the set strategy.

For example, when the IT Vision is that the vast majority if not all (business) services are running in The Cloud [scroll down for a footnote on this 'vision'], and the IT Strategy is to implement in order a highly virtualized infrastructure which is then complemented with cloud administration services such that a private cloud is the result and finally all systems are moved to a public cloud as a means of out-sourcing the infrastructure.
The roadmap might be to initially select and implement virtualization technology, then migrate core systems to platforms that are supported in a virtualized environment, then select a cloud provider that supports the selected virtualization technology, then implement iteratively cloud administration tooling.
The Enterprise Architect is then responsible for ensuring that a set of policies, standards and guidelines are defined, implemented and enforced, that enforces applications to be cloud-ready. This may mean that certain paradigms are followed, for example that new infrastructure software and applications running on them are horizontally scalable in order to increase capacity such that performance requirements can be met.

It is utterly impossible to enforce compliance to IT Strategy for all systems, applications, projects etc. The Enterprise Architect, in collaboration with the business, will have to classify each project, system, application as being strategic, tactical or opportunistic.
Strategic projects (implementing core systems) will have to be completely compliant with the IT Strategy. These projects are implementing systems and applications that are planned to last for a long time within the enterprise and perform core functionality to the business. Typically they are planned to outlive the IT Vision or even the business vision.
Tactical projects can deviate from the IT Strategy but are required to be in-line with the IT Strategy. You could say they have to be in the spirit of the IT Strategy. These projects are typically implementing core business functions but for any reason cannot be implemented completely in-line with the IT Strategy. For example because the vendor does not yet support the selected virtualization technology for all components of the application.
Opportunistic projects are those projects that have a strong business driver to be implemented within a short time frame. Possibly because the particular bandwagon needs to be jumped on.

The interesting thing about an enterprise architecture is that treats those components that implement it on a physical level (hardware and software) as black boxes. At least to a certain level. The Enterprise Architect is concerned with the concepts of the various components but not so much with their implementation. By this I mean that the Enterprise Architect defines for example that all applications will have to run on a common application server infrastructure. This is the concept of farming, the infrastructure consists of a set of farms that host applications and applications components (application server farms, messaging farms, database farms). The idea behind this might be that infrastructural software is to be considered a commodity. How these farms are actually implemented (clusters of active nodes, clusters of active/passive nodes, etc) is not so much the concern of the Enterprise Architect but more for the infrastructure architects (middleware, database, etc), who in turn will be required to implement these farms compliant to the Enterprise Architecture. Even more interesting is that the Enterprise Architect is responsible for documenting the complete enterprise architecture as he own the enterprise architecture.

Although I realize that I haven't mentioned anything about the information architecture in this post, this is where I want to leave it to. Me for myself am not entirely sure whether the information architecture is part of the Enterprise IT Architect's responsibility. The thing with the Information Architecture is that it, in theory, is a business affair. The information architecture should be owned by the business. But practically I've seen it considered an IT affair in most pretty much every organization I've been. This is mainly due to the fact that the data architecture and information architecture are considered one and the same, which is in my opinion, because in most organizations it is considered to be impossible to create one enterprise wide information architecture. As a result, data architectures are not derived from a global or even local information architecture but from the physical needs of applications. But this is something worth its own post which I will work on some other time.

So this is what I consider the role of the Enterprise Architect as a role within an organization. But, there's more... Since the Enterprise Architect is one of the most senior roles within an IT department, if not the most senior role, I also think that as an employee within the enterprise the Enterprise Architect has also the responsibilities that come with this level seniority. In many cases the Enterprise Architect will have to act out of his seniority instead of his role. And this responsibility differs significantly from one enterprise to another.

Thanks for reading all the way to the end. It's become a long text, but I hope I've been able to explain my view on my role as an Enterprise Architect. Probably there's a lot you disagree with and I would love to hear about it.


Footnote on the vision of having everything in "The Cloud"; Of course, it is not clear as to what the cloud actually is, even when you take the definition provided bij NIST. But the problem here is that everything in the Cloud is considered an IT Vision, which is of course not correct... but not uncommon, unless the  Cloud is considered placeholder terminology where the IT Vision is actually to no longer own physical infrastructure, not even from an outsourcing perspective.

April 28, 2011

Introducing my new blog on IT Architecture and Strategy

This is my blog on IT Architecture and Strategy. On regular basis I will post on topics that seem to be hot in the world of IT and that are either architecturally or strategically of interest.
Sometimes I'll comment on trends and sometimes I'll blog about my views and opinions on IT in general.

I hope you'll enjoy it and am looking forward to your views and opinions.