ZapThink </>
Business Agility as an Emergent Property of SOA
Document ID: ZAPFLASH-20081024 | Document Type: ZapFlash
/By: Jason Bloomberg/
Posted: Oct. 24, 2008
As students go through our Licensed ZapThink Architect
<http://www.zapthink.com/lza.html> (LZA) course, they experience a
series of "aha" moments, as we systematically tear down their
preconceptions about what Service-Oriented Architecture (SOA) is -- and
what it is not. But perhaps the biggest aha moment of all, however, is
when they realize that implementing SOA isn't traditional systems
engineering (TSE) at all, but rather a fundamentally different approach
to dealing with complexity in the IT environment. Needless to say, this
realization is an especially big wakeup call for people with TSE
backgrounds!
The fundamental shift in thinking is this: TSE focuses on building big
systems out of small components, where the behavior of the resulting
system depends directly on the properties of the components.
Essentially, TSE boils down to a "connecting things" way of thinking
about distributed computing, where integration is the central activity,
and what you end up with when you're done with all the integrating is at
best what you expected to build.
SOA, on the other hand, calls for an entirely different approach. In the
SOA context, we focus on building and maintaining the Business Service
abstraction <http://www.zapthink.com/report.html?id=ZAPFLASH-2007719>,
which supports inherently unpredictable behavior as the business
composes Services to support fundamentally dynamic business processes.
Essentially, with SOA we're building for change, while with TSE, we're
building for stability. The problem with stability, of course, is it
only takes the business so far -- if the organization requires business
agility, then they're much better off implementing SOA.
*Business Agility as an Emergent Property*
Business agility, which ZapThink defines as being able to quickly and
efficiently respond to changes in the business environment and to
leverage such change for competitive advantage, is the primary strategic
motivation for most SOA initiatives. What is it about SOA, however, that
enables such agility? Unlike with TSE, where the properties of the
resulting system depend upon the properties of the components that make
up the system, the individual elements of a SOA implementation,
including Services, Service compositions, policies, contracts, and the
like, somehow contribute to the agility benefit
<http://www.zapthink.com/report.html?id=ZAPFLASH-2008117>, even though
each of them don't exhibit business agility themselves.
We call properties that certain systems exhibit that their individual
components do not /emergent/ properties. In addition, we call systems
that exhibit emergent properties complex systems
<http://en.wikipedia.org/wiki/Complex_system>. Complex systems theory is
an exploding area of study today, because so many different systems in
the world exhibit emergent properties. Emergent properties include
everything from friction to traffic jams to the human mind, and the
field of complex systems theory is gradually explaining many of the more
subtle aspects of the world around us.
*Complex Systems Engineering: The Key to Implementing SOA*
Explaining natural phenomena is one thing; /building/ a complex system
is quite another. We call such a practice Complex Systems Engineering
(CSE). If you're looking to engineer a system, where your desired
outcome is an emergent property like business agility, then TSE won't do
-- you need to take a CSE approach. As a result, it is imperative that
architects looking to implement SOA take such an approach, because
business agility is a critically important emergent property, and in
many ways defines the success criteria for SOA initiatives. Leveraging
complex systems best practices, therefore, may be able to give us some
important insight into how to deliver on the business agility promise,
and perhaps more importantly, how to avoid impediments that might
prevent a SOA project from providing the required agility.
A fascinating paper from 2006 by David A. Fisher from Carnegie Mellon
University's Software Engineering Institute
<http://www.sei.cmu.edu/publications/documents/06.reports/06tr003.html>
provides a marvelous starting point for a discussion of SOA in the
context of a particular type of complex system known as a system of
systems. According to Fisher's paper, systems of systems are
qualitatively different from traditional large-scale systems. Such
systems of systems are far more complex, and involve independently
managed and operated components. Furthermore, systems of systems depend
upon other systems that are outside the control of their owners and
users. As a result, TSE approaches and methods are often inadequate or
inappropriate for systems of systems.
Systems of systems have unique characteristics that distinguish them
from traditional monolithic systems, including higher levels of
complexity, flexibility, and emergent behavior. Such characteristics
result from the operational and managerial independence of their
constituent parts, from independent evolution, and from the character of
emergent effects, while traditional monolithic systems depend on central
control, global visibility, hierarchical structures, and coordinated
activities. As a result, such traditional systems are unable to exploit
the advantages of emergent behavior like business agility.
*Is a SOA Implementation a Complex System?*
Many of the characteristics of such systems of systems line up quite
neatly with how ZapThink sees SOA. The core SOA principle of loose
coupling in particular leads to the system of systems requirement of
autonomous systems. In fact, complex systems theory recognizes that a
result of tightly coupling systems is the fact that accidental failures
of individual subsystems lead to cascading failures across the entire
system. Furthermore, systems of systems depend upon interoperation among
systems, rather than integration, where integration of subsystems leads
to a unified system, while interoperation among systems is the
combination of autonomous systems into a system of systems.
Interoperation in systems of systems demands what Fisher calls a
node-centric perspective, where each constituent (a Service provider or
consumer, say) views the system from its own individual perspective. For
each node, a node-centric perspective describes the interactions with
its immediate neighbors based upon available metadata. In this view, the
behavior of the overall system of systems depends upon the interactions
between Service providers and consumers. An individual constituent node
need have no knowledge of the details of other nodes that it doesn't
interoperate with.
To achieve this interoperation, systems of systems should be
interdependent with other systems beyond their boundaries, where it's
impossible to predict the resulting outcomes of the architecture.
Furthermore, requirements for systems of systems that interoperate are
constantly changing and imprecisely known, as they typically are for SOA
implementations. Interoperation also encourages distributed control,
which aligns nicely with the business empowerment benefit of SOA.
It seems, therefore, that an architectural approach like SOA that leads
to loosely coupled, autonomous, interoperable Services in response to
dynamic business requirements is a perfect example of a system of
systems. Fisher, however, makes an interesting claim to the contrary:
that SOA implementations, at least at the time he wrote his article,
don't provide sufficient interoperation in systems of systems because
they assume an absence of emergent behavior or at least fail to
recognize and provide support for managing emergent behavior. The
question of whether SOA leads to emergent behavior like business
agility, therefore, depends upon whether complex systems theory applies
to SOA at all.
*Taking a CSE Approach to SOA*
If business agility is an example of emergent behavior, and we expect
SOA implementations to provide such agility, then how can we say that
SOA implementations assume an absence of emergent behavior? This
conceptual disconnect is at the heart of the aha moment our students
experience in our course. Far too often, people assume that TSE is
sufficient for implementing SOA, and TSE thinking excludes emergent
behavior as a possible outcome. We see this misconception all the time,
whenever an organization believes that they must purchase integration
software like an Enterprise Service Bus
<http://www.zapthink.com/report.html?id=ZAPFLASH-2008530> (ESB) in order
to implement SOA. TSE means connecting things, so the obvious place to
start is with something that helps you connect things -- but that
approach misses the boat entirely.
Fisher's second point is also well-taken. To put this point in another
way, we would need to manage a SOA implementation's emergent behavior in
order to achieve the benefits that systems of systems exhibit. ZapThink
hammers home this point whenever we talk about business empowerment
<http://www.zapthink.com/report.html?id=ZAPFLASH-2008212>. If we
mistakenly conclude that the sole point of SOA is to build all these
flexible Services and then simply turn them over to the business to
compose willy-nilly, then it's true we'd never be able to achieve the
business agility benefit, because such a haphazard lack of control would
lead to immediate chaos. On the contrary, the key to the business
empowerment benefit of SOA is /governance/ -- a set of organizing
principles that tempers the emergent properties of the architecture,
thus providing the essential management of emergent properties that SOA
requires for us to consider SOA implementations to be complex systems.
*The ZapThink Take*
There is an important insight here which is the essential point of this
ZapFlash: for SOA to be successful, organizations must take a CSE
approach to their implementation, /including the implementation of SOA
governance/. Our natural tendency would be to take a TSE approach to
governance, where we hammer out organization-wide policies and put in
place a systematic governance infrastructure to enforce those policies.
While such an approach might be integral to a traditional architecture
that we've implemented with TSE approaches, taking such a heavy-handed
approach to SOA governance will stifle the implementation's emergent
properties, most notably business agility. ZapThink calls this situation
the "big brother effect," where excess governance can lead to the
failure of the SOA initiative.
Instead, it is essential to take a CSE approach to SOA governance.
Represent policies as metadata, and leverage SOA governance
infrastructure to manage and enforce those policies in the context of
Services. As the SOA implementation matures, leverage SOA governance
best practices to support overall IT governance, and in turn, corporate
governance. This approach to SOA governance "in the broad" not only
improves the organization's governance overall, it is also essential to
promoting the emergent properties of the SOA implementation. Your SOA
initiative depends on it.
