[service-orientated-architecture] Janecek on reducing complexity

[Gervas Douglas]

<<Whenever IT professionals discuss the benefits of SOA - full utilization of IT resources, reduced infrastructure costs, and the agility to deliver new services quickly - they usually issue a caveat. They warn potential adopters of the inherent complexity of SOA, and point out that unless SOA is properly managed and governed, it's apt to fail. It's a valid concern, which is why I recommend that IT professionals seek a SOA solution that combines SOA governance, quality, and management. But as SOA has matured and tools have been developed to cope with complexity, complexity itself has become less of an issue. In fact, here at HP, we've discovered something that may surprise IT managers who implement SOA to gain the usual benefits. When it comes to integrating large, complex IT environments, SOA has a significant additional benefit: It actually reduces complexity. And we use it for that purpose in integrating our comprehensive suite of IT management solutions by substituting business services for point integrations. The HP IT Management Portfolio: A Use Case To help our customers take a lifecycle approach to IT management, we use SOA to integrate a set of software that includes solutions for portfolio management, SOA governance, quality assurance, application management, business service management, IT service management, and business service automation. In the process, we've encountered and solved a problem many of our large enterprise customers face - the overwhelming complexity of using traditional enterprise application integration (EAI) techniques. The story of how we integrate our software portfolio is a compelling use case for SOA as both an example and a set of best practices for integrating large complex environments. Using our own SOA software solutions, we've substituted SOA-based services for EAI-style hub-and-spoke integrations. And in the process, we've given our developers best practices, visibility into the inherent complexity of integrated resources, and a mechanism for putting everyone's expectations on the table - time, cost, resources consumed, and impact on service levels. Standards Are Not Enough Before I go into detail about how we apply SOA concepts to integrations, I'd like to comment briefly on the role of standards. Standards are important in EAI, since the more standards-based systems are the less work needs to be done writing APIs that make disparate systems talk to one another. And while this is extremely important, it's not sufficient to overcome the complexity of large-system integration. Using SOAP, SWDL, Atom, or similar standards can help systems interface with systems, but it doesn't make services more interoperable or useful. SOA processes and solutions add essential factors such as shared guidelines on how to externalize data using these standards and a way to express identifications, versions, business taxonomies, and relationships. It's very easy to serialize a Java object into a custom piece of XML code and then put it in a SOAP envelope. But the result can still be chaos if the process isn't guided by effective architectural governance. In short, standards are essential, but standards aren't enough. They help systems talk to systems, but they don't do enough to help systems work well with other systems. They don't supply agreements on the meanings of terms, rules for data storage and format, or contracts that govern the interaction of organizations. And SOA does. The Importance of Contracts SOA brings several advantages to the process of integration, but if I had to pick one as the most important, I would point to contracts. The whole concept of SOA is based on business services, and services are based on contracts. Unfortunately, most people consider contracts in SOA to be based simply on the technical agreement at the service-interface or message-format level. But a real contract is much more. A SOA contract is an agreement between the provider of the service and the consumer of the service - an agreement that covers all the important aspects of the relationship. Examples include the following, although there are many more aspects that can and should be described in the contract: When will the service be available? What is the expected performance of the service? Are there limits on the number of times I can call the service? Who is responsible for correcting faults? What is the change process? What are the escalation procedures? How can I request modifications to the service? Contracts give visibility into services and their relationships, yield better alignment with business service management, and reveal integrations that might break or otherwise impact seemingly unrelated services. And they capture the roles and the responsibilities of both the consumer and provider. Contracts are an important part of SOA governance precisely because of the inherent complexity that distributed and loosely-coupled systems create. They are a best practice developed to address and mitigate the problem of complexity. So SOA governance mechanisms work just as well to counter the complexity of enterprise integration, and that's why here at HP we use our own SOA governance approach and software in the process of integrating our portfolio. SOA puts the business logic where it will be invoked automatically - in the business service rather than buried in an integration server. Contracts yield important benefits while an integration project is going on, and they build in the mechanism for successfully scaling and expanding later. They also support the lifecycle approach by extending visibility across development, testing, and monitoring. The Importance of Policy Management Another key benefit that SOA brings to integration is consistency, driven by policy management. Integration projects need to comply with corporate policies for interoperability, security, and performance. In many cases, they also have to comply with external policies imposed by regulations such as SOX and HIPAA. Enabling policy compliance and reporting are inherent capabilities of our SOA governance solution. A Model to Address the Key Requirements of Integration Combining the HP lifecycle approach to IT management, the principles of SOA, and the expertise we have built around SOA, HP has developed a model that enables us to integrate components of our portfolio quickly, cost-effectively, and in a way that supports future growth and change. We take basic factors into consideration before we undertake an integration project, some of which go beyond software architecture: Overall architecture Solutions integration Standards Service model governance Data model governance Our working version of this model addresses all five factors by providing governance, a roadmap, communication, visibility, and alignment between our business units and our R&D organization. It also gives us a methodology, key services, and standards. And it helps our teams agree on priorities, roles, and responsibilities; select tools and define architecture styles; and agree on how services will be architected, described, and published. Under this model, every integration effort is split into two parts: delivery and governance. The integrations are governed centrally by our Applications Strategy Team (AST), which is our SOA Center of Excellence. The actual delivery of the integrations remains in the hands of our Product Strategy Teams (PSTs) and are handled just as if they were any other product feature. This separation of roles allows for scalable execution of large portfolios of integrations. Business Requirements First Within this framework, we first deal with the "why" and "what," and then with the "how." We always try to design our integrations to minimize middleware infrastructure requirements. Even sending one message from service A to service B might require sophisticated middleware if the services have poor interfaces or inappropriate interaction options. On the other hand, well-designed services require nothing more than the ubiquitous Internet infrastructure. So we avoid the kind of high-level use case analysis often seen in EAI projects that contain long lists of low-level requirements containing WS-* acronyms. While low-level requirements become critical at later stages of the lifecycle, we start from a different place. When defining solutions that span multiple products, we first define the use cases and agree on priorities of subordinate use cases (which are often individual integrations or product enhancements). These use cases help us define services and basic interaction patterns. At the service and interaction level, we insist on formal descriptions. Governance is extremely important at this level. If the high-level decisions are wrong, we're apt to end up with services that are too granular or processes that are overly complicated. Good governance helps us identify and eliminate overlaps, minimize hardware requirements, consolidate databases where necessary, and form solution-level teams that involve business-level stakeholders for clear communication. Once the business requirements are firm, we can map them to the product level, integrating products in a way that addresses overall business requirements rather than simply bringing two systems together. Combining SOA with EAI While SOA delivers essential capabilities lacking in traditional EAI practices, we understand that for some integrations, EAI is still necessary. If very tight integration is required, or both applications require their own databases, we might still use EAI techniques. But even when we do, we give the integration a service façade so that we have an integration catalog for governance purposes and compliance reporting. If your organization is still taking an EAI approach to integrating large, complex environments, and especially if you're experiencing high costs and low responsiveness to business needs, then I recommend that you consider SOA as an alternative approach. It can pay off in increased visibility, better decision-making, flexibility, change resilience, reduced ongoing maintenance, lower cost, and - believe it or not - less complexity.>> You can read this at: http://soa.sys-con.com/node/914596 Gervas