Guest column: Expensive data pitfalls in service-oriented architectures today

SOAs themselves are not the problem. The fundamental problem is the great complexity of IT infrastructures and the absence of an integrated architectural foundation that enables interoperability among silos of heterogeneous applications built up over years and even decades. In many organisations, well over 50 percent of the IT budget is devoted to building and maintaining points of integration among legacy systems specific to finance, supply chain, customer management, and other mission-critical functions.

CIOs and IT managers are confronted with:

• High costs for IT development, deployment, and maintenance


• Poor IT infrastructure flexibility in meeting new business demands


• Inconsistent, inaccurate business data across the enterprise

Integrating such fragmented architectures has emerged as one of the greatest challenges facing IT and business, with bottom-line business implications. Business prosperity increasingly depends on a global view of customers, suppliers, products, and partners—an ideal not achievable without integration.

SOA provides an elegant solution for dealing with the complexity of application integration in the enterprise. However, to efficiently handle the growing demands for timely information in the enterprise, the underlying IT infrastructure must also support the real-time provisioning of highly fragmented data.

EAI and other application integration-centric technologies are not designed to address data integration issues. Implementers embarking on the path of SOA, have quickly realised that SOA presents the following data-centric challenges that cannot be handled by their SOA platform providers:

• Heterogeneous data sources distributed across the enterprise and beyond


• Inconsistent and constantly changing data structures


• Poor data quality that is often difficult to measure or monitor


• Lack of agreement or visibility (single-view) into critical information assets

SOA implementations thus lack a proactive data integration foundation, resulting in massive hand-coding efforts to address granular data complexities underlying the business logic. This leads to the proliferation of brittle point-to-point connectivity, which is exactly the problem that SOA set out to solve. Others may be handicapped in data functionality, or exacerbate issues with inconsistent data across the enterprise.

To avoid costly setbacks and implement an SOA that provides integration at both the application level and the data level, IT decision makers should proactively define business requirements for data integration in an SOA, scrutinise the capabilities of SOA solutions to deal with complex data issues, and recognise functional distinctions between EAI, data integration, and other SOA technologies.

In particular, IT architects implementing an SOA infrastructure should consider five key data integration elements for delivering holistic, accurate and timely information to an SOA:

• Data Semantics: The business context behind data definitions for concepts such as customer address, product category, or employee type


• Data Quality: Improving the accuracy and consistency of the “dirty data” common in disparate applications and legacy systems


• Data Governance: Data and metadata lineage, management, documentation, reporting, and auditing tools that help satisfy Sarbanes-Oxley and other regulatory requirements


• Data Access: Broad reach into structured, semi-structured, and unstructured data in hierarchical and relational databases, mainframe systems, files and documents, and applications

An SOA offers an ideal framework for EAI and enterprise data integration technologies to complement each other, with EAI orchestrating processes and transactions, and a data integration platform executing complex data integration functions. Without a sophisticated data integration technology as a foundation, an SOA will suffer expensive limitations in its ability to fully access and leverage data, and deliver on its potential as a transformative architecture for IT and business.

Laurie Newman is managing director for Informatica A/NZ.

architecturescolumndataexpensiveguesthardwareinpitfallsserviceorientedtoday