Author: Jane Griffin
Summary: Implementing the proper architecture for efficient, integrated information logistics management is a wise investment that facilitates the process of inter-enterprise collaboration and enables an organization to make the most of resources.
Cost-effective logistics management has forever been the bane of most companies’ existence. The bulk of many organizations’ budgets – both for information technology (IT) and process improvement initiatives – is spent improving the processes involved in getting materials and products through the supply chain with maximum efficiency and minimum cost. Why, then, are most companies still struggling to find and implement new technologies for integrating the technical architectures that form the backbone of their logistics networks? Simply put, the technologies are there, but fitting them together in a profit maximizing, effort-minimizing combination is a Byzantine labyrinth that most executive managers are loath to tackle.
The problem in maximizing the efficiency of logistics networks is now, as it has been for decades, that information, applications and processes have been stovepiped. These vertical silos of information have engendered years of disarray and inefficiency in logistics management, customer service and analysis capability. In this environment, processes and information systems are separate, uncommunicative parts of the organization that often work at cross- purposes.
However, the picture is not as dim as it may look. There is a new, two-part IT model to build, manage and integrate logistics networks. The first part of the model is a technical architecture that deploys the best-of- breed extraction, transformation and loading (ETL) tools; enterprise application integration (EAI) tools; and business analytic (BA) tools. The second part of the model is the use of these sophisticated tools to perform complex analysis to enable better demand forecasting and inter-enterprise collaboration between trading partners.
The current crop of integration and reporting tools has matured tremendously over the past several years, and with that maturity has come a new ability for organizations to integrate their systems (including data warehouses and inter-enterprise applications) in new and cost-effective ways. In fact, the current level of cooperation between vendors in the EAI, ETL and BA tool sectors makes it possible to build integrated technical architectures that will help create smooth, efficient information logistics management.
Collaboration will be the key to success. At the dawn of 2000, Gartner predicted a technology transition in the market – the merging or replacing of old-line, standalone ETL, EAI and BA tools with flexible, scalable, intelligent information networks that route data to and from information-craving entities based on prescribed business rules. “By 2001, most Type A enterprises will participate in between two and five collaborative pilots, with channel masters focused on singular processes (e.g., make, move and buy) at industry pinch points,” said Gartner’s Lora Cecere and Karen Peterson.1
This transition will not take place overnight, but it will most likely grow from vendors already in this marketplace. Moreover, it will not be a true marriage of technologies to create a “super toolset.” Instead, the transition will occur in the form of vendor cooperation that moves beyond marketplace posturing and individual profit- seeking to true collaboration with the ultimate goal of customer satisfaction on a wide scale.
This model can only be built with cooperation and collaboration – both by tool vendors and trading partners – in developing and using ETL, EAI and BA tools. How badly is inter-enterprise collaboration needed? According to IDC estimates, the total market for both intra- and inter- enterprise integration was $800 million in 1999 and will grow to $8.9 billion in 2004.2 The need is there, and the vendors will fill it – and savvy organizations will take advantage of the new technologies.
The foundation of information interchange (and inter-enterprise collaboration and managing information logistics) is moving data. However, moving data has become more complex than ever before; and e-commerce is producing higher and higher volumes of Web data that must be collected, transformed, loaded and analyzed. In the past, data was loaded into or extracted from applications in large, structured batches at predetermined times. There was no flexibility and hardly any extensibility in terms of what types of data could be moved or what languages the data-movement (ETL) tools could handle. Initial ETL tools occupied this cloistered data universe – and they worked well in it.
The new IT model requires that information logistics management move beyond the basic process of integrating data from disparate sources and be able to extract, transform and load both structured and unstructured data in batch and near real-time mode. To meet the need, vendors are transforming their tools into an integration technology that solves issues at the data level and beyond. Some ETL vendors are making their tools critical to the process of sharing data from data warehousing to business-to-business (B2B). They are developing a new generation of ETL tools that provide high-level functionality and performance as well as user-friendly features such as Web access capabilities.
The most important new improvement in ETL tools has been the development of extensible architectures. Essentially, extensibility means that the ETL tool has the capability to evolve as the technology it serves changes. Some benefits of extensibility are:
The ability to generate extract, transform and load programs for legacy-based applications (IMS, COBOL, C, etc.) as well as newer environments such as C++, XML, Java, etc.
The capacity to extract and load data in both batch and near real-time mode, directly or via interfaces to messaging software such as IBM’s MQ Series.
Multi-environment functionality through extensive customization capability. This capability is especially important to meet the needs of ever-changing protocols and network environments.
The ability to perform source-to-target data mapping, as well as to quickly and easily program transformations using built-in mechanisms that collect meta data and create data documentation integrally with the development process.
The key element that these features have in common is easier access and ability to move critical data to and from data-craving applications and their business unit owners in common. However, while these ETL tools provide the underpinnings for the new information logistics management model, they do not function in a vacuum. They function best as enablers for the new generation of EAI and data analysis tools.
Even EAI tools need a fresh look. Like ETL tools, EAI tools aren’t the complete answer to better business integration and data analysis. However, relying on EAI to correct all IT problems has disappointed many organizations that have jumped on the EAI bandwagon. No tool by itself is an easy fix for problems. EAI tools have to provide high-level functionality and integrate easily with other pieces of the technical architecture to provide a complete, seamless solution. There are two required elements for new-generation EAI tools: a holistic integration model and easy integration with trading partners.
Best-of-breed EAI tools will include most, if not all, of the following features. Specifically they will:
Enable workflow automation for customer-facing processes – such as tier-2 customer support – that are still largely performed by humans to create a holistic view of processes and supporting IT applications.
Provide vendor partners with the ability to develop and integrate a virtual marketplace, using the Web as a vehicle.
Enable integration – through adapters, XML or messaging technologies – of trading-partner systems so that information can be distributed to those partners that do not have mature B2B commerce systems.
Provide the ability to collect and share meta data and create data documentation integrally with the development process.
Extensive integration capability is the common theme of the functionality set. EAI tools that offer a combination of most of the integrative features listed will provide the most collaborative possibilities for integrating any information logistics model. These EAI tools will push the integration envelope. They will enable partner networks to build technical architectures that move beyond the borders of the host organization and integrate selected, critical vendor-partner applications to form a seamless information logistics- monitoring and management network.
Moving and integrating data is still not enough to build an optimally integrated information logistics management structure. To really tap the power of supply chain related data, organizations must also integrate and implement sophisticated data management and analysis tools into their information logistics-management technical architectures. Today’s best BA tools provide the backbone for increasingly complex analysis and efficient integration of trading partners, irrespective of the technology platforms and architectures that individual partners may employ.
This new breed of BA tools provides access to real-time, Web-based applications to connect to the real-time databases that contain specific client profiles, information about the anticipated delivery cycles and schedules, production cycles for clients and sales to insure that demands can be met and inventory levels for clients and retailers.
Some capabilities of the new breed of BA tools include:
Ability to deploy data analysis solutions multiple environments and platforms such as Internet/intranet, client/server, mainframe or hybrids of Web, client/server and/or mainframes.
Advanced data mining functions such as patching algorithms for data anomalies or missing data, binning and clustering algorithms, model creation, multivariate forecasting, time-series and causal analysis, etc.
Deployment capabilities that enable the integration of custom, external and legacy applications into the analysis piece of the technical architecture. The best tools integrate via open APIs built on stable platforms such as Microsoft NT, UNIX, etc. This capability can be deployed over multiple platforms.
Deployment capabilities that extend to the Internet and/or intranet environment. This is the most critical functionality for inter-enterprise collaboration.
Provide the ability to collect and share meta data and create data documentation integrally with the development process.
These tools make the job of analyzing data easier and more efficient. The upside can be huge. The new breed of business analytic tools collect the interactions taking place and send them securely to corporate data warehouses for analysis and action. These BA tools “close the loop” on the information logistics management cycle. They provide vendors, hosts and customers the ability to analyze their integrated data with a view that spans organizations – not just internal departments.
How can the marriage of these new-generation ETL, EAI and BA tools enable such gains in efficiency? By facilitating the process of inter-enterprise collaboration. In essence, the process that once was fine-tuned to specific organizations must now become borderless. It must be flexible enough to move beyond organizational boundaries and navigate different levels and quantities of interaction between trading partners. It must also be complex and mature enough to function in a myriad of technical environments.
Not surprisingly, inter-enterprise collaboration begins with the planning and development process. In traditional EDI information exchange environments, trading partners produced massive volumes of information that they threw at their partners, hoping it would get there – and back – in some recognizable and usable format. The new generation ETL, EAI and BA tools – with their multilevel sharing, integration and analysis capabilities – will enable trading partners to use shared information to coordinate product planning and development as well as resource allocation to meet each other’s needs.
The new generation ETL tools provide better access to data, EAI tools provide better integration of IT applications and BA tools provide more mature and complex analysis capability. These tools make it possible to predict financially impacting shifts in demand or other market conditions that before seemed like anomalies or sporadic fluctuations. What’s more, trading partners can share information about trends that are specific to their organization and that may have a ripple effect across the entire supply chain. Once these trends are identified, however, the trading partners – who have the access to the information they need to really collaborate – can brainstorm a solution to the problem. Figure 1 represents an information architecture that facilitates extraprise collaboration.
Figure 1: Information Architecture for Extraprise Collaboration
These tools will also generate savings – mostly in the area of inventory management. It has been estimated that a typical manufacturer with revenues of $1 billion and inventory of $250 million can reduce his or her inventory level by as much as $150 million to $100 million or below with efficient, integrated supply chain management.3 How could such a huge savings be possible? By making the entire supply chain more efficient through the use of more accurate and timely information.
Better ability to slice and dice data enables more complex analysis of shipping methods and levels in order to minimize shipping costs. Inventory cycle-counting programs can be honed to match inventory items with proper counting cycles. Finally, with the collaboration ability provided via Web portals, more accurate production estimates and order schedules can be provided to trading partners in almost real time, and master production schedules can be tightened in terms of accuracy – both in numbers and time estimates.
The current downturn in the economy has hit hard. Not since the mid-1980s has there been more of a need to make the most of available resources. With the new crop of ETL, EAI and BA tools, that task has become easier. Implementing the proper architecture for efficient, integrated information logistics management won’t be cheap, but with a potential upside in the hundreds of millions of dollars, it sure is a good investment.
1. Cecere, Lora and Karen Peterson. “Supply-Chain Collaboration Defined.” Gartner. 14 Aug. 2000. http://www4.gartner.com/Init.
2. Teagarden, Gretchen. WEBM: Initiating Coverage. 5 Sep. 2000. Salomon Smith Barney.
3. Goldstein, Alan. “Assembling Wealth: i2 Founder’s Factory Software Fills High-Demand Niche.” Dallas Morning News. 6 Oct. 1997: 1D, 4D.
Jane Griffin is a Deloitte Consulting LLP partner. Griffin has designed and built business intelligence solutions and data warehouses for clients in numerous industries. She may be reached via e-mail at email@example.com.
Copyright 2007, SourceMedia and DM Review.