Reading time: 9 minutes
ESB Enterprise Service Bus – a bus to the destination
30 / 08 / 2016
Data is among the company’s most valuable resources right after humans. It is essential to the proper operation and profitability of the company.
Data collection is half the battle, as it should be used appropriately, keeping in mind that it is often distributed due to the variety of systems which support the business. In such a case one should consider the support of the ESB Enterprise Service Bus, which enables the efficient transfer of data between applications.
Where and why should you use an Enterprise Service Bus?
Enterprises often use different systems to support their business. The more complex the business, the more the need for the implementation of various parallel solutions. Just imagine the example of a sales company. It usually needs tools to support product management by PIM, sales by ERP, warehouse and freight forwarding by WMS, human resources by CRM, and often manages the website content through the CMS system or an online store through the e-commerce system. In addition, mobile applications are also more and more needed.
Of course, these systems can exist in various other combinations and configurations depending on the type of business. However, there is always the problem of data integration. If we have so many parallel systems working for one organization, we should, and even have to, make them effectively cooperate with one another, as they usually need various information coming from each of them.
However, redundancy and storing the same data in different locations is difficult to maintain and highly error-prone. Ideally, such applications or systems exchange data in a relatively quick and simple way. Being able to seamlessly integrate further systems would be a good thing as well…
How does ESB work?
Does it make sense to connect each pair of applications or systems separately for the purpose of data exchange? Let’s look at an example in our environment. Do the Internet cables run separately from the provider to each building where access to the Internet is needed? No, because it is just not economical.
The line is laid out along the buildings which need to access it. Not every building needs a separate line directly from the provider. It can use one common line, simply by connecting it using a suitable connector. Over time, other buildings can also connect to such a line and use the same link.
By analogy, buildings are the equivalent of applications that need access to data exchange. They can receive and transmit data without the need for a direct cable connection to each recipient. There is no need to connect them directly in pairs, because you can use one common data exchange medium where more applications which need data exchange will be able to connect in the future. ESB (Enterprise Service Bus) meets these types of expectations.
Let’s look at two ways of data exchange to understand their operation. The first is data exchange via point-to-point connections, the other uses the data bus. You can see the difference at first glance: the complexity of processes in the case of direct connections versus simplified and minimized processes using ESB.
Even more benefits of an ESB
The communication between applications is homogenous in the case of an Enterprise Service Bus implementation. They do not have to contact one another directly in a variety of configurations, because that’s what a data bus is for. You do not have to integrate every application with all those with which it should interoperate. Just one integration of the application with the bus is enough to exchange data with other integrated applications.
ESB is also a great way to integrate data between desktop and mobile systems. Mobile applications can become part of the business and internal systems via the bus. On one hand, ESB can support some part of the logic, e.g. transforms, and on the other hand, the application integrated into the bus can handle a given logic. This solution enables the building of a system compatible with SOA (Service-Oriented Architecture), which meets business needs and cuts down on user intervention in the technical details of the system. Services provided by the system can operate independently, but also carry out highly complex business processes when exchanging data.
Modification, extension or appending successive system elements does not involve much time and effort, because it is enough to integrate the new application using the bus based on any data exchange protocol. New connections need to be established between applications, end-points, and the ESB data bus after the implementation of the bus. You will probably need to leave the integrated systems and external applications in the currently used form of communication due to the requirements of partners and existing integrations, although there is the option of integration via ESB.
It should be noted that the diversity of connectors and flexibility of the bus enables the connection of even archaic systems, which exist in virtually every company which does business long enough. Legacy systems, despite the lack of reasonable interfaces, can be plugged into the bus and “revived”, or allow easy transfer of data to newer systems. All this is due to the fact that ESB enables the creation of standardized APIs for applications which do not have them.
Some more things to keep in mind before implementing an Enterprise Service Bus:
- The use of ESB becomes reasonable when integrating three or more applications/services (it is better to use point-to-point connectors for two).
- Reckless implementation of “future-ready” solutions is not a good idea, because you may find out that nobody will use them at all.
- The implementation of the ESB will bring about the greatest benefits if using multiple communication protocols.
- It is worth considering whether we need the messages routing capability, such as forking and aggregating the flow of messages, or a content-based routing.
- ESB usually provides a robust and scalable set of services ready for use by other applications. It is good to identify the needs in this area.
- It is better to avoid the implementation of the integration with more applications from the very beginning, but start with 3-4 systems and add other ones successively. This will allow you to avoid struggling with all problems at once and solve them gradually to facilitate the implementation of further integrations.
Like any new solution, the use of Enterprise Data Bus also involves consecutive stages of the implementation process, such as:
- Launching the bus with several major integrations
- Bus testing, resolving errors
- Extending the bus by subsequent integrations
- Bus testing/resolving errors
- Acceptance tests of the complete implementation
- Production roll-out of the bus
– Bus maintenance
– Maintenance of the bus documentation
– Integrating new applications
First stage — analysis, is particularly important because it determines the success of the entire implementation. Your plan should be as following:
- Gathering requirements
- Specification of system requirements
- Assessment of the reasonableness of implementation
- Non-functional analysis
- Assessment of the feasibility of implementation
- Analysis and developing of the information model at the company (what information is created and where, why and how it is related, processed, and transmitted)
- Determining where the original (source, reference) versions of information are and where the secondary ones (e.g. copies) are
- Organizing data exchange by bringing about the hierarchical network structure (source of reference and users)
- Determining the discovered old and unused applications and those with duplicate functions
- Functional analysis
- Disabling unnecessary applications or excluding them from integration
- Determination of connectors and the data exchange logic for all applications integrated with the bus
- Implementation support/troubleshooting
- Keeping documentation on changes/extensions/integrations of successive system.
Are you looking for the best solution for you? Consider Mule ESB!
There are a number of data bus solutions. Mule ESB is a distinctive one, and it deserves special attention. It is an open-source tool, but its quality, functionality and reliability are as high as in enterprise-class systems. Mule ESB is a powerful framework allowing for the implementation of ESB architecture.
By using it you can create a scalable data bus which supports multiple integrated applications that you can connect using a variety of communication protocols. Integrated applications can be end-points here, for example, via Web services, databases, JMS, e-mail (POP3, SMTP, IMAP), TCP/UDP sockets, FTP, HTTP, file system or XMP protocol.
Data exchange may also be carried out by different types. The flexibility of Mule ESB provides room for the integration of various components. The fact that the solution has been designed using proven design standards, e.g. SEDA (Staged Event Driven Architecture) to enable efficient data processing at high server load is also an advantage.
In addition, Mule ESB has a number of features enabling the automation of work and advanced configuration. These include:
- Data transformers
- Automatic message queuing
- Automatic creating and maintaining of component pools enabling parallel message processing
- Extended message routing options
- Extended message filtering
Mule ESB is a tool with an established strong market position. It is available for free (Community) and as a paid version (Enterprise). But do not be discouraged by the free access to the Mule ESB Community solution, because it is a very high-end tool offering a wide range of features in spite of not having to be paid for.
More demanding customers, particularly those expecting some additional, advanced features and fast and extensive technical support by the manufacturer, can choose the Mule ESB Enterprise solution. It has additional advanced features in addition to support, including: configuration and management streamlining tools (e.g. DataMapper), tools for additional analysis and statistics, additional controlling tools, more enhanced security features and additional ready-made connectors (e.g. SAP Connector). When making the decision, you can check out the general comparison of both versions.
The vendor also enables change from the Community version to Enterprise, because the latter is based on the former, expanding it by additional elements.
Figure 4: An example of Mule ESB usage
In a few words
Given the growth of the enterprise and the scale of deployment of new applications or entire systems, it is worthwhile to seriously consider the implementation of a data bus. This translates to lower maintenance and implementation costs of new systems, streamlines the data flow and reduces the risk of errors. The ESB bus also provides great opportunities for integration with various entities and dynamic expansion.
However, do not implement the data bus rashly and without consideration, because it becomes one of the most important elements of the entire architecture.
Data buses in the form of Mule ESB have been used successfully in implementations by Grupa Unity at large companies which rationally consider their actions and think forward. Conscientious clients, such as Leroy Merlin, Volkswagen Group Polska or TADO appreciated the advantage of the medium, which consolidates the flow of information between their many advanced systems.