| Lesson 5 | Data integration middleware |
| Objective | Describe the function of data integration middleware |
Data Integration Middleware
What is Data Integration Middleware?
Data integration middleware is a software layer that enables different applications and data sources to communicate and share data with each other. It is designed to overcome the challenges of data fragmentation, where data is stored in different formats, locations, and systems, making it difficult to access and use effectively. Data integration middleware provides a set of tools and services that enable developers to connect different data sources, map data between different formats, transform and enrich data, and synchronize data across different systems. It helps to ensure that data is consistent, accurate, and up-to-date, regardless of where it is stored or how it is accessed. Some common features of data integration middleware include:
Data integration middleware provides services that allow applications and development tools to view different underlying data structures and data sources as a single data source. This means that an application is not affected by physical implementation of data structure. The illustration below explains why this is important:
- Data Mapping: Tools that enable developers to map data between different formats, such as between different databases or data warehouses.
- Data Transformation: Tools that allow developers to transform and enrich data as it moves between different systems, such as by adding new fields or calculating new values.
- Data Quality: Services that help to ensure that data is accurate, consistent, and conforms to defined standards or rules.
- Data Synchronization: Tools that enable data to be synchronized between different systems, ensuring that changes made in one system are reflected in all others.
- Connectivity: Services that enable different applications and systems to connect and communicate with each other, regardless of the underlying technology or platform.
Data integration middleware provides services that allow applications and development tools to view different underlying data structures and data sources as a single data source. This means that an application is not affected by physical implementation of data structure. The illustration below explains why this is important:
| A table can be created within a specific database, but to the user it will look like any database table. | Data integration middleware can also make multiple tables look as if they are one. |
| Like other middleware tools, data integration middleware abstracts the details of underlying constructs such as data constructs, and it makes it easier for the software layer above (and the user) to use those constructs. Undoubtedly the most popular method of accessing data through this abstracted layer is throughODBC[1] JDBC[2]. | |
Remote Data Integration
Just as RPCs deal with process interoperability between remote programs, there are products, known as
SQL Gateways, that support data integration remotely.
SQL gateways are useful when dealing with applications that have a high number of temporarily connected users and that possess heterogeneous database sources. This is frequently true of B2C applications. Many application servers now possess a similar form of SQL gateway functionality.
The next lesson discusses distributed object middleware.
The next lesson discusses distributed object middleware.
[1]
ODBC(Open Database Connectivity): OBDC inserts a middle layer, better known as a database driver, making it possible to
access any data from any application, regardless of which database management system handles the data. For this to occur, the application must be capable of issuing ODBC commands and the database management system must be capable of responding to them.
[2]
JDBC (Java Database Connectivity): A programming interface that lets Java applications access a database via the SQL language. Since Java interpreters (Java Virtual Machines) are available for all major client platforms, this allows a platform-independent database application to be written. In 1996, JDBC was the first extension to the Java platform. JDBC is the Java counterpart of Microsoft's ODBC.