Interoperability: The challenges of connecting heterogeneous systems
Chief Technology Officer of CargoX
Abstract: On defining different types of interoperability and how they affect connections between companies, governments, and systems.
Keywords: interoperability, technical, semantic, ontology, blockchain, IT, legal
Interoperability has always been a challenge. The same issue repeats itself over and over in a variety of contexts, be it in legacy industries, or new and exciting technologies. Common cases of interoperability (or a lack thereof) have affected businesses and individuals alike. Slightly older readers will remember the VHS vs. Betamax war and the HD-DVD vs Blu-ray debacle. Sometimes, interoperability is hindered on purpose to lock users into a specific ecosystem. And that is the reason you cannot FaceTime on an Android phone or Chromecast from an Apple device.
The biggest challenges in interoperability, however, arise in the business sector. They appear when companies, governments, or systems within one entity need to interact with one another.
This article aims to explore the different types of interoperability, why they exist, and the best ways to solve the issues surrounding them.
Interoperability challenges stem from two situations:
When the same idea or concept is developed independently in different parts of the world and both gain momentum. Now two or more systems with similar ideas behind need to interact. A typical example would be healthcare informatics, where years were needed to come up with a standard to exchange data.
With technological leaps, where being the first to market is more important than connecting with existing systems. This has happened, for example, with blockchain technologies. To this day we still do not have a great way to share information between two distinct blockchains.
Both cases lead to the same challenge – getting the distinct systems to cooperate.
The term "interoperability" was coined in the late 1960s. It is defined as “capable of being used or operated reciprocally”. The New European Interoperability Framework interprets the interoperability on a more business level as “the ability of organizations to interact towards mutually beneficial goals.”
The need for interoperability came with the advent of computing, and the need to exchange data between discreet systems. Before the first computers, interoperability was not the issue, as people are adroit and apt to understanding non-structured data.
Interoperable systems optimize information exchange. Without interoperability, data needs to be exported from one system and – most often than not – typed into another system manually. By connecting the systems, the whole chain can function faster. By reducing manual labor, costs are slashed and issues stemming from human error (omits, typos, and misinformation) are reduced to practically zero.
Types of interoperability
The definition of interoperability has grown over the years and does not solely rely on the act of “technical” connectivity alone. Various authors structure interoperability into different levels. Let us give a few examples.
Aris Ouksel and Amit Sheth divide interoperability into:
semantic, defined as “the ability to exchange “the meaning” of data, usually based on a set of high-level data interchange standards,”
structural, defined as “compatibility of schemas and data-modeling constructs,”
syntactic, defined as “differences in machine-readable aspects of data representation and formatting,” and
system interoperability, defined as “the ability of two systems to connect”.
The IDEAS framework defines three levels of interoperability between co-operating enterprises:
The new European Interoperability Framework (EIF) (EU) defines interoperability as:
legal, “ensuring that organizations operating under different legal frameworks, policies, and strategies are able to work together,”
organizational, “the way in which public administrations align their business processes, responsibilities, and expectations to achieve commonly agreed and mutually beneficial goals. In practice, organizational interoperability means documenting and integrating or aligning business processes and relevant information exchanged,”
semantic, “the precise format and meaning of exchanged data and information is preserved and understood throughout exchanges between parties, in other words ‘what is sent is what is understood,” and
technical interoperability which “covers the applications and infrastructures linking systems and services. Aspects of technical interoperability include interface specifications, interconnection services, data integration services, data presentation and exchange, and secure communication protocols".
Yet UN/CEFACT speaks about:
Looking at all these definitions, a common theme emerges: interoperability is hard. And with new technologies and the explosion of new actors and systems, it is only going to get more complex.
To sum up: looking at interoperability from the bird’s-eye view perspective, the authors define interoperability on three levels, as discussed below.
While technical interoperability was a great issue in the past, it is much less of a challenge than it used to be. Network connections have pretty much standardized to TCP/IP. AppleTalk, IPX, and similar protocols have gone the way of the Dodo. Level 7 connectivity relies almost exclusively on HTTP.
We can still find older protocols hanging around, such as X.25 and EDI. They are, however, being rapidly replaced with the technologies mentioned, as they are much simpler to implement, integrate, and maintain.
While there are several standards for web services, the world has pretty much settled on REST and JSON. Some older services still rely on SOAP/XML, though.
Practically all operating systems and programming languages in use today support this technology stack, and technical interoperability ought to be pretty much solved.
Alas, this was the case until blockchain came along. Blockchain, by design, is an immutable and “closed-circuit” technology: it cannot query the outside world. It can operate only on the data contained within. While the problem is being mitigated with the use of oracles, we have yet to find an efficient solution to this challenge.
While connectivity seems trivially easy nowadays, understanding transferred data is a completely different challenge. People are extremely good at deciphering complex data structures. For example, we can pretty much understand a well-known form (e.g. a bill of lading), even if the fields are labeled in a language we do not understand. For computers, though, this is a challenge.
That is why it’s essential to have standardization bodies (such as UN/CEFACT, ISO, ANSI, NIST, DCSA, W3C – to name a few), where interested parties can come together and define a schema for the given ontology.
This schema defines the meaning of data: which fields exist, and what their meaning and allowed content are.
It is essential, though, that these governing bodies do not fall into the fallacy of relying on (too many) technical standards to define the semantics. Too often the standards prefer one technology. Consequently, these standards age more quickly and become obsolete sooner.
Organizational and legal interoperability
Interestingly enough, even if organizations can exchange data and computers can understand the data, the old ways still persist. This is largely due to a missing legal framework.
A good example would be the bill of lading: while most countries have accepted the notion that the document may be presented in an electronic form (Given that certain security prerequisites are met), some laws still stipulate that this must be a paper document. In other cases, some countries (or organizations) might require certain data to be present on the document, while the origin party simply doesn’t collect or even have access to this data.
The sad truth is that organizational and legal interoperability is usually the hardest nut to crack, and is what most often takes the longest to resolve. An external push can help, though. It is just that lately, due to the COVID-19 pandemic, India has shifted gears and focused on electronic document interchange. In a surprising move, Brazil has also opened its customs to accept the electronic bill of lading during this time.
Interoperability does present challenges but this is something that cannot be avoided.
We can generally structure it into technical (how), semantic (what) and legal (why) interoperability. Technical interoperability has the longest track record, but new challenges arise with every new technology. The semantic interoperability is essential for understanding the exchanged data. The legal interoperability is the most complex issue, as it requires the alignment of laws and regulations of multiple countries and governing bodies.
Your free demo session with our experts
Please fill out the form below, and we will get back to you as soon as possible.