This the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Design Details

1: Topology

Intercompany (Master) Data Management enables Business Central Companies to share data. Data is serialized to Messages and organized in Topics by the Publisher. The Subscriber can then setup Subscriptions on the Topics thereby receive the data.

This section will cover the underlying thoughts and architecture behind IDM.

Messaging pattern

Data is transferred between Companies in so called Messages. A Message that is made available in a Topic by a Publisher is referred to as a Topic Message and a Message that has been received by a Subscriber is known as a Subscription Message.

IDM has been built with inspiration from the Publish-Subscribe pattern as well as the Client-Server pattern making it a hybrid that benefits from strengths in both models. The main sequence for new data to be published is as follows.

User changes some data.
Data change is registered by Publisher.
Publisher serializes changed data into Messages.
Publisher notifies Subscriber about new Messages.
Subscriber requests new Messages from Publisher.
Publisher returns new Messages to Subscriber.
Subscriber saves each received Message.
Subscriber registers information about consumed Messages with Publisher.
Subscriber deserializes and processes each Message.

The above flow is illustrated in the following sequence diagram. Messaging Pattern

The main sequence is a hybrid of Publish-Subscribe and Client-Server and the following table lists key features from each model which have been implemented in the general Messaging Pattern.

Pattern	Highlights
Publish-Subscribe	Publisher serializes data to Messages without direct data transfer. Publisher organizes Messages in Topics. A Topic can have multiple Subscribers. Publisher immediately notifies Subscribers. Subscribers registers Message consumption with Publisher.
Client-Server	Messages are transferred by request of Subscriber. Data serialization and Message Processing is aligned between Topic and Subscription. Publisher has knowledge about each Subscriber. Publisher and Topic is kept within the same System. A Publisher can only post Messages to its own Topic.

Messages are organized in Topics that are made available to multiple Subscribers. A Publisher posts new messages to the Topic and notifies Subscribers as illustrated on the following flow chart.

Publish-Subscribe

This allows for a decoupling between publishing new data and consumption, where Subscribers will receive the new data. Data is serialized and stored as Topic Messages before notifying Subscribers which has the following benefits.

Messages can quickly be returned to Subscribers when needed as no further processing is required.
Messages can be marked to only be published to a subset of Subscribers.

Each Topic can have multiple Subscribers and the Publisher does not necessarily need to know each Subscriber beforehand.

Client-Server

Data is only transferred when Subscribers requests new Messages from Publisher as shown the below illustration.

Client-Server

In this way the Subscriber can decide when data transfer should take place. The Topic is placed in the Publisher system to allow for the Topic to contain full information about the communications protocol which is managed by the Message Broker.

Message Broker

Functionality in IDM is grouped into the following three main areas.

Area	Description
Data Serialization	Convert data to and from text.
Data Transfer	Notify Subscriber and transfer Message.
Message Processing	Parse and map Message to data.

IDM has been implemented with extensibility in mind and functions involving these three areas are placed in Codeunit Message Broker IDMYME. In this way it is possible to use any of the predefined interfaces as well as have a partner implement new ones to fit your specific needs while still making use of the overall IDM framework. Each implementation consists of a number of explicitly required functions defined by the AL interface as well as implicitly required Subscriber functions which allows for additional features and differentiations to be implemented with fallback to default functions.

The following sections will describe each of the three interfaces in detail.

Data Serialization

In Business Central data is stored as records in tables and can not be directly transferred to other Companies. Therefore when data is saved as Messages by the Publisher it is first serialized to text and when Messages are processed by the Subscriber it is deserialized back from text to data.

Serialization can be performed in a number of ways and the key point is that serialization and deserialization should be done in more or less the same way when both Publisher and Subscriber is a Business Central Company. IDM therefore provides an interface for Serialization that can both be defined on Topic and Subscription to ensure consistency during integration.

The technical name for the Data Serialization interface is IData Serialization IDMYME and the following table outlines it from a programming point of view.

Function	Type	Area	Description
Serialize()	Interface	Serialization	Invoked when data is serialized.
Deserialize()	Interface	Serialization	Invoked when data is deserialized.

IDM comes with a default Json Serialization but the Data Serialization interface can be extended to match your specific needs.

Data Transfer

In IDM, Data Transfer consists of three areas:

Area	Description
Notification	Publisher sends notification to Subscriber that new Messages have been published.
Topic Metadata	Subscriber receives Topic Metadata from Publisher to understand the Message context.
Message	Subscriber receives Topic Messages from Publisher and saves it as Subscription Messages.

In all three areas, both Publisher and Subscriber must request and respond accordingly to ensure consistency. To help this along Data Transfer has been implemented in a single interface, IData Transfer IDMYME, that is defined on Parties (IDM) in by both Publisher and Subscriber.

and the following table outlines it from a programming point of view.

Function	Type	Area	Description
NotifySubscriber()	Interface	Notification	Invoked when a Publisher notifies a Subscriber about new Messages.
FindTopics()	Interface	Topic Metadata	Invoked when Subscriber lists Topics from a Publisher.
FindICParties()	Interface	Topic Metadata	Invoked when Subscriber lists Parties (IDM) from a Publisher.
RequestNewMessages()	Interface	Message	Invoked when a Subscriber requests new Messages from a Publisher.
RegisterConsumedMessages()	Interface	Message	Invoked when a Subscriber registers Message Consumption with a Publisher.
GetTopicTables()	Interface	Topic Metadata	Invoked when a Subscriber lists Topic Tables from a Publisher.

Message Processing

Each Subscription Message is processed by the Subscriber and the default behaviour is to map tables and fields more or less 1:1 resulting in data from Publisher being replicated to Subscriber. However, you might require some specific Message Processing where underlying data or procedures are taking into use and to support this the Message Processing interface has been provided.

The technical name for the Message Processing interface is IMessage Processing IDMYME and the following table outlines it from a programming point of view.

Function	Type	Area	Description
ProcessMessage()	Interface	Message Processing	Invoked when Message Processing is triggered.
OnInitSubscriptionTables()	Subscriber	Subscription Metadata	Function OnInitSubscriptionTables in Codeunit Message Broker IDMYME is invoked whenever a Subscription initiates a Subscription Table from Topic Table metadata received from the Publisher. This function is optional and the default initiation will be used if an interface does not implement this function.
OnInitTargetField2SourceField()	Subscriber	Subscription Metadata	Function OnInitTargetField2SourceField in Codeunit Message Broker IDMYME is invoked whenever a Subscription Field is initiated from a Source Field. This function is optional and the default initiation will be used if an interface does not implement this function.

Enum Message Processing Type

The Role of the Publisher

A Publisher is a Party where a Topic has been created in the its company. The Publisher posts data to a Topic and notifies Subscribers about new Messages. When Subscribers requests new Messages then it is the Publisher that responds to those requests and returns new Topic Messages to Subscribers as well as Topic Metadata.

Topic

Published data is organized in Topics which have the following key features.

Feature	Description
Data content	Tables and Fields to be included is defined on each Topic.
Subscription Filter	Topic can be defined to only allow Subscription from some Subscribers.
Message Filter	Topic Messages can be filtered to only be available to some Subscribers.
Metadata	Topic can provide Metadata to Subscribers to ease the process of setting up new Subscriptions.
Message Queue	Published data is saved as Topic Messages to allow for immediate delivery upon demand where it is returned to Subscribers is chronological order.
Notification	Information about notification sent to Subscribers is stored in Topic Register.

The Role of the Subscriber

A Party where a Subscription has been created in its company is considered as being a Subscriber. The Subscriber requests new Messages from a Topic at a Publisher and each Message is then saved and later on processed. When receipt is completed then the Subscriber register receipt of Messages back to the Publisher.

Subscription

When a Subscriber wishes to receive data from a Topic a Subscription is created meaning there is a direct correlation between a Topic from a Publisher and a Subscription. Subscription has the following key features.

Feature	Description
Topic	Subscription contains information about Publisher and Topic.
Target data	Target Tables and -Fields is defined on each Subscription with a mapping from the Topics data content which is referred to as Source Tables and -Fields.
Message Queue	Received data is saved as Subscription Messages to allow for asynchronous Message Processing in chronological order.
Consumption	Information about last Message received from Publisher is stored in Subscription Register.

1 - Topology

Learn about different configuration of multiple companies in multiple environments

Single environment

Hub and spoke

flowchart TB
    Contoso --> Fabricam
    Contoso --> Northwind
    Contoso --> ProseWare

Mesh

flowchart LR
    Contoso <--> Fabricam
    Contoso --> Northwind 
    Fabricam --> ProseWare   
    Northwind <--> ProseWare
    Northwind --> Fabricam
    Contoso --> ProseWare

flowchart TB
    Contoso --Chart of Accounts--> Fabricam
    ProseWare --Items--> Northwind 
    Contoso --Chart of Accounts--> Northwind     
    Fabricam --Items--> ProseWare       
    Northwind --Resources--> ProseWare    
    Northwind --Resources--> Fabricam    
    Contoso --Chart of Accounts--> ProseWare    
    ProseWare --Items--> Contoso

Multiple Environments

flowchart TB
    subgraph BC_Env_US
        direction LR
        Contoso 
        Fabricam
    end
    subgraph BC_Env_GB
        direction LR
        Northwind 
        ProseWare
    end
    ASB[(Azure Service Bus)]    
    Contoso<-->ASB
    Fabricam<-->ASB
    Northwind<-->ASB
    ProseWare<-->ASB