Understanding the Network Delivery Layer (NDL)

PubSub Engines running on different computers within a single system have a peer-to-peer relationship, never connecting directly with each other over the network. Instead, communication is based on the User Datagram Protocol (UDP), which is a connectionless multicast (that is, "broadcast") protocol that is fast but does not guarantee message delivery or order. This broadcast traffic is encrypted so that only PubSub Engines within the same system can understand the messages sent between peers.

When a PubSub Engine receives a subscription from a connected subscriber but cannot find it in the cache of local publications or identify it as registered to a local persistence store, the engine assumes there is no local publisher. It broadcasts the subscription to its peers using the Network Delivery Layer (NDL) to see if any of them can handle it. When the other engines receive the subscription, they record the subscription as coming from a peer rather than from a directly connected subscriber.

If a receiving peer finds a match in its cache of locally published data and an integrity update was requested, the data can be published immediately using the NDL. When the peer who sent the subscription receives the broadcast publication, it forwards the publication to its subscriber using the Local Delivery Layer (LDL).

If not found in the cache, the peer PubSub Engine does nothing until a local publisher publishes on the topic. When this happens—and for all but one engine in the system, it should never happen—the engine broadcasts the publication to its peers.

The below diagram illustrates this scenario where the publisher and subscriber are running on different computers:

Note: Engines are free to respond to or ignore the messages received from their peers. For example, an engine will never broadcast a local publication to the network unless it has first received a subscription for the topic from one of its peers.

If the network is busy, peer messages may be lost or received out of order. For this reason, the messages include a unique sequence number generated by the transmitting peer, which can be used to detect mis-ordered or dropped messages. Messages are buffered so that if they are received in the wrong order they can be reordered automatically. While processing the messages from each peer, if the PubSub Engine notices a message was missed, it will broadcast a request to the peer to resend the messages starting from the oldest missed message.

PubSub communications between connected AVEVA Enterprise SCADA systems is the responsibility of the Inter-System Publish/Subscribe (ISPS) agents, not the NDL. The NDL cannot be used, because each system is configured for a different multicast address. In addition, multicast traffic between systems is frequently blocked by firewall rules.