How to run OPC UA stack open62541 with Realtime PubSub
on Realtime Linux and TSN using Intel I210 Ethernet controller
This quick start guide serves as a starting point for a user in learning/evaluating OPC UA including TSN technology
for their products/projects.
This quick start guide uses “Open Source OPC UA stack open62541 with Pub/Sub feature” and leverages the TSN features
available on “standard Linux kernel + PREMPT_RT patches” on an x86 system with Intel I210 Ethernet controller
Prerequisites
Hardware requirements:
Two 4-core Intel x86-based systems (e.g. Apollo Lake or Whiskey Lake architecture) with I210 Ethernet controller.
The two nodes can be setup in one of the two network topologies as shown below

Software requirements:
Debian GNU/Linux 10.6 (buster: debian-10.6.0-amd64-xfce-CD-1 ) (Note: Please understand that we cannot provide individual version of the Quick Start Guide for the wide variety of available Linux distributions. We selected Debian 10, as it is one of the most popular distributions and known to be stable and suitable for our purpose.)
Run the following command to update the node after installing the OS
sudo apt-get update && apt-get upgrade
Download default RT image using the following command
sudo apt-get install linux-image-rt-amd64
Reboot the node and ensure that you land in the newly installed RT kernel
sudo reboot
Environment setup
After reboot you can check the kernel version by using the below command
uname -r
Sample output

Enter as root user
sudo su
To download the package for the demo
wget https://www.kalycito.com/wp-content/uploads/2021/01/demo_package.tar
Unzip the package using following command and enter into the demo_package folder
tar -xvf demo_package.tar && cd demo_package
The setup.sh available in the package does the following,
- Updates the installed packages in the node
- Installs iproute2 package for kernel version 4.19
- Installs Linux PTP version 2.0
- Checks if 8021q module is loaded, if not adds the same
Run the setup.sh script
./setup.sh
Setup static IP address to the south bound interface (I210) which is connected peer-to-peer between two nodes for PubSub communication
vi /etc/network/interfaces
Copy and paste the following lines in “/etc/network/interfaces” file and replace “X” with a suitable suffix number to setup IP address for I210 interface
In our case the interface name is enp2s0, replace this with your I210 interface name
Example: Node 1 IP address: 192.168.0.1, Node 2 IP address: 192.168.0.2
auto enp2s0
iface enp2s0 inet static
address 192.168.0.X
netmask 255.255.255.0
broadcast 192.168.0.255
auto enp2s0.8
iface enp2s0.8 inet static
address 192.168.8.X
netmask 255.255.255.0
Note: The PubSub TSN applications are configured to run with VLAN Id 8. For this purpose, VLAN configuration is done in the I210 interface.
An example interface file:

Restart the networking service using the following command
/etc/init.d/networking restart
Verify the IP address and connection between two nodes. From node 1 ping node 2 and check vice versa
ip a
ping 192.168.0.X
The application_dependencies.sh available in the package does the following,
- Configures traffic control parameters for transmit and receive
- Sets Egress and Ingress policies
- Tunes real-time behaviours
- Runs Linux PTP and PHC2SYS
To run the node as PTP master use the following command
./application_dependencies.sh -i <IFACE> -m
To run the node as PTP slave use the following command
./application_dependencies.sh -i <IFACE> -s
The following error occurs while running application_dependencies.sh, since no ntuple configuration will be available to delete initially.

The output log of PTP can be verified by giving the following command
tail -f /var/log/ptp4l.log
An example log file at PTP master:
An example log file of PTP slave:
To view PHC2SYS log:
tail -f /var/log/phc2sys.log
An example log file at PHC2SYS log:
To configure the node permanently as PTP master or PTP slave, use the following command to create a bootup script
To configure as PTP master:
echo ./<FILE_PATH>/application_dependencies.sh -i <IFACE> -m >> /etc/rc.local
(or) PTP slave:
echo ./<FILE_PATH>/application_dependencies.sh -i <IFACE> -s >> /etc/rc.local
To make the bootup script as executable
chmod +x /etc/rc.local
To execute the bootup script
/etc/rc.local
Note: From next reboot, the bootup script will be executed automatically.
To view rc.local log:
cat /var/log/rc.local.log
Test setup
Image below shows the test setup followed to run the PubSub application

You can follow either of the above listed network to run the application
Steps to run OPC UA PubSub application
Clone the open62541 stack with the following command
git clone https://github.com/open62541/open62541.git
Fetch the pull request and create new branch with the following command
cd open62541/
git fetch origin pull/3996/head:local_branch && git checkout local_branch
Image attached below shows the overview of the target applications

T1 – Ethernet packet publish timestamp at userspace
T2 – Ethernet packet timestamp at kernel space
T3 – Ethernet incoming packet timestamp at kernelspace
T4 – Ethernet packet subscribe timestamp at userspace
T5 – Ethernet loopback packet publish timestamp at userspace
T6 – Ethernet loopback packet outgoing timestamp at kernelspace
T7 – Ethernet loopback packet incoming timestamp at kernelspace
T8 – Ethernet loopback packet subscriber timestamp at userspace
The application in Node 1 encodes and publishes a packet to Node 2, and the Node 2 decodes the received publish packet and loops back the same to make the published data finally available to the application at Node 1. The application at Node 1 finally computes the round-trip time (T8-T1) it takes to publish and receive back the same data. To keep track of the sequence in which the packets were sent and received at both ends, the payload in the OPC UA PubSub publish packet contains a counter variable that increments in every application cycle
Follow the steps given below to build the application,
mkdir build
cd build/
cmake -DUA_BUILD_EXAMPLES=ON -DUA_ENABLE_PUBSUB=ON -DUA_ENABLE_PUBSUB_ETH_UADP=ON ..
The following error in CMake build is expected and can be ignored

Build the application by giving the following command in open62541/build directory
make -j4
Note: To build a single sample, you can use “make -j4 <Sample_name>”. For example, make -j4 pubsub_TSN_loopback
Running OPC UA Pub/Sub application
Note: It is always recommended to run pubsub_TSN_loopback application first to avoid missing the initial data published by pubsub_TSN_publisher.
Run the Loopback application in Node 2 by entering the following command
./bin/examples/pubsub_TSN_loopback -interface <IFACE> -enableBlockingSocket
Run the Publisher application in Node 1 by entering the following command
./bin/examples/pubsub_TSN_publisher -interface <IFACE> -enableBlockingSocket -enableLatencyCsvLog
Note: -enableLatencyCsvLog calculates the RTT of the counterData that is published and received back. Since RTT is calculated by the pubsub_TSN_publisher application, -enableLatencyCsvLog is not available in pubsub_TSN_loopback application.
If -enableLatencyCsvLog is enabled, the latencyT1toT8.csv file will be generated in the build folder (only after the application is terminated), containing the RTT latency, missed counters and repeated counters. And the application can be terminated by using “Ctrl + C”.
An example latencyT1toT8.csv log file will look as shown below.

To visualize the data, the csv file should be imported into a statistical computing program such as R and the data displayed as histogram along with minimum and maximum values. A script mkloghisto.R is part of the demo package. To install R and execute the script in the system, follow the below commands
apt-get install r-base
Rscript <FILE_PATH>/mkloghisto.R latencyT1toT8.csv
The graph will be generated in the latencyT1toT8-log.pdf file in the current directory. As the application is run in 250us, the expected RTT latency is 1ms (4x cycletime). The x-axis in the graph shows the deviation of the obtained RTT latency from 1ms and y-axis shows the number of samples. A sample graph is shown below, where the obtained RTT latency deviates from 1ms in the range of -31 to 32 us.

Optional Steps:
To know more usage
./bin/examples/pubsub_TSN_loopback -help
./bin/examples/pubsub_TSN_publisher -help