How to run OPC UA stack open62541 with Realtime PubSub
on Realtime Linux and TSN using Intel I210 Ethernet controller

This quick start guide serves as a starting point for a user in learning/evaluating OPC UA including TSN technology
for their products/projects.

This quick start guide uses “Open Source OPC UA stack open62541 with Pub/Sub feature” and leverages the TSN features
available on “standard Linux kernel + PREMPT_RT patches” on an x86 system with Intel I210 Ethernet controller

Prerequisites

Hardware requirements:

Two 4-core Intel x86-based systems (e.g. Apollo Lake or Whiskey Lake architecture) with I210 Ethernet controller.

The two nodes can be setup in one of the two network topologies as shown below

Software requirements:

Debian GNU/Linux 10.6 (buster: debian-10.6.0-amd64-xfce-CD-1 ) (Note: Please understand that we cannot provide individual version of the Quick Start Guide for the wide variety of available Linux distributions. We selected Debian 10, as it is one of the most popular distributions and known to be stable and suitable for our purpose.)

Run the following command to update the node after installing the OS

sudo apt-get update && apt-get upgrade

Download default RT image using the following command

sudo apt-get install linux-image-rt-amd64

Reboot the node and ensure that you land in the newly installed RT kernel

sudo reboot

Environment setup

After reboot you can check the kernel version by using the below command

uname -r

Sample output

Enter as root user

sudo su

To download the package for the demo

wget https://www.kalycito.com/wp-content/uploads/2021/01/demo_package.tar

Unzip the package using following command and enter into the demo_package folder

tar -xvf demo_package.tar && cd demo_package

The setup.sh available in the package does the following,

Updates the installed packages in the node
Installs iproute2 package for kernel version 4.19
Installs Linux PTP version 2.0
Checks if 8021q module is loaded, if not adds the same

Run the setup.sh script

./setup.sh

Setup static IP address to the south bound interface (I210) which is connected peer-to-peer between two nodes for PubSub communication

vi /etc/network/interfaces

Copy and paste the following lines in “/etc/network/interfaces” file and replace “X” with a suitable suffix number to setup IP address for I210 interface

In our case the interface name is enp2s0, replace this with your I210 interface name

Example: Node 1 IP address: 192.168.0.1, Node 2 IP address: 192.168.0.2



auto enp2s0

iface enp2s0 inet static

    address 192.168.0.X

    netmask 255.255.255.0

    broadcast 192.168.0.255


auto enp2s0.8

iface enp2s0.8 inet static

    address 192.168.8.X

    netmask 255.255.255.0

Note: The PubSub TSN applications are configured to run with VLAN Id 8. For this purpose, VLAN configuration is done in the I210 interface.

An example interface file:

Restart the networking service using the following command

/etc/init.d/networking restart

Verify the IP address and connection between two nodes. From node 1 ping node 2 and check vice versa

ip a 

ping 192.168.0.X

The application_dependencies.sh available in the package does the following,

Configures traffic control parameters for transmit and receive
Sets Egress and Ingress policies
Tunes real-time behaviours
Runs Linux PTP and PHC2SYS

To run the node as PTP master use the following command

./application_dependencies.sh -i <IFACE> -m

To run the node as PTP slave use the following command

./application_dependencies.sh -i <IFACE> -s

The following error occurs while running application_dependencies.sh, since no ntuple configuration will be available to delete initially.

The output log of PTP can be verified by giving the following command

tail -f /var/log/ptp4l.log

An example log file at PTP master:

An example log file of PTP slave:

To view PHC2SYS log:

tail -f /var/log/phc2sys.log

An example log file at PHC2SYS log:

To configure the node permanently as PTP master or PTP slave, use the following command to create a bootup script

To configure as PTP master:

echo ./<FILE_PATH>/application_dependencies.sh -i <IFACE> -m >> /etc/rc.local

(or) PTP slave:

echo ./<FILE_PATH>/application_dependencies.sh -i <IFACE> -s >> /etc/rc.local

To make the bootup script as executable

chmod +x /etc/rc.local

To execute the bootup script

/etc/rc.local

Note: From next reboot, the bootup script will be executed automatically.

To view rc.local log:

cat /var/log/rc.local.log

Test setup

Image below shows the test setup followed to run the PubSub application

You can follow either of the above listed network to run the application

Steps to run OPC UA PubSub application

Clone the open62541 stack with the following command

git clone https://github.com/open62541/open62541.git

Fetch the pull request and create new branch with the following command

 cd open62541/ 

git fetch origin pull/3996/head:local_branch && git checkout local_branch

Image attached below shows the overview of the target applications

T1 – Ethernet packet publish timestamp at userspace

T2 – Ethernet packet timestamp at kernel space

T3 – Ethernet incoming packet timestamp at kernelspace

T4 – Ethernet packet subscribe timestamp at userspace

T5 – Ethernet loopback packet publish timestamp at userspace

T6 – Ethernet loopback packet outgoing timestamp at kernelspace

T7 – Ethernet loopback packet incoming timestamp at kernelspace

T8 – Ethernet loopback packet subscriber timestamp at userspace

The application in Node 1 encodes and publishes a packet to Node 2, and the Node 2 decodes the received publish packet and loops back the same to make the published data finally available to the application at Node 1. The application at Node 1 finally computes the round-trip time (T8-T1) it takes to publish and receive back the same data. To keep track of the sequence in which the packets were sent and received at both ends, the payload in the OPC UA PubSub publish packet contains a counter variable that increments in every application cycle

Follow the steps given below to build the application,

mkdir build

cd build/

 cmake -DUA_BUILD_EXAMPLES=ON -DUA_ENABLE_PUBSUB=ON -DUA_ENABLE_PUBSUB_ETH_UADP=ON ..

The following error in CMake build is expected and can be ignored

Build the application by giving the following command in open62541/build directory

make -j4

Note: To build a single sample, you can use “make -j4 <Sample_name>”. For example, make -j4 pubsub_TSN_loopback

Running OPC UA Pub/Sub application

Note: It is always recommended to run pubsub_TSN_loopback application first to avoid missing the initial data published by pubsub_TSN_publisher.

Run the Loopback application in Node 2 by entering the following command

./bin/examples/pubsub_TSN_loopback -interface <IFACE> -enableBlockingSocket

Run the Publisher application in Node 1 by entering the following command

./bin/examples/pubsub_TSN_publisher -interface <IFACE> -enableBlockingSocket -enableLatencyCsvLog

Note: -enableLatencyCsvLog calculates the RTT of the counterData that is published and received back. Since RTT is calculated by the pubsub_TSN_publisher application, -enableLatencyCsvLog is not available in pubsub_TSN_loopback application.

If -enableLatencyCsvLog is enabled, the latencyT1toT8.csv file will be generated in the build folder (only after the application is terminated), containing the RTT latency, missed counters and repeated counters. And the application can be terminated by using “Ctrl + C”.

An example latencyT1toT8.csv log file will look as shown below.

To visualize the data, the csv file should be imported into a statistical computing program such as R and the data displayed as histogram along with minimum and maximum values. A script mkloghisto.R is part of the demo package. To install R and execute the script in the system, follow the below commands

 apt-get install r-base 
 
 Rscript <FILE_PATH>/mkloghisto.R latencyT1toT8.csv

The graph will be generated in the latencyT1toT8-log.pdf file in the current directory. As the application is run in 250us, the expected RTT latency is 1ms (4x cycletime). The x-axis in the graph shows the deviation of the obtained RTT latency from 1ms and y-axis shows the number of samples. A sample graph is shown below, where the obtained RTT latency deviates from 1ms in the range of -31 to 32 us.

Optional Steps:

To know more usage

 ./bin/examples/pubsub_TSN_loopback -help 

./bin/examples/pubsub_TSN_publisher -help

Download Whitepaper