How to run OPC UA stack open62541 with Realtime PubSub
on Realtime Linux and TSN using Intel I210 Ethernet controller

 

This quick start guide serves as a starting point for a user in learning/evaluating OPC UA including TSN technology
for their products/projects.

This quick start guide uses “Open Source OPC UA stack open62541 with Pub/Sub feature” and leverages the TSN features
available on “standard Linux kernel + PREMPT_RT patches” on an x86 system with Intel I210 Ethernet controller

 

Prerequisites

 

Hardware requirements:

Two 4-core Intel x86-based systems (e.g. Apollo Lake or Whiskey Lake architecture) with I210 Ethernet controller.

The two nodes can be setup in one of the two network topologies as shown below

QSG_test_setup  

Software requirements:

Debian GNU/Linux 10.6 (buster: debian-10.6.0-amd64-xfce-CD-1 ) (Note: Please understand that we cannot provide individual version of the Quick Start Guide for the wide variety of available Linux distributions. We selected Debian 10, as it is one of the most popular distributions and known to be stable and suitable for our purpose.)

Run the following command to update the node after installing the OS

sudo apt-get update && apt-get upgrade
 

Download default RT image using the following command

sudo apt-get install linux-image-rt-amd64
 

Reboot the node and ensure that you land in the newly installed RT kernel

sudo reboot
 

Environment setup

 

After reboot you can check the kernel version by using the below command

uname -r
 

Sample output

RT kernel version 4.19.11  

Enter as root user

sudo su
 

To download the package for the demo

wget https://www.kalycito.com/wp-content/uploads/2021/01/demo_package.tar
 

Unzip the package using following command and enter into the demo_package folder

tar -xvf demo_package.tar && cd demo_package
 

The setup.sh available in the package does the following,

  • Updates the installed packages in the node
  • Installs iproute2 package for kernel version 4.19
  • Installs Linux PTP version 2.0
  • Checks if 8021q module is loaded, if not adds the same
 

Run the setup.sh script

./setup.sh
 

Setup static IP address to the south bound interface (I210) which is connected peer-to-peer between two nodes for PubSub communication

vi /etc/network/interfaces
 

Copy and paste the following lines in “/etc/network/interfaces” file and replace “X” with a suitable suffix number to setup IP address for I210 interface

In our case the interface name is enp2s0, replace this with your I210 interface name

Example: Node 1 IP address: 192.168.0.1, Node 2 IP address: 192.168.0.2

auto enp2s0
iface enp2s0 inet static
    address 192.168.0.X
    netmask 255.255.255.0
    broadcast 192.168.0.255
auto enp2s0.8
iface enp2s0.8 inet static
    address 192.168.8.X
    netmask 255.255.255.0

Note: The PubSub TSN applications are configured to run with VLAN Id 8. For this purpose, VLAN configuration is done in the I210 interface.

 

An example interface file:

Restart the networking service using the following command

/etc/init.d/networking restart
 

Verify the IP address and connection between two nodes. From node 1 ping node 2 and check vice versa

ip a
ping 192.168.0.X
 

The application_dependencies.sh available in the package does the following,

  • Configures traffic control parameters for transmit and receive
  • Sets Egress and Ingress policies
  • Tunes real-time behaviours
  • Runs Linux PTP and PHC2SYS
 

To run the node as PTP master use the following command

./application_dependencies.sh -i <IFACE> -m
   

To run the node as PTP slave use the following command

./application_dependencies.sh -i <IFACE> -s
   

The following error occurs while running application_dependencies.sh, since no ntuple configuration will be available to delete initially.

The output log of PTP can be verified by giving the following command

tail -f /var/log/ptp4l.log
 

An example log file at PTP master:

QSG_ptp4l_master_log  

An example log file of PTP slave:

QSG_ptp4l_log_slave  

To view PHC2SYS log:

tail -f /var/log/phc2sys.log
 

An example log file at PHC2SYS log:

QSG_phc2sys_log  

To configure the node permanently as PTP master or PTP slave, use the following command to create a bootup script

To configure as PTP master:

echo ./<FILE_PATH>/application_dependencies.sh -i <IFACE> -m >> /etc/rc.local

(or) PTP slave:

echo ./<FILE_PATH>/application_dependencies.sh -i <IFACE> -s >> /etc/rc.local

To make the bootup script as executable

chmod +x /etc/rc.local
   

To execute the bootup script

/etc/rc.local

Note: From next reboot, the bootup script will be executed automatically.

To view rc.local log:

cat /var/log/rc.local.log
 

Test setup

 

Image below shows the test setup followed to run the PubSub application

QSG_test_setup  

You can follow either of the above listed network to run the application

 

Steps to run OPC UA PubSub application

 

Clone the open62541 stack with the following command

git clone https://github.com/open62541/open62541.git
 

Fetch the pull request and create new branch with the following command

cd open62541/
git fetch origin pull/3996/head:local_branch && git checkout local_branch
 

Image attached below shows the overview of the target applications

  OPC UA PubSub application
 

T1 – Ethernet packet publish timestamp at userspace

T2 – Ethernet packet timestamp at kernel space

T3 – Ethernet incoming packet timestamp at kernelspace

T4 – Ethernet packet subscribe timestamp at userspace

T5 – Ethernet loopback packet publish timestamp at userspace

T6 – Ethernet loopback packet outgoing timestamp at kernelspace

T7 – Ethernet loopback packet incoming timestamp at kernelspace

T8 – Ethernet loopback packet subscriber timestamp at userspace

The application in Node 1 encodes and publishes a packet to Node 2, and the Node 2 decodes the received publish packet and loops back the same to make the published data finally available to the application at Node 1. The application at Node 1 finally computes the round-trip time (T8-T1) it takes to publish and receive back the same data. To keep track of the sequence in which the packets were sent and received at both ends, the payload in the OPC UA PubSub publish packet contains a counter variable that increments in every application cycle

 

Follow the steps given below to build the application,

mkdir build
 
cd build/
 
cmake -DUA_BUILD_EXAMPLES=ON -DUA_ENABLE_PUBSUB=ON -DUA_ENABLE_PUBSUB_ETH_UADP=ON ..
 

The following error in CMake build is expected and can be ignored

Build the application by giving the following command in open62541/build directory

make -j4

Note: To build a single sample, you can use “make -j4 <Sample_name>”. For example, make -j4 pubsub_TSN_loopback

 

Running OPC UA Pub/Sub application

 

Note: It is always recommended to run pubsub_TSN_loopback application first to avoid missing the initial data published by pubsub_TSN_publisher.

Run the Loopback application in Node 2 by entering the following command

./bin/examples/pubsub_TSN_loopback -interface <IFACE> -enableBlockingSocket
 

Run the Publisher application in Node 1 by entering the following command

./bin/examples/pubsub_TSN_publisher -interface <IFACE> -enableBlockingSocket -enableLatencyCsvLog
 

Note: -enableLatencyCsvLog calculates the RTT of the counterData that is published and received back. Since RTT is calculated by the pubsub_TSN_publisher application, -enableLatencyCsvLog is not available in pubsub_TSN_loopback application.

If -enableLatencyCsvLog is enabled, the latencyT1toT8.csv file will be generated in the build folder (only after the application is terminated), containing the RTT latency, missed counters and repeated counters. And the application can be terminated by using “Ctrl + C”.

An example latencyT1toT8.csv log file will look as shown below.

 

To visualize the data, the csv file should be imported into a statistical computing program such as R and the data displayed as histogram along with minimum and maximum values. A script mkloghisto.R is part of the demo package. To install R and execute the script in the system, follow the below commands

apt-get install r-base
Rscript <FILE_PATH>/mkloghisto.R latencyT1toT8.csv
 

The graph will be generated in the latencyT1toT8-log.pdf file in the current directory. As the application is run in 250us, the expected RTT latency is 1ms (4x cycletime). The x-axis in the graph shows the deviation of the obtained RTT latency from 1ms and y-axis shows the number of samples. A sample graph is shown below, where the obtained RTT latency deviates from 1ms in the range of -31 to 32 us.

Optional Steps:

To know more usage

./bin/examples/pubsub_TSN_loopback -help
./bin/examples/pubsub_TSN_publisher -help