How to run OPC UA stack open62541 with Realtime PubSub
on Realtime Linux and TSN using Intel i210 Ethernet card

 

This quick start guide serves as a starting point for a user in learning/evaluating OPC UA including TSN technology
for their products/projects.

This quick start guide uses “Open Source OPC UA stack open62541 with Pub/Sub feature” and leverages the TSN features
available on “standard Linux kernel + PREMPT_RT patches” on an x86 PC hardware with Intel i210 Ethernet card

 

Prerequisites

 

Hardware requirements:

2 numbers of 4 core Intel X86 processor PCs (at least Apollolake processor) with i210 network interface card

The two nodes can be setup in any of the two different network as shown below

QSG_test_setup  

Software requirements:

Debian GNU/Linux 10.6 (buster: debian-10.6.0-amd64-xfce-CD-1 ) (Note: We used Debian for our test setup, however the steps given below should work with any Ubuntu distribution).

Run the following command to update the node after installing the OS

apt get update && apt get upgrade
 

Download default RT image using the following command

sudo apt-get install linux-image-rt-amd64
 

Reboot the node and ensure that you land in the newly installed RT kernel

sudo reboot
 

Environment setup

 

After reboot you can check the kernel version by using the below command

uname -r
 

Sample output

RT kernel version 4.19.11  

Enter as root user

sudo su
 

Download the package for the demo. Click here to download the package file

Navigate to downloaded demo_package.tar folder path

Unzip the package using following command and enter into the demo_package folder

tar -xvf demo_package.tar
 
cd demo_package
 

Run the setup.sh script available in the package.

The setup.sh does the following,

  • Update the installed packages in the node
  • Install iproute2 package for kernel version 4.19
  • Install Linux PTP version 2.0
  • Check if 8021q module is loaded, if not add the same
 

Make setup.sh script as executable by modifing the permission

chmod +x setup.sh
 

Run the setup.sh script

./setup.sh
 

Setup static IP address to the south bound interface (i210) which is connected peer-to-peer between two nodes for PubSub communication

vi /etc/network/interfaces
 

Copy and paste the following lines in “/etc/network/interfaces” file and replace “X” with a suitable suffix number to setup IP address for i210 interface

In our case the interface name is enp2s0, replace this with your i210 interface name

Example: Node 1 IP address: 192.168.0.1, Node 2 IP address: 192.168.0.2

auto enp2s0
iface enp2s0 inet static
    address 192.168.0.X
    netmask 255.255.255.0
    broadcast 192.168.0.255
auto enp2s0.8
iface enp2s0.8 inet static
    address 192.168.8.X
    netmask 255.255.255.0

 

Restart the networking service using the following command

/etc/init.d/networking restart
 

Verify that the configured IP address

ip a
 

Run the application_dependencies.sh script available in the package.

The application_dependencies.sh does the following,

  • Configure traffic control parameters for transmit and receive
  • Set Egress and Ingress policies
  • Tune real-time behaviours
  • Run Linux PTP and PHC2SYS
 

Make application_dependencies.sh script as executable by modifing the permission

chmod +x application_dependencies.sh
   

To run the node as PTP master use the following command

./application_dependencies.sh -i <IFACE> -m
   

To run the node as PTP slave use the following command

./application_dependencies.sh -i <IFACE> -s
   

For further information, run with -h option

./application_dependencies.sh -h
   

The output log of PTP can be verified by giving the following command

tail -f /var/log/ptp4l.log
 

An example log file at PTP master:

QSG_ptp4l_master_log  

An example log file of PTP slave:

QSG_ptp4l_log_slave  

To view PTP error log:

tail -f /var/log/ptp4l.err.log
 

An example ptp4l error log:

QSG_ptp4l_err_log

Note: Ensure that “send sync failed” does not occur repeatedly within a short interval of time

 

To view PHC2SYS log:

tail -f /var/log/phc2sys.log
 

An example log file at PHC2SYS log:

QSG_phc2sys_log  

To view PHC2SYS error log:

tail -f /var/log/phc2sys.err.log
 

An example PHC2SYS error log file:

QSG_phc2sys_err_log

Note: Ensure that “clockcheck: clock jumped forward or running faster than expected!” does not occur repeatedly within a short interval of time

 

To configure the node permanently as PTP master or PTP slave, you will have to create a bootup script

Create bootup script

vi /etc/rc.local
   

Copy and paste the below lines in rc.local file,

if PTP master: ./<FILE_PATH>/application_dependencies.sh -i <IFACE> -m

(or)

PTP slave: ./<FILE_PATH>/application_dependencies.sh -i <IFACE> -s

in the bootup script and make it executable

chmod +x /etc/rc.local
   

To execute the bootup script

/etc/rc.local
   

Note: From next reboot, the bootup script will be executed automatically.

To view rc.local log:

cat /var/log/rc.local.log
 

Test setup

 

Image below shows the test setup followed to run the PubSub application

QSG_test_setup  

You can follow either of the above listed network to run the application

 

Steps to run OPC UA PubSub application

 

Clone the open62541 stack with the following command

git clone https://github.com/open62541/open62541.git
 

Image attached below shows the overview of the target applications

  OPC UA PubSub application
 

T1 – Ethernet packet publish timestamp at userspace

T2 – Ethernet packet timestamp at kernel space

T3 – Ethernet incoming packet timestamp at kernelspace

T4 – Ethernet packet subscribe timestamp at userspace

T5 – Ethernet loopback packet publish timestamp at userspace

T6 – Ethernet loopback packet outgoing timestamp at kernelspace

T7 – Ethernet loopback packet incoming timestamp at kernelspace

T8 – Ethernet loopback packet subscriber timestamp at userspace

The application in Node 1 encode and publish a packet to Node 2, and the Node 2 decodes the received publish packet and loops back the same to make the published data finally available to the application at Node 1. The application at Node 1 finally computes the round-trip time (T8-T1) it takes to publish and receive back the same data. To keep track of the sequence in which the packets were sent and received at both ends, the payload in the OPC UA PubSub publish packet contains a counter variable that increments in every application cycle

 

Follow the steps given below to build the application,

cd open62541/
 
mkdir build
 
cd build/
 
cmake -DUA_BUILD_EXAMPLES=OFF -DUA_ENABLE_PUBSUB=ON -DUA_ENABLE_PUBSUB_CUSTOM_PUBLISH_HANDLING=ON -DUA_ENABLE_PUBSUB_ETH_UADP=ON -DUA_ENABLE_PUBSUB_ETH_UADP_ETF=ON ..
 

Optional Steps:

1. To run the application in different cycle time, modify the following #define in pubsub_TSN_loopback.c and pubsub_TSN_publisher.c present in directory open62541/examples/pubsub_realtime/

#define CYCLE_TIME 0.25

Note: Cycle time in milliseconds

 

2. To view the data written in csv files, uncomment the following #define in pubsub_TSN_loopback.c and pubsub_TSN_publisher.c present in directory open62541/examples/pubsub_realtime/

#define UPDATE_MEASUREMENTS
 

Image attached below shows the overview of the .csv log files generated by the applications

OPC UA PubSub application csv log
 

pubsub_TSN_publisher generates publisher_T1.csv, subscriber_T8.csv log files and
pubsub_TSN_loopback generates publisher_T5.csv, subscriber_T4.csv log files respectively

 

Build the application by giving the following command in open62541/build directory

make -j4
 

In the Node 1 (publisher):

gcc -O2 ../examples/pubsub_realtime/pubsub_TSN_publisher.c -I../include -I../plugins/include -Isrc_generated -I../arch/posix -I../arch -I../plugins/networking -I../deps/ -I../src/server -I../src -I../src/pubsub bin/libopen62541.a -lrt -lpthread -D_GNU_SOURCE -o ./bin/pubsub_TSN_publisher
 

In the Node 2 (subscriber):

gcc -O2 ../examples/pubsub_realtime/pubsub_TSN_loopback.c -I../include -I../plugins/include -Isrc_generated -I../arch/posix -I../arch -I../plugins/networking -I../deps/ -I../src/server -I../src -I../src/pubsub bin/libopen62541.a -lrt -lpthread -D_GNU_SOURCE -o ./bin/pubsub_TSN_loopback
 

Running OPC UA Pub/Sub application

 

Note: It is always recommended to run pubsub_TSN_loopback application first to avoid missing the initial data published by pubsub_TSN_publisher.

Run the Loopback application in Node 2 by entering the following command

sudo ./bin/pubsub_TSN_loopback <IFACE>
 

Run the Publisher application in Node 1 by entering the following command

sudo ./bin/pubsub_TSN_publisher <IFACE>
 

Note: If #define UPDATE_MEASUREMENTS is enabled, the CSV files will be generated in the build directory containing the counter values and timestamp for benchmarking. Make sure you terminate both the applications by pressing “Ctrl + C”

An example .csv log file will look as shown below. The 1st column contains counter data and the 2nd column contains the timestamp value

QSG_phc2sys_err_log