My Favourite QNX Feature

Working on a project with my university's rocketry team got me thinking about my favourite QNX feature from when I was playing with that RTOS. One of the primary things that makes QNX unique is the fact that it is a microkernel operating system. This means that the code which actually runs in "kernel mode" is very limited, so things like device drivers run entirely in user mode (and are registered 'on-demand'). The kernel operates via message passing , which also means its very scalable. There is a limited set of messages to pass, and everything else is built on top of them. This makes scaling to SMP systems easier, but also unlocks some special features, including my favourite QNX feature.

QNX Neutrino's modular architecture (diagram taken from QNX 7.10 documentation)

The messages passed within the kernel to accomplish tasks are (as the name suggests) just messages. This means that the medium through which they are passed can be changed. Enter Qnet , a special module added to the QNX networking stack that allows kernel messages to be passed via the network. Personally, Qnet was (and still probably is) one of the coolest things I have ever seen software used for. Since it just becomes an additional medium for passing kernel messages, it allows QNX systems to do crazy things over the network completely unbeknownst to the user program.

As an example of what you can do with Qnet, imagine some character device for a serial driver. If you echo into the file, you write some output on the serial device (let's say UART). If you cat the device, you listen to the serial output.

$ ls /dev
ser0
$ echo "hello" > /dev/ser0
$ cat /dev/ser0
world!

Now, if you're running Qnet, you are able to see the file systems of other machines running Qnet on the network. So let's say that in addition to our machine, there is another one running Qnet that has a serial device /dev/ser1 . Let's say it has the hostname "machineb". We can run the following commands from our machine:

$ echo "hello" > /net/machineb/dev/ser1
$ cat /net/machineb/dev/ser1
world!

Yes, that's right! You can interact with any character device on another machine running Qnet simply by using the right pathname prefix to identify the machine you want to interact with. All discoverable hosts are mounted in a file system under /net/hostname by Qnet. This is a toy example with a fake serial device, but it works for every single driver.

Let's say you wrote some command line utility for scanning the I2C bus called scanner , and it works locally. The only argument it takes is the file path to the I2C bus you want to scan. Well, with absolutely no changes to your binary (which was written in POSIX C, perhaps even ported from another POSIX OS), you can use it to scan the I2C bus connected to Machine B. This works even if you are in Canada and Machine B is in Russia, provided you set up the network you're using properly to allow both devices to connect to each other and be discoverable. The only change is running scanner /net/machineb/dev/i2c0 instead of scanner /dev/i2c0 . Qnet handles everything else under the hood by passing kernel message over the network instead of on the local machine.

As you can imagine, this is an incredible advantage for distributed systems. Binaries written in POSIX C, even ported from Linux, now work over the network to interact with other machines for completely free (free as in no additional work, a QNX license costs a lot of money). Of course this comes with some security risks, since Qnet is not incredibly secure and advertises other machines running Qnet on the network. However, it is an extremely powerful tool nonetheless and is a very good example of the power of a microkernel.

Unfortunately, it appears that Qnet has been discontinued in the newer release of QNX 8.0 (Qnet was present on 7.1) because of the networking stack changing to io-sock from io-pkt . I very much hope they bring it back at some point, since it was an incredibly powerful tool.

I began thinking of this after working the rocketry team's network-operated control system. The system uses W5500-EVB Picos on both the controller side and the pad side (controlled side) to act as network accessible MCUs. I wrote a message specification for messages passed between the two, used to define actuation commands as well as telemetry. However, I was thinking that it would be nice to have a log store to record any logs leading up to and after any failures, to better diagnose system issues while in production. The production system is remote controlled because it controls a highly pressurized system for a hybrid rocket engine, so it's not possible to be up close monitoring a serial output stream. I thought of logging to a SmartFS flash file system on board the device, but the W5500-EVB Pico does not have very much memory. It would also require grabbing the logs off the device post-failure somehow, and ensuring that they aren't overwritten at boot.

The W5500-EVB Pico

I thought that since the device was connected to the network during regular operation, it would be ideal to have the device output logging information over the network instead. NuttX's syslog implementation implements many different sink types, but among them is not a network sink. I thought immediately of how easy it would be to implement using Qnet, and then even how I would be able to implement the control client by just using Qnet paths to directly interact with the actuator GPIO character devices on the pad device. I would even be able to read the device drivers for all of the sensors directly that way! However, QNX is not the easiest system to use and using Qnet for such an application (while very easy) would likely be overkill. While I figure out what the best way to implement a syslog network sink would be, I'm wondering how something like Qnet could be implemented for NuttX. Maybe some agreed upon protocol between network stacks that can send information from read , write , open , close , ioctl (and etc.) calls over the network for call and response. That way paths with /net/hostname prefixes could possibly be intercepted by the file system layer and have any calls performed on them forwarded to the correct machine.