Nov 282019
 

This is more of a brain dump for yours truly, since as a day job, I’m dealing with OpenThread a lot, and so there’s lots of little tricks out there for building it on various platforms and configurations. The following is not so much a how-to guide, but a quick brain dump of different things I’ve learned over the past year or so of messing with OpenThread.

Verbose builds

To get a print out of every invocation of the toolchain with all flags, specify V=1 on the call to make:

$ make -f examples/Makefile-${PLATFORM} …${ARGS}… V=1

Running one step at a time

To disable the parallel builds when debugging the build system, append BuildJobs=1 to your make call:

$ make -f examples/Makefile-${PLATFORM} …${ARGS}… BuildJobs=1

Building a border router NCP image

General case

$ make -f examples/Makefile-${PLATFORM} …${ARGS}… BORDER_AGENT=1 BORDER_ROUTER=1 COMMISSIONER=1 UDP_FORWARD=1

For TI CC2538

# Normal CC2538 (e.g. Zolertia Firefly)
$ make -f examples/Makefile-cc2538 BORDER_AGENT=1 BORDER_ROUTER=1 COMMISSIONER=1 UDP_FORWARD=1

# CC2538 + CC2592
$ make -f examples/Makefile-cc2538 BORDER_AGENT=1 BORDER_ROUTER=1 COMMISSIONER=1 UDP_FORWARD=1 CC2592=1

If you want to run the latter image on a WideSky Hub, you’ll need to edit examples/platform/cc2538/uart.c and comment out line 117 before compiling as it uses a simple diode for 5V to 3.3V level shifting, and this requires the internal pull-up to be enabled:

115     // rx pin
116     HWREG(IOC_UARTRXD_UART0) = IOC_PAD_IN_SEL_PA0;
117     HWREG(IOC_PA0_OVER)      = IOC_OVERRIDE_DIS; // ← comment out this to allow UART RX to work
118     HWREG(GPIO_A_BASE + GPIO_O_AFSEL) |= GPIO_PIN_0;

For Nordic nRF52840

# Nordic development board (PCA10056) via J2 (near battery)
$ make -f examples/Makefile-nrf52840 BORDER_AGENT=1 BORDER_ROUTER=1 COMMISSIONER=1 UDP_FORWARD=1

# Ditto, but instead using J3 (near middle of bottom edge)
$ make -f examples/Makefile-nrf52840 BORDER_AGENT=1 BORDER_ROUTER=1 COMMISSIONER=1 UDP_FORWARD=1 USB=1

# Nordic dongle (PCA10059)
$ make -f examples/Makefile-nrf52840 BORDER_AGENT=1 BORDER_ROUTER=1 COMMISSIONER=1 UDP_FORWARD=1 USB=1 BOOTLOADER=USB

I’m working on what needs to be done for the Fanstel BT840X… watch this space.

Building certification test images

For CC2538

$ make -f examples/Makefile-cc2538 BORDER_ROUTER=1 COMMISSIONER=1 DHCP6_CLIENT=1 JOINER=1

Running CI tests outside of Travis CI

You will need:

  • Python 3.5 or later. 3.6 recommended.
  • pycryptodome (not pycrypto: if you get an AttributeError referencing AES.MODE_CCM, that’s why!
  • enum34 (for now… I suspect this will disappear once the Python 2.7 requirement is dropped for Android tools)
  • ipaddress
  • pexpect

The test suites work by running the POSIX ot-cli-${TYPE} (where ${TYPE} is ftd or mtd).

Running tests

From the root of the OpenThread tree:

$ make -f examples/Makefile-posix check

Making tests

The majority of the tests lurk under tests/scripts/thread-cert. Don’t be fooled by the name, lots of non-Thread tests live there.

The file node.py wraps pexpect up in an object with methods for calling the various CLI commands or waiting for things to be printed to the console.

The tests themselves are written using using the standard Python unittest framework, with setUp creating a few Node objects (node.py) and tearDown cleaning them up. The network is simulated.

The test files must be executable, and call unittest.main() after checking if the script is called directly.

Logs during the test runs

During the tests, the logs are stashed in build/${CHOST}/tests/scripts/thread-cert and will be named ${SCRIPTNAME}.log. (e.g test_coap_observe.py.log). Also present is the .pcap file (Packet dump).

Logs after the tests

The test suite will report the logs are at tests/scripts/thread-cert/test-suite.log. This is relative to the build directory… so look in build/${CHOST}/tests/scripts/thread-cert/test-suite.log.

make pretty with clang 8.0

Officially this isn’t supported, but you can “fool” OpenThread’s build system into using clang 8.0 anyway:

$ cat ~/bin/clang-format-6.0 
#!/bin/bash

if [ "$1" == "--version" ]; then
        echo "clang-format version 6.0"
else
        clang-format "$@"
fi

Put that file in your ${PATH}, make it executable, then OpenThread will think you’ve got clang-format version 6 installed. This appears to work without ill effect, so maybe a future release of OpenThread will support it.

May 242019
 

So, in my workplace we’re developing a small energy/water metering device, which runs on a 6LoWPAN network and runs OpenThread-based firmware. The device itself is largely platform-agnostic, requiring a simple CoAP gateway to provide it with a configuration blob and history write end-point. The gateway service I’m running is intended to talk to WideSky.

One thorny issue we need to solve before deploying these things in the wild, is over-the-air updates. So we need a way to transfer the firmware image to the device over the mesh network. Obviously, this firmware image needs to be digitally signed and hashed using a strong cryptographic hash — I’ve taken care of this already. My problem is downloading an image that will be up to 512kB in size.

Thankfully, the IETF has thought of this, the solution to big(gish) files over CoAP is Block-wise transfers (RFC-7959). This specification gives you the ability to break up a large payload into smaller chunks that are powers of two in size between 16 and 2048 bytes.

6LoWPAN itself though has a limitation: the IEEE 802.15.4 radio specification it is built on cannot send frames bigger than 128 bytes. Thus, any message sent via this network must be that size or smaller. IPv6 has a minimum MTU of 1280 bytes, so how do they manage? They fragment the IPv6 datagram into multiple 802.15.4 frames. The end device re-assembles the fragments as it receives them.

The catch is, if a fragment is lost, you lose the entire datagram, there’s no repeats of individual fragments, the entire datagram must be re-sent. The question in my mind was this: Is it faster to rely on block-wise transfers to break the payload up and make lots of small requests, or is it faster to rely on 6LoWPAN fragmentation?

The test network here has a few parts:

  • The target device, which will be downloading a 512kB firmware image to a separate SPI flash chip.
  • The border router, which provides a secure IPv6 tunnel to a cloud host.
  • The cloud server which runs the CoAP service we’ll be talking to.

The latency between the office (in Brisbane) and the cloud server (in Sydney) isn’t bad, about 30~50ms. The CoAP service is built using node-coap and coap-polka.

My CoAP requests have some overheads:

  • The path being downloaded is about 19 bytes long.
  • There’s an authentication token given as a query string, so this adds an additional 12 bytes.

The data link is not 100% reliable, with the device itself dropping some messages. This is leading to some retransmits. The packet loss is not terrible, but probably in the region of around 5%. Over this slightly lossy link, I timed the download of my 512kB firmware image by my device with varying block size settings.

Note that node-coap seems to report a “Bad Option” error for szx=0 and szx=7, even though both are legitimately within specification. (I’d expect node-coap to pass szx=7 through and allow the application to clamp it to 6, but it seems node-coap‘s behaviour is to report “Bad option”, but then pass the payload through anyway.)

Size Exponent (szx)Block sizeStart time (UTC)End time (UTC)Effective data rate
6102403:27:0403:37:52809B/s
551203:41:2503:53:40713B/s
425603:57:1504:16:16458B/s
312804:17:4604:54:17239B/s
26404:56:0905:54:53150B/s
13223:31:3301:39:4468B/s

It remains to be seen how much multiple hops and outdoor atmospherics affect the situation. A factor here is how quickly the device can turn-around between processing a response and sending the next request, which in this case is governed by the speed of the SPI flash and its driver.

Effects on “busy” networks

So, I haven’t actually done any hard measurements here, but doing testing on a busier network with about 30 nodes, the block size equation tips more in favour of a smaller block size.

I’ll try to quantify the delays at some point, but right now 256 byte blocks are the clear winner, with 512 and 1024 byte block transfers proving highly unreliable. The speed advantage between 1k and 512 bytes in ideal conditions was a big over 10%… which really doesn’t count for much. At 256 bytes, the speed difference was about 43%, quite significant. You’re better off using 512-byte blocks if the network is quiet.

On a busy network, with all the retransmissions, smaller is better. No hard numbers yet, but right now at 256 byte blocks, the effective rate is around 118 bytes/sec. I’ll have to analyse the logs a bit to see where 512/1024 byte block sizes sat, but the fact they rarely completed says it all IMO.

Slow and steady beats fast and flakey!