May 072017
 

So, in amongst my pile of crusty old hardware is the old netbook I used to use in the latter part of my univerity days. It is a Lemote Yeeloong, and sports a ~700MHz Loongson 2F CPU (MIPS III little endian ISA) and 1GB RAM.

Back in the day it was a brilliant little machine. It came out of the box running a localised (for China) version of Debian, and had pretty much everything you’d need. I natually repartitioned the machine, setting up Gentoo and I had a separate partition for Debian, so I could actually dual-boot between them.

Fast forward 10 years, the machine runs, but the battery is dead, and Debian no longer supports MIPS-III machines. Debian Jessie does, but Stretch, likely due for release some time this year, will not, if you haven’t got a CPU that supports mips32r2 or mips64r2, you’re stuffed.

I don’t want to throw this machine away.  Being as esoteric as it is, it is an unlikely target for theft, as to the casual observer, it’ll just be “some crappy netbook”.  If someone were to try and steal it, there’s a very high probability I’ll recover it with my data because the day its PMON2000 boot firmware successfully boots a x86-64 OS like Ubuntu or Windows without the assistance of a VM of some kind would be the day Satan puts a requisition order in for anti-freeze and winter mittens.

My use case is for a machine I can take with me on the bicycle.  My needs aren’t huge: I won’t be playing video on this thing, it’ll be largely for web browsing and email.  The web browser needs to support JavaScript, so that rules out options like ELinks or Dillo, my preferred browser is Firefox but I’ll settle for something Webkit-based if that’s all that’s out there.

So what operating systems do I have for a machine that sports a MIPS-III CPU and 1GB RAM?  Fedora has a MIPS port, but that, like Debian, is for the newer MIPS systems.  Arch Linux too is for newer architectures.

I could bootstrap Alpine Linux… and maybe that’s worth looking into, they seem to be doing some nice work in producing a small and capable Linux distribution.  They don’t yet support MIPS though.

Linux From Scratch is an option, if a little labour intensive.  (Been there, done that.)

OpenBSD directly supports this machine, and so I gave OpenBSD 6.0 a try.  It’s a very capable OS, and while it isn’t Linux, there isn’t much that an experienced Linux user like myself needs to adapt to in order to effectively use the OS.  pkgsrc is a great asset to OpenBSD, with a large selection of pre-built packages already available.  Using that, it is possible to get a workable environment up and running very quickly.  OpenBSD/loongson uses the n64 ABI.

Due to licensing worries, they use a particularly old version of binutils as their linker and assembler.  The plan seems to be they wish to wean themselves off the GNU toolchain in favour of LLVM.  At this time though, much of the system is built using the GNU toolchain with some custom patches.  I found that, on the Yeeloong, 1GB RAM was not sufficient for compiling LLVM, even after adding additional swap files, and some packages I needed weren’t available in pkgsrc, nor would they build with the version of GNU tools available.

Maybe as they iron out the kinks in their build environment with LLVM, this will be worth re-visiting.  They’ve done a nice job so far, but it’s not quite up to where I need it to be.

Gentoo actually gives me the choice of two possible ABIs: o32 and n32o32 is the old 32-bit ABI, and suffers a number of performance problems, but generally works.  It’s what Debian Jessie and earlier supplies, and what their mips32 port will produce from Stretch onwards.

n32 is the MIPS equivalent of what some of you may know as x32 on AMD64 platforms, it is a 32-bit environment with 64-bit long pointers… the idea being that very few applications actually benefit from the use of 64-bit data types, and so the usual quantities like int and long remain the same as what they’d be on o32, saving memory.  The long long data type gets a boost because, although “32-bit”, the 64-bit operations are still available for use.

The trouble is, some applications have problems with this mode.  Either the code sees “mips64” in the CHOST and assumes a full 64-bit system (aka n64), or it assumes the pointers are the same width as a long, or the build system makes silly assumptions as to where things get put.  (virtualenv comes to mind, which is what started me on this journey.  The same problem affects x32 on AMD64.)

So I thought, I’d give n64 a try.  I’d see if I can build a cross-compiler on my AMD64 host, and bootstrap Gentoo from that.

Step 1: Cross-compiler

For the cross-compiler, Gentoo has a killer feature that I have not seen in too many other distributions: crossdev.  This is a toolchain build tool that can generate cross-compiler toolchains for most processor architectures and environments.

This is installed by running emerge sys-devel/crossdev.

A gotcha with hardened

I run “hardened” AMD64 stages on my machines, and there’s a little gotcha to be aware of: the hardened USE flag gets set by crossdev, and that can cause fun and games if, like on MIPS, the hardening features haven’t been ported.  My first attempt at this produced a n64 userland where pretty much everything generated a segmentation fault, the one exception being Python 2.7.  If I booted with init=/bin/bash (or init=/bin/bb), my virtual environment died, if I booted with init=/usr/bin/python2.7, I’d be dropped straight into a Python shell, where I could import the subprocess module and try to run things.

Cleaning up, and forcing crossdev to leave off hardened support, got things working.

Building the toolchain

With the above gotcha in mind:

# crossdev --abis n64 \
           --env 'USE="-hardened"' \
           -s4 -t mips64el-unknown-linux-gnu

The --abis n64 tells crossdev you want a n64 ABI toolchain, and the --env will hopefully keep the hardened flag unset. Failing that, try this:

# cat > /etc/portage/package.use/mips64 <<EOF
cross-mips64el-unknown-linux-gnu/binutils -hardened
cross-mips64el-unknown-linux-gnu/gcc -hardened
cross-mips64el-unknown-linux-gnu/glibc -hardened
EOF

If you want a combination of specific toolchain components to try, I’m using:

  • Binutils: 2.28
  • GCC: 5.4.0-r3
  • glibc: 2.25
  • headers: 4.10

Step 2: Checking our toolchain

This is where I went wrong the first time, I tried building the entire OS, only to discover I had wasted hours of CPU time building non-functional binaries. Save yourself some frustration. Start with a small binary to test.

A good target for this is busybox. Run mips64el-unknown-linux-gnu-emerge busybox, and wait for a bit.

When it completes, you should hopefully have a busybox binary:

RC=0 stuartl@beast ~ $ file /usr/mips64el-unknown-linux-gnu/bin/busybox 
/usr/mips64el-unknown-linux-gnu/bin/busybox: ELF 64-bit LSB executable, MIPS, MIPS-III version 1 (SYSV), statically linked, for GNU/Linux 3.2.0, stripped

Testing busybox

There is qemu-user-mips64el, but last time I tried it, I found it broken. So an easier option is to use real hardware or QEMU emulating a full system. In either case, you’ll want to ensure you have your system-of-choice running with a working 64-bit kernel already, if your real hardware isn’t already running a 64-bit Linux kernel, use QEMU.

For QEMU, the path-of-least-resistance I found was to use Debian. Aurélien Jarno has graciously provided QEMU images and corresponding kernels for a good number of ports, including little-endian MIPS.

Grab the Wheezy disk image and the corresponding kernel, then run the following command:

# qemu-system-mips64el -M malta \
    -kernel vmlinux-3.2.0-4-5kc-malta \
    -hda debian_wheezy_mipsel_standard.qcow2 \
    -append "root=/dev/sda1 console=ttyS0,115200" \
    -serial stdio -nographic -net nic -net user

Let it boot up, then log in with username root, password root.

Install openssh-client and rsync (this does not ship with the image):

# apt-get update
# apt-get install openssh-client rsync

Now, you can create a directory, and pull the relevant files from your host, then try the binary out:

# mkdir gentoo
# rsync -aP 10.0.2.2:/usr/mips64el-unknown-linux-gnu/ gentoo/
# chroot gentoo bin/busybox ash

With luck, you should be in the chroot now, using Busybox.

Step 3: Building the system

Having done a “hello world” test, we’re now ready to build everything else. Start by tweaking your /usr/mips64el-unknown-linux-gnu/etc/portage/make.conf to your liking then adjust /usr/mips64el-unknown-linux-gnu/etc/portage/make.profile to point to one of the MIPS profiles. For reference, on my system:

RC=0 stuartl@beast ~ $ ls -l /usr/mips64el-unknown-linux-gnu/etc/portage/make.profile
lrwxrwxrwx 1 root root 49 May  1 09:26 /usr/mips64el-unknown-linux-gnu/etc/portage/make.profile -> /usr/portage/profiles/default/linux/mips/13.0/n64
RC=0 stuartl@beast ~ $ cat /usr/mips64el-unknown-linux-gnu/etc/portage/make.conf 
CHOST=mips64el-unknown-linux-gnu
CBUILD=x86_64-pc-linux-gnu
ARCH=mips

HOSTCC=x86_64-pc-linux-gnu-gcc

ROOT=/usr/${CHOST}/

ACCEPT_KEYWORDS="mips ~mips"

USE="${ARCH} -pam"

CFLAGS="-O2 -pipe -fomit-frame-pointer"
CXXFLAGS="${CFLAGS}"

FEATURES="-collision-protect sandbox buildpkg noman noinfo nodoc"
# Be sure we dont overwrite pkgs from another repo..
PKGDIR=${ROOT}packages/
PORTAGE_TMPDIR=${ROOT}tmp/

ELIBC="glibc"

PKG_CONFIG_PATH="${ROOT}usr/lib/pkgconfig/"
#PORTDIR_OVERLAY="/usr/portage/local/"

Now, you should be ready to start building:

# mips64el-unknown-linux-gnu-emerge -e \
    --keep-going -j6 --load-average 12.0 @system

Now, go away, and do something else for several hours.  It’ll take that long, depending on the speed of your machine.  In my case, the machine is an AMD Phenom II x6 with 8GB RAM, which was brand new in 2010.  It took a good day or so.

Step 4: Testing our system

We should have enough that we can boot our QEMU VM with this image instead.  One way of trying it would be to copy across the userland tree the same way we did for pulling in busybox and chrooting back in again.

In my case, I took the opportunity to build a kernel specifically for the VM that I’m using, and made up a disk image using the new files.

Building a kernel

Your toolchain should be able to cross-build a kernel for the virtual machine.  To get you started, here’s a kernel config file.  Download it, decompress it, then drop it into your kernel source tree as .config.

Having done that, run make olddefconfig ARCH=mips to set the defaults, then make menuconfig ARCH=mips and customise to your hearts content. When finished, run make -j6 vmlinux modules CROSS_COMPILE=mips64el-unknown-linux-gnu- to build the kernel and modules.

Finally, run make modules_install firmware_install INSTALL_MOD_PATH=$PWD/modules CROSS_COMPILE=mips64el-unknown-linux-gnu- to install the kernel modules and firmware into a convenient place.

Making a root disk

Create a blank, raw disk image using qemu-img, then partition it as you like and mount it as a loopback device:

# qemu-img create -f raw gentoo.raw 8G
# fdisk gentoo.raw
(do your partitioning here)
# losetup -P /dev/loop0 $PWD/gentoo.raw

Now you can format the partitions /dev/loop0pX as you see fit, then mount them in some convenient place. I’ll assume that’s /mnt/vm for now. You’re ready to start copying everything in:

# rsync -aP /usr/mips64el-unknown-linux-gnu/ /mnt/vm/
# rsync -aP /path/to/kernel/tree/modules/ /mnt/vm/

You can use this opportunity to make some tweaks to configuration files, like updating etc/fstab, tweaking etc/portage/make.conf (changing ROOT, removing CBUILD), and setting up a getty on ttyS0. I also like to symlink lib to lib64 in non-multilib environments such as this: Don’t symlink lib and lib64! See below.

# cd /mnt/vm
# mv lib/* lib64
# rmdir lib
# ln -s lib64 lib
# cd usr
# mv lib/* lib64
# rmdir lib
# ln -s lib64 lib

When you’re done, unmount.

First boot

Run QEMU with the following arguments:

# qemu-system-mips64el -M malta \
    -kernel /path/to/your/kernel/vmlinux \
    -hda /path/to/your/gentoo.raw \
    -append "root=/dev/sda1 console=ttyS0,115200 init=/bin/bash" \
    -serial stdio -nographic -net nic -net user

It should boot straight to a bash prompt. Mount the root read/write, and then you can make any edits you need to do before boot, such as changing the root password. When done, re-mount the root as read-only, then exec /sbin/init.

# mount / -o rw,remount
# passwd
… etc
# mount / -o ro,remount
# exec /sbin/init

With luck, it should boot to completion.

Step 5: Making the VM a system service

Now, it’d be real nice if libvirt actually supported MIPS VMs, but it doesn’t appear that it does, or at least I couldn’t get it to work.  virt-manager certainly doesn’t support it.

No matter, we can make do with a telnet console (on loopback), and supervisord to daemonise QEMU.  I use the following supervisord configuration file to start my VMs:

[unix_http_server]
file=/tmp/supervisor.sock   ; (the path to the socket file)

[supervisord]
logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB        ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10           ; (num of main logfile rotation backups;default 10)
loglevel=info                ; (log level;default info; others: debug,warn,trace)
pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false               ; (start in foreground if true;default false)
minfds=1024                  ; (min. avail startup file descriptors;default 1024)
minprocs=200                 ; (min. avail process descriptors;default 200)

; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL  for a unix socket

[program:qemu-mips64el]
command=/usr/bin/qemu-system-mips64el -cpu MIPS64R2-generic -m 2G -spice disable-ticketing,port=5900 -M malta -kernel /home/stuartl/kernels/qemu-mips/vmlinux -hda /var/lib/libvirt/images/gentoo-mips64el.raw -append "mem=256m@0x0 mem=1792m@0x90000000 root=/dev/sda1 console=ttyS0,115200" -chardev socket,id=char0,port=65223,host=::1,server,telnet,nowait -chardev socket,id=char1,port=65224,host=::1,server,telnet,nowait -serial chardev:char0 -mon chardev=char1,mode=readline -net nic -net bridge,helper=/usr/libexec/qemu-bridge-helper,br=br0

The following creates two telnet sockets, port 65223 is the VM’s console, 65224 is the QEMU control console. The VM has the maximum 2GB RAM possible and uses bridged networking to the network bridge br0. There is a graphical console available via SPICE.

All telnet and SPICE interfaces are bound to loopback, so one must use SSH tunnelling to reach those ports from another host. You can change the above command line to use VNC if that’s what you prefer.

At this point, the VM should be able to boot on its own. I’d start with installing some basic packages, and move on from there. You’ll find the environment is very sparse (my build had no Perl binary for example) but the basics for building everything should be there.

You may also find that what is there, isn’t quite installed right… I found that sshd wasn’t functional due to missing users… a problem soon fixed by doing an emerge -K openssh (the earlier step will have produced binary packages).

In my case, that’s installing a decent text editor (vim) and GNU screen so I can start a build, then detach.  Lastly, I’ll need catalyst, which is Gentoo’s release engineering tool.

At the moment, this is where I’m at.  GNU screen has indirectly pulled in Perl as a dependency, and that is building as I type this.  It is building faster than the little netbook does, and I have the bonus that I can throw more RAM at the problem than I can on the real hardware. The plan from here:

  1. emerge -ek @system, to build everything that got missed before.
  2. ROOT=/tmp/seed emerge -eK @system, to bundle everything up into a staging area
  3. populating /tmp/seed/dev with device files
  4. tar-ing up /tmp/seed to make my initial “seed” stage for catalyst.
  5. building the first n64 stages for Gentoo using catalyst
  6. building the packages I want for the netbook in a chroot
  7. transferring the chroot to the netbook

Symlinking lib and lib64… don’t do it!

So, I was doing this years ago when n32 was experimental.  I recall it being necessary then as this was before Portage having proper multilib support.  The earlier mipsel n32 stages I built, which started out from kanaka‘s even more experimental multilib stages, required this kludge to work-around the lack of support in Portage.

Portage has changed, it now properly handles multilib, and so the symlink kludge is not only not necessary, it breaks things rather badly, as I discovered.  When packages merge files to /lib, rather than following the symlink, they’ll replace it with a directory.  At that point, all hell breaks loose, because stuff that “appeared” in /lib before is no longer there.

I was able to recover by rsync-ing /lib64 to /lib, which isn’t a pretty solution, but it’ll be enough to get an initial “seed” stage.  Running that seed stage through Catalyst will clean up the remnants of that bungle.

Mar 262017
 

Yesterday’s post was rather long, but was intended for mostly technical audiences outside of amateur radio.  This post serves as a brain dump of volatile memory before I go to sleep for the night.  (Human conscious memory is more like D-RAM than one might realise.)

Radio interface

So, many in our group use packet radio TNCs already, with a good number using the venerable Kantronics KPC3.  These have a DB9 port that connects to the radio and a second DB25 RS-323 port that connects to the computer.

My proposal: we make an audio interface that either plugs into that DB9 port and re-uses the interface cables we already have, or directly into the radio’s data port.

This should connect to an audio interface on the computer.

For EMI’s sake, I’d recommend a USB sound dongle like this, or these, or this as that audio interface.  I looked on Jaycar and did see this one, which would also work (and burn a hole in your wallet!).

If you walk in and the asking price is more than $30, I’d seriously consider these other options.  Of those options, U-Mart are here in Brisbane; go to their site, order a dongle then tell the site you’ll come and pick it up.  They’ll send you an email with an order number when it’s ready, you just need to roll up to the store, punch that number into a terminal in the shop, then they’ll call your name out for you to collect and pay for it.

Scorptec are in Melbourne, so you’ll have to have items shipped, but are also worth talking to.  (They helped me source some bits for my server cluster when U-Mart wouldn’t.)

USB works over two copper pairs; one delivers +5V and 0V, the other is a differential pair for data.  In short, the USB link should be pretty immune from EMI issues.

At worst, you should be able to deal with it with judicious application of ferrite beads to knock down the common mode current and using a combination of low-ESR electrolytic and ceramic capacitors across the power rails.

If you then keep the analogue cables as short as absolutely possible, you should have little opportunity for RF to get in.

I don’t recommend the TigerTronics Signalink interfaces, they use cheap and nasty isolation transformers that lead to serious performance issues.

Receive audio

For the receive audio, we feed the audio from the radio and we feed that via potentiometer to a 3.5mm TRS (“phono”) plug tip, with sleeve going to common.  This plugs into the Line-In or Microphone input on the sound device.

Push to Talk and Transmit audio

I’ve bundled these together for a good reason.  The conventional way for computers to drive PTT is via an RS-232 serial port.

We can do that, but we won’t unless we have to.

Unless you’re running an original SoundBLASTER card, your audio interface is likely stereo.  We can get PTT control via an envelope detector forming a minimal-latency VOX control.

Another 3.5mm TRS plug connects to the “headphone” or “line-out” jack on our sound device and breaks out the left and right channels.

The left and right channels from the sound device should be fed into the “throw” contacts on two single-pole double-throw toggle switches.

The select pin (mechanically operated by the toggle handle) on each switch thus is used to select the left or right channel.

One switch’s select pin feeds into a potentiometer, then to the radio’s input.  We will call that the “modulator” switch; it selects which channel “modulates” our audio.  We can again adjust the gain with the potentiometer.

The other switch first feeds through a small Schottky diode then across a small electrolytic capacitor (to 0V) then through a small resistor before finally into the base of a small NPN signal transistor (e.g. BC547).  The emitter goes to 0V, the collector is our PTT signal.

This is the envelope detector we all know and love from our old experiments with crystal sets.  In theory, we could hook a speaker to the collector up to a power source and listen to AM radio stations, but in this case, we’ll be sending a tone down this channel to turn the transistor, and thus or PTT, on.

The switch feeding this arrangement we’ll call the “PTT” switch.

By using this arrangement, we can use either audio channel for modulation or PTT control, or we can use one channel for both.  1200-baud AFSK, FreeDV, etc, should work fine with both on the one channel.

If we just want to pass through analogue audio, then we probably want modulation separate, so we can hold the PTT open during speech breaks without having an annoying tone superimposed on our signal.

It may be prudent to feed a second resistor into the base of that NPN, running off to the RTS pin on an RS-232 interface.  This will let us use software that relies on RS-232 PTT control, which can be added by way of a USB-RS232 dongle.

The cheap Prolific PL-2303 ones sold by a few places (including Jaycar) will work for this.  (If your software expects a 16550 UART interface on port 0x3f8 or similar, consider running it in a virtual machine.)

Ideally though, this should not be needed, and if added, can be left disconnected without harm.

Software

There are a few “off-the-shelf” packages that should work fine with this arrangement.

AX.25 software

AGWPE on Windows provides a software TNC.  On Linux, there’s soundmodem (which I have used, and presently mirror) and Direwolf.

Shouldn’t need a separate PTT channel, it should be sufficient to make the pre-amble long enough to engage PTT and rely on the envelope detector recognising the packet.

Digital Voice

FreeDV provides an open-source digital voice platform system for Windows, Linux and MacOS X.

This tool also lets us send analogue voice.  Digital voice should be fine, the first frame might get lost but as a frame is 40ms, we just wait before we start talking, like we would for regular analogue radio.

For the analogue side of things, we would want tone-driven PTT.  Not sure if that’s supported, but hey, we’ve got the source code, and yours truly has worked with it, it shouldn’t be hard to add.

Slow-scan television

The two to watch here would be QSSTV (Linux) and EasyPal (Windows).  QSSTV is open-source, so if we need to make modifications, we can.

Not sure who maintains EasyPal these days, not Eric VK4AES as he’s no longer with us (RIP and thank-you).  Here, we might need an RS-232 PTT interface, which as discussed, is not a hard modification.

Radioteletype

Most is covered by FLDigi.  Modes with a fairly consistent duty cycle will work fine with the VOX PTT, and once again, we have the source, we can make others work.

Custom software ideas

So we can use a few off-the-shelf packages to do basic comms.

  • We need auditability of our messaging system.  Analogue FM, we can just use a VOX-like function on the computer to record individual received messages, and to record outgoing traffic.  Text messages and files can be logged.
  • Ideally, we should have some digital signing of logs to make them tamper-resistant.  Then we can mathematically prove what was sent.
  • In a true  emergency, it may be necessary to encrypt what we transmit.  This is fine, we’re allowed to do this in such cases, and we can always turn over our audited logs for authorities anyway.
  • Files will be sent as blocks which are forward-error corrected (or forward-erasure coded).  We can use a block cipher such as AES-256 to encrypt these blocks before FEC.  OpenPGP would work well here rather doing it from scratch; just send the OpenPGP output using FEC blocks.  It should be possible to pick out symmetric key used at the receiving end for auditing, this would be done if asked for by Government.  DIY not necessary, the building blocks are there.
  • Digital voice is a stream, we can use block ciphers but this introduces latency and there’s always the issue of bit errors.  Stream ciphers on the other hand, work by generating a key stream, then XOR-ing that with the data.  So long as we can keep sync in the face of bit errors, use of a stream cipher should not impair noise immunity.
  • Signal fade is a worse problem, I suggest a cleartext (3-bit, 4-bit?) gray-code sync field for synchronisation.  Receiver can time the length of a fade, estimate the number of lost frames, then use the field to re-sync.
  • There’s more than a dozen stream ciphers to choose from.  Some promising ones are ACHTERBAHN-128, Grain 128a, HC-256, Phelix, Py, the Salsa20 family, SNOW 2/3G, SOBER-128, Scream, Turing, MUGI, Panama, ISAAC and Pike.
  • Most (all?) stream ciphers are symmetric.  We would have to negotiate/distribute a key somehow, either use Diffie-Hellman or send a generated key as an encrypted file transfer (see above).  The key and both encrypted + decrypted streams could be made available to Government if needed.
  • The software should be capable of:
    • Real-time digital voice (encrypted and clear; the latter being compatible with FreeDV)
    • File transfer (again, clear and encrypted using OpenPGP, and using good FEC, files will be cryptographically signed by sender)
    • Voice mail and SSTV, implemented using file transfer.
    • Radioteletype modes (perhaps PSK31, Olivia, etc), with logs made.
    • Analogue voice pass-through, with recordings made.
    • All messages logged and time-stamped, received messages/files hashed, hashes cryptographically signed (OpenPGP signature)
    • Operation over packet networks (AX.25, TCP/IP)
    • Standard message forms with some basic input validation.
    • Ad-hoc routing between interfaces (e.g. SSB to AX.25, AX.25 to TCP/IP, etc) should be possible.
  • The above stack should ideally work on low-cost single-board computers that are readily available and are low-power.  Linux support will be highest priority, Windows/MacOS X/BSD is a nice-to-have.
  • GNU Radio has building blocks that should let us do most of the above.
Mar 252017
 

So, there’s been a bit of discussion lately about our communications infrastructure. I’ve been doing quite a bit of thinking about the topic.

The situation today

Here in Australia, a lot of people are being moved over to the National Broadband Network… with the analogue fixed line phone (if it hasn’t disappeared already) being replaced with a digital service.

For many, their cellular “mobile” phone is their only means of contact. More than the over-glorified two-way radios that was pre-cellular car phones used by the social elites in the early 70s, or the slightly more sophisticated and tennis-elbow inducing AMPS hand-held mobile phones that we saw in the 80s, mobile phones today are truly versatile and powerful hand-held computers.

In fact, they are more powerful than the teen-aged computer I am typing this on. (And yes, I have upgraded it; 1GB RAM, 250GB mSATA SSD, Linux kernel 4.0… this 2GHz P4 still runs, and yes I’ll update that kernel in a moment. Now, how’s that iPhone 3G going, still running well?)

All of these devices are able to provide data communications throughput in the order of millions of bits per second, and outside of emergencies, are generally, very reliable.

It is easy to forget just how much needs to work properly in order for you to receive that funny cat picture.

Mobile networks

One thing that is not clear about the NBN, is what happens when the power is lost. The electricity grid is not infallible, and requires regular maintenance, so while reliability is good, it is not guaranteed.

For FTTP users, battery backup is an optional extra. If you haven’t opted in, then your “land line” goes down when the power goes out.

This is not a fact that people think about. Most will say, “that’s fine, I’ve got my mobile” … but do you? The typical mobile phone cell tower has several hours battery back-up, and can be overwhelmed by traffic even in non-emergencies.They are fundamentally engineered to a cost, thus compromises are made on how long they can run without back-up power, and how much call capacity they carry.

In the 2008 storms that hit The Gap, I had no mobile telephone coverage for 2 days. My Nokia 3310 would occasionally pick up a signal from a tower in a neighbouring suburb such as Keperra, Red Hill or Bardon, and would thus occasionally receive the odd text message… but rarely could muster the effective radiated power to be able to reply back or make calls. (Yes, and Nokia did tell me that internal antennas surpassed the need for external ones. A 850MHz yagi might’ve worked!)

Emergency Services

Now, you tell yourself, “Well, the emergency services have their own radios…”, and this is correct. They do have their own radio networks. They too are generally quite reliable. They have their problems. The Emergency Alerting System employed in Victoria was having capacity problems as far back as 2006 (emphasis mine):

A high-priority project under the Statewide Integrated Public Safety Communications Strategy was establishing a reliable statewide paging system; the emergency alerting system. The EAS became operational in 2006 at a cost of $212 million. It provides coverage to about 96 per cent of Victoria through more than 220 remote transmitter sites. The system is managed by the Emergency Services Telecommunications Agency on behalf of the State and is used by the CFA, VICSES and Ambulance Victoria (rural) to alert approximately 37,400 personnel, mostly volunteers, to an incident. It has recently been extended to a small number of DSE and MFB staff.

Under the EAS there are three levels of message priority: emergency, non-emergency, and administrative. Within each category the system sends messages on a first-in, first-out basis. This means queued emergency messages are sent before any other message type and non-emergency messages have priority over administrative messages.

A problem with the transmission speed and coverage of messages was identified in 2006. The CFA expressed concern that areas already experiencing marginal coverage would suffer additional message loss when the system reached its limits during peak events.

To ensure statewide coverage for all pagers, in November 2006 EAS users decided to restrict transmission speed and respond to the capacity problems by upgrading the system. An additional problem with the EAS was caused by linking. The EAS can be configured to link messages by automatically sending a copy of a message to another pager address. If multiple copies of a message are sent the overall load on the system increases.

By February 2008 linking had increased by 25 per cent.

During the 2008 windstorm in Victoria the EAS was significantly short of delivery targets for non-emergency and administrative messages. The Emergency Services Telecommunications Agency subsequently reviewed how different agencies were using the system, including their message type selection and message linking. It recommended that the agencies establish business rules about the use of linking and processes for authorising and monitoring de-linking.

The planned upgrade was designed to ensure the EAS could cope better with more messages without the use of linking.

The upgrade was delayed several times and rescheduled for February 2009; it had not been rolled out by the time of Black Saturday. Unfortunately this affected the system on that day, after which the upgrade was postponed indefinitely.

I can find mention of this upgrade taking place around 2013. From what I gather, it did eventually happen, but it took a roasting from mother nature to make it happen. The lesson here is that even purpose built networks can fall over, and thus particularly in major incidents, it is prudent to have a back-up plan.

Alternatives

For the lay person, CB radio can be a useful tool for short-range (longer-than-yelling-range) voice communications. UHF CB will cover a few kilometres in urban environments and can achieve quite long distances if good line-of-sight is maintained. They require no apparatus license, and are relatively inexpensive.

It is worth having a couple of cheap ones, a small torch and a packet of AAA batteries (stored separately) in the car or in a bag you take with you. You can’t use them if they’re in a cupboard at home and you’re not there.

The downside with the hand-helds, particularly the low end ones, is effective radiated power. They will have small “rubber ducky” antennas, optimised for size, and will typically have limited transmit power, some can do the 5W limit, but most will be 1W or less.

If you need a bit more grunt, a mobile UHF CB set and magnetic mount antenna could be assembled and fitted to most cars, and will provide 5W transmit power, capable of about 5-10km in good conditions.

HF (27MHz) CB can go further, and with 12W peak envelope power, it is possible to get across town with one, even interstate or overseas when conditions permit. These too, are worth looking at, and many can be had cheaply second-hand. They require a larger antenna however to be effective, and are less common today.

Beware of fakes though… A CB radio must meet “type approval”, just being technically able to transmit in that band doesn’t automatically make it a CB, it must meet all aspects of the Citizens Band Radio Service Class License to be classified a CB.

If it does more than 5W on UHF, it is not a UHF CB. If it mentions a transmit range outside of 476-478MHz, it is not a UHF CB.  Programming it to do UHF channels doesn’t change this.

Similarly, if your HF CB radio can do 26MHz (NZ CB, not Australia), uses FM instead of SSB/AM (UK CB, again not Australia), does more than 12W, or can do 28-30MHz (10m amateur), it doesn’t qualify as being a CB under the class license.

Amateur radio licensing

If you’ve got a good understanding of high-school mathematics and physics, then a Foundation amateur radio license is well within reach.  In fact, I’d strongly recommend it for anyone doing first year Electrical Engineering … as it will give you a good practical grounding in electrical theories.

Doing so, you get to use up to 10W of power (double what UHF CB gives you; 6dB can matter!) and access to four HF, one VHF and one UHF band using analogue voice or hand-keyed Morse code.

You can then use those “CB radios” that sell on eBay/DealExtreme/BangGood/AliExpress…etc, without issue, as being un-modified “commercial off-the-shelf”, they are acceptable for use under the Foundation license.

Beyond Voice: amateur radio digital modes

Now, all good and well being able to get voice traffic across a couple of suburban blocks. In a large-scale disaster, it is often necessary to co-ordinate recovery efforts, which often means listings of inventory and requirements, welfare information, etc, needs to be broadcast.

You can broadcast this by voice over radio… very slowly!

You can put a spreadsheet on a USB stick and drive it there. You can deliver photos that way too. During an emergency event, roads may be in-passable, or they may be congested. If the regular communications channels are down, how does one get such files across town quickly?

Amateur radio requires operators who have undergone training and hold current apparatus licenses, but this service does permit the transmission of digital data (for standard and advanced licensees), with encryption if needed (“intercommunications when participating in emergency services operations or related training exercises”).

Amateur radio is by its nature, experimental. Lots of different mechanisms have been developed through experiment for intercommunication over amateur radio bands using digital techniques.

Morse code

The oldest by far is commonly known as “Morse code”, and while it is slower than voice, it requires simpler transmitting and receiving equipment, and concentrates the transmitted power over a very narrow bandwidth, meaning it can be heard reliably at times when more sophisticated modes cannot. However, not everybody can send or receive it (yours truly included).

I won’t dwell on it here, as there are more practical mechanisms for transmitting lots of data, but have included it here for completeness. I will point out though, due to its simplicity, it has practically no latency, thus it can be faster than SMS.

Radio Teletype

Okay, there are actually quite a few modes that can be described in this manner, and I’ll use this term to refer to the family of modes. Basically, you can think of it as two dumb terminals linked via a radio channel. When you type text into one, that text appears on the other in near real-time. The latency is thus very low, on par with Morse code.

The earliest of these is the RTTY mode, but more modern incarnations of the same idea include PSK31.

These are normally used as-is. With some manual copying and pasting pieces of text at each end, it is possible to encode other forms of data as short runs of text and send files in short hand-crafted “packets”, which are then hand-deconstructed and decoded at the far end.

This can be automated to remove the human error component.

The method is slow, but these radioteletype modes are known for being able to “punch through” poor signal conditions.

When I was studying web design back in 2001, we were instructed to keep all photos below 30kB in size. At the time, dial-up Internet was common, and loading times were a prime consideration.

Thus instead of posting photos like this, we had to shrink them down, like this. Yes, some detail is lost, but it is good enough to get an “idea” of the situation.

The former photo is 2.8MB, the latter is 28kB. Via the above contrived transmission system, it would take about 20 minutes to transmit.

The method would work well for anything that is text, particularly simple spread sheets, which could be converted to Comma Separated Values to strip all but the most essential information, bringing file sizes down into realms that would allow transmission times in the order of 5 minutes. Text also compresses well, thus in some cases, transmission time can be reduced.

To put this into perspective, a drive from The Gap where that photo was taken, into the Brisbane CBD, takes about 20 minutes in non-peak-hour normal circumstances. It can take an hour at peak times. In cases of natural disaster, the roads available to you may be more congested than usual, thus you can expect peak-hour-like trip times.

Radio Faximile and Slow Scan Television

This covers a wide variety of modes, ranging from the ancient like Hellschreiber which has its origins in the German Military back in World War II, various analogue slow-scan television modes through to the modern digital slow-scan television.

This allows the transmission of photos and visual information over radio. Some systems like EasyPAL and its elk (based on HamDRM, a variant of Digital Radio Mondiale) are in fact, general purpose modems for transmitting files, and thus can transmit non-graphical data too.

Transmit times can vary, but the analogue modes take between 30 seconds and two minutes depending on quality. For the HamDRM-based systems, transmit speeds vary between 86Bps up to 795kBps depending on the settings used.

Packet Radio

Packet radio is the concept of implementing packet-switched networks over radio links. There are various forms of this, the most common in amateur radio being PACTOR, WINMOR, the 1200-baud AFSK and 9600-baud FSK and 300-baud AFSK packet modes.

300-baud AFSK is normally used on HF links, and hails from experiments using surplus Bell 103 modems modified to work with radio. Similarly, on VHF and UHF FM radio, experiments were done with surplus Bell 202 modems, giving rise to the 1200-baud AFSK mode.

The 9600-baud FSK mode was the invention of James Miller G3RUH, and was one of the first packet radio modes actually developed by radio amateur operators for use on radio.

These are all general-purpose data modems, and while they can be used for radioteletype applications, they are designed with computer networking in mind.

The feature facilities like automatic repeating of lost messages, and in some cases support forward error correction. PACTOR/WINMOR is actually used with the Winlink radio network which provides email services.

The 300-baud, 1200-baud and 9600-baud versions generally use a networking protocol called AX.25, and by configuring stations with multiple such “terminal node controllers” (modems) connected and appropriate software, a station can operate as a router, relaying traffic received via one radio channel to a station that’s connected via another, or to non-AX.25 stations on Winlink or the Internet.

It is well suited to automatic stations, operating without human intervention.

AX.25 packet and PACTOR I are open standards, the later PACTOR modems are proprietary devices produced by SCS in Germany.

AX.25 packet is capable of transmit speeds between 15Bps (300 baud) and 1kBps (9600 baud). PACTOR varies between 5Bps and 650Bps.

In theory, it is possible to develop new modems for transmitting AX.25, the HamDRM modem used for slow-scan television and the FDMDV modem used in FreeDV being good starting points as both are proven modems with good performance.

These simply require an analogue interface between the computer sound card and radio, and appropriate software.  Such an interface made to link a 1200-baud TNC to a radio could be converted to link to a low-cost USB audio dongle for connection to a computer.

If someone is set up for 1200-baud packet, setting up for these other modes is not difficult.

High speed data

Going beyond standard radios, amateur radio also has some very high-speed data links available. D-Star Digital Data operates on the 23cm microwave band and can potentially transmit files at up to 16KBps, which approaches ADSL-lite speeds. Transceivers such as the Icom ID-1 provide this via an Ethernet interface for direct connection to a computer.

General Electric have a similar offering for industrial applications that operates on various commercial bands, some of which can reach amateur frequencies, thus would be usable on amateur bands. These devices offer transmit speeds up to 8KBps.

A recent experiment by amateurs using off-the-shelf 50mW 433MHz FSK modules and Realtek-based digital TV tuner receivers produced a high-speed speed data link capable of delivering data at up to 14KBps using a wideband (~230kHz) radio channel on the 70cm band.  They used it to send high definition photos from a high-altitude balloon.

The point?

We’ve got a lot of tools at our disposal for getting a message through, and collectively, 140 years of experience at our disposal. In an emergency situation, that means we have a lot of different options, if one doesn’t work, we can try another.

No, a 1200-baud VHF packet link won’t stream 4k HD video, but it has minimal latency and will take less than 20 minutes to transmit a 100kB file over distances of 10km or more.

A 1kB email will be at the other end before you can reach for your car keys.  Further experimentation and development means we can only improve.  Amateur radio is far from obsolete.

Nov 202016
 

The Yaesu FT-897D has the de-facto standard 6-pin Mini-DIN data jack on the back to which you can plug a digital modem.  Amongst the pins it provides is a squelch status pin, and in the past I’ve tried using that to drive (via transistors) the carrier detect pin on various computer interfaces to enable the modem to detect when a signal is incoming.

The FT-897D is fussy however.  Any load at all pulling this pin down, and you get no audio.  Any load.  One really must be careful about that.

Last week when I tried the UDRC-II, I hit the same problem.  I was able to prove it was the UDRC-II by construction of a crude adapter cable that hooked up to the DB15-HD connector, converting that to Mini-DIN6: by avoiding the squelch status pin, I avoided the problem.

One possible solution was to cut the supplied Mini-DIN6 cable open, locate the offending wire and cut it.  Not a solution I relish doing.  The other was to try and fix the UDRC-II.

Discussing this on the list, it was suggested by Bryan Hoyer that I use a 4.7k pull-up resistor on the offending pin to 3.3V.  He provided a diagram that indicated where to find the needed signals to tap into.

With that information, I performed the following modification.  A 1206 4.7k resistor is tacked onto the squelch status pin, and a small wire run from there to the 3.3V pin on a spare header.

UDRC-II modification for Yaesu FT-897D

UDRC-II modification for Yaesu FT-897D

I’m at two minds whether this should be a diode instead, just in case a radio asserts +12V on this line, I don’t want +12V frying the SoC in the Raspberry Pi.  On the other hand, this is working, it isn’t “broke”.

Doing the above fixed the squelch drive issue and now I’m able to transmit and receive using the UDRC-II.  Many thanks to Bryan Hoyer for pointing this modification out.

Nov 122016
 

So, recently, the North West Digital Radio group generously donated a UDRC II radio control board in thanks for my initial work on an audio driver for the Texas Instruments TLV320AIC3204 (yes, a mouthful).

This board looks like it might support the older Pi model B I had, but I thought I’d play it safe and buy the later revision, so I bought version 3 of the Pi and the associated 7″ touch screen.  Thus, an order went to RS for a whole pile of parts, including one Raspberry Pi3 computer, a blank 8GB MicroSD card, a power supply, the touch screen kit and a case.

Fitting the UDRC

To fit the UDRC, the case will need some of the plastic cut away,  rectangular section out of the main body and a similarly sized portion out of the back cover.

Modifications to the case

Modifications to the case

When assembled, the cut-away section will allow the DB15-HD and Mini-DIN6 connectors to protrude out slightly.

Case assembled with modifications

The UDRC needs some minor modifications too for the touch screen.  Probe around, and you’ll find a source of 5V on one of the unpopulated headers.  You’ll want to solder a two-pin header to here and hook that to the LCD control board using the supplied jumper leads.  If you’ve got one, use a right-angled header, otherwise just bend a regular one like I did.

5V supply for the LCD on the UDRC

5V supply for the LCD on the UDRC

You’ll note I’ve made a note on the DB15-HD, a monitor does NOT plug in here.

From here, you should be ready to load up a SD card.  NWDR recommend the use of Compass Linux, which is a Raspbian fork configured for use with the UDRC.  I used the lite version, since it was smaller and I’m comfortable with command lines.

Configuring screen rotation

If you try to boot your freshly prepared SD card, the first thing you’ll notice is that the screen is up-side-down.  Clearly a few people didn’t communicate with each-other about which way was up on this thing.

Before you pull the SD card out, it is worth mounting the first partition on the SD card and editing config.txt on the root directory of that partition. If doing this on a Windows computer ensure your text editor respects Unix line endings! (Blame Microsoft. If you’re doing this on a Mac, Linux, BSD or other Unix-ish computer, you have nothing to worry about.)

Add the following to the end of the file (or anywhere really):

# Rotate the screen the "right way up"
lcd_rotate=2

Now save the file, unmount the SD card, and put it in the Pi before assembling the case proper.

Setting up your environment

Now, if you chose the lite option like I did, there’ll be no GUI, and the touch aspect of the touchscreen is useless.  You’ll need a USB keyboard.

Log in as pi (password raspberry), run passwd to change your password, then run sudo -s to gain a root shell.

You might choose like I did to run passwd again here to set root‘s password too.

After that, you’ll want to install some software.  Your choice of desktop environment is entirely up to you, I prefer something lightweight, and have been using FVWM for years, but there are plenty of choices in Debian as well as the usual suspects (KDE, Gnome, XFCE…).

For the display manager, I’ll choose lightdm. We also need an on-screen keyboard. I tried a couple, including matchbox-keyboard and the rather ancient xvkbd. Despite its age, I found xvkbd to be the most usable.

Once you’ve decided what you want, run apt-get install with your list of packages, making sure to include xvkbd and lightdm in your list.  Other applications I included here were network-manager-gnome, qasmixer, pasystray, stalonetray and gkrellm.

Enabling the on-screen keyboard in lightdm

Having installed lightdm and xvkbd, you can now configure lightdm to enable the accessibility options.

Open up /etc/lightdm/lightdm-gtk-greeter.conf, look for the line show-indicators and tack ;~a11y on the end.

Now down further, look for the commented out keyboard setting and change that to keyboard=xvkbd. Save and close the file, then run /etc/init.d/lightdm restart.

You should find yourself staring at the log-in screen, and lo and behold, there should be a new icon up the top-right. Tapping it should bring up a 3 line menu, the bottom of which is the on-screen keyboard.

On-screen keyboard in lightdm

On-screen keyboard in lightdm

The button marked Focus is what you hit to tell the keyboard which application is to receive the keyboard events.  Tap that, then the application you want.  To log in, tap Focus then the password field.  You should be able to tap your password in followed by either the Return button on the virtual keyboard or the Log In button on the form.

Making FVWM touch-friendly

I have a pretty old configuration that has evolved over the last 10 years using FVWM that was built around keyboard-centric operation and screen real-estate preservation.  This configuration mainly needed two changes:

  • Menus and title bar text enlarged to make the corresponding UI elements finger-friendly
  • Adjusting the size of the FVWM BarButtons to suit the 800×480 display

Rather than showing how to do it from scratch, I’ll just link to the configuration tarball which you are welcome to play with.  It uses xcalendar which isn’t in the Debian repositories any more, but is available on Gentoo mirrors and can be built from source (you’ll want to install xutils-dev for xmake), stalonetray and gkrellm are both in the standard Debian repositories.

FVWM on the Raspberry Pi

FVWM on the Raspberry Pi

Enabling the right-click

This took a bit of hunting to figure out.  There is a method that works with Debian Wheezy which allows right-clicks by way of long presses, but this broke in Jessie, and the 2016-05-23 release of Compass Linux is built on the latter.  So another solution is needed.

Philipp Merkel however, wrote a little daemon called twofing.  Once installed, doing a right click is simply a two-fingered tap on the screen, there’s support for other two-fingered gestures such as pinching and rotation as well.  It is available on Github, and I have forked this, adding some udev rules and scripts to integrate it into the Raspberry Pi.

The resulting Debian package is here.  Download the .deb, run dpkg -i on it, and then re-start the Raspberry Pi (or you can try running udevadm trigger and re-starting X).  The udev rules should create a /dev/twofingtouch symbolic link and the installed Xsession.d/Xreset.d scripts should take care of starting it with X and shutting it down afterwards.

Having done this, when you log in you should find that twofing is running, and that right clicks can be performed using a two-fingered prod.

Finishing up

Having done the configuration, you should now have a usable workhorse for numerous applications.  The UDRC shows up as a second sound card and is accessible via ALSA.  I haven’t tried it out yet, but it at least shows up in the mixer application, so the signs are there.  I’ll be looking to add LinBPQ and FreeDV into the mix yet, to round the software stack off to make this a general purpose voice/data radio station for emergency communications.

Nov 062016
 

Sometimes, it is desirable to have a TLS-based VPN tunnel for those times when you’re stuck behind an oppressive firewall and need to have secure communications to the outside world.  Maybe you’re visiting China, maybe you’re making an IoT device and don’t want to open your customers’ networks to world+dog by making your device easy to compromise (or have it pick on Brian Krebs).

OpenVPN is able to share a port with a non OpenVPN server.  When a tunnel is established, it looks almost identical to HTTPS traffic because both use TLS.  The only dead giveaway would be the OpenVPN session lasts longer, but then again, in this day of websockets and long polling, who knows how valid that assumption will be?

The lines needed to pull this magic off?  Here, we have sniproxy listening on port 65443. You can use nginx, Apache, or any other HTTPS web server here.  It need only be listening on the IPv4 loopback interface (127.0.0.1) since all connections will be from OpenVPN.

port 443
port-share localhost 65443

There’s one downside.  OpenVPN will not listen on both IPv4 and IPv6.  In fact, it takes a ritual sacrifice to get it to listen to an IPv6 socket at all.  On UDP, it’s somewhat understandable, and yes, they’re working on it.  On TCP, it’s inexcusable, the problems that plague dual-stack sockets on UDP mostly aren’t a problem on TCP.

It’s also impossible to selectively monitor ports.  There’s a workaround however.  Two, in fact.  Both involve deploying a “proxy” to re-direct the traffic.  So to start with, change that “port 443” to another port number, say 65444, and whilst you’re there, you might as well bind OpenVPN to loopback:

local 127.0.0.1
port 65444
port-share localhost 65443

Port 443 is now unbound and you can now set up your proxy.

Workaround 1: redirect using xinetd

The venerable xinetd superserver has a rather handy port redirection feature.  This has the bonus that the endpoint need not be on the same machine, or be dual-stack.


service https_port_forward
{
flags = IPv6               # Use AF_INET6 as the protocol family
disable = no               # Enable this service
type = UNLISTED            # Not listed in standard system file
socket_type = stream       # Use "stream" socket (aka TCP)
protocol = tcp             # Protocol used by the service
user = nobody              # Run proxy as user 'nobody'
wait = no                  # Do not wait for close, spawn a thread instead
redirect = 127.0.0.1 65444 # Where OpenVPN is listening
only_from = ::/0 0.0.0.0/0 # Allow world + dog
port = 443                 # Listen on port 443
}

Workaround 2: socat and supervisord

socat is a Swiss Army knife of networking, able to tunnel just about anything to anything else.  I was actually going to deploy that route, but whilst I was waiting for socat and supervisord to install, I decided to explore xinetd‘s capabilities.  Both will do the job however.

There is a catch though, socat does not daemonise. So you need something that will start it automatically and re-start it if it fails. You might be able to achieve this with systemd, here I’ll use supervisord to do that task.

The command to run is:
socat TCP6-LISTEN:443,fork TCP4:127.0.0.1:65444

and in supervisord you configure this accordingly:

[program:httpsredirect]
directory=/var/run
command=socat TCP6-LISTEN:443,fork TCP4:127.0.0.1:65444"
redirect_stderr=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
autostart=true
autorestart=true

Oct 132016
 

Well, today’s mail had a surprise.  Back about 6 years ago, I was sub-contracted to Jacques Electronics to help them develop some device drivers for their video intercom system.  At the time, they were using TI’s TLV320AIC3204 and system-on-modules based on the Freescale i.MX27 SoC.

No driver existed in the ALSA tree for this particular audio CODEC, and while TI did have one available under NDA, the driver was only licensed for use with a TI OMAP SoC.  I did what just about any developer would do, grabbed the closest-looking existing ALSA SoC driver, ripped it apart and started hacking.  Thus I wound up getting to grips with the I²S infrastructure within the i.MX27 and taming the little beast that is the TLV320AIC3204, producing this patch.

As the code was a derivative work, the code was automatically going to be under the GPLv2 and thus was posted on the ALSA SoC mailing list for others to use.  This would help protect Jacques from any possible GPL infringement regarding the use of that driver.  I was able to do this as it was a clean-room implementation using only material in TI’s data sheet, thus did not contain any intellectual property of my then-employer.

About that time I recall one company using the driver in their IP camera product, the driver itself never made it into the mainline kernel.  About 6 months later, another driver for the TLV320AIC3204 and 3254 did get accepted there, I suspect this too was a clean-room implementation.

Fast forward to late August, I receive an email from Jeremy McDermond on behalf of the Northwest Digital Radio.  They had developed the Universal Digital Radio Controller board for the Raspberry Pi series of computers based around this same CODEC chip.  Interestingly, it was the ‘AIC3204 driver that I developed all that time before that proved to be the code they needed to get the chip working.  The chip in question can be seen up the top-right corner of the board.

Universal Digital Radio Controller

Timely, as there’s a push at the moment within Brisbane Area WICEN Group to investigate possible alternatives to our aging packet radio system and software stack.  These boards, essentially being radio-optimised sound cards, have been used successfully for implementing various digital modes including AX.25 packet, D-Star and could potentially do FreeDV and other digital modes.

So, looks like I’ll be chasing up a supplier for a newer Raspberry Pi board, and seeing what I can do about getting this device talking to the world.

Many thanks to the Northwest Digital Radio company for their generous donation! 🙂

Aug 132016
 

Sometimes I wonder.  Take this evening for example.

I recently purchased some microcontrollers to evaluate for a project, some Atmel ATTiny85s, because they have a rather nice PLL function which means they can do VHF-speed PWM, and some NXP LPC810s, because they happen to be the only DIP-package ARM chip on the market I know of.

The project I’m looking at is a re-work of my bicycle horn… the ATMega32U4 works well, but the LeoStick boards are expensive compared to a bare DIP MCU, and the wiring inside the original prototype is a mess.  I also never got USB working on them, so there’s no point in a USB-capable MCU.

I initially got ATMega1284s owing to the flash storage, but these being 40-pin DIPs, they’re bigger than anticipated, and the fact they’ve got dual USARTs, lots of GPIOs and plenty of storage space, I figured I’d put them aside for another project.

What to use?  Well I have some AT89C2051s from way back (but no programmer for them), some ATTiny24As which I bought for my solar cluster project, an ATMega8L from another project, a LeoStick (Arduino Leonardo clone).  The LeoStick I’m in the process of turning into a debugWire debugger so that I can figure out what the ADCs are doing in my cluster’s power controller (ATTiny24A).

I started building a programmer for the ‘2051s using my ATMega8L last weekend.  The MAX232 IC I grabbed for serial I/O was giving me jibberish, and today I confirmed it was misbehaving.  The board in general is misbehaving in that after flashing the MCU, it seems to stay in reset, so I’ve got more work to do.  If I got that going, I was thinking I could have PCM recordings in an I²C EEPROM and use port 1 on the ‘2051 with an R2R ladder DAC to play sound.  (These chips do not feature PWM.)

Thinking this morning, I thought the LPC810 might be worth a shot.  It only has 4kB of flash, half that of the ATTiny85, and doesn’t have as impressive PWM capabilities, but is good enough.  I really need about 16kB to store the waveforms in flash.  I do have some I²C EEPROMs, mostly <2kB ones that are sourced off old motherboards, but also a handful of 32kB ones that I had just bought especially for this… but then left behind on my desk at work.

I considered audio compression, and experimenting with ADPCM-style techniques, came to the conclusion that I didn’t like the reduced audio quality.  It really sounded harsh.  (Okay, I realise 4-bits per sample is never going to win over the audiophiles!)

Maybe instead of PCM, I could do a crude polyphonic synthesizer?  My horn effect is in fact synthesized using a Python script: the same can be done in C, and the chip probably has the CPU grunt to do it.  It’d save the flash space as I’d be basically doing “poor man’s MIDI” on the thing.  Similar has been done before on lesser hardware.

I did some rough design of data structures.  I figured out a data structure that would allow me to store the state of a “voice” in 8 bytes, and could describe note and timing events in 8-byte blocks.  So in a 2kB EEPROM, I’d store 256 notes, and could easily accommodate 8 or 16 voices in RAM, provided the CPU could keep up at 30MHz.

So, I pull a chip out, slap it in my breadboard, and start hooking it up to power, and to my shiny new USB-TTL serial cable.  Fire up lpc21isp and, nothing, no response from the chip.  Huh?  Check wiring, probe around, still nothing.  Tried different baud rates, etc.  No dice.

This stubborn chip was not going to talk to lpc21isp.  Okay, let’s see if it’ll do SWD.  I dig out my STLink/V2 and hook that up.

OpenOCD reports no response from the device.

Great, maybe a dud chip.  After a good hour or so of fruitless poking and prodding, I pull it out of the breadboard and go to get another from the tube it came from when I notice “Atmel” written on the tube.

I look closer at the chip: it was an ATTiny85!  Different pin-out, different ISP procedure, and even if the .hex file had uploaded, it almost certainly would not have executed.

Swap the chip for an actual LPC810, and OpenOCD reports:

Open On-Chip Debugger 0.10.0-dev-00120-g7a8915f (2015-11-25-18:49)
Licensed under GNU GPL v2
For bug reports, read
http://openocd.org/doc/doxygen/bugs.html
Info : auto-selecting first available session transport "hla_swd". To override use 'transport select '.
Info : The selected transport took over low-level target control. The results might differ compared to plain JTAG/SWD
adapter speed: 10 kHz
adapter_nsrst_delay: 200
Info : Unable to match requested speed 10 kHz, using 5 kHz
Info : Unable to match requested speed 10 kHz, using 5 kHz
Info : clock speed 5 kHz
Info : STLINK v2 JTAG v23 API v2 SWIM v4 VID 0x0483 PID 0x3748
Info : using stlink api v2
Info : Target voltage: 2.979527
Warn : UNEXPECTED idcode: 0x0bc11477
Error: expected 1 of 1: 0x0bb11477
in procedure 'init'
in procedure 'ocd_bouncer'

I haven’t figured out the cause of this yet, whether the ST programmer doesn’t like talking to a competitor’s part. It’d be nice to get SWD going since single-stepping code and peering into memory really spoils a developer like myself. I try lpc21isp again.

Success!  I see a LED blinking, consistent with the demo .hex file I loaded.  Of course now the next step is to try building my own, but at least I can load code onto the device now.

Jul 222016
 

Seems spying on citizens is the new black these days, most government “intelligence” agencies are at it in one form or another. Then the big software companies feel left out, so they join in the fun as well, funneling as much telemetry into their walled garden as possible. (Yes, I’m looking at you, Microsoft.)

This is something I came up with this morning. It’s incomplete, but maybe I can finish it off at some point. I wonder if Cortana has a singing voice?

Partial lyrics for the ASIO/GCHQ/NSA song book