Nov 202016
 

The Yaesu FT-897D has the de-facto standard 6-pin Mini-DIN data jack on the back to which you can plug a digital modem.  Amongst the pins it provides is a squelch status pin, and in the past I’ve tried using that to drive (via transistors) the carrier detect pin on various computer interfaces to enable the modem to detect when a signal is incoming.

The FT-897D is fussy however.  Any load at all pulling this pin down, and you get no audio.  Any load.  One really must be careful about that.

Last week when I tried the UDRC-II, I hit the same problem.  I was able to prove it was the UDRC-II by construction of a crude adapter cable that hooked up to the DB15-HD connector, converting that to Mini-DIN6: by avoiding the squelch status pin, I avoided the problem.

One possible solution was to cut the supplied Mini-DIN6 cable open, locate the offending wire and cut it.  Not a solution I relish doing.  The other was to try and fix the UDRC-II.

Discussing this on the list, it was suggested by Bryan Hoyer that I use a 4.7k pull-up resistor on the offending pin to 3.3V.  He provided a diagram that indicated where to find the needed signals to tap into.

With that information, I performed the following modification.  A 1206 4.7k resistor is tacked onto the squelch status pin, and a small wire run from there to the 3.3V pin on a spare header.

UDRC-II modification for Yaesu FT-897D

UDRC-II modification for Yaesu FT-897D

I’m at two minds whether this should be a diode instead, just in case a radio asserts +12V on this line, I don’t want +12V frying the SoC in the Raspberry Pi.  On the other hand, this is working, it isn’t “broke”.

Doing the above fixed the squelch drive issue and now I’m able to transmit and receive using the UDRC-II.  Many thanks to Bryan Hoyer for pointing this modification out.

Nov 122016
 

So, recently, the North West Digital Radio group generously donated a UDRC II radio control board in thanks for my initial work on an audio driver for the Texas Instruments TLV320AIC3204 (yes, a mouthful).

This board looks like it might support the older Pi model B I had, but I thought I’d play it safe and buy the later revision, so I bought version 3 of the Pi and the associated 7″ touch screen.  Thus, an order went to RS for a whole pile of parts, including one Raspberry Pi3 computer, a blank 8GB MicroSD card, a power supply, the touch screen kit and a case.

Fitting the UDRC

To fit the UDRC, the case will need some of the plastic cut away,  rectangular section out of the main body and a similarly sized portion out of the back cover.

Modifications to the case

Modifications to the case

When assembled, the cut-away section will allow the DB15-HD and Mini-DIN6 connectors to protrude out slightly.

Case assembled with modifications

The UDRC needs some minor modifications too for the touch screen.  Probe around, and you’ll find a source of 5V on one of the unpopulated headers.  You’ll want to solder a two-pin header to here and hook that to the LCD control board using the supplied jumper leads.  If you’ve got one, use a right-angled header, otherwise just bend a regular one like I did.

5V supply for the LCD on the UDRC

5V supply for the LCD on the UDRC

You’ll note I’ve made a note on the DB15-HD, a monitor does NOT plug in here.

From here, you should be ready to load up a SD card.  NWDR recommend the use of Compass Linux, which is a Raspbian fork configured for use with the UDRC.  I used the lite version, since it was smaller and I’m comfortable with command lines.

Configuring screen rotation

If you try to boot your freshly prepared SD card, the first thing you’ll notice is that the screen is up-side-down.  Clearly a few people didn’t communicate with each-other about which way was up on this thing.

Before you pull the SD card out, it is worth mounting the first partition on the SD card and editing config.txt on the root directory of that partition. If doing this on a Windows computer ensure your text editor respects Unix line endings! (Blame Microsoft. If you’re doing this on a Mac, Linux, BSD or other Unix-ish computer, you have nothing to worry about.)

Add the following to the end of the file (or anywhere really):

# Rotate the screen the "right way up"
lcd_rotate=2

Now save the file, unmount the SD card, and put it in the Pi before assembling the case proper.

Setting up your environment

Now, if you chose the lite option like I did, there’ll be no GUI, and the touch aspect of the touchscreen is useless.  You’ll need a USB keyboard.

Log in as pi (password raspberry), run passwd to change your password, then run sudo -s to gain a root shell.

You might choose like I did to run passwd again here to set root‘s password too.

After that, you’ll want to install some software.  Your choice of desktop environment is entirely up to you, I prefer something lightweight, and have been using FVWM for years, but there are plenty of choices in Debian as well as the usual suspects (KDE, Gnome, XFCE…).

For the display manager, I’ll choose lightdm. We also need an on-screen keyboard. I tried a couple, including matchbox-keyboard and the rather ancient xvkbd. Despite its age, I found xvkbd to be the most usable.

Once you’ve decided what you want, run apt-get install with your list of packages, making sure to include xvkbd and lightdm in your list.  Other applications I included here were network-manager-gnome, qasmixer, pasystray, stalonetray and gkrellm.

Enabling the on-screen keyboard in lightdm

Having installed lightdm and xvkbd, you can now configure lightdm to enable the accessibility options.

Open up /etc/lightdm/lightdm-gtk-greeter.conf, look for the line show-indicators and tack ;~a11y on the end.

Now down further, look for the commented out keyboard setting and change that to keyboard=xvkbd. Save and close the file, then run /etc/init.d/lightdm restart.

You should find yourself staring at the log-in screen, and lo and behold, there should be a new icon up the top-right. Tapping it should bring up a 3 line menu, the bottom of which is the on-screen keyboard.

On-screen keyboard in lightdm

On-screen keyboard in lightdm

The button marked Focus is what you hit to tell the keyboard which application is to receive the keyboard events.  Tap that, then the application you want.  To log in, tap Focus then the password field.  You should be able to tap your password in followed by either the Return button on the virtual keyboard or the Log In button on the form.

Making FVWM touch-friendly

I have a pretty old configuration that has evolved over the last 10 years using FVWM that was built around keyboard-centric operation and screen real-estate preservation.  This configuration mainly needed two changes:

  • Menus and title bar text enlarged to make the corresponding UI elements finger-friendly
  • Adjusting the size of the FVWM BarButtons to suit the 800×480 display

Rather than showing how to do it from scratch, I’ll just link to the configuration tarball which you are welcome to play with.  It uses xcalendar which isn’t in the Debian repositories any more, but is available on Gentoo mirrors and can be built from source (you’ll want to install xutils-dev for xmake), stalonetray and gkrellm are both in the standard Debian repositories.

FVWM on the Raspberry Pi

FVWM on the Raspberry Pi

Enabling the right-click

This took a bit of hunting to figure out.  There is a method that works with Debian Wheezy which allows right-clicks by way of long presses, but this broke in Jessie, and the 2016-05-23 release of Compass Linux is built on the latter.  So another solution is needed.

Philipp Merkel however, wrote a little daemon called twofing.  Once installed, doing a right click is simply a two-fingered tap on the screen, there’s support for other two-fingered gestures such as pinching and rotation as well.  It is available on Github, and I have forked this, adding some udev rules and scripts to integrate it into the Raspberry Pi.

The resulting Debian package is here.  Download the .deb, run dpkg -i on it, and then re-start the Raspberry Pi (or you can try running udevadm trigger and re-starting X).  The udev rules should create a /dev/twofingtouch symbolic link and the installed Xsession.d/Xreset.d scripts should take care of starting it with X and shutting it down afterwards.

Having done this, when you log in you should find that twofing is running, and that right clicks can be performed using a two-fingered prod.

Finishing up

Having done the configuration, you should now have a usable workhorse for numerous applications.  The UDRC shows up as a second sound card and is accessible via ALSA.  I haven’t tried it out yet, but it at least shows up in the mixer application, so the signs are there.  I’ll be looking to add LinBPQ and FreeDV into the mix yet, to round the software stack off to make this a general purpose voice/data radio station for emergency communications.

Oct 132016
 

Well, today’s mail had a surprise.  Back about 6 years ago, I was sub-contracted to Jacques Electronics to help them develop some device drivers for their video intercom system.  At the time, they were using TI’s TLV320AIC3204 and system-on-modules based on the Freescale i.MX27 SoC.

No driver existed in the ALSA tree for this particular audio CODEC, and while TI did have one available under NDA, the driver was only licensed for use with a TI OMAP SoC.  I did what just about any developer would do, grabbed the closest-looking existing ALSA SoC driver, ripped it apart and started hacking.  Thus I wound up getting to grips with the I²S infrastructure within the i.MX27 and taming the little beast that is the TLV320AIC3204, producing this patch.

As the code was a derivative work, the code was automatically going to be under the GPLv2 and thus was posted on the ALSA SoC mailing list for others to use.  This would help protect Jacques from any possible GPL infringement regarding the use of that driver.  I was able to do this as it was a clean-room implementation using only material in TI’s data sheet, thus did not contain any intellectual property of my then-employer.

About that time I recall one company using the driver in their IP camera product, the driver itself never made it into the mainline kernel.  About 6 months later, another driver for the TLV320AIC3204 and 3254 did get accepted there, I suspect this too was a clean-room implementation.

Fast forward to late August, I receive an email from Jeremy McDermond on behalf of the Northwest Digital Radio.  They had developed the Universal Digital Radio Controller board for the Raspberry Pi series of computers based around this same CODEC chip.  Interestingly, it was the ‘AIC3204 driver that I developed all that time before that proved to be the code they needed to get the chip working.  The chip in question can be seen up the top-right corner of the board.

Universal Digital Radio Controller

Timely, as there’s a push at the moment within Brisbane Area WICEN Group to investigate possible alternatives to our aging packet radio system and software stack.  These boards, essentially being radio-optimised sound cards, have been used successfully for implementing various digital modes including AX.25 packet, D-Star and could potentially do FreeDV and other digital modes.

So, looks like I’ll be chasing up a supplier for a newer Raspberry Pi board, and seeing what I can do about getting this device talking to the world.

Many thanks to the Northwest Digital Radio company for their generous donation! 🙂

Nov 212015
 

Well, in the last post I started to consider the thoughts of building my own computer from a spare 386 CPU I had liberated from an old motherboard.

One of the issues I face is implementing the bus protocol that the 386 uses, and decoding of interrupts.  The 386 expects an 8-bit interrupt request number that corresponds to the interrupting device.  I’m used to microcontrollers where you use a single GPIO line, but in this case, the interrupts are multiplexed.

For basic needs, you could do it with a demux IC.  That will work for a small number of interrupt lines.  Suppose I wanted more though?  How feasible is it to support many interrupt lines without tying up lots of GPIO lines?

CANBus has an interesting way of handling arbitration.  The “zeros” are dominant, and thus overrule “ones”.  The CAN transceiver is a full-duplex device, so as the station is transmitting, it listens to the state of the bus.  When some nodes want to talk (they are, of course, oblivious to each-others’ intentions), they start sending a start-bit (a zero) which synchronises all nodes, then begin sending an address.

While each node is sending the same “bit value”, the receiving nodes see that value.  As each node tries sending a 1 while the others are sending 0’s, it sees the disparity, and concludes that it has lost arbitration.  Eventually, you’re left with a single node that then proceeds to send its CANBus frame.

Now, we don’t need the complexity of CANBus to do what we’re after.  We can keep synchronisation by simple virtue that we can distribute a common clock (the one the CPU runs at).  Dominant and recessive bits can be implemented with transistors pulling down on a pull-up resistor, or a diode-OR: this will give us a system where ‘1’s are dominant.  Good enough.

So I figured up Logisim to have a fiddle, came up with this:

Interrupt controller using logic gates

Interrupt controller using logic gates

interrupt.circ is the actual LogiSim circuit if you wanted to have a fiddle; decompress it.  Please excuse the mess regarding the schematic.

On the left is the host-side of the interrupt controller.  This would ultimately interface with the 386.  On the right, are two “devices”, one on IRQ channel 0x01, the other on 0x05.  The controller handles two types of interrupts: “DMA interrupts”, where the device just wants to tell the DMA controller to put data into memory, or “IRQ”s, where we want to interrupt the CPU.

The devices are provided with the following control signals from the interrupt controller:

Signal Controlled by Description
DMA Devices Informs the IRQ controller if we’re interrupting for DMA purposes (high) or if we need to tell the CPU something (low).
IRQ Devices Informs the IRQ controller we want its attention
ISYNC Controller Informs the devices that they have the controller’s attention and to start transmitting address bits.
IRQBIT[2…0] Controller Instructs the devices what bit of their IRQ address to send (0 = MSB, 7 = LSB).
IDA Devices The inverted address bit value corresponding to the bit pointed to by IRQBIT.
IACK Devices Asserted by the device that wins arbitration.

Due to the dominant/recessive nature of the bits, the highest numbered device wins over lesser devices. IRQ requests also dominate over DMA requests.

In the schematic, the devices each have two D-flip-flops that are not driven by any control signals.  These are my “switches” for toggling the state of the device as a user.  The ones feeding into the XOR gate control the DMA signal, the others control the IRQ line.

Down the bottom, I’ve wired up a counter to count how long between the ISYNC signal going high and the controller determining a result.  This controller manages to determine which device requested its attention within 10 cycles.  If clocked at the same 20MHz rate as the CPU core, this would be good enough for getting a decoded IRQ channel number to the data lines of the 386 CPU by the end of its second IRQ acknowledge cycle, and can handle up to 256 devices.

A logical next step would be to look at writing this in Verilog and trying it out on an FPGA.  Thanks to the excellent work of Clifford Wolf in producing the IceStorm project, it is now possible to do this with completely open tools.  So, I’ve got a Lattice iCE40HX-8K FPGA board coming.  This should make a pretty mean SDRAM controller, interrupt controller and address decoder all in one chip, and should be a great introduction into configuring FPGAs.

Nov 072015
 

Well, I’ve been thinking a lot lately about single board computers. There’s a big market out there. Since the Raspberry Pi, there’s been a real explosion available to the small-end of town, the individual. Prior to this, development boards were mostly in the 4-figures sort of price range.

So we’re now rather spoiled for choice. I have a Raspberry Pi. There’s also the BeagleBone Black, Banana Pi, and several others. One gripe I have with the Raspberry Pi is the complete absence of any kind of analogue input. There’s an analogue line out, you can interface some USB audio devices (although I hear two is problematic), or you can get an I2S module.

There’s a GPU in there that’s capable of some DSP work and a CLKOUT pin that can generate a wide range of frequencies. That sounds like the beginnings of a decent SDR, however one glitch, while I can use the CLKOUT pin to drive a mixer and the GPIOs to do band selection, there’s nothing that will take that analogue signal and sample it.

If I want something wider than audio frequencies (and even a 192kHz audio CODEC is not guaranteed above ~20kHz) I have to interface to SPI, and the pickings are somewhat slim. Then I read this article on a DIY single board computer.

That got me thinking about whether I could do my own. At work we use the Technologic Systems TS-7670 single-board computers, and as nice as those machines are, they’re a little slow and RAM-limited. Something that could work as a credible replacement there too would be nice, key needs there being RS-485, Ethernet and a 85 degree temperature rating.

Form factor is a consideration here, and I figured something modular, using either header pins or edge connectors would work. That would make the module easily embeddable in hobby projects.

Since all the really nice SoCs are BGA packages, I figured I’d first need to know how easy I could work with them. We’ve got a stack of old motherboards sitting in a cupboard that I figured I could raid for BGAs to play with, just to see first-hand how fine the pins were. A crazy thought came to me: maybe for prototyping, I could do it dead-bug style?

Key thing here being able to solder directly to a ball securely, then route the wire to its destination. I may need to glue it to a bit of grounded foil to keep the capacitance in check. So, the first step I figured, would be to try removing some components from the boards I had laying around to see this first-hand.

In amongst the boards I came across was one old 386 motherboard that I initially mistook for a 286 minus the CPU. The empty (PLCC) socket is for an 80387 math co-processor. The board was in the cupboard for a good reason, corrosion from the CMOS battery had pretty much destroyed key traces on one corner of the board.

Corrosion on a motherboard caused by a CMOS battery

Corrosion on a motherboard caused by a CMOS battery

I decided to take to it with the heat gun first. The above picture was taken post-heatgun, but you can see just how bad the corrosion was. The ISA slots were okay, and so where a stack of other useful IC sockets, ICs, passive components, etc.

With the heat gun at full blast, I’d just wave it over an area of interest until the board started to de-laminate, then with needle-nose pliers, pull the socket or component from the board. Sometimes the component simply dropped out.

At one point I heard a loud “plop”. Looking under the board, one of the larger surface-mounted chips had fallen off. That gave me an idea, could the 386 chip be de-soldered? I aimed the heat-gun directly at the area underneath. A few seconds later and it too hit the deck.

All in all, it was a successful haul.

Parts off the 386 motherboard

Parts off the 386 motherboard

I also took apart an 8-bit ISA joystick card. It had some nice looking logic chips that I figured could be re-purposed. The real star though was the CPU itself:

Intel NG80306SX-20

Intel NG80306SX-20

The question comes up, what does one do with a crusty old 386 that’s nearly as old as I am? A quick search turned up this scanned copy of the Intel 80386SX datasheet. The chip has a 16-bit bus with 23 bits worth of address lines (bit 0 is assumed to be zero). It requires a clock that is double the chip’s operating frequency (there’s an internal divide-by-two). This particular chip runs internally at 20MHz. Nothing jumped out as being scary. Could I use this as a practice run for making an ARM computer module?

A dig around dug up some more parts:

More parts

More parts

In this pile we have…

I also have some SIMMs laying around, but the SDRAM modules look easier to handle since the controllers on board synchronise with what would otherwise be the front-side bus.  The datasheet does not give a minimum clock (although clearly this is not DC; DRAM does need to be refreshed) and mentions a clock frequency of 33MHz when set to run at a CAS latency of 1.  It just so happens that I have a 33MHz oscillator.  There’s a couple of nits in this plan though:

  • the SDRAM modules a 3.3V, the CPU is 5V: no problem, there are level conversion chips out there.
  • the SDRAM modules are 64-bits wide.  We’ll have to buffer the output to eight 8-bit registers.  Writes do a read-modify-write cycle, and we use a 2-in-4 decoder to select the CE pin on two of the registers from address bits 1 and 2 from the CPU.
  • Each SDRAM module holds 32MB.  We have a 23-bit address bus, which with 16-bit words gives us a total address space of 16MB.  Solution: the old 8-bit computers of yesteryear used bank-switching to address more RAM/ROM than they had address lines for, we can interface an 8-bit register at I/O address 0x0000 (easily decoded with a stack of Schottky diodes and a NOT gate) which can hold the remaining address bits mapping the memory to the lower 8MB of physical memory.  We then hijack the 386’s MMU to map the 8MB chunks and use the page faults to switch memory banks.  (If we put the SRAM and ROM up in the top 1MB, this gives us ~7MB of memory-mapped I/O to play with.)

So, not show stoppers.  There’s an example circuit showing interfacing an ATMega8515 to a single SDRAM chip for driving a VGA interface, and some example code, with comments in German. Unfortunately you’d learn more German in an episode of Hogan’s Heroes than what I know, but I can sort-of figure out the sequence used to read and write from/to the SDRAM chip. Nothing looks scary there either.  This SDRAM tutorial seems to be a goldmine.

Thus, it looks like I’ve got enough bits to have a crack at it.  I can run the 386 from that 33MHz brick; which will give me a chip running at 16.5MHz.  Somewhere I’ve got the 40MHz brick laying around from the motherboard (I liberated that some time ago), but that can wait.

A first step would be to try interfacing the 386 chip to an AVR, and feed it instructions one step at a time, check that it’s still alive.  Then, the next steps should become clear.

Sep 272015
 

Well, lately I’ve been doing a bit of work hacking the firmware on the Rowetel SM1000 digital microphone.  For those who don’t know it, this is a hardware (microcontroller) implementation of the FreeDV digital voice mode: it’s a modem that plugs into the microphone/headphone ports of any SSB-capable transceiver and converts FreeDV modem tones to analogue voice.

I plan to set this unit of mine up on the bicycle, but there’s a few nits that I had.

  • There’s no time-out timer
  • The unit is half-duplex

If there’s no timeout timer, I really need to hear the tones coming from the radio to tell me it has timed out.  Others might find a VOX feature useful, and there’s active experimentation in the FreeDV 700B mode (the SM1000 currently only supports FreeDV 1600) which has been very promising to date.

Long story short, the unit needed a more capable UI, and importantly, it also needed to be able to remember settings across power cycles.  There’s no EEPROM chip on these things, and while the STM32F405VG has a pin for providing backup-battery power, there’s no battery or supercapacitor, so the SM1000 forgets everything on shut down.

ST do have an application note on their website on precisely this topic.  AN3969 (and its software sources) discuss a method for using a portion of the STM32’s flash for this task.  However, I found their “license” confusing.  So I decided to have a crack myself.  How hard can it be, right?

There’s 5 things that a virtual EEPROM driver needs to bear in mind:

  • The flash is organised into sectors.
  • These sectors when erased contain nothing but ones.
  • We store data by programming zeros.
  • The only way to change a zero back to a one is to do an erase of the entire sector.
  • The sector may be erased a limited number of times.

So on this note, a virtual EEPROM should aim to do the following:

  • It should keep tabs on what parts of the sector are in use.  For simplicity, we’ll divide this into fixed-size blocks.
  • When a block of data is to be changed, if the change can’t be done by changing ones to zeros, a copy of the entire block should be written to a new location, and a flag set (by writing zeros) on the old block to mark it as obsolete.
  • When a sector is full of obsolete blocks, we may erase it.
  • We try to put off doing the erase until such time as the space is needed.

Step 1: making room

The first step is to make room for the flash variables.  They will be directly accessible in the same manner as variables in RAM, however from the application point of view, they will be constant.  In many microcontroller projects, there’ll be several regions of memory, defined by memory address.  This comes from the datasheet of your MCU.

An example, taken from the SM1000 firmware, prior to my hacking (stm32_flash.ld at r2389):

/* Specify the memory areas */
MEMORY
{
  FLASH (rx)      : ORIGIN = 0x08000000, LENGTH = 1024K
  RAM (rwx)       : ORIGIN = 0x20000000, LENGTH = 128K
  CCM (rwx)       : ORIGIN = 0x10000000, LENGTH = 64K
}

The MCU here is the STM32F405VG, which has 1MB of flash starting at address 0x08000000. This 1MB is divided into (in order):

  • Sectors 0…3: 16kB starting at 0x08000000
  • Sector 4: 64kB starting at 0x0800c000
  • Sector 5 onwards: 128kB starting at 0x08010000

We need at least two sectors, as when one fills up, we will swap over to the other. Now it would have been nice if the arrangement were reversed, with the smaller sectors at the end of the device.

The Cortex M4 CPU is basically hard-wired to boot from address 0, the BOOT pins on the STM32F4 decide how that gets mapped. The very first few instructions are the interrupt vector table, and it MUST be the thing the CPU sees first. Unless told to boot from external memory, or system memory, then address 0 is aliased to 0x08000000. i.e. flash sector 0, thus if you are booting from internal flash, you have no choice, the vector table MUST reside in sector 0.

Normally code and interrupt vector table live together as one happy family. We could use a couple of 128k sectors, but 256k is rather a lot for just an EEPROM storing maybe 1kB of data tops. Two 16kB sectors is just dandy, in fact, we’ll throw in the third one for free since we’ve got plenty to go around.

However, the first one will have to be reserved for the interrupt vector table that will have the space to itself.

So here’s what my new memory regions look like (stm32_flash.ld at 2390):

/* Specify the memory areas */
MEMORY
{
  /* ISR vectors *must* be placed here as they get mapped to address 0 */
  VECTOR (rx)     : ORIGIN = 0x08000000, LENGTH = 16K
  /* Virtual EEPROM area, we use the remaining 16kB blocks for this. */
  EEPROM (rx)     : ORIGIN = 0x08004000, LENGTH = 48K
  /* The rest of flash is used for program data */
  FLASH (rx)      : ORIGIN = 0x08010000, LENGTH = 960K
  /* Memory area */
  RAM (rwx)       : ORIGIN = 0x20000000, LENGTH = 128K
  /* Core Coupled Memory */
  CCM (rwx)       : ORIGIN = 0x10000000, LENGTH = 64K
}

This is only half the story, we also need to create the section that will be emitted in the ELF binary:

SECTIONS
{
  .isr_vector :
  {
    . = ALIGN(4);
    KEEP(*(.isr_vector))
    . = ALIGN(4);
  } >FLASH

  .text :
  {
    . = ALIGN(4);
    *(.text)           /* .text sections (code) */
    *(.text*)          /* .text* sections (code) */
    *(.rodata)         /* .rodata sections (constants, strings, etc.) */
    *(.rodata*)        /* .rodata* sections (constants, strings, etc.) */
    *(.glue_7)         /* glue arm to thumb code */
    *(.glue_7t)        /* glue thumb to arm code */
	*(.eh_frame)

    KEEP (*(.init))
    KEEP (*(.fini))

    . = ALIGN(4);
    _etext = .;        /* define a global symbols at end of code */
    _exit = .;
  } >FLASH…

There’s rather a lot here, and so I haven’t reproduced all of it, but this is the same file as before at revision 2389, but a little further down. You’ll note the .isr_vector is pointed at the region called FLASH which is most definitely NOT what we want. The image will not boot with the vectors down there. We need to change it to put the vectors in the VECTOR region.

Whilst we’re here, we’ll create a small region for the EEPROM.

SECTIONS
{
  .isr_vector :
  {
    . = ALIGN(4);
    KEEP(*(.isr_vector))
    . = ALIGN(4);
  } >VECTOR


  .eeprom :
  {
    . = ALIGN(4);
    *(.eeprom)         /* special section for persistent data */
    . = ALIGN(4);
  } >EEPROM


  .text :
  {
    . = ALIGN(4);
    *(.text)           /* .text sections (code) */
    *(.text*)          /* .text* sections (code) */

THAT’s better! Things will boot now. However, there is still a subtle problem that initially caught me out here. Sure, the shiny new .eeprom section is unpopulated, BUT the linker has helpfully filled it with zeros. We cannot program zeroes back into ones! Either we have to erase it in the program, or we tell the linker to fill it with ones for us. Thankfully, the latter is easy (stm32_flash.ld at 2395):

  .eeprom :
  {
    . = ALIGN(4);
    KEEP(*(.eeprom))   /* special section for persistent data */
    . = ORIGIN(EEPROM) + LENGTH(EEPROM) - 1;
    BYTE(0xFF)
    . = ALIGN(4);
  } >EEPROM = 0xff

Credit: Erich Styger

We have to do two things. One, is we need to tell it that we want the region filled with the pattern 0xff. Two, we need to make sure it gets filled with ones by telling the linker to write one as the very last byte. Otherwise, it’ll think, “Huh? There’s nothing here, I won’t bother!” and leave it as a string of zeros.

Step 2: Organising the space

Having made room, we now need to decide how to break this data up.  We know the following:

  • We have 3 sectors, each 16kB
  • The sectors have an endurance of 10000 program-erase cycles

Give some thought as to what data you’ll be storing.  This will decide how big to make the blocks.  If you’re storing only tiny bits of data, more blocks makes more sense.  If however you’ve got some fairly big lumps of data, you might want bigger blocks to reduce overheads.

I ended up dividing the sectors into 256-byte blocks.  I figured that was a nice round (binary sense) figure to work with.  At the moment, we have 16 bytes of configuration data, so I can do with a lot less, but I expect this to grow.  The blocks will need a header to tell you whether or not the block is being used.  Some checksumming is usually not a bad idea either, since that will clue you in to when the sector has worn out prematurely.  So some data in each block will be header data for our virtual EEPROM.

If we don’t care about erase cycles, this is fine, we can just make all blocks data blocks, however it’d be wise to track this, and avoid erasing and attempting to use a depleted sector, so we need somewhere to track this.  256 bytes gives us enough space to stash an erase counter and a map of what blocks are in use within that sector.

So we’ll reserve the first block in the sector to act as this index for the entire sector.  This gives us enough room to have 16-bits worth of flags for each block stored in the index.  That gives us 63 blocks per sector for data use.

It’d be handy to be able to use this flash region for a few virtual EEPROMs, so we’ll allocate some space to give us a virtual ROM ID.  It is prudent to do some checksumming, and the STM32F4 has a CRC32 module, so in that goes, and we might choose to not use all of a block, so we should throw in a size field (8 bits, since the size can’t be bigger than 255).  If we pad this out a bit to give us a byte for reserved data, we get a header with the following structure:

15 14 13 12 11 10 19 8 7 6 5 4 3 2 1 0
+0 CRC32 Checksum
+2
+4 ROM ID Block Index
+6 Block Size Reserved

So that subtracts 8 bytes from the 256 bytes, leaving us 248 for actual program data. If we want to store 320 bytes, we use two blocks, block index 0 stores bytes 0…247 and has a size of 248, and block index 1 stores bytes 248…319 and has a size of 72.

I mentioned there being a sector header, it looks like this:

15 14 13 12 11 10 19 8 7 6 5 4 3 2 1 0
+0 Program Cycles Remaining
+2
+4
+6
+8 Block 0 flags
+10 Block 1 flags
+12 Block 2 flags

No checksums here, because it’s constantly changing.  We can’t re-write a CRC without erasing the entire sector, we don’t want to do that unless we have to.  The flags for each block are currently allocated accordingly:

15 14 13 12 11 10 19 8 7 6 5 4 3 2 1 0
+0 Reserved In use

When the sector is erased, all blocks show up as having all flags set as ones, so the flags is considered “inverted”.  When we come to use a block, we mark the “in use” bit with a zero, leaving the rest as ones.  When we erase, we mark the entire flags block as zeros.  We can set other bits here as we need for accounting purposes.

Thus we have now a format for our flash sector header, and for our block headers.  We can move onto the algorithm.

Step 3: The Code

This is the implementation of the above ideas.  Our code needs to worry about 3 basic operations:

  • reading
  • writing
  • erasing

This is good enough if the size of a ROM image doesn’t change (normal case).  For flexibility, I made my code so that it works crudely like a file, you can seek to any point in the ROM image and start reading/writing, or you can blow the whole thing away.

Constants

It is bad taste to leave magic numbers everywhere, so constants should be used to represent some quantities:

  • VROM_SECT_SZ=16384:
    The virtual ROM sector size in bytes.  (Those watching Codec2 Subversion will note I cocked this one up at first.)
  • VROM_SECT_CNT=3:
    The number of sectors.
  • VROM_BLOCK_SZ=256:
    The size of a block
  • VROM_START_ADDR=0x08004000:
    The address where the virtual ROM starts in Flash
  • VROM_START_SECT=1:
    The base sector number where our ROM starts
  • VROM_MAX_CYCLES=10000:
    Our maximum number of program-erase cycles

Our programming environment may also define some, for example UINTx_MAX.

Derived constants

From the above, we can determine:

  • VROM_DATA_SZ = VROM_BLOCK_SZ – sizeof(block_header):
    The amount of data per block.
  • VROM_BLOCK_CNT = VROM_SECT_SZ / VROM_BLOCK_SZ:
    The number of blocks per sector, including the index block
  • VROM_SECT_APP_BLOCK_CNT = VROM_BLOCK_CNT – 1
    The number of application blocks per sector (i.e. total minus the index block)

CRC32 computation

I decided to use the STM32’s CRC module for this, which takes its data in 32-bit words.  There’s also the complexity of checking the contents of a structure that includes its own CRC.  I played around with Python’s crcmod module, but couldn’t find some arithmetic that would allow it to remain there.

So I copy the entire block, headers and all to a temporary copy (on the stack), set the CRC field to zero in the header, then compute the CRC. Since I need to read it in 32-bit words, I pack 4 bytes into a word, big-endian style. In cases where I have less than 4 bytes, the least-significant bits are left at zero.

Locating blocks

We identify each block in an image by the ROM ID and the block index.  We need to search for these when requested, as they can be located literally anywhere in flash.  There are probably cleverer ways to do this, but I chose the brute force method.  We cycle through each sector and block, see if the block is allocated (in the index), see if the checksum is correct, see if it belongs to the ROM we’re looking for, then look and see if it’s the right index.

Reading data

To read from the above scheme, having been told a ROM ID (rom), start offset and a size, the latter two being in byte sand given a buffer we’ll call out, we first need to translate the start offset to a sector and block index and block offset.  This is simple integer division and modulus.

The first and last blocks of our read, we’ll probably only read part of.  The rest, we’ll read entire blocks in.  The block offset is only relevant for this first block.

So we start at the block we calculate to have the start of our data range.  If we can’t find it, or it’s too small, then we stop there, otherwise, we proceed to read out the data.  Until we run out of data to read, we increment the block index, try to locate the block, and if found, copy its data out.

Writing and Erasing

Writing is a similar affair.  We look for each block, if we find one, we overwrite it by copying the old data to a temporary buffer, copy our new data in over the top then mark the old block as obsolete before writing the new one out with a new checksum.

Trickery is in invoking the wear levelling algorithm on an as-needed basis.  We mark a block obsolete by setting its header fields to zero, but when we run out of free blocks, then we go looking for sectors that are full of obsolete blocks waiting to be erased.  When we encounter a sector that has been erased, we write a new header at the start and proceed to use its first data block.

In the case of erasing, we don’t bother writing anything out, we just mark the blocks as obsolete.

Implementation

The full C code is in the Codec2 Subversion repository.  For those who prefer Git, I have a git-svn mirror (yes, I really should move it off that domain).  The code is available under the Lesser GNU General Public License v2.1 and may be ported to run on any CPU you like, not just ST’s.

Sep 262015
 

Well, a little nit I have to pick with chip manufacturers. On this occasion, it’s with ST, but they all do it, Freescale, TI, Atmel…

I’m talking about the assumptions they make about who uses their site.

Yes, I work as a “systems engineer” (really, programmer and network administrator, my role is more IT than Engineering).  However, when I’m looking at chip designs and application notes, that is usually in my recreation.

This morning, I had occasion to ask ST a question about one of their application notes.  Specifically AN3969, which deals with emulating an EEPROM using the in-built flash on a STM32F4 microcontroller.  Their “license” states:

   License

      The enclosed firmware and all the related documentation are not covered
      by a License Agreement, if you need such License you can contact your
      local STMicroelectronics office.

      THE PRESENT FIRMWARE WHICH IS FOR GUIDANCE ONLY AIMS AT PROVIDING
      CUSTOMERS WITH CODING INFORMATION REGARDING THEIR PRODUCTS IN ORDER FOR
      THEM TO SAVE TIME. AS A RESULT, STMICROELECTRONICS SHALL NOT BE HELD
      LIABLE FOR ANY DIRECT, INDIRECT OR CONSEQUENTIAL DAMAGES WITH RESPECT
      TO ANY CLAIMS ARISING FROM THE CONTENT OF SUCH FIRMWARE AND/OR THE USE
      MADE BY CUSTOMERS OF THE CODING INFORMATION CONTAINED HEREIN IN
      CONNECTION WITH THEIR PRODUCTS.

Hmm, not licensed, but under a heading called “license”. Does that mean it’s public domain? Probably not. Do I treat this like MIT/BSD license? I’m looking to embed this into LGPLed firmware that will be publicly distributed: I really need an answer to this.  So over to the ST website I trundle.

I did have an account, but couldn’t think of the password.  They’ve revamped their site and I also have a new email address, so I figure, time for a new account.  I click their register link, and get this form:

ST Website registration

ST Website registration

Now, here’s where I have a gripe. Why do they always assume I am doing this for work purposes? This is something pretty much all the manufacturers do. The assumption is WRONG. My account on their website has absolutely nothing to do with my employer. I am doing this for recreation! Therefore, should not, mention them in any way.

Yet, they’re mandatory fields. I guess ST get a lot of employees of the “individual – not a company” company.

I filled out the form, got an email with a confirmation link which I click, and now this is something a lot of companies, not just chip makers, get wrong. Apart from the “wish it was” two factor (you can tell my answer was bogus), they dictate some minimum requirements, but then enforce undisclosed maximum requirements on the password.

ST Website password

ST Website password

WTF? “Special” characters? You mean like printable-ASCII characters? Or did a vertical tab slip in there somehow?  Password security, done properly, should not care how long, or how complex you choose to make your password: so long as it meets a minimum standard.  A maximum length in the order of 64 bytes or more might be reasonable, as might be a restriction to what can be typed on a “standard” US-style keyboard layout may be understandable.

In this case, the password had some punctuation characters.  Apparently these are “special”.  If they restrict them because of possible SQL injection, then I’m afraid ST, you are doing it wrong!  A base64 or hex encoded hash from something like bcrypt, PKCS12 or the like, should make such things impossible.

Obviously preventing abuse by preventing someone from using the dd-dump of a full-length Blu-ray movie as a password is perfectly acceptable, but once hashed, all passwords will be the same size and will contain no “special” characters that could upset middleware.

Sure, enforce a large maximum length (not 20 characters like eBay, but closer to 100) so that any reasonably long password won’t overflow a buffer.  Sure, enforce that some mixed character classes be used.  But don’t go telling people off for using a properly secure password!

May 032015
 

The Problem

I’ve been running a station from the bicycle for some time now and I suppose I’ve tried a few different battery types on the station.

Originally I ran 9Ah 12V gel cells, which work fine for about 6 months, then the load of the radio gets a bit much and I find myself taking two with me on a journey to work because one no longer lasts the day.  I replaced this with a 40Ah Thundersky LiFePO4 pack which I bought from EVWorks, which while good, weighed 8kg!  This is a lot lighter than an equivalent lead acid, gel cell or AGM battery, but it’s still a hefty load for a bicycle.

At the time that was the smallest I could get.  Eventually I found a mob that sold 10Ah packs. These particular cells were made by LiFeBatt, and while pricey, I’ve pretty much recouped my costs. (I’d have bought and disposed of about 16 gel cell batteries in this time at $50 each, versus $400 for one of these.)   These are what I’ve been running now since about mid 2011, and they’ve been pretty good for my needs.  They handle the load of the FT-857 okay on 2m FM which is what I use most of the time.

A week or two back though, I was using one of these packs outside with the home base in a “portable” set-up with my FT-897D.  Tuned up on the 40m WICEN net on 7075kHz, a few stations reported that I had scratchy audio.  Odd, the radio was known to be good, I’ve operated from the back deck before and not had problems, what changed?

The one and only thing different is I was using one of these 10Ah packs.  I’ve had fun with RF problems on the bicycle too.  On transmit, the battery was hovering around the 10.2V mark, perhaps a bit low.  Could it be the radio is distorting on voice peaks due to input current starvation?  I tried after the net swapping it for my 40Ah pack, which improved things.  Not totally cleared up, but it was better, and the pack hadn’t been charged in a while so it was probably a little low too.

The idea

I thought about the problem for a bit.  SSB requires full power on voice peaks.  For a 100W radio, that’s a 20A load right now.  Batteries don’t like this.  Perhaps there was a bit of internal resistance from age and the nature of the cells?  Could I do something to give it a little hand?

Supercapacitors are basically very high capacity electrolytic capacitors with a low breakdown voltage, normally in the order of a few volts and capacitances of over a farad.  They are good for temporarily storing charge that needs to be dumped into a load in a hurry.  Could this help?

My cells are in a series bank of 4: ~3.3V/cell with 4 cells gives me 13.2V.  There’s a battery balancer already present.  If a cell gets above 4V, that cell is toast, so the balancer is present to try to prevent that from happening.  I could buy these 1F 5.5V capacitors for only a few dollars each, so I thought, “what the hell, give it a try”.  I don’t have much information on them other that Elna Japan made them.  The plan was to make some capacitor “modules” that would hook in parallel to each cell.

My 13.2V battery pack, out of case

My 13.2V battery pack, out of its case

Supercapacitors

Supercapacitors

For my modules, the construction was simple, two reasonably heavy gauge wires tacked onto the terminals, the whole capacitor then encased in heatshrink tubing and ring lugs crimped to the leads. I was wondering whether I should solder a resistor and diode in parallel and put that in series with the supercap to prevent high in-rush current, but so far that hasn’t been necessary.

The re-assembled pack

I’ve put the pack back together and so far, it has charged up and is ready to face its first post-retrofit challenge.  I guess I’ll be trying out the HF station tomorrow to see how it goes.

Assembled pack

Assembled pack

The Verdict

Not a complete solution to the RF feedback, it seems to help in other ways. I did a quick test on the drive way first with the standard Yaesu handmic and with the headset. Headset still faces interference problems on HF, but I can wind it up to about 30W~40W now instead of 20.

More pondering to come but we’ll see what the other impacts are.

Mar 192015
 
Tropical Cyclone Nathan, Forecast map as of 2:50PM

Tropical Cyclone Nathan, Forecast map as of 2:50PM

This cyclone has harassed the far north once already, wobbled out in the Pacific like a drunken cyclist as a tropical low, has gained strength again and is now making a bee-line for Cape Flattery.

As seen, it also looks like doing the same stunt headed for Gove once it’s finished touching up far north Queensland.  Whoever up there is doing this rain dancing, you can stop now, it’s seriously pissing off the weather gods.

National and IARU REGION III Emergency Frequencies (Please keep clear and listen for emergency traffic)

  • 80m
    • 3.600MHz LSB (IARU III+WICEN)
  • 40m
    • 7.075MHz LSB (WICEN)
    • 7.110MHz LSB (IARU III)
  • 20m
    • 14.125MHz USB (WICEN)
    • 14.300MHz USB (IARU III)
    • 14.183MHz USB: NOT an emergency frequency, but Queensland State WICEN hold a net on this frequency every Sunday morning at around 08:00+10:00 (22:00Z Saturday).
  • 15m
    • 21.190MHz USB (WICEN)
    • 21.360MHz USB (IARU III)
  • 10m
    • 28.450MHz USB (WICEN)

I’ll be keeping an ear out on 14.125MHz in the mornings.

Update 20 March 4:31am: It has made landfall between Cape Melville and Cape Flattery as a category 4 cyclone.

Mar 022015
 

Well, it’s been a year and a half since I last posted details about the bicycle mobile station.  Shortly after getting the Talon on the road, setting it up with the top box and lighting, and having gotten the bugs worked out of that set-up, I decided to get a second mounting plate and set my daily commuter up the same way, doing away with the flimsy rear basket in place of a mounting plate for the top box.

VK4MSL/BM today after the trip home from work.

VK4MSL/BM today after the trip home from work.

That particular bike people might recognise from earlier posts, it’s my first real serious commuter bike. Now in her 5th year, has done over 10200km since 2012.  (The Talon has done 5643km in that time.) You can add to that probably another 5000km or so done between 2010 and 2012. It’s had a new rear wheel (a custom one, after having a spate of spoke breakages) and the drive chain has been upgraded to 9-speed. The latter upgrade alone gave it a new lease on life.

Both upgrades being money well spent, as was the upgrade to hydraulic brakes early in the bike’s lifetime. I suppose I’ve spent in those upgrades alone close to $1400, but worth it as it has given me quite good service so far.

As for my time with the top box. Some look at it and quiz me about how much weight it adds. The box itself isn’t that heavy, it just looks it. I can carry a fair bit of luggage, and at present, with my gear and tools in it it weighs 12kg. Heavy, but not too heavy for me to manage.

Initially when I first got it, it was great, but then as things flexed over time, I found I was intermittently getting problems with the top box coming off.  This cost me one HF antenna and today, the top box sports a number of battle-scars as a result.

The fix to this?  Pick a spot inside the top box that’s clear of the pannier rack rails and the rear tyre, and drill a 5mm hole through.  Then, when you mount the top box, insert an M5 bolt through the mounting plate and into the bottom of the top box and tighten with a 5mm wing nut.  The box now will not come loose.

vk4msl-bm-2713

Still lit up like a Christmas tree from this morning’s ride.

The lights still work, and now there’s a small rear-view camera.  On the TODO list is to rig up a 5V USB socket to power that camera with so that it isn’t draining the rather small internal battery (good for 4 hours apparently).

The station has had an upgrade, I bought a new LDG Z-100Plus automatic tuner which I’m yet to fully try out.  This replaces the aging Yaesu FC-700 manual tuner which, although very good, is fiddly to use in a mobile set-up and doesn’t tune 6m easily.

One on-going problem now not so much on the Boulder but on the Talon is an issue with pannier racks failing.  This started happening when I bought the pannier bags, essentially on the side I carry my battery pack (2kg), I repeatedly get the pannier rack failing.  The racks I use are made by Topeak and have a built-in spacer to accomodate disc brake calipers.  This seems to be a weak spot, I’ve now had two racks fail at about the same point.

Interestingly on the Boulder which also has disc brakes, I use the standard version of the same rack, and do not get the same failures.  Both are supposedly rated to 25kg, and my total load would be under 16kg. Something is amiss.

A recurring flaw with the Topeak racks

I’m on the look-out for a more rugged rack that will tolerate this kind of usage, although having seen this, I am inspired to try a DIY solution.  Then if a part fails, I can probably get replacement parts in any typical hardware store.  A hack saw and small drill are not heavy items to carry, and I can therefore get myself out of trouble if problems arise.