Feb 282015

Well, it’s been about 7½ years since I bought my first bike and started riding, and really, about 5 years since I started riding seriously as a means of transportation.

In late 2011 my father and I went halves in a pair of GPS/CB radio units, it was a 2 for 1 deal and so we bought these two units at about $400 each, normally they’d be about $700 individually. So there I started logging the distance I covered. I just used the in-built odometer on the GPS, resetting it when the bike went in for service.

When I got the mountain bike, I realised I needed to track the distance covered by each bike to ensure they all went in at their 1000km service on-time. So being a programmer by trade, I coded up a crude CGI/Perl script that used a SQLite back-end to log the odometer readings. It was a simple HTML form where I could enter the distance at regular intervals.  Crucially, it worked with the “feature phone” I used at the time.

The SQL views (no such thing as stored procedures in standard SQLite3) took care of actually calculating the differentials and so I used that to track my progress. So far so good. I’ve now had this in place since mid-2012 and I’ve brought in some of my data from early 2012, thus I’m now starting to see some trends.

Distances by year

Year Distance (km)
2012 5594.9
2013 4837.78
2014 4593.42

Am I getting lazy? Well, hard to say there. I go out less on the weekends and have also optimised my routes to reduce distances somewhat.  Some of this is weather-dependent, in the heat one does not feel like going outdoors.

Distance by month-of-year

Month Distance (km)
01 282.59
02 406.20
03 409.10
04 377.42
05 511.29
06 493.36
07 330.01
08 532.05
09 494.21
10 470.14
11 370.27
12 394.13

I’m not sure why there’s a lull in activity around July, but the most active months seem to be May and August.  The lull in January can be somewhat attributed to the end of the Christmas break.  I guess if anything, I should aim to be more active in July when the weather is the coolest.

Guess I’ll be keeping an eye on what happens over time with these stats and see if I can get them up a bit.

The following graph will continuously update as I pump data in. We’ll see what happens.

Distance by month-of-year

Feb 202015


As an update on this…

Due to some issues (browser pop up behavior for example), with the Superfish Visual Discovery browser add-on, we have temporarily removed Superfish from our consumer systems until such time as Superfish is able to provide a software build that addresses these issues. As for units already in market, we have requested that Superfish auto-update a fix that addresses these issues.

To be clear, Superfish comes with Lenovo consumer products only and is a technology that helps users find and discover products visually. The technology instantly analyzes images on the web and presents identical and similar product offers that may have lower prices, helping users search for images without knowing exactly what an item is called or how to describe it in a typical text-based search engine.

The Superfish Visual Discovery engine analyzes an image 100% algorithmically, providing similar and near identical images in real time without the need for text tags or human intervention. When a user is interested in a product, Superfish will search instantly among more than 70,000 stores to find similar items and compare prices so the user can make the best decision on product and price.

Superfish technology is purely based on contextual/image and not behavioral. It does not profile nor monitor user behavior. It does not record user information. It does not know who the user is. Users are not tracked nor re-targeted. Every session is independent. When using Superfish for the first time, the user is presented the Terms of User and Privacy Policy, and has option not to accept these terms, i.e., Superfish is then disabled.

Mark Hopkins, Lenovo Support

That’s alright Mark, I’ve permanently removed Lenovo from my list of future suppliers. If I buy a Lenovo product, I’m going to insist the machine is delivered to me completely formatted of hardware and supplied with media to do a clean installation since it is clear you cannot be trusted to put an OS on a computer and not botch it in some manner.

I think there should be a law against this sort of bundling: too long machines have been delivered with crippling bloatware that either wastes system resources, causes security headaches or both. Sure, bundle some software, BUT ASK THE CUSTOMER BEFORE YOU INSTALL IT!

Feb 012015

How do software companies get things so wrong?  I aim this at both Google and Apple here, as both are equally as guilty of this.  Maybe Microsoft can learn from this?

So you see something on an “app store” that might be useful on the device you’re using.  Ohh, a free download, great!  Let’s download.  You click the link, and immediately get asked for a login and password.  There’s no option to proceed without.  They insist you create an account with them even if it’s the one and only thing you’re interested from them.

In the past my gripe has been with the Google Play store.  Even the “free” apps, require you to log in.  Ohh, and to add insult to injury, the Google Play store doesn’t just expect any Google account, it has to be one of their Gmail accounts.  Back in the late 90s I had an email address with most providers as the average quota was about 5MB.  I’ve had a mailbox of my own with a multi-gigabyte (actually limited only by disk capacity) “quota” since 2002, I have no use for Gmail, and only keep my old Yahoo! address (created in 1999) for historic reasons.

I have an Android phone (release 4.1: thanks to ZTE’s backward thinking and short attention span), and thankfully there’s the F-Droid store which has sufficient applications for my use.  So I can work around the Google Play store most of the time and so far, haven’t needed anything from there.

Today, my gripe is at Apple, and the “app” in question is MacOS X, which cannot be obtained anywhere else.

With all the high-profile attacks on websites that store user accounts, one has to ask, why?  It’s one extra username and password, which given the frequency I’m likely to use it, will have to be written down and stored somewhere secure as it won’t get sufficient use to commit it to memory.  Before people point out password managers, I’d like to point out one thing: it’s still writing it down!

There’s absolutely no need for an “app store” to know your email address, usernames, passwords, or any details.  If you are actually purchasing an application, they only need enough information to process a payment.

Usually this is by a debit/credit card, so they need to know the details on the card.  An alternative might be direct deposit through a bank, at which point they need to supply you with details on how to make the payment — details that include the information they need to match your payment in their ledger to your store purchase.  At no point do they need anything else.

For convenience an email address might be supplied so they can confirm your order or contact you if there’s a problem, however for debit/credit cards, this happens so quickly that it can be achieved via the web browser.

Despite this, they insist on you providing just about everything.

I’m no stranger to the “app store” concept.  Linux and BSD distributions have had this sort of concept for years.  BSD has had ports for as long as I can remember.  Debian had apt since 1998, Gentoo has had portage since its inception in 2003 and RPM-based distributions have had yum for some time too.

None of these actually need to know who you are in order to download a package.  Admittedly none of these are geared toward commercial sales of software, and so lack the ability to prompt for credentials or payment information.

Since both Google Play and the Apple App store have solved the former problem, I see no reason why they couldn’t solve the latter.  I don’t want to post anything to the site, I don’t want to leave feedback as I can hardly comment on something I haven’t received yet, and I don’t know when I’ll next visit the site.

If I was going to be back repeatedly, sure, I’ll make an account.  It’ll make everyone’s lives easier. (Including the blackhats!)  But I’m not.  I have a late-2008 model MacBook, probably the oldest machine that Apple support for their latest OS.  The machine dual-boots MacOS X 10.6 and Gentoo Linux, and spends 99% of its time in the latter OS.

Given the age of the machine and the frequency at which I use its native OS, it is not worth me spending a lot of time or expense updating it.  A 2GHz Core 2 Duo with 8GB RAM and a 750GB HDD is good enough for many tasks under Linux, but is the bare minimum to run OS X 10.10.  The only reason this machine doesn’t grace my desk at work anymore is the fact the lack of ports (USB in particular) proved to be a right pain.

Why update?  Well, applications these days seem to expect at least MacOS X 10.7 now.  I either have to build everything myself or update the OS, so I’m investigating the possibility of updating the OS to see if it’s feasible.  Apparently it’s a free download, so why not?

Well, why not indeed!  Instead of having a simple http, https or ftp link to the file in question (maybe a .dmg image) for software they’re not actually selling to me in the traditional sense, they instead insist on making me jump through hoops like requiring their “app store” client — so I can’t just grab the link, tell the web server here to download the file then grab it from there when I’m ready.

Since I can’t do the download any other way than via their “app store” client, I have to remain booted in MacOS X in order to download it regardless of what I might otherwise wish to do the machine and what OS that requires.

However, before I can even think about starting the download, I’ve got to register an account, supplying a username and password for something that will probably be used exactly once.  Details that they have to pay people big money to store securely.

Instead of spending some money paying someone to add an extra one-off button and form to their “app store” clients, they instead spend significantly more on infrastructure designed to meet the privacy requirements of various laws to store user information that simply is not necessary for the transaction to proceed.

In light of the sophistication of the modern cracker and the cut-throat nature of the mobile market, is this such a wise use of company funds?

Jan 032015

I’ve been riding on the road now for some years, and while I normally try to avoid it, I do sometimes find myself riding on the road itself rather than on the footpath or bicycle path.

Most of the time, the traffic is fine.  I’m mindful of where everyone is, and there aren’t any problems, but I have had a couple of close calls from time to time.  Close calls that have me saying “ode for a horn”.

By law we’re required to have a bell on our bikes.  No problem there, I have a mechanical one which is there purely for legal purposes.  If I get pulled over by police, and they ask, I can point it out and demonstrate it.  Requirement met?  Tick to that.

It’s of minimal use with pedestrians, and utterly useless in traffic.

Early on with my riding I developed a lighting system which included indicators.  Initially this was silent, I figured I’d see the lights flashing, but after a few occasions forgetting to turn indicators off, I fitted a piezo buzzer.  This was an idea inspired by the motorcycles ridden by Australia Post contractors, which have a very audible buzzer.  Jaycar sell a 85dB buzzer that’s waterproof, overkill in the audio department but fit for purpose.  It lets me know I have indicators on and alerts people to my presence.

That is, if they equate the loud beep to a bicycle.  Some do not.  And of course, it’s still utterly useless on the road.

I figured a louder alert system was in order.  Something that I could adjust the volume on, but loud enough to give a pedestrian a good 30 seconds warning.  That way they’ve got plenty of time to take evasive action while I also start reducing speed.  It’s not that I’m impatient, I’ll happily give way, but I don’t want to surprise people either.  Drivers on the other hand, if they do something stupid it’d be nice to let them know you’re there!

My workplace looks after a number of defence bases in South-East Queensland, one of which has a railway crossing for driver training.  This particular boom gate assembly copped a whack from a lightning strike, which damaged several items of equipment, including the electronic “bells” on the boom gate itself.  These “bells” consisted of a horn speaker with a small potted PCB mounted to the back which contained an amplifier and bell sound generator.  Apply +12V and the units would make a very loud dinging noise.  That’s in theory; in practise, all that happened was a TO-220 transistor got hot.  Either the board or the speaker (or both) was faulty.

It was decided these were a write-off, and after disassembly I soon discovered why: the voice coils in the horn speakers had been burnt out.  A little investigation, and I figured I could replace the blown out compression drivers and get the speakers themselves working again, building my own horn.

A concept formed: the horn would have two modes, a “bell” mode with a sound similar to a bicycle bell, and a “horn” mode for use in traffic.  I’d build the circuit in parts, the first being the power amplifier then interface to it the sound effect generator.

To make life easier testing, I also decided to add a line-in/microphone-in feature which would serve to debug construction issues in the power amplifier and add a megaphone function.  (Who knows, might be handy with WICEN events.)

Replacing the compression drivers

Obviously it’d be ideal to replace it with the correct part, but looking around, I couldn’t see anything that would fit the housing.  That, and what I did see, was more expensive than buying a whole new horn speaker.

There was a small aperture in the back about 40mm in diameter.  The original drivers were 8ohms, and probably rated at 30W and had a convex diaphragm which matched the concave geometry in the back of the horn assembly.

Looking around, I saw these 2W mylar cone speakers.  Not as good as a compression driver, but maybe good enough?  It was cheap enough to experiment.  I bought two to try it out.

I got them home, tacked some wires onto one of them and plugged it into a radio.  On its own, not very loud, but when I held it against the back of a horn assembly, the amplification was quite apparent.  Good enough to go further.  I did some experiments with how to mount the speakers to the assembly, which required some modifications to be made.

I soon settled on mounting the assembly to an aluminium case with some tapped holes for clamping the speaker in place.  There was ample room for a small amplifier which would be housed inside the new case, which would also serve as a means of mounting the whole lot to the bike.

Bell generator

I wasn’t sure what to use for this, I had two options: build an analogue circuit to make the effect, or program a microcontroller.  I did some experiments with an ATMega8L, did manage to get some sound out of it using the PWM output, but 8kB of flash just wasn’t enough for decent audio.

A Freetronics LeoStick proved to be the ticket.  32kB flash, USB device support, small form factor, what’s not to like?  I ignored the Arduino-compatible aspect and programmed the device directly.  Behind the novice-friendly pin names, they’re an ATMega32U4 with a 16MHz crystal.  I knocked up a quick prototype that just played a sound repeatedly.  It sounded a bit like a crowbar being dropped, but who cares, it was sufficient.

Experimenting with low-pass filters I soon discovered that a buffer-amp would be needed, as any significant load on the filter would render it useless.

A 2W power amplifier

Initially I was thinking along the lines of a LM386, but after reading the datasheet I soon learned that this would not cut it.  They are okay for 500mW, but not 2W.  I didn’t have any transistors on hand that would do it and still fit in the case, then I stumbled on the TDA 1905.  These ICs are actually capable of 5W into 4 ohms if you feed them with a 14V supply.  With 9V they produce 2.5W, which is about what I’m after.

I bought a couple then set to work with the breadboard.  A little tinkering and I soon had one of the horn speakers working with this new amplifier.  Plugged into my laptop, I found the audio output to be quite acceptable, in fact turned up half-way, it was uncomfortable to sit in front of.

I re-built the circuit to try and make use of the muting feature.  For whatever reason, I couldn’t get this to work, but the alternate circuit provided a volume control which was useful in itself.

The pre-amplifier

For the line-level audio, there’s no need for anything more fancy than a couple of resistors to act as a passive summation of the left and right channels, however for a microphone and for the LeoStick, I’d need a preamp.  I grabbed a LM358 and plugged that into my breadboard alongside the TDA1905.

Before long, I had a working microphone preamp working using one half of the LM358, based on a circuit I found.  I experimented with some resistor values and found I got reasonable amplification if I upped some of the resistor values to dial the gain back a little.  Otherwise I got feedback.

For the LeoStick, it already puts out 5V TTL, so a unity-gain voltage follower was all that was needed.  The second half of the LM358 provided this.  A passive summation network consisting of two resistors and DC-blocking capacitor allowed me to combine these outputs for the TDA1905.

One thing I found necessary, the TDA1905 and LM358 misbehave badly unless there’s a decent size capacitor on the 9V rail.  I found a 330uF electrolytic helped in addition to the datasheet-prescribed 100nF ceramics.

Power supply

Since I’m running on batteries with no means of generating power, it’s important that the circuit does not draw power when idle.  Ideally, the circuit should power on when either I:

  • plug the USB cable in (for firmware update/USB audio)
  • toggle the external source switch
  • press the bell button

We also need two power rails: a 9V one for the analogue electronics, and a 5V one for the LeoStick.  A LM7809 and LM7805 proved to be the easiest way to achieve this.

To allow software control of the power, a IRF9540N MOSFET was connected to the 12V input and supplies the LM7809.  The gate pin is connected to a wired-OR bus.  The bell button and external source switch connect to this bus with signal diodes that pull down on the gate.

Two BC547s also have collectors wired up to this bus, one driven from the USB +5V supply, and the other from a pin on the LeoStick.  Pressing the Bell button would power the entire circuit up, at which point the LeoStick would assert its power on signal (turning on one of the BC547s) then sample the state of the bell button and start playing sound.  When it detects the button has been released, it finishes its playback and turns itself off by releasing the power on signal.

Sound effect generator

Earlier I had prototyped a bell generator, however it wasn’t much use as it just repeatedly made a bell noise regardless of the inputs.  To add insult to injury, I had lost the source code I used.  I had a closer look at the MCU datasheet, deciding to start from a clean slate.

The LeoStick provides its audio on pin D11, which is wired up to Port B Pin 7.  Within the chip, two possible timers hook up: Timer 0, which is an 8-bit timer, and Timer 1, which is 16-bits.  Both are fed from the 16MHz system clock.  The bit depth affects the PWM carrier frequency we can generate, the higher the resolution, the slower the PWM runs.  You want the PWM frequency as high as possible, ideally well above 20kHz so that it’s not audible in the audio output, and obviously well above the audio sampling rate.

At 16MHz, a 16-bit timer would barely exceed 240Hz, which is utterly useless for audio.  A 10-bit timer fares better, with 15kHz, older people may not hear it but I certainly can hear 15kHz.  That leaves us with 8-bits which gets us up to 62kHz.  So no point in using Timer 1 if we’re only going to be using 8-bits of it, we might as well use Timer 0.

Some of you familiar with this chip may know of Timer 4, which is a high-speed 10-bit timer fed by a separate 64MHz PLL.  It’s possible to do better quality audio from here, either running at 10-bits with a 62kHz carrier, or dropping to 8-bits and ramping the frequency to 250kHz.  Obviously it’d have been nice, but I had already wired things up by this stage, so it was too late to choose another pin.

Producing the output voltage is only half the equation though: once started, the PWM pin will just output a steady stream of pulses, which when low-passed, produces a DC offset.  In order to play sound, we need to continually update the timer’s Capture Compare register with each new sample at a steady rate.

The most accurate way to do this, is to use another timer.  Timer 3 is another 16-bit timer unit, with just one capture compare output available on Port C pin 3.  It is an ideal candidate for a timer that has no external influence, so it gets the job of updating the PWM capture compare value with new samples.

Timer 1 is connected to pins that drive two of the three LEDs on the LeoStick, with Timer 4 driving the remaining one, so if I wanted, I could have LEDs fade in and out with it instead of just blinking.  However, my needs are basic, and I need something to debounce switches and visibly blink LEDs.  So I use that with a nice long period to give me a 10Hz timer.

Here is the source code.  I’ll add schematics and other notes to it with time, but the particular bits of interest for those wanting to incorporate PWM-generated sound in their AVR projects are the interrupt routine and the sound control functions.

To permit gapless playback, I define two buffers which I alternate between, so while one is being played back, the other can be filled up with samples.  I define these on line 139 with the functions starting at line 190.  The interrupt routine that orchestrates the playback is at line 469.

When sound is to be played, the first thing that needs to happen is for the initial buffer to be loaded with samples using the write_audio function.  This can either read from a separate buffer in RAM (e.g. from USB) or from program memory.  One of the options permits looping of audio.  Having loaded some initial data in, we can then call start_audio to set up the two timers and get audio playback rolling.  start_audio needs the sample rate to configure the sample rate timer, and can accept any sample rate that is a factor of 16MHz (so 8kHz, 16kHz up to 32kHz).

The audio in this application is statically compiled in, taking the form of an array of uint8_t‘s in PROGMEM.

Creating the sounds

I initially had a look around to see if I could get a suitable sound effect.  This proved futile, I was ideally looking around for a simple openly-licensed audio file.  Lots of places offered something, but then wanted you to sign up or pay money.  Fine, I can understand the need to make a quid, and if I were doing this a lot, I’d pay up, but this is a once-off.

Eventually, I found some recordings which were sort of what I was after, but not quite.  So I downloaded these then fired up Audacity to have a closer look.

The bicycle bell

Bicycle bells have a very distinctive sound to them, and are surprisingly complicated.  I initially tried to model it as an exponentially decaying sinusoid of different frequencies, but nothing sounded quite right.

The recording I had told me that the fundamental frequency was just over 2kHz.  Moreover though, the envelope was amplitude-modulated by a second sinusoid: this one about 15Hz.  Soon as I plugged this second term in, things sounded better.  This script, was the end result.  The resulting bell sounds like this:

So somewhat bell-like.  To reduce the space, I use a sample rate of 6.4kHz.  I did try a 4kHz sample rate but was momentarily miffed at the result until I realised what was going on: the bell was above the Nyquist frequency at 4kHz, 6.4kHz is the minimum practical rate that reproduces the audio.

I used Audacity to pick a point in the waveform for looping purposes, to make it sound like a bell being repeatedly struck.

The horn

I wanted something that sounded a little gutsy.  Like an air-horn on a truck.  Once again, I hit the web, and found a recording of a train horn.  Close enough, but not long enough, and a bit noisy.  However opening it up in Audacity and doing a spectrum analysis, I saw there were about 5 tones involved.  I plugged these straight into a Python script and decided to generate those directly.  Using a raised cosine filter to shape the envelope at the start and end, and I soon had my horn effect.  This script generates the horn.  The audio sounds like this:

Using other sound files

If you really wanted, you could use your own sound recordings.  Just keep in mind the constraints of the ATMega32U4, namely, 32kB of flash has to hold both code and recordings.  An ATMega64 would do better.  The audio should be mono, 8-bits and unsigned with as lower sample rate as you can get away with.  (6.4kHz proved to be sufficient for my needs.)

Your easiest bet would be to either figure out how to read WAV files (in Python: wave module), or alternatively, convert to raw headerless audio files, then code up a script that reads the file one byte at a time. The Python scripts I’ve provided might be a useful starting point for generating the C files.

Alternatively, you can try interfacing a SDCard and embedding a filesystem driver and audio file parser (not sure about WAVE but Sun Audio is easily parsed), this is left as an exercise for the adventurous.

Finishing up

I’ll put schematics and pictures up soonish.  I’m yet to try mounting the whole set up, but so far the amplifier is performing fine on the bench.

Dec 102014

This afternoon we started getting some unusual calls.  Now I hate those survey calls, or telemarketing calls wanting to sell you some kind of service.  I especially hate them when they’re delivered by a recorded voice, and there’s a special place in HELL for those which claim to have found “problems with your computer”.

My troubles started earlier this afternoon.  Having gotten home from work around 3PM, I make a call to my father to find out what was happening tonight, got no answer, and so I just hung up rather than leaving a message (it wasn’t important).  He rang back and we had a quick discussion.

Some time later, the phone rings.  Now, normally when the phone rings, it’s two bursts, then silence, then two more bursts, then silence … etc.  This had a different initial rhythm: one long burst, then silence, then the usual pattern.  I answered, only to be greeted by silence, then an automated voice.  I hung up straight away.

Normally that’d be the end of it.  Then history repeats itself, after 5 minutes the phone ring again.  Same pattern.  I answer, and get the same silence, followed by a voice recording.  I hang up, again.

Cue this happening about 3 or 4 times.  So I look up the Telstra website and found their help-desk number.  I also paid a visit to the Do Not Call register for good measure.  (We had done it before, but maybe it had expired?).  A computer system answers (typical), and after answering a few prompts, I’m told there will be a 7 minute wait.

Well, 7 minutes turned out to be 25 minutes, but who’s counting?  I guess Steven Travalgia is right about the “variable viscosity of time” theory, it certainly applies to help-desk queues!  That said, at least I wasn’t getting nuisance calls.

I explain the situation to the operator.  Naturally, not being the account holder, they cannot do much, but at least there’s a record of me calling, they mention they can enable tracing to find out what’s going on.  They give me a direct line for their unwanted calls department, and I reply stating I’ll take some logs of what happens and call that number when I have some evidence.

17:04 4 rings, dial tone on pick up
17:05 3 rings, stopped ringing before answer
17:12 2 rings, dial tone on pick up
17:52 Answered and recorded.

I recorded this (apologies for the clipping, my mic gain was up a bit high):

Now it’s worth noting that nothing currently plugged into the phone line can receive SMS messages.  Our phone line terminates in our garage at a ADSL2+ central splitter (installed by yours truly).

One CAT5e cable is divided into one ADSL circuit and 3 voice circuits and runs into the office, providing service for the ADSL router/modem, a multi-function fax/printer/scanner, a General Electric speaker phone (with corroding AA batteries, so maybe that phone will go in the bin now), a (Telstra-branded) cordless phone base station and a 56k modem.

The other feed coming out of the splitter box is original house wiring, and terminates upstairs with an old Telecom Australia Touchfone 200 that probably remembers the days of our house having a 6-digit number.  (Our line is that old.)

Nothing that will receive messages, or confuse the hell out of the delivery centre.  It seems if there’s nothing on the line, they just keep ringing persistently, making the service a very cheap and efficient way to harass someone at all hours of the night!

Sadly, a quick search does not tell one how to disable this service.  I have no reason to receive SMS messages on a land-line, I have a mobile for that.  If I find out how, I’ll be updating this.

Dec 042014

Just recently I’ve been looking into asynchronous programming.

Previously I had an aversion to asynchronous code due to the ugly twisted web of callback functions that it can turn into. However, after finding that having a large number of threads blocking on locks and semaphores still manages to thrash a machine, I’ve come to the conclusion that I should put aside my feelings and try it anyway.

Our codebase is written in Python 2.7, sadly, not new enough to have asyncio. However we do plan to eventually move to Python 3.x when things are a bit more stable in the Debian/Ubuntu department (Ubuntu 12.04 didn’t support it and there are a few sites that still run it, one or two still run 10.04).

That said, there’s thankfully a port of what became asyncio in the form of Trollius.

Reading through the examples though still had me lost and the documentation is not exactly extensive. In particular, coroutines and yielding. The yield operator is not new, it’s been in Python for some time, but until now I never really understood it or how it was useful in co-operative programming.

Thankfully, Sahand Saba has written a guide on how this all works:

I might put some more notes up as I learn more, but that guide explained a lot of the fundamentals behind a lot of event loop frameworks including asyncio.

Nov 052014

Just because I effectively turned down offers to work for you doesn’t mean I’m okay with your customers having a crack at my server:


The IP has just been banned by Fail2Ban after
3 attempts against SSH.

Here is more information about

# ARIN WHOIS data and services are subject to the Terms of Use
# available at: https://www.arin.net/whois_tou.html
# If you see inaccuracies in the results, please report at
# http://www.arin.net/public/whoisinaccuracy/index.xhtml

# The following results may also be obtained via:
# http://whois.arin.net/rest/nets;q=

NetRange: -
NetName:        GOOGLE-CLOUD
NetHandle:      NET-107-167-160-0-1
Parent:         NET107 (NET-107-0-0-0-0)
NetType:        Direct Allocation
OriginAS:       AS15169
Organization:   Google Inc. (GOOGL-2)
RegDate:        2014-01-24
Updated:        2014-01-24
Comment:        *** The IP addresses under this netblock are in use by Google Cloud customers *** 
Comment:        Please direct all abuse and legal complaints regarding these addresses to the 
Comment:        GC Abuse desk (google-cloud-compliance@google.com). Complaints sent to 
Comment:        any other POC will be ignored.
Ref:            http://whois.arin.net/rest/net/NET-107-167-160-0-1

OrgName:        Google Inc.
OrgId:          GOOGL-2
Address:        1600 Amphitheatre Parkway
City:           Mountain View
StateProv:      CA
PostalCode:     94043
Country:        US
RegDate:        2006-09-29
Updated:        2013-10-18
Comment:        *** The IP addresses under this Org-ID are in use by Google Cloud customers ***
Comment:        Please direct all abuse and legal complaints regarding these addresses to the
Comment:        GC Abuse desk (google-cloud-compliance@google.com).  Complaints sent to 
Comment:        any other POC will be ignored.
Ref:            http://whois.arin.net/rest/org/GOOGL-2

OrgAbuseHandle: GCABU-ARIN
OrgAbuseName:   GC Abuse
OrgAbusePhone:  +1-650-253-0000 
OrgAbuseEmail:  google-cloud-compliance@google.com
OrgAbuseRef:    http://whois.arin.net/rest/poc/GCABU-ARIN

OrgTechHandle: ZG39-ARIN
OrgTechName:   Google Inc
OrgTechPhone:  +1-650-253-0000 
OrgTechEmail:  arin-contact@google.com
OrgTechRef:    http://whois.arin.net/rest/poc/ZG39-ARIN

OrgNOCName:   GC Abuse
OrgNOCPhone:  +1-650-253-0000 
OrgNOCEmail:  google-cloud-compliance@google.com
OrgNOCRef:    http://whois.arin.net/rest/poc/GCABU-ARIN

# ARIN WHOIS data and services are subject to the Terms of Use
# available at: https://www.arin.net/whois_tou.html
# If you see inaccuracies in the results, please report at
# http://www.arin.net/public/whoisinaccuracy/index.xhtml



Geez, you’re getting as bad as another mob I could mention, although in your favour, you at least make it clear from the WHOIS data that it’s a guest on your network that’s stirring up trouble.

Oct 262014

Earlier this week I had an idea.  We’ve got an old clock radio that picks up interference from the fridge when it turns on and the buttons on it are starting to fail with age.

I thought: “Why not build a new one?”

So the requirements are simple.  We need a real-time clock, display driver, and of course, a receiver.  The unit we have spends most of its time tuned to 792kHz AM (4QG or “ABC Radio National”), so a simple direct conversion receiver was what I was thinking of.  But what about the LO?

Now I do have some clock radio ICs that implement the timing circuitry, alarm function and LED panel driver somewhere in a junk box.  You feed them with the 50Hz or 60Hz waveform that comes out of the transformer and they use that as the timing source.  Easy to use a 555 timer for the time source, and I’d make a traditional receiver.  Another option is to use a AVR microcontroller, I have a few ATMega8Ls in the junk box with a NXP I2C RTC chip which I also have a few of.

The ATMega8L has a couple of PWM channels one 16-bit and one 8-bit: could they be used as an LO?

So: after digging around and locating my bought-years-ago and not-yet-used AVR programmer, and dusting off a breadboard that had an ATMega8L on it from a previous experiment I set to work.

This page explains in good detail how the PWM channels work. I started with those examples as a guide and tweaked from there.

For the PWM channel to work as a receiver LO, I want it to cover 540kHz to ~2MHz, with reasonable granularity. Question is, how far can I crank this? I have a 4MHz crystal, not the fastest I can use with this chip, but the absolute top of the range for the ATMegas isn’t much higher: 16MHz or maybe 20MHz. So if you’ve got a 16MHz crystal, you can expect to quadruple what I do here.

I started off with some blink code. If you take out all the delays, you get the following code:

#include <avr/io.h>
int main(void)
        DDRB |= (1 << DDB1);
        while (1)
                PORTB ^= (1 << DDB1);

and the following waveform:

Waveform done in software with GPIOs

Waveform done in software with GPIOs

The yellow waveform there is off one of the crystal pins. The cyan one is the PWM pin output, which in this case is a software driven GPIO. Even if this one worked, you wouldn’t want to do it this way unless your chip was doing only this task, and who’d use a programmable chip like an ATMega8L for that?

So, after reading through the documentation and examples, I loaded in the following code:

#include <avr/io.h>

#define TCCR1_COM1A     1
#define TCCR1_COM1B     0
#define TCCR1_FOC1A     0
#define TCCR1_FOC1B     0
#define TCCR1_WGM1      0xf
#define TCCR1_ICNC1     0
#define TCCR1_ICES1     0
#define TCCR1_CS1       1
#define TCCR1A_VAL      (                       \
                        (TCCR1_COM1A    << 6)   \
                |       (TCCR1_COM1B    << 4)   \
                |       (TCCR1_FOC1A    << 3)   \
                |       (TCCR1_FOC1B    << 2)   \
                |       (TCCR1_WGM1 & 0x3)      )
#define TCCR1B_VAL      (                       \
                        (TCCR1_ICNC1    << 7)   \
                |       (TCCR1_ICES1    << 6)   \
                |       (((TCCR1_WGM1 & 0xc) >> 2) << 3) \
                |       TCCR1_CS1)

int main (void)
        DDRB |= (1 << DDB1);
        OCR1A = 0x001;
        TCCR1A = TCCR1A_VAL;
        TCCR1B = TCCR1B_VAL;

The frequency can be adjusted by playing with OCR1A. If I leave it at 1 (basically as fast as the PWM can go) I get the following:

Waveform from AVR PWM

Waveform from AVR PWM

Bump it up one, and it sinks to 600kHz. Way too coarse for what I want sadly. I guess I was hopeful, but maybe the above might serve as a useful spring-off point for experiments with PWM.

Oct 042014

This was sent to me by email.  While I don’t normally air political issues here, I think the original author of this, whoever that was, makes some very valid points.

The politicians themselves, in Canberra, brought it up, that the Age of Entitlements is over:

The author is asking each addressee to forward this email to a minimum of twenty people on their address list; in turn ask each of those to do likewise. At least 20 if you can. In three days, most people in Australia will have this message.

This is one idea that really should be passed around because the rot has to stop somewhere.

Proposals to make politicians shoulder their share of the weight now that the Age of Entitlement is over

1. Scrap political pensions.

Politicians can purchase their own retirement plan, just as most other working Australians are expected to do.

2. Retired politicians (past, present & future) participate in Centrelink.

A Politician collects a substantial salary while in office but should receive no salary when they’re out of office.

Terminated politicians under 70 can go get a job or apply for Centrelink unemployment benefits like ordinary Australians.

Terminated politicians under 70 can negotiate with Centrelink like the rest of the Australian people.

3. Funds already allocated to the Politicians’ retirement fund be returned immediately to Consolidated Revenue.

This money is to be used to pay down debt they created which they expect us and our grandchildren to repay for them.

4. Politicians will no longer vote themselves a pay raise. Politicians pay will rise by the lower of, either the CPI or 3%.

5. Politicians lose their privileged health care system and participate in the same health care system as ordinary Australian people.

i.e. Politicians either pay for private cover from their own funds or accept ordinary Medicare.

6. Politicians must equally abide by all laws they impose on the Australian people.

7. All contracts with past and present Politicians men/women are void effective 31/12/14.

The Australian people did not agree to provide perks to Politicians, that burden was thrust upon them.

Politicians devised all these contracts to benefit themselves.

Serving in Parliament is an honour not a career.

The Founding Fathers envisioned citizen legislators, so our politicians should serve their term(s), then go home and back to work.

If each person contacts a minimum of twenty people, then it will only take three or so days for most Australians to receive the message. Don’t you think it’s time?

THIS IS HOW YOU FIX Parliament and help bring fairness back into this country!

If you agree with the above, pass it on.

Sep 292014

Well, it’s been a busy year so far for security vulnerabilities in open-source projects.  Not that those have been the only two bugs, they’re just two high-profile ones that are getting a lot of media attention.

Now, a number of us do take sheer delight in pointing and laughing when one of the big boys, whether they be based in Redmond or California, makes a security balls-up on a big scale.  After all, people pay big dollars to use some of that software, and many are dependent on it for their livelihoods.

The question does get raised though, what do you trust more?  A piece of software whose code is a complete secret, or the a piece of software anyone can audit?  Some argue the former, because anyone can find the holes in the latter and exploit them.  Some argue the latter, since anyone can find the holes and fix them.  Not being able to see the code doesn’t guarantee a lack of security issues however, and these last two headline-making bugs is definitely evidence that having the code isn’t a guarantee to a bug-free utopia.

There is no guarantee either way.

I’ve seen both open-source systems and high-end commercial systems both perform well and I’ve seen both make a dismal failure.  Bad code is bad code, no matter what the license, and even having the source available doesn’t mean you can fix it as first one must be able to understand what its intent is.  Information Technology in particular seems to attract the technologically inept but socially capable types that are able to talk their way into nearly any position, and so you wind up with the monstrosities that you might see on The Daily WTF.  These same people lurk amongst open-source circles too, and there are those who just make an honest mistake.  Security is hard, and it can be easy to overlook a possible hole.

I run Gentoo here, have done so now since 2004 (damn, 10 years already, but I digress…).  I’ve been building my own stage 3 tarballs from scratch since 2010.  July 2010 I bought my current desktop, a 6-core AMD Phenom machine, and so combined with the 512Kbps ADSL I had at the time, it was faster for me to compile stage 3 tarballs for the various systems (i386, AMD64 and about 6 different MIPS builds) than to download the sources.  If I wanted an up-to-date stage 3, I just took my last build, ran it through Gentoo Catalyst, and out came a freshly built tarball.

I still obtain my operating systems that way.  Even though I’ve upgraded the ADSL, I still use the same scripts that used to produce the official Gentoo/MIPS media.

This means I could audit every piece of software that forms my core system.  I have the source code there, all of it.  Not many Linux users have this, most have it at arms reach (i.e. an apt-get source ${PACKAGE} away), or at worst, a polite email/letter to their supplier (e.g. Netcomm will supply sources for their routers for a ~AU$10 fee), however I already have it.

So did I do any audits?  Have I done any audits?  No.  Ultimately I just blindly trust what comes down the wire, and to some, that is arguably no better than just blindly trusting what Apple and Microsoft produce.

Those who say that, do have a point.  I didn’t pick up on HeartBleed, nor on ShellShock, and I probably haven’t spotted what will become the next headline-grabbing bug.  There’s a lot of source code that goes into a GNU/Linux system, and if I were to sit there and audit it, myself, it’d take me a lifetime.  It’d cost me a fortune to pay a team to analyse it.

However, I at least have the choice of auditing parts of it.  I’ll never be able to audit the copies of Microsoft Windows, or the one copy of Apple MacOS X I have.  For those, I’m reliant on the upstream vendors to audit, test and patch their code, I cannot do it myself.

For the open-source software though, it’s ultimately my choice.  I can do it myself, I can also pay someone to do it, I’ve simply chosen not to at this time.  This is an important distinction that the anti-open-source camp seem to forget.

As for the quality factor: well I’ve spent more time arguing with some piece of proprietary software and having trouble getting it to do something I need it to do, or fixing up some cock up caused by a bug in the said software.  One option, I spend hours arguing with it to make it work, and have to pay good money for the privilege.  The other, they money stays in my pocket, and in theory I can re-build it to make it work if needed.  One will place arbitrary restrictions on how I use the software as an end user, forcing me to spend money on more expensive licenses, the other will happily let me keep pushing it until I hit my system’s technical limits.

Neither offer me any kind of warranty regarding to losses I might suffer as a result of their software (I’m sorry, but US$5.00 is as good as worthless), so the money might as well stay in my pocket while I learn something about the software I use.

I remain in control of my destiny that way, and that is the way I’d like to keep it.