Apr 102015
 

This afternoon, whilst waiting for a build job to complete I thought I’d do some further analysis on my annual mileage.

Now I don’t record my odometer readings daily (perhaps I should), but I do capture them every Sunday morning.  So I can possibly assume that the distance done for each day of a “run” is the total distance divided by the number of days.  I’m using a SQLite3 database to track this, question is, how do I extract this information?

This turned out to be the key to the answer.  I needed to enumerate all the days between two points.  SQLite3 has a julianday function, and with that I have been able to extract the information I need.

My database schema is simple. There are two tables:
CREATE TABLE bikes (id integer primary key not null, description varchar(64));
CREATE TABLE odometer (timestamp datetime not null default current_timestamp, action char(8) not null, bike_id integer not null, odometer real not null, constraint duplicate_log unique (timestamp, action, bike_id) on conflict replace);

Then there are the views.
CREATE VIEW run_id as select s.rowid as start_id, (select rowid from odometer where bike_id=s.bike_id and timestamp > s.timestamp and action='stop' order by timestamp asc limit 1) as stop_id from odometer as s where s.action='start';
CREATE VIEW "run" AS select start.timestamp as start_timestamp, stop.timestamp as stop_timestamp, start.bike_id as bike_id, start.odometer as start_odometer, stop.odometer as stop_odometer, stop.odometer-start.odometer as distance,julianday(start.timestamp) as start_day, julianday(stop.timestamp) as stop_day from (run_id join odometer as start on run_id.start_id=start.rowid) join odometer as stop on run_id.stop_id=stop.rowid;

The first view breaks up the start and stop events, and gives me row IDs for where each “run” starts and stops. I then use that in my run view to calculate distances and timestamps.

Here’s where the real voodoo lies, to enumerate days, I start at the very first timestamp in my dataset, find the Julian Day for that, then keep adding one day on until I get to the last timestamp. That gives me a list of Julian days that I can marry up to the data in the run view.

CREATE VIEW distance_by_day as
SELECT day_of_year, avg_distance FROM (
SELECT days.day - julianday(date(days.day,'start of year')) as day_of_year, sum(run.distance/max((run.stop_day-run.start_day),1))/count(*) as avg_distance
FROM run,
(WITH RECURSIVE
days(day) as (
SELECT julianday((select min(timestamp) from odometer))
union all
SELECT day+1 from days
limit cast(round(julianday((select max(timestamp) from odometer))-julianday((select min(timestamp) from odometer))) as int)
) SELECT day from days) as days
where
run.start_day < = days.day AND run.stop_day >= days.day
group by day_of_year) dist_by_doy;

This is the result.

Distance by Day Of Year

Distance by Day Of Year

Apr 062015
 

I’ve been a long time user of PGP, had a keypair since about 2003.  OpenPGP has some nice advantages in that it’s a more social arrangement in that verification is done by physically meeting people.  I think it is more personal that way.

However, you still can get isolated islands, my old key was a branch of the strong set, having been signed by one person who did do a lot of key-signing, but sadly thanks to Heartbleed, I couldn’t trust it anymore.  So I’ve had to start anew.

The alternate way to ensure communications is to use some third party like a certificate authority and use S/MIME.  This is the other side of the coin, where a company verifies who you are.  The company is then entrusted to do their job properly.  If you trust the company’s certificate in your web browser or email client, you implicitly trust every non-revoked valid certificate that company has signed.  As such, there is a proliferation of companies that act as a CA, and a typical web browser will come with a list as long as your arm/leg/whatever.

I’ve just set up one such certificate for myself, using StartCOM‘s CA as the authority.  If you trust StartCOM, and want my GPG key, you’ll find a S/MIME signed email with my key here.  If you instead trust my GPG signature and want my S/MIME public key, you can get that here.  If you want to throw caution to the wind, you can get the bare GPG key or S/MIME public key instead.

Update: I noticed GnuPG 2.1 has been released, so I now have an ECDSA key; fingerprint B8AA 34BA 25C7 9416 8FAE  F315 A024 04BC 5865 0CF9.  You may use it or my existing RSA key if your software doesn’t support ECDSA.

Mar 312015
 

Every now and again, I get some invite to join some social network or to “Confirm I know ${PERSON}”.  It doesn’t really matter what platform you choose, the end result is the same.

There are a couple of reasons I do not participate on these systems.

Number one is that when I get home, I typically cannot be stuffed having much to do with computers.  Sure I’ll be checking what news has happened during the day, I have an RSS aggregator for that.  I’ll check what the weather is doing tomorrow (as a cyclist, this is of high importance).  I’ll check emails, as that is how I keep in touch with people (I know, how quaint, but it works).

Maybe check some public forums/newsgroups.  That’s about it.  I really cannot be stuffed doing much else having fought with computers all day, I’m really not in the mood when I get home.

Second reason is the nature of these social networks.  Much of the interaction happens behind the scenes.  You’re “sharing” to an audience behind closed doors.  That audience has to be a member of that same group to even see your material.  Don’t believe me?  Try to log into Facebook and then go “Friend” a Google+ user and share something with them.  Or how about go log into Google+ and “circle” (is that the term?) a Linked In user.  These new-breed “social” networks are about as anti-social as it gets.

They’re today’s bulletin board system.  I’m sorry, are we really abandoning the World Wide Web for what’s little more than a BBS?  About the only difference I see is that JavaScript and HTML5 replace the ANSI/DEC escape sequences of old.  It’s still an isolated silo from which your information is locked in and only those who have opted into that network can participate.

A forum can be publicly viewed in most cases, in fact forums generally have trouble attracting an audience if they’re not publicly visible.

Finally there’s the privacy.  Sharing what you’re doing in your day to day life is one thing.  You can share a lot without giving much away personally.  “Friend”-ing people however is basically uploading your social graph, one link at a time.  One’s social graph is one of the most personal things one can expose about themselves, and increasingly the privacy policies of these social networks have been found lacking in several aspects.

Companies have been working on sophisticated ways in order to search and map out these social graphs, they would not be doing this if there wasn’t financial incentive to do so.  Knowing who someone’s friends are is the first step in being able to manipulate that person.  I’d rather not be someone’s puppet.

There’s also the phishing risk.  They’re popular sites to try and spoof.  I recently received one alleging to be from The Register author Simon Rockman.  It could be authentic, but then again, anyone could sign up for an account on one of these social network, claim to be someone they’re not, and try to lure you in.  I’ve got no way of verifying this, and with a broken “Reply-to” header, in the bin it goes.

So, next time you think of putting my email address in to a form on a social network page to invite me to join, don’t bother.  I do respond to emails, I even respond to comments left here (unless they’re spam), but I will not respond to social network invites, in fact I may not even receive them.

Mar 192015
 
Tropical Cyclone Nathan, Forecast map as of 2:50PM

Tropical Cyclone Nathan, Forecast map as of 2:50PM

This cyclone has harassed the far north once already, wobbled out in the Pacific like a drunken cyclist as a tropical low, has gained strength again and is now making a bee-line for Cape Flattery.

As seen, it also looks like doing the same stunt headed for Gove once it’s finished touching up far north Queensland.  Whoever up there is doing this rain dancing, you can stop now, it’s seriously pissing off the weather gods.

National and IARU REGION III Emergency Frequencies (Please keep clear and listen for emergency traffic)

  • 80m
    • 3.600MHz LSB (IARU III+WICEN)
  • 40m
    • 7.075MHz LSB (WICEN)
    • 7.110MHz LSB (IARU III)
  • 20m
    • 14.125MHz USB (WICEN)
    • 14.300MHz USB (IARU III)
    • 14.183MHz USB: NOT an emergency frequency, but Queensland State WICEN hold a net on this frequency every Sunday morning at around 08:00+10:00 (22:00Z Saturday).
  • 15m
    • 21.190MHz USB (WICEN)
    • 21.360MHz USB (IARU III)
  • 10m
    • 28.450MHz USB (WICEN)

I’ll be keeping an ear out on 14.125MHz in the mornings.

Update 20 March 4:31am: It has made landfall between Cape Melville and Cape Flattery as a category 4 cyclone.

Mar 192015
 

Hi all,

This is more a note to self for future reference.  Qt has a nice handy reference counting memory management system by means of QSharedPointer and QWeakPointer.  The system is apparently thread-safe and seems to be totally transparent.

One gotcha though, is two QSharedPointer objects cannot share the same pointer unless one is cloned from the other (either directly or via QWeakPointer).  The other is that you must leave deletion of the object to QSharedPointer.  You’ve given it your precious pointer, it has adopted it and while you may call the object, it is no longer yours, so don’t go deleting it.

So you create an object, you want to pass a reference to yourself to some other object.  How?  Like this?

QSharedPointer<MyClass> MyClass::ref() {
    return QSharedPointer<MyClass>(this); /* NO! */
}

No, not like that! That will create QSharedPointer instances left right and centre. Not what you want to do at all. What you need to do, is create the initial reference, but then store a weak reference to it. Then all future calls, you simply call the toStrongRef method of the weak reference to get a QSharedPointer that’s linked to the first one you handed out.

Then, having done this, when you create your new object, you create it with the new keyword as normal, take a QSharedPointer reference to it, then forget all about the original pointer! You can get it back by calling the data method of the pointer object.

To make it simple, here’s a base class you can inherit to do this for you.

    #include <QWeakPointer>
    #include <QSharedPointer>

    /*!
     * Self-Reference helper.  This allows for objects to maintain
     * references to "this" via the QSharedPointer reference-counting
     * smart pointers.
     */
    template<typename T>
    class SelfRef {
        public:
            /*!
             * Get a strong reference to this object.
             */
            QSharedPointer<T>    ref()
            {
                QSharedPointer<T> this_ref(this->this_weak);
                if (this_ref.isNull()) {
                    this_ref = QSharedPointer<T>((T*)this);
                    this->this_weak = this_ref.toWeakRef();
                }
                return this_ref;
            }

            /*!
             * Get a weak reference to this object.
             */
            QWeakPointer<T>        weakRef() const
            {
                return this->this_weak;
            }
        private:
            /*! A weak reference to this object */
            QWeakPointer<T>        this_weak;
    };

Example usage:

#include <iostream>
#include <stdexcept>
#include "SelfRef.h"

class Test : public SelfRef<Test> {
        public:
                Test()
                {
                        std::cout << __func__ << std::endl;
                        this->freed = false;
                }
                ~Test()
                {
                        std::cout << __func__ << std::endl;
                        this->freed = true;
                }

                void test() {
                        if (this->freed)
                                throw std::runtime_error("Already freed!");
                        std::cout
                                << "Test object is at "
                                << (void*)this
                                << std::endl;
                }

                bool                    freed;
                QSharedPointer<Test>    another;
};

int main(void) {
        Test* a = new Test();
        if (a != NULL) {
                QSharedPointer<Test> ref1 = a->ref();
                if (!ref1.isNull()) {
                        QSharedPointer<Test> ref2 = a->ref();
                        ref2->test();
                }
                ref1->test();
        }
        a->test();
        return 0;
}

Note that the line before the return is a deliberate use after free bug to prove the pointer really was freed.  Also note that the idea of setting a boolean flag to indicate the constructor has been called only works here because nothing happens between that use after free attempt and the destructor being called.  Don’t rely on this to see if your object is being called after destruction.  This is what the output session from gdb looks like:

RC=0 stuartl@rikishi /tmp/qtsp $ make CXXFLAGS=-g
g++ -c -g -I/usr/share/qt4/mkspecs/linux-g++ -I. -I/usr/include/qt4/QtCore -I/usr/include/qt4/QtGui -I/usr/include/qt4 -I. -I. -o test.o test.cpp
g++ -Wl,-O1 -o qtsp test.o    -L/usr/lib64/qt4 -lQtGui -L/usr/lib64 -L/usr/lib64/qt4 -L/usr/X11R6/lib -lQtCore -lgthread-2.0 -lglib-2.0 -lpthread 
RC=0 stuartl@rikishi /tmp/qtsp $ gdb ./qtsp 
GNU gdb (Gentoo 7.7.1 p1) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./qtsp...done.
(gdb) r
Starting program: /tmp/qtsp/qtsp 
warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Test
Test object is at 0x555555759c90
Test object is at 0x555555759c90
~Test
terminate called after throwing an instance of 'std::runtime_error'
  what():  Already freed!

Program received signal SIGABRT, Aborted.
0x00007ffff5820775 in raise () from /lib64/libc.so.6
(gdb) bt
#0  0x00007ffff5820775 in raise () from /lib64/libc.so.6
#1  0x00007ffff5821bf8 in abort () from /lib64/libc.so.6
#2  0x00007ffff610cd75 in __gnu_cxx::__verbose_terminate_handler() ()
   from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so.6
#3  0x00007ffff6109ec8 in ?? () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so.6
#4  0x00007ffff6109f15 in std::terminate() () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so.6
#5  0x00007ffff610a2e9 in __cxa_throw () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so.6
#6  0x0000555555555cea in Test::test (this=0x555555759c90) at test.cpp:20
#7  0x0000555555555315 in main () at test.cpp:41
(gdb) up
#1  0x00007ffff5821bf8 in abort () from /lib64/libc.so.6
(gdb) up
#2  0x00007ffff610cd75 in __gnu_cxx::__verbose_terminate_handler() ()
   from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so.6
(gdb) up
#3  0x00007ffff6109ec8 in ?? () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so.6
(gdb) up
#4  0x00007ffff6109f15 in std::terminate() () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so.6
(gdb) up
#5  0x00007ffff610a2e9 in __cxa_throw () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so.6
(gdb) up
#6  0x0000555555555cea in Test::test (this=0x555555759c90) at test.cpp:20
20                                      throw std::runtime_error("Already freed!");
(gdb) up
#7  0x0000555555555315 in main () at test.cpp:41
41              a->test();
(gdb) quit
A debugging session is active.

        Inferior 1 [process 17906] will be killed.

Quit anyway? (y or n) y

You’ll notice it fails right on that second last line because the last QSharedPointer went out of scope before this.  This is why you forget all about the pointer once you create the first QSharedPointer.

To remove the temptation to use the pointer directly, you can make all your constructors protected (or private) and use a factory that returns a QSharedPointer to your new object.

A useful macro for doing this:

/*!
 * Create an instance of ClassName with the given arguments
 * and immediately return a reference to it.
 *
 * @returns	QSharedPointer<ClassName> object
 */
#define newRef(ClassName, args ...)	\
	((new ClassName(args))->ref().dynamicCast<ClassName>())
Mar 022015
 

Well, it’s been a year and a half since I last posted details about the bicycle mobile station.  Shortly after getting the Talon on the road, setting it up with the top box and lighting, and having gotten the bugs worked out of that set-up, I decided to get a second mounting plate and set my daily commuter up the same way, doing away with the flimsy rear basket in place of a mounting plate for the top box.

VK4MSL/BM today after the trip home from work.

VK4MSL/BM today after the trip home from work.

That particular bike people might recognise from earlier posts, it’s my first real serious commuter bike. Now in her 5th year, has done over 10200km since 2012.  (The Talon has done 5643km in that time.) You can add to that probably another 5000km or so done between 2010 and 2012. It’s had a new rear wheel (a custom one, after having a spate of spoke breakages) and the drive chain has been upgraded to 9-speed. The latter upgrade alone gave it a new lease on life.

Both upgrades being money well spent, as was the upgrade to hydraulic brakes early in the bike’s lifetime. I suppose I’ve spent in those upgrades alone close to $1400, but worth it as it has given me quite good service so far.

As for my time with the top box. Some look at it and quiz me about how much weight it adds. The box itself isn’t that heavy, it just looks it. I can carry a fair bit of luggage, and at present, with my gear and tools in it it weighs 12kg. Heavy, but not too heavy for me to manage.

Initially when I first got it, it was great, but then as things flexed over time, I found I was intermittently getting problems with the top box coming off.  This cost me one HF antenna and today, the top box sports a number of battle-scars as a result.

The fix to this?  Pick a spot inside the top box that’s clear of the pannier rack rails and the rear tyre, and drill a 5mm hole through.  Then, when you mount the top box, insert an M5 bolt through the mounting plate and into the bottom of the top box and tighten with a 5mm wing nut.  The box now will not come loose.

vk4msl-bm-2713

Still lit up like a Christmas tree from this morning’s ride.

The lights still work, and now there’s a small rear-view camera.  On the TODO list is to rig up a 5V USB socket to power that camera with so that it isn’t draining the rather small internal battery (good for 4 hours apparently).

The station has had an upgrade, I bought a new LDG Z-100Plus automatic tuner which I’m yet to fully try out.  This replaces the aging Yaesu FC-700 manual tuner which, although very good, is fiddly to use in a mobile set-up and doesn’t tune 6m easily.

One on-going problem now not so much on the Boulder but on the Talon is an issue with pannier racks failing.  This started happening when I bought the pannier bags, essentially on the side I carry my battery pack (2kg), I repeatedly get the pannier rack failing.  The racks I use are made by Topeak and have a built-in spacer to accomodate disc brake calipers.  This seems to be a weak spot, I’ve now had two racks fail at about the same point.

Interestingly on the Boulder which also has disc brakes, I use the standard version of the same rack, and do not get the same failures.  Both are supposedly rated to 25kg, and my total load would be under 16kg. Something is amiss.

A recurring flaw with the Topeak racks

I’m on the look-out for a more rugged rack that will tolerate this kind of usage, although having seen this, I am inspired to try a DIY solution.  Then if a part fails, I can probably get replacement parts in any typical hardware store.  A hack saw and small drill are not heavy items to carry, and I can therefore get myself out of trouble if problems arise.

Feb 282015
 

Well, it’s been about 7½ years since I bought my first bike and started riding, and really, about 5 years since I started riding seriously as a means of transportation.

In late 2011 my father and I went halves in a pair of GPS/CB radio units, it was a 2 for 1 deal and so we bought these two units at about $400 each, normally they’d be about $700 individually. So there I started logging the distance I covered. I just used the in-built odometer on the GPS, resetting it when the bike went in for service.

When I got the mountain bike, I realised I needed to track the distance covered by each bike to ensure they all went in at their 1000km service on-time. So being a programmer by trade, I coded up a crude CGI/Perl script that used a SQLite back-end to log the odometer readings. It was a simple HTML form where I could enter the distance at regular intervals.  Crucially, it worked with the “feature phone” I used at the time.

The SQL views (no such thing as stored procedures in standard SQLite3) took care of actually calculating the differentials and so I used that to track my progress. So far so good. I’ve now had this in place since mid-2012 and I’ve brought in some of my data from early 2012, thus I’m now starting to see some trends.

Distances by year

Year Distance (km)
2012 5594.9
2013 4837.78
2014 4593.42

Am I getting lazy? Well, hard to say there. I go out less on the weekends and have also optimised my routes to reduce distances somewhat.  Some of this is weather-dependent, in the heat one does not feel like going outdoors.

Distance by month-of-year

Month Distance (km)
01 282.59
02 406.20
03 409.10
04 377.42
05 511.29
06 493.36
07 330.01
08 532.05
09 494.21
10 470.14
11 370.27
12 394.13

I’m not sure why there’s a lull in activity around July, but the most active months seem to be May and August.  The lull in January can be somewhat attributed to the end of the Christmas break.  I guess if anything, I should aim to be more active in July when the weather is the coolest.

Guess I’ll be keeping an eye on what happens over time with these stats and see if I can get them up a bit.

The following graph will continuously update as I pump data in. We’ll see what happens.

Distance by month-of-year

Feb 202015
 

All,

As an update on this…

Due to some issues (browser pop up behavior for example), with the Superfish Visual Discovery browser add-on, we have temporarily removed Superfish from our consumer systems until such time as Superfish is able to provide a software build that addresses these issues. As for units already in market, we have requested that Superfish auto-update a fix that addresses these issues.

To be clear, Superfish comes with Lenovo consumer products only and is a technology that helps users find and discover products visually. The technology instantly analyzes images on the web and presents identical and similar product offers that may have lower prices, helping users search for images without knowing exactly what an item is called or how to describe it in a typical text-based search engine.

The Superfish Visual Discovery engine analyzes an image 100% algorithmically, providing similar and near identical images in real time without the need for text tags or human intervention. When a user is interested in a product, Superfish will search instantly among more than 70,000 stores to find similar items and compare prices so the user can make the best decision on product and price.

Superfish technology is purely based on contextual/image and not behavioral. It does not profile nor monitor user behavior. It does not record user information. It does not know who the user is. Users are not tracked nor re-targeted. Every session is independent. When using Superfish for the first time, the user is presented the Terms of User and Privacy Policy, and has option not to accept these terms, i.e., Superfish is then disabled.

Mark Hopkins, Lenovo Support

That’s alright Mark, I’ve permanently removed Lenovo from my list of future suppliers. If I buy a Lenovo product, I’m going to insist the machine is delivered to me completely formatted of hardware and supplied with media to do a clean installation since it is clear you cannot be trusted to put an OS on a computer and not botch it in some manner.

I think there should be a law against this sort of bundling: too long machines have been delivered with crippling bloatware that either wastes system resources, causes security headaches or both. Sure, bundle some software, BUT ASK THE CUSTOMER BEFORE YOU INSTALL IT!

Feb 012015
 

How do software companies get things so wrong?  I aim this at both Google and Apple here, as both are equally as guilty of this.  Maybe Microsoft can learn from this?

So you see something on an “app store” that might be useful on the device you’re using.  Ohh, a free download, great!  Let’s download.  You click the link, and immediately get asked for a login and password.  There’s no option to proceed without.  They insist you create an account with them even if it’s the one and only thing you’re interested from them.

In the past my gripe has been with the Google Play store.  Even the “free” apps, require you to log in.  Ohh, and to add insult to injury, the Google Play store doesn’t just expect any Google account, it has to be one of their Gmail accounts.  Back in the late 90s I had an email address with most providers as the average quota was about 5MB.  I’ve had a mailbox of my own with a multi-gigabyte (actually limited only by disk capacity) “quota” since 2002, I have no use for Gmail, and only keep my old Yahoo! address (created in 1999) for historic reasons.

I have an Android phone (release 4.1: thanks to ZTE’s backward thinking and short attention span), and thankfully there’s the F-Droid store which has sufficient applications for my use.  So I can work around the Google Play store most of the time and so far, haven’t needed anything from there.

Today, my gripe is at Apple, and the “app” in question is MacOS X, which cannot be obtained anywhere else.

With all the high-profile attacks on websites that store user accounts, one has to ask, why?  It’s one extra username and password, which given the frequency I’m likely to use it, will have to be written down and stored somewhere secure as it won’t get sufficient use to commit it to memory.  Before people point out password managers, I’d like to point out one thing: it’s still writing it down!

There’s absolutely no need for an “app store” to know your email address, usernames, passwords, or any details.  If you are actually purchasing an application, they only need enough information to process a payment.

Usually this is by a debit/credit card, so they need to know the details on the card.  An alternative might be direct deposit through a bank, at which point they need to supply you with details on how to make the payment — details that include the information they need to match your payment in their ledger to your store purchase.  At no point do they need anything else.

For convenience an email address might be supplied so they can confirm your order or contact you if there’s a problem, however for debit/credit cards, this happens so quickly that it can be achieved via the web browser.

Despite this, they insist on you providing just about everything.

I’m no stranger to the “app store” concept.  Linux and BSD distributions have had this sort of concept for years.  BSD has had ports for as long as I can remember.  Debian had apt since 1998, Gentoo has had portage since its inception in 2003 and RPM-based distributions have had yum for some time too.

None of these actually need to know who you are in order to download a package.  Admittedly none of these are geared toward commercial sales of software, and so lack the ability to prompt for credentials or payment information.

Since both Google Play and the Apple App store have solved the former problem, I see no reason why they couldn’t solve the latter.  I don’t want to post anything to the site, I don’t want to leave feedback as I can hardly comment on something I haven’t received yet, and I don’t know when I’ll next visit the site.

If I was going to be back repeatedly, sure, I’ll make an account.  It’ll make everyone’s lives easier. (Including the blackhats!)  But I’m not.  I have a late-2008 model MacBook, probably the oldest machine that Apple support for their latest OS.  The machine dual-boots MacOS X 10.6 and Gentoo Linux, and spends 99% of its time in the latter OS.

Given the age of the machine and the frequency at which I use its native OS, it is not worth me spending a lot of time or expense updating it.  A 2GHz Core 2 Duo with 8GB RAM and a 750GB HDD is good enough for many tasks under Linux, but is the bare minimum to run OS X 10.10.  The only reason this machine doesn’t grace my desk at work anymore is the fact the lack of ports (USB in particular) proved to be a right pain.

Why update?  Well, applications these days seem to expect at least MacOS X 10.7 now.  I either have to build everything myself or update the OS, so I’m investigating the possibility of updating the OS to see if it’s feasible.  Apparently it’s a free download, so why not?

Well, why not indeed!  Instead of having a simple http, https or ftp link to the file in question (maybe a .dmg image) for software they’re not actually selling to me in the traditional sense, they instead insist on making me jump through hoops like requiring their “app store” client — so I can’t just grab the link, tell the web server here to download the file then grab it from there when I’m ready.

Since I can’t do the download any other way than via their “app store” client, I have to remain booted in MacOS X in order to download it regardless of what I might otherwise wish to do the machine and what OS that requires.

However, before I can even think about starting the download, I’ve got to register an account, supplying a username and password for something that will probably be used exactly once.  Details that they have to pay people big money to store securely.

Instead of spending some money paying someone to add an extra one-off button and form to their “app store” clients, they instead spend significantly more on infrastructure designed to meet the privacy requirements of various laws to store user information that simply is not necessary for the transaction to proceed.

In light of the sophistication of the modern cracker and the cut-throat nature of the mobile market, is this such a wise use of company funds?

Jan 032015
 

I’ve been riding on the road now for some years, and while I normally try to avoid it, I do sometimes find myself riding on the road itself rather than on the footpath or bicycle path.

Most of the time, the traffic is fine.  I’m mindful of where everyone is, and there aren’t any problems, but I have had a couple of close calls from time to time.  Close calls that have me saying “ode for a horn”.

By law we’re required to have a bell on our bikes.  No problem there, I have a mechanical one which is there purely for legal purposes.  If I get pulled over by police, and they ask, I can point it out and demonstrate it.  Requirement met?  Tick to that.

It’s of minimal use with pedestrians, and utterly useless in traffic.

Early on with my riding I developed a lighting system which included indicators.  Initially this was silent, I figured I’d see the lights flashing, but after a few occasions forgetting to turn indicators off, I fitted a piezo buzzer.  This was an idea inspired by the motorcycles ridden by Australia Post contractors, which have a very audible buzzer.  Jaycar sell a 85dB buzzer that’s waterproof, overkill in the audio department but fit for purpose.  It lets me know I have indicators on and alerts people to my presence.

That is, if they equate the loud beep to a bicycle.  Some do not.  And of course, it’s still utterly useless on the road.

I figured a louder alert system was in order.  Something that I could adjust the volume on, but loud enough to give a pedestrian a good 30 seconds warning.  That way they’ve got plenty of time to take evasive action while I also start reducing speed.  It’s not that I’m impatient, I’ll happily give way, but I don’t want to surprise people either.  Drivers on the other hand, if they do something stupid it’d be nice to let them know you’re there!

My workplace looks after a number of defence bases in South-East Queensland, one of which has a railway crossing for driver training.  This particular boom gate assembly copped a whack from a lightning strike, which damaged several items of equipment, including the electronic “bells” on the boom gate itself.  These “bells” consisted of a horn speaker with a small potted PCB mounted to the back which contained an amplifier and bell sound generator.  Apply +12V and the units would make a very loud dinging noise.  That’s in theory; in practise, all that happened was a TO-220 transistor got hot.  Either the board or the speaker (or both) was faulty.

It was decided these were a write-off, and after disassembly I soon discovered why: the voice coils in the horn speakers had been burnt out.  A little investigation, and I figured I could replace the blown out compression drivers and get the speakers themselves working again, building my own horn.

A concept formed: the horn would have two modes, a “bell” mode with a sound similar to a bicycle bell, and a “horn” mode for use in traffic.  I’d build the circuit in parts, the first being the power amplifier then interface to it the sound effect generator.

To make life easier testing, I also decided to add a line-in/microphone-in feature which would serve to debug construction issues in the power amplifier and add a megaphone function.  (Who knows, might be handy with WICEN events.)

Replacing the compression drivers

Obviously it’d be ideal to replace it with the correct part, but looking around, I couldn’t see anything that would fit the housing.  That, and what I did see, was more expensive than buying a whole new horn speaker.

There was a small aperture in the back about 40mm in diameter.  The original drivers were 8ohms, and probably rated at 30W and had a convex diaphragm which matched the concave geometry in the back of the horn assembly.

Looking around, I saw these 2W mylar cone speakers.  Not as good as a compression driver, but maybe good enough?  It was cheap enough to experiment.  I bought two to try it out.

I got them home, tacked some wires onto one of them and plugged it into a radio.  On its own, not very loud, but when I held it against the back of a horn assembly, the amplification was quite apparent.  Good enough to go further.  I did some experiments with how to mount the speakers to the assembly, which required some modifications to be made.

I soon settled on mounting the assembly to an aluminium case with some tapped holes for clamping the speaker in place.  There was ample room for a small amplifier which would be housed inside the new case, which would also serve as a means of mounting the whole lot to the bike.

Bell generator

I wasn’t sure what to use for this, I had two options: build an analogue circuit to make the effect, or program a microcontroller.  I did some experiments with an ATMega8L, did manage to get some sound out of it using the PWM output, but 8kB of flash just wasn’t enough for decent audio.

A Freetronics LeoStick proved to be the ticket.  32kB flash, USB device support, small form factor, what’s not to like?  I ignored the Arduino-compatible aspect and programmed the device directly.  Behind the novice-friendly pin names, they’re an ATMega32U4 with a 16MHz crystal.  I knocked up a quick prototype that just played a sound repeatedly.  It sounded a bit like a crowbar being dropped, but who cares, it was sufficient.

Experimenting with low-pass filters I soon discovered that a buffer-amp would be needed, as any significant load on the filter would render it useless.

A 2W power amplifier

Initially I was thinking along the lines of a LM386, but after reading the datasheet I soon learned that this would not cut it.  They are okay for 500mW, but not 2W.  I didn’t have any transistors on hand that would do it and still fit in the case, then I stumbled on the TDA 1905.  These ICs are actually capable of 5W into 4 ohms if you feed them with a 14V supply.  With 9V they produce 2.5W, which is about what I’m after.

I bought a couple then set to work with the breadboard.  A little tinkering and I soon had one of the horn speakers working with this new amplifier.  Plugged into my laptop, I found the audio output to be quite acceptable, in fact turned up half-way, it was uncomfortable to sit in front of.

I re-built the circuit to try and make use of the muting feature.  For whatever reason, I couldn’t get this to work, but the alternate circuit provided a volume control which was useful in itself.

The pre-amplifier

For the line-level audio, there’s no need for anything more fancy than a couple of resistors to act as a passive summation of the left and right channels, however for a microphone and for the LeoStick, I’d need a preamp.  I grabbed a LM358 and plugged that into my breadboard alongside the TDA1905.

Before long, I had a working microphone preamp working using one half of the LM358, based on a circuit I found.  I experimented with some resistor values and found I got reasonable amplification if I upped some of the resistor values to dial the gain back a little.  Otherwise I got feedback.

For the LeoStick, it already puts out 5V TTL, so a unity-gain voltage follower was all that was needed.  The second half of the LM358 provided this.  A passive summation network consisting of two resistors and DC-blocking capacitor allowed me to combine these outputs for the TDA1905.

One thing I found necessary, the TDA1905 and LM358 misbehave badly unless there’s a decent size capacitor on the 9V rail.  I found a 330uF electrolytic helped in addition to the datasheet-prescribed 100nF ceramics.

Power supply

Since I’m running on batteries with no means of generating power, it’s important that the circuit does not draw power when idle.  Ideally, the circuit should power on when either I:

  • plug the USB cable in (for firmware update/USB audio)
  • toggle the external source switch
  • press the bell button

We also need two power rails: a 9V one for the analogue electronics, and a 5V one for the LeoStick.  A LM7809 and LM7805 proved to be the easiest way to achieve this.

To allow software control of the power, a IRF9540N MOSFET was connected to the 12V input and supplies the LM7809.  The gate pin is connected to a wired-OR bus.  The bell button and external source switch connect to this bus with signal diodes that pull down on the gate.

Two BC547s also have collectors wired up to this bus, one driven from the USB +5V supply, and the other from a pin on the LeoStick.  Pressing the Bell button would power the entire circuit up, at which point the LeoStick would assert its power on signal (turning on one of the BC547s) then sample the state of the bell button and start playing sound.  When it detects the button has been released, it finishes its playback and turns itself off by releasing the power on signal.

Sound effect generator

Earlier I had prototyped a bell generator, however it wasn’t much use as it just repeatedly made a bell noise regardless of the inputs.  To add insult to injury, I had lost the source code I used.  I had a closer look at the MCU datasheet, deciding to start from a clean slate.

The LeoStick provides its audio on pin D11, which is wired up to Port B Pin 7.  Within the chip, two possible timers hook up: Timer 0, which is an 8-bit timer, and Timer 1, which is 16-bits.  Both are fed from the 16MHz system clock.  The bit depth affects the PWM carrier frequency we can generate, the higher the resolution, the slower the PWM runs.  You want the PWM frequency as high as possible, ideally well above 20kHz so that it’s not audible in the audio output, and obviously well above the audio sampling rate.

At 16MHz, a 16-bit timer would barely exceed 240Hz, which is utterly useless for audio.  A 10-bit timer fares better, with 15kHz, older people may not hear it but I certainly can hear 15kHz.  That leaves us with 8-bits which gets us up to 62kHz.  So no point in using Timer 1 if we’re only going to be using 8-bits of it, we might as well use Timer 0.

Some of you familiar with this chip may know of Timer 4, which is a high-speed 10-bit timer fed by a separate 64MHz PLL.  It’s possible to do better quality audio from here, either running at 10-bits with a 62kHz carrier, or dropping to 8-bits and ramping the frequency to 250kHz.  Obviously it’d have been nice, but I had already wired things up by this stage, so it was too late to choose another pin.

Producing the output voltage is only half the equation though: once started, the PWM pin will just output a steady stream of pulses, which when low-passed, produces a DC offset.  In order to play sound, we need to continually update the timer’s Capture Compare register with each new sample at a steady rate.

The most accurate way to do this, is to use another timer.  Timer 3 is another 16-bit timer unit, with just one capture compare output available on Port C pin 3.  It is an ideal candidate for a timer that has no external influence, so it gets the job of updating the PWM capture compare value with new samples.

Timer 1 is connected to pins that drive two of the three LEDs on the LeoStick, with Timer 4 driving the remaining one, so if I wanted, I could have LEDs fade in and out with it instead of just blinking.  However, my needs are basic, and I need something to debounce switches and visibly blink LEDs.  So I use that with a nice long period to give me a 10Hz timer.

Here is the source code.  I’ll add schematics and other notes to it with time, but the particular bits of interest for those wanting to incorporate PWM-generated sound in their AVR projects are the interrupt routine and the sound control functions.

To permit gapless playback, I define two buffers which I alternate between, so while one is being played back, the other can be filled up with samples.  I define these on line 139 with the functions starting at line 190.  The interrupt routine that orchestrates the playback is at line 469.

When sound is to be played, the first thing that needs to happen is for the initial buffer to be loaded with samples using the write_audio function.  This can either read from a separate buffer in RAM (e.g. from USB) or from program memory.  One of the options permits looping of audio.  Having loaded some initial data in, we can then call start_audio to set up the two timers and get audio playback rolling.  start_audio needs the sample rate to configure the sample rate timer, and can accept any sample rate that is a factor of 16MHz (so 8kHz, 16kHz up to 32kHz).

The audio in this application is statically compiled in, taking the form of an array of uint8_t‘s in PROGMEM.

Creating the sounds

I initially had a look around to see if I could get a suitable sound effect.  This proved futile, I was ideally looking around for a simple openly-licensed audio file.  Lots of places offered something, but then wanted you to sign up or pay money.  Fine, I can understand the need to make a quid, and if I were doing this a lot, I’d pay up, but this is a once-off.

Eventually, I found some recordings which were sort of what I was after, but not quite.  So I downloaded these then fired up Audacity to have a closer look.

The bicycle bell

Bicycle bells have a very distinctive sound to them, and are surprisingly complicated.  I initially tried to model it as an exponentially decaying sinusoid of different frequencies, but nothing sounded quite right.

The recording I had told me that the fundamental frequency was just over 2kHz.  Moreover though, the envelope was amplitude-modulated by a second sinusoid: this one about 15Hz.  Soon as I plugged this second term in, things sounded better.  This script, was the end result.  The resulting bell sounds like this:

So somewhat bell-like.  To reduce the space, I use a sample rate of 6.4kHz.  I did try a 4kHz sample rate but was momentarily miffed at the result until I realised what was going on: the bell was above the Nyquist frequency at 4kHz, 6.4kHz is the minimum practical rate that reproduces the audio.

I used Audacity to pick a point in the waveform for looping purposes, to make it sound like a bell being repeatedly struck.

The horn

I wanted something that sounded a little gutsy.  Like an air-horn on a truck.  Once again, I hit the web, and found a recording of a train horn.  Close enough, but not long enough, and a bit noisy.  However opening it up in Audacity and doing a spectrum analysis, I saw there were about 5 tones involved.  I plugged these straight into a Python script and decided to generate those directly.  Using a raised cosine filter to shape the envelope at the start and end, and I soon had my horn effect.  This script generates the horn.  The audio sounds like this:

Using other sound files

If you really wanted, you could use your own sound recordings.  Just keep in mind the constraints of the ATMega32U4, namely, 32kB of flash has to hold both code and recordings.  An ATMega64 would do better.  The audio should be mono, 8-bits and unsigned with as lower sample rate as you can get away with.  (6.4kHz proved to be sufficient for my needs.)

Your easiest bet would be to either figure out how to read WAV files (in Python: wave module), or alternatively, convert to raw headerless audio files, then code up a script that reads the file one byte at a time. The Python scripts I’ve provided might be a useful starting point for generating the C files.

Alternatively, you can try interfacing a SDCard and embedding a filesystem driver and audio file parser (not sure about WAVE but Sun Audio is easily parsed), this is left as an exercise for the adventurous.

Finishing up

I’ll put schematics and pictures up soonish.  I’m yet to try mounting the whole set up, but so far the amplifier is performing fine on the bench.