May 142022
 
Seriously Amazon, why?

So, yeah, I’m trying to convert music wish-list entries to actual recordings in my music collection (as I won’t hear many of these on the radio anymore). I must stress I do want to support the artist by buying at least one license to their work. Preferably in a lossless form like CD or FLAC, but LP will do if the other two aren’t available. Heck, I even have a cassette player if it comes to that!

I don’t want to pirate music. That was something I did in the last century because I didn’t have money — those MP3s are deleted long ago (they got thrown out around 2004 or so; for both technical and legal reasons).

Making legally-purchased copies unobtainable does not help make this happen! Making copies unobtainable encourages piracy!

In this case, someone does have a copy for sale. There’s even an “Add To Cart” button to indicate a desire to purchase. Guess what, it just tells me “Not Added” when I click it. Can I contact Amazon about it? Not that I can see!

Seems the recording industry and the retailers are their own worst enemy on this front. Too distracted by the modern “hip” stuff than the stuff the rest of us actually listen to.

May 082022
 

So, this is quite sad news… I learned this on Friday morning that one of Brisbane’s longer-serving radio stations will be taken over by new management and will change its format from being a “classic hits” music station, to being a 24/7 sports coverage station.

It had been operated by the Australian Radio Network who had recently done a merger with a rival network, Grant Broadcasters, picking up their portfolio which included their portfolio of stations which included a number of other Greater Brisbane region stations. This tipped them over the edge and so they had to let one go, the unlucky victim was their oldest: 4KQ.

Now, you’re thinking, big deal, there are lots of radio stations out there, including Internet radio. Here’s why this matters. Back in the 90s, pretty much all of the stations here in Brisbane were locally run. They might’ve been part of a wider network, but generally, the programming about shows and music was decided on by people in this area. Lots of songs were hits only in Brisbane. There are some songs that did not make the music charts anywhere else world-wide. But, here in Brisbane, we requested those songs.

Sometimes the artists knew about this, sometimes not.

Over time, other stations have adjusted their format, and in many cases, abandoned local programming, doing everything from Sydney and Melbourne. Southern Cross Austereo tried this with Triple M years ago, and in the end they had to reverse the decision as their ratings tanked and complaints inundated the station.

4KQ represented one of the last stations to keep local programming. I’m not sure how many still do, but in particular this station was unique amongst the offerings in this area due to its wide coverage of popular music spanning 1960 ~ 1995, and in particular, its focus on the Brisbane top-40 charts.

Some of the radio programs too were great: Brent James in particular had an art for painting a picture of Brisbane at that time for both people who were there to experience it, those who missed out because they lived someplace else, and people like myself who were either too young to remember or not alive at the time in the first place. A lot of their other staff too, had a lot of music knowledge and trivia — yes you can reproduce the play lists with one’s own music collection, but the stories behind the hits are harder to replicate. Laurel Edwards is due to celebrate her 30th year with the station — that’s a long commitment, and it’s sad to think that this will be her last through no fault or decision of her own.

It’s loss as a music station is a major blow to the history of this city. To paraphrase Joni Mitchell, they’ve torn down Festival Hall to put up an apartment block!

A new normal

The question is, where to now? The real sad bit is that this was a successful station that was only culled because of a regulatory compliance issue: ARN now had too many stations in the Greater Brisbane area, and had to let one go. They reluctantly put it up for sale, and sure enough, a buyer took it, but that buyer was not interested in preserving anything other than the frequency, license and broadcast equipment.

In some ways, AM is a better fit for the yap-fest that is SEN-Q. They presently broadcast on DAB+ at 24kbps in essentially AM-radio quality. 4KQ has always been a MW station, originally transmitting at 650kHz back in 1947, moving to 690kHz a year later… then getting shuffled up 3kHz to its present-day 693kHz in 1978 when the authorities (in their wisdom at the time) decided to “make room” by moving all stations to a 9kHz spacing.

Music has never been a particularly good fit for AM radio, but back in 1947 that was the only viable option. FM did exist thanks to the work of Edwin Armstrong, but his patents were still active back then and the more complicated system was less favourable to radio manufacturers at a time when few could afford a radio (or the receiver license to operate it). So AM it was for most broadcasters of that time. “FM radio” as we know it today, wouldn’t come into existence in Brisbane until around 1980, by which time 4KQ was well-and-truly established.

The question remains though… ratings were pretty good, clearly there is demand for such a station. They had a winning formula. Could an independent station carry forward their legacy?

The options

So, in July we’ll have to get used to a new status-quo. It’s not known how long this will last. I am not advocating vigilante action against the new owners. The question will be, is there enough support for a phoenix to rise out of the ashes, and if so, how?

Existing station adopting 4KQ’s old format?

This might happen. Not sure who would be willing to throw out what they have now to try this out but this may be an option. There are a few stations that might be “close enough” to absorb such a change:

  • 4BH (1116kHz AM) does specialise in the “older” music, but it tends to be the softer “easy listening” stuff, they don’t do the heavier stuff that 4KQ and others do. (e.g. you won’t hear AC/DC)
  • KIIS 97.3 (97.3MHz FM) was 4KQ’s sister station, at present they only do music from the 80s onwards.
  • Triple M (104.5MHz FM) would be their closest competitor. They still do some 60s-80s stuff, but they’re more focused on today’s music. There’s a sister-station, Triple M Classic Rock (202.928MHz DAB+) but they are an interstate station, with no regional focus.
  • Outside of Brisbane, River 94.9 (94.9MHz FM) in Ipswich would be the closest to 4KQ. They make frequent mentions of 4IP and its charts. Alas, they are likely beaming west as they are not receivable in this part of Brisbane at least. (VK4RAI on the other hand, located on the same tower can be received, and worked from here… so maybe it’s just a case of more transmit power and a new antenna to service Brisbane?)

I did a tune-around the other day and didn’t hear anything other than those which was in any way comparable.

Interesting aside, 4IP of course was the hit station of its day. These days, if you look up that call-sign, you get directed to RadioTAB… another sports radio station network. Ironic that its rival meets the same fate at the hands of a rival sports radio network.

A new station?

Could enough of us band together and start afresh? Well, this will be tough. It’d be a nice thing if we could, and maybe provide work for those who started the year thinking their job was mostly secure only to find they’ve got two more months left… but the tricky bit is we’re starting from scratch.

FM station?

A new FM station might be ideal in terms of suiting the format, and I did look into this. Alas, not going to happen unless there’s a sacrifice of some sort. I did a search on the ACMA license database; putting in Mt. Coot-tha as the location (likely position of hypothetical transmitter, I think I chose Ch 9 site, but any on that hill will do), giving a radius of 200km and a frequency range of 87-109MHz.

Broadcast FM radio stations are typically spaced out every 800kHz; so 87.7MHz, 88.5MHz, 89.3MHz, … etc. Every such frequency was either directly taken, or had a station within 400kHz of it. Even if the frequency “sounded” clear, it likely was being used by a station I could not receive. A big number of them are operated by churches and community centres, likely low-power narrowcast stations.

The FM broadcast band, as seen from a roof-top 2m “flower pot” in The Gap.

There’s only two ways a new station can spring up on FM in the Brisbane area:

  • an existing station closes down, relinquishing the frequency
  • all the existing stations reduce their deviation, allowing for new stations to be inserted in between the existing ones

The first is not likely to happen. Let’s consider the latter option though. FM bandwidth is decided by the deviation. That is, the modulating signal, as it swings from its minimum trough to its maximum peak, causes the carrier of the transmitter to deviate above or below its nominal frequency in proportion to the input signal amplitude. Sometimes the deviation is almost identical to the bandwidth of the modulating signal (narrowband FM) or sometimes it’s much greater (wideband FM).

UHF CB radios for example; deviate either 2.5kHz or 5kHz, depending on whether the radio is a newer “80-channel” device or an older “40-channel” device. This is narrowband FM. When the ACMA decided to “make room” on UHF CB, they did so by “grandfathering” the old 40-channel class license, and decreeing that new “80-channel” sets are to use a 2.5kHz deviation instead of 5kHz. This reduced the “size” of each channel by half. In between each 40-channel frequency, they inserted a new 80-channel frequency.

This is simple enough with a narrowband FM signal like UHF CB. There’s no sub-carriers to worry about, and it’s not high-fidelity, just plain old analogue voice.

Analogue television used FM for its audio, and in later years, did so in stereo. I’m not sure what the deviation is for broadcast FM radio or television, but I do know that the deviation used for television audio is narrower than that used for FM radio. So evidently, FM stereo stations could possibly have their deviation reduced, and still transmit a stereo signal. I’m not sure what the trade-off of that would be though. TV stations didn’t have to worry about mobile receivers, and most viewers were using dedicated, directional antennas which better handled multi-path propagation (which would otherwise cause ghosting).

Also, TV stations to my knowledge, while they did transmit sub-carriers for FM stereo, they didn’t transmit RDS like FM radio stations do. Reducing the deviation may have implications on signal robustness for mobile users and for over-the-air services like RDS. I don’t know.

That said, lets suppose it could be done, and say Triple M (104.5MHz) and B105 (105.3MHz) decided to drop their deviation by half: we could then maybe squeeze a new station in at 104.1MHz. The apparent “volume” of the other two stations would drop by maybe 3dB, so people will need to turn their volume knobs up higher, but might work.

I do not know however if this is technically possible though. In short, I think we can consider a new FM station a pipe dream that is unlikely to happen.

New AM station?

A new AM station might be more doable. A cursory look at the same database, putting in much the same parameters but this time, a 300km radius and a frequency range of 500kHz-1.7MHz, seems to suggest there are lots of seemingly “unallocated” 9kHz slots. I don’t know what the frequency allocation strategy is for AM stations within a geographic area. I went a wider radius because MW stations do propagate quite far at night: I can pick up 4BU in Bundaberg and ABC Radio Emerald from my home.

The tricky bit is physically setting up the transmitter. MW transmitters are big, and use lots of power. 4KQ for example transmitted 10kW during daylight hours. Given it’s a linear PA in that transmitter, that means it’s consuming 20kW, and when it hits a “peak” it will want that power now!

The antennas are necessarily large; 693kHz has a wavelength of 432m, so a ¼-wave groundplane is going to be in the order of 100m tall. You can compromise that a bit with some clever engineering (e.g. see 4QR’s transmitter site off the Bruce Highway at Bald Hills — guess what the capacitance hat on the top is for!) but nothing will shrink that antenna into something that will fit a suburban back yard.

You will need a big open area to erect the antenna, and that antenna will need an extensive groundplane installed in the ground. The stay-wires holding the mast up will also need a big clearance from the fence as they will be live! Then you’ve got to keep the transmitter fed with the power it demands.

Finding a place is going to be a challenge. It doesn’t have to be elevated for MW like it does for VHF services (FM broadcast, DAB+), but the sheer size of the area needed will make purchasing the land expensive.

And you’ve got to consider your potential neighbours too, some of whom may have valid concerns about the transmitter: not liking the appearance of a big tower “in their back yard”, concerns about interference, concerns about “health effects”… etc.

DAB+?

This could be more doable. I don’t know what costs would be, and the big downside is that DAB+ radios are more expensive, as well as the DAB+ signal being more fragile (particularly when mobile). Audio quality would be much better than AM, but not quite as good as FM (in my opinion).

It’d basically be a case of opening an account with Digital Radio Broadcasting Pty Ltd, who operate the Channel 9A (202.928MHz) and Channel 9B (204.64MHz) transmitters. Then presumably, we’d have to encode our audio stream as HE-AAC and stream it to them somehow, possibly over the Internet.

The prevalence of “pop-up” stations seems to suggest this method may be comparatively cost-effective for larger audiences compared to commissioning and running our own dedicated transmitter, since the price does not change whether we have 10 listeners or 10000: it’s one stream going to the transmitter, then from there, the same signal is radiated out to all.

Internet streaming?

Well, this really isn’t radio, it’s an audio stream on a website at this point. The listener will need an Internet connection of their own, and you, the station operator, will be paying for each listener that connects. The listener also pays too: their ISP will bill them for data usage.

A 64kbps audio stream will consume around 230MB every 8 hours. If you stream it during your typical 8-hour work day, think a CD landing on your desk every 3 days. That’s the data you’re consuming. That data needs to be paid for, because each listener will have their own stream. If there’s only a dozen or so listeners, Internet radio wins … but if things get big (and 4KQ’s listenership was big), it’ll get expensive fast.

The other downside is that some listeners may not have an Internet connection, or the technical know-how to stream a radio station. I for example, do not have Internet access when riding the bicycle, so Internet radio is a no-go in that situation. I also refuse to stream Internet radio at work as I do not believe I should be using a workplace Internet connection for personal entertainment.

Staff?

The elephant in the room is staffing… there’s a workforce that kept 4KQ going who would soon be out of work, would they still be around if such a station were to materialise in the near future? I don’t know. Some of the announcers may want a new position in the field, others may be willing to go back to other vocations, and some are of an age that they may decide hanging up the headphones sounds tempting.

I guess that will be a decision for each person involved. For the listeners though, we’ve come to know these people, and will miss not hearing from them if they do wind up not returning to the air.

In the meantime

What am I doing now? Well, not saving up for a broadcast radio license (as much as my 5-year-old self would be disgusted at me passing up such an opportunity). I am expanding my music collection… and I guess over the next two months, I’ll be taking special note of songs I listen to that aren’t in my collection so I can chase down copies: ideally CDs or FLAC recordings (legally purchased of course!)… or LPs if CDs are too difficult.

Record companies and artists could help here — there are services like ZDigital that allow people to purchase and download individual songs or full albums in FLAC format. There are also lots of albums that were released decades ago, that have not been re-released by record companies. Sometimes record companies don’t release particular songs because they seemingly “weren’t popular”, or were popular in only a few specific geographic areas (like Brisbane).

People like us do not want to pirate music. We want to support the artists. Their songs did get played on radio, and still do; but may not be for much longer. Not everything is on Spotify, and sometimes that big yellow taxi has a habit of taking those hits away that you previously purchased. They could help themselves, and the artists they represent, by releasing some of these “less popular” songs as FLAC recordings for people to purchase. (Or MP3 if they really insist… but some of us prefer FLAC for archival copies.)

The songs have been produced, the recordings already exist, it seems it’s little skin of their nose to just release them as digital-only singles on these purchase-for-download platforms. I can understand not wanting to spend money pressing discs and having to market and ship them, but a file? Some emails, a few signed agreements and one file transfer and it’s done. Not complicated or expensive.

Please, help us help you.

Anyway… I guess I have a shopping list to compile.

Dec 062021
 

Tonight I learned something disturbing… I heard hear-say evidence that someone I know, had made the decision to obtain a fraudulent COVID-19 vaccination certificate for the purpose of bypassing the upcoming restrictions due to be applied on the 17th December, 2021.

Now, it comes as no surprise that people will want to dodge this. I won’t identify the individual who is trying to dodge the requirements in this case, nor will I reveal my source. As what I have is hear-say evidence, this is not admissible in a court of law, and it would be wrong for me to name or identify the person in any way.

No doubt though, the authorities have considered this possibility. They cracked down on one “doctor”, who was found to be issuing fraudulent documents a little over a month ago. She isn’t the first, won’t be the last either. It’s not entirely clear looking at the Queensland Government website what the penalties are for supplying fraudulent documentation. One thing I know for certain, I do not want to be on the receiving end. I do not want to have to justify my presence because someone I go to a restaurant with chooses to break the rules.

My biggest fear in this is two-fold:

  1. Fear of prosecution from association with the individual committing fraud
  2. Fear of knee-jerk restrictions being applied to everybody because a small number could not follow the rules

We’ve seen #2 already this pandemic. It’s why we’ve got this silly check-in program in the first place. I’ve already made my thoughts clear on that.

What worries me is it’s unknown at this stage how the certificate can be verified. There are two possible ways I can think of: the Individual Healthcare Identifier and the Document number, both of which appear on the MyGov-issued certificates. Are the staff members at venues able to validate these documents somehow? How do they know they’re looking at a genuine certificate? Is it a matter of blind-faith, or can they punch these details in and come up with something that says yay or nay?

I’m guessing the police have some way of verifying this, but, as a staff member at a venue, do you really want to be calling the police on patrons just because you have a “gut feeling” that something is fishy? How is this going to be policed really?

Surprise!

Let’s play devil’s advocate and suggest that indeed, there will be surprise inspections by the constabulary. Presumably they have a way of validating these certificates, otherwise what is the point? Now, suppose for arguments sake, one or two people are found to be holding fraudulent documents.

What then? Clearly, the guilty parties will have some explaining to do. What about the rest of us at that table, are we guilty by association? How about the business owner? The staff who were working that shift?

Cough! Sneeze! I’m not feeling well!

The other prospect is even worse, suppose that a few of us come down with an illness, get tested, and it winds up being one of the many strains floating around. Maybe it’s original-recipe COVID-19, maybe it’s Alpha, or Delta… this new Omicron variant… would you like some Pi with that? (You know, the irrational one that never ends!)

You’ve had to check-in (or maybe you don’t, but others you were with did, and they say you were there too — and CCTV backs their story up). Queensland Health looks up your details, and hang on, you’re not vaccinated. They check with venue staff, “Ohh yes, that person did show me a certificate and it looked valid”.

Hmmm, dear sir/madam, could you please show us your certificate? Ohh, you haven’t got one? The staff at the restaurant say you do. BUSTED! You’d either be charged for failing to follow a health direction, or charged with fraud, possibly both.

What’s worse with this hypothetical situation is that you and the people you’re with are then exposed to a deadly virus. At least with the surprise inspection in the previous hypothetical situation no one gets sick.

The end game

Really, I hope that we can move on from this. The worst possible situation we can wind up with is that the privilege of going out and doing things is revoked from everybody because a small minority (less than 10% of the Queensland population) refuse to do the right thing by everyone else.

I don’t want to be hassled by staff at the door everywhere I go. This will not end if people keep flouting rules! It used to be just hospitality venues where you needed to sign-in, it was done on paper, and life was simple, but then Queensland Health learned that today’s adults can’t write properly. If they mandate proprietary check-in software programs, then those of us who do not have a suitable phone are needlessly excluded from participation in society through no fault of their own.

We will eventually get to the stage where we treat COVID-19 like every other coronavirus out there. The common flu is, after all, a member of that same family, and we never needed check-in programs for that. Some aged-care centres will insist on seeing vaccination certificates, but you could get a coffee without fear of being interrogated. We are not there yet though. We’ve probably got another year of this… so we’re maybe ⅔ of the way through. Please don’t blow it for all of us!

Dec 012021
 

You’d be hard pressed to find a global event that has brought as much pandamonium as this COVID-19 situation has in the last two years. Admittedly, Australia seems to have come out of it better than most nations, but not without our own tortise and hare moment on the vaccination “stroll-out”.

One area where we’re all slowly trying to figure out a way to get along, is in contact tracing, and proving vaccination status.

Now, it’s far from a unique problem. If Denso Wave were charging royalties each time a QR code were created or scanned, they’d be richer than Microsoft, Amazon and Apple put together by now. In the beginning of the pandemic, when a need for effective contact tracing was first proposed, we initially did things on paper.

Evidently though, at least here in Queensland, our education system has proven ineffective at teaching today’s crop of adults how to work a pen, with a sufficient number seemingly being unable to write in a legible manner. And so, the state government here mandated that all records shall be electronic.

Now, this wasn’t too bad, yes a little time-consuming, but by-in-large, most of the check-in systems worked with just your phone’s web browser. Some even worked by SMS, no web browser or fancy check-in software needed. It was a bother if you didn’t have a phone on you (e.g. maybe you don’t like using them, or maybe you can’t for legal reasons), but most of the places where they were enforcing this policy, had staff on hand that could take down your details.

The problems really started much later on when first, the Queensland Government decided that there shall be one software package, theirs. This state was not unique in doing this, each state and territory decided that they cannot pool resources together — wheels must be re-invented!

With restrictions opening up, they’re now making vaccination status a factor in deciding what your restrictions are. Okay, no big issue with this in principle, but once again, someone in Canberra thought that what the country really wanted to do was to spend all evening piss-farting around with getting MyGov and ther local state/territory’s check-in application to talk to each-other.

MyGov itself is its own barrel of WTFs. Never needed to worry about it until now… it took 6 attempts with pass to come up with a password that met its rather loosely defined standards, and don’t get me started on the “wish-it-were two-factor” authentication. I did manage to get an account set-up, and indeed, the COVID-19 certificate is as basic as they come; a PDF genrated using the Eclipse BIRT Report Engine, on what looks to be a Linux machine (or some Unix-like system with a /opt directory anyway). The PDF itself just has the coat-of-arms in the background, and some basic text describing whom the certificate is for, what they got poked with and when. Nothing there that would allow machine-verification whatsoever.

The International version (which I don’t have as I lack a passport), embeds a rather large and complicated QR-code which embeds a JSON data structure (perhaps JOSE? I didn’t check) that seems to be digitally signed with an ECC-based private key. That QR code pushes the limits of what a standard QR code can store, but provided the person scanning it has a copy of the corresponding public key, all the data is there for verification.

The alternative to QRZilla, is rather to make an opaque token, and have that link through to a page with further information. This is, after all, what all the check-in QR codes do anyway. Had MyGov embedded such a token on the certificate, it’d be a trivial matter for the document to be printed out, screen-shotted or opened in, an application that needs to check it, and have that direct whatever check-in application to make an API call to the MyGov site to verify the certificate.

But no, they instead have on the MyGov site in addition to the link that gives you the rather bland PDF, a button that “shares with” the check-in applications. To see this button, you have to be logged in on the mobile device running the check-in application(s). For me, that’s the tablet, as my phone is too old for this check-in app stuff.

When you tap that button, it brings you to a page showing you the smorgasboard of check-in applications you can theoretically share the certificate with. Naturally, “Check-in Queensland” is one of those; tapping it, it takes you to a legal agreement page to which you must accept, and after that, magic is supposed to happen.

As you can gather, magic did not happen. I got this instead.

I at least had the PDF, which I’ve since printed, and stashed, so as far as I’m concerned, I’ve met the requirements. If some business owner wants to be a technical elitist, then they can stick it where it hurts.

In amongst the instructions, it makes two curious points:

  • iOS devices, apparently Safari won’t work, they need you to use Chrome on iOS (which really is just Safari pretending to be Chrome)
  • Samsung’s browser apparently needs to be told to permit opening links in third-party applications

I use Firefox for Android on my tablet as I’m a Netscape user from way-back. I had a look at the settings to see if something could help there, and spotted this:

Turning the Open links in apps option on, I wondered if I could get this link-up to work. So, dug out the password, logged in, navigated to the appropriate page… nada, nothing. They changed the wording on the page, but the end result was the same.

So, I’m no closer than I was; and I think I’ll not bother from here on in.

As it is, I’m thankful I don’t need to go interstate. I’ve got better things to do than to muck around with a computer every time I need to go to the shops! Service NSW had a good idea in that, rather than use their application, you could instead go to a website (perhaps with the aide of someone who had the means), punch in your details, and print out some sort of check-in certificate that the business could then scan. Presumably that same certificate could mention vaccination status.

Why this method of checking-in hasn’t been adopted nation-wide is a mystery to me. Seems ridiculous that each state needs to maintain its own database and software, when all these tools are supposed to be doing the same thing.

In any case, it’s a temporary problem: I for one, will be uninstalling any contact-tracing software at some point next year. Once we’re all mingling out in public, sharing coronaviruses with each-other, and internationally… it’ll be too much of a flood of data for each state’s contact tracers to keep up with everyone’s movements.

I’m happy to just tell my phone, tablet or GPS to record a track-log of where I’ve been, and maybe keep a diary — for the sake of these contact tracers. Not hard when they make an announcement that ${LOCATION} is a contact site; me to check, “have I been to ${LOCATION}?” and get in touch if I have, turning over my diary/track logs for contact tracers to do their work. It’ll probably be more accurate than what all these silly applications can give them anyway.

We need to move on, and move forward.

Oct 072021
 

Recently, I noticed my network monitoring was down… I hadn’t worried about it because I had other things to keep me busy, and thankfully, my network monitoring, whilst important, isn’t mission critical.

I took a look at it today. The symptom was an odd one, influxd was running, it was listening on the back-up/RPC port 8088, but not 8086 for queries.

It otherwise was generating logs as if it were online. What gives?

Tried some different settings, nothing… nada… zilch. Nothing would make it listen to port 8086.

Tried updating to 1.8 (was 1.1), still nothing.

Tried manually running it as root… sure enough, if I waited long enough, it started on its own, and did begin listening on port 8086. Hmmm, I wonder. I had a look at the init scripts:

#!/bin/bash -e

/usr/bin/influxd -config /etc/influxdb/influxdb.conf $INFLUXD_OPTS &
PID=$!
echo $PID > /var/lib/influxdb/influxd.pid

PROTOCOL="http"
BIND_ADDRESS=$(influxd config | grep -A5 "\[http\]" | grep '^  bind-address' | cut -d ' ' -f5 | tr -d '"')
HTTPS_ENABLED_FOUND=$(influxd config | grep "https-enabled = true" | cut -d ' ' -f5)
HTTPS_ENABLED=${HTTPS_ENABLED_FOUND:-"false"}
if [ $HTTPS_ENABLED = "true" ]; then
  HTTPS_CERT=$(influxd config | grep "https-certificate" | cut -d ' ' -f5 | tr -d '"')
  if [ ! -f "${HTTPS_CERT}" ]; then
    echo "${HTTPS_CERT} not found! Exiting..."
    exit 1
  fi
  echo "$HTTPS_CERT found"
  PROTOCOL="https"
fi
HOST=${BIND_ADDRESS%%:*}
HOST=${HOST:-"localhost"}
PORT=${BIND_ADDRESS##*:}

set +e
max_attempts=10
url="$PROTOCOL://$HOST:$PORT/health"
result=$(curl -k -s -o /dev/null $url -w %{http_code})
while [ "$result" != "200" ]; do
  sleep 1
  result=$(curl -k -s -o /dev/null $url -w %{http_code})
  max_attempts=$(($max_attempts-1))
  if [ $max_attempts -le 0 ]; then
    echo "Failed to reach influxdb $PROTOCOL endpoint at $url"
    exit 1
  fi
done
set -e

Ahh right, so start the server, check every second to see if it’s up, and if not, just abort and let systemd restart the whole shebang. Because turning the power on-off-on-off-on-off is going to make it go faster, right?

I changed max_attempts to 360 and the sleep to 10.

Having fixed this, I am now getting data back into my system.

Jun 202021
 

So, today on the radio I heard that from this Friday, our state government was “expanding” the use of their Check-in Queensland program. Now, since my last post on the topic, I have since procured a new tablet. The tablet was purchased for completely unrelated reasons, namely:

  1. to provide navigation assistance, current speed monitoring and positional logging whilst on the bicycle (basically, what my Garmin Rino-650 does)
  2. to act as a media player (basically what my little AGPTek R2 is doing — a device I’ve now outgrown)
  3. to provide a front-end for a SDR receiver I’m working on
  4. run Slack for monitoring operations at work

Since it’s a modern Android device, it happens to be able to run the COVID-19 check-in programs. So I have COVIDSafe and Check-in Queensland installed. For those to work though, I have to run my existing phone’s WiFi hotspot. A little cumbersome, but it works, and I get the best of both worlds: modern Android + my phone’s excellent cell tower reception capability.

The snag though comes when these programs need to access the Internet at times when using my phone is illegal. Queensland laws around mobile phone use changed a while back, long before COVID-19. The upshot was that, while people who hold “open” driver’s licenses may “use” a mobile phone (provided that they do not need to handle it to do so), anybody else may not “use” a phone for “any purpose”. So…

  • using it for talking to people? Banned. Even using “hands-free”? Yep, still banned.
  • using it for GPS navigation? Banned.
  • using it for playing music? Banned.

It’s a $1000 fine if you’re caught. I’m glad I don’t use a wheelchair: such mobility aids are classed as a “vehicle” under the Queensland traffic act, and you can be fined for “drink driving” if you operate one whilst drunk. So traffic laws that apply to “motor vehicles” also apply to non-“motor vehicles”.

I don’t have a driver’s license of any kind, and have no interest in getting one, my primary mode of private transport is by bicycle. I can’t see how I’d be granted permission to do something that someone on a learner’s permit or P1 provisional license is forbidden from doing. The fact that I’m not operating a “motor vehicle” does not save me, the drink-driving in a wheelchair example above tells me that I too, would be fined for riding my bicycle whilst drunk. Likely, the mobile phones apply to me too. Given this, I made the decision to not “use” a mobile phone on the bicycle “for any purpose”. “For any purpose” being anything that requires the device to be powered on.

If I’m going to be spending a few hours at the destination, and in a situation that may permit me to use the phone, I might carry it in the top-box turned off (not certain if this is permitted, but kinda hard to police), but if it’s a quick trip to the shops, I leave the mobile phone at home.

What’s this got to do with the Check-in Queensland application or my new shiny-shiny you ask? Glad you did.

The new tablet is a WiFi-only device… specifically because of the above restrictions on using a “mobile phone”. The day those restrictions get expanded to include the tablet, you can bet the tablet will be ditched when travelling as well. Thus, it receives its Internet connection via a WiFi access point. At home, that’s one of two Cisco APs that provide my home Internet service. No issue there.

If I’m travelling on foot, or as a passenger on someone else’s vehicle, I use the WiFi hot-spot function on my phone to provide this Internet service… but this obviously won’t work if I just ducked up the road on my bike to go get some grocery shopping done, as I leave the phone at home for legal reasons.

Now, the Check-in Queensland application does not work without an Internet connection, and bringing my own in this situation is legally problematic.

I can also think of situations where an Internet connection is likely to be problematic.

  • If your phone doesn’t have a reliable cell tower link, it won’t reliably connect to the Internet, Check-in Queensland will fail.
  • If your phone is on a pre-paid service and you run out of credit, your carrier will deny you an Internet service, Check-in Queensland will fail.
  • If your carrier has a nation-wide whoopsie (Telstra had one a couple of years back, Optus and Vodafone have had them too), you can find yourself with a very pretty but very useless brick in your hand. Check-in Queensland will fail.

What can be done about this?

  1. The venues could provide a WiFi service so people can log in to that, and be provided with limited Internet access to allow the check-in program to work whilst at the venue. I do not see this happening for most places.
  2. The Check-in Queensland application could simply record the QR code it saw, date/time, co-visitors, and simply store it on the device to be uploaded later when the device has a reliable Internet link.
  3. For those who have older phones (and can legally carry them), the requirement of an “application” seems completely unnecessary:
    1. Most devices made post-2010 can run a web browser capable of running an in-browser QR code scanner, and storage of the customer’s details can be achieved either through using window.localStorage or through RFC-6265 HTTP cookies. In the latter case, you’d store the details server-side, and generate an “opaque” token which would be stored on the device as a cookie. A dedicated program is not required to do the function that Check-in Queensland is performing.
    2. For older devices, pretty much anything that can access the 3G network can send and receive SMS messages. (Indeed, most 2G devices can… the only exception I know to this would be the Motorola MicroTAC 5200 which could receive but not send SMSes. The lack of a 2G network will stop you though.) Telephone carriers are required to capture and verify contact details when provisioning pre-paid and post-paid cellular services, so already have a record of “who” has been assigned which telephone number. So why not get people to text the 6-digit code that Check-In Queensland uses, to a dedicated telephone number? If there’s an outbreak, they simply contact the carrier (or the spooks in Canberra) to get the contact details.
  4. The Check-in Queensland application has a “business profile” which can be used for manual entry of a visitor’s details… hypothetically, why not turn this around? Scan a QR code that the visitor carries and provides. Such QR codes could be generated by the Check-in Queensland website, printed out on paper, then cut out to make a business-card sized code which visitors can simply carry in their wallets and present as needed. No mobile phone required! For the record, the Electoral Commission of Queensland has been doing this for our state and council elections for years.

It seems the Queensland Government is doing this fancy “app” thing “because we can”. Whilst I respect the need to effectively contact-trace, the truth is there’s no technical reason why “this” must be the implementation. We just seem to be playing a game of “follow the shepherd”. They keep trying to advertise how “smart” we are, why not prove it?

Apr 232021
 

So, about 10 years ago, I started out as a contractor with a local industrial automation company, helping them integrate energy meters into various energy management systems.

Back then, they had an in-house self-managed corporate email system built on Microsoft Small Business Server. It worked, mostly, but had the annoyance of being a pariah regarding Internet standards… begrudgingly speaking SMTP to the outside world and mangling RFC822 messaging left-right and centre any chance it got. Ohh, and if you didn’t use its sister product, Microsoft Outlook, you weren’t invited!

Thankfully, as a contractor, I was largely insulated from that horror of a mail system… I had my own, running postfix + dovecot. That worked. Flawlessly for my needs. Emails were stored in the Maildir format, so back-ups were easy, if I couldn’t find something over IMAP, a ssh into the server was all I needed to unleash grep on the mailstore. Prior to this, I’ve used various combinations of Sendmail, Qmail, qpsmtpd for MTA and uw-imapd, Binc IMAP and finally dovecot. I used SpamAssassin for mail filtering, configured the server with a variety of RBLs, and generally enjoyed a largely spam-free and easy life.

A year or two into this arrangement, my workplace’s server had a major meltdown… they apparently had hit some internal limit on the Microsoft server, and on receipt of a few messages, it just crashed. Restore from back-up, all good, then some more incoming emails, down she went. In a hurry for an alternative, they grabbed an old box, loaded it up with an Ubuntu server fork and configured Zarafa groupware which sat atop the postfix MTA.

It was chosen because it was feature-wise, similar, to the Microsoft option. Unfortunately, it was also architecturally similar, with the mailstore being stored in MySQL using a bizzare schema that tried to replicate how Microsoft Exchange stored emails… meaning any header that Zarafa didn’t understand, got stripped… and any character that didn’t fit in the mailstore’s LATIN1 table character set got replaced with ?. Yes Mr. ????????? we’ll be onto that support request right away! One thing that I will say in Zarafa’s defence though, is that they at least supported IMAP (even if their implementation was primitive, it mostly “worked”), and calendaring was accessible using CalDAV.

That was the server I inherited as mail server administrator. We kept it going like that for a couple of years, but over time, the growing pains became evident… we had to move… again. By this stage, we were using Thunderbird as our standard email client, the Lightning extension for calendaring. On the fateful weekend of the 13-14th February, 2016, after a few weeks of research and testing, we moved again; to a combination of postfix, dovecot and SoGO providing calendaring/webmail. Like the server I had at home, email was stored in Maildir mail stores, which meant back-ups were as simple as rsync, selective restoring of a mail folder was easy, we could do public folders. People could use any IMAP compatible mail client: Thunderbird, Outlook, mutt, Apple Mail… whatever floated their boat.

I was quite proactive about the spam/malware situation… there was an extensive blacklist I maintained on that server to keep repeat offenders out. If you used a server at OVH or DigitalOcean for example, your email was not welcome, connections to port 25/tcp were rejected. Anything that did get through brought to my attention, I would pass the email through Spamcop for analysis and reporting, and any repeat offenders got added to the blacklist. I’d have liked to improve on the malware scanning… there are virus scanners that will integrate into Postfix and I was willing to set something up, but obviously needed management to purchase something suitable to do that.

Calendaring worked too… about the only thing that was missing was free-busy information, which definitely has its value, but it was workable. Worst case in my opinion is maybe replace SoGO with something else, but for now, it worked.

Fast forward to March 29th this year. New company has bought up my humble abode… and the big wigs have selected… Microsoft! No consultation. No discussion. The first note I got regarding this was a company-wide email stating we’d be migrating over the Easter long week-end.

I emailed back, pointing out a few concerns. I was willing to give Microsoft a second chance. For my end as a end user, I really only care about one thing: that the server communicates with the software on my computer with agreed “standard” protocols. For email that is IMAP and SMTP. For calendaring that is CalDAV. I really don’t care how it’s implemented, so long as it implements it properly. They do their end of the bargain by speaking an agreed protocol correctly… I’ll do my end by selecting a standards-compliant email/calendar client. All good.

I was assured that yes, it would do this. Specifically, I was shown this page as evidence. Okay, I thought, lets see how it goes. Small Business Server was from 2003… surely Microsoft has learned something in 18 years. They’ve been a lot more open about things, adopting support for OpenDocument in Office, working with Novell on .NET, ditching Visual Source Safe and embracing git so much so they acquired Github… surely things have improved.

Tuesday, 6th April, we entered a new world. A world were public folders were gone. A world with no calendaring. I’m guessing the powers at be have decided I do not need to see public folders, after all, RFC2342 has been around since the 90s… and even has people from Microsoft working on it! It’s possible they’re still migrating them from the old server, but 3 weeks seems a stretch.

Fine, I can live without public folders for now. Gone are the days where I interacted with customers on a regular basis and thus needed to file correspondence. The only mail folder I had much to do with of late was a public folder called Junk Mail which I used to monitor for spam to report and train the spam filter with.

Calendaring, I’ll admit I don’t use much… but to date, I have no CalDAV URI to configure my client with. I did some digging this morning. Initial investigations suggest that Microsoft still lives in the past. Best they can offer is a “look-but-not-touch” export. Useless.

But wait, there’s a web client! Yeah great… let’s cram it all in a web browser. I have to deal with Slack and its ugly bloat because voice chat doesn’t work in anything else. Then there’s the thorny of web-based email and why I think that is a bad idea. No, just because a web client works for you, or a particular brand desktop client works for you, does not mean it will work for everybody.

The frustration from this end right now is that I’m trapped with nowhere to go. I’m locked in to supporting myself and Sam (I made a commitment to my dying grandmother that he’d be cared for) for another 10 years at least (who knows how long he’ll live for, he’s 7 now and Emma lived to nearly 18), so suicide isn’t an option right now, nor is simply quitting and living on the savings I have.

Most workplaces seem to be infected with this groupware-malware, so switching isn’t a viable option either. Office365 apparently has a REST API, so maybe that’s the next point of call: see if I can write a proxy to bolt-on such an interface.

Apr 112021
 

So, for the past 12 months we’ve basically had a whirlwind of different “solutions” to the problem of contact tracing. The common theme amongst them seems to be they’re all technical-based, and they all assume people carry a smartphone, registered with one of the two major app stores, and made in the last few years.

Quite simply, if you’re carrying an old 3G brick from 2010, you don’t exist to these “apps”. Our own federal government tried its hand in this space by taking OpenTrace (developed by the Singapore Government and released as GPLv3 open-source) and rebadging that (and re-licensing it!) as COVIDSafe.

This had very mild success to say the least, with contact tracers telling us that this fancy “app” wasn’t telling them anything new. So much focus has been put on signing into and out of venues.

To be honest, I’m fine with this until such time as we get this gift from China under control. The concept is not what irks me, it’s its implementation.

At first, it was done on paper. Good old fashioned pen and paper. Simple, nearly foolproof, didn’t crash, didn’t need credit, didn’t need recharging, didn’t need network coverage… except for two problems:

  1. people who can’t successfully operate a pen (Hmm, what went wrong, Education Queensland?)
  2. people who can’t take the process seriously (and an app solves this how?)

So they demanded that all venues use an electronic system. Fine, so we had a myriad of different electronic web-based systems, a little messy, but it worked, and for the most part, the venue’s system didn’t care what your phone was.

A couple, even could take check-in by SMS. Still rocking a Nokia 3210 from 1998? Assuming you’ve found a 2G cell tower in range, you can still check in. Anything that can do at least 3G will be fine.

An advantage of this solution is that they have your correct mobile phone number then and it’s a simple matter for Queensland Health to talk to Telstra/Optus/Vodaphone/whoever to get your name and address from that… as a bonus, the cell sites may even have logs of your device’s IMEI roaming, so there’s more for the contact tracing kitty.

I only struck one venue out of dozens, whose system would not talk to my phone. Basically some JavaScript library didn’t load, and so it fell in a heap.

Until yesterday.

The Queensland Government has decided to foist its latest effort on everybody, the “Check-in Queensland” app. It is available on Google Play Store and Apple App Store, and their QR codes are useless without it. I can’t speak about the Apple version of the software, but for the Android one, it requires Android 5.0 or above.

Got an old reliable clunker that you keep using because it pulls the weakest signals and has a stand-by time that can be measured in days? Too bad. For me, my Android 4.1 device is not welcome. There are people out there for whom, even that, is a modern device.

Why not buy a newer phone? Well, when I bought this particular phone, back in 2015… I was looking for 3 key features:

  1. Make and receive (voice) telephone calls
  2. Send and receive short text messages
  3. Provide a Internet link for my laptop via USB/WiFi

Anything else is a bonus. It has a passable camera. It can (and does) play music. There’s a functional web browser (Firefox). There’s a selection of software I can download (via F-Droid). It Does What I Need It To Do. The battery still lasts 2-3 days between charges on stand-by. I’ve seen it outperform nearly every contemporary device on the market in areas with weak mobile coverage, and I can connect an external antenna to boost that if needed.

About the only thing I could wish for is open-source firmware and a replaceable battery. (Well, it sort-of is replaceable. Just a lot of frigging around to get at it. I managed to replace a GPS battery, so this should be doable.)

So, given this new check-in requirement, what is someone like me to do? Whilst the Queensland Government is urging people to install their application, they recognise that there are those of us who cannot because we lack anything that will run it. So they ask that venues have a device on hand that can be used to check visitors in if this situation arises.

My little “hack” simply exploits this:

# This file is part of pylabels, a Python library to create PDFs for printing
# labels.
# Copyright (C) 2012, 2013, 2014 Blair Bonnett
#
# pylabels is free software: you can redistribute it and/or modify it under the
# terms of the GNU General Public License as published by the Free Software
# Foundation, either version 3 of the License, or (at your option) any later
# version.
#
# pylabels is distributed in the hope that it will be useful, but WITHOUT ANY
# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
# A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along with
# pylabels.  If not, see <http://www.gnu.org/licenses/>.

import argparse
import labels
import time
from reportlab.lib.units import mm
from reportlab.graphics import shapes
from reportlab.lib import colors
from reportlab.graphics.barcode import qr

rows = 4
cols = 2
# Specifications for Avery C32028 2×4 85×54mm
specs = labels.Specification(210, 297, cols, rows, 85, 54, corner_radius=0,
        left_margin=17, right_margin=17, top_margin=31, bottom_margin=32)

def draw_label(label, width, height, checkin_id):
    label.add(shapes.String(
        42.5*mm, 50*mm,
        'COVID-19 Check-in Card',
        fontName="Helvetica", fontSize=12, textAnchor='middle'
    ))
    label.add(shapes.String(
        42.5*mm, 46*mm,
        'The Queensland Government has chosen to make the',
        fontName="Helvetica", fontSize=8, textAnchor='middle'
    ))
    label.add(shapes.String(
        42.5*mm, 43*mm,
        'CheckIn QLD application incompatible with my device.',
        fontName="Helvetica", fontSize=8, textAnchor='middle'
    ))
    label.add(shapes.String(
        42.5*mm, 40*mm,
        'Please enter my contact details into your system',
        fontName="Helvetica", fontSize=8, textAnchor='middle'
    ))
    label.add(shapes.String(
        42.5*mm, 37*mm,
        'at your convenience.',
        fontName="Helvetica", fontSize=8, textAnchor='middle'
    ))

    label.add(shapes.String(
        5*mm, 32*mm,
        'Name: Joe Citizen',
        fontName="Helvetica", fontSize=12
    ))
    label.add(shapes.String(
        5*mm, 28*mm,
        'Phone: 0432 109 876',
        fontName="Helvetica", fontSize=12
    ))
    label.add(shapes.String(
        5*mm, 24*mm,
        'Email address:',
        fontName="Helvetica", fontSize=12
    ))
    label.add(shapes.String(
        84*mm, 20*mm,
        'myaddress+c%o@example.com' % checkin_id,
        fontName="Courier", fontSize=12, textAnchor='end'
    ))
    label.add(shapes.String(
        5*mm, 16*mm,
        'Home address:',
        fontName="Helvetica", fontSize=12
    ))
    label.add(shapes.String(
        15*mm, 12*mm,
        '12 SomeDusty Rd',
        fontName="Helvetica", fontSize=12
    ))
    label.add(shapes.String(
        15*mm, 8*mm,
        'BORING SUBURB, QLD, 4321',
        fontName="Helvetica", fontSize=12
    ))

    label.add(shapes.String(
        2, 2, 'Date: ',
        fontName="Helvetica", fontSize=10
    ))
    label.add(shapes.Rect(
        10*mm, 2, 12*mm, 4*mm,
        fillColor=colors.white, strokeColor=colors.gray
    ))
    label.add(shapes.String(
        22.5*mm, 2, '-', fontName="Helvetica", fontSize=10
    ))
    label.add(shapes.Rect(
        24*mm, 2, 6*mm, 4*mm,
        fillColor=colors.white, strokeColor=colors.gray
    ))
    label.add(shapes.String(
        30.5*mm, 2, '-', fontName="Helvetica", fontSize=10
    ))
    label.add(shapes.Rect(
        32*mm, 2, 6*mm, 4*mm,
        fillColor=colors.white, strokeColor=colors.gray
    ))
    label.add(shapes.String(
        40*mm, 2, 'Time: ',
        fontName="Helvetica", fontSize=10
    ))
    label.add(shapes.Rect(
        50*mm, 2, 6*mm, 4*mm,
        fillColor=colors.white, strokeColor=colors.gray
    ))
    label.add(shapes.String(
        56.5*mm, 2, ':', fontName="Helvetica", fontSize=10
    ))
    label.add(shapes.Rect(
        58*mm, 2, 6*mm, 4*mm,
        fillColor=colors.white, strokeColor=colors.gray
    ))

    label.add(shapes.String(
        10*mm, 5*mm, 'Year',
        fontName="Helvetica", fontSize=6, fillColor=colors.gray
    ))
    label.add(shapes.String(
        24*mm, 5*mm, 'Month',
        fontName="Helvetica", fontSize=6, fillColor=colors.gray
    ))
    label.add(shapes.String(
        32*mm, 5*mm, 'Day',
        fontName="Helvetica", fontSize=6, fillColor=colors.gray
    ))
    label.add(shapes.String(
        50*mm, 5*mm, 'Hour',
        fontName="Helvetica", fontSize=6, fillColor=colors.gray
    ))
    label.add(shapes.String(
        58*mm, 5*mm, 'Minute',
        fontName="Helvetica", fontSize=6, fillColor=colors.gray
    ))

    label.add(qr.QrCodeWidget(
            '%o' % checkin_id,
            barHeight=12*mm, barWidth=12*mm, barBorder=1,
            x=73*mm, y=0
    ))

# Grab the arguments
OCTAL_T = lambda x : int(x, 8)
parser = argparse.ArgumentParser()
parser.add_argument(
        '--base', type=OCTAL_T,
        default=(int(time.time() / 86400.0) << 8)
)
parser.add_argument('--offset', type=OCTAL_T, default=0)
parser.add_argument('pages', type=int, default=1)
args = parser.parse_args()

# Figure out cards per sheet (max of 256 cards per day)
cards = min(rows * cols * args.pages, 256)

# Figure out check-in IDs
start_id = args.base + args.offset
end_id = start_id + cards
print ('Generating cards from %o to %o' % (start_id, end_id))

# Create the sheet.
sheet = labels.Sheet(specs, draw_label, border=True)

sheet.add_labels(range(start_id, end_id))

# Save the file and we are done.
sheet.save('checkin-cards.pdf')
print("{0:d} cards(s) output on {1:d} page(s).".format(sheet.label_count, sheet.page_count))

That script (which may look familiar), generates up to 256 check-in cards. The check-in cards are business card sized and look like this:

That card has:

  1. the person’s full name
  2. a contact telephone number
  3. an email address with a unique sub-address component for verification purposes (compatible with services that use + for sub-addressing like Gmail)
  4. home address
  5. date and time of check-in (using ISO-8601 date format)
  6. a QR code containing a “check-in number” (which also appears in the email sub-address)

Each card has a unique check-in number (seen above in the email address and as the content of the QR code) which is derived from the number of days since 1st January 1970 and a 8-bit sequence number; so we can generate up to 256 cards a day. The number is just meant to be unique to the person generating them, two people using this script can, and likely will, generate cards with the same check-in ID.

I actually added the QR code after I printed off a batch (thought of the idea too late). Maybe the next batch will have the QR code. This can be used with a phone app of your choosing (e.g. maybe use BarcodeScanner to copy the check-in number to the clip-board then paste it into a spreadsheet, or make your own tool) to add other data. In my case, I’ll use a paper system:

The script that generates those is here:

# This file is part of pylabels, a Python library to create PDFs for printing
# labels.
# Copyright (C) 2012, 2013, 2014 Blair Bonnett
#
# pylabels is free software: you can redistribute it and/or modify it under the
# terms of the GNU General Public License as published by the Free Software
# Foundation, either version 3 of the License, or (at your option) any later
# version.
#
# pylabels is distributed in the hope that it will be useful, but WITHOUT ANY
# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
# A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along with
# pylabels.  If not, see <http://www.gnu.org/licenses/>.

import argparse
import labels
import time
from reportlab.lib.units import mm
from reportlab.graphics import shapes
from reportlab.lib import colors

rows = 4
cols = 2
# Specifications for Avery C32028 2×4 85×54mm
specs = labels.Specification(210, 297, cols, rows, 85, 54, corner_radius=0,
        left_margin=17, right_margin=17, top_margin=31, bottom_margin=32)

def draw_label(label, width, height, checkin_id):
    label.add(shapes.String(
        42.5*mm, 50*mm,
        'COVID-19 Check-in Log',
        fontName="Helvetica", fontSize=12, textAnchor='middle'
    ))

    label.add(shapes.Rect(
        1*mm, 3*mm, 20*mm, 45*mm,
        fillColor=colors.lightgrey,
        strokeColor=None
    ))
    label.add(shapes.Rect(
        41*mm, 3*mm, 28*mm, 45*mm,
        fillColor=colors.lightgrey,
        strokeColor=None
    ))

    for row in range(3, 49, 5):
        label.add(shapes.Line(1*mm, row*mm, 84*mm, row*mm, strokeWidth=0.5))
    for col in (1, 21, 41, 69, 84):
        label.add(shapes.Line(col*mm, 48*mm, col*mm, 3*mm, strokeWidth=0.5))

    label.add(shapes.String(
        2*mm, 44*mm,
        'In',
        fontName="Helvetica", fontSize=8
    ))

    label.add(shapes.String(
        22*mm, 44*mm,
        'Check-In #',
        fontName="Helvetica", fontSize=8
    ))

    label.add(shapes.String(
        42*mm, 44*mm,
        'Place',
        fontName="Helvetica", fontSize=8
    ))

    label.add(shapes.String(
        83*mm, 44*mm,
        'Out',
        fontName="Helvetica", fontSize=8, textAnchor='end'
    ))

# Grab the arguments
parser = argparse.ArgumentParser()
parser.add_argument('pages', type=int, default=1)
args = parser.parse_args()

cards = rows * cols * args.pages

# Create the sheet.
sheet = labels.Sheet(specs, draw_label, border=True)

sheet.add_labels(range(cards))

# Save the file and we are done.
sheet.save('checkin-log-cards.pdf')
print("{0:d} cards(s) output on {1:d} page(s).".format(sheet.label_count, sheet.page_count))

When I see one of these Check-in Queensland QR codes, I simply pull out the log card, a blank check-in card, and a pen. I write the check-in number from the blank card (visible in the email address) in my log with the date/time, place, and on the blank card, write the same date/time and hand that to the person collecting the details.

They can write that into their device at their leisure, and it saves time not having to spell it all out. As for me, I just have to remember to write the exit time. If Queensland Health come a ringing, I have a record of where I’ve been on hand… or if I receive an email, I can use the check-in number to validate that this is legitimate, or even tell if a venue has on-sold my personal details to an advertiser.

I guess it’d be nice if the Queensland Government could at least add a form to their fancy pages that their flashy QR codes send people to, so that those who do not have the application can still at least check-in without it, but that’d be too much to ask.

In the meantime, this at least meets them half-way, and hopefully does so which ensures minimal contact and increases efficiency.

Dec 162020
 

Well, this month has been a funny one. When we moved to the NBN back in March, we went from having a 500GB a month quota, to a 100GB a month, with a link speed of 50Mbps.

That seemed, at the time, like a reasonable compromise, since much of the time, my typical usage has been around 60~70GB a month. There’s no Netflicks subscriptions here, but my father does hit YouTube rather hard, and I lately have been downloading music (legally) from time to time.

This year has also seen me working from home, and doing a lot of Slack and Zoom calls. Zoom in particular, is pricey quota-wise, since everyone insists on running webcams. Despite this, the extra Internet use has been manageable. Couple of times we got around 90GB, maybe sailing close to the 100GB, but never over. This is what it looked like last month:

November’s Internet quota usage

This month, that changed:

Internet usage this month

Now, the start of the month data got missed because of a glitch between collectd and the Internode quota monitoring script I have. Two of the spikes can be attributed to:

  • the arrival of a Windows 10-based laptop doing its out-of-box updates (~4GB)
  • my desktop doing its 3-monthly OS updates (~5GB)

That isn’t enough to account for why things have nearly doubled though. A few prospects were in my mind:

  • a web-based script going haywire in a browser (this has happened, and cost me dearly, before)
  • genuine local user Internet activity increases
  • website traffic increases
  • server or workstation compromise

Looking over the netflow data

Now, last time I had this happen, I did two things:

  • I set up collectd/influxdb/Grafana to be able to monitor my Internet usage and quota
  • I set up nfcapd on the border router to monitor my usage

This is pretty easy to set up in OpenBSD, and well worth doing.

I keep about 30 days’ worth of netflow data on the border router. So naturally, I haul that back to my workstation and run nfdump over it to see what jumps out.

Looking through the list of “flows”, one target identified was a development machine hosted at Vultr… checking the IP address, revealed it was one of the WideSky test instances my workplace uses, about 5GB of HTTP requests and about 4GB of VPN traffic — admittedly the couple of WideSky hubs I have here have the logging settings cranked high.

That though doesn’t explain it. The bulk of the traffic was scattered amongst a number of hosts. I didn’t see it until I tried aggregating it by /16 subnet:

RC=0 stuartl@rikishi /tmp $ nfdump -R /tmp/nfcapd -A srcip,dstip -o long6 -O bytes 'net 114.119.0.0/16'  
Date first seen          Duration Proto                             Src IP Addr:Port                                 Dst IP Addr:Port     Flags Tos  Packets    Bytes Flows
2020-11-27 23:11:30.000 1630599.000 0                             150.101.176.226:0     ->                         114.119.146.185:0     ........   0    4.7 M    6.8 G  2535
2020-11-22 13:02:41.000 2099541.000 0                             150.101.176.226:0     ->                         114.119.133.234:0     ........   0    4.3 M    6.1 G  2376
2020-11-18 14:38:42.000 2439079.000 0                             150.101.176.226:0     ->                         114.119.140.107:0     ........   0    3.8 M    5.4 G  2418
2020-11-20 10:43:58.000 2280070.000 0                             150.101.176.226:0     ->                          114.119.141.52:0     ........   0    3.7 M    5.3 G  2421
2020-11-21 22:34:35.000 2151244.000 0                             150.101.176.226:0     ->                         114.119.159.109:0     ........   0    3.4 M    4.9 G  2446
2020-11-24 00:11:52.000 1972657.000 0                             150.101.176.226:0     ->                          114.119.136.13:0     ........   0    3.4 M    4.8 G  2399
2020-11-25 04:24:32.000 1870854.000 0                             150.101.176.226:0     ->                         114.119.136.215:0     ........   0    3.3 M    4.8 G  2473
2020-11-24 15:49:55.000 1916848.000 0                             150.101.176.226:0     ->                           114.119.151.0:0     ........   0    3.0 M    4.4 G  2435
2020-11-27 20:15:43.000 1641316.000 0                             150.101.176.226:0     ->                         114.119.129.181:0     ........   0    2.6 M    3.7 G  2426
2020-11-27 21:38:37.000 1636635.000 0                             150.101.176.226:0     ->                          114.119.159.16:0     ........   0    2.5 M    3.6 G  2419
2020-11-27 23:11:30.000 1630599.000 0                             114.119.146.185:0     ->                         150.101.176.226:0     ........   0    4.1 M  175.9 M  2535
…
2020-11-19 22:02:04.000     0.000 0                             150.101.176.226:0     ->                         114.119.138.111:0     ........   0        3      132     1
2020-11-25 03:37:11.000     0.000 0                             150.101.176.226:0     ->                          114.119.152.27:0     ........   0        3      132     1
2020-12-06 19:59:49.000     0.000 0                             150.101.176.226:0     ->                         114.119.151.153:0     ........   0        3      132     1
2020-11-22 08:23:11.000     0.000 0                             150.101.176.226:0     ->                          114.119.130.23:0     ........   0        3      132     1
2020-11-25 15:43:47.000     0.000 0                             150.101.176.226:0     ->                         114.119.128.219:0     ........   0        3      132     1
2020-11-24 09:05:13.000     0.000 0                             150.101.176.226:0     ->                          114.119.140.85:0     ........   0        3      132     1
Summary: total flows: 56059, total bytes: 51.7 G, total packets: 65.0 M, avg bps: 150213, avg pps: 23, avg bpp: 794
Time window: 2020-11-13 11:01:52 - 2020-12-16 20:19:41
Total flows processed: 39077053, Blocks skipped: 0, Bytes read: 2698309352
Sys: 3.744s flows/second: 10436251.9 Wall: 15.108s flows/second: 2586482.6 

51.7GB in a month!!! Drilling further, I noted it was mostly targeted at TCP ports 80 and 443, and UDP port 53. Web traffic, in other words. Reverse look-up on a randomly selected IP showed the reverse pointer petalbot-xxx-xxx-xxx-xxx.aspiegel.com, and indeed, in server logs for various sites I host, I saw PetalBot in the user agent.

Plucking some petals off PetalBot

So, I needed to put the brakes on this somehow. I’m fine with them indexing my site, just they should have some consideration and restraint about how quickly they do so.

Thus, I amended pf.conf:

# Rate-limited "friends"
ratelimit_dst4="{ 114.119.0.0/16 }"
#ratelimit_dst6="{ }"

# Traffic shaping queues
queue root on $external  bandwidth 25M max 25M
queue slow parent root   bandwidth 256K max 512K
queue bulk parent root   bandwidth 25M default

# …

# Rate-limit certain targets
pass out on egress proto { tcp, udp, icmp } from any to $ratelimit_dst4 modulate state (pflow) set queue slow
#pass out on egress proto { tcp, udp, icmp6 } from any to $ratelimit_dst6 modulate state (pflow) set queue slow

So, the first line defines the root queue on my external interface, and sets the upload bandwidth for 25Mbps (next month, I will be dropping my speed to 25Mbps in favour of an “unlimited” quota).

Then, I define a queue which is restricted to 256kbps (peak 512kbps), and define all traffic going to a specific list of networks, to use that queue. PetalBot should now see a mere 512kbps at most from this end, which should severely crimp how quickly it can guzzle my quota, whilst still permitting it to index my site.

Yesterday, PetalBot chewed through 8GB… let’s see what it does tomorrow.