January 2022

Adventures with UniFi controllers and APs

We’ve had WiFi in one form or another for some years on this network. Originally it started with an interest in the Brisbane Mesh metropolitan area network which more-or-less imploded around 2006 or so. Back then, I think I had one of the few WiFi access points in The Gap. 2.4GHz was basically microwave ovens and not much else. The same is not true today.

WiFi networks in my local area, 2.4GHz isn’t as quiet as it once was.

Since then, the network has changed a bit: from a little D-Link 802.11b AP, we moved to a Prism54g WiFi card (that I still have) with hostapd, using OpenVPN to provide security. That got replaced by a Telstra-branded Netcomm WiFi router which I figured out supported WPA-Enterprise, so I went down the rabbit hole of setting up FreeRADIUS, and we ran that until a lightning strike blew it up. The next consumer AP that replaced it was a miserable failure, so it’s been business APs since then.

Initially a Cisco WAP4410N, which was a great little AP… worked reliably for years, but about 12 months ago I noticed it was dropping packets occasionally and getting a bit intermittent. Thinking that maybe the device is past its prime, I bought a replacement: a WAP150, which proved to be a bit disappointing. Range wasn’t as good compared to the WAP4410N, and I soon found myself moving the WAP150 downstairs to service the network there and re-instating the WAP4410N.

In particular, one feature I liked about the two Cisco units is they support 802.1Q VLANs, with the ability to assign a different WiFi SSID to each. The 4410N could do 4 SSIDs, the 150 8. This is a feature that consumer APs don’t do, and it is a handy feature here as it enables me to have a “work” LAN (with VPN to my workplace) and a “home” LAN which everybody else uses.

Years ago, our Internet usage was over a 512kbps/128kbps ADSL link, and it was mostly Internet browsing… so intermittent packet loss wasn’t a big deal… one AP did just fine. Now with the move to NBN, our telephone service is a VoIP service, and I’m finding that WiFi IP phones are very picky about APs. We have three IP phones and an ATA… the ATA (Grandstream HT814) is Ethernet of course, as is one of the IP phones (Grandstream GXP1615), but the other two IP phones are WiFi (Aristel Wi-Fi Genius X1+ and Grandstream WP810).

The Aristel device in particular, was really choppy… and the first one sent out seemed to be a DoA, with poor performance even when right beside the AP. A replacement was provided under RMA, and this one performed much better, but still suffered intermittent loss. The Grandstream WP810 in general worked, but there were noticeable dead spots in a few areas around the house.

The final straw with the existing pair of APs came at the last Brisbane WICEN meeting, conducted over Zoom… both APs seem to suffer a problem where they started dropping packets and glitching badly. A power-cycle “fixes” the problem, but the issue returns after a week or two. Clearly, they were no longer up to snuff.

The replacements

APs

I procured the following replacements:

I went the long-range one for upstairs since it’s in a high spot (sitting atop a stereo speaker on a top shelf in my room) so would be able to “radiate” over a long distance to hopefully reach down the drive way and into the back-yard.

The other one is to fill in dead spots downstairs, and since it’s going to be pretty much sitting at waist level, there’s no point in it being “long range”.

The devices I bought were purchased through mWave (here and here), as they had them in stock at the time.

Power injectors

These are 48V passive PoE devices… so to make them go, you need a separate power injector. The “standard” Ubiquiti power injector was out-of-stock, but I wanted these to work on 12V anyway, so I looked around for a suitable option. Core Electronics do have some step-up converters which work great for 24V devices, but the range available doesn’t quite reach 48V. I did find though that Telco Antennas sell these 48V PoE injectors. (They also sell the APs here and here, but were out-of-stock at the time of purchase.)

Admittedly, they’re 10/100Mbps only, which means you don’t quite get the full throughput out of the WiFi6 APs, but meh, it’s good enough… if the IP phones need more than 100Mbps, they’ll run up against the 25Mbps limit of the NBN link!

Controller

These APs, unlike the Cisco devices they’re replacing (and everything else I’ve used prior), these have no built-in management interface, they talk to a network controller device… normally the UniFi Cloud Key. I had a run-in with the first generation of these at the Stirling’s Crossing Endurance Centre. For a big network, the idea of a central device does make a lot of sense (that site has 5 UAP-AC-Ms and 3 8-port PoE switches), but for a two-AP network like mine it seemed overkill.

One thing I learned, is these things positively DO NOT like being power-cycled! Repeated power-cycling corrupts the database in very short order, and you find yourself restoring configurations from a back-up soon after. So I was squeamish about buying one of these. The second generation version has its own back-up battery, but reports suggest they can be just as unreliable. In any case, they were out of stock everywhere, and I didn’t want to spring the extra cash for the “plus” model (that has a HDD… not much use to me) or the Dream Machine router.

I did consider using a Raspberry Pi 3, in fact that was my original plan… I had one spare, and so started down the path of setting it up as a UniFi controller… however, ran into two road blocks:

  • UniFi controller at this time requires Java v8… Debian Bullseye ships with v11 minimum
  • UniFi controller needs MongoDB 3.4, which isn’t available on Debian Bullsye on ARM64

I could compile MongoDB, but Java is a whole other issue, and lots of people have complained loudly about this very limitation. If there was one big gripe I’ve got, this would be it.

I did some further research: Ubuntu 20.04 does offer a Java 8 runtime, and on AMD64, I can use existing binaries for MongoDB. I looked around and purchased this small-form-factor PC. Windows 10 went bye byes once I managed to hit F1 at the right point in the BIOS set-up, and Ubuntu 20.04 was PXE-loaded. I could then follow the standard instructions to install via APT. The controller seems to be working fine using OpenJDK JRE v8. I’d recommend this over the licensing quagmire that is using Oracle JRE.

Installation

With a controller, and all the requisite bits, things went smoothly. I found at first, the controller insisted on using 192.168.1.0/24 addresses to talk to the APs… so wound up setting that up in the netplan config. I later found that the UniFi controller won’t let you set a network subnet address unless you turn off Auto Scale Network.

Setting the network subnet is not possible until “Auto Scale Network” is disabled.

So maybe from here-on-in, new APs will appear in the correct subnet, but to be honest, it’s no big deal either way, unless an AP has an untimely end, I shouldn’t need to buy new ones for a while!

Auto-negotiation quirks with Cisco switches

One oddity I noticed was the upstairs (U6 LR) AP was reluctant to communicate via Ethernet, instead funnelling its traffic via the downstairs AP. While it’s handy they can do that, means I don’t necessarily need to worry about powering the upstairs switches in a power outage, the AP should be able to use its Ethernet back-end.

The downstairs one was having no problems, and the set-up was similar: switch port → PoE injector → AP, via short cables. I tried a few different cables with no change. Logged into the switch and had a look, it was set to auto-negotiate, which was working fine downstairs. The downstairs switch is a Netgear GS748T, whereas the one upstairs is a Cisco SG200-08 (not the P version that does PoE).

I found I could log into the AP over SSH (you can provide your SSH key via the UniFi controller)… so I logged in as root and had a look around. They run Linux with (a sadly tainted due to ubnthal.ko and ubnt_common.ko) kernel 4.4, and a Busybox/musl environment with an ARM64 CPU. (Well, the U6 LRs are ARM64, the U6 Lites are MediaTek MT7621s… mipsel32r2 with kernel 5.4.0 and not tainted.) ip told me that eth0 was up, and that the AP’s IP address was assigned to br0 which was also up. brctl told me that eth0 was enslaved by br0. Curiously, /sys/class/net/eth0/carrier was reporting 1, which disagreed with what the switch was telling me.

On a hunch, I tried turning off auto-negotiation, forcing instead 100Mbps full-duplex. Bingo, a link LED appeared. The topology showed the AP was now wired, not talking via downstairs.

Network topology shown in the UniFi Controller UI

Switched back to auto-negotiation, and the AP switched to being a wireless extender with the link LED disappearing from the switch. This may be a quirk of the PoE injectors I’m using, which do not handle 100Mbps, and maybe the switch hasn’t realised this because the AP otherwise “advertises” 1Gbps link capability. For now, I’m leaving that switch port locked at 100Mbps full-duplex. If you have problems with an AP showing up via Ethernet, here’s a place that is worth checking.

Experimentation with cycle clothing: 8 years later

Well, some might recall a few years ago I was trying ideas for cycle clothing, and later followed up with some findings.

My situation has changed a bit… the death of a former work colleague shook me up quite a bit, and while I have been riding, I haven’t been doing it nearly as much. Then, COVID-19 reared its ugly head.

Suffice to say, my commute is now one side of the bedroom to the other. Right at this moment, I’m in self-imposed lockdown until I can get my booster shot: I had my second AstraZenica shot on the 4th November, and the Queensland Government has moved the booster shots to being 3 months after the second shot, so for me, that means I’m due on the 4th February. I’m already booked in with a local chemist here in The Gap, I did that weeks ago so that the appointment would be nailed to the floor, and thus currently I’m doing everything in my power to ensure that appointment goes ahead on-time.

I haven’t been on the bike much at all. That doesn’t mean though that I stop thinking about how I can make my ride more comfortable.

Castle Clothing Coveralls

Yes, I’m the one clad in yellow far left.

They had quite few positives:

  • They were great in wet weather
  • They were great in ambient temperatures below 20°C
  • The pocket was handy for storing keys/a phone/a wallet
  • They had good visibility day and night
  • They keep the wind out well. (On the Main Range, Threadbo Top Station was reporting 87km/hr wind gusts that day.)

But, they weren’t without their issues:

  • They’re (unsurprisingly) no good on a sunny summer’s day (on the day that photo was taken, it was borderline too hot, weather prediction was for showers and those didn’t happen)
  • They’re knackered after about 30 washes or so: the outer waterproof layer peels off the lining
  • In intermittent rain / sunshine, they’d keep you dry during the rainy bit, but when the sun came out, you’d get steamed

To cap it off, they’re no longer being manufactured. Castle Clothing have basically canned them. They’ve got a plain yellow version with no stripes, but otherwise, nothing like their old product. I wound up buying 4 of them in the end… the first two had to be chucked because of the aforementioned peeling problem, the other two are in good condition now, but eventually they’ll need replacement.

Mammoth Workwear do have some alternatives. The “Supertouch” ones I have tried, they’re even shorter lived than the Castle ones, and feel like wearing a plastic bag. The others are either not night-time visible, or they’re lined for winter use.

So, back to research again.

Zentai suits?

Now, I know I’ve said previously I’m no MAMIL… and for the most part I stand by this. I did try wearing a stinger suit on the bike once… on the plus side they are very breathable, so quite comfortable to ride in. BUT, three negatives with stinger suits:

That got me thinking, what’s the difference between a stinger suit and an open-face zentai suit? Not a lot. The zentai suit, if it has gloves, can be bought as a “mitten” or (more commonly) a proper multi-finger glove version. They come in a lot more colours than a stinger suit does. They’re about the same price. And there’s no logos, just plain colours (or you can do various patterns/designs if that’s your thing).

A downside is that the zipper is at the back, which means answering calls from nature is more difficult. But then again, some stinger suits and most wetsuits also feature a back-entry.

I’ve got two coming to try the idea out. I suspect they’ll get worn over other clothing, I’ll just duck into a loo, take my shirt off, put the zentai suit on, then jump on the bike to ride to my destination… that way my shirt isn’t soaked with sweat. We’ll see.

One is a black one, which was primarily bought to replace one of the stinger suits for swimming activities, but I can also evaluate the fabric too (it is the usual lycra material).

The other is a silver one (thus a lycra/latex blend), to try out the visibility — it’ll be interesting to see whether it’s somewhat water-repellent due to the latex mix in the material, and see what effect this has on sweat.

Both of these are open-face! You should never try swimming with a full-face zentai suit. I can’t imagine getting caught in the rain ending well either, and the ability to see where you’re going is paramount when operating any vehicle (especially a bicycle)!

They’ll turn up in a week or two, I can try them out then. Maybe won’t be the final solution, but it may answer a few questions.

Heavy Wet/Cold weather gear

So, with the lighter-weight class out of the way, that turns my attention to what to do in truly foul weather, or just bitterly cold weather.

Now, let me define the latter: low single digits °C. Possibly with a westerly breeze carrying it. For some reading this, this will feel like a hot summer’s day, but for those of us in Brisbane, temperatures this low are what we see in the middle of winter.

The waterproof overalls I was wearing before worked well in dry-but-cold weather, however I did note my hands copped the cold… I needed gloves. The ends of the legs also could get tangled with the chain if I wasn’t careful, and my shoes would still get wet. Riggers boots work okay for this, but they’re hard to come by.

I happened to stumble on Sujuvat ratkaisut Oy, who do specialist wet-weather clothing meant for Europe. Meeko (who runs the site) has a commercial relationship with a few manufacturers, notably AJGroup who supply the material for a lot of Meeko’s “extreme” range.

The suits are a variant of PVC, which will mean they’re less breathable than what I have now, but should also mean they’re a lot more durable. There’s a decent range of colours available, with many options having the possibility of reflective bands, attached gloves and attached wellington boots. It’s worth noting the BikeSuit (no longer available) I was looking at 8 years ago was also a PVC outfit.

In the winter time, the big problem is not so much sweat, but rather, sweat being hit by wind-chill. Thus I’m ordering one of the Extreme Drainage Coveralls to try them out.

I’ve seen something similar out of AliExpress, however the options there are often built for the Chinese market… so rarely feature size options that fit someone like myself. Most of the Chinese ones are dark colours, with one “tan”-coloured option listed, and a couple of rubber ones that were lighter colours (a dark “pink”, and a yellow). Some of the rubber ones also had a strange opening arrangement: a tube opening in the stomach, which you pulled yourself through, then clamped shut with a peg. Innovative, but looks very untidy and just begging to get caught in something! I’ll stick with something a bit more conventional.

The coverall I’m ordering will be a 500g/m² white fabric… so about twice the weight of my current Castle workwear overalls (which are about 330g/m²), and will have the gloves and boots attached. I’m curious to see how that’s done up close, and see how it works out in my use case.

Being a white rather than a yellow/orange will make them less visible in the day time, but I suspect this won’t be much of an issue as it’s night-time visibility I’m particularly after. Also, being white instead of a “strong” fluro colour will likely be better at horse endurance rides, as horses tend to react to fluro colours.

The zip arrangement intrigues me as well… it’s been placed up high so that you can pretty much wade into water up to your chest and not get wet. There’s a lighter-weight option of the same suit, however with fewer options for colours. If the extreme version doesn’t work out for cycling, I might look at this alternative (the bike doesn’t react to strong colours like a horse does).

There’s about a 2-month lead-time on this gear because it’s made-to-order, a reasonable trade-off given you get to more-or-less get it made exactly how you want it. Looking around, I’m seeing off-the-shelf not-customisable outfits at AU$400 a pop, €160 (~AU$252) is looking a good option.

The fact that this is being run as a small side-hustle is commendable. I look forward to seeing the product.

Demise of classic hardware

So, this year I had a new-year’s resolution of sorts… when we first started this “work from home” journey due to China’s “gift”, I just temporarily set up on the dinner table, which was of course, meant to be another few months.

Well, nearly 2 years later, we’re still working from home, and work has expanded to the point that a move to the office, on any permanent basis, is pretty much impossible now unless the business moves to a bigger building. With this in mind, I decided I’d clear off the dinner table, and clean up my room sufficiently to set up my workstation in there.

That meant re-arranging some things, but for the most part, I had the space already. So some stuff I wasn’t using got thrown into boxes to be moved into the garage. My CD collection similarly got moved into the garage (I have it on the computer, but need to retain the physical discs as they represent my “personal use license”), and lo and behold, I could set up my workstation.

The new workspace

One of my colleagues spotted the Indy and commented about the old classic SGI logo. Some might notice there’s also an O2 lurking in the shadows. Those who have known me for a while, will remember I did help maintain a Linux distribution for these classic machines, among others, and had a reasonable collection of my own:

My Indy, O2 and Indigo2 R10000
The Octane, booting up

These machines were all eBay purchases, as is the Sun monitor pictured (it came with the Octane). Sadly, fast forward a number of years, and these machines are mostly door stops and paperweights now.

The Octane’s and Indigo2’s demises

The Octane died when I tried to clean it out with a vacuum cleaner, without realising the effect of static electricity generated by the vacuum cleaner itself. I might add mine was a particularly old unit: it had a 175MHz R10000 CPU, and I remember the Linux kernel didn’t recognise the power management circuitry in the PSU without me manually patching it.

The Indigo2 mysteriously stopped working without any clear reason why, I’ve never actually tried to diagnose the issue.

That left the Indy and the O2 as working machines. I haven’t fired them up in a long time until today. I figured, what the hell, do they still run?

Trying the Indy and O2 out

Plug in the Indy, hit the power button… nothing. Dead as a doornail. Okay, put it aside… what about the O2?

I plug it in, shuffle it closer to the monitor so I can connect it. ‘Lo and behold:

The O2 lives!

Of course, the machine was set up to use a serial console as its primary interface, and boot-up was running very slow.

Booting up… very slowly…

It sat there like that for a while, figuring the action was happening on a serial port, I went to go get my null modem cable, only to find a log-in prompt by the time I got back.

Next was remembering what password I was using when I last ran this machine. We had the OpenSSL heartbleed vulnerability happen since then, and at about that time, I revoked all OpenPGP keys and changed all passwords, so it isn’t what I use today. I couldn’t get in as root, but my regular user account worked, and I was able to change the root password via sudo.

Remembering my old log-in credentials, from 22 years ago it seems

The machine soon crashed after that. I tried rebooting, this time I tweaked some PROM settings (and yes, I was rusty remembering how to do it) to be able to see what was going. (I had the null modem cable in hand, but didn’t feel like trying to blindly find the serial port at the back of my desktop.)

Changing PROM settings
The subsequent boot, and crash

Evidently, I had a dud disk. This did not surprise me in the slightest. I also noticed the PSU fan was not spinning, possibly seized after years of non-use.

Okay, there were two disks installed in this machine, both 80-pin SCA SCSI drives. Which one was it? I took a punt and tried the one furtherest away from the I/O ports.

Success, she boots now

I managed to reset the root password, before the machine powered itself off (possibly because of overheating). I suspect the machine will need the dust blown out of it (safely! — not using the method that killed the Octane!), and the HDDs will need replacements. The guilty culprit was this one (which I guessed correctly first go):

a 4GB HDD was a big drive back in 1998!

The computer I’m typing this on has a HDD that stores 1000 of these drives. Today, there are modern alternatives, such as SCSI2SD that could get this machine running fully if needed. The tricky bit would be handling the 80-pin hot-swap interface. There’d be some hardware hacking needed to connect the two, but AU$145 plus an adaptor seems like a safer bet than trying some random used HDD.

So, replacement for the HDDs, a clean-out, and possibly a new fan or two, and that machine will be back to “working” state. Of course the Linux landscape has moved on since then, Debian no longer support the MIPS4 ISA that the RM5200 CPU understands, Gentoo still could work on this though, and maybe OpenBSD still support this too. In short, this machine is enough of a “go-er” that it should not be sent to land-fill… yet.

Turning my attention back to the Indy

So the Indy was my first SGI machine. Bought to better understand the MIPS processor architecture, and perhaps gain enough understanding to try and breathe life into a Cobalt Qube II server appliance (remember those?), it did teach me a lot about Linux and how things vary between platforms.

I figured I might as well pop the cover and see if there’s anything “obviously” wrong. The procedure I was rusty on, but I recalled there was a little catch on the back of the case that needed to be release before the cover slid off. So I lugged the 20″ CRT off the top of the machine, pulled the non-functioning Indy out, and put it on the table to inspect further.

Upon trying to pop the cover (gently!), the top of the case just exploded. Two pieces of the top cover go flying, and the RF shield parts company with the cover’s underside.

The RF shield parted company with the underside of the lid

I was left with a handful of small plastic fragments that were the heat-set posts holding the RF shield to the inside of the lid.

Some of the fragments that once held the RF shield in place

Clearly, the plastic has become brittle over the years. These machines were released in 1993, I think this might be a 1994-model as it has a slightly upgraded R4600 CPU in it.

As to the machine itself, I had a quick sticky-beak, there didn’t seem to be any immediately obvious things, but to be honest, I didn’t do a very thorough check. Maybe there’s some corrosion under the motherboard I didn’t spot, maybe it’s just a blown fuse in the PSU, who knows?

The inside of the Indy

This particular machine had 256MB RAM (a lot for its day), 8-bit Newport XL graphics, the “Indy Presenter” LCD interface (somewhere, we have the 15″ monitor it came with — sadly the connecting cable has some damaged conductors), and the HDD is a 9.1GB HDD I added some time back.

Where to now?

I was hanging on to these machines with the thinking that someone who was interested in experimenting with RISC machines might want them — find them a new home rather than sending them to landfill. I guess that’s still an option for the O2, as it still boots: so long as its remaining HDD doesn’t die it’ll be fine.

For the others, there’s the possibility of combining bits to make a functional frankenmachine from lots of parts. The Indy will need a new PROM battery if someone does manage to breathe life into it.

The Octane had two SCSI interfaces, one of which was dead — a problem that was known-of before I even acquired it. The PROM would bitch and moan about the dead SCSI interface for a good two minutes before giving up and dumping you in the boot menu. Press 1, and it’d hand over to arcload, which would boot a Linux kernel from a disk on the working controller. Linux would see the dead SCSI controller, and proceed to ignore it, booting just fine as if nothing had happened.

The Indigo2 R10000 was always the red-hedded step child: an artefact of the machine’s design. The IP22 design (Indy and Indigo2) was never designed with the intent of being used with a R10000 CPU, and the speculative execution features played havoc with caching on this design. The Octane worked fine because it was designed from the outset to run this CPU. The O2 could be made to work because of a quirk of its hardware design, but the Indigo2 was not so flexible, so kernel-space code had to hack around the problem in software.

I guess I’d still like to see the machines go to a good home, but no idea who that’d be in this day and age. Somewhere, I have a partial disc set of Irix 6.5, and there’s also a 20″ SGI GDM5410 monitor (not the Sun monitor pictured above) that, at last check, did still work.

It’ll be a sad day when these go to the tip.