May 072017
 

So, in amongst my pile of crusty old hardware is the old netbook I used to use in the latter part of my univerity days. It is a Lemote Yeeloong, and sports a ~700MHz Loongson 2F CPU (MIPS III little endian ISA) and 1GB RAM.

Back in the day it was a brilliant little machine. It came out of the box running a localised (for China) version of Debian, and had pretty much everything you’d need. I natually repartitioned the machine, setting up Gentoo and I had a separate partition for Debian, so I could actually dual-boot between them.

Fast forward 10 years, the machine runs, but the battery is dead, and Debian no longer supports MIPS-III machines. Debian Jessie does, but Stretch, likely due for release some time this year, will not, if you haven’t got a CPU that supports mips32r2 or mips64r2, you’re stuffed.

I don’t want to throw this machine away.  Being as esoteric as it is, it is an unlikely target for theft, as to the casual observer, it’ll just be “some crappy netbook”.  If someone were to try and steal it, there’s a very high probability I’ll recover it with my data because the day its PMON2000 boot firmware successfully boots a x86-64 OS like Ubuntu or Windows without the assistance of a VM of some kind would be the day Satan puts a requisition order in for anti-freeze and winter mittens.

My use case is for a machine I can take with me on the bicycle.  My needs aren’t huge: I won’t be playing video on this thing, it’ll be largely for web browsing and email.  The web browser needs to support JavaScript, so that rules out options like ELinks or Dillo, my preferred browser is Firefox but I’ll settle for something Webkit-based if that’s all that’s out there.

So what operating systems do I have for a machine that sports a MIPS-III CPU and 1GB RAM?  Fedora has a MIPS port, but that, like Debian, is for the newer MIPS systems.  Arch Linux too is for newer architectures.

I could bootstrap Alpine Linux… and maybe that’s worth looking into, they seem to be doing some nice work in producing a small and capable Linux distribution.  They don’t yet support MIPS though.

Linux From Scratch is an option, if a little labour intensive.  (Been there, done that.)

OpenBSD directly supports this machine, and so I gave OpenBSD 6.0 a try.  It’s a very capable OS, and while it isn’t Linux, there isn’t much that an experienced Linux user like myself needs to adapt to in order to effectively use the OS.  pkgsrc is a great asset to OpenBSD, with a large selection of pre-built packages already available.  Using that, it is possible to get a workable environment up and running very quickly.  OpenBSD/loongson uses the n64 ABI.

Due to licensing worries, they use a particularly old version of binutils as their linker and assembler.  The plan seems to be they wish to wean themselves off the GNU toolchain in favour of LLVM.  At this time though, much of the system is built using the GNU toolchain with some custom patches.  I found that, on the Yeeloong, 1GB RAM was not sufficient for compiling LLVM, even after adding additional swap files, and some packages I needed weren’t available in pkgsrc, nor would they build with the version of GNU tools available.

Maybe as they iron out the kinks in their build environment with LLVM, this will be worth re-visiting.  They’ve done a nice job so far, but it’s not quite up to where I need it to be.

Gentoo actually gives me the choice of two possible ABIs: o32 and n32o32 is the old 32-bit ABI, and suffers a number of performance problems, but generally works.  It’s what Debian Jessie and earlier supplies, and what their mips32 port will produce from Stretch onwards.

n32 is the MIPS equivalent of what some of you may know as x32 on AMD64 platforms, it is a 32-bit environment with 64-bit long pointers… the idea being that very few applications actually benefit from the use of 64-bit data types, and so the usual quantities like int and long remain the same as what they’d be on o32, saving memory.  The long long data type gets a boost because, although “32-bit”, the 64-bit operations are still available for use.

The trouble is, some applications have problems with this mode.  Either the code sees “mips64” in the CHOST and assumes a full 64-bit system (aka n64), or it assumes the pointers are the same width as a long, or the build system makes silly assumptions as to where things get put.  (virtualenv comes to mind, which is what started me on this journey.  The same problem affects x32 on AMD64.)

So I thought, I’d give n64 a try.  I’d see if I can build a cross-compiler on my AMD64 host, and bootstrap Gentoo from that.

Step 1: Cross-compiler

For the cross-compiler, Gentoo has a killer feature that I have not seen in too many other distributions: crossdev.  This is a toolchain build tool that can generate cross-compiler toolchains for most processor architectures and environments.

This is installed by running emerge sys-devel/crossdev.

A gotcha with hardened

I run “hardened” AMD64 stages on my machines, and there’s a little gotcha to be aware of: the hardened USE flag gets set by crossdev, and that can cause fun and games if, like on MIPS, the hardening features haven’t been ported.  My first attempt at this produced a n64 userland where pretty much everything generated a segmentation fault, the one exception being Python 2.7.  If I booted with init=/bin/bash (or init=/bin/bb), my virtual environment died, if I booted with init=/usr/bin/python2.7, I’d be dropped straight into a Python shell, where I could import the subprocess module and try to run things.

Cleaning up, and forcing crossdev to leave off hardened support, got things working.

Building the toolchain

With the above gotcha in mind:

# crossdev --abis n64 \
           --env 'USE="-hardened"' \
           -s4 -t mips64el-unknown-linux-gnu

The --abis n64 tells crossdev you want a n64 ABI toolchain, and the --env will hopefully keep the hardened flag unset. Failing that, try this:

# cat > /etc/portage/package.use/mips64 <<EOF
cross-mips64el-unknown-linux-gnu/binutils -hardened
cross-mips64el-unknown-linux-gnu/gcc -hardened
cross-mips64el-unknown-linux-gnu/glibc -hardened
EOF

If you want a combination of specific toolchain components to try, I’m using:

  • Binutils: 2.28
  • GCC: 5.4.0-r3
  • glibc: 2.25
  • headers: 4.10

Step 2: Checking our toolchain

This is where I went wrong the first time, I tried building the entire OS, only to discover I had wasted hours of CPU time building non-functional binaries. Save yourself some frustration. Start with a small binary to test.

A good target for this is busybox. Run mips64el-unknown-linux-gnu-emerge busybox, and wait for a bit.

When it completes, you should hopefully have a busybox binary:

RC=0 stuartl@beast ~ $ file /usr/mips64el-unknown-linux-gnu/bin/busybox 
/usr/mips64el-unknown-linux-gnu/bin/busybox: ELF 64-bit LSB executable, MIPS, MIPS-III version 1 (SYSV), statically linked, for GNU/Linux 3.2.0, stripped

Testing busybox

There is qemu-user-mips64el, but last time I tried it, I found it broken. So an easier option is to use real hardware or QEMU emulating a full system. In either case, you’ll want to ensure you have your system-of-choice running with a working 64-bit kernel already, if your real hardware isn’t already running a 64-bit Linux kernel, use QEMU.

For QEMU, the path-of-least-resistance I found was to use Debian. Aurélien Jarno has graciously provided QEMU images and corresponding kernels for a good number of ports, including little-endian MIPS.

Grab the Wheezy disk image and the corresponding kernel, then run the following command:

# qemu-system-mips64el -M malta \
    -kernel vmlinux-3.2.0-4-5kc-malta \
    -hda debian_wheezy_mipsel_standard.qcow2 \
    -append "root=/dev/sda1 console=ttyS0,115200" \
    -serial stdio -nographic -net nic -net user

Let it boot up, then log in with username root, password root.

Install openssh-client and rsync (this does not ship with the image):

# apt-get update
# apt-get install openssh-client rsync

Now, you can create a directory, and pull the relevant files from your host, then try the binary out:

# mkdir gentoo
# rsync -aP 10.0.2.2:/usr/mips64el-unknown-linux-gnu/ gentoo/
# chroot gentoo bin/busybox ash

With luck, you should be in the chroot now, using Busybox.

Step 3: Building the system

Having done a “hello world” test, we’re now ready to build everything else. Start by tweaking your /usr/mips64el-unknown-linux-gnu/etc/portage/make.conf to your liking then adjust /usr/mips64el-unknown-linux-gnu/etc/portage/make.profile to point to one of the MIPS profiles. For reference, on my system:

RC=0 stuartl@beast ~ $ ls -l /usr/mips64el-unknown-linux-gnu/etc/portage/make.profile
lrwxrwxrwx 1 root root 49 May  1 09:26 /usr/mips64el-unknown-linux-gnu/etc/portage/make.profile -> /usr/portage/profiles/default/linux/mips/13.0/n64
RC=0 stuartl@beast ~ $ cat /usr/mips64el-unknown-linux-gnu/etc/portage/make.conf 
CHOST=mips64el-unknown-linux-gnu
CBUILD=x86_64-pc-linux-gnu
ARCH=mips

HOSTCC=x86_64-pc-linux-gnu-gcc

ROOT=/usr/${CHOST}/

ACCEPT_KEYWORDS="mips ~mips"

USE="${ARCH} -pam"

CFLAGS="-O2 -pipe -fomit-frame-pointer"
CXXFLAGS="${CFLAGS}"

FEATURES="-collision-protect sandbox buildpkg noman noinfo nodoc"
# Be sure we dont overwrite pkgs from another repo..
PKGDIR=${ROOT}packages/
PORTAGE_TMPDIR=${ROOT}tmp/

ELIBC="glibc"

PKG_CONFIG_PATH="${ROOT}usr/lib/pkgconfig/"
#PORTDIR_OVERLAY="/usr/portage/local/"

Now, you should be ready to start building:

# mips64el-unknown-linux-gnu-emerge -e \
    --keep-going -j6 --load-average 12.0 @system

Now, go away, and do something else for several hours.  It’ll take that long, depending on the speed of your machine.  In my case, the machine is an AMD Phenom II x6 with 8GB RAM, which was brand new in 2010.  It took a good day or so.

Step 4: Testing our system

We should have enough that we can boot our QEMU VM with this image instead.  One way of trying it would be to copy across the userland tree the same way we did for pulling in busybox and chrooting back in again.

In my case, I took the opportunity to build a kernel specifically for the VM that I’m using, and made up a disk image using the new files.

Building a kernel

Your toolchain should be able to cross-build a kernel for the virtual machine.  To get you started, here’s a kernel config file.  Download it, decompress it, then drop it into your kernel source tree as .config.

Having done that, run make olddefconfig ARCH=mips to set the defaults, then make menuconfig ARCH=mips and customise to your hearts content. When finished, run make -j6 vmlinux modules CROSS_COMPILE=mips64el-unknown-linux-gnu- to build the kernel and modules.

Finally, run make modules_install firmware_install INSTALL_MOD_PATH=$PWD/modules CROSS_COMPILE=mips64el-unknown-linux-gnu- to install the kernel modules and firmware into a convenient place.

Making a root disk

Create a blank, raw disk image using qemu-img, then partition it as you like and mount it as a loopback device:

# qemu-img create -f raw gentoo.raw 8G
# fdisk gentoo.raw
(do your partitioning here)
# losetup -P /dev/loop0 $PWD/gentoo.raw

Now you can format the partitions /dev/loop0pX as you see fit, then mount them in some convenient place. I’ll assume that’s /mnt/vm for now. You’re ready to start copying everything in:

# rsync -aP /usr/mips64el-unknown-linux-gnu/ /mnt/vm/
# rsync -aP /path/to/kernel/tree/modules/ /mnt/vm/

You can use this opportunity to make some tweaks to configuration files, like updating etc/fstab, tweaking etc/portage/make.conf (changing ROOT, removing CBUILD), and setting up a getty on ttyS0. I also like to symlink lib to lib64 in non-multilib environments such as this: Don’t symlink lib and lib64! See below.

# cd /mnt/vm
# mv lib/* lib64
# rmdir lib
# ln -s lib64 lib
# cd usr
# mv lib/* lib64
# rmdir lib
# ln -s lib64 lib

When you’re done, unmount.

First boot

Run QEMU with the following arguments:

# qemu-system-mips64el -M malta \
    -kernel /path/to/your/kernel/vmlinux \
    -hda /path/to/your/gentoo.raw \
    -append "root=/dev/sda1 console=ttyS0,115200 init=/bin/bash" \
    -serial stdio -nographic -net nic -net user

It should boot straight to a bash prompt. Mount the root read/write, and then you can make any edits you need to do before boot, such as changing the root password. When done, re-mount the root as read-only, then exec /sbin/init.

# mount / -o rw,remount
# passwd
… etc
# mount / -o ro,remount
# exec /sbin/init

With luck, it should boot to completion.

Step 5: Making the VM a system service

Now, it’d be real nice if libvirt actually supported MIPS VMs, but it doesn’t appear that it does, or at least I couldn’t get it to work.  virt-manager certainly doesn’t support it.

No matter, we can make do with a telnet console (on loopback), and supervisord to daemonise QEMU.  I use the following supervisord configuration file to start my VMs:

[unix_http_server]
file=/tmp/supervisor.sock   ; (the path to the socket file)

[supervisord]
logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB        ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10           ; (num of main logfile rotation backups;default 10)
loglevel=info                ; (log level;default info; others: debug,warn,trace)
pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false               ; (start in foreground if true;default false)
minfds=1024                  ; (min. avail startup file descriptors;default 1024)
minprocs=200                 ; (min. avail process descriptors;default 200)

; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL  for a unix socket

[program:qemu-mips64el]
command=/usr/bin/qemu-system-mips64el -cpu MIPS64R2-generic -m 2G -spice disable-ticketing,port=5900 -M malta -kernel /home/stuartl/kernels/qemu-mips/vmlinux -hda /var/lib/libvirt/images/gentoo-mips64el.raw -append "mem=256m@0x0 mem=1792m@0x90000000 root=/dev/sda1 console=ttyS0,115200" -chardev socket,id=char0,port=65223,host=::1,server,telnet,nowait -chardev socket,id=char1,port=65224,host=::1,server,telnet,nowait -serial chardev:char0 -mon chardev=char1,mode=readline -net nic -net bridge,helper=/usr/libexec/qemu-bridge-helper,br=br0

The following creates two telnet sockets, port 65223 is the VM’s console, 65224 is the QEMU control console. The VM has the maximum 2GB RAM possible and uses bridged networking to the network bridge br0. There is a graphical console available via SPICE.

All telnet and SPICE interfaces are bound to loopback, so one must use SSH tunnelling to reach those ports from another host. You can change the above command line to use VNC if that’s what you prefer.

At this point, the VM should be able to boot on its own. I’d start with installing some basic packages, and move on from there. You’ll find the environment is very sparse (my build had no Perl binary for example) but the basics for building everything should be there.

You may also find that what is there, isn’t quite installed right… I found that sshd wasn’t functional due to missing users… a problem soon fixed by doing an emerge -K openssh (the earlier step will have produced binary packages).

In my case, that’s installing a decent text editor (vim) and GNU screen so I can start a build, then detach.  Lastly, I’ll need catalyst, which is Gentoo’s release engineering tool.

At the moment, this is where I’m at.  GNU screen has indirectly pulled in Perl as a dependency, and that is building as I type this.  It is building faster than the little netbook does, and I have the bonus that I can throw more RAM at the problem than I can on the real hardware. The plan from here:

  1. emerge -ek @system, to build everything that got missed before.
  2. ROOT=/tmp/seed emerge -eK @system, to bundle everything up into a staging area
  3. populating /tmp/seed/dev with device files
  4. tar-ing up /tmp/seed to make my initial “seed” stage for catalyst.
  5. building the first n64 stages for Gentoo using catalyst
  6. building the packages I want for the netbook in a chroot
  7. transferring the chroot to the netbook

Symlinking lib and lib64… don’t do it!

So, I was doing this years ago when n32 was experimental.  I recall it being necessary then as this was before Portage having proper multilib support.  The earlier mipsel n32 stages I built, which started out from kanaka‘s even more experimental multilib stages, required this kludge to work-around the lack of support in Portage.

Portage has changed, it now properly handles multilib, and so the symlink kludge is not only not necessary, it breaks things rather badly, as I discovered.  When packages merge files to /lib, rather than following the symlink, they’ll replace it with a directory.  At that point, all hell breaks loose, because stuff that “appeared” in /lib before is no longer there.

I was able to recover by rsync-ing /lib64 to /lib, which isn’t a pretty solution, but it’ll be enough to get an initial “seed” stage.  Running that seed stage through Catalyst will clean up the remnants of that bungle.

Oct 132016
 

Well, today’s mail had a surprise.  Back about 6 years ago, I was sub-contracted to Jacques Electronics to help them develop some device drivers for their video intercom system.  At the time, they were using TI’s TLV320AIC3204 and system-on-modules based on the Freescale i.MX27 SoC.

No driver existed in the ALSA tree for this particular audio CODEC, and while TI did have one available under NDA, the driver was only licensed for use with a TI OMAP SoC.  I did what just about any developer would do, grabbed the closest-looking existing ALSA SoC driver, ripped it apart and started hacking.  Thus I wound up getting to grips with the I²S infrastructure within the i.MX27 and taming the little beast that is the TLV320AIC3204, producing this patch.

As the code was a derivative work, the code was automatically going to be under the GPLv2 and thus was posted on the ALSA SoC mailing list for others to use.  This would help protect Jacques from any possible GPL infringement regarding the use of that driver.  I was able to do this as it was a clean-room implementation using only material in TI’s data sheet, thus did not contain any intellectual property of my then-employer.

About that time I recall one company using the driver in their IP camera product, the driver itself never made it into the mainline kernel.  About 6 months later, another driver for the TLV320AIC3204 and 3254 did get accepted there, I suspect this too was a clean-room implementation.

Fast forward to late August, I receive an email from Jeremy McDermond on behalf of the Northwest Digital Radio.  They had developed the Universal Digital Radio Controller board for the Raspberry Pi series of computers based around this same CODEC chip.  Interestingly, it was the ‘AIC3204 driver that I developed all that time before that proved to be the code they needed to get the chip working.  The chip in question can be seen up the top-right corner of the board.

Universal Digital Radio Controller

Timely, as there’s a push at the moment within Brisbane Area WICEN Group to investigate possible alternatives to our aging packet radio system and software stack.  These boards, essentially being radio-optimised sound cards, have been used successfully for implementing various digital modes including AX.25 packet, D-Star and could potentially do FreeDV and other digital modes.

So, looks like I’ll be chasing up a supplier for a newer Raspberry Pi board, and seeing what I can do about getting this device talking to the world.

Many thanks to the Northwest Digital Radio company for their generous donation! 🙂

Sep 292014
 

Well, it’s been a busy year so far for security vulnerabilities in open-source projects.  Not that those have been the only two bugs, they’re just two high-profile ones that are getting a lot of media attention.

Now, a number of us do take sheer delight in pointing and laughing when one of the big boys, whether they be based in Redmond or California, makes a security balls-up on a big scale.  After all, people pay big dollars to use some of that software, and many are dependent on it for their livelihoods.

The question does get raised though, what do you trust more?  A piece of software whose code is a complete secret, or the a piece of software anyone can audit?  Some argue the former, because anyone can find the holes in the latter and exploit them.  Some argue the latter, since anyone can find the holes and fix them.  Not being able to see the code doesn’t guarantee a lack of security issues however, and these last two headline-making bugs is definitely evidence that having the code isn’t a guarantee to a bug-free utopia.

There is no guarantee either way.

I’ve seen both open-source systems and high-end commercial systems both perform well and I’ve seen both make a dismal failure.  Bad code is bad code, no matter what the license, and even having the source available doesn’t mean you can fix it as first one must be able to understand what its intent is.  Information Technology in particular seems to attract the technologically inept but socially capable types that are able to talk their way into nearly any position, and so you wind up with the monstrosities that you might see on The Daily WTF.  These same people lurk amongst open-source circles too, and there are those who just make an honest mistake.  Security is hard, and it can be easy to overlook a possible hole.

I run Gentoo here, have done so now since 2004 (damn, 10 years already, but I digress…).  I’ve been building my own stage 3 tarballs from scratch since 2010.  July 2010 I bought my current desktop, a 6-core AMD Phenom machine, and so combined with the 512Kbps ADSL I had at the time, it was faster for me to compile stage 3 tarballs for the various systems (i386, AMD64 and about 6 different MIPS builds) than to download the sources.  If I wanted an up-to-date stage 3, I just took my last build, ran it through Gentoo Catalyst, and out came a freshly built tarball.

I still obtain my operating systems that way.  Even though I’ve upgraded the ADSL, I still use the same scripts that used to produce the official Gentoo/MIPS media.

This means I could audit every piece of software that forms my core system.  I have the source code there, all of it.  Not many Linux users have this, most have it at arms reach (i.e. an apt-get source ${PACKAGE} away), or at worst, a polite email/letter to their supplier (e.g. Netcomm will supply sources for their routers for a ~AU$10 fee), however I already have it.

So did I do any audits?  Have I done any audits?  No.  Ultimately I just blindly trust what comes down the wire, and to some, that is arguably no better than just blindly trusting what Apple and Microsoft produce.

Those who say that, do have a point.  I didn’t pick up on HeartBleed, nor on ShellShock, and I probably haven’t spotted what will become the next headline-grabbing bug.  There’s a lot of source code that goes into a GNU/Linux system, and if I were to sit there and audit it, myself, it’d take me a lifetime.  It’d cost me a fortune to pay a team to analyse it.

However, I at least have the choice of auditing parts of it.  I’ll never be able to audit the copies of Microsoft Windows, or the one copy of Apple MacOS X I have.  For those, I’m reliant on the upstream vendors to audit, test and patch their code, I cannot do it myself.

For the open-source software though, it’s ultimately my choice.  I can do it myself, I can also pay someone to do it, I’ve simply chosen not to at this time.  This is an important distinction that the anti-open-source camp seem to forget.

As for the quality factor: well I’ve spent more time arguing with some piece of proprietary software and having trouble getting it to do something I need it to do, or fixing up some cock up caused by a bug in the said software.  One option, I spend hours arguing with it to make it work, and have to pay good money for the privilege.  The other, they money stays in my pocket, and in theory I can re-build it to make it work if needed.  One will place arbitrary restrictions on how I use the software as an end user, forcing me to spend money on more expensive licenses, the other will happily let me keep pushing it until I hit my system’s technical limits.

Neither offer me any kind of warranty regarding to losses I might suffer as a result of their software (I’m sorry, but US$5.00 is as good as worthless), so the money might as well stay in my pocket while I learn something about the software I use.

I remain in control of my destiny that way, and that is the way I’d like to keep it.

Sep 082013
 

Well, some might remember my time with a cheap and nasty Android tablet (some might call these “landfill Android”).  The device packaging did not once even acknowledge the fact that there was GPL’ed software onboard, let alone how one obtains the source.

I discovered it was based around the Vimicro VC0882 SoC.  Turns out, that’s the same as the ViewSonic ViewPad 10e, who do release their kernel sources on their knowledge base.

Thank-you ViewSonic, you have just helped me greatly!  Maybe I should track down one of your tablets and buy one in appreciation.

Feb 172013
 

Well, I was half expecting that it’d happen one day. Gentoo Bug 89744, the bug that saw me promoted to developer on the Gentoo/MIPS team, is now a retirement bug in full swing.

To those in the Gentoo community, I say, thank-you for putting up with me for so long. It is probably time that I moved on though. Real life has meant I’ve got practically no time during my working week to do anything meaningful, and after a week of arguing with computers (largely Ubuntu-based) I come home on a Friday evening not feeling like even looking at a computer. Some weekends, the computer has stayed in my backpack, and not been removed until the following Monday when I return to work.

Thus the time has come, I must be going.

That said, I mostly did enjoy the time I had as a developer. I still remain a Gentoo user, as that seems to be the OS that best fits my usage patterns, and I might pop up from time to time, but I’ll probably maintain a fairly low profile from now on.

I actually didn’t notice the account being shut down, only discovered today in fact, that the redhatter@gentoo.org email address was not working from an Amateur radio colleague. It’s then I thought to have a quick gander and found out what had happened.

This does leave me with two Lemote boxes, that technically no longer belong here.

Remember these? They’re looking for a home now!

I shall enquire, and find out where to send the boxes themselves, or a donation to cover their cost. It is not right that they remain here without some sort of compensation.

This leaves some people without a means of contacting me.  I don’t bother with the instant messengers these days, and definitely not Skype.

Plain old email still works though, you can contact me at stuartl at longlandclan dot yi dot org from now on.  As for the old links on dev.gentoo.org, terribly sorry but that’s outside my control now.


Update:

The Lemote boxes now have a home. Thanks Anthony!

Apr 012012
 

Well, it’s been a while since I touched this tablet.  I basically chucked it in a corner in disgust after it shat itself rather unceremoniously on the trip before we even got to the NSW/Victorian border.  By “shat” itself, I mean corrupting files on the internal microSD card, intermittent device resets, display flickers, all the hallmarks of a dry joint.

The seller on eBay that sold me the device have been completely unresponsive as to the problems, so looks like I kissed about $250 goodbye.  Ahh well, such is life.  They are still being sold on eBay, but buyer beware, they are cheap, and it’s pot luck whether yours is cheerful, or nasty like mine.  If you want something reliable, look elsewhere.

Having made this mistake, well, I’m looking to make lemonade from the lemon.  First step, was to figure out what on earth I had.  So out with the screwdriver.

You’ll notice on the top and bottom of the unit, there are four small plugs concealing screws.  These hold the LCD screen assembly in place.  Undo these, then you need to carefully work your way around and release the clips that hold the LCD screen assembly.  Do not try to detach the LCD touch panel from the LCD!  I initially couldn’t get it to budge, so I tried doing exactly this in the hunt for possible hidden screws (there were none).  This was the end result:

Shattered Flytouch III touch panel

Why one should not try to detach the touch panel.

Never mind I say… the unit was just about destined for the bin as it was.  External USB HID devices work for what I’m after, but it’ll mean any touch-related fun is out unless I can pick up a replacement 4-wire panel.  Element14 and RS have them at >$80, to which I say, bugger it, I’ll do without.

Having pulled the unit apart, the main PCB is held to the back shell by a few screws, one thing is immediately apparent.  The whole device is based on what looks to be a fairly generic System-on-Module based around the Vimicro VC0882BCXA System-on-Chip, and the Vimicro VC7822EL companion chip.

Flytouch III PCB

Top left is a Wifi module based on the Realtek RTL8111, and to the right, the GPS module (which hooks to one of the serial ports from what I recall).  Down the bottom of the image are the USB ports.  Near the HDMI socket is a Silicon Image SiI9022ACNU HDMI transmitter.

The system on module looks interesting, and I’m curious to find out more about it, as for hobby projects, the pins are not too small to deal with using a soldering iron.  The OS and boot loader exist on the microSD card.  I tried putting a 16GB card in, but evidently I wasn’t getting the partition table right as it wouldn’t boot.  I haven’t tried hooking up a serial port as yet, so it’s hard to know what is wrong.  Some research indicates that ttyS0 lurks on this board just near the aforementioned microSD socket:

The system on module within the Flytouch III

The system on module within the Flytouch III

I haven’t spotted the bad joints that were giving me grief. In fact, having gotten it out of the case, I find the top USB port (flakey from day one) seems to be behaving, and I’ve had no issues with it running with the case apart.  Otherwise I’d be running a soldering iron over a few joints just to make sure everything was right.

Next step?  Well for now, I’ll put it back together (minus touchscreen) and put it aside.  I’ll have a look at tacking a connector onto those serial pins, with a level shifter so I can interact with the serial console.

Having gotten bootloader access, I should be able to debug the SD card cloning issue, then I can have a close gander at what the current u-boot and kernel are doing to tickle the hardware.  End game?  Well, Android isn’t much use without a touch screen, so I’ll be probably hacking together a Gentoo-based environment with some amateur radio related software.  We shall see.

Dec 202011
 

Well, further analysis today. The Flytouch III seems to boot off an embedded SD card. I don’t know if it is removable or not, for now I’ll assume no.

Having gained root access earlier, I was able to use dd and nc to siphon off a copy of the internal SD card, which appears as /dev/block/mmcblk0. To grab a copy, first plug the unit into Ethernet (it’ll be faster, trust me) and have another Linux box handy:

Start up netcat on a Linux system:
$ busybox nc -l -p 8123 > tablet.img

Then on the tablet, become root:
$ /system/bin/su

Then start copying to the other system (here; its IP is 12.23.34.45):
# dd if=/dev/block/mmcblk0 | nc 12.23.34.45 8123

Sit back and wait, it should be done in about 5 minutes. Now if you look at the partition table, you’ll see the following:

Disk tablet.img: 482 cylinders, 255 heads, 63 sectors/track
Units = sectors of 512 bytes, counting from 0

   Device Boot    Start       End   #sectors  Id  System
tablet.img1            63   5535320    5535258   b  W95 FAT32           < -- User applications, data live here
tablet.img2       5535321   7612181    2076861   5  Extended
tablet.img3       7612248   7677783      65536  bb  Boot Wizard hidden  <-- Kernel?
tablet.img4       7677784   7743319      65536  bb  Boot Wizard hidden  <-- UBoot?
tablet.img5       5535384   6059608     524225  83  Linux               <-- /system partition
tablet.img6       6059672   7595608    1535937  83  Linux               <-- Android internal?
tablet.img7       7595672   7611992      16321  83  Linux               <-- ???

Partitions 3 and 4 are a complete mystery. They're not a standard Linux file system, but, the former appears to hold a copy of the Linux kernel, and the latter seems to hold a copy of UBoot. You can bust the image apart using the following script:


/sbin/sfdisk -uS -l tablet.img | grep ^tablet.img | while read part; do
pn=$( echo "$part" | cut -c 11-11 );
s=$( echo "$part" | cut -c 13-25 );
l=$( echo "$part" | cut -c 36-48 );
echo "[$pn][$s][$l]";
dd if=tablet.img of=tablet-$pn.img skip=$(( $s )) count=$(( $l ));
done

You might have to play with column offsets.

The initial part of partition 3 looks like this:

00000000  41 4e 44 52 4f 49 44 21  c0 d7 4b 00 00 80 00 10  |ANDROID!..K.....|
00000010  b5 2a 15 00 00 00 00 11  00 00 00 00 00 00 f0 10  |.*..............|
00000020  00 01 00 10 00 08 00 00  00 00 00 00 00 00 00 00  |................|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000240  b8 29 4b 8c 7c d2 1f 65  cf b3 3a 78 bc 87 c0 61  |.)K.|..e..:x...a|
00000250  2e 24 79 a5 00 00 00 00  00 00 00 00 00 00 00 00  |.$y.............|
00000260  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000800  27 05 19 56 43 d9 c4 f4  4e ab c7 11 00 4b d7 80  |'..VC...N....K..|
00000810  80 00 80 00 80 00 80 00  d5 42 0e 53 05 02 02 00  |.........B.S....|
00000820  4c 69 6e 75 78 2d 32 2e  36 2e 33 35 2e 37 00 00  |Linux-2.6.35.7..|
00000830  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000840  d3 f0 21 e3 10 9f 10 ee  56 00 00 eb 05 a0 b0 e1  |..!.....V.......|
00000850  52 00 00 0a 6c 00 00 eb  05 80 b0 e1 4f 00 00 0a  |R...l.......O...|
00000860  7b 00 00 eb 13 00 00 eb  c0 d0 9f e5 00 e0 8f e2  |{...............|
00000870  10 f0 8a e2 30 5f 11 ee  02 50 85 e3 30 5f 01 ee  |....0_...P..0_..|
00000880  02 00 80 e3 1f 50 a0 e3  10 5f 03 ee 10 4f 02 ee  |.....P..._...O..|

Note the rather prominent “Linux-2.6.35.7”. Similarly, if we pick through partition 4:

00020eb0  11 12 a0 41 10 13 a0 51  30 1c 81 41 10 02 a0 e1  |...A...Q0..A....|
00020ec0  1e ff 2f e1 ff ff ff ff  ff ff ff ff ff ff ff ff  |../.............|
00020ed0  00 10 05 60 20 10 05 60  00 13 05 60 20 13 05 60  |...` ..`...` ..`|
00020ee0  40 13 05 60 00 16 05 60  20 16 05 60 00 19 05 60  |@..`...` ..`...`|
00020ef0  20 19 05 60 00 1c 05 60  20 1c 05 60 40 1c 05 60  | ..`...` ..`@..`|
00020f00  55 2d 42 6f 6f 74 20 32  30 31 30 2e 30 36 20 28  |U-Boot 2010.06 (|
00020f10  4f 63 74 20 32 39 20 32  30 31 31 20 2d 20 31 37  |Oct 29 2011 - 17|
00020f20  3a 32 37 3a 30 31 29 00  18 13 ea 80 20 13 ea 80  |:27:01)..... ...|
00020f30  27 13 ea 80 2e 13 ea 80  35 13 ea 80 3c 13 ea 80  |'.......5...< ...|
00020f40  43 13 ea 80 4a 13 ea 80  51 13 ea 80 58 13 ea 80  |C...J...Q...X...|
00020f50  5f 13 ea 80 67 13 ea 80  6f 13 ea 80 77 13 ea 80  |_...g...o...w...|
00020f60  7f 13 ea 80 87 13 ea 80  8f 13 ea 80 97 13 ea 80  |................|
Dec 162011
 

Yes, I’ve joined this century and bought myself a tablet. Lately, I’ve found myself needing some means of navigating in strange areas whilst on the bicycle, and while pieces of paper work — if you’re organised enough to print them out in advance and not ride too fast (otherwise they disappear with the wind), I’ve found there are a number of shortcomings with this.

Since I like open source, and didn’t like the idea of spending several hundred on a hand-held GPS with proprietary firmware & map data which I need to constantly purchase updates for, I opted for the cheapskate route.  I picked up a GL4Ever Flytouch III Tablet off eBay.  The unit I have came loaded with Android 2.3 (Gingerbread).

Ultimately I may replace the OS, or at least, the kernel, soon as I have sources for it, but in the meantime, it runs what it came with.  I have however, already managed to gain root access.

Those who might do a search for how to do so, may come across this guide.  I tried this first, and found I had no joy.  USB Debugging was enabled out-of-the-box on the unit I have, but z4root did not successfully enable root access.  The following are my notes on how I gained a shell with root access on the device.  Ohh, and I warn you, there is no warranty given in the instructions below.  If it breaks, you get to keep the pieces.

  1. Download and install Gingerbreak.
  2. Run Gingerbreak, it will run for a while, before resetting the device.  Upon starting, you should now notice you have a Superuser application installed.
  3. Next, install Android Terminal.
  4. Now, run /system/bin/su.

/bin/su is a symbolic link to /bin/busybox which was installed without the setuid bit, and is broken anyway, you’ll find if you do add a setuid bit, it will report that it can’t find the ‘root‘ user.  This system has no /etc/passwd or equivalent user database, so it has no idea who ‘root‘ is, but it knows who UID 0 is, and that’s what matters.  The latter ‘su‘ you’ll find has the necessary permissions, and knows about UID 0.

Other things I’ve found… the operating system lurks on a SD card embedded in the device.  Or at least, it’s presented as a SD card; /dev/block/mmcblk0.  The user-accessible SD-card port is /dev/block/mmcblk1.  You can verify this by ejecting the card, doing a ls /dev/block, then inserting a card and repeating.

On my TODO list, is to make a DD-copy of this block device, and pick through to see how one swaps out the kernel.  I’ll post notes if I figure this out.  I am also yet to obtain the kernel sources, I’ll chase those up before long.

Sep 132011
 

Hi all…

I got fed up of restoring my firmware for the Broadcom wireless chip in my late-2008 model MacBook.  Anyone who has one of these might find the current in-tree versions of net-wireless/b43-firmware is missing files needed by the modern b43 driver (namely ucode16_mimo.fw), and net-wireless/b43-fwcutter doesn’t well, cut it, for extracting the newer files.

If you’ve got a newer 802.11n-based Broadcom chip, you might find the following ebuilds handy:

  • net-wireless/b43-firmware-5.10.56.27.3
  • net-wireless/b43-fwcutter-015 and net-wireless/b43-fwcutter-9999

The first is the firmware mentioned in this post.  It needs a newer fwcutter binary than is provided in Portage.  You’ve got the choice of the latest version, or the bleeding edge via git.  Both work at time of writing, although neither are guaranteed.

The ebuilds are not in-tree, I’ll leave that for the actual maintainer for these ebuilds to pick them up if desired, I’ve put them in an overlay accessed via the following command:

git clone git://git.longlandclan.yi.org/overlays/b43.git

Or you can take a squiz via gitweb.

Jun 232011
 

It was interesting when I posted a news article to the WIA regarding IPv6, how quickly it got shot down by “experts”.

A recent addition to our network was a 2008-model Apple MacBook, which I have dual-booting Gentoo and MacOS X 10.6.7 nicely.  One quirk of this particular laptop though, is that it will, when running its native OS, intermittently drop off the IPv4-only network.

The first tip-off to this is usually things like Skype ceasing to work.  Then I’ll notice DNS isn’t resolving (DNS is IPv6-accessible, but not many systems support RDNSS).

As a work-around to the problem, and also for my own self-education, I decided I’d have a crack at getting NAT64 and DNS64 to work.  What are they exactly?

NAT64 as the name suggests, is a variant of NAT that translates IPv6 to IPv4.  In doing so, allowing my MacBook that’s just disappeared from the face of the IPv4-only world, to still access the IPv4-part of the Internet.

DNS64 is a service that synthesizes AAAA records for host names that do not provide one.

The two work together to provide Internet access to an IPv6 only host.

What you will need to know

Make sure you have the following information on-hand.  I’ll use the following examples:

  • Your server’s IPv4 address on the local network: e.g. 192.168.0.1/24
  • IPv4 NAT address pool: This must not overlap with your existing networks.
    Examples use 192.168.255.0/24, I used 172.16.24.0/24
  • TAYGA’s tunnel IPv4 address: This will be the first address in the above subnet (i.e. 172.16.24.1)
  • Your network’s IPv6 subnet: e.g. 2001:dead:beef:1200::/56
  • IPv6 NAT address pool: This needs to be a non-overlapping portion of your address space.  In my case, I’m borrowing a /56 from AARNet, and I used a /64 for this, setting the lower 8-bits of the prefix to 0x64.  It only needs to be /96 in size.
    Example used: 2001:dead:beef:1264::/96

NAT64 setup

To get NAT64 working; start by installing TAYGA.  This is a userspace daemon uses the TUN device to route between IPv4 and IPv6.  On Gentoo, begin by running emerge tayga.  (You may need to keyword it.)

This installs the binary, but crucially, it comes with no init scripts.  You will need to create one yourself like this:

#!/sbin/runscript
# Copyright 1999-2004 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: /var/cvsroot/gentoo-x86/sys-libs/gpm/files/gpm.rc6,v 1.12 2004/07/15 01:02:02 agriffis Exp $

depend() {
    need gw6c net.nat64
}

start() {
    ebegin "Starting tayga"
    start-stop-daemon --start --quiet -p /var/run/tayga.pid \
        --exec /usr/sbin/tayga -- \
        -u nobody -g nogroup \
        --pidfile /var/run/tayga.pid
    eend ${?}
}

stop() {
    ebegin "Stopping tayga"
    start-stop-daemon --stop --quiet --pidfile /var/run/tayga.pid
    eend ${?}
}

Now, edit /etc/tayga.conf, using /etc/tayga.conf.example as a guide.  There are comments provided.  The following are the settings I used (with the above addresses):

#
# TAYGA's IPv4 address.  This is NOT your router's IPv4 address!  TAYGA
# requires its own address because it acts as an IPv4 and IPv6 router, and
# needs to be able to send ICMP messages.  TAYGA will also respond to ICMP
# echo requests (ping) at this address.
#
# This address can safely be located inside the dynamic-pool prefix.
#
# Mandatory.
#
ipv4-addr 172.16.24.1
# ... etc ...
#
# The NAT64 prefix.  The IPv4 address space is mapped into the IPv6 address
# space by prepending this prefix to the IPv4 address.  Using a /96 prefix is
# recommended in most situations, but all lengths specified in RFC 6052 are
# supported.
#
# This must be a prefix selected from your organization's IPv6 address space
# or the Well-Known Prefix 64:ff9b::/96.  Note that using the Well-Known
# Prefix will prohibit IPv6 hosts from contacting IPv4 hosts that have private
# (RFC1918) addresses, per RFC 6052.
#
# The NAT64 prefix need not be specified if all required address mappings are
# listed in `map' directives.  (See below.)
#
# Optional.
#
prefix 2001:dead:beef:1264::/96
#
# Dynamic pool prefix.  IPv6 hosts which send traffic through TAYGA (and do
# not correspond to a static map or an IPv4-translatable address in the NAT64
# prefix) will be assigned an IPv4 address from the dynamic pool.  Dynamic
# maps are valid for 124 minutes after the last matching packet is seen.
#
# If no unassigned addresses remain in the dynamic pool (or no dynamic pool is
# configured), packets from unknown IPv6 hosts will be rejected with an ICMP
# unreachable error.
#
# Optional.
#
dynamic-pool 172.16.24.0/24
# ... etc ...

Now, having done this… you just need to make sure the nat64 device gets created and initialised by openrc.  In /etc/conf.d/net:

# NAT64 configuration for TAYGA
config_nat64=(
   "192.168.0.1/24"
   "2001:dead:beef:1264::1/64"
)
routes_nat64=(
        "172.16.24.0/24"
        "2001:dead:beef:1264::/96"
)

preup() {
        case ${IFACE} in
                nat64)
                        /usr/sbin/tayga --mktun
                        ;;
        esac
}

Now, symlink /etc/init.d/net.lo to /etc/init.d/net.nat64, start the tayga service, and you should find that you can ping e.g. 2001:dead:beef:1264::8.8.8.8 (8.8.8.8 is Google DNS).

DNS64 setup

All good and well if you know the IP addresses, but most people don’t.  Now emerge totd.  Use /usr/share/doc/totd-VERSION/totd.conf.sample.bz2 as an example for configuring /etc/totd.conf:

; $Id: totd.conf.sample,v 1.9 2003/09/17 15:56:20 dillema Exp $
; Totd sample configuration file
; you can have multiple forwarders, totd will always prefer
; forwarders listed early and only use forwarders listed later
; if the first ones are unresponsive.
forwarder 2001:dead:beef:1234::1 port 65053
forwarder 127.0.0.1 port 65053
forwarder 8.8.8.8 port 53
forwarder 8.8.4.4 port 53

; you can have multiple prefixes or even no prefixes at all
; totd uses them in round-robin fashion
prefix 2001:dead:beef:1264::
; the port totd listens on for incoming requests
port 53
; the pidfile to use (default: /var/run/totd.pid)
pidfile /var/run/totd.pid
; interfaces totd listens on (UDP only for now and not on Linux)
; If left out totd will only open wildcard sockets.
;interfaces br0
; 6to4 reverse lookup
stf

In my case, I have a local caching name-server (BIND), which I’ve moved to port 65053. The trick-or-treat daemon now sits on port 53 where the rest of the network expects it. You can now start the totd service, point your /etc/resolv.conf files to it, and everything should Just Work.

Testing

Easiest way is to shut off IPv4, and set up /etc/resolv.conf on your client with the IPv6 address of your server running totd.  You should now be able to browse IPv4-only sites as if IPv4 were running.  I achieved this test by plugging into Ethernet, turning off wicd (it kept wanting to start-up dhcpcd), then manually bringing the interface up and configuring /etc/resolv.conf.