qunilator-installer
Writes a QUniLator release image to an SD card, and
sets the SD card up before it has ever booted.
Writing the image is the one step of setting a QUniLator up that happens on your
own machine, and it is the step that loses people: find the image, find a
writer, find the SD card, get the device name right. This does the same thing on
macOS, Windows and Linux, and refuses to write anything that is not a removable
disk.
Written and used on macOS and Linux. The Windows path is written but has never
been run — see the end of this file.
Run it with nothing
qunilator-installer
It asks its way through: which QUniLator the image is for, whether to fetch the
newest release or use a file, which SD card of the ones it can see, whether to
set the machine up now and under what name — and then shows all of it back,
changeable line by line, before anything is written. That is what somebody meets
who ran the program rather than typing a command line; on Windows a console
program double-clicked opens a window, and a usage message would close it again
before it could be read.
Given a terminal it draws a screen — arrow keys, a progress bar, a review you
can walk through and edit. Given anything else — a pipe, output to a file, a
console that will not take escape sequences — it asks the same questions a line
at a time, which is also what makes the whole flow drivable from a script.
A run that fails says so where it can be read. The failure stays on the
screen, under the step it stopped in and what the run had already said, until a
key dismisses it. On Windows there is a second reason for that: a program the
system raises for administrator rights is given a console window of its own and
closes it the moment the process ends, so a run that owns its window waits to be
dismissed at the end of a good run too — that is where what to do with the
QUniLator next has just been printed.
ctrl-c leaves at once, wherever it is. During a download or a write that
means the work stops too, not just the screen: the loops that are reading and
writing are told to stop, so it is gone in the time one chunk takes rather than
at the end of three gigabytes. It then says what that left behind — an SD card
part written has to be written again.
What works today
qunilator-installer -get_sdcards
qunilator-installer -board qbone -latest -device /dev/disk4
qunilator-installer -board unibone -fetch
qunilator-installer -write qbone-dist.img.zst -device /dev/disk4 [-digest ]
qunilator-installer -verify qbone-dist.img.zst -device /dev/disk4
qunilator-installer -write_seed -user [-hostname ] [-ssh_key ]
qunilator-installer -version
-board says which QUniLator, because a release carries an image per bus and the
emulator is compiled for one of them: qbone drives QBUS, unibone drives
UNIBUS. Writing the wrong one produces an SD card that boots and then finds no cape
it recognises.
-latest fetches that image from the newest release and writes it in one go;
-fetch stops after the download; -release v1.16.0 names another one. The
digest published beside the image comes down with it, so the write checks the
download without being asked. An image already in the directory at its published
size is not fetched again — half a gigabyte should not come down twice because a
second SD card is being written.
-write decompresses by extension — .img, .img.zst as the releases ship,
.img.xz, .img.gz, .img.bz2, .zip — writes to the SD card, and then checks
it. Progress and rate are printed as it goes, measured against the image's own
size, which a .zst records in its frame header and an .xz in its index.
The decompressor keeps up with the SD card. A release ships .img.zst,
which this decompresses at around 250 MB/s — ten times what a card takes — on
one processor and with no tool installed beside it. The .img.xz that older
releases carry still writes, at the 5-25 MB/s a pure Go xz decoder manages.
The decompressing runs ahead of the writing, a few chunks at a time, so a
write costs the slower of the two rather than both added together.
Run with sudo, it does not leave root-owned files behind. Only the raw
device ever needs the privilege; a downloaded image and its digest are given
back to whoever invoked sudo, so they can be deleted, moved or written over
afterwards like anything else in that directory.
The download is checked before anything is written. A QUniLator release
publishes <img />.sha256 beside each image; download both and the digest is
found and checked without being asked for — seconds against the minutes an SD card
takes to be written with a bad image. -digest gives one by hand, and an
image with no digest beside it says so and writes anyway.
The check reads a spread of windows rather than the whole SD card. The image is
hashed in 1 MB windows as it streams past on its way in, and afterwards those
windows are read back — about 35 MB in 33 places for a 3.5 GB image, seconds
instead of minutes. What goes wrong with SD cards is coarse: one that accepts
writes and stores nothing, a counterfeit that claims a size it has not got and
wraps its writes around, a reader dropping data. All three change what those
windows hold, and the window at offset zero is checked last, because that is the
one a counterfeit has overwritten with the image's end. What sampling gives up
is a single bad block between windows; -full_verify reads everything back and
costs about as long again as the write. -no_verify skips the check.
The check happens through the same open device as the write, and that matters: a
desktop mounts an SD card the moment the device is closed and writes its own
housekeeping into the first filesystem it finds. On macOS that lands in the FAT32
free-cluster count a thousand bytes into the partition, so an SD card that was
written perfectly no longer matches the image seconds later.
That is also why -verify on its own — a full comparison, run separately —
may report a difference in that same place: it compares the SD card as it is now,
and if the desktop has mounted it since, the free-space bookkeeping has moved.
The difference is real and the tool reports it rather than papering over it.
Every refusal comes before any work, and what the SD card is comes first of all:
pointing at a system disk is answered with that, not with "run me again with
sudo" — which somebody would otherwise do, and find out afterwards.
-write_seed is the half that needs no raw disk access and no administrator
rights: point it at the SD card's mounted BOOT volume — /Volumes/BOOT on
macOS, E:\ on Windows, /media/you/BOOT on Linux — and it writes the setup
file that the first boot reads. Useful on its own if you wrote the image with
something else.
The password is not written down
A QUniLator carries one identity: the same name and password open the web
interface, the SMB, FTP and SFTP shares of the image library, and an ssh
login. Those three check it in three different shapes, so the tool derives all
three and writes those instead of the password:
| checked by | shape |
|---|
| the web interface | PBKDF2-HMAC-SHA256, 120000 rounds over a random 16-byte salt |
the Linux account, and ssh | crypt(3), $6$ sha512crypt |
| the file shares | NT hash — MD4 over the password in UTF-16LE |
None of the three yields the other two, which is why all three are written and
why a file carrying only one is refused by the reader. The password itself
appears nowhere on the SD card, which matters because passwords get reused and a
FAT partition your workstation mounts is one an unrelated backup may carry off.
The file is deleted by the boot that reads it.
Getting it
Binaries are attached to each release, one per platform:
| |
|---|
| macOS | qunilator-installer-darwin-arm64, -darwin-amd64 |
| Linux | qunilator-installer-linux-amd64, -linux-arm64, -linux-arm |
| Windows | qunilator-installer-windows-amd64.exe |
SHA256SUMS is published beside them. On macOS the binaries are not signed yet,
so clear the quarantine flag before running one:
xattr -d com.apple.quarantine qunilator-installer-darwin-arm64.
Building
go build ./cmd/qunilator-installer
go test ./...
One static binary per platform, cross-compiled from one machine, no runtime to
install.
What is not built yet
- Writing an SD card on Windows. The code is there — physical drives are
enumerated and described through
IOCTL_STORAGE_QUERY_PROPERTY, the volumes
on the card are locked and dismounted for as long as the drive is open, and
the drive is opened with FILE_FLAG_NO_BUFFERING so a read after a write
reads the SD card. A Windows run reaches the write: the listing, the
questions, the download and the check of it happen there. What the write
itself does has not been seen. The parts that could be pinned down without
Windows are — the sector splitting and the buffer alignment that path depends
on are ordinary Go, and tested on every platform. Treat the rest as unproven
until somebody has an SD card that boots from it.
- Signing and notarisation, without which macOS and Windows meet the
operator with a warning at the worst possible moment.
Where the format is defined
The reader is 10.05_web/2_src/webseed.cpp in
QUniLator, and the format is documented
in the manual's install page. The
parameters above are that reader's, so changing them there is a
config_version bump here.