DevOps for the Sinclair Spectrum - Part 1

Updated:

This article is part of a series

  • Table Of Contents - Full table of contents in Part 1
  • Part 1 - Introduction, hardware, development environment, Windows/Linux buildchain and tools
  • Part 2 - The server environment and building the first prototype
  • Part 3 - The backend server daemon, pipelines and unit tests
  • Part 4 - Wrap-up, other sites and final thoughts

Discussion on Hacker News Discussion on lobste.rs

NOTE : You can now access the TNFS site through an emulated ZX Spectrum running in a web browser at jsspeccy.markround.com!

Intro

When I was around 8-9 years old, I received a Sinclair ZX Spectrum home computer for my birthday. One of my earliest memories I remember is sitting with my Dad, reading the manual to work out the magic commands to load the games from cassette tape. Whilst the games (Manic Miner and Starstrike II were the two I remember trying out first) were cutting-edge for a 1980s computer system and I spent many hours playing them, I was always more interested in the manual. Like many systems of the day which “booted” straight into a BASIC prompt, the documentation included a full introduction to programming the Speccy in BASIC as well as some tantalising (but over-my-head at the time) memory maps and details of coding in Z80 assembly language.

To say I was obsessed would be an understatement: My Mum would diligently make computer-themed birthday cakes pretty much every year after that, as I pored over manuals, magazines and learned as much as I could about this fascinating new world. Of course, I eventually upgraded to a Commodore Amiga and then migrated to the PC world while at University, where the world of Unix systems opened up. It’s been a long time since those first days pecking at a rubbery keyboard in the early ’80s, but I never forgot the humble system that started me off on my journey and ultimately led me into an awesome career.

We’re fast approaching the 40th birthday of the Sinclair Spectrum in 2022, and to keep myself occupied during COVID lockdowns I decided it would be a lot of fun to go back and re-visit the computer that started it all for me. I set about coding and building the infrastructure for a Spectrum-based community project (website at tnfs.markround.com) incorporating my current-day tools and knowledge, hence the title of this series of posts.

The enterprise grew into a curious mix of old and new: Container-based pipelines with Ruby server-side components, all interacting with Spectrum BASIC and z80 assembly code, running on real 1980s hardware with a TCP/IP connection. If you’ve ever wondered how to unit-test Sinclair BASIC programs in GitOps pipelines running on Kubernetes clusters, this is the set of articles for you ;)

The site has already attracted a community of users and even got featured on The Spectrum Show #107!

The above video is a great overview of using an older version of the site as well as some of the hardware details, skip to around 3:33 in to see it in action.

This series of articles covers the technical details and “behind the scenes” processes of building a Spectrum platform/application using modern toolsets. As this is one of the longest things I’ve written, I’ve split it into multiple posts and included a Table Of Contents below so you can jump between sections. There’s also a set of permalinks at the top of each article linking to the other parts to make navigation easier.

Table Of Contents

The hardware

Although a very simple machine, the ZX Spectrum did go through a number of revisions before it was discontinued in 1992. The final model produced by Amstrad in the UK was the +3, sporting an obscure 3-inch floppy drive with 173Kb capacity, “proper” keyboard, 128Kb RAM and a AY-3-8912 sound chip (essential for any scene demos!) All this was based around the same 8-bit Z80 processor running at 3.5Mhz as the original Spectrum. This was the last model I owned as a kid, and when I found one in good quality on eBay I snapped it up.

The floppy drive had been replaced with a Gotek drive that emulates floppies through .DSK files saved on a FAT-formatted USB stick. This provided a way to get code and software into the machine without having to load from cassette tape: I could download DSK games from the internet, or create my own code in an emulator and then transfer things over via “sneakernet”.

Another great addition which helps the Speccy to exist in a modern world is the ZX-VGA-JOY add-on card I also purchased from eBay. This card connects to the Speccy’s rear expansion I/O port and provides a Kempston-compatible joystick interface which works nicely with my collection of Amiga joysticks and gamepads. But most importantly, it also sends crystal-clear video signals over a standard VGA connector. The Speccy existed in a world where most users would have it hooked up to an old fuzzy CRT analogue TV set, but using this card means I can simply hook it up to any of the modern flat-screen displays I have. I currently have it connected to an ASUS VB199T-P which I’ve also used with my Amiga for that authentic old-school 5:4 square aspect ratio.

Development

As much as I wanted to sit at the speccy and write code directly on it, I found the experience just too frustrating. For one thing, my typing over the years has got way faster than my young “pecking slowly at the keys” tempo and I found the poor old computer just couldn’t keep up. I was getting constant missed letters along with that infurating syntax error “BOOP” noise. I also found it very hard to adapt to the odd keyboard layout again and found myself accidentally mashing the GRAPH or EDIT keys way too often.

While I could have adapted my typing habits, a more pressing matter was the fact that the Spectranet card I later added (see below!) forces the Spectrum into 48k mode. This meant any programming had to be done using the horribly convoluted keyword entry system that was one of the original Spectrum’s “lovable” quirks. And by “loveable”, I mean “rage inducing”. The 48K model keyboard looked like this:

You’ll notice that the keys have lots of BASIC keywords printed on them in different colours. This was a laughable attempt to make BASIC entry on the squishy rubber “dead flesh” keyboard faster - what it means in practice is that when in Keyword mode, you press e.g. the J key and the LOAD keyword appears. To get the red characters, you hold down SYMBOL SHIFT, so most kids quickly learned the J-SYMBOL SHIFT-P-P combo to enter LOAD "" to load games from tape.

All well and good, but then to get at some of the other keywords in green, or the ones above and below the key, you then had to press various combinations of SYMBOL SHIFT and CAPS SHIFT; sometimes together, sometimes one after the other. All guaranteed to bring on more frantic beeping than a newbie trying to exit vi as you get the combinations wrong; the Red Mist descending, followed closely by crippling wrist pain in a few minutes flat.

To make matters worse, my Spectrum +3 doesn’t actually have any of the keyword legends printed on the keyboard, the expectation being that you’d spend your time in the new improved “modern” (for 1987) BASIC editor. Which means either sitting at the keyboard with a photo of a 48K keyboard next to you for reference, or giving up and choosing to do the bulk of your development on another system. I’m sorry, 8-year old me, but I’ve just got too used to the luxuries of modern development. I chose the latter option.

Spectranet

So now I had to work out a way of getting code written on my laptop into the Spectrum. There’s no end of development options and workflows when using an emulator, but I wanted to get my code over to the Speccy with the minimum of fuss for testing on the real hardware. My first thought was to find a way of producing .TAP or .TZX files, writing them to a SD card and then plugging them into an interface such as a DivMMC. An alternative route might also have been to use the Gotek drive fitted in my +3, and transfer .DSK disk image files over via a USB stick. But that’s a lot of fiddling around and trotting forwards and backwards between machines.

That’s when I discovered the Spectranet card. It adds an Ethernet interface to a Spectrum and comes with a great BIOS that adds a bunch of commands to Spectrum BASIC. It mounts filesystems into the Spectrum using a simple “TNFS” protocol, so that they function much the same as a SMB or NFS share might on Windows or Unix systems. The server process runs on a Linux/Windows/Mac system, and the Spectrum can then accesses files on the TNFS server as if they were attached to the Spectrum directly. It’s sort of like a massive network-attached hard disk.

For example, you can use %cat to catalogue the files in a filesystem (like ls or dir) and %tapein loads a .TAP image as a virtual cassette tape. Other commands are provided which allow you to load code or snapshots directly into the Spectrum’s memory as well as interact with TCP services from BASIC. As mentioned above, there’s a great review of the card and functions on The Spectrum Show YouTube channel.

To my mind, this is perhaps the most elegant solution for developing Spectrum software on a PC: You can build and develop locally, test in an emulator (even including Spectranet emulation) and easily deploy to a server and run on the real hardware in seconds. Best of all, if you save your code as a .TAP-format file named boot.zx in the root of your TNFS server, you can load and run it automatically when you turn your Spectrum on via an autoboot function.

Note as of end of December 2021: Spectranet cards are quite hard to come by now, but you can still find some clones on eBay (e.g. “ShamaZX NET”). You can always use emulation though in the meantime. UPDATE 16th February 2022: ByteDelight just announced on their Facebook group that they have ordered another batch of cards! Keep checking out their website for any updates.

There are a few public TNFS sites on the Internet where people have taken this idea and run with it; it’s kind of like a website or BBS system for Spectrums; the main difference being that the code is downloaded and run directly on your machine instead of being accessed via a terminal emulator or browser. I also review and provide screenshots for my favourite sites in Part 4.

This is where the idea occured to me: My BASIC skills may be old and rusty, and I never progressed much beyond rudimentary Z80 assembly language - but I was pretty sure I could build a pretty nice TNFS site. I’d always wanted to run a BBS back when I first had my Amiga (and the ‘Scene’ was all the rage), so this seemed like an ideal fun project that would serve as a great re-introduction to Spectrum coding, and give something back to the community that had influenced me so much when I was growing up. To work!

Windows/Linux Buildchain

My current development environment of choice is Windows 11/WSL2/VScode/Docker. I’ve been through both Mac and Linux (since 2000 - I do not miss having to recompile my kernel for sound card support…) on the desktop, and this is just what gives me minimal grief at the moment. That’s not to say I’m overly fond of it, but things tend to “just work”. You can see an example screenshot of a typical development session below:

Using WSL and Docker means I’m free to easily bounce between my laptop and Linux/Kubernetes-based servers and keep things consistent. As I wanted to eventually automate everything (notice the Concourse pipelines running in the background - I’ll cover all that in later parts), I went looking for something that could be run in a container that could compile my code alongside the rest of the infrastructure pipelines.

Emulators

Since the Spectrum has been around for such a long time, there’s no shortage of emulators available, covering pretty much every platform from old DOS-based things to modern mobile operating systems. My current preferred emulator is the venerable Free Unix Spectrum Emulator FUSE, which despite the name, has been ported to Windows, MacOS, and even Haiku and AmigaOS. Apart from the benefit of being available on all of my systems, it’s also one of the more complete emulators out there and can simulate a very complete and wide-ranging set of Spectrum models/clones and additional expansion cards including the Spectranet. You can see it displaying the blue & white TNFS pre-boot sequence in the screenshot above.

Like the other tools I used, this could also be run from within a container, using Xvfb to provide a pseudo graphical environment. I’ll talk about that a bit more in Part 3, though as I came up with one of my favourite hacks with it!

Compilers

As I’d be writing my code in VSCode, I needed something that could turn my BASIC and Assembler programs into a format that the Spectrum could load. Fortunately, there’s quite a few such projects and I ended up settling on the Pasmo cross-assembler and zmakebas for my BASIC code. Both of which output files in .TAP format which can be loaded into an emulator or renamed to .ZX and dropped into a TNFS share for loading into a real Spectrum.

Pasmo

I’m only slowly starting to dip back into low-level coding again, but there’s a few useful pre-written Z80 assembly routines I’ve used in my site such as the handy 64-column text driver used in the articles and other sections to work around the somewhat limited and chunky 32-column character set used by default on the Speccy. But more on that later…

Pasmo is included in many Linux distributions, including Debian and Ubuntu, which I tend to use for all my projects and as Docker base images. It’s therefore a quick apt-get away and from there it’s simple to compile your code to a TAP format binary which can be used by emulators such as Fuse or loaded directly into Spectranet systems:

$ pasmo --tap 64.asm 64.tap

zmakebas

There’s a bunch of tools available to turn plain text Sinclair BASIC listings into executable formats for emulators and real hardware alike. I started off using txt2bas, but this had a number of drawbacks for my way of working. I settled on the awesome zmakebas at a suggestion of a user on the Spectrum Computing Forums. One of the key advantages this tool had was it was written in portable C so I could use it on all my systems. It also supports using indentation and labels instead of line-numbers in Sinclair BASIC. This is another massive improvement which as far as I’m concerned is a must-have for bringing things into a vaguely more modern era.

Without this feature, you’d be stuck with the way BASIC worked back in the ’80s: All lines had to start with a unique, incrementing number. When you wanted to call a sub-routine, you’d GOSUB 1000 for example, and just have to remember that 1000 was your screen-clearing routine or whatever. Horrible, and a nightmare when you’re writing terrible broken code rapidly iterating prototypes. With this, I can now write in a way that feels more natural after years with more modern programming languages and environments. So for example, here’s an abbreviated start of my code that draws the main welcome screen on the TNFS site:

@init:
  let savescroll=1
  let offset=6
  let c=20: gosub @clearscreen

@udg:
  restore
  data 0,0,0,255,0,0,0,0
  for n=0 to 7
      read char
      poke usr chr$ 145+n, char
  next n
  ...
  ...

@mainmenu1:
  let offset=6
  let c=17: gosub @clearscreen
  print at 4,3; paper 1; ink 7; "                         "
  ...
  ...

If you look at the screenshot taken from the Speccy, you’ll see how the labels and subroutine calls have all been automatically translated to line numbers, and the indentation has been flattened.

I had a few friendly chats with that user from the Spectrum Computing Forums and we worked on a few additions to zmakebas together, adding support for Spectranet commands and fixing a few other minor bugs. Thanks for all the help, “flatduckrecords”!

It’s simple to build from that repo on a stock distribution with build tools installed: A quick make leaves you with a binary that you can copy into your $PATH.

z88dk

z88dk is a C compiler targeting the z80 (yes, really!). It can output executable code for a variety of old systems, the Spectrum included. I haven’t yet written any C code for this project, but I did need a compiler to build SnapCTerm so I could add a telnet console to the site, so a user can jump straight into the many telnet BBSes still running.

As a bonus, they have a Docker image along with their other Downloads.

Docker

I’m a big believer in creating containerised reproducable builds for my development tools. It really helps keep things portable among any system with a OCI runtime available, and has the great advantage that they can also be used in CI/CD workflows.

Using the z88dk container as my base, I built a new tools container that included Pasmo and my patched zmakebas. I’ve added this container to my CI/CD pipelines and it’s publicly available on Docker Hub at markdr/zxdev, if you want to try any of this yourself. Using this image I can build my code in the container, mapping in the directory holding my code like so:

docker run --rm -ti \
  -v $PWD:/src \
  --user $(id -u):$(id -g) \
  markdr/zxdev \
  pasmo --tap 64.asm 64.tap

Graphics

Although the site is written primarily in Sinclair BASIC, I still wanted to jazz it up a bit. Unfortunately, the Spectrum is not exactly a graphical prodigy; even amongst it’s peers of the time it compared poorly against the C64 and CPC range for example. Although I’d argue its ridiculous attribute-clash issues and garish palette lent the Spectrum a certain “crap, but in a loveable way” aesthetic charm! It’s amazing what can be done given limitations such as “no more than 2 colours in an 8x8 character square”.

Pixel art

A quick win to make the site look more appealing is a simple header image and placeholder for the menu/section title. This does mean sacrificing 5 text rows out of the available 22 (the last 2 8-pixel high rows of the standard Speccy BASIC screenmode are reserved for error messages and user input) but I think I still managed to get a half-decent effect. I used ZX Paintbrush running under Windows to draw the font-based logo.

I used a TTF font I’d used before on my musical projects and coloured it in. For Nostalgia’s sake, I saved the image in the Spectrum’s native SCREEN$ format (a dump of the 6912-byte length screen memory from address 16384 onwards - I still remember that from back then!) and loaded it into The Artist II, which I used as a kid.

After a few years on my original Spectrum journey, I’d saved up for a Datel Electronics Mouse which came with a copy of The Artist II. It cost me the princely sum of £49 which was a big deal to 11-year old me. Well worth it though! I remember hours playing with making graphics and feeling like I had a “proper” computer like the Amiga I eventually upgraded to. I used The Artist running through an emulator (I still need to find a Datel mouse for my real Speccy - if you have one, contact me!) to finish off the “stars” and dithering detail on the menu bar which brought back a lot of memories.

UDGs

Another way a text-mode BASIC program can look a little snazzier is to make use of User Defined Graphics which allow you to extend the Speccy’s character set with your own 8x8 blocks. If you look at the menu code shown earlier, you’ll see the POKE commands being used to define the dithered separator lines used in the main menu. Fortunately, there are again easier options today than the original method of hand-drawing things out on school graph-paper and converting from binary!

I used a web-based UDG builder to visually build up the required values to define the character. You can see it in the screenshot being used to create the yellow icon I use to indicate a file has comments attached to it (DATA 0,126,129,129,129,110,48,96 if you were curious…)

64-character mode

To fit more text on the screen than the blocky 32x24 text area usually allows, I used a 64-column text driver written in Z80 assembly language. Although it’s based on code from 1984 (just 2 years after the Speccy’s release), I really like how compact and straight-forward it is when compared to some of the more extravagant (yet admittedly very cool) print routines from later years. It’s also extremely easy to incorporate into BASIC code; The routine in BASIC code (using zmakebas syntax) to load the assembled TAP-format binary over a Spectranet TNFS connection is:

1
2
3
4
@init:
  clear 49999
  %load "/64.tap" code
  randomize usr 50000

It makes use of the little-used streams facility built into Spectrum BASIC so you can simply print to channel #4 to fit a lot more characters on the screen at the expense of a slight decrease in readability:

1
print #4; "Hello 64-character world!"

I managed to get colour into this driver with some limitations (more details in Part 3) and also used little tricks from the past like building the menu separator bar at the bottom using coloured undercore characters. In short, I tried to make the site as appealing as it could be given the limitations I had to work with; not least being my own rusty 8-bit graphical “skills”!

Next steps

Continue on to Part 2 where I discuss the server-side components and build the first iteration of the site.