markround.com

Code, Amiga, Sound Engineering and other things of interest...

SetCmd Amiga Development Part 2

| Comments

As I mentioned in my last post, I’ve been developing a new “SetCmd” tool for my Amiga systems, both for my own practical usage and also to serve as a re-introduction to the Amiga developer environment. This series of blog posts will cover the progress of this tool, as well as explore the challenges and technologies from my perspective of a returning AmigaOS fan.

I’ve learned a lot over the past few days and even though the tool is currently very much a “prototype” AmigaDOS script, it is working well enough at this stage that it has really helped my day-to-day activities at the Amiga shell. There’s a few screenshots at the end of this post showing my current progress, but to start off with I wanted to share my first “brain dump” covering some of the things I have picked up and/or re-learned :

Installer progress

As I want to make this tool as Amiga-native as possible, I also want to provide documentation in AmigaGuide format (an interactive hypertext-based system, somewhat similar to HTML), and an installer using the standard Amiga Installer utility. I decided to tackle the installer script first, and have already learned quite a bit.

There is apparently a new Python-based installer system present in AmigaOS 4 which can create much nicer and more modern-looking installer wizards (such as the SDK installer – I looked at this amongst other things in another post), but I chose to stick with the “classic” installer tool for several reasons:

  • At some point, I would also like to provide this tool for classic AmigaOS 3.x so I don’t want to make too many choices at this point that limit my options.
  • There are many Installer scripts to use as examples.
  • Pretty much every Amiga user is familiar with the Installer tool.
  • I remember it well from my classic Amiga days and always wanted to learn it!

I started off by examining some existing Installer scripts to get an idea of what the language looks like. It’s an odd sort of LISP-based system, which uses some patterns I found extremely jarring at first. For example, in most languages I am comfortable with, comparision is done like if thing == value. In the Installer language, it looks instead like (if (= thing value)). Still, once I’d got used to that it wasn’t too bad, and I discovered the language is pretty full-featured with procedures, control statements and lots of useful helper commands/functions.

As an example, here’s a block of code which copies the setcmd program file over to a previously created directory, the value of which has been assigned to the #dname variable:

1
2
3
4
5
6
7
8
(copyfiles
  (source "setcmd")
  (dest #dname)
  (prompt ("Copy SetCmd program file?"))
  (confirm "expert")
  (all)
  (help @copyfiles-help)
)

And there’s also a variant which can copy wildcards which is handy when copying .info files along with something – note the pattern parameter is now used and source is empty:

1
2
3
4
5
6
7
8
(copyfiles
  (source "")
  (pattern "SetCmd.guide#?")
  (dest #dname)
  (prompt ("Copy SetCmd AmigaGuide documentation?"))
  (confirm "expert")
  (help @copyfiles-help)
)

And for running commands, you can use the run command, along with cat (short for “concatenate”) to build up the command string :

1
(run (cat "C:MakeLink FROM " #dname "/cmds/setcmd/release TO SETCMD:setcmd SOFT"))

I initially found it very difficult to find any definitive guide for this language as nothing was showing up on web searches. After struggling through with trial and error and only example scripts to help, I remembered that this is Amiga-land so there’s probably documentation in a native format. And sure enough, searching Aminet brought me to the Installer dev package. It dates from around 1996 which is shortly before I first left the Amiga scene but still seems to be a relevant source of information. The supplied Installer.guide is a great resource, and you can see it open below along with my Installer work in progress:

I also discovered that you can pass useful information and configuration which controls the Installer tool through standard Amiga tooltypes:

The more I get back into the Amiga, the more I appreciate these details and features that never seemed to make it into other systems!

Version control

As I use version control (usually git) for all my other projects on Unix, Mac and other systems like Haiku[1], I wanted to do the same for this project. However, this is something I’m kind of stuck with at the moment, so I’m curious what other Amiga developers use. Simply put, the only tools I could find (after an admittedly short search) were either very old or incomplete. There is for example, a port of Subversion to AmigaOS 4, but I consider that a technical step back and the port is already 10 years old.

My options for using git seem quite limited – all I could find was sgit. This is sort of working OK at the moment, I have managed to initialise a local repository and commit code but various things don’t seem to work quite how I would expect. I still haven’t worked out how to add and push to an origin at Github.com, for example. Diffs and .gitignore files also don’t seem to work although that’s probably something I’ve done wrong, and many of the commands don’t accept the same arguments as I’m used to.

Still, I’m extremely grateful for it as having a limited SCM system is infinitely better than the old approach of millions of files all called things like final_1.bak-v2-bug-fix_DO_NOT_DELETE which you can never remember what they are or why they were labelled like that!

I’ll keep investigating – one option might be to see if the bundled Python installation is up to running Mercurial. I once submitted a patch for this to work on Irix, so maybe in the future they’ll get another patch from The Weirdo Who Likes Old Systems ;)

Remote access

This is more a “nice-to-have” but I admit to being stumped right now. The only way I have of developing this tool is by sitting at my Amiga X5000 which is fun (especially with AmigaAmp blasting some classic 90’s MODs in the background!), but it would be nice to be able to continue on with my work while sitting with a laptop in my living room. So far, the only route seems to be emulation on a Mac/Windows/Linux laptop which is not ideal.

I can’t seem to find a SSH or Telnet server for AmigaOS 4 (plenty of clients are available) which would have let me work from the shell remotely. There is a port of VNC, but using that for development work over Wifi is painful. I’d appreciate it if anyone has any insights – is there anything obvious I’m overlooking ?

Documentation

Brief shout out to whoever maintains the AmigaOS Documentation Wiki – this has been an invaluable resource. I managed to get myself back up to speed with AmigaDOS after a multi-decade absence using the great documentation, particularly these fantastic guides :

  • AmigaDOS Commands – all commands described along with plenty of helpful usage examples and tips. The documentation for the list command is a revelation – I had always been mystified by that command back in the day!
  • Using Scripts – lots of great examples and tips on how to write scripts, accept arguments and so on.
  • AmigaDOS Advanced Features – more useful tips including a whole section on escape codes for colourised output. I remember working these all out by hand on my A1200 and working out what codes I needed to build up ASCII-art screens for my old scene group!

I saved all these guides locally as PDFs using the “Save as PDF” option from the Odyssey browser. I would have loved to have a resource like this back when I was starting out on my old Amiga, so again a big “Thank You!” to whoever is responsible for maintaining this wiki.

Onwards

I am now able to use the setcmd running through itself to develop it. This has been very useful and has further validated my need for such a tool – take a look at the example below, where the setcmd installation is pointing to my development version in my Work: volume :

But now I want to switch to my release version, where I’ve installed it to my usual Software:Programs/ directory :

Note that the version number changes in the header of the command, and setcmd show setcmd shows the versions changing. Also note the liberal use of AmigaDOS Assigns – one of my favourite features of this OS, and one that I really miss on other systems. VMS had something similar with System Logicals, and of course Unix systems have symlinks but they don’t work in the same way, and aren’t nearly as useful or efficient.

My next steps are to finish the Installer script, then learn enough AmigaGuide markup so I can together a set of documentation for my tool. A few more bug fixes and general polishing, and I should be ready to release it to the world! This is just the start, however. I have more plans like supporting AmigaOS3.x – probably limited to OS3.9 as that’s all I have readily available at the moment but I do have a set of 3.1.4 ROMs being shipped.

I’d also like to try out different editors, and use my setcmd tool as a jumping-off point into re-learning C on the Amiga, eventually re-implementing it as a pure C program instead of the “prototype” script.

Part of the reason I started off on this journey was to develop my skills to a point where I can eventually start contributing more to the Amiga scene anyway, so hopefully learning enough C (and working out the build process, accessing Amiga libraries and so on) will get me to a point where I can help bring some more tools across to our platform.

Until next time…

[1]=Haiku is a recreation of the BeOS, which I used extensively after I left the Amiga scene. I’ll write more about this OS in some other posts, but it’s nothing short of amazing what this small group of dedicated developers has accomplished. I’ve also been continuing in my mission to submit patches to open-source projects for obscure platforms ;)

Amiga Project in Progress - Setcmd

| Comments

I’ve been having a lot of fun with my X5000 over the past few weeks (more blog posts to come!) but I’ve been working on something recently that I wanted to share. I’ve been enjoying re-learning AmigaDOS and as an exercise for myself, set about building a tool I plan on releasing in the near future. Inspired by some Linux distributions’ “alternatives” system, It’s called setcmd (short for “Set Command”) and lets you easily and quickly switch between different versions of a command while in the AmigaDOS shell.

It’s been particularly useful for me as I experiment with different versions of the UAE emulator. I wanted to share a quick “sneak peak” at my work so far, but bear in mind this is all a “work in progress” and I have a lot more to learn! Anyway, here’s the “usage” screen that provides an overview of the program options:

And here’s a quick walkthrough demonstrating how I use it: First, let’s add a new command under setcmd control. I picked “UAE” for this test, as I frequently want to switch between versions while I’m testing compatibility with my various emulated systems:

I added the command, and now setcmd list shows that there is a new uae command available (note that the setcmd script is also managed by setcmd, so I can easily test new versions!)

A setcmd show uae shows that it’s just a “stub” entry for now; As no versions have been added, if we try to run it we just get a message telling us to add some versions.

OK, so let’s do that – I have two main builds of UAE I switch between: the system-provided one, and the new UAE-1.0.0 I downloaded from OS4Depot:

I’ve now setup two “versions” of UAE, one called system, and another called 1.0 – versions in setcmd can be any format so it’s easier for you to label them according to your needs. So, let’s see it in action! First, let’s try setting the version to “system”, and run the which command to verify that it’s pointing to the correct location:

And now, let’s switch it to the “1.0” version and again verify the path has changed:

There you have it! I can now easily and quickly switch between multiple versions of tools. It’s been a really fun experience diving back into Amiga development again, and I hope to release setcmd on OS4Depot.net soon. First though, I have to tidy it up, remove some bugs and I also want to learn the AmigaGuide and Installer languages so I can provide documentation and an installer to turn this into a “proper” tool. I look forward to my first proper Amiga software release in decades!

Backups on the Amiga X5000

| Comments

Happy New Year everyone! I’ve got big plans for my Amiga projects in 2019, but thought I’d start off the New Year with a blog post on a not-particularly “exciting” topic, but an important one nonetheless: Backups. As I am experimenting more with my X5000 and Amiga OS 4.1, I’ve been getting particularly “twitchy” that I didn’t have a solid backup/restore plan in place, particularly as some of my experiments will invariably go wrong and I’ll need a way to roll back my changes to a known-good state. I spent a few days researching and implementing a backup strategy that’s ideal for my needs and hopefully there will be something of use to other Amiga owners too.

My requirements were:

  • Create a spare bootable system partition that always has a working copy of Amiga OS 4.1 complete with all my settings and tools
  • Backup my system and other partitions to directories on another disk
  • Create archive files of these backups so they can be stored on another system

I’ll run through my steps for each of these areas in turn. Firstly, I wanted to create a spare bootable partition so that even if I totally screw up something in the AmigaOS boot process, I can still get back to a working system. To do this, I decided to use some of the spare space I had left unpartitioned when I set up my X5000.

Preparing the backup boot partition

The first step was to run Media Toolbox and select my internal boot device which is connected to the on-board p5020 SATA ports:

Next, I selected my boot device which is the 256GB Sandisk SSD installed by AmigaKit:

And here’s what my partition layout looked like, with a decent amount of space free for a secondary boot volume:

I know AmigaOS is a pretty light operating system, but I first checked how much space was actually in use on my current System volume using the AmigaDOS info command:

Not bad, just around 900Mb in use. I decided to create a 5Gb partition for my new boot volume; this was way more than I needed but then disk space is cheap, and it’s always better to have spare capacity in the future:

I named this partition DH3 as I’d already used DH0 to DH2 for my System, Software, and Work volumes. I then set the filesystem to SFS2 as that’s what I’m using on all my other volumes and it’s proven to be reliable so far. It doesn’t however have any recovery tools unlike NGFS, which means that a backup plan is even more important as I’ll have no choice other than to reformat and restore data if any corruption occurs!

The next step was to make sure that I had set the various options needed to allow booting from this new partition. I ticked the “Bootable” checkbox, and gave it a Boot Priority of -20, as that is lower than my current System: volume, which has a boot priority of -15. So this means it should only be picked automatically as a boot volume if my main system volume is unavailable:

With that done, I clicked on “OK – accept changes”, and back at the unit selection screen I clicked on “Save to disk” to write the new partition table to the SSD:

I then formatted the volume (using “quick format” – never do a full format of a SSD!) in Workbench, named it SystemBackup and I was ready to start copying data over.

Making the backup

I wanted to make the backups using something a little more efficient than the plain old AmigaDOS COPY ALL. On Unix systems, there’s a fantastic tool called rsync that can be used to archive directories by only copying over the changes between them. This means that the first time may take a little while as it has to do a full copy, but after that any subsequent runs will be much faster as only the delta will be transferred.

I tried a few of the available tools that can do incremental copies (as well as correctly preserve all the AmigaDOS file attributes) and eventually settled on the fantastic BackUp from OnyxSoft. It’s a 68k binary but works very well on both classic and NG systems – another great example of how AmigaOS 4 can run old system-legal classic software. It has a nice GUI for one-off operations and can also be scripted from the CLI (more on that later). I unpacked this tool into my usual Software:Programs directory, and ran the first backup manually. I did this by picking the Source and Target volumes and clicked the “Backup” button:

This started the backup running and produced some short output showing which files were being worked on:

And a short while later, the backup had completed:

I then ran the backup again to verify that incremental backups were working; sure enough, it completed in a fraction of the time and only transferred modified files.

Uboot modifications

Now I had my backup boot partition ready, I somehow had to find a way to boot into it. On my classic Amigas this would have been straightforward, using the “early boot menu” you can access by holding down both mouse buttons when powering on the system. There didn’t appear to be anything like this in the Uboot firmware on the X5000, though. The only options I saw on that menu are for booting into AmigaOS, MorphOS or Linux. The “Boot options” submenu just lets you pick whether to boot from HD, USB or optical media. So I knew I’d have to mess around with Uboot configuration directly, using the “Command Line” option from the boot menu.

Update : Trevor Dickinson of A-EON has provided a comment below that corrects my assumption above! There is apparently an early boot menu, you just have to press the mouse buttons together at the right time. Many thanks Trevor, for the update! My steps below are still valid, but it’s good to know about this extra environment. I’ll explore this in more detail later on…

The first thing I wanted to do was run a printenv command from Uboot and note the values. This is very important as it’s quite possible to make your system unbootable by changing things here, so having a record of settings is very important in case you need to roll back your changes:

The first parameter caught my eye – amigaboot_quiet. I had no idea what this did, but I hoped it would show some more verbose output during boot. I thought I might then see something in the boot logs which might point me in the right direction to set up multi-partition boot. So I set this value to N (hopefully to disable “quiet booting”) using the setenv command, and then ran boota to start the AmigaOS boot process off:

By sheer luck, this turned out to be exactly the parameter I was looking for! Now, when the boot process starts, I get a menu allowing me to pick a boot volume:

I was really happy that my first guess seemed to take me in the right direction :) I then simply used the cursor keys to move the selector arrow down to select the new backup boot volume, and pressed Return:

This started booting and loading kernel modules:

And then a few moments later, I got the OS 4.1 splash screen and then my Workbench desktop appeared. I celebrated briefly, but I quickly noticed there was a slight problem as it didn’t appear to have worked as I had expected. The system did boot, but when I checked my assigns I saw that SYS: (the boot volume) was still pointing to my System volume on DH0, along with all my other assigns:

This meant that although it appeared that the Kickstart kernel and modules etc. had been loaded from my backup partition on DH3, the boot had continued from DH0, run the startup-sequence from there and so on. So I was getting closer, but there was still obviously a last piece of the puzzle to put together.

I then went back to reading more about Uboot and the AmigaOS 4 boot process. I eventually stumbled upon a thread on the Amigans.net forums where someone was trying to do exactly the same thing as I was!

It turns out there’s one more important variable to set: os4_bootdevice should be set to auto. With this in place, the boot sequence will continue from whichever volume the Kickstart and Kicklayout file were loaded from. Otherwise, it appears the boot process will continue from whichever partition has the highest boot priority – which would always be my original installation on DH0:. I tried setting the two Uboot variables again before trying to boot from the new partition again:

I again picked the backup partition from the boot menu and this time, I had success:

You can see above that my SYS: volume is now pointing to my backup volume, along with all the other assigns. With that work verified, I went into Uboot for the last time, set the two variables I needed and then saved the configuration to the flash device with the saveenv command to make my modifications permanent:

Now, every time I boot I get the boot selection menu and can pick which partition to boot into.

Remainder of backups

Now that I had my bootable System: backup partition, I wanted to also create clones of my other volumes to make sure I have backups for my Software: and Work: partitions. For this, I just wrote a very simple AmigaDOS script (saved into my Work:c directory for all my personal scripts and tools) that runs OnyxSoft BackUp in CLI mode. It first updates my bootable clone, then clones everything (including the System volume again) into a Backups:Clones/ directory on a separate disk. This means that even if the SanDisk SSD holding my volumes dies, I still have a backup on another disk:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
SET backup_bindir Software:Programs/BackUp

SET system_source System:
SET system_clone SystemBackup:
SET software_source Software:
SET work_source Work:
;
; Bootable clone
;
echo "Cloning $system_source to $system_clone"
$backup_bindir/BackUp SOURCE $system_source TARGET $system_clone BACKUP AUTOQUIT

;
; Backup clones on separate Backups: volume
;
echo "Cloning $system_source to Backups:Clones/System"
$backup_bindir/BackUp SOURCE $system_source TARGET Backups:Clones/System BACKUP AUTOQUIT

echo "Cloning $software_source to Backups:Clones/Software"
$backup_bindir/BackUp SOURCE $software_source TARGET Backups:Clones/Software BACKUP AUTOQUIT

echo "Cloning $work_source to Backups:Clones/Work"
$backup_bindir/BackUp SOURCE $work_source TARGET Backups:Clones/Work BACKUP AUTOQUIT

;
; Display backups usage
;
info Backups:

This took a long time the first time I ran it, but thanks to BackUp only copying changed files, it now takes only a minute or so each time I run this script.

I was now feeling a lot safer with my volumes all backed up, but I had one more step to complete, namely taking these backups and turning them into single-file archives which I could then transfer off my X5000 to another system, such as my home NAS and S3 for off-site backups. After all, it’s no good having all your backups on one system if that system completely fails or is damaged in some way. My years of running Unix systems professionally has taught me the “3-2-1” backup rule: A good backup plan must have at least 3 copies of the data, 2 of which should be on different media, and 1 should be stored off-site.

Archives

So, the final step was working out how to take these backup directories and archive them into single files. My first thought was to use LHA which is pretty much the standard Amiga archival utility. But there was a problem: As it turns out, most “classic” Amiga software is not large-file aware, which means that LHA could only create 2GB archives (which would have been unthinkably huge back in the day!).

This was OK for my system volume which was under 1GB in size, but it would not suffice for my Software and Work volumes which were much larger. I went searching for an alternative solution which could handle creating large files, as well as preserving all the important Amiga filesystem attributes. I posted a question over on the helpful AmigaWorld forums and quickly got a suggestion to try AmiDVD which sounded like a good tool to try as it’s bundled with AmigaOS 4.1.

I’m pleased to say this worked out very well, and was simple to use. I just used the “Create Image” tab, and tested it first by using my Work: volume which was around 40GB in size:

With this done, I then mounted the resulting .iso file and browsed it to verify that my files were all present and still had the correct filesystem attributes set. In the screenshot below you can see the ISO file mounted as a virtual drive, and you can see from the shell and Workbench window that the ISO file is around 40GB in size. I also browsed to the backup of my Work:c directory where I work on my own scripts and verified that the AmigaDOS filesystem attributes were still set correctly:

I then just had to re-run through the process again to create backups of my other volumes. Here’s the results, showing again the large filesizes created with no problem:

I could then transfer these files off my X5000 and onto my Synology NAS and other systems. I would ideally have liked a tool that could be driven from the command-line so I could include in my backup script, but AmiDVD is very straightforward and it doesn’t take much effort to click a few times to create updated backup archives. Still, I’ll continue searching and if I find something I can use from the shell I will update this blog post to cover it. Maybe mkisofs would be a good starting point…

Anyway, now I have a solid backup plan in place (and learned a lot more about my X5000 and the world of Next-Gen Amiga systems!), I can press on with having fun experimenting with my new system and I look forward to blogging more this year. Hope it’s a great year for all you Amigans out there and remember: Only Amiga Makes It Possible!

Fuse, SOBJS and Multiple Assigns: Amiga OS 4 Learning Curve Part 1

| Comments

A little while ago, an updated port of the FUSE ZX Spectrum emulator was uploaded to OS4Depot. My first computer was a ZX Spectrum 48k, although I eventually ended up with a +3 model before I upgraded to my first Amiga. I was looking forward to getting this emulator installed so I could re-visit some of the classic 8-bit games I used to play; my favorite game is still Manic Miner although I never manage to get past the “Eugene’s Lair” level on my first attempt!

I hit a few stumbling blocks along the way though, so this “simple” exercise turned into a trip down the rabbit-hole which ended up teaching me a lot about my new X5000 system and AmigaOS 4. What follows is pretty much a blow-by-blow account of what I discovered along the way. As I’m still pretty new to OS4, I’m sure there’s probably a much neater solution to all this though, so any comments and feedback are all welcome!

Anyway, here’s the first problem I hit: I downloaded the ZIP archive for FUSE, unpacked it with UnArc and tried to launch it by double-clicking on the Icon. Instead of seeing the FUSE window appear, I instead got this error message:

I remember having this kind of issue all the time on my classic Amigas – a program would require some library that I didn’t have installed. Back in the day, if I was lucky I’d get an error message telling me what to download, or a note in a README guide somewhere. When that failed, I’d usually break out the fantastic and indispensable SnoopDOS utility to find out what a program was looking for. In this case however, it was clearly telling me what file I needed, but I had no idea where to install it. On my classic Amiga systems, standard Amiga libraries (with the .library extension) simply got dropped in the LIBS: volume, which was usually in the Libs directory on your startup disk. But I had no idea what to do with ELF-format shared libraries.

Anyway, I searched on OS4Depot for “libpng”, and found this package. Looking at the contents of the archive, I could see this contained a libpng16.so.16.34.0 file. This looked to be exactly what I was after! I downloaded the libpng archive, and unpacked it to a new directory in my Downloads folder (I really dislike archives that unpack into the current directory; it makes cleaning up and keeping track of where files came from so much harder). This left me with a bunch of library files and an AutoInstall file. I looked at this, and it turned out to be a simple AmigaDOS script:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Delete SDK:Local/clib2/bin/libpng#? QUIET
Delete SDK:Local/newlib/bin/libpng#? QUIET

Delete SDK:Local/clib2/lib/pkgconfig/libpng#? QUIET
Delete SDK:Local/newlib/lib/pkgconfig/libpng#? QUIET

Delete SDK:Local/clib2/lib/libpng#? QUIET
Delete SDK:Local/newlib/lib/libpng#? QUIET

Delete SDK:Local/clib2/include/libpng#? ALL QUIET
Delete SDK:Local/newlib/include/libpng#? ALL QUIET

Delete SDK:Local/clib2/include/png.#? QUIET
Delete SDK:Local/clib2/include/pngconf.#? QUIET
Delete SDK:Local/clib2/include/pnglibconf.#? QUIET
Delete SDK:Local/newlib/include/png.#? QUIET
Delete SDK:Local/newlib/include/pngconf.#? QUIET
Delete SDK:Local/newlib/include/pnglibconf.#? QUIET

Copy Local/ ALL CLONE QUIET SDK:local/
CD SDK:local/newlib/lib
MakeLink libpng.so libpng16.so.16.34.0 SOFT
MakeLink libpng16.so.16 libpng16.so.16.34.0 SOFT
MakeLink libpng16.so libpng16.so.16.34.0 SOFT

Fairly rudimentary, but it’s clear what it’s doing: It first deletes any existing libpng-related files, and then copies things to the required locations, and sets up some symbolic links in SDK:local/newlib/lib. If you’re familiar with how Unix systems handle shared libraries, this should look pretty similar. I also discovered from that script that AmigaDOS can make use of symbolic links – I’m not sure when support for that was added, but I don’t recall it being around on my classic Amigas.

Only problem was, I didn’t have any SDK: assign, or any directory structure resembling the archive on my system. After some searching online, it appears this is created by the official SDK for AmigaOS 4, which can be downloaded from the Hyperion website. I picked the most recent version, SDK_53.30.lha (Software Development Kit for AmigaOS 4.1 Final Edition) and downloaded it.

After I’d unpacked it, I ran through the supplied installation utility, only changing the location to my standard Software: assign. I selected a “Full Install”, as this bundles a lot of very useful-looking GNU tools and utilities, as well as include files and compilers:

Definitely a lot to go exploring through on another occasion! I noticed that it had also modified my S:User-Startup file to make sure the SDK: assign is created, and it also dropped in another startup file that is run on boot:

1
2
3
4
;BEGIN AmigaOS 4.1 SDK
assign SDK: Software:SDK
execute SDK:S/sdk-startup
;END AmigaOS 4.1 SDK

Looking at SDK:S/sdk-startup, it appears that this sets up assigns and other things that are needed for the bundled compilers and utilities. In any case, I now had my SDK: assign, so I could go back to my libpng download directory and run execute AutoInstall from an AmigaDOS shell. This completed, and I could then see that the libpng files were copied to the correct destination under my SDK root. So, I clicked on the FUSE icon again, and…. same error: “Failed to load shared object libpng16.so.16.34.0”. Damn.

At this point, I discovered there was a SOBJS: assign which appeared to point to the System:Sobjs directory which also seemed to be full of .so library files. I decided to try and copy the libpng library there:

1
copy SDK:local/newlib/lib/libpng16.so.16.34.0 system:SObjs/

And I then re-launched Fuse. Now, I saw a different error – this time, complaining about not being able to find libz.so.1. One step forward, two steps back…. This libz.so.1 file however was in the SDK:local/newlib/lib directory, but I decided against copying everything in there over as it seemed pretty hacky, and a sure-fire way to screw something up. Plus, it seemed that there must be a way of adding this SDK lib directory to the system-wide library search path.

Now, on Linux or Unix systems, I’d have looked for a way to either re-compile Fuse with the correct -L and -R flags, or found a way of adding the SDK to the LD_LIBRARY_PATH variable. In this case though, I was totally stuck and in a very unfamiliar place. I shelved my “get FUSE installed” project for the time being, and instead took a brief diversion into getting QT installed. Quite by chance, I noticed that it added something very interesting to the end of my S:User-Startup:

1
2
3
4
5
;BEGIN Qt-4.7
assign Qt: "Software:Programs/Qt-4.7"
path Qt:bin ADD
assign sobjs: Qt:lib ADD
;END Qt-4.7

Whoah. It looked like it was appending a path onto an existing Assign – I had no idea that AmigaDOS supported this! I’ve used the ADD syntax to the path command many times when adding additional directories full of tools, but never thought to try it with an Assign. It looks like it was added into AmigaDOS release 2, so it’s always been available to me ever since I got my first A600. Another new thing I learned!

So, I then added the following to the end of my S:User-startup:

1
2
; Needed for FUSE etc to pick up on new libraries
assign Sobjs: SDK:local/newlib/lib ADD

I rebooted, clicked on the Fuse icon, and…..(cue drum-roll)

Success! Although, I have to admit that at this point, I lost around 2-3 hours downloading and playing tons of classic games from the World Of Spectrum archives :)

However, this little distraction did teach me some new things:

  • AmigaDOS supports both hard and soft symbolic links.
  • Assign can take an ADD parameter which appends directories onto a logical assign.
  • The SOBJS: assign appears to be where programs look for ELF-format shared libraries on startup.
  • The OS4 SDK contains some very useful CLI tools and is worth installing on any Amiga OS 4 system even if you don’t intend to develop software with it.
  • Even after 30-odd years, there’s still something new for me to learn about classic AmigaDOS commands!

Finally, I’d just like to note that shortly after I solved the FUSE problem, someone very helpfully posted a comment which highlights an alternative fix – simply copying the libpng library to the FUSE directory. Although I wish I had thought of trying that, this little diversion proved to be a very useful exercise and I’m glad I got to dig a little deeper “under the hood” of my new X5000.

Classic Amiga Emulation on the X5000

| Comments

While I’ve been having a lot of fun with the new software written specifically for AmigaOS 4, the bulk of my software is still “classic” titles that used to run on my A1200. One of the first things I did when I set up my X5000 was to transfer my old Amiga’s hard drive over so I could continue running this library of software. I also wanted to set up an emulation of my A1200 so I can quickly launch a classic Workbench 3.9 session and pick up all my old projects and bits of code I’d written over the years.

Fortunately, the X5000 and AmigaOS 4 offers a variety of ways of running all your old software. For this blog post, I’ll break it down into four areas:

  • Running software from disk images (such as ADF, DMS etc.)
  • Running games and demos from WHDLoad
  • Full system emulation with E-UAE
  • Running 68k system software

Disk images

The first “mode” I wanted to explore was running classic Amiga games and demos from ADF disk images.

I discovered that thanks to the pre-installed RunInUAE tool, this was about as easy as it gets – you simply double-click a supported disk image filetype such as an ADF or DMS image, and RunInUAE then boots a pre-configured fullscreen classic Amiga environment with your disks attached.

It relies on the filename to select the model of classic Amiga to emulate so if your disk image has the letters AGA in the filename, then an A1200 will be emulated. Otherwise, it falls back to a classic A500 setup. It also supports multi-disk demos and games very simply: If the filenames for your disk images are identical apart from a trailing numerical character, it will “insert” both disks into the system. As an example, here’s what my directory of classic “Grapevine” disk magazines looks like after I’d renamed them according to RunInUAE’s standards:

By default, the emulation runs in fullscreen mode. You can quit the emulation by holding down Ctrl and pressing the Alt and q keys, and Ctrl-Alt-s switches out of fullscreen mode. You can also tweak the settings for an individual game by dropping it onto the RunInUAE window and right-clicking to see a menu where you can change the input devices (joysticks etc.), screenmode and other things. This creates a customised .uaerc file for that particular image which you can see opened in Multiview below:

Here, I’ve switched to windowed mode instead of full-screen. You can also see in the generated .uaerc that it provides a Workbench volume providing a 3.1 installation mounted on a virtual DH0:. This installation lives under System:Emulation/AmigaDisks/Workbench3.1 – this will be important later on… In addition, my usual volumes (Software, Work and System) are also mounted into the virtual Amiga in read-only mode.

A nice touch is that the DiskImageGUI utility also supports Amiga disk images, so you can also mount the images as virtual IDFn: volumes. As an example, I’ve mounted the Grapevine disk image with Mounter and you can see it show up on my X5000 desktop, accessible just like a real floppy image:

The DiskImageGUI runs as a commodity after you close the window, so you can always bring it back with the shortcut Ctrl-Alt-d.

WHDload

WHDLoad is an awesome tool developed for classic 68k Amigas that makes lots of games and demos runnable straight from a hard drive, even for programs that didn’t natively support a HD install option. There are lots of archives available on the Internet of thousands of classic games and demos, all pre-configured and ready to go. These can also run on the X5000 and AmigaOS 4 through RunInUAE, as it provides a WHDLoad “shim” which boots an emulated Amiga and runs the real WHDLoad from inside that.

However, the bundled WHDLoad binary is pretty old, and is also unregistered so before a game starts you are presented with a “please register” message and a timer that counts down before the game will actually start. Newer versions of WHDLoad include numerous improvements and also remove the registration requirement, so the first thing I did was update the installation. I went to the WHDLoad homepage and downloaded the latest “usr” package which was a standard LHA archive. After extracting it, I ran through the installer utility, but (and this is important!) I selected the C directory under System:Emulation/AmigaDisks/Workbench3.1 as the destination. Do not install this into your AmigaOS 4 System volume, as otherwise it’ll overwrite the WHDLoad “shim”. You can see the correct path specified in the screenshot below:

Once this was done, all my WHDLoad games and demos worked exactly as they did on my A1200! Here’s what the emulation screen displays briefly before the game starts, showing the updated WHDLoad install:

Joystick tweaks

The only AmigaInput supported gamepad I have for my X5000 doesn’t let me use the “hat” switchpad in most games. For some things this is fine, but for other games I prefer the feel of the classic pad instead of using the joystick nubs. I found a fantastic tool on http://os4depot.net/ called AmigaInputAnywhere which lets you use an AmigaInput device to trigger keypresses or other actions. With this, I could set my gamepad’s “hat” controller to trigger the numeric keys (8=Up, 2=Down, 4=Left, 6=Right, 5=Fire) which can then be selected as an emulated Joystick in RunInUAE. For the games where I preferred the Hat switch, I just opened up RunInUAE and dropped the game icons on the open window, right clicked and selected Per-game settings –> Set joystick to –> kbd1 (number pad). In the next screenshot, you can see the AmigaInput prefs open, along with my AmigaInputAnywhere configuration and a test session in the shell where moving the hat switchpad generated a stream of number-pad keypresses:

Full system emulation

Although I could now run my old games and demos through RunInUAE, I also wanted to emulate my A1200 as a “full system” which boots into my Workbench 3.9 installation so I can continue with my old projects. I had the directories prepared with the contents of my A1200’s hard disk copied over; this was covered in my earlier blog post here.

To setup this emulation environment, I decided to use an updated E-UAE which has been released on OS4Depot here. This version of E-UAE has JIT support which can add a significant speed boost, although it’s worth experimenting with it as some things can become unstable when enabling it. I also definitely didn’t want to interfere with my stock E-UAE installation. Even though it’s old, it still seems to work fine for my disk images and WHDLoad games, so instead I unpacked this updated release into a seperate directory at Software:Programs/E-UAE_1.0.0.

I then created the Configuration file at Software:Programs/E-UAE_1.0.0/.uaerc based on a template from Amigaone X5000 Blog. I made a few simple changes: Disabled the floppy drive sound, enabled the JIT (after some experimentation, my installation seemed to run better with it) and changed the paths to my emulated hard drives and ROM image. For reference, my slightly modified .uaerc file is listed below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
kickstart_rom_file=A1200:rom/amiga-os-310-a1200.rom
kickstart_key_file=A1200:rom/rom.key

amiga.floppy_path=A1200:Floppies

floppy_speed=400
floppy0sound=0
scsi=true
scsi_device=true
show_leds=false
filesystem2=rw,DH0:System:A1200:Drives/System,1
filesystem2=rw,DH1:Software:A1200:Drives/Software,0
filesystem2=rw,DH2:Work:A1200:Drives/Work,0

joyport0=mouse
joyport1=joy0

cpu_type=68040
cpu_compatible=false
cpu_cycle_exact=false
cpu_speed=max

chipset=aga
immediate_blits=yes
collision_level=full

chipmem_size=16
bogomem_size=0

fastmem_size=0
z3mem_size=256
gfxcard_size=32

sound_output=exact
sound_channels=mixed
sound_frequency=44100
sound_latency=120
sound_max_buff=8192

amiga.screen_type=ask
amiga.publicscreen=ask
gfx_fullscreen_amiga=false

amiga.use_dither=false
gfx_framerate=1

# Display settings
gfx_width=1024
gfx_height=768
gfx_width_windowed=720
gfx_height_windowed=568
gfx_fullscreen_amiga=no
gfx_fullscreen_picasso=yes
gfx_lores=false
gfx_linemode=double
gfx_correct_aspect=no
gfx_center_horizontal=smart
gfx_center_vertical=smart

# Miscellaneous
use_debugger=no
ppc.use_tbc=false

# JIT
cachesize=8192
cpu_compatible=false
cpu_cycle_exact=false
blitter_cycle_exact=false
comp_constjump=yes
comp_trustbyte=indirect
comp_trustword=indirect
comp_trustlong=indirect
comp_optimize=true

I created a little script file to launch the emulation environment, this was a simple text file with the following lines:

1
2
cd Software:Programs/E-UAE_1.0.0
uae

I saved this as “a1200” in my Work:c directory I keep for my own scripts and tools (this is also added to my path in my S:User-Startup). I also ran Protect Work:c/a1200 +s to mark it as an executable script. So that I could also launch this from X-Dock, I copied over a nice .info icon file to this directory and renamed it as a1200.info. I then modified the tooltypes as per the screenshot below:

This sets the default tool to C:IconX which launches script files, and also set the window to the NIL: device so I didn’t get a screen of UAE debugging output.

With that in place, all I have to do now is click on the icon in my X-Dock, and I get a windowed emulation of my A1200 just as I’d left it!

I can easily put this into fullscreen mode by using the Ctrl-Alt-s shortcut, and I also discovered that Ctrl-Alt-g “grabs” the mouse pointer inside the emulation which is really useful when it’s running in windowed mode.

Classic 68k binaries

AmigaOS 4 also includes a built-in JIT 68k emulation layer called “Petunia” that allows classic system-legal 68k binaries and libraries to run unmodified. For these to run, they must only use OS system calls and avoid “hitting the hardware” – so anything that bangs on the custom chipset can’t be used (although there’s always RunInUAE for such things).

Many tools and utilities work just fine however, especially more recent 68k binaries. In the screenshot below, I’m running the 68k binaries of the Workbench 3.1 “calculator” tool, and also AmTelnet for some classic BBS action. AmTelnet is a particularly good example of what can run: It’s a full MUI program which also installs some custom 68k libraries. Apart from the occasional Grim Reaper crash on launch, it functions just like a PPC-native program. It’s fully integrated into the updated Workbench environment, and if I never looked at the task listing in SysMon and saw the “68k” field in the “Arch” column, I’d have thought it was a native application!

So there you have it! I hope that’s been of some help and interest :) I’ll continue to update this blog with more of my Amiga experiments, and feel free to leave a comment with any questions!

New Amiga X5000

| Comments

As you may have seen with my latest music project, I’ve been getting back into the Amiga scene in a big way over the last year. Granted, a large part of this is nostalgia on my part; the Amiga was a lifeline to me during my teenage years and was responsible for starting my twin interests of computing and music. But I’ve always been amazed by the sheer tenacity of the Amiga scene – nearly 30 years on from when I first got my Amiga 600, the scene is still going (albeit fractured into different camps) and new systems are still being created for this legendary platform.

Sadly, my A1200 recently died and while I haven’t given up on it yet, it’s going to take me some time to source spare parts and experiment with fixing it. This left a large Amiga-shaped hole to be filled, and I’ve always been fascinated by the next-gen PPC systems. I finally took the plunge and ordered my maxed-out X5000 from AmigaKit in the UK, with the aim of getting a more modern, reliable system and moving all my projects from the A1200 over onto it.

Although I used OS 3.x pretty heavily “back in the day”, I jumped ship from the Amiga scene in the late 90s/early 2000s before PPC accelerators had really become wide-spread. I’ve also never had access to the current breed of next-gen PPC systems, so this series of Amiga-related posts is intended to document my adventures and experiments with my new Amiga, from the perspective of an old Amigan getting back into the scene. Hopefully, once I’ve repaired my A1200 there also will be a series of posts on the classic 68k scene. I’m also a little rusty on all things Amiga, so if I’ve made any glaring mistakes please feel free to contact me or leave a comment and I’ll fix it!

While I’m at it, I should give a special mention to two fantastic blogs:

Both of these provided a wealth of information on next-gen systems, and I was particularly pleased to see Epsilon received his X5000 only a few days before I got mine. I know there will be a fair amount of overlap between my blog posts and theirs, but I hope my take on the X5000 will maybe help convince a few more old Amigans to join the party!

Anyway, back to the X5000. About a week after I ordered it, it had been built and tested (great communication from AmigaKit by the way – they’re only a small operation, but I have to hand it to them keeping the Amiga dream alive!) and I received a shipping notification and tracking number. The following day, this very exciting parcel is what turned up on my doorstep:

Very well packaged and protected during the journey.

And here’s the assorted collection of “extras” that came with this X5000. The brown cardboard box was taped inside the case, in one of the spare HD slots. Inside was a very useful collection of spare screws, bolts, blanking panels and a cable. I’m not entirely sure what the cable was for, it looked like something left over from the motherboard. I’ll keep it anyway just in case it turns out to be useful!

I also received a USB drive pre-installed with AmigaOS 4.1 FE pre-release for the X5000 and A-Eon Enhancer Software. This can be used as a recovery drive, so the first thing I did while I was getting the rest of the kit ready was to copy the contents to my NAS using an Apple Macbook Pro laptop and the dd command, so I have the USB image safe somewhere else in case I ever need it.

As the USB recovery drive comes pre-installed with the Enhancer package, you can also continue to access your SFS/02 partitions and filesystems – apparently, SFS2 support has been dropped from recent updates of AmigaOS 4.1. It seems these days as though Hyperion isn’t really actively developing AmigaOS 4 any further, and therefore much of the updates to drivers and the rest of the OS is now coming directly from A-Eon instead. Having the USB drive pre-installed and ready to boot is a real nice touch as it also boots straight to a desktop like a Linux “live CD”, so can be used for repair work in case you muck something up like your startup-sequence and need a way to edit it.

Along with the USB drive, I also receieved a registration card for AmigaOS, a DVD copy of the Enhancer Pack, and a printed manual for the motherboard. Creating an account and registering on the Hyperion website also gave me access to a ISO image download of “Amiga OS 4.1 Final Edition Pre-Release”, which I also downloaded to keep safe. I don’t think I’ll ever use this image as the USB recovery stick has everything I need on it, but it’s nice to have anyway.

Speaking of the motherboard, here’s a shot looking down on the inside of the X5000 after I removed the side panel to check everything was properly connected:

A nice, tidy case with plenty of room for expansion! There’s a single stick of RAM and a PCI sound card along with the PCIe RadeonHD graphics card pre-installed.

I then put the cover back on, connected my USB mouse and keyboard (Logitech K120) to the top two black USB sockets at the rear, hooked up power and turned the system on. I was greeted with the awesome animated boot logo:

Once the ball starts spinning, you can press the space key to get at the early-boot menu, or you can simply wait and after some seconds it boots straight into AmigaOS. Under a minute later, I was looking at my brand new Workbench! An awesome moment, and I was really excited to start exploring the new OS and machine. Note that my “screenshots” at this step are all taken with my phone camera as I was going to reformat and re-partition my drives so didn’t yet want to mess around with taking screengrabs and worrying how to transfer them across.

Anyway, here’s what I saw after first boot – a 1290x1080 HD workbench with an animated “CANDI” background and a few widgets like the clock and calendar:

One thing to note is that the keyboard didn’t seem to function as expected unless it was plugged into one of the black USB ports. I could type OK, and it worked in the U-boot environment, but the Windows keys weren’t recognised as “Amiga” keys, and I also had problems with the other modifier keys like Ctrl and Alt not being recognised. Moving the keyboard to the black USB ports seemed to resolve this, though.

I clicked around a few drawers and experimented with some of the included programs, but had to stop the fun as I had work to do installing an additional SSD before I could really continue. My plan was to keep the SSD shipped with the X5000 as a “System and Software” volume, and install a second SSD for my Work volume. I also planned on making room for some additional partitions which I may require later if I wanted to experiment with MorphOS or different filesystems.

Pretty much any SSD should work, but I decided on a Samsung EVO as I’ve had good experience with them in the past. here’s my new Samsung SSD installed in one of the spare mounting brackets from the X5000:

I then had to remove the other side-panel so I could access the bundle of SATA and power cables that had been cable-tied neatly together. First step was to free these up by cutting some ties so I had enough room to move cables around to their desired position. As the motherboard only has two SATA connectors, my plan was simply to disconnect the DVD drive (I very rarely use optical media these days) and use that SATA port for my new SSD. I’ll later add an additional supported PCI SATA controller so I can re-connect the DVD drive and maybe an additional drive for later use.

Here’s my new drive in place with SATA and power connected:

There was, however, a small problem I encountered during this work!

When I looked closely at the SATA connectors, I noticed that glue had been applied around them. You can see this in the close-up image below; presumably this was to stop connectors coming loose in transit. This was a good idea, but it made things a little difficult when it came to moving things around inside.

Anyway, the connector plugged into the DVD drive was pretty firmly glued at the DVD end, so I decided instead to leave that end of the connector in place and remove the cable from the motherboard end. I carefully applied gentle pressure to pull the cable free from the motherboard and – disaster! The SATA socket actually lifted clear off the motherboard, still attached to the cable and leaving some very gentle looking pins exposed. Argh! It seems the SATA connectors (and maybe others) are not firmly held in place on the board like they are on most PC motherboards I’ve previously used.

I stopped what I was doing, calmed down, made myself a coffee and then went back to work (making very, very certain I was earthed with an anti-static wristband) and carefully pushed the SATA connector back onto the motherboard pins with the help of a magnifying glass and a pair of insulated tweezers I got from an old toolset. It eventually slotted home without damaging or bending any of the pins and I was good to go; a hair-raising moment though and I’d advise extreme caution when attempting this on any X5000 motherboard!

So, with my new SSD attached I quickly checked everything looked OK from Uboot. I simply pressed the space bar when the boing ball started spinning, and selected the System Information menu option. I saw the following screen displayed which confirmed my new SSD drive was recognised:

So, now on to partitioning and laying things out to my usual preferences. On all my Amigas, I usually have 3 partitions :

  • DH0: (System) : Boot drive containing the OS and essential libs
  • DH1: (Software) : All programs, games, 3rd party libs etc.
  • DH2: (Work) : My equivalent of a “Home” directory. Pictures, music, downloads, code projects etc.

The X5000 came with a single 240GB SanDisk SSD installed, and this was the default partition layout:

There was one 20GB System partition, and the remainder of the disk was left as a single large “Work” partition. I wanted to reduce this in size to use as my Software volume, and as I mentioned earlier, also allow some space at the end of the drive for a later MorphOS partition. I have no plans right now to install MorphOS, but I can see that it’ll be something worth experimenting with, especially if the on-board graphics card gets driver support (currently, to use MorphOS on the X5000, you’ll need to replace the supplied graphics card). So, I reduced DH1 in size, leaving enough space to create some more partitions as needed:

I then also partitioned my new SSD and added a single large Work partition. A quick reboot and format of the drives later, and I was ready to start installing software, copying data over and customising the install!

So now my next step was to transfer all the data over from my A1200. I removed the internal CF card from it, and placed it in an external USB adapter that supports CF, SD and micro-SD. The adpter is a “Cateck Aluminum Superspeed USB 3.0 Multi-in-1 3-Slot Card Reader”, which I purchased from Amazon here. It works very nicely in Linux, Windows, Mac OS and AmigaOS! Here’s the A1200s CF card inserted and ready for transfer:

I now faced another problem – I could mount the partitions using the “Mounter” utility after selecting the usbdisk.device device, but the A1200s partitions were labelled as DH0: to DH2:, which clashed with my internal drives. I found the documentation that said that I should alter the tooltypes of the Mounter utility to include NOUNMOUNT, which should have renamed the partitions and prevented the internal drives form unmounting:

However, no matter what I did, I couldn’t get this to work. So instead, I took a different approach – I simply opened the A1200s drive in Media Toolbox, and changed the A1200 partitions to become DH3: to DH5:. With this done, I could mount them without the internal drives unmounting, and could start to copy everything over. I first created a new directory and aliased this under my work partition to hold all my A1200 files:

1
2
3
4
5
6
makedir Work:A1200
Assign A1200: Work:A1200
makedir A1200:Drives
makedir A1200:Drives/System
makedir A1200:Drives/Software
makedir A1200:Drives/Work

And I also added the Assign command to my S:User-Startup for later use.

I started copying the files by using Directory Opus as you can see in this next screenshot, with the A1200 partitions showing up on the right hand side of the screen with their OS3.x icons:

Later on, I reverted back to a simple AmigaDOS shell session and did something like this for the remaining partitions:

1
2
copy DH4: A1200:Drives/Software all
copy DH5: A1200:Drives/Work all

After some more work customising my “classic” environment (details in a forthcoming post), here’s my desktop as it was at the end of my first day:

You can see I’ve tweaked it to my liking, added a classic Amiga wallpaper, setup X-Dock with a few useful shortcuts, and I also have my A1200 emulated in E-UAE, along with my old WHDload games ready for action. Setting up and tweaking my emulation configuration is definitely too much for this already very long post, so I’ll save those details for another time.

Until then, I hope this has been an interesting overview of my new system!

Flashback

| Comments

This is my rock/metal cover of a tune from the classic Mahoney & Kaktus Amiga demo, “Sounds of Gnome” (http://www.pouet.net/prod.php?which=5583). Specifically, the tune was called “Jobba” and the intro also borrows from the intro song on The Great Giana Sisters.

I loved this track when I was a kid, and watching M&K demos (along with the other late 80s/early 90s classics from Anarchy, Rebels, Animate!, D.O.C. etc.) was what drew me into the UK demoscene (shout out to any old members of NFA!). I remember watching this on a friend’s A500 and it was one of the first things I ran on my first Amiga (A600, eventually traded up to an A1200).

As well as getting me into the scene, the Amiga also kick-started my interest in making computer-based music. Mahoney (who wrote the original tune) was also responsible for NoiseTracker which was also my first introduction to making music for myself. I eventually ended up on OctaMed and now my Amiga is solely there for retro sessions as I’m on Cubase these days. But I still miss the DIY approach of Trackers, and I learnt so much from playing around with other people’s MODs.

The Amiga was a real life-line to a geeky kid like I was back then, and it set me on my path for both my career working in IT, and my hobby of making music. The Amiga scene gave me friendship, an escape from troubles of teenage life, and showed me that it’s only imagination that holds you back from achieving something that’s supposed to be “impossible”.

So this is a tribute to the Amiga scene really; I’m not exactly the worlds greatest guitarist but I had the time of my life recording this. I also stuck a shot in, right at the end of my pimped-out A1200 (HXCE CF, Blizzard 1230, 16Mb. Indivision fixer) playing the original demo and track.

Hope you enjoy, and remember : Only Amiga Makes It Possible!

The River

| Comments

Here’s my latest music project – a little progressive rock/metal track called “The River”. Based on some riffs I had knocking around in my head for several years now, and I finally got them down in a form I’m pretty happy with!

Three Years of Tiller

| Comments

On July 18th, 2014 – 3 years to the day of writing this post – I pushed the first public code of Tiller to Github. Back then, it was just a simple little tool that I wrote (mostly as a learning exercise), found useful and thought others may like to use.

Since then, there have been:

  • Nearly 60,000 downloads from rubygems.org
  • 575 commits
  • 244 stars on Github
  • 54 releases
  • 52 issues closed
  • 16 new plugins created
  • 11 pull requests from other developers
  • Over 2,000 lines of documentation written

And countless emails, comments and discussions on the chatroom.

I’d be the first to admit that it’s certainly not the best code you’ll ever see, and in terms of project size it’s tiny compared to some of the other things you’ll see hosted on Github. But something that was once solely mine has clearly resonated with others (The 1.1.0 release actually contained more of other people’s code than my own changes!), spawned a community and has found a place as a building block in projects I’d never have imagined, from people I’ve never even met.

It’s a feeling I didn’t expect to experience outside of my musical endeavours. As when I’ve slaved over a mixing desk for days or spent hours trying to perfect a bass riff, it’s been a labour of love. For me, it’s the same nerve-racking sensation publishing a mastered musical track, or pushing a new code release to rubygems.org. I find myself second-guessing my decisions every time, and – despite working on these things because it’s like an itch I have to scratch – I can’t help but wonder whether anyone will like what I’ve just created.

Just like making music, I strongly believe that code can be an art form. It’s something that ends up reflecting a little of yourself – and whatever your endeavours, you’ll only ever get better at as you continue, receive helpful criticism and words of encouragement.

So here we are, 3 years later from that first tentative code push. It’s immensely satisfying to see a community spring up around this little tool, and I want to take a moment to thank everyone who has ever emailed me, submitted a bug report, come up with suggestions, submitted code or otherwise joined in helping this project fill a niche. You’re my encouragement to keep up with this project, learn more, grow as a developer and above all – keep having fun!

Recent updates

Anyway, enough of my meandering musings.. It’s been quite a while since I last posted about Tiller, but that doesn’t mean things have been quiet! On the contrary, development is busier than ever and there have been a lot of new features and improvements added over the last year. There is of course the Changelog where you can see all these updates, but I thought I’d take the opportunity to provide a quick round-up of some of the highlights.

0.9.0

Tiller 0.9.0 was released on the 10th of August 2016, and the 0.9 series included a lot of new features and some nice improvements:

Precedence changes

In previous versions of Tiller, global values were merged together in the order that plugins were loaded. Then, the same was done for template values. Finally, template values were merged over the top of global values. As had been discussed in the Gitter chatroom many times, this led to some counter-intuitive behaviour! Starting with Tiller 0.9.0, the behaviour has now been greatly simplified.

We now go through the plugins in order, and for each one we merge template values over global values, then proceed onto the next plugin. To put it simply: A template value will take priority over a global value, and any value from a plugin loaded later will take priority over any previously loaded plugins.

Vaults!

The 0.9.x series also saw two “Vault” plugins. The first supports Hashicorp’s Vault product and was provided in an awesome Pull Request including test cases and documentation from the fantastic liquid-sky.

The second plugin adds support for Ansible Vault – a simple way of encrypting a YAML file and passing it into a Docker container. I’m particularly fond of this plugin as it means it’s trivial to ship a safely encrypted set of YAML values or credentials inside your container, and unlock it at run-time. Even if you’re not using Ansible for orchestration or configuration management (and you should!), Ansible vault is simple to use:

1
2
$ ansible-vault create my_secret_vars.yaml
Vault password: <enter password here>

And that’s it! You can then edit the file with ansible-vault edit and bundle it into your container. See the plugin documentation for more examples.

Config.d

Tiller 0.9.5 added support for merging configuration in a config.d directory structure (and 0.9.6 improved it). This was another simple change suggested by a user (thanks, rafik777!) but means it’s easier now to create layered containers and split configuration out over multiple files.

1.0.0

The big 1.0.0 release! This marked the start of using semantic versioning, and as always, a big priority is never to break existing installs. I now strongly recommend a modern Ruby installation (see the requirements page for more details) but will continue to test on older installations.

Apart from some nice new features such as exec on write and dynamic values, the big highlight for me of this series so far is the 1.1.0 release. As I mentioned above, for the first time since I started this little project, I made a release with more of other people’s changes in it than my own code!

The future

There’s several new features coming in future releases, and I’ve already started to lay the foundations for plugin versioning. This means I (and other plugin authors) can experiment with new features or radically alter the internal workings of Tiller without breaking compatibility with existing installations. The first thing that will use this will be a new feature to specify multiple “targets” per template. So you’ll be able to do something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
vhost_template.erb:
  - target: /etc/httpd/sites_available/my_site.conf
    user: root
    group: httpd
    perms: 0644
    values:
      site_name: "My site"
      site_fqdn: www.mysite.example.com
  - target: /etc/httpd/sites_available/other_site.conf
    user: root
    group: httpd
    perms: 0644
    values:
      site_name: "My other site"
      site_fqdn: www.myothersite.example.com

… and so on. As always, feel free to submit an issue, join the chatroom or send me an email with any suggestions or improvements you’d like to see. And keep having fun!

Tiller 0.8.0 Released!

| Comments

Tiller has just seen its 0.8.0 release, and as you’d expect from a major version bump, it’s a big one. There’s some breaking changes if you write your own plugins – nothing too major, a quick search & replace should fix things for you. But, more importantly, there’s a major new feature which brings two big improvements.

I’ve added a “helper modules” feature, which lets you group together custom utility functions in Ruby that can be called from within templates. And I’ve included a built-in function called Tiller::render that lets you include and parse nested sub-templates from another template.

Sub-Templates

Now you can include other templates in your templates by using the built-in Tiller::render helper function. This may be useful if you have a large configuration file to generate: you can now split it up into separate files for each section and combine them together at run-time. Or, you may need to generate a lot of similar configuration blocks (for example, Nginx virtual hosts). You can now iterate over a list and render a single sub-template for each block.

For example, if you have a template called main.erb, you can include another template called sub.erb by calling this function inside main.erb:

main.erb
1
2
3
This is the main.erb template. 
This will include the sub.erb template below this line:
<%= Tiller::render('sub.erb') -%>

You can nest sub-templates as deeply as you wish, so you can have sub-templates including another sub-template and so on.

Helper modules

The sub-template system builds on top of a new “helper module” feature. A helper module is intended to group together small blocks of code that you can include in your templates to perform mark-up or other utility functions. They aren’t intended to replace the existing Data- and Template-source plugins; if you need to get some values into your templates, or hook up to some external service, these are probably still the best way to go about it.

But you can see how if you had a more complicated transformation to do (e.g. convert markdown text into HTML) or needed to include some logic in a function, this would help clean up your templates, as well as keep a clean separation of code and configuration.

As an example, this is how you’d add a Lorem Ipsum generator for filling in place-holder text. We’ll simply wrap the excellent forgery gem, so first make sure you have it installed:

1
2
3
4
5
$ gem install forgery
Successfully installed forgery-0.6.0
Parsing documentation for forgery-0.6.0
Done installing documentation for forgery after 0 seconds
1 gem installed

I’ll also assume you are using the default directory for custom plugins; if you want to change this, use the -l/--lib-dir option to Tiller to point to somewhere else.

First, create a file named /usr/local/lib/tiller/helper/lorem_ipsum.rb , and put the following code inside:

lorem_ipsum.rb
1
2
3
4
5
6
7
require 'forgery'

module Tiller::LoremIpsum
  def self.words(num)
    Forgery(:lorem_ipsum).words(num)
  end
end

Note that I defined this inside a custom namespace so it doesn’t clash with anything else. You can then load this module by adding the following to the top-level of your common.yaml:

1
helpers: [ "lorem_ipsum" ]

Now, in your templates you can call this function like so:

1
This is some place-holder content : <%= Tiller::LoremIpsum.words(10) %>

When you run Tiller with the -v (verbose) flag, you’ll see Helper modules loaded ["lorem_ipsum"] amongst the output, and your template will contain the following text when generated :

1
This is some place-holder content : lorem ipsum dolor sit amet consectetuer adipiscing elit proin risus