Adjusting my point of reference, and all becomes clear.

I have been scratching my head over the logic implemented in the routines that calculate object intersection for some time now, and it is the reason they remained unnamed until now, despite the fact that I knew their purpose. I had assumed all along, for no good reason and without giving it further thought, that an object's (x,y) position in 3D space was defined by the location of the corner closest to the origin, and the width and depth were the dimensions along the X and Y axis respectively. It occurred to me today however (whilst not even at the computer), that the position could instead be the center of the object, with the width and depth being the X and Y radii respectively. Looking at the aforementioned routines in a new light, the logic became obvious. Doh!

All variables and bit fields (flags) have been decoded and documented. All routines (with the exception of Z-ordering) have been documented. I now have stubs for all of the audio routines in the C port, but have not implemented any audio logic as of yet. IOW the entire program, again with the exception of Z-ordering, has now been reverse-engineered!

I do know of one bug in my C port that causes special objects to re-appear after you've picked them up whenever you re-enter the room. I'm hoping that's the same bug that I saw last week when picking up a special object that caused Sabreman to fall through the floor indefinitely!?!

Next task though is to reverse-engineer the Z-order algorithm and port it to C. I suspect the maths will be quite simple, it'll be the recursion that will be trickier to unravel and port to C.

Speaking of which, the code lends itself to a C implementation remarkably well. In fact there are only two (2) cases when stacks are adjusted and JP is used to jump to code elsewhere - exiting a screen and ending the game. As it stands I've used setjmp/longjmp in an attempt to preserve the implementation, but I have 2 issues with this. Whilst the exit screen case works fine, for some reason I can't fathom the end game case crashes. But more significantly, some development platforms (eg. Neo Geo) don't actually have implementations of setjmp/longjmp. So I'll need to find a work-around soon.

MUST. IGNORE. TRS-80. GRAFYX. SOLUTION. PORT...

Only a few 'bits' remain unknown

Not surprisingly, given the time of year I haven't had much time in the last week or so to work on this project.

That said, I've managed to make a little headway. I've been focused on getting the remainder of the unknown variables (bit flags) decoded and am happy to say that only a handful of bits from a single byte in the object table remains.

The object table contains 3 bytes of flags/counters; the first mostly defined in hard-coded object data and partially documented on the Icemark Knight Lore Data Format site. The 2nd I've now fully decoded, and comprises a counter in the top nibble and flags in the bottom nibble. The 3rd I've partially decoded, but the byte itself is used for different purposes for each of the objects. For some objects it's a counter, others it denotes direction, and others again different bits for different flags. At last count I still have at least 7 uses of flags to decode. I do have clues for 3 of those (all related), which incidentally appear to be consistent for all objects. So that leaves around 4.

For those of you wondering how I have reverse-engineered (and ported) pretty much the entire code base, yet don't understand the purpose of a handful of flags - in most instances the flags are tested and routines simply conditionally exited prematurely, sometimes on the basis of multiple criteria. So it's not always clear what individual flags mean, until you've examined all uses of a flag together. And in some cases, I just haven't gotten around to doing exactly that just yet.

I know of one bug in my C port that occurs in some cases when you pick up a special item, though it is admittedly difficult to play the trickier screens without proper Z-order. For this reason, and the relative lack of bugs, I've concluded that once all bit flags are decoded, there's no escaping attacking the Z-order algorithm. Only then will I be able to properly play-test the game.

It may surprise some of you that I'd never- and I mean never - played the game when I started this port. In fact, even now I've played this far longer on my C port than on any ZX Spectrum emulator. I was so impressed by the praise this game received in Retro Gamer magazines, the screenshots I saw in the same, and the YouTube video I watched briefly, that I didn't need to play the game. In fact, I actually didn't want to; I wanted to experience it for the first time on my own port. It will certainly be the case for the first game I finally play all the way through and win.

Anyway, time will continue to be fleeting in the short term and so I'll hack away at the remainder of the flags and then when the silly season winds down I'll tackle the Z-order algorithm, then hopefully spend most of my time playing the game in order to eliminate the last niggly bugs from my port.

That might be a few weeks off yet as I'll be heading up the coast for a week in the new year and have resolved to leave the laptop at home, the idea being to spend my time after the kids are in bed watching the cricket, reading books on my tablet and if I absolutely must, conversing with my wife. ;)

And I've been thinking about prospective ports again, and the TRS-80 Model 4 is always in the back of my mind. I only just realised in the last day or so that, aside from keyboard input which is trivial, I really only need to re-code two rather straightforward routines to get it running on the Model 4. This is because all rendering is done on a screen buffer in normal memory, with just two blit routines that transfer it to the actual screen. Now I'm sure it'll run into bandwidth issues on the TRS-80's Grafyx Solution hires board, and it may perhaps not been particularly playable because of the slow-down, but at least it'll run and it will look damn impressive - unquestionably worth the exercise given the relatively trivial effort required. It'll probably be my first port after the PC since I've effectively got fully relocatable Z80 source code now.

All over bar the Z-ordering

By my reckoning, the C port is complete except for a single routine - the Z-ordering algorithm.

Although the exact purpose of a handful of routines and flags are still unknown, I do understand the gist of them (i.e. what they are trying to achieve) and it is of course still possible to complete the (rest of the) port; which is exactly what I've done. Naturally a few more mysteries were uncovered during the process. As it stands, all bytes in the graphic object structure are known (interestingly 4 of them are unused, and another 4 have dual-purpose), although some bit flags are still to be decoded. All but five (5) global variables are known, and having a quick look at those just now it looks like they've simply escaped my attention, rather than being difficult to deduce.

UPDATE: All global variables have been named

So the game is playable in the sense that you can move around, interact with objects, move from screen to screen, drop/pickup special objects and be killed. There were, however, one or more bugs introduced when I added the object interaction routines that have affected the object movements; guards spin on their heels, balls don't bounce, gates don't move up/down. Whilst not easy to find in 4,200 lines of code, they'll likely be simple typo's.

UPDATE: The bug that prevents objects from moving correctly has been fixed. The game is pretty much playable now, with perhaps a couple of minor bugs that occur infrequently.

[Oh, there's no audio yet either. There's a surprising number of audio routines, some of which have quite a lot of code in them to produce what appear to be semi-random sounds.]

I'm going to stick to the plan of completing the game (fixing the bugs) before tackling the Z-order code. Having done some analysis of the Z-order routine, it doesn't appear to have any effect outside rendering, so it should be possible to have a fully playable game without it. I'll endeavour to decode the last remaining bit flags during the process.

It's exciting to be so close to having a fully-commented disassembly and a fully functional direct C port of the game. It's been a lot of work, but the subsequent porting to 16-bit platforms should be almost trivial in comparison. The Amiga will probably be the best candidate, although I'm still very keen to attempt a Neo Geo port, if only for the challenge! I don't dare think of the work involved in a 6809 port at this point...

I'm hoping my next post will be to announce a fully-functional game, sans Z-ordering.

Man or beast?

A week since my last post and I'm happy to say that I've made very good progress in that time. So much so, that you can control Sabreman turning, walking and jumping; he looks around randomly and at midnight he transforms into Sabre Wulf.

I was hoping to be able to post a short video of  Sabreman walking around, and perhaps other objects moving as well, but trying to find any sort of free video screen/window recording software without the risk of installing all manner of malware on your machine is simply too great. So until I get a recommendation from a trusted source, a few still frames will have to suffice.

Sabreman looking around

At night time you transform into Sabre Wulf

In fact, the game isn't that far off being playable - the sun and moon move across the frame and the days tick over - and there's not a huge amount of code left to reverse-engineer or port to C. I've gone about as far as I can now without tackling at least one of the two most technical aspects of the program - object interaction - which is little more than 3D collision-detection.

So late last night with only about 30 mins free I decided on a small fun diversion and instead worked on getting a raw dump of the loader screen displayed properly on my port. I'm yet to actually see how the screen is rendered during loading from tape, but I'm thinking of ultimately emulating that on my port, which will be a challenge on the Neo Geo.

Displayed from a raw dump of Spectrum video and attribute memory

The object interaction is required to move from screen to screen, where the object handler routine for the arches plays a key role. Once the generic interaction routine is done, the other objects shouldn't pass through one-another and aside from aesthetics, the game should be all-but-done. I'm even feeling less daunted by the Z-order rendering algorithm now.

Definitely getting to the business end of the whole process now!

ADDENDUM: I should add that the bug I thought I'd found was not, in fact, a bug.

It's interesting the obscure - but extremely helpful - stuff you find when you search the internet hard enough. I think I found a bug in the wipe routine tonight, so I went searching for reports of graphics glitches in the game; I haven't found any (yet) but did come across this blog entry right here on blogspot.com.au! Talk about perfect timing! ;)

That aside, I'm really down to the nuts and bolts of the code now. In fact, most of the rendering of all objects, both static and dynamic, is complete, excepting for sprites concerned with the player (of which there are quite a few) and interactions between objects. So it's time to roll up my sleeves and properly analyse the routines I've - temporarily - stashed in the too hard basket.

Actually the routine to wipe the old sprite was a bit of a diversion; I've been working on completing the high level game loop code (finished) and am about to turn my attention to having the keyboard inputs actually control the player. That will allow me to test all the player-related sprite handler routines before diving into the object interactions.

So if you smell something burning...

Cross-platform ports

I've been cranking the handle and have finished at least templates for each of the sprite update/handler routines. About half of them are completely implemented, so each room is now lively animated with moving blocks, balls, ghosts etc. The core engine is now 2,500 lines of C code.

Needing a break from translating Z80 assembler into C, and inspired by a query from a friend as to why I was using Allegro rather than SDL, I decided to take a step back and re-architect the code for cross-platform building as I did for Lode Runner.

As for Allegro vs SDL, it is simply a case of what I know. Allegro is simple to use but has some annoying kinks, so I was up for trying something new. After poring over tutorials and the API reference for at least 10 minutes, I had the Knight Lore menu screen rendering in a window! Unfortunately I then spent the next few hours trying to get keyboard input working, and have still yet to do so. Whilst I currently have both Allegro and SDL projects, the jury is out on SDL atm.

PC platform library issues aside, I now have template projects that build - and run - on Amiga, Sega Genesis and Neo Geo targets. The core code runs, but there's no graphical output yet. The Amiga and Sega Genesis projects at least display some text to show it's running. I should stress that this does not necessarily mean that I will be completing ports for all of these platforms. The Amiga is probable, I'd certainly like to do the Neo Geo if I can overcome a technical issue, but the Sega Genesis is probably unlikely.

With that out of the way, I should resist further work on the Amiga and/or Neo Geo ports and return to the C core implementation itself. There are probably a few more sprites that I can knock over quickly - such as the guards - but then I'll have to have a full implementation of the game loop and all its subtleties before going any further, as a lot of the remaining handler routines rely on that framework to be in place.

During this process I've had more of an insight into the mechanics of the code and I must admit it's not bad at all. For example, some of the game's state is implicit in the sprite number of particular objects, and the entire game logic is implemented within the object handler routines.

So from here-on in, the game will soon become playable. The player's character already appears at the start of the game from a cloud of sparkles, he just can't move yet.

UPDATE: One of the guard types and the wizard are fully animated and walk around the room now!
 

Don't believe everything you see!

Whilst things are progressing, it hasn't perhaps been as quickly as I'd like because I've been chasing down a couple of 'bugs' in my C implementation. I won't bore you with too many details, but I was testing the animation of the moving blocks when I came across room #29.

Whilst I hadn't noticed any issues with the room rendering prior to this, room #29 was obviously completely wrong...

This can't be right!?!

Spectrum Original

After spending a few hours chasing down this issue, including painstakingly comparing the graphics object table between the original (memory dump in MESS) and my C code, I couldn't see any differences at all.  And then it hit me...

Because I haven't implemented the Z-order logic yet, the rendering of objects is done in arbitrary order (more correctly, the order in which they are defined in the location table). Of course I realised this - what I didn't realise though, is that it could create the type of optical illusion you see above! If you follow the outline of the block structure with your eyes, it is actually correct.

Lesson learned, I'm continuing to work my way through the object handler routines. In fact, player, guards and wizard aside, I've finished most of the simpler objects. What I haven't done, though, is the low-level object collision/interaction routine, so atm (movable) objects fall through other objects to the floor, and pass through them when moving around. But they do move in the manner intended and are confined by the physical boundaries of the room.

Fun fact #34: spiked balls don't drop immediately in rooms with odd-numbered location ID!

There was movement at the station...

I've made some pretty decent progress today at lunchtime and this evening, deducing the purpose of no less than three more bytes in the object table, identifying another two as flags (leaving just 6!) - and the purpose of 3 of those bits - and a handful of global variables.

I thought I'd choose an object with the simplest movement - the portcullis that moves only up and down (i.e. along the Z axis) - and try to reverse engineer the handler for that object. It was quite successful, so much so that I now have a fully animated portcullis! Not unexpectedly this involved some generic low-level movement routines which are used by all the moving objects, such as bounds checking on the object which, of course, are also now implemented in C.

I guess I'll stick with the objects with the simplest movement logic, and work my way through to more complex object behaviour. Somewhere along the line the rest of the object table bytes should fall into place. Note that there is quite a lot of code to get through, and the next update of any significance may be a while away yet.

No more candy, back to meat and potatoes.

I've got all the static objects rendered in their correct locations now; the only objects remaining are those relating to the player and/or are dynamically generated and/or move. Because I can't really test any of those though, I'll leave them until I have implemented the code to allow me to do so. In summary, most of the objects are rendered and it's time to proceed to the next stage.

The choice I have now is to either analyse in minute detail and then implement the sprite priority (Z order) code, or continue with reverse-engineering the rest of the code.

I've chosen to go with the latter - I'm on a roll and I don't want to risk getting bogged down in the detail of one particular routine which is, ultimately, purely cosmetic (albeit important). I figure I can tackle that right at the end whilst I can see the light at the end of the tunnel.

The key data structure in the game is an array of objects that is constructed before you enter each room (screen), and contains an entry for each sprite including the player, special objects, and foreground & background objects. Each entry is 32 bytes and contains not only 3D location & 2D rendering information, but also state information for objects for example, that move.

Last night I deduced the purpose of another two of those 32 bytes, now leaving 11 (plus a few flag bits) still remaining unknown. And whilst trying to get to sleep shortly after it occurred to me what one of the unknown flags could be - but I'm yet to test that theory.

Besides those 11 bytes, there's around 30 other 'global' variables whose purpose also remains unknown. A few of those are simple flags accessed in only a few places in the code (which shouldn't be difficult to deduce) and at least a few others are temporary storage for the recursive Z order routine. Most of the game state is stored in the aforementioned object table.

EDIT: In one fell swoop I've confirmed my theory about the flag and also crossed another three global variables off the list!

And finally, it's clear that the sprite adjustment routines are actually the object handler/update routines (and to give you an idea of the work involved, whilst there are 188 sprites defined, there are 'only' 50 distinct handler routines), so I've got a bit of renaming to do in the disassembly and my C code. I am confident now though that I have full understanding of all the code and data structures and there should be no surprises from hereon-in; the puzzle is starting to come together.

I've got a fair bit of code to implement now as I need to flesh out the main loops and the routines I've already reverse-engineered but haven't yet bothered to port to C (as they don't play an important part in the rendering) and then start to implement the bulk of the handler routines as I reverse engineer them. There's a bit of code re-use so the C port might get a little messy at the lower levels and it won't be easy to keep track of what has and hasn't been fully implemented.

The next bit of eye candy probably won't be for a while, though it'll likely be a video.

What a difference a line makes!

I realised this morning that adding masking should be trivial - and it was basically a single line.

So here's the difference:

No masking

With masking

Note that the Z-order (sprite priority) has still not been implemented, hence the issue with the arch on the right hand side of the screen. But you can now clearly see the gargoyles sitting on the blocks.

Unmasked

I've worked my way through all the rooms now, adding the sprite adjustment routines for each of the foreground and background objects and implementing within each the logic that is strictly concerned with the sprite adjustment. As a result, the initial state of all rooms are now rendered 'correctly'; at least as far as pixel placement is concerned. What remains are a handful of adjustment routines for the player and the special objects that need to be collected and placed in the cauldron.

I suspect the 'sprite adjustment' routines are actually responsible for handling all the object interactions, so when I confirm that I'll update my nomenclature accordingly.

In the mean time, once I have finished the remaining sprite adjustments I'll implement the sprite masking, which I've thus far ignored. That will produce more pleasing results but the sprite priority (Z order) is still yet to be done.

No sprite masking as yet

You can see in the rendering above that the gargoyles sitting on the blocks are 'lost' in the blocks and the bricks on the walls, as is the arch on the western (front, right) wall. They're actually rendered last in the scene, but without the masking, which produces a black outline, they blend into the objects behind them.

From there I face another choice on how to proceed, but I'll discuss that next post.

I've also been thinking about potential target platforms for a port. The C code will allow me to port to the Amiga at least (I should also take the opportunity to do an Atari ST port while I'm at it), and the Neo Geo (hopefully) with some effort. The Sega Genesis will be more difficult than I first thought, and perhaps not even viable, as it will require use of the rather bandwidth-limited pseudo-bitmap graphics mode. I'll produce a PC port of course but I doubt anyone will be interested enough to look at it. Maybe a mobile platform will be a possibility?

As for assembler ports, the Coco3 was always the original target. But I've also been considering the TRS-80 (Grafyx Solution) - the results won't be stellar without double-buffering but perhaps still impressive enough to warrant a play through? Any other Z80-based bitmapped systems I've neglected?

Foreground objects

Another quick session, added and debugged logic to render the foreground objects.

Room ID=46

Like the background objects, there's plenty of per-sprite code to wade through, including what I believe is code to move the objects around. So basically, the bulk of the program. After rendering's done, there's very little code left to reverse-engineer!

In my PC port, I can move from room to room by simply pressing N, S, E or W keys.

From here-on in there's a lot of handle-cranking...

And the name of the game is...

A few more sessions and I've managed to reverse-engineer and implement enough to render the first room which, incidentally, contains no foreground objects. If this screen shot isn't enough to give the game away, then you need to hand back your Retro Gamer T-shirt.

A seminal title for the ZX Spectrum

At this point I'm obviously not concerned with colour. Technically the bitmaps are all monochrome and the game sets the attribute (ink) for different areas on the screen. Besides, not unlike Lode Runner, the game is perfectly playable in monochrome as is evident from the BBC Micro port.

Recent developments include identification and partial commenting of the recursive sprite/object priority-encoder algorithm (which I've yet to port) and - quite obviously - the 3D-to-2D translation and rendering routine.

For reasons not readily apparent to me, each of the game's sprites has a routine that tweaks the display coordinates; some are hard-coded and some depend on other factors. Why at least the hard-coded values weren't implicit in the sprite data is beyond me...

Next step is to render a room with foreground objects, which are handled differently to the background objects since multiples of which can appear anywhere in the room, and can also move around. However I believe I've already reverse-engineered most of that logic already. Like the background objects, it'll be a matter of implementing all the per-sprite tweaks that will take most of the time.

Still a few holes in my understanding of the data structures and variables but it's all coming together and I feel I'm heading towards the home stretch; less analysis and more handle-cranking - which isn't a bad thing. I still find the idea of a 6809 port a little daunting, but it's also a good chance to hone my 6809 coding skillz.

Chipping away slowly

Still progressing, even if slowly, each time I open up the disassembly to look at it.

I mentioned in a previous post a 'related' project; I'll talk about it more now since I had a closer look at it today.

The project involved modifying a ZX Spectrum emulator specifically for this game in order to render the screen using the OpenGL library on a modern platform - pretty interesting idea! Tonight I decided to read up on the technical details of the project - written in Spanish - and learned a thing or two about it. There's even information on the game in there that contradicts what I've read on another site, and I suspect that when I've digested it fully it may actually clear up some confusion I've had thus far in certain areas.

Technical details aside, the rendering algorithm looks a lot simpler and cleaner in C code. Unfortunately at this point, sections of it don't appear to bear any resemblance whatsoever to the original Z80 code; at least I can't find the corresponding implementation. To further obfuscate matters, the code - comments, variables and functions - is written entirely in Spanish. Early days yet, however.

Regardless, I have a feeling that I'm not too far off a breakthrough and although there's a fair amount of code to implement in C before I can render all of the game's screens, at the same time I'm also pretty confident that it will only take a small amount of code to render one specific location.

Jumping the gun

Further analysis of the code reveals a very different story to that I painted last post. I am, in fact, quite a way off from being able to render a screen accurately. Indeed, the amount of code I need to reverse-engineer before being able to render came as quite a shock! I assumed that most of the remaining code was to handle special objects and AI; however every object to be rendered calls into a jump table that calls common routines at least 2 layers deep, and all are rife with magic numbers.

I'm sure though, that I only have to understand a handful of them before the rest of them simply fall into place. And that is what I attempted to do tonight, with moderate success. Still, I've only just scratched the surface.

The up side is that once I can render a screen, there's not much of the game left to reverse-engineer. Underneath the impressive graphics, the game mechanic is deceptively simple; something later games in the series built upon with an enhanced engine.

One other thing I now understand; there is a utility available that supposedly lets you dissect the game's graphics and data structures, allowing you to study sprites, objects, room contents, and the entire map - the ultimate idea being that you could update or create your own game within the confines of the existing data structures. The only thing is, the menu option to render the room doesn't actually do anything!

Well that's because you have to reverse-engineer pretty much the entire game to actually render a single room and notably, the game disassembly available on the same site covers around 5-10% of the game code. No doubt I'm not the first one to be taken by surprise!

However, I do hope to go one better than the author of the above-mentioned utility!

UPDATE: The author of said utility has responded to my email claiming it was mere laziness that prevented him from finishing his utility! I suspect the truth of the matter - evidenced by the (lack of) information available on the site - is that he either had no idea what lay ahead, or more likely was put off by the effort required (most of the original reverse-engineering effort was done by someone else).

Next post: The reveal?

It's been a slow day today so I took the opportunity at lunchtime to work on the project.

I finished coding the routines that build the display list for the screen rendering and for a quick test, simply added a call to the rendering routine for each sprite in the list. The result; not surprisingly the sprite positioning needs adjustments but the display is encouragingly similar to what I would expect to see!

So obviously now I'll be reverse-engineering the rendering logic in minute detail and updating my C source as I progress. The next milestone is to have the screen rendering 100% accurately. Then I can finally post a screenshot and no doubt reveal the identity of the game!

At the risk of giving it away now, this game has seen a few ports (some home-brew) and a few remakes on more modern platforms. Despite that, and also occasional claims of having reverse-engineered the game, I can't find more than what I would describe as a preliminary attempt at a disassembly anywhere on the net. I don't like to re-invent the wheel where avoidable, but in this case I can't help feeling I'm actually doing just that. Regardless, I've put a fair amount of effort into this and it only makes sense to complete it myself now.

Having said all that, I did only just today find a rather unique project from 2006 that I will describe in a later post for fear of giving too much away.

As I've alluded to in previous posts, I've got some ambitious plans for this port, particularly where the Coco3 is concerned. I'm quite excited to see where I can take it, but I've still got many more months of work before I can even kick it off.

Getting ready to render!

A few more hours tonight on the port. Good progress.

The focus was on the dynamic data structures; how they are built, what their relationships are, and how they are used. As a result I've updated the disassembly with more accurate and descriptive labels and comments, including some of the work done by others that preceded me.

The good news is that I now understand most of what is required to render a screen. The game builds a table of (simple) sprites to be rendered; the first 2 are the player, the next 2 are special objects, and the remaining (up to 36) comprise background, then foreground sprites. I'll go into more detail in another post.

The bad news is that each sprite has a pre-render routine that fiddles bits and bytes in the sprite rendering table, and at this point I don't know what half the bytes in the table do. That said, I haven't even attempted to reverse-engineer them just yet.

Next session I'll complete the coding of the data structure building in C, and then start on the actual rendering just to see what falls out without the per-sprite data fiddling. I don't think it'll be long before I have something that vaguely resembles a proper screen!

All play and no work...

Fast approaching the end of the work trip and I've only just managed to find the time and muster the inclination to work on the ZX Spectrum port tonight. Between jet lag, dinners out, Skyping the family, watching Alaska-based reality TV on Discovery Channel and other distractions, I haven't quite achieved as much as I'd forecast on the project.

That said, the C implementation is shaping up nicely. Tonight involved importing data, and defining & initialising the object-related data structures that are used in the main rendering logic. Rewriting it in C definitely assists in understanding the relationships between the various data structures.

As I've mentioned previously, the output of this process should be a fully-rendered screen and a significant portion of the core game engine (reverse-engineering) complete. Another hint: there were a couple of games that followed this one, both updating the engine in the process. I would imagine that it wouldn't be too difficult to port these to the Coco3 once I've finished this one.

I've just discovered that Retro Gamer magazine did a feature on this game some time back. Just downloading a digital edition now so I can have a read before I fall asleep tonight.

Only 2 more nights here, and at least one of those is ear-marked for dinner, so I probably won't get a lot more done before returning home.

Going loopy!

Thought I'd post a quick update on the ZX Spectrum game reverse-engineering, since the last few posts have been off-topic and you'd be forgiven for thinking that I'd been irretrievably diverted from the task at hand.

Quite to the contrary I've done a few short sessions here-and-there and am progressing somewhat, chipping away at my understanding of the screen rendering and what is done where. This has, for the first time, involved running MESS and patching code. I think I've got a reasonable view of the code structure now - I've identified the main game (nested) loops and the purpose of most of the routines called from there - and it will probably be in the next session or two that I will have sufficient information to be able to return to the screen rendering in C.

The rendering code deals with all the major game data structures and whilst it means that there's a reasonable amount of work involved in re-implementing it, even in C, it of course implies that a lot of the game's core engine code will then be complete.

Once I have the rendering complete I'll post a brief description of the main engine structure because I think it's interesting, as well as the requisite eye candy.

From there the task gets a little harder as the remainder will involve the 'AI' aspects of the game and a multitude of specific-purpose variables and flags that are hard to deduce from the listing alone. This is generally where you have to start actually playing the game (more) and poking around in memory.

I've got two weeks overseas travel (USA) for work coming up this weekend, which will actually mean I'll get more time to work on this project (sad, lonely nights in a foreign motel room) so I'll ensure my laptop is up-to-date with the latest tools and source before I go.

I've also had an interesting idea about what could be done with this project once the reverse-engineering and/or porting is complete which I think will particularly interest Coco fans, but I'll save that for another post.

Madness from my sick bed

Struck down with a stomach bug today, unable to sleep much and trying to keep my mind off my physical ailments, my thoughts turned - naturally - to porting projects.

Now I have to warn you that this idea was borne from a stomach ache and nausea-induced haze, so you'll have to forgive the madness.

Of course plenty of clones of arcade games were produced on all of the 8-bit microcomputers back in the day. But what if one were to take one of those clones, and port it back to the original arcade hardware!?!

 Why on Earth would you bother? I hear you ask. And as I sit here gradually recuperating after a couple of pieces of Vegemite toast and some paracetamol, even I must admit that the idea sounds sillier by the minute.

Regardless, I did try to entertain the idea briefly. And I was thinking a good candidate would be one that came from the same CPU - to greatly reduce the effort required - and obviously similar hardware architecture. The only one that sprung easily to mind was a couple of rather decent Joust ports on the Coco; Lancer and Buzzard Bait (I did have a preference back in the day, but can't recall which).

Both the arcade and Coco are 6809 machines with a bit-mapped display, although the pixel mapping differs somewhat. I would imagine there's sufficient ROM and RAM space on the arcade platform. So not too far-fetched; it's worth noting that the arcade Tutankham didn't require a lot of patching to run on the Coco3! What it would require though is a pretty comprehensive reverse-engineering of the Coco code in order to account for the pixel mapping differences.

Maybe one day someone will pay me to complete absolutely pointless ports?

Project List

I was looking for a way to include links on the sidebar of this blog and stumbled across the mechanism for adding pages! So you'll note in the sidebar to the right is a link to the 'Project List' which, as you would suspect, is a list of my past and current porting projects.

Apple II Lode Runner for the Coco3 - recap

Just tonight listening to The CoCo Crew Podcast Episode #3 I heard myself get a mention in relation to my Lode Runner port for the Coco3, which was both flattering (thanks John & Neil) and embarrassing (thanks John & Neil). So I thought I'd give an update on the status of the project.

The port itself is all-but-complete and, for those that missed the announcement, a playable demo version is available for download. FTR I have done one (cosmetic) fix since that release. The only issues remaining are that the data for all 150 levels does not fit in the non-banked 64K address space, and there is no mechanism for loading/saving high scores. I do, however, have plans to overcome these - in time.

My intention is to release a cartridge version of the port initially, for which I plan to design my own flash-based cartridge, with a few extra features - including the ability to save high scores. Although the cartridge will be made available with Lode Runner (and Championship Lode Runner) installed, it will also be able to be programmed by the end user for their own (additional) cartridge images. There will be a limited run with a case and a printed sticker for those that wish to have a physical copy of the game. More on this when the time comes.

Sometime after the cartridge release, I plan on releasing the disk version - which will be freeware.

The question on everyone's lips is, of course, when will this happen? Right now, it's difficult to say. I am in the middle of a rather significant upheaval in my working life; the plan was to actually reduce my working hours and free up some more time for family and hobbies, however somehow I've managed to do the exact opposite - for the next 5-6 months at least. But at least it's all part of the process in setting myself up later for less work and more play.

Hope that clears things up for now. As was also alluded to in the podcast, and of course documented right here in this blog, I am working on another game port to the Coco3; I should explain that pure software work is a lot more conducive to "casual efforts" and of course does not require any capital outlay. So whilst I don't have the resources to dedicate to the flash cartridge design at this point, I can sometimes find the time to do a bit of reverse-engineering.

C9

No feedback thus far from the Microbee Volcano Hunter port, so in the mean time I've resisted other temptations and RETURNED to the ZX Spectrum game. I've got some additional annotation for the listing courtesy of George but failing any further developments, I'll wrap Volcano Hunter up soon and send the final source listing back to David Smith to do with as he pleases.

After a few more casual scans of the (aforementioned ZX Spectrum game) listing, I've deduced that the 'easy' bits are done and the remaining reverse-engineering is going to take some effort.

The next task is to definitively identify the top-level routine that renders the game play area of the screen and comment that as I translate it to C. Once that is complete, I have a feeling that more of the code will fall into place, as a lot of the data structures are related to the rendering. I'll also post some eye candy at that point and thus reveal the identity of the game.

Trying to ignore other ideas

The Microbee port of Volcano Hunter is complete, pending exhaustive testing. In the end the changes were almost trivial, for the most part cut-and-paste from my previous Microbee port, Donut Dilemma. In the end all that was required, aside from the usual preamble to initialise the Microbee PCG, was modifying hard-coded references/offsets to video memory with a .define'd constant, patching the keyboard and sound routines, and modifying the (one-and-only) delay routine to cater for a faster Z80. Almost too easy.

And of course immersing oneself in such a port - however briefly - and interacting with the enthusiast community, inevitably some suggestion or idea pops up from nowhere. Somehow the suggestion to port the MSX version of Lode Runner to the Microbee popped into my head; after doing the TRS-80 version I've little doubt it can be done; it's just a matter of whether the pixel-addressable graphics mode on the Microbee has sufficient bandwidth.

Then I started thinking about an enhanced port of Volcano Hunter, with a scrolling background (as opposed to the original flip-screen) on a (slightly) more capable platform - I'm thinking SNES or Amiga for example. Or maybe even Neo Geo... arrghh!!

And during discussions of the port with a friend, the topic turned to Prince Of Persia... I've had plans on that for quite a while now, only I'd forgotten that the source code is freely available. Ports are soooo much easier with the source... also trying to convince said friend to reverse-engineer DKJr; would be awesome to release a physical Neo Geo cartridge with ports of the arcade version of both Donkey Kong and Donkey Kong Jr!

But I need to ignore these for now and get back to my ZX Spectrum port before I forget how it all works. It may even be possible to port that to the Microbee - at least it's already Z80!

Short diversion

I mentioned Volcano Hunter in my previous post; I've contacted the game's author David Smith and he has sent me his original source listing - in the form of iPhone photos of fan-fold paper sheets!

Starting with a known-good TRS-80 CMD file (game executable) I've completed annotating the disassembly with all of David's labels and comments and I've even preserved the radixes of all his numeric literals! The ASM file now assembles and produces a byte-for-byte copy of the original CMD file - and is also fully relocatable!

Now that David has an electronic copy of his original source listing that assembles on a freely-available, modern cross-assembler (ASZ80), I'm going to start on the Microbee port!

Then it's back to the ZX Spectrum game, where things are starting to look interesting!

Status Update

Nothing ground-breaking yet but I am working on getting the Status portion of the screen rendered, which is done every iteration of the main game loop.

The main reason for this update though is to mention a recent discovery and the idea for a port that it subsequently spawned. Volcano Hunter was a TRS-80 Model I game that I played for many, many hours back in the day, even writing a utility to facilitate printing the game map - over 200 screens - on our Epson MX80 printer. The author had a web page about it quite a few years ago, but he never responded to my email and it subsequently disappeared.

Whilst browsing the AtariAge forums yesterday though, I came across a post about it and whilst Googling it, I discovered the author has - in 2011 - created a Facebook page for it. Not only that, but he seems to still be quite fond of the game, creating a playable online version, and printing themed T-shirts and keyrings.

That got me thinking again about an idea I've had for a while, namely porting a TRS-80 game to another machine besides the Microbee. Sure, the Coco would be nice, if possible, but what would really be cool in my opinion would be a very unlikely target - the Neo Geo! I'd dismissed the idea in the past as being not practical, but lately I've done some more thinking about it and I'm sure it can be done.

Anyway, whilst Volcano Hunter is out of the question for now, I've thought about doing a quick proof-of-concept port just for fun; something quite small, and ideally something I already have the source code for. Right now I'm thinking Tandy Invaders would be a good candidate!

Let the porting begin

I've begun my experiment - porting the mystery game to C on the PC. It's rendering text on the main menu and the call tree has been coded for the sprite rendering. Next task is to import all the sprite data/tables and implement the low-level rendering.

I'm keeping as close as possible to the structure of the original Z80 code; memory variables are defined as globals and register contents generally passed as parameters to functions. Variables and functions exist in the source in the same address order as the Z80 code, which I've always done in the past when porting regardless of target language.

I'm thinking of actually using the resultant C code as the reference for the 6809 port, rather than the Z80 code. The register set and, in some cases, the instructions are so different that I may ultimately produce cleaner code in doing so. Hence the reason I'm adhering to the original structure - it will still produce a faithful port if done right. It'll be interesting to see how this approach pans out.

I was going to release more clues as to the game's identity as I progressed but I can't really think of any cool way to do so that aligns nicely with my progress. So for now I will reveal, as someone has correctly speculated, that the game is indeed from the library of the Sinclair ZX Spectrum.

Meanwhile, I've been listening to quite a few episodes from a couple of Atari-centric podcasts, namely ANTIC and Player/Missile. I'm currently listening to episode #14 of the latter, in which a panel of 5 each choose 5 of their favourite 8-bit Atari games.

It occurred to me today that I should try to choose a seminal game from each platform to port to the Coco3. So far I've done Apple II (with a couple more in mind), I'm doing ZX Spectrum now, so perhaps I should attempt an Atari title next. Now I really, really do want to like the Atari but I'm having a hard time being impressed with any of the games. For example the praise for Jumpman was unanimously glowing on the above-mentioned podcast, but from what I've seen on YouTube, I'm thoroughly unimpressed - for example the animation of the player is absolutely horrid!

Hopefully I'll find one eventually... Cavelord looks interesting...

Easier to C what's happening!

Still managing the odd hour here and there and starting to get a real feel for the structure of the code and how everything hangs together. It's a fine line between trying to get a good understanding, and not getting bogged down in low-level implementation details that can wait until porting is under way.

I'm considering a slightly different approach for this port. Because so much of the binary is actually data structures, and a fair amount of the same maintained in RAM, plus the fact that a lot of the code is concerned with rendering the screen, I'm actually toying with the idea of doing a complete port in C (on the PC) before tackling the 6809 translation.

The code itself looks quite well structured, with very little jumping around, which of course lends itself to a high-level implementation. And debugging C code to parse data structures and render graphics on the PC is a lot more attractive than debugging 6809 in the MESS debugger running a Coco emulator.

I think I've just made up my mind to start coding in C next session.

Low hanging fruit

Steady but slow progress as time is in short supply; a major upheaval in my working life and plenty of things happening around the house in my private life. This probably won't change for a month or two yet either. Still, I try to grab an hour or so 'watching' TV with the wife or just before bed one or two nights per week.

I'm still getting a general feel for the code, jumping around and commenting sections that are easy to identify - low hanging fruit. I'll probably persist with this strategy until pickings become meagre.

The use of IX and IY registers appear to be limited to only a few data structures, so I'm feeling a bit more confident about a 6809 translation.

Not sure at this point when I'll revisit the rendering and/or code my first 6809 instruction for the port. I could write the main menu easily as it is now, but that's not very exciting at all.

Flippin' Hell

Another couple of short sessions - reverse-engineering the sprite rendering code.

Something I haven't seen before; read on! Sprites can be flipped horizontally and/or vertically on the screen. There's a single copy of the sprite data in memory due to memory size restrictions. Rather than flip during rendering, the sprite data is actually flipped - if required - in-place and then rendered, with each sprite data block keeping track of the current flip state. Clever, if sprites aren't flipped alternatively on the same screen (at least not very often).

Sprites must also be rendered at any bit offset and this is done at display time however - via self-modifying code. Not exactly uncommon for this type of operation though.

Generally with the low-level routines such as rendering there's no sense at all attempting to replicate the existing code since the underlying hardware can be quite different and often I'll convert the graphics data to a more suitable format. It also makes sense to optimise these routines for the CPU and hardware of the target platform. You'll find, for example, the Coco code for rendering graphics in Lode Runner bares absolutely no resemblance to the original Apple II code, and nor should it. The same will be true for the Coco code for sprite rendering in this port.

So with this in mind, I'm already thinking about optimisations that can be done on a machine with considerably more RAM. Like pre-flipping all the sprites and just choosing the appropriate memory bank to render to the screen. Perhaps even pre-rendering the bit offset sprites as well, especially if/when using 4-colour mode which will require only 2 offsets rather than 8!

As I delve into the code more, I'm feeling a little better about the port. Since so much of the game is concerned with rendering graphics, there may be less code that warrants line-by-line translation.

I'll probably spend the next few sessions doing more reverse-engineering until I fully understand all the rendering of sprites, complex objects and screens.

Made some time at lunch today to work on the new port again.

Added some more structure to the graphics data and commented the basic text display routines, allowing me to decode the font data which I subsequently rendered on the PC. I continued attempting to decode the graphics and managed to render the raw sprite data and had partial success rendering compound objects (made from multiple sprite blocks).

The game font in technicolour glory!

Here is where the format documentation on the net is somewhat lacking (and in some cases, actually not quite right) as I am yet to understand exactly how compound objects are put together. I made some basic assumptions and roughly half the objects look OK. I was hoping to be able to render a game screen in its entirety, but I'll have to defer any further progress on that front until I reverse-engineer the sprite/object rendering code.

What I've seen of the code doesn't bode well for portability to another CPU, however. The Z80 alternate register set, IX & IY, and self-modifying code are all used throughout the game. Preserving colour information is also going to be a task in itself. That said, I may need to take a different approach to my other ports, and rather than attempt a line-by-line translation, perhaps a routine-by-routine translation would be a better approach.

Oh well, if it was easy, every man and his dog would be doing it!

Jump, jump, jump, if you feel you want to

Got another couple of hours in tonight - mostly defining code and data sections and transcribing descriptions of the data blocks. Most of the code/data definitions are done.

The disassembly documents little more than the above, with a few dozen easy-to-deduce routines having been labelled; mainly to do with displaying on the screen, menu selections, playing audio and accepting user input. Pretty much the same routines I start with for all my disassemblies. Beyond that, very little has been commented, and literally only a handful of variables have been identified. It's a good head start, but soon it'll have reached its usefulness.

I haven't really looked much at the code, but I have noticed plenty of jump tables defined, and also fairly extensive use of the IX and IY registers. That's going to be... interesting... to port to the 6809.

I think next session I'll render the graphics objects on the PC.

Glutton for punishment

Back onto the subject of my new port - I've been toying with the idea of (also) porting it to the TRS-80 Model 4 (with hires board). Avid followers of this blog may recall that I originally attempted to port Lode Runner from the Apple II to both the TRS-80 Model 4 and the Coco in parallel. I gave it a good go but not long after rendering the title and level graphics I decided to discontinue the Model 4 port; 6502 to both Z80 and 6809 proved to be too much work.

I will reveal that this time around, the original game is actually written in Z80 (hint #1). The two main issues with a Model 4 port are the limitation of monochrome graphics, and the graphics bandwidth required for the animation in the game.

The original game doesn't make much use of colour at all - in fact it has even seen monochrome ports to other platforms in the past (hint #2). And the animation is quite modest for the most part, and arguably some slow-down wouldn't completely break the game. In any case, given the relatively little amount of work required, it may even be worth it even if it only ran 'properly' in an emulator, or on an enhanced (no wait state) graphics board which, BTW, is also on my to-do list.

Dusting off Neo Kong

John Kowalski has remixed Donkey Kong on the Coco 3 (added additional gameplay elements) and is reportedly in the process of back-porting his code to the original arcade hardware!

Anyway, I thought I'd dust off the arcade disassembly and send him a copy, which then resulted in John asking me a few questions about my Neo Geo port. And since I don't have a video of the latest developments, I thought I'd record a quick one tonight and post it on the blog.

To re-iterate, the disassembly of the arcade original is roughly 50% complete, and accordingly the Neo Geo port is also roughly 50% complete. What remains is the ironing out of a bug when Mario jumps, then adding all of the dynamic objects in the game, scoring and sound.

On the shoulders of giants

Got a little bit more time tonight and have loaded the binary into IDAPro and started on the disassembly. Still in the process of defining data and code segments - there's a lot of data in this game. I have a partial disassembly and data format document to work from, and I'm still in the process of transcribing that information into IDAPro. Probably have a few more sessions before I have to start thinking for myself.

Before I get too far into the disassembly I'll revisit the data format and write some code to render all the graphics objects on the PC. Then I'll write code to render each of the screens. Once that's done I should be very familiar with the data formats in the code.

Hopefully next update, which may not be for a few weeks, will have some eye candy - and no doubt will then reveal the game that I'm porting.

Interestingly, the graphics and controls may well be suitable for a Neo Geo port too...

It's a new day, it's a new game...

Far from finishing Donkey Kong or Lode Runner, I haven't touched any port nor any hardware project since the birth of my son in February.

Donkey Kong is waiting on the Neo Geo Flash Cartridge, which is laid out and ready for review but I currently don't have the funds to produce the prototype so it's on ice temporarily.

Similarly Lode Runner is waiting on the Coco Flash Cartridge, which I haven't even started.

So in order to get back into the swing of things, I've decided to start work on a new port to get me enthused. The target platform is, once again, the Coco 3. At this stage I'm not prepared to divulge the source machine (except to say that it's not the Apple II this time), nor the game itself, but suffice it to say that it was a landmark title in the day, specifically for its graphics.

I've been impressed with this title since I first read about it, and have had the idea of porting it for quite a while, despite the fact that I've never seriously played it. No doubt that will change during this process. Somewhat fortuitously the game has been disassembled to some degree and the data structures reasonably well documented, though work on it seems to have halted over a decade ago.

I'll say no more about it at this time, until I'm absolutely certain that a port is feasible (I'm fairly sure it is) and I have some eye candy to show off. Tonight was the maiden session and I've loaded the program image into a RAM array on the PC and wrote code to dump the data structures purely to familiarise myself with them.

The plan is to render the screens on the PC, and then start actually porting the code - which includes documenting the original platform source - to the point where I can do the same on the Coco 3.

Let's get this show on the road... again...

Tonight I decided to end the procrastination and crack open my Retro Ports again. The goal tonight was to fix the monochrome rendering of the GAME OVER animation in the Coco 3 port of Lode Runner. This is in preparation for delivery of a handful of Coco cartridge cases I'm expecting in the next few weeks. Not that I've designed the PCB yet...

It's amazing how much you forget when you haven't touched a project for a long time. I thought I had the bulk of it documented/scripted but there's a lot of detail you don't realise that you have in your head after working intensely on a project for months at a time.

In the end I managed to produce - and test - a monochrome rendering but I still need to reorganise the graphics mapping to accommodate both monochrome and colour variations. That's for another night...

Next I'd like to return to the C version - in particular the Neo Geo port - and fix the AI. I can't quite recall what else was lacking in the C version, but from memory it was very, very close (aside from sound). Once the AI is fixed I should be in a position to release a demo NGCD image.

All this has been driven by my urge to progress on the Neo Geo flash cartridge. I've been hanging out in the #neogeodev IRC channel again, and there's some talk of development and porting and it got me enthusiastic again. I got so far with Lode Runner it would be a shame not to finish it off properly. And once the generic C core is complete, porting to other platforms - like the Amiga and Sega Genesis - should be relatively straightforward.

Then of course there's Donkey Kong (Neo Geo) which to-date is, IMHO, still my most impressive effort. I'm determined to finish that off as well - I've put a lot of hours into that but it's languishing at 50% complete so there's still a bit of work to do.

 So much to do, so little time...