Previously, I had a bug on my emulator with the way the map is rendered.

With the bug

How it should look like

After I wrote the previous article, I posted it on the SMS power discord, asking for ideas on this corruption bug. Many great people chimed in.

A question I had was if I could dump the tileset to look at the video ram of the VDP to find out if anything might be corrupted.

Interlude: Tilesets

A tileset is the list of tiles (sometimes called "patterns" or "characters") in memory that can be be used for background or sprites.

On a typical Game Gear game, those live in the ROM, then are copied in Video RAM (VRAM) once the ROM is mapped in the address space. They can only be displayed by the VDP from the VRAM.

The VRAM can hold 512 tiles, and each tile is 8x8 pixel. So I initially rendered the memory region in a 256x128 tileset (32x16 tiles)

But the map looked weird, and someone rightfully told be it would look much better if rendered in the other dimension (16x32 tiles):

The map is entirely visible! It's as if the artists worked exactly like this. We can clearly see the corruption bug in the first two tile lines.

Something feels off though: it seems like the bottom half (usually used for sprites) is using the wrong palette. In the VDP, background can use both palette 0 and 1 (0 by default), but sprites always use palette 1. Let's render the bottom half with palette 1 instead:

Transparency is also respected which you'll be able to see by dragging the image on a desktop browser (grey backgrounded added for readability).

But something is still missing. Like those sprites are split or something. What if we tried to honor the SIZE bit for the bottom half in order to see double-size sprites ?

Much better ! We can clearly see Dr Eggman, numbers, as well as the level lines. Now that I had a nice-looking tileset, maybe I could try to find the source of the corruption ?

VRAM read/write CPU buffer

An SMS Power member suggested that it might be related to the incorrect reads from the VRAM. What did they mean ?

Here is how I imagined the a VDP read and write to VRAM happened:

CPU would use an OUT instruction to the VDP to select the I/O address, and then would do an IN for the VRAM read; VDP would respond by fetching the data and giving it to the CPU.

For the write, after an OUT instruction to set the I/O address, and an other OUT would write the data directly to VRAM.

But of course that's not how any of this this works. Because of physics. There's a real latency to do I/O to and from VRAM. And to work around that, the VDP has a 1-byte "cache" buffer used for VRAM transactions.

A more accurate version would look like this:

There are actually two types of OUT addresses to set an I/O address: one for setting a read address, and another for a write address. And they have different behaviour, which is not at all apparent in the official Sega Game Gear Hardware reference manual.

Setting a read address, will immediately trigger the read to VRAM. This read will be stored in the 1-byte buffer in the VDP. The subsequent IN instruction will read data directly from this buffer, instead of from VRAM. This way the latency stays manageable. And just after the each read, from VRAM, the VDP will auto-increment the VRAM address, and fetch the next byte.

For writes, the story is similar, except setting the write address has no other side effect. Doing a write will first write data in the 1-byte buffer, write the buffer to VRAN, and then auto-increment the address.

Why the auto-increment ? Because it allows this kind of pattern:

Sequential access. The address only needs to be set once, and then the following I/O will automatically be at the next address. For reads, it also means that the next byte should already be pre-fetched and readily available in the buffer. It's a particularity of Reads: they only happen from the 1-byte cache buffer, and trigger the next byte fetch.

You probably see where I'm going with that. What if you do a read without any previous read or read address setup ?

You have an edge case ! You'll read whatever was in the buffer in the VDP, and this might not be what you expect.

Here is another edge case to show a more likely sequence of operations:

Setting a read address will do the fetch in the 1-byte buffer, and also auto-increment the VRAM address in the VDP; so if the next instruction is a write instead, the write will happen at address + 1 ! And if you do a subsequent read, you'll read the value of what you previously wrote, which is not really useful.

The way the hardware works is in fact more complex than what I expected. And this was widely known at the time, the developers used this behaviour in actual ROMs, for example to skip a byte after a read and write at the following byte.

Once I fixed that I noticed that Sonic Triple Trouble's demo behaviour changed, which broke some tests. I can no longer automatically reproduce the screenshots from the previous article, because they were generated with a buggy VDP read/write implementation.

And this bug ?

Unfortunately, adding the 1-byte buffer did not fix the map corruption bug.

So I had to go deeper, and look at the code being executed. In its current form, gears does not have a debugger; but there a few features: pressing space can stop/start the execution, I also added single-frame stepping with another key. And there's a parameter in the code to print every executed instruction. Coupling that with tracking down VRAM accesses (read and write), I noticed something weird, a dozen or so instructions before the corrupt data was written: two consecutive read instructions, that went from address 1023 to 0, with no address setup in the middle. It's as if the auto-increment wasn't working !

Once this issue was noticed, finding the buggy code and fixing it was quite simple. I had put a wrong constant for VRAM address auto-increment, but only for reads; I later removed open-coded constants to prevent this type of issue.

Finally, here is the tileset, as generated without the corruption issue:

Startup tileset

I also generated the tileset for the starting animation of Sonic 1 for fun:

Bonus: region

As I was tracking down this issue, I wondered if this could be related to some other feature/device I didn't implement. I looked at the system I/O port and found that I did not implement the region bit, hardcoding World (instead of Japan). I was surprised to see my regression tests start failing when I changed the region:

Sonic splash screen on World region.

Sonic splash screen on Japan region.

The ™ disappeared ! I remember reading about this online, so this is already widely known, but it still surprised me. It also affects the Press Start screen:

Sonic Press Start screen on World region.

Sonic Press Start screen on Japan region.

What's next ?

Many games still don't work at all, starting with Sonic 2 that does not display anything, just a black screen. I might look into what happens there next. I suspect it might be something related to timings or interrupts, but who knows ?