Super Metroid ripping project

[Raccoon Sam]

On July 18th, 2013 around 12:07 AM, ittan-moment made an important observation.

The super Metroid section's more than a bit dated as it were. Big over sized sheets, superfluous frames, and several missing parts.

i've been driven to the point of italics. the man is right.
look at this shit. LOOK AT IT. What is this, Ninsprites? yeah i exaggerated, no offense to original rippers. they ain't bad but cmon they could be a lot better.

In the same thread we kinda went off-topic but in a good way, with hints of new, fresh, accurate rips in the horizon. Well, I'm more of a technical smash-the-game-apart-guy more than an old school screenshot-til-you-drop-guy and I've dwelled into Super Metroid in the past. This is why I suggest that we create SMUSHER. The Super Metroid Ultimate SHEet Ripper. god that was lame.

You see, most of Super Metroid's sprites follow a pretty straightforward format. I've written about it in the past, but it was only a scratch of the surface. Now I've made a much more thorough document that should give the project programmer all the info he/she needs to integrate to the viewer.
In short, the goal is to have one easy-to-use program that can view every* individual sprite in the game. I can't program worth shit but here's what I know of the game's sprites. The ROM we'll be using is a headerless [!] (JU) SMC, MD5 hash 21f3e98df4780ee1c667b84e57d88675.

I. – DNA

At least every Samus-interacting sprite in the game has a unique, 64-byte DNA. It essentially tells what the game must know about the sprite to summon it, like where its graphics are stored, what its name is, hitbox size, etc. Most of the bytes in the DNA are useless to us because over half of the DNA defines non-visual stuff like initiation routines, "what sound to make", "how to explode" etc. Out of the 64 bytes, only twelve bytes are important to us. Size, Palette pointer, Master Bank, Piece amount, Graphics address and Enemy name.
The giant array of 64-byte DNA after another is located in 0x104EBF. There are at least 165 unique DNAs of which at least three are unidentified or broken.
165 * 64 = 10560 bytes of DNA data.

II. – EBI's sequence

Before we begin, you must make yourself familiar with converting SNES addresses to PC addresses. The pointers and words in the ROM are always little-endian and, of course, SNES addresses instead of PC.
From this point onwards, I'll also write in Courier because it's a monospaced font.

So, right. For our example, I'll use a randomly chosen DNA. This DNA is EBI's.
In 0x10663F, (SNES address $A0:E63F), we see 64 bytes. I've noted each.

SIZEB PALET ..... ..... ..... ..... MB .. ..... ..... ..... PIECE ..... ..... ..... ..... .....
00 06 87 86 2C 01 64 00 10 00 14 00 A8 00 4A 00 00 00 E0 87 03 00 00 00 1B 89 0F 80 4C 80 41 80

..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... GRAPHADR .. ..... ..... ENAME
00 00 04 00 00 00 00 00 0C 8B 00 00 00 00 00 00 06 8B 12 8B 00 00 00 94 B1 05 50 F3 CE EF 23 DE

legend:
SIZEB = How many bytes of data to rip from the ROM for this sprite to use. In this case, 0x0600.
PALET = Pointer to the 32-byte palette in the same bank as MB. In this case, $A8:8687, which is 0x140687 in PC. That's where the palette is.
..... = irrelevant bytes.
MB = Master Bank. Combined with many things, for example, the palette pointer.
PIECE = How many parts the sprite is made from. In this case 0x0003.
GRAPHADR = Address where the graphics are loaded. In this case, $B1:9400 which is 0x189400 in PC. That's where the graphics are.
ENAME = Pointer to enemy name, always from bank $34. In this case, $34E23, which is 0x1A5E23 in PC. That' where the name is, in ASCII. Not all enemies have names. Dunno if this is even useful for the program, but I guess it isn't that irrelevant.

III. – Frame Duration/Pointer Arrays

"But, but, there isn't a byte in the enemy's DNA at all for where the tilemap data is!"
Correct, but apparently $20 bytes after PALET there's usually† something called the Frame Duration/Pointer Array.

Counting (PALET + 0x20) will get you the offset where a Frame Duration/Pointer Array of the sprite in question begins. We want to see EBI's FD/PA, so let's count, PALET ($A8:8687) plus 0x20 equals $A8:86A7, which is 0x1406A7 in PC.

Here's a snippet from that very address (0x1406A7):

DURAT FRMPT DURAT FRMPT DURAT FRMPT DURAT FRMPT DURAT FRMPT DURAT FRMPT
0A 00 59 8B 0A 00 88 8B 0A 00 B7 8B 0A 00 E6 8B 0A 00 15 8C 0A 00 44 8C

legend:
DURAT = Duration of the frame. Exactly how long the frame is displayed in the sprite's animation. I'm not sure whether should this be considered an irrelevant byte, since these end up in a static PNG sheet anyway. Maybe useful for animators?
FRMPT = Frame pointer. This one's the important one. Remember how the first guide I posted tells you how the sprites are built? A frame pointer tells exactly where that info for one frame of the current sprite is. These frames are also always from the same bank as MB. In this case, $A8:8B59, which is 0x140B59 in PC.

Let's head over there. (I formatted these because you know how the format works already) 0x140B59:
09 00
FE 81 FB 06 21
EE 81 FB 04 21
FE 81 EB 02 21
EE 81 EB 00 21
02 80 FB 08 21
01 80 01 08 21
FF 81 07 0C 21
FE 01 10 26 21
FE 01 08 25 21

And THOSE, my friend, are the instructions of how to build the first frame of EBI. Those nine parts make this:

IV. – Final words

"But, but, where are the graphics loaded from?"
GRAPHADR, silly. Which was 0x189400. 4bpp planar, composite (2x2bpp):


"But, but, he looks all crazy colored! Where's the palette?"
PALET, silly. Which was 0x140687. 15bpp BGR:


"But, but, does he need all those graphics himself?"
No, silly. The correct amount of bytes the character in question needs is determined by SIZEB. Which was $600 bytes:


"But, but, that's only one frame! What about the next one?"
Let's return to the frame pointer array. We already covered the $A8:8B59 (which lasted 0x000A frames, by the way), the next frame according to the frame pointer array is $A8:8B88, which is 0x140B88 in PC:
09 00
FE 81 FB 06 21
EE 81 FB 04 21
FE 81 EB 02 21
EE 81 EB 00 21
02 80 FC 08 21
01 80 02 08 21
FF 81 07 0C 21
FE 01 10 26 21
FE 01 08 25 21

I made a GIF of this one so you get the idea:

The succeeding block/tile goes always behind the preceding one. In this one, tile $25 is the 'bottom' one, thus 'behind' everything. But hey, now we already have the second frame!!


"But, but, how do we know when to stop? How many frames does a sprite have?"


uh



Yeah that's kinda where the fun ends. I don't know. I can't know. There's no single table of how many individual frames a sprite has because some variable frames are built on-the-fly by arbitrarily executed code.
That's why I propose the viewer should look something like this:


That isn't meant to be final, but just to get the idea. It reads the DNA array, 64 bytes at a time. Clicking 'next' goes to the next DNA and so on. You could also type the address by hand (or via dropdown menu?).
The little hex viewer down there shows the Frame Duration/Pointer Arrays of current DNA (which it gets by counting PALET + $20, remember?). Then you could click(?) on the nearby four byte DURAT/FRMPT-combos and they'd get updated to the sprite viewer.
Fake Kraid, though, is an example. It's PALET is 0x13198C, and adding $20 to it gets you to 0x1319AC. If you try to read the data from that address, it will load it incorrectly. Fake Kraid's first real DURAT/FRMPT-combo is at 0x1319AE, two bytes after the expected.
Like I mentioned in †, it's only usually. As we just saw, Fake Kraid's stuff is $22 bytes after the palette. That's why an 'explorer' is needed.

Also here's a bonus gif of building Fake Kraid's first frame.



uuuhghgh too much text for one day. Questions? Suggestions? Ideas? I'm out for now. apologies for any typos or inconsistencies..