[PS2] Need help figuring out what to do next!

[Samevi]

Hello,

I'm trying to rip from the game Zero no Tsukaima - Maigo no Period to Ikusen no Symphony, something that's been a passion project of mine for the past while. Problem is, I'm stumped. I've got a file called SCENEDAT.bin which I know contains TIM2 files thanks to opening it in a hex editor, but I'm not sure how to turn this into a format from which I can readily grab said TIM2s.

File here, in case you want to check for yourself.

[Raccoon Sam]

Usually when dealing with this exact issue, TextEr (https://wiki.vg-resource.com/PlayStation...8TextER.29) is the best bet. However, in this case I got the error:

Code:

Looking for TIM2 and TIM3 file(s):

Sorry pal. No TIM2/TIM3 files have been detected

Which unfortunately means that the textures are compressed.
Naive compression analysis is possible and I've done it before but it requires a raw sample from a save state, so I don't know if I can do much without one.

If you can run the game in PCSX2, please make a save state from somewhere with graphics and we can try extracting the sample from memory. Bear in mind the scene has to contain legitimate graphics, so a scene with an FMV playing doesn't count.

[Samevi]

Usually when dealing with this exact issue, TextEr (https://wiki.vg-resource.com/PlayStation...8TextER.29) is the best bet. However, in this case I got the error:

Code:

Looking for TIM2 and TIM3 file(s):

Sorry pal. No TIM2/TIM3 files have been detected

Which unfortunately means that the textures are compressed.
Naive compression analysis is possible and I've done it before but it requires a raw sample from a save state, so I don't know if I can do much without one.

If you can run the game in PCSX2, please make a save state from somewhere with graphics and we can try extracting the sample from memory. Bear in mind the scene has to contain legitimate graphics, so a scene with an FMV playing doesn't count.

I guess compression should've been my first thought. Anyways, as requested, here's a PCSX2 save state that contains gameplay graphics.
Just to make sure I understand correctly, it's possible to rip all of the game's assets at once through this method?

[Raccoon Sam]

Excellent. After unzipping the save state I ran texter again through eeMemory.bin and this time got

Code:

Looking for TIM2 and TIM3 file(s):

Found 13 TIM2 and 0 TIM3 in eeMemory.bin.

Extraction in progress

TIM2: [100%]

Which makes sense because the game had decompressed a few TIMs into memory at runtime, and because a save state is a representation of the game state, texter was able to snag the decompressed TIM from there.
Here's the loot: https://drive.google.com/drive/folders/1...sp=sharing

Now it is time to analyse the compressed TIMs from SCENEDAT.bin and understand what the function is that transforms the compressed data into the uncompressed data.

I am analysing the differences now and it looks doable. Just need a bit of time to work it out, should be done soon hopefully.

Just to make sure I understand correctly, it's possible to rip all of the game's assets at once through this method?

yes. When done, my tool will decompress everything from the scenedat, and I'm assuming it's all the graphics in the game

[Samevi]

Excellent. After unzipping the save state I ran texter again through eeMemory.bin and this time got

Code:

Looking for TIM2 and TIM3 file(s):

Found 13 TIM2 and 0 TIM3 in eeMemory.bin.

Extraction in progress

TIM2: [100%]

Which makes sense because the game had decompressed a few TIMs into memory at runtime, and because a save state is a representation of the game state, texter was able to snag the decompressed TIM from there.
Here's the loot: https://drive.google.com/drive/folders/1...sp=sharing

Now it is time to analyse the compressed TIMs from SCENEDAT.bin and understand what the function is that transforms the compressed data into the uncompressed data.

I am analysing the differences now and it looks doable. Just need a bit of time to work it out, should be done soon hopefully.

Just to make sure I understand correctly, it's possible to rip all of the game's assets at once through this method?

yes. When done, my tool will decompress everything from the scenedat, and I'm assuming it's all the graphics in the game

That's beautiful. Thank you very much!

[Raccoon Sam]

Just giving a quick update: There are more compressed TIM2s for other graphics than that those of the character portraits and whatnot in the other DAT file on the disc. The tool should handle those as well.
And then the bad news: after some deeper analysis I've found the compression, like I said, only looks doable. In reality it looks deceivingly simple but in actually appears to be very complex, even nonsensical at times.
I was expecting to whip out a simple script that just churns through the TIMs and decompresses everything in one evening as a fun programming exercise, but now that I actually understand that the rabbit hole goes a bit deeper than I anticipated, I'll need way more time with this. Sorry!

In short: I'm still interested in reverse engineering the compression but it wasn't as simple as I thought.

[Samevi]

Just giving a quick update: There are more compressed TIM2s for other graphics than that those of the character portraits and whatnot in the other DAT file on the disc. The tool should handle those as well.
And then the bad news: after some deeper analysis I've found the compression, like I said, only looks doable. In reality it looks deceivingly simple but in actually appears to be very complex, even nonsensical at times.
I was expecting to whip out a simple script that just churns through the TIMs and decompresses everything in one evening as a fun programming exercise, but now that I actually understand that the rabbit hole goes a bit deeper than I anticipated, I'll need way more time with this. Sorry!

In short: I'm still interested in reverse engineering the compression but it wasn't as simple as I thought.

That's fine! I'm more than willing to wait for this - and with any luck, it could be useful for understanding other PS2 games with similar compression methods as well.
On that note, I'm more than willing to compensate you for your time (at least a little. I'm afraid my budget isn't too big right now) once all is said and done, just let me know.

[Samevi]

Hey, just wanted to bump and check if there was any progress on that decompression. Not trying to rush or anything though so no sweat if you don't have it yet.

[Raccoon Sam]

Hey there!
Bad news – I really can't wrap my head around it, I'm afraid :C

I've got samples for a compressed TIM2 file ("paktim" is just my nomenclature) extracted from the SCENEDAT and a legitimate, uncompressed counterpart TIM2 for it extracted from your save state. You can download them here.

The first eight bytes in the paktim are a header; 40 64 04 00 01 00 00 00 which should be read as two words, 00046440 00000001 which in turn just define the uncompressed size (0x46440 or 287808 bytes, which unsurprisingly is the file size of the legitimate TIM2 file) and the amount of files in the entry (1, which unsurprisingly is the ... uh, amount of files)


After that, you read a byte as the variable instruction.
Logical AND instruction with 0x1F and add 2 to it to get the variable amount.
Shift instruction right five times to get the variable method.

According to the method and amount, read bytes from the bytestream:
Method 7 is just "read amount bytes and write them to output".
Method 6 is "read 1 byte as surrogate and then repeat that byte exactly amount times to the output".
Method 0 is "repeat FF exactly amount times to the output".

The other methods are.... yeah, I dunno. Some methods look like they make sense but then the same method used later in the file does something completely different. I really can't figure out what the trick here is. Really hope someone else takes a look at this, it's driving me nuts!!

This is the TIM2 we're dealing with, for what it's worth.