|
Post by TrekkiesUnite118 on Oct 7, 2018 5:44:58 GMT
I removed the MDT file that was causing the game to crash and replaced it with it's Japanese counterpart for the time being while I figure out whats wrong. On the bright side though some of the other areas look to be working ok with some minor graphical glitches: From the looks of it, in this area the palette data is wrong. So I probably need to figure out where it's trying to read from the wrong spot and fix it. The English PS1 version added/removed chunks of null data from the files, probably to increase/reduce padding for this very issue.
|
|
|
Post by TrekkiesUnite118 on Oct 6, 2018 5:42:34 GMT
So I've been able to figure out how to get the game to use the 8x16 font by default. Turns out the reason I couldn't find the code that I needed to change is because it doesn't exist in the games compiled code. Instead those instructions are generated at run time based on values the game reads from other files. This could be a form of compression, or it could be some kind of automated code generation. I'm not really sure which.
However, I was able to find what value it's using to generate the code that determines what font to use. Turns out it's in the SCNRL.BIN file. At offset 0x22A4 is the value 0x9D. This value get's XOR'd with the value 0x54 to create 0xC9. This corresponds to the And instruction for SH-2. Changing this value to 0xB4 will instead result in the value 0xE0 being created, which is the Mov instruction we need instead to use the 8x16 Japanese font.
Now the game is still crashing after the intro cutscene when it tries to load Parm. However I think this is due to the MDT file having an issue with it, as when I swap back in the original Japanese one it no longer crashes. So I'm going to need to dig back into these files and figure out whats wrong.
|
|
|
Post by TrekkiesUnite118 on Oct 4, 2018 6:09:36 GMT
So time for some news. First thanks to some help from Klarth/Mr.Conan over at SegaXtreme/Romhacking.net I've found out how to enable the 8x16 English font. The address 0x0601CEA4 has a byte that seems to control this. When it's set to x00 we get the 16x16 Japanese font. When it's set to x01 we get the ASCII 8x16 font. Digging further in the debugger for YabaSanshiro I was able to find the assembly code that controls this. At 0x0601E2A4 there's these lines: extu.b r0, r0 shlr2 r0 and #0x01, r0 mov.b r0, @r3
The last 2 lines are the important code here. This code sets r0 to x00 and then writes it to the address at r3. That address happens to be 0x0601CEA4. Doing some testing, I was able to successfully force the game to use the 8x16 font by changing the code to this: extu.b r0, r0 shlr2 r0 mov #0x01, r0 mov.b r0, @r3
This seems to do the trick. When viewing the memory address for this code I found these values: 600C 4009 C901 2300 This appears to align with the Opcodes for these instructions according to the SH-2 manual. If I'm understanding things correctly, I believe I need to change it to this: 600C 4009 E001 2300 Which would align with the Opcode for the replacement mov instruction. The problem is however, I can't seem to find this sequence of hex values anywhere in the game's executable, or anywhere in the games BIN/ISO for that matter. I found 2 values that were close, but they didn't seem to make any difference when I changed them. So I'm not sure what to do here. Moving on however, I have some more news concerning this. I have continued work on my program for converting the files and I believe I may have successfully converted the PS1 script data over to the Saturn format. When combined with the memory edits above, I was able to get this: Now this still isn't perfect. As while this cutscene plays fine, the game seems to hang afterwards when it tries to load Parm. The music and sound effects play, but the screen is black and the game doesn't seem to be progressing any further. Also not all the sound effects and voice clips seem to be playing during that first cutscene. So probably some debugging is needed to find out what's gone wrong. However the fact this seems to work in some cases looks promising that we can convert the English PS1 files over to the Saturn game rather easily. I intend to release my code as I get it cleaned up as it's a bit sloppy at the moment. But if I get it working nicely, I'd imagine it could make it really easy for any future retranslation projects to apply their translation to both versions of the game.
|
|
|
Post by TrekkiesUnite118 on Sept 22, 2018 20:36:58 GMT
So I wrote a program that allows me to parse the MDT and MDP files from the Saturn and Playstation versions. This basically strips the first 512 bytes and reads them as a pointer table. This information is then used to parse the rest of the file. Knowing that 0x60 marks the table entry for the script, I wrote some code to copy the bytes defined in the table and write them to a file. This has essentially allowed me to dump the raw script for the game out of all the MDT/MDP files.
I've made it so it also writes the pointer table to a file, as well as the pre-script and post-script sections to files. This way it should be easy to make some code to do the reverse and put all the pieces back together and recalculate the pointer table.
On to the more interesting stuff though. After dumping the script for the Japanese Saturn, Japanese PS1, and English PS1 versions, I discovered that the Saturn and Japanese PS1 script data is identical. The initial cutscene I was looking at appears to be a special snowflake where they've added a bunch of additional control codes in the PS1 version. After that, things start lining up between the two. The only difference is the non-script parts of the data are byte swapped due to endian differences.
This is making it really easy to start identifying pointer values and control codes by comparing the different versions. I'm still figuring things out in these files, but I'm starting to think it might be possible to write a program that could automatically convert the English PS1 script files into a format the Saturn is expecting.
|
|
|
Post by TrekkiesUnite118 on Sept 18, 2018 4:43:48 GMT
Ok, so I think I've cracked the code of the MDT/MDP file headers. This information seems to hold true for both the Saturn and PS1 files, the only difference being that in the PS1 files pointers are Little Endian where as in the Saturn files they're Big Endian. - The header is 512 Bytes in size.
- The header seems to consist of just a pointer table
- Each entry is 8 bytes total in size
- The first 4 bytes seem to be the place in the file where the data lives
- The second 4 bytes seem to be the size of the data.
- The entry that tells us where the text lives and how big it is seems to be at x60 every time.
- There's an entry at x58 that seems to point to a 24 byte piece of data right before the text. Perhaps this is a header or something?
- Consistently I'm seeing in the English PS1 version that any addresses that come after the address defined at x60 have their values increased by the increase in size of the text data, and any values that come before it are untouched.
- I'm seeing that padding is being removed in the PS1 files of the exact amount of bytes added to the text.
- Another oddity I'm seeing is that the last entry in the table at x1F8 seems to also have it's size increased by the increase in text size. But I'm not sure what that value represents or what data was added.
Some other oddities I'm seeing is that some entries don't seem to make sense as the addresses are beyond the size of the file, but they're still updated in the English PS1 version. With this info I imagine I should be able to make a program that can do the following:
- Parse the MDT files.
- Extract the script portions.
- Put the MDT file back together with a new script portion.
- Recalculate the pointer table.
The hard part to automate in code I'd imagine would be to find a portion of padding in the file that can be removed to keep the file size the same.
|
|
|
Post by TrekkiesUnite118 on Sept 16, 2018 1:18:29 GMT
So after making a topic on RomHacking.net it was revealed that the 8x16 English font is actually in the Saturn version, and it's in ASCII format. To use it you just need to start your text line with the code 03 and then you can write ASCII encoded text. I tested this and got these results: So basically this makes things a lot easier I'd imagine. Now it's just figure out the control codes, any pointers in the files for text size and location. With that information it probably wouldn't be too difficult to write a tool to dump the JP script, and then another to inject an English script with pointer values adjusted for expanded text. Though it would still probably be beneficial to make some kind of hack to make the 8x16 font the default font so you don't have to waste a byte with the 03 control code. I'm guessing this is what the English PS1 version did as I'm not seeing any evidence of a control code like that being used in it's files. EDIT: So I think I might be figuring out how the script pointers work based on comparing the PS1 versions, and then spot checking with the Saturn version. So here is what I believe to be one complete text routine from the file BA38.MDT at x05B4: This highlighted portion I believe is a header of some kind. It seems to change from text sample to text sample, so there's probably optional values that are in some text boxes but not in others. Now this next highlighted portion is the important part. This value I believe represents how many bytes until the end of this text box. In this example it's 5A: If we then move forward 5A that will bring us to the end of this block of data. The end of a block of text data seems to be tagged with this possible footer: After this the next text box starts. I've been able to keep following this pattern and consistently keep finding what appears to be the correct number of bytes and the start of the next text box. I'm going to keep testing this but so far it's looking promising. Also, another interesting thing of note is that it seems that the English PS1 versions files seem to be close to if not the same size as the Japanese PS1 files. It looks like Sony removed unnecessary padding to keep the files close to the same size from what I can tell.
|
|
|
Post by TrekkiesUnite118 on Sept 15, 2018 7:19:45 GMT
So I've started putting together a font table for the game as I'm figuring out the encoding more. I have it done for the English portions of the Font, the Hiragana, and the Katakana as well as the special characters. I think I have an idea of how the Kanji encoding works, but it's going to be some trial and error to get it right. I figure if I can get this made it should make it at the very least easier to identify the Japanese text in a hex editor, as well as possibly make it easier to dump the script. It could also make it easier to spot the control codes. I've also been thinking about the issue of getting the font size changed from 16x16 to 8x16. CyberWarriorX has some good notes in this old thread about how the font and it's compression system works: segaxtreme.net/threads/saturn-hacking-notes.4934/Basically I think this is going to require some ASM hacking of the text decompression routine and possibly the text drawing routine. The Decompression routine works by decoding the first 8x16 portion of the character, followed by the next 8x16 portion. I'd imagine this would need to be changed so only the first half is decoded and treated as one character. I'd also imagine that this would impact the pointer table of the FDATA.BIN file. Finally depending on if the game is spacing the text based on font width or if it's spacing it in 16x16 increments regardless of it's size, would determine if the text drawing routine would need to be modified. This is something that is way above my skill level so I'm probably going to need some help on this one. I was planning on reaching out to CyberWarriorX and seeing if he had any thoughts on it or if he wanted to jump in and help. If anyone here wants to give it a shot feel free to jump in. I was also thinking about reaching out for help on ROMHacking.net. I have however had some success figuring out a bit more about the control codes. The code 090F that I before thought was end of text I believe is actually 1 code plus the start of another. The first code is 09. This seems to indicate a pause in text drawing. It seems to be used to pause the text to keep it in sync with pauses in the voices. The next code is actually 0FXX. The XX I've seen be any value from 01 up to 0A so far. This seems to be used for various different things. What I've found so far though is the following: 0F03 seems to be related to controlling the Left Portrait. 0F21 seems to be related to controlling the Right Portrait. The next code I was able to figure out is this one: 0A 0C 00 XX This code tells the game to switch to a different portrait, where XX is the value for the portrait you want displayed. So for example if you see the following: 0F 03 0A 0C 00 XX It would appear to mean "change the left portrait to portrait XX". I was able to confirm this in testing by changing the portraits in the intro cutscene to arbitrary values: There's still quite a few different values for 0F XX however, and I haven't quite been able to nail down what they all do yet.
|
|
|
Post by TrekkiesUnite118 on Sept 13, 2018 3:38:29 GMT
So I decided to play around with the font tonight. I did a simple test where I took the English PS1 font and stretched it to now be 16x16 in size. I then copy pasted it into the Japanese Font file in ASCII order. This way in theory the script files could be edited into English with ASCII encoding to make things easy. And other than a few letters being off, it proved to be successful: I'm attributing the few incorrect characters here to two things: 1) I was probably off in my count for the ... character. 2) I probably missed a control code that I haven't figured out yet that controls Kanji characters. Obviously I didn't edit much beyond this point but the game still prints the font out ok. It's just now everything is truly gibberish as expected:
|
|
|
Post by TrekkiesUnite118 on Sept 11, 2018 4:56:37 GMT
So doing some more digging I've found a couple more nuggets of info: - The Japanese PS1 and Japanese Saturn version use the same encoding for their text.
- The English PS1, Japanese PS1, and Saturn versions seem to use the same control codes with some catches
- The PS1 versions seem to add additional values in the codes between text lines/new boxes.
- Some codes are also used differently.
So with the control codes, I've figured this out looking at the file BA38.MDP/BA38.MDT in the different versions. This is the file that contains the intro scene I edited earlier among other things. When the scene starts, we can see this code before the text across all 4 versions: 0901 090B 8020 0F03 0A0C 0030 It would seem that this indicates the following: - Start a new text box for a new character.
- Text Box is on Top
- Character Portrait is on the Left
- The character is Baal.
- Face palming.
When he speaks again however, we now see this: 0901 0F03 0A0C 0030 This seems to indicate the following: - Start a new text box
- Same Position
- Same Portrait Position
- Character is Baal.
- Face Palming.
The next big code we see is this: 0901 0F03 0A0C 002E This seems to indicate the following: - Start a new text box
- Same Position
- Same Portrait Position
- Character is Baal.
- Relaxed face.
Next in this scene the character Mullen is the one speaking so we now see this: 0901 090B 8021 0F21 0A0C 0022 This seems to indicate the following: - Start a new text box for a new character.
- Text Box is on the Bottom
- Character Portrait is on the Right
- The character is Mullen.
- Serious? Face.
Next, Baal speaks again, only now with this code and it's slightly different in the English PS1 version. In the Japanese and Korean versions it's: 0108 0F03 0A0C 002F In the English version it's this: DC08 0F03 0A0C 002F This would seem to reflect the following: - Use the previous text box settings (Position, Portrait Position, etc.).
- Character is Baal again.
- Mild Chuckle Face?
Next Mullen speaks again and we see a code that differs between the English version and the other versions. In the Japanese and Korean versions it's: 0108 0F21 0A0C 0021 In the English version it's this: AA08 0F21 0A0C 0021 This seems to reflect the following: - Use the previous text box settings (Position, Portrait Position, etc.).
- Character is Mullen again.
- Relaxed face.
So this would seem that the value 0108 does mean use the previous text box settings. Next we see this code come up quickly: 01 0F21 0A0C 0026 This seems to indicate that in the same text box mid sentence we need to change the face portrait to a different one. This is used in the cutscene to change Mullen's expression. It would seem that the value 01 is an interrupt to tell the game to change portraits. Now I can keep going but I think the patterns should start to become obvious. I've also noticed the following values coming up to signify the end of a line in the text box or the end of a textbox: 0E00
090F The problem however is I can't seem to find consistency for which is used where. The PS1 sometimes uses 0E00 where the Saturn version uses 090F, but then there's instances where both are using the same values. Now after the end code however, there seems to be varying bytes of data. The size of bytes seems to be the same in the Japanese and Korean Saturn versions, and in many cases the same values. However when we compare to the Japanese PS1 version, the size of bytes between these gaps is different. There's additional data added in. However there is still some data that matches, however bytes are flipped which is consistent with endian differences. This would seem to indicate these are perhaps pointers. I have no idea what the additional data that's added into the PS1 version is though. Obviously more research is needed here as well as some experimentation, but this seems to be consistent when stepping through the script file in sync with the cutscenes so far.
|
|
|
Post by TrekkiesUnite118 on Sept 10, 2018 4:05:27 GMT
In comparing the script files between the Korean version and the Japanese version I think I've figured out some of the encoding. I still need to figure out the control codes for the text boxes, but the text itself is actually pretty simple. Basically the hex values for the text correspond to the sequence number of the font character in the font table. However the values are the decimal value of the font sequence number + 31, and then turned into Hex. So for example looking at the font charts above, the character A is 11th in the font table. If you want to have A displayed, you need to put the value 2A in the MDT file. This is because 11 + 31 = 42, and 42 in Hex is 2A. I was able to test this theory by replacing the text in the intro scene so in the first two text boxes Baal should now say: And low and behold: As I said, I still need to figure out the control codes, but if someone wants to take a look at it, the file for this area is BA38.MDT in the FIELD directory. The offset for this cutscene and where I started making changes is x5AA. Now the odd thing here is there's over 1000 characters in the font if I'm counting correctly. But this encoding can at most go up to 255 if I'm understanding it correctly. So I'm not sure how it's dealing with characters beyond that limit. However for an English hack we might not even need to care about that as we wouldn't need over 1000 characters to begin with. So honestly, once the control codes are figured out I think this might actually be a pretty straight forward hack. The biggest issue to do first would be to change the game to operate on an 8x16 font instead of a 16x16 font so the English PS1 font can be used. After that it would just be injecting the new font and then editing the script files. This is probably what they did for the Korean translation, but without the font size change, as Korean is still a Fixed Width Font so they could stick with the 16x16 font size easily. So this is what I've found over the weekend. I figured I'd post it here as well since I figured you guys would find this of interest.
|
|
|
Post by TrekkiesUnite118 on Sept 10, 2018 4:05:08 GMT
This is a repost from what I posted on SegaXtreme. But over the weekend I thought I'd snoop around in Grandia's files and do some comparing and contrasting with the US and JP PS1 releases to see how different they are. The thought process was to see if it would be possible to extract the English script out of the PS1 version, and possibly use that as a base to translate the Saturn version. So far this is what I found: - The File Structure between the two games is actually rather similar, unlike that of Lunar Silver Star Story.
- That said there are still some differences. Some files don't exist on the Saturn version and some don't exist on the PS1 version. However this seems to be more aimed at bin files and system specific formatted stuff like audio files, video files, fonts, etc.
- I believe I've found the script for the English version. In both versions there's a Field directory. In this directory there are numerous files with hex values for their file names. In the Saturn data these are .MDT files, in the PS1 version these are .MDP files. I'm pretty sure these contain the games script.
- The English PS1 script is completely uncompressed in the .MDP files. You can bring them up in a Hex Editor and read the ASCII encoded script.
- The font for the Saturn version is in the FDATA.BIN directory as mentioned before by CyberWarriorX on SegaXtreme, and his compression and decompression tools that he wrote appear to work just fine. If you run them on FDATA.BIN you can get the 4BPP font and can load it up in a tile editor of your choice.
- The FONT does have an all Caps English font in it, but from the looks of things I'd bet this one isn't a variable width font.
- I've not been able to really find where the menu text lives. In the PS1 version I've found a couple pieces here and there. For example in the Battle directory there's some pieces in it's DAT files, but these files appear to be one of the few areas where the Saturn and PS1 versions are structured differently. Basically some files the PS1 has don't exist on the Saturn and vice versa.
The Saturn version also recently had a Korean Translation released. So I took al peak at the Korean translated version and this is what I've found: - File sizes appear to be identical to the Japanese version.
- The font has definitely been changed as I'd expect it to be.
- The MDT files have been modified, which would seem to confirm that those contain the script.
- ITGET.BIN has been modified. Looks like they changed character names from Japanese to somewhat English versions. Which looking at the changes and gameplay footage, I'm guessing this is where the menu text lives. This is probably the equivalent of the ITEM.BIN file in the PS1 version.
- CMDMN.BIN has been changed. Again this looks like either Menu text, guessing command related based on what's in it and the name.
- CNFIG.BIN has also been changed. This again looks to be like menu text, guessing Config menu based on the name.
- SHOP.BIN has been changed. This is most likely Shop Menu text by the looks of things.
- SVLD.BIN has been changed. Again looks to be Menu text, I'm guessing this is for the Save/Load screen based on whats in it and the name.
- All the AMAP#.ADD files have been modified. My guess is this is text in the map screens or dungeons?
- WDATA.DAT has been altered, my guess is this is Weapons info maybe?
- SWN6M.DAT has been altered, no idea what this one is.
I then dug into the English PS1 version a bit more. I was able to get the PS1 English font. After looking at it I'm starting to wonder if the English version actually supports VWF. Reason being the English font is 8x16 in size, but every letter is scaled to that size and is perfectly centered. And when watching it in action you can actually see some spots where the spacing looks a bit odd, specifically around thin letters like Lower case L's and I's: And here are the fonts for each version of interest. Original Japanese Saturn: Hacked Korean Saturn Font: English PS1 Font: There were some 8x16 Japanese characters in the English PS1 font as well, but I didn't see a reason to include them. More findings continued in the next post...
|
|
|
Post by TrekkiesUnite118 on Aug 23, 2017 2:58:27 GMT
MsTea here was kind enough to send me the IOS files to play with, so here's the result of a quick and dirty conversion and insertion of the first 3 story audio files:
As you can see, the game no longer crashes or cuts off the clips. So these seem to fit better into the limitations. And a quick glance shows that most if not all of them are smaller than the original Saturn files, so in theory they should all fit in easily.
The only odd thing is there's an awful hiss during silent portions. I don't remember getting that with the PS1 conversions so perhaps I converted something wrong or audacity is having issues.
|
|
|
Post by TrekkiesUnite118 on Aug 12, 2017 5:40:45 GMT
As said on the other thread this is nice to see. I've been meaning to come back to this so this might just be the kick in the butt I needed. If you need any help let me know.
|
|
|
Post by TrekkiesUnite118 on Jan 8, 2016 2:21:08 GMT
Interesting to look at those links you provided. Ive always wondered why Sonic CD wasnt included on Sonic Jam and perhaps it was just programmed alot differently. The answer is much more simple. It wouldn't fit on the disc and would have required a 2nd disc.
|
|
|
Post by TrekkiesUnite118 on Jan 7, 2016 6:21:37 GMT
Another one I had that might be fun for a group project was to see if recreating Sonic CD for the Saturn was a possibility with it. Most of the info for creating an accurate classic Sonic engine is on SonicRetro, so I'd imagine it could be pretty doable. I like that idea a lot. Well if we wanted to jump into doing that, the info on Sonic's physics and what not can be found here: info.sonicretro.org/Sonic_Physics_Guideinfo.sonicretro.org/SCHG:Sonic_CD
|
|