Hacking [REVERSE ENGINEERING] Finding the correct encoding for hiragana text

[Serberker]

Hi all!

I'm kinda stuck with the encoding of a game.
The ingame text is hiragana, but the hex values don't match up.

What is the typical encoding of japan psp games?
It seems the characters are stored on 2 bytes, but UTF16-LE, and shift-jis doesn't return the correct characters.

I doubt that they have put an offset in the encoding or that they used custom encoding.

Any help / hint / idea is greatly appreciated.

 

[FAST6191]

Also I think it's unlikely that they used custom encoding.

Why? The vast majority of console games in history have used custom encodings, and while the PSP was in an era where we started to see more common encodings used by default it was so very far from unheard of for custom encodings to be used.

 

[Serberker]

You won
Found the custom encoding table, and it works flawlessly.
Feeling confident as it has latin characters too.

 

[crimsonwolf8439]

You won
Found the custom encoding table, and it works flawlessly.
Feeling confident as it has latin characters too.

How large was the table?

 

[Serberker]

How large was the table?

2171 characters. (the file itself is 4342bytes+12bytes header)
Why do you ask?

 

[crimsonwolf8439]

2171 characters. (the file itself is 4342bytes+12bytes header)
Why do you ask?

Looking for a reference in my game

 

[Killrain]

Serberker, cool.
How did you find that custom encoding table?

 

[Serberker]

Statistics

The game itself have like 22k files. I modified my extraction script to group by and count the file extensions.
The result was ~16 (or 18 can't recall atm) different extensions.
I checked the files with the lowest count and the encoding table was there, with the extension of .TTL

In my case the files with the highest count were texture and model files.
That was followed by the animation files. This count was lower because static models (like trees, houses) don't have an animation.
And then varius script and other files.

 

[Killrain]

Hmm, interesting.
Thanks!