How to read an obscure .data file

[MechPilot524]

I'm looking to read the contents of a .data file which is part of a game which came out a long time ago, going into beta circa 2004. It's called NavyField: Fleet Mission.

 

It would be too easy if the file was in plaintext or if it was properly Romanized, but of course... it isn't. Maybe NP++ isn't using the right shiftkey, or maybe it has a rudimentary level of data encryption; I don't know. Copying and pasting the Asian characters hasn't yielded anything that makes sense; in Chinese or Japanese it looks like random characters giving me random words, like "Moth" or "Feeling" in a file that based on its name should be talking about the naval guns in the game.

 

I've had a little exposure to bitstream analysis and I have fledgling knowledge of programming and coding, but I don't know the approach for something like this. I also don't know of any tools I can use to derive things like frame width, or how to see which bits/bytes are checksums, if any bytes are going to be markers. Lastly, I'm clueless as to how programming languages change if they're made by a foreign country - one whose primary language is very different from English.

 

I've uploaded the "Hong Kong" and "China" versions of the "same file" along with the North America file. Interestingly, they do not contain the Chinese/Japanese characters, but I've had just as much success reading them.

 

How could I go about analyzing, processing, or reverse engineering the files?

Gunset_EN-US.Data

Gunset_ZH-CN.Data

Gunset_ZH-HK.Data

[Mira Yurizaki]

Here's the first thing you have to answer: what does this data contain? If we don't know what this file is supposed to contain, we're looking at a dozens of different things we can try to figure out, assuming the data isn't compressed or encrypted.

[MechPilot524]

It is believed this file contains attributes for the game's various naval guns.

 

For example, cannon X has range of Y, with an accuracy rating of 22. It probably also specifies requirements to equip, such as the cannon requires a slot of 100 space, weighs 44 tons, and requires a level 45 gunner to operate.

 

I do not expect it to encapsulate other files such as images or sounds, as those game assets are already cataloged and known. It's probably not compressed, as the game's sprites and audio are not compressed. I wouldn't know if it's encrypted.

[mariushm]

The files could be "encrypted" with some key. For example let's say you have a 32 byte key. Take one byte from original file and xor with first byte from the key, save result,  loop for 32 bytes, then go back to first byte in key and so on.

 

Doesn't seem to be compressed with some common compression algorithm like zlib (deflate), 7zip etc

 

Could be some ancient compression algorithm like LHA for example, lots of Asian programmers wrote custom versions of such algorithms back then.

 

For funsies, I checked if it's a little endian vs big endian issue (assuming data is saved as UTF-16, two bytes per character, I imported the file into Adobe Audition as raw audio mono little endian and exported to raw mono big endian, basically same file just two bytes at a time flipped - no luck. Funny enough TRID thinks it found a 100% match on the new file and that it's a  PrintFox/Pagefox bitmap (var. P) (1000/1)  ... but most likely it's a false positive.

 

If the files had some kinds of records with a fixed structure and there's no compression or encryption, you should see a lot of same byte values while comparing English vs HK files for example (numbers would be the same in both files, and only text strings would be different).  The fact that both files have same file size (140428 bytes) makes me think there's no compression involved (because some Asian texts would use more bytes compared to English text if using UTF-8 or UTF-16, more code points, and Asian text would probably be less compressible).

So, I'm leaning towards using a sort of encryption... maybe some kind of bit flips/byte swaps or substitution cyphers

 

ps. Maybe try opening the game executable and search for string texts, maybe there's some text that looks like a password, then try something like an online xor decrypt tool : https://www.browserling.com/tools/xor-decrypt

 

[JacobFW]

One thing to try first is to try opening it with common programs.  7zip, Excel, VLC.  Likely won't be anything, but it won't take much time and it will at least rule those out.

 

Next thing I'd to do is search the game's name on google along with "modding" or "porting".  Pretty much anything that would involve digging into the specific files of this game.  In case you don't know, google also has a feature where you can limit the search results to webpages that existed during a certain timeframe.  It's not perfect but I've found it helpful in the past.

 

Second would be to do some research on the game, go back and find any articles about the game around the time it came about, maybe us archive.org to find any webpages that have been taken down, to find out about what engine it used, who the developers were, what other games they worked, then go to a forum dedicated to game modding, and ask if anyone has any experience with similar games/made by the same people.  Remember, most people are lazy, so it's highly unlikely they would invent a brand new way to save data if an existing library could already do it for them (it would also be stupid if was a better solution than anything they could quickly develop).  Generally the most people will do might be to mod an existing solution.

 

If however you're looking for something else you can do right now without waiting for someone else, then two other things I'd suggest would be to change the encoding on Notepad++ just in case they're using something else.  The other thing would be to download a hex editor, and view the file through there.  I've found that helpful for inspecting database files in the past.

 

Good luck!

 

 

 

[MechPilot524]

Just now, JacobFW said:

One thing to try first is to try opening it with common programs.  7zip, Excel, VLC.  Likely won't be anything, but it won't take much time and it will at least rule those out.

 

Next thing I'd to do is search the game's name on google along with "modding" or "porting".  Pretty much anything that would involve digging into the specific files of this game.  In case you don't know, google also has a feature where you can limit the search results to webpages that existed during a certain timeframe.  It's not perfect but I've found it helpful in the past.

 

Second would be to do some research on the game, go back and find any articles about the game around the time it came about, maybe us archive.org to find any webpages that have been taken down, to find out about what engine it used, who the developers were, what other games they worked, then go to a forum dedicated to game modding, and ask if anyone has any experience with similar games/made by the same people.  Remember, most people are lazy, so it's highly unlikely they would invent a brand new way to save data if an existing library could already do it for them (it would also be stupid if was a better solution than anything they could quickly develop).  Generally the most people will do might be to mod an existing solution.

 

If however you're looking for something else you can do right now without waiting for someone else, then two other things I'd suggest would be to change the encoding on Notepad++ just in case they're using something else.  The other thing would be to download a hex editor, and view the file through there.  I've found that helpful for inspecting database files in the past.

 

Good luck!

 

 

 

Oh yeah. I've been a fan of this game for years. What I do know is that this is a proprietary in-house game engine coded by one author who left the small developer a decade ago, and with him left their ability to do much with the code. It's been years and years of band-aids since the author parted ways.

It's also been so long that any trails into the game's actual source code are cold. I was in a modding community and we have a tool to open the game's graphical Sprite files and audio assets, but we never had anything to look at gameplay mechanics, partly because we couldn't mess with them. Maybe it's because they've been encrypted all these years. On many files N++ and HxD can make some sense of them, but these .data files make no sense.

 

I don't really understand what you mean by N++ encoding - you mean DOS vs Windows ANSI?

 

Marius' assessment makes sense, I'll see if I can break the encryption. It can't be anything too crazy, right?

[JacobFW]

 

Regarding the encryption, just as an fyi your game came out a few years after the AES Contest, so I imagine many companies were clamoring to say they were using the latest and greatest military grade encryption.

 

I am highly, highly skeptical about it being encrypted though. It might be an SQLite file?

 

Are you able to see any plaintext at all?  If I might suggest, make a copy of it, open it in Notepad++, and do a replace in regex mode

 

Use this as your search string

[^\0 -~]+

and replace it with nothing.  This should remove any characters that aren't ascii from the file, and let you see what remains.

 

 

[MechPilot524]

 

2 hours ago, JacobFW said:

 

I am highly, highly skeptical about it being encrypted though. It might be an SQLite file?

 

Are you able to see any plaintext at all?  If I might suggest, make a copy of it, open it in Notepad++, and do a replace in regex mode

 

So I've got 50,300 normal characters in one line consisting with the occasional null character. It seems pretty scrambled; from the seemingly random brackets and parentheses, it probably isn't code?

 

On 6/24/2019 at 1:11 PM, mariushm said:

ps. Maybe try opening the game executable and search for string texts, maybe there's some text that looks like a password, then try something like an online xor decrypt tool : https://www.browserling.com/tools/xor-decrypt

 

I'm scrolling through the file and I've found a few spots where it lists the entire alphabet.

 

Should I expect a password to be an exact number of bytes or characters, or for it to be defined in a certain way? I've got maybe 2,200 lines here to sift through, and I'm not sure what the environment around such a password might look like.

 

 

In the main executable, I did a little sifting and it may use zlib (it has that inflate 1.1.3 Copyright 1995-98 Mark Adler, as well as unzip 0.15  Copyright 1998 Gilles Vollant). Both of which seem to be for open-source decompression, though zlib also seems to deal in decryption.

 

A couple hundred lines later, I also see the mark "Microsoft Base Cryptographic Provider v1.0".

[mariushm]

zlib/unzip may also be used to decompress PNG files. If present, encryption is needed to unpack password protected zip files. Or may be used in other parts of game. Presence of text doesn't guarantee its use on those files.

 

Can you actually say the name of the game, link to game info etc?

 

If I find the time I may look at it out of curiosity.

[MechPilot524]

NavyField: Resurrection of the Steel Fleet.

It's now known as NavyField: Fleet Mission after it was acquired by a Hong Kong company, KupaiSky, circa 2017.

The game's website is here. It's a very quiet game now, but used to be somewhat popular during its heyday. Some fans in Europe saved a version of the game previously and run a fan server, but it's extremely small.

 

But are there any clues as to where the password in the main executable might be?

[straight_stewie]

It may not actually be encrypted. It could just be POD. Try opening the file(s) in question in FlexHex and see if they make any sense.

 

Disregard that. I didn't notice your most recent replies.