Monkey Island Special Edition - File Formats

**bgbennyboy** · 07-15-2009, 01:20 PM

I'll be posting here about the file formats in MI:SE.

So far:

The sounds and music are in xbox wavebanks so should be easy to extract.

The rest of the resources are in Monkey1.pak - which seems to be unencrypted so far (I can see an offset table and name directory anyway)

**bgbennyboy** · 07-15-2009, 01:43 PM

Images seem to be DXT textures, unfortunately they arent embedded in dds files for easy viewing

Silvor · 07-15-2009, 01:47 PM

Hopefully getting voices on the old art won't be too hard.
Here's hoping.

Jeroen_JRP · 07-15-2009, 02:05 PM

Do you think we could get it to play without steam?

**bgbennyboy** · 07-15-2009, 02:11 PM

Quote:

Originally Posted by Jeroen_JRP

Do you think we could get it to play without steam?

No. I'm not interested in bypassing DRM just looking into the resource files. If you want it without steam LEC are selling it from other online providers.

Jeroen_JRP · 07-15-2009, 02:15 PM

Yeah, I'm waiting for that, untill now only Direct2Drive in US and Canada

**bgbennyboy** · 07-15-2009, 02:25 PM

Update - Serge has put together a quick dumper for the Monkey1.pak. Meanwhile I'm knocking up a quick dumper for the wavebank music.

daltysmilth · 07-15-2009, 02:48 PM

Cool.

**bgbennyboy** · 07-15-2009, 06:15 PM

I'm having problems with the audio atm, some of the audio dumps correctly as pcm wav's, however some appears to be WMA from the format tag but doesnt play back. Either they arent WMA and my understanding of the format tag is incorrect or there's some sort of added obfuscation on the wma files

Serge · 07-15-2009, 10:30 PM

Finished the bulk of "Remonkeyed Explorer" which opens the main resource file and allows you to save all the files inside. Should be out later this week (probably Friday).

Examples of contents inside the .pak file:
The original monkey1.001 and monkey1.000 files (unchanged) from the Monkey Madness CD-ROM version
Backgrounds
Costumes (i.e. character art)
Room definitions (in some binary encoded XML - not looking into that before Friday)
Fonts (stored as PNG)
Shaders (.fx files)
tweaks.txt (containing some settings for strange things in the engine)
hints.csv (binary file with all the hints)
etc.

No music or speech - that's in the separate xwb files (which BG is working on).

I've managed to "decode" the backgrounds and costumes to .dds (really just a matter of adding the DDS header with width/height etc.) - probably adding decoding from DDS to PNG format before Friday, although I'm not sure that I'll make it combine the various tiles that make up a single background in that short amount of time.

Various examples and screenshots (please don't hotlink - my bandwidth can't take it):

http://www.jither.net/media/images/remonkeyed.png (screenshot)
http://www.jither.net/media/images/woohoo.png (first tile of background art I got decoded)
http://www.jither.net/media/images/ghostpig.png (a random costume decoded)

Oh yes, it's "Remonkeyed Explorer", because apparently an internal name for the Special Edition is "Remonkeyed Game".

cocomonk22 · 07-16-2009, 04:46 AM

If the original 001 and 000 files are unchanged, that explains why there are no voices in classic mode and that the SE was just made to be in sync with the original game.

Serge · 07-16-2009, 05:17 AM

Nah, I think (as I did before the game came out) that it's always the original game running, and the new art is "projected onto it".

Silvor · 07-16-2009, 05:24 AM

I find it annoying that the old art version runs in completely full screen mode, compared to the old dos version which had some black bars around it. Just makes it look ultra blocky. Wonder why that is.

And there seems to be some music bugs in the old art version as well, like voodoo house music keeps playing after you leave the voodoo house sometimes. So they must have done something to the old game files I think.

Serge · 07-16-2009, 05:30 AM

Nothing was done to the old game files (they're binary identical), but since the SCUMM engine running it has had to be updated, there are probably bugs inthere.

Silvor · 07-16-2009, 05:31 AM

Ahh, yeah that would explain it.

Serge · 07-16-2009, 05:43 AM

Might add that the "projected art" "theory" (I'm quite sure it's accurate) means that there are hacks done in various spots to expand the original game.

I.e., the addition of Spiffy's close-up, the different readings of insult lines, depending on whether they're decided right by the script or not, etc.

Laserschwert · 07-16-2009, 05:49 AM

If it's possible to extract graphics from the game, would it be possible to edit those files, and put them back in the game?

Serge · 07-16-2009, 05:57 AM

Most likely, yes. As long as they have the correct format, and correct layout, it'd be as simple as replacing the files. Since the (DXT) graphics are deterministic in size (always compressed 1:4), if they keep the same layout, they keep the same file size and the file offsets in the rest of the file don't change, so there shouldn't even be a problem if there's hacky hardcoded direct offsets in the .exe (which is unlikely these days anyway).

Not something I'm looking into for a while, but I'll probably release some specs when I get back from vacation in a few weeks (if BG or someone else hasn't already).

It's even likely, as in GrimE, that just placing the new art in the proper subfolder, with the proper name, in the correct format, might just mean it's loaded into the game. Since that's easier (and very unlikely not to be implemented) while developing. If it's not turned off on release.

Serge · 07-16-2009, 06:07 AM

Just for fun, by the way, here's most of Guybrush's costume:

http://www.jither.net/media/images/guybrush.png
http://www.jither.net/media/images/guybrush2.png
http://www.jither.net/media/images/guyho.png

Laserschwert · 07-16-2009, 07:24 AM

Mh, do I smell "hair patch"?

ThunderPeel2001 · 07-16-2009, 08:34 AM

I hope so

It looks like the 2.5.gb PC download WAS for all the hires artwork... The game must to able to run at some insane resolutions!

Serge · 07-16-2009, 08:48 AM

Well, the .PAK is around 1.2GB and 99.999% of that space is taken up by images - mostly split up into 1024x1024 texture tiles (like the ones posted above). Such a tile is a bit over 1MB, if compressed - saw a few 4MB tiles which may be uncompressed RGB.

The remaining data is mostly 500 byte-65KB description files, shader scripts, some PNG textures and the two original resource files from Monkey Madness.

parabolee · 07-16-2009, 10:11 AM

Look like we are in for some interesting patches

Although I wouldn't want to change Guybrush myself. Love him.

Cool to fix the Scumm Bar sign (smiley face) and the crappy text on Stan's sign though.

Kev63 · 07-16-2009, 11:29 AM

Hi everyone !

Serge, do you intend to release your tool to unpack Monkey1.pak ? It would be useful for foreigners like me, as I'm french I could put the Monkey1.000 and Monkey1.001 files from the original french version. Thanks

...

jott · 07-16-2009, 11:42 AM

Hello,

I just discovered this thread. Just wanted to throw in, that I also wrote a little tool to dump the Monkey1.pak file and write out DDS textures.

Serge already started a proper GUI which would ultimately allow easier modding I suppose.

Anyway, here is my (hackish) source in plain C:
http://slexy.org/view/s21wNL2X3x

Sample image (copyright LucasArts):
http://i36.tinypic.com/b8wby9.jpg

@Serge, two questions though:
Do you plan to release the source code of your tool?
And do you know what the table at 0x28 (referred at 0x8) is for?

I also haven't looked at the binary xml yet...

jott · 07-16-2009, 12:23 PM

Just a note:
The game seems to run fine without the Monkey1.pak when all files are extracted in the same directory (such as my little tool does).

Silvor · 07-16-2009, 12:26 PM

Very interesting.
I wonder if that will help to get voices when you switch to the old art in some way.

Laserschwert · 07-16-2009, 12:26 PM

Which means changing files without repacking, yay!

**bgbennyboy** · 07-16-2009, 12:28 PM

Thanks for the dumper and source jott. Lets all try not to duplicate each other's work here. Lets try and post what we are working on/discover so we don't overlap.

[Edit] If it runs without the pak thats excellent news! Dont even have to write a pak compiler.

Serge · 07-16-2009, 12:38 PM

Quote:

Originally Posted by Kev63

Hi everyone !

Serge, do you intend to release your tool to unpack Monkey1.pak ? It would be useful for foreigners like me, as I'm french I could put the Monkey1.000 and Monkey1.001 files from the original french version. Thanks

...

I do - as mentioned above, I planned on Friday - not sure at the moment, since I have a lot of work-work and leaving on vacation tomorrow. But maybe. In any event, Jott already has something out.

Quote:

@Serge, two questions though:
Do you plan to release the source code of your tool?
And do you know what the table at 0x28 (referred at 0x8) is for?

Source code: Not at the moment, simply because it shares a lot of code with a longer term "secret" project

, which will probably be open source. Shares as in "copy/paste and quick hacks" which means it's not even close to clean and won't be maintained - since the original project already decodes PAK also. Only did this tool to get something out quick.

As for 0x28 - no idea yet. Did all this rather quickly last night, so when it worked, I actually stopped looking at the package format and focused on the DXT stuff. Fast enough to not find any file count in the format either, which means at the moment, it simply reads the offset table until it reaches the known position of the name table.

And yes, Remonkeyed Explorer also decodes all files to the same folders if wanted (that's what the "Use original subfolder" checkbox is there for). Great news that unpackaged resource use wasn't disabled for the release

**bgbennyboy** · 07-16-2009, 12:45 PM

Continuing the division of labour - I'll work on a quick 'texture replacer' program, for the moaners who want to do a guybrush "hair patch".

I'd also appreciate any input from anyone on the audio for the new music inside the soundbanks. The format tag identifies it as WMA (from what I know about the format tag) but It doesnt look like WMA. Either its not WMA or its encrypted/obfuscated somehow.

Silvor · 07-16-2009, 12:53 PM

About that "hair patch" thing.
I wonder if the people who want to fix this knows how much work they'll actually have to do.
Guybrush does have quite a few elaborate animations he goes trough during the course of the game.
Not gonna be a small job to animate the new hair on all of them.

jott · 07-16-2009, 12:54 PM

Speaking of duplicate work, bgbennyboy...

Here is a little tool to convert DDS back to DXT:
http://slexy.org/view/s2laFdoAjF

So here is a shot of a modded Monkey Island SE:
http://i33.tinypic.com/2v2a4jm.jpg

All seems to work out nicely... though some manual steps are needed ATM.

**bgbennyboy** · 07-16-2009, 01:01 PM

Nice one Jott

I'll be doing a GUI tool, like I usually do. If you are working on a similar app please let me know and I'll stop and concentrate on something else

Laserschwert · 07-16-2009, 01:03 PM

The best thing would of course be if the game uses the files from the PAK file, with the exception of files that are also contained separately in the game folder. But I guess the PAK is the game's first choice, so separate files would be ignored.

**elTee** · 07-16-2009, 01:04 PM

Incredible work guys, hopefully these 'exploits' won't be patched on Steam.

Tomas · 07-16-2009, 01:08 PM

I had a very brief look at the audio files. I used this program to extract the files from the wavebanks. The .WAV-files seem to play without doing anything to them. The .WMA-files do not as bg says. Opened them in a hex editor and concluded that most of the start with the hex values 09 00 00 03 (I was looking at the MusicNew.xwb file). Also opened up some .WMA-files i had lying on my HD. Many of them has this byte pattern (09 00 00 03) at what seems to be the end of the header or the beginning of the data. I don't know exactly how this program (unxwb) extracts the files but maybe the .WMA-files are stored without the header and it needs to be reconstructed somehow? I don't know, maybe this could give some ideas. I don't have time to look into at it at the moment.

jott · 07-16-2009, 01:09 PM

@Laserschwert:
The PAK file is indeed the fallback! So just putting files in the right directory will replace them in-game.

@bgbennyboy:
No, I'm not working on any GUI. Maybe I'll try to find out something about the XML later.

Laserschwert · 07-16-2009, 01:12 PM

Quote:

Originally Posted by jott

@Laserschwert:
The PAK file is indeed the fallback! So just putting files in the right directory will replace them in-game.

So I have to put only the files I want changed in the game's folder, and it uses them instead?? That'd be awesome!

**bgbennyboy** · 07-16-2009, 01:12 PM

@Tomas - good to see you again! Everyones coming out of the woodwork recently

The idea about the WMA just missing a header sounds reasonable, I'll try and find time to look at it again this weekend.