Requesting Advice on Compressing 'Dense' Files

[Expack2]

I've run into a roadblock while trying to create a pre-installed archive for 3-D Ultra Minigolf to be uploaded onto Abandonia. The problem lies in compressing the game's RDX data files, which contain the minigolf holes and the main menu. I've tried making them submit size-wise using standard ZIP, 7-Zip x64, and WinRAR, but the files refuse to compress well. Here's the best results I've had from trying to compress them:

Uncompressed: 229MB
Standard ZIP: 195MB
WinRAR (Best compression, solid archive): 186MB
7-Zip (256MB dictionary, solid archive): 181MB

Does anyone have any sage advice as to how to make 3-D Ultra Minigolf's RDX files submit? If someone would be willing to share any, I'd really appreciate it!

[dosraider]

Are you asking to get the archive below 150MB?

You could try UHARC, it compresses a lot better then the 'classic' formats, but most n00bs will choke on it ..... there will be a lot of "WTF is this" posts ....:lol:

[Expack2]

Are you asking to get the archive below 150MB?

You could try UHARC, it compresses a lot better then the 'classic' formats, but most n00bs will choke on it ..... there will be a lot of "WTF is this" posts ....:lol:
While I do thank you for the suggestion, dosraider, WinUHA only succeeded to get the RDX files down to 184MB - definitely over the 150MB limit for Abandonia's FTP site. :(

[dosraider]

Ah well, was worth the try.

But I see that 7Zip gets the thing down to a 181MB
7-Zip (256MB dictionary, solid archive): 181MB
Mayhaps they will accept it .... ? :whistling:

[Scatty]

I've run into a roadblock while trying to create a pre-installed archive for 3-D Ultra Minigolf to be uploaded onto Abandonia. The problem lies in compressing the game's RDX data files, which contain the minigolf holes and the main menu. I've tried making them submit size-wise using standard ZIP, 7-Zip x64, and WinRAR, but the files refuse to compress well. Here's the best results I've had from trying to compress them:

Uncompressed: 229MB
Standard ZIP: 195MB
WinRAR (Best compression, solid archive): 186MB
7-Zip (256MB dictionary, solid archive): 181MB

Does anyone have any sage advice as to how to make 3-D Ultra Minigolf's RDX files submit? If someone would be willing to share any, I'd really appreciate it!
If you aren't able to compress files far enough below their original size that usually means that those files are themselves compressed to a degree, using usually own compression algorithm made by the game programmers, and there's no way to compress already compressed files efficiently except using some algorithm that would require 5x the CPU speed of today's computers to work (shortly - not possible). Think of it as if you would be trying to compress a ZIP archive of a high compression level further into a RAR archive, you won't reach much compression there also.
Thus 181MB is probably the highest compression you can reach on those files.

[The Fifth Horseman]

As you are aware, we're using standard ZIP archives (max compression setting) for games hosted on Abandonia.
The only way to cut down the game's size beyond what that compression provides is by removing ("ripping") some of the game assets (typically videos, music and sound FX, in that order).

[Expack2]

As you are aware, we're using standard ZIP archives (max compression setting) for games hosted on Abandonia.
The only way to cut down the game's size beyond what that compression provides is by removing ("ripping") some of the game assets (typically videos, music and sound FX, in that order).
Unfortunately, Horseman, the RDX files that 3-D Ultra Minigolf uses contain all the required information for the minigolf holes and the main menu, so getting rid of any of those, being that they'd be the largest files in the archive, would prevent the game from being fully playable. So, I guess that means that 3-D Ultra Minigolf will have to come to Abandonia as part of the ISO Cellar... :(

EDIT: Dang, where are my manners? Thanks for trying to help me out anyways, Horseman! Oh, and thanks for the explanation about custom compression methods, Scatty!

[The Fifth Horseman]

What size is the smallest RDX you've got there? I'd like to take a look at the file structure.

[Expack2]

What size is the smallest RDX you've got there? I'd like to take a look at the file structure.
The smallest one is the RDX file for the main menu; I've take the liberty of attaching it to this post. If you need a larger RDX file, the next largest is the one for the Rocket minigolf hole, which weighs in at 4.63MB.

Note that while the RDX files are small on their own, there's twenty of them in the DATA folder created by the "Large Install" option for 3-D Ultra Minigolf. All twenty files bring the total size of the DATA folder up to 229MB!

[The Fifth Horseman]

Oh man. That's not RDX... that's RBX. Aka "almost exactly the same format I've hacked before". One or two more of those files would be good, so that I can work out the differences between this application and the one in Cyberstorm 2 (another Sierra game released around the same time period).

Then it's time for an extractor, figuring out how to dummy out the WAX files and writing a program to pack the stuff back into an RBX. If they are anywhere near close to the WAX format I've dealt with before, they consist of a standarised header followed by a stream of audio data - either uncompressed or encoded using the IMA ADPCM algorithm.
That's probably the part responsible for the huge archive size, BTW.

EDIT: Nevermind, got the minimal format differences cracked now. The problem was in my own assinine old code. The WAX audio appears to be very close (though not identical) to CS2's.
I'll catch some sleep and then look into dummying out the files. FYI, the WAX files take up 3/4rd of that RBX's size.

[Expack2]

Oh man. That's not RDX... that's RBX. Aka "almost exactly the same format I've hacked before". One or two more of those files would be good, so that I can work out the differences between this application and the one in Cyberstorm 2 (another Sierra game released around the same time period).

Then it's time for an extractor, figuring out how to dummy out the WAX files and writing a program to pack the stuff back into an RBX. If they are anywhere near close to the WAX format I've dealt with before, they consist of a standarised header followed by a stream of audio data - either uncompressed or encoded using the IMA ADPCM algorithm.
That's probably the part responsible for the huge archive size, BTW.
Sorry this took so long, Horseman, but here's the link (http://dl.dropbox.com/u/13771683/MinigolfRBX.zip) to a zip file I prepared that contains all the RBX files for 3-D Ultra Minigolf's holes.

[The Fifth Horseman]

See if the game works after replacing the original main.rbx with the attached one.

[The Fifth Horseman]

If the RBX in the post above worked fine, then unpack the files attached in this archive to where the RBX'es are and run launcher.exe.
It will unpack the contents of the RBX'es, truncate the sound files and then repack everything back into new, valid RBX'es.

Mind you, this was written specifically for the versions of RBX and WAX formats used by your particular game. It won't work for other games unless they use the same exact implementation of both.

[Expack2]

See if the game works after replacing the original main.rbx with the attached one.
Unfortunately, I just tested the file and the game immediately crashed. (Guess it's back to the drawing board...) Sorry it took so long to test the replacement file...I've been really busy today.

[The Fifth Horseman]

Unfortunately, I just tested the file and the game immediately crashed. (Guess it's back to the drawing board...) I was hoping that wouldn't happen. Oh, well. :)
Take two attached - rather than truncating the sound files, we'll pad them with zeroes instead.

[Expack2]

I was hoping that wouldn't happen. Oh, well. :)
Take two attached - rather than truncating the sound files, we'll pad them with zeroes instead.
Oddly enough, Horseman, even though your new RBX recompiler creates playable RBX files, only the main menu's RBX compresses well. The main menu compresses to 15% of its original size; the other RBX files only compress to around 80-85% of their original size. Arrgh, so close yet so far! :mhh:

[The Fifth Horseman]

Obviously, there must be some other data boosting the size.

Put this where the RBX files are, run it and post the txt file it generates.

[Expack2]

Obviously, there must be some other data boosting the size.

Put this where the RBX files are, run it and post the txt file it generates.
As requested, I've attached the TXT file that your RBX scanner provided when in the 3-D Ultra Minigolf DATA folder.

[The Fifth Horseman]

*^_^ Seems I missed a small piece of default behavior in the scanner. As a result, it reported only the contents of a single RBX rather than all of them. *^_^
Fixed version attached.

[Expack2]

*^_^ Seems I missed a small piece of default behavior in the scanner. As a result, it reported only the contents of a single RBX rather than all of them. *^_^
Fixed version attached.Here's the TXT file generated from your fixed RBX scanner, Horseman!