A.D.A. Amiga Demoscene Archive

Hi Guys

Wondered if anyone had any source for doing perform Delta optimisation of sample data for the amiga? Its that thinkg you do to 8bit sample data which doesnt affect the quality but helps with compression?

It's easy. You just transcode absolute values into relative values before saving, and inverse that after loading. The first value stays absolute.
So this sequence:

will be delta-encoded as:

and after inversing it will turn back into:

Just thinking of it, but after the delta encoding (and if it's for a better compression ratio) you can try to do a Burrows-Wheeler transform (see: http://en.wikipedia.org/wiki/Burrows-Wheeler_trans form).

For selecting the marker byte, several choice is possible:

* suppose the delta encoding will not generate any 0x7f (or 0x80) value (not the best choice ;-) )
* parse the delta encoding and find a value not used and write it for example just before the first value (do the BW-transform on all values except the first)...

Or if you don't care about a little loss, transform any delta 0x7f (or 0x80) to the nearest one, and use the value you have just freed as a marker... (Or find the less used value, and transform it to the nearest one...)

The delta operation (1st order derivative) is just the first step in delta compression. After you have differentiated the sequence, you should apply some form of lossless compression to it. Simple RLE will give very good results for simple waveforms.

Sometimes you can compress the sequence simply decreasing the number of bits per sample after differentiation. I recently stumbled upon an article about compressing and decompressing a sinusoid this way (really a classic problem in amiga democoding). The author demonstrated that the 3rd order derivative of a sine wave (24 bits, 1024 samples) can be stored in only 3-bits per sample and the unpacking code is really simple. Link: http://yehar.com/blog/?p=1220

i personally just used crunchmania.

Indeed if you are packing your data using a general purpose compressor anyway, the best result is usually obtained by just doing the delta transformation and storing the resulting values in individual bytes (even though they might be representable using fewer bits). The compressor will take care of the rest.

As for Crunchmania - last time I looked (that's many years ago, admittedly), the decrunch header did not flush the caches after decompression, which means it doesn't work reliably on anything other than a 68000. Has that flaw been fixed?

Of course, my personal recommendation would be to forget everything about Crunchmania and just use my cruncher. ;)

I think the Crunchmania decrunch-header problem has never been fixed, so indeed, using Crunchmania to crunch an executable directly cannot be recommended*!

What I meant to say in respect to the original question of packing sample data was that I personally just used "Crunchmania Data" option , i.e. I embedded the Crunchmania decrunch code in my program, decrunched the compressed data at runtime, and flushed the cache afterwards to be safe. I have used that method without problems since 1997.

Admittedly a lot of time has passed since then and I am sure that Blueberry's cruncher is even more efficient. So basically: prepare your data for compression, then use the best available cruncher.
---
* From my memory that Crunchmania header only ever caused problems on with copyback cache, which are only found on 68040 and 68060. ?

Yes, without copyback cache, the problem only arises if the memory area written to happens to be already loaded into the instruction cache. For dynamically self-modifying code, that will often be the case, but for typical executable decrunching, it is unlikely.

As for the original topic of this thread, here's some code:

As for the crunchmania cache issue; I usually use FixCrm to patch crm-crunched executables for 040/060 (adds 12bytes or so)

Nice one, cheers for the ex-sample, pardon the pun.

My main problem now is the music disk i'm working one has 50 - 100 samples in total, accross 11 modules. Does anyone know how I could wrap this little routine up into CLI exe?

Actually, scrub that. I just worked out how to save a file to the hard drive so I'm going to write a routine which deltas all the samples and saves them for me.