okay... as far as i recall the TIFF image format is an open standard (could be wrong - i've not researched the matter).

i was rather chagrined today to find that when i tried to save my 16 bit tiffs that resulted from decoding some RAW camera files i had, LZW results in LARGER files than i started with when working with 16 bits. zip isn't good with this either, but rar is handier.


here's where my idea comes. i decided to save 1 channel of a 16 bit tiff to a headerless data file and open it up in cooledit. after attaining a decent looking file (i had to unfortunately keep colours under 128 to avoid offsetting fullscale to minus fullscale in cooledit, effectively giving me 15 bits instead of 16) i decided to save to FLAC at full compression.

i got a smaller file than RAR, and a very good ratio over my original file.

original tiff = 7572kB
lzw tiff = 7961kB (WTF?)
original in rar at maximum = 4391kB
FLAC compressed raw data (compression set to 8) = 4035kB

the speed was comparable (faster?) to the rar file, and the size was smaller.

so would it be possible for FLAC to be used in 16 bit TIFF compression? i don't know enough about the format, but this seems intriguing.

You just discovered that lossy (1 channel, 16->15 bits) compression works better than lossless, I think.

Try PNG and a PNG optimizer, it will probably work better than TIFF/LZW.

well... not really that, i was testing a hunch, and unfortunately cooledit doesn't open 16 bit headerless in an efficient way. it places 0 on the zero DC line (duh) and when luminance goes above 127 (in 8-bit speak) it ends up at -32767, causing shall we say sub-optimal conditions for lossless compression.

besides, i compressed the exact same image for all the other techniques listed to make sure the reduction in data from my reference image didn't screw things up.

if there were a tool in cooledit to offset positive samples by -1, then invert and offset negative samples by +1, i wouldn't have to resort to limiting the luminance range... it was a testing hack.

yes i hadn't tried PNG, as i'm working within photoshop (that doesn't support saving to 16-bit PNG, last time i checked).

i just read the TIFF specification (google), and it seems that invariably the LZW algo works in 8 bits no matter what the image depth is. in my case it's splitting 1 pixel into 2 8-bit "characters" and then compressing. this appears to result in disaster. (i'll leave the explanation to somebody who knows more aboot lossless compression;))

i hadn't thought about PNG... if i can find a relatively hassle-free way of getting my 16 bit pics into it, i'll give it a try on the same image and post results. of course, FLAC could be built into PNG as well.

[edit]

this thread is probably semi-off topic for an audio forum, but it may interest somebody. my results are repeatable of course.

[edit edit]

oh, man, thanks for the tipoff for PNG files... after a search i found www.fnordware.com who offer 16 bit PNG and 16 bit JPEG2000 I/O plugs for photoshop.

but after compressing the exact same file with these two lossless formats, FLAC still wins! strange, considering it was developed for audio.

lossless jp2 = 4303kB
PNG = 4403kB

Is it

a) reproducable for different files

b) actually possible to reverse the process? (And did you test if it actually works?)

PS. You can 'optimize' PNG's compression, which can often buy quite a bit extra. Look for PNGCrush for example.

hehe. scientific rigour...

a) haven't tried yet, but i see no reason for it not to be

b) see a)

i've heard of PNG crush, but haven't used it as image compression isn't overtly important to me. the only reason i attempted the above was because i was attempting to send 16-bit tiffs to a friend who needed them, and was horrified to find the (full colour and bit depth...) files came out bigger when i used photoshop's tiff implementation. of course rar helped a lot here, but it sparked the thought that most file compression isn't optimised for 16 bit data rather than 8 bit characters. naturally i realised that lossless audio compressors for the most part ARE optimised for 16 bit depths.

now of course i only tested out 1 file, and was quite excited to find the result to be encouraging. no ironclad conclusions can be made on the basis of 1 file, but i thought it might be interesting to developers of lossless codecs that their work had other potential applications.

if you like i'll do a scientific (at least, MORE scientific... i'm a multimedia guy after all and not a scientist) test with several files that i'll make available for download. as the files i'm working with are rather large, i'll only do this if there's enough interest in this (and i'll probably use smaller pixel dimensions, but this will no doubt skew the result... resampling reduces noise, which will make lossless coding more efficient, etc).

as far as recovering the original data from cooledit... well i'll leave that in the "theoretically possible" basket. i'm not sure i can actually get bit-identical results, but most probably because of the compromises one must make to get cooledit and photoshop to play fair.

[edit]

looks like the files are identical when decompressed. unfortunately i had to compare by loading the files in photoshop, saving as raw files (identical settings of course), then loading the resulting raw files in cooledit and wave-subtracting.

i've forgotten how to bit compare in windows command-prompt... i'd appreciate if someone could tell me

just tried with another image... this one was much smaller pixel dimensions (640x480 vs 2272x1704) and consequently smoother. it was slightly overexposed (but not clipped) leading to slightly higher similarity in the resulting "wave" i suppose.

again, it compressed better with FLAC than the other methods i tried. there's definitely something in this, if it could be made to work within an image format.

i found PNGcrush doesn't appear to make much difference with these images (using -brute i only was able to shed 94kb from a 4.5MB file).


stats for this new file (it's a picture of a friend, so i wont upload it to protect privacy... i know it sounds lame).


original = 609kB
(16bit, greyscale, treated as above, with levels not exceeding half of fullscale)

png = 453kB
(from the fnord "superpng" photoshop plug, set to max)

rar = 418kB
(compression on max)

lossless jp2 = 394kB
(again, fnord plugin)

flac = 374kB
(compression on 8)


[edit]

i forgot to mention above explicitly, but...

a) looks like it is reproducible with at least 2 files

b) it appears to be reversible, if you trust 16 bit wave subtraction yielding perfect zero for each and every sample

hope i've covered all my bases... i understand skepticism of course... i'm not the sort to deliberately embarrass myself by posting fallacies

i hope somebody finds this information useful

Interesting discussion. If you want to try an actual file compare, without using cooledit's wave sub, in windows the command is fc. "help fc" will get you all the switches, but basically fc /b file1 file2 will do a bit binary comparison.

FC! that was it... i only used it once so i forgot what the command was

Have you also tried JPEG2000 lossless mode? Depending on the image data, you may see better results than with PNG, even after optimization.

As for PNG optimization, here is my insane combo:

1) Remove unwanted metadata with TweakPNG.
2) Squeeze it down with OptiPNG, either with -o6, or with -o7 if you have a lot of time, or if the image is rather small.
3) Attempt to squeeze it further with advancecomp's advpng, using maximum settings. (-z -4)
4) If advpng failed to reduce it, repeat with advdef, with the same settings.

Optionally, use TweakPNG again to add metadata, such as author name. Chances are, you won't want to use advpng if you don't want to lose any of the metadata you saw in the first step, as it deletes everything that doesn't affect the image.

I had a similar idea a while ago:
http://www.hydrogenaudio.org/forums/index....topic=14103&hl=

It's all a bit pointless, really, but was fun to try nonetheless.

For lossless compression of photo data JPEG2000 lossless is the right thing and for graphics PNG is perfect.

If you want to avoid wrap-around problems, just substract 128 from each pixel value. Then you'll use the correct -128..127 audio range instead of 0..255.

Edit: As cabbagerat said, you better try JPEG2000 on your images. On photographic pictures it can compress much better than PNG in lossless mode.

MugFunky tried JEPG2000 in his second post . . .

yeah... it performs the 2nd best compared to flac which i really find amazing... i mean how much development went into jpeg2000 lossless, and along comes an audio codec of all things and beats it (by only a few k).

i can't subtract 128 from an image, as there's nothing darker than black (except in HDRshop, but that's 32bit not 16)... and i can't wrap in cooledit AFAIK, or i just end up clipping.

yes, it's not at all practical, but if flac were adapted for images it could make quite a large splash. especially if image specific issues were sorted... i could see it delivering stunning compression ratios from a lot of images.

Well, you have values between 0..255, which makes -128..127 after substraction. You just have to use tools that won't clip negative values up to zero.

heh. it was actually kind of simple for me to load a tiff straight into the CLI of flac, by specifying raw samples, 16 bit unsigned.

tested some colour images... didn't perform as well as JPEG2000 in the few cases i tried (no matter what my settings, and i tried a LOT). i wish channel coupling worked for 3 channels

i also tried different channel interleaving with photoshop (interleaved RGBRGBRGB or straight one-after-the-other RRRGGGBBB). looks like interleaved worked best, but still not as good as jpeg2000. pity, as i see some potential tweakage possibilities here.

it was actually real easy to pull the original TIFF file out of the flac as well... just decode to raw again. bit-identical.

if you have 16 bits red, 16 bits green, 16 bits blue, the best you'll probably get with flac is to interleave rgb for each pixel and use

--channels=3 --bps=16

(and match the endian-ness and signed-ness of course).

also set the blocksize to be the # of pixels per line (or 2 lines worth, or 3... experiment). this will probably only help if you can get the color data to align with FLAC frame boundaries; don't know enough about TIFF to know if this is possible.

Josh

edit: actually, alignment is even more important... say the TIFF was one byte header followed by raw pixel data. then each 16bit number would have 8 bits of one channe and 8 of another which would screw up the prediction. a version of FLAC that uderstood TIFF structure could probably get much better compression.

yeah, i experimented and came up with similar settings, though for some reason channels = 1 gave smaller files.

i actually set the blocksize to ~10 lines... as many as i could get less than 65535 (the pic is a 16-bit decode of a CCD RAW capture from my spanking new camera... 2272 pix wide)

the little "clicks" that indicate new lines don't seem to affect compression much (maybe i missed something), as i got comparable results using headerless and tiff, and keeping blocksize to multiples of 2272.

i wish i knew enough about how flac worked, but for now i'll stay with jpeg-2000 for colour, and flac for black-and-white.