Hi,
I'm just about convinced that, for archive purposes, lossless compression is the way forward, especially since large HD's are relatively cheap (as are CDR's).

The question is.....do I go with Monkeys, LPAC, FLAC or whatever?
I cannot see me ever using anything other than Windows so different platforms are not really an issue.

Any advice or suggestions would be most welcome (incidentally, I rip as one large file with cue sheet using EAC if that is of any significance)

Thanks,
auldyin

I would suggest Monkey's Audio for the simple reason that it compresses better and, usually, faster.

But it's only available for Windows. Since you said that's no problem...

Regards;

Roberto.

The format really doesn't matter. Monkey's Audio will probably compress a few percents more, which saves you some insignificient amount of disk space. Should you ever change your mind, you can always uncompress and recompress into another format without any loss.

I too recommend Monkey's Audio, as it has a neat interface, but Windows only.

I'd go w/ FLAC, but that's just me.

Despite the fact that you're only using Windows, you might want to at some point play them on some other (future) computer. Monkey might not be available for that platform / device. Re-compressing is still a hassle.

It's hard to predict the future, but if I were to put my crystal ball aside and make an (un)educated guess, I would bet that if any lossless format were to become a de facto standard in the near future (as MP3 has with lossy), FLAC would have the highest probability at this point in the game of becoming so, simply because it is open, cross-platform, very portable*, easy to work with, and it works really well (the Winamp plugin has never skipped a beat, and with PP's gapless output... good stuff .

It would, because of these qualities, be a good candidate for device integration as well (a future lossless Rio-style device / car audio player...?).

That having been said, a lot of ppl use SHN right now... but then again, some of those users are beginning to migrate over to FLAC... so who knows?

Just my $0.02

--

* I've compiled and run FLAC on Windows, OSX, Linux, FreeBSD, & OpenBSD w/out any incident.

I haven't seen much discussion of it on this forum - other than the rare occasions on which I bring it up myself, that is - but in audio-trading circles (Grateful Dead/Phish/John Coltrane/Pink Floyd/what have you...) there IS already a de facto standard for lossless compression: Shorten.

I'm not joking. There are a lot of folks using it. The compression ratio is not the highest available perhaps (yes, Monkey's Audio will beat it by a few percentage points), but it is definitely one of the most portable formats (Windows, Linux and Mac ports are already available as open source freeware) and has more existing popular support than any competing standard.

etree.org (http://etree.org) offers a download of the CLI versions of this software, although there is (hopefully) a new Windows frontend in the works from one of our other respected Project Mayhem contributors....

For more info, you might check these links.

Windows: http://etree.org/shncom.html
Linux: http://etree.org/linux.html
Macintosh: http://etree.org/macos.html

Sincerely,
- M.

... there is also a Winamp plugin for Shorten. You can download it from this page: http://etree.org/shnamp.html

- M.

That is true. Shorten has a large following among certain folks. Someone once requested a ShortenDrop. Was that you M? I made it, quite some time ago, and never knew who to send it to. I'll try to get it updated and see if Roberto's interested in hosting it. BTW, I got an e-mail from David Bryant, the author of WavPack yesterday. He's working on a new version, which has a new decorrelator that gives much better performance in lossless mode and somewhat better performance in the hybrid mode. Sounds quite interesting!

Of course I am
Why didn't you send me yet?


Obs: Please, send me on monday. You already know why.

Well, compared to lossy encoding, there are not yet many lossless audio compressor users. My guess is FLAC might become the widest used lossless compressor in the future. FLAC already has hardware-support. It has other features that Shorten lacks (for example streaming support). Seems that FLAC is much more actively developed. Shorten seems to offer near worst compression of all lossless audio compressors.

For personal use, I see no reason why not use for example Monkey's Audio.

Imho, Lpac and Flac are more reliable than Monkey (look at its forum).

Bye, dB

I'd recommend WavPack since it's the best lossless format when it comes to speed/compression ratio...

Monkey's audio might be good because of the tags, but it's far too slow when using the highest compression ratio...

a) MA is really slow when used in highest compression mode, but why use it? He disrecommends it himself. Use normal or high compression.

b) Both FLAC and MA have pages comparing the different compression formats, and both pages have MA ranked above WavPack.

Since FLAC is a competitior, I see no reason to doubt that data. OTOH, you didn't give any, so I very much doubt your statement.

--
GCP

Here are some comparison pages:
FLAC's comparison: http://flac.sourceforge.net/comparison.html
Monkey's Audio 's comparison: http://www.monkeysaudio.com/comparison.html
Speek's comparison http://home.wanadoo.nl/~w.speek/comparison.htm

From this page, I gather WavPac is a bit faster but compresses quite a bit worse.

Perhaps MA in fast mode would be a good matchup then

--
GCP

hmm it may be me but it only takes around 10-20 sec to encode a song in MA's high compression

Actually I´d say it´s pretty much the opposite... Those benchmarks are old... I did a quick testing of my own with my Athlon 800 using Windows XP...

WavPack 3.93 (01-10-02) and Flac 1.0.2 vs. LPAC 1.40 using Madonna´s song "Frozen" in 48kHz, 16 bit LPCM audio (58,3 MB [61 160 516 bytes]):

CODE

Format    Switches  Time         Size                         Compression



WavPack   -h        16.42188 s   31,0 MB (32 547 257 bytes)   ~46.78%

Flac      -5        36.62891 s   31,5 MB (33 071 767 bytes)   ~45.92%

Flac      -8        153.2422 s   31,4 MB (32 938 980 bytes)   ~46.14%

LPAC      -3        57.00781 s   29,9 MB (31 396 127 bytes)   ~48.66%

As far as I can tell, Flac doesn´t even stand a chance against WavPack... Neither in compression ratio nor in speed...

[b]Edit: Just added LPAC...

I'm willing to give up a 1% decrease in file size, and a halving in compression time, for the ability to actually compress and decompress the format in the first place.

If you want to be able to read your files on any system, and still have decent compression ratios, your *only* choice is FLAC. That's it. That's why FLAC is the preferred format for audio samples on this board.

Monkey's Audio and WavPack are Windows only, and thus utterly useless, unless you like playing around with undocumented file formats. LPAC is slightly more cross platform, but is still an undocumented format, and only available on a few OSes. Shorten is widely used, but the source code is now extremely hard to find, and it has poor compression ratios, and the format is not by default seekable.

<rant>
This isn't some l33t warez board - Windows only file formats *ARE* *NOT* *ACCEPTABLE*.
</rant>

Jon??? Where was the smiley after your rant???

WavPack is also the only one with a built in lossy - recovery feature. JohnV had a good point on this issue. Would the RIAA have the legal authority to hunt down recovery files?

Well that´s just the problem... [b]auldyin (the starter of this thread) didn´t bother about any other platform than Windows so your argument is a bit wrong in that way... He might never share his music with anyone else, so why should he use an inferior format? If you´re bound to Windows and will never share your music WavPack is the only format to use... But if you want to share it, Flac will undoubtly be a better choice...

IIRC the last link has benchmarks with WanPack 3.91 or 3.92.



Don't change the subject. We were talking about MA, which you conveniently excluded from the benchmark.

I know FLAC isn't the best, but it's free software and runs on any flatform. That's why it's recommended for samples on this board.

--
GCP

I hope you mean WavPack...? Anyway, I used 3.93 when benchmarking...Does Monkey's audio have any command line encoder? If not, I cannot make an exact benchmark with my timer program... But if there is I´ll do some benchmarks right away...Well I agree that it´s the best lossless format for cross platform useage, but if there´s no chance what so ever that you´ll change platform or trade/share your music, then you can just as well stick with WavPack which has a very good size/compression time ratio...

Is that a laptop?

Yeah, mac.exe is the command line console encoder.

Update:

Monkey's audio 3.95a1
WavPack 3.93
flac 1.0.2
LPAC 1.40

Still using Madonna´s song "Frozen" in 48kHz, 16 bit stereo LPCM... 58,3 MB (61 160 516 bytes)Compression/time is the percentage of compression divided by the total time of encoding... Logically speaking, the higher value the better compression per time "unit"... Sort of...

Perhaps I should test some other samples too?

Guys guys guys... (gals?) ... all of 'em compress closely enough that size isn't much of an issue. If speed is the issue, then look at that. If cross-platform compatibility (whatever OS you use) is the issue, then look at that. Whatever is the main issue for you, if any!

For purposes of trading between a wide variety of systems, FLAC makes the most sense... right now. Any or all of these encoders could go cross-platform in the future, if they get popular enough. Given, open source is a benefit too, but IMHO not critical for encoding/decoding purposes (if you like programming...)

As for FLAC becoming the defacto standard, I wouldn't count on it. The "best" rarely gets to be the standard, in the real world -- other factors (like popularity & blind chance) determine success. So all of us might be doing some re-encoding a few years from now, cuz *the* format might not have been invented yet.

@l3maniac: btw, why doesn't flacdrop support dropping directories on it?

I just have to update the benchmarks because the previous ones sucked... First of all, I had an application running in the background which sucked processing power like a spunge in a glass filled with water, and second of all, the compression/time "calculations" weren´t really useful...

Green

This is very nice, but pointless. Yes, Monkey's Audio is the best Windows only music compression algorithm, but we already knew that.


It doesn't work like that. Closed file formats almost always tend to stay closed, unless you get a large enough group of people together to reverse engineer them. And, given that we have a 'good enough' format in FLAC, there is no motivation to reverse engineer the Monkey's Audio or whatever files formats. As for relying on the people themselves to open the format / port the software - there are many cases that show that this mostly doesn't happen.

I think you can all stop arguing now.

Hmm... I've NEVER seen FLAC take THAT long to compress anything... for me it's usually just a few seconds w/ -8...

Perhaps the fact that it's at 48kHz is causing the problem. I'm gonna do some tests of my own with FLAC and see if I can't pin it down...

Well, I played with it, and as it turns out, FLAC is just slow as hell with -8. I didn't notice that before, since I just use it from EAC / Grip, and do it in the background.

It is a touch slower compressing a file resampled to 48kHz than the standard 44kHz. But still.

However, I wonder how much optimizing could be done to FLAC. It's written to be portable... but I wonder what it would do to compile it with some scary opt flags, Intel compiler, whatnot... I think I'll jack with it a bit...

Eh, anyway, I still like it, so there.

And being **extremely** portable, and Open Source, I don't have to think about it becoming abandonware (plus, I can't run Monkey, LPAC, or WAVPACK on OpenBSD or OSX...). And since there is already hardware player support for FLAC, who knows?...

Anyway, speed isn't as relevant to me as compression (good) and availability (best), but I would like to see FLAC get a better score on speed anyway...

It seems very reasonable to me that Project Mayhem is standardizing on FLAC -- that's as it should be, since it enables everyone to unpack samples.

However, at the present time, I don't think the same attitude should necessarily extend to individual archiving. Right now, use what you like best (and can use), and wait for the lossless formats to "shake out."

FLAC could definitely stand some optimizing, and i don't want to use it yet for personal archiving. Monkey's Audio seems to be totally closed & proprietary (maybe even headed in the direction of ad-ware), and I don't want to use that either... for me, LPAC is a good compromise -- at least until FLAC gets better optimized.

No, you get the best ratio (lossless audio compression) with OptimFrog (windows only).

about lossless audio compression: imho is more important reliable than compression (and Monkey is not reliable).

Bye, dB

I tried OptimFrog last month and was completely unimpressed.
1. It did NOT make smaller files than Monkey.
2, No winamp plugin.
3. You want to talk SLOW???

I talked about compression ratio only

Monkey's Audio 3.95a1 (extra high)
OptimFrog 4.1 (extra mode (3)/speedup 2)

- Jan Hammer - New York Theme.wav (39838220 bytes)
Jan Hammer - New York Theme.ape (25439204 bytes)
Jan Hammer - New York Theme.frog (25267935 bytes)
- Don Henley - Dirty Laundry.wav (59733788 bytes)
Don Henley - Dirty Laundry.ape (36158676 bytes)
Don Henley - Dirty Laundry.frog (36022617 bytes)
- Jan Hammer - Crockett's Theme.wav (36526604 bytes)
Jan Hammer - Crockett's Theme.ape (20015312 bytes)
Jan Hammer - Crockett's Theme.frog (19935864 bytes)
- Depeche Mode - Little 15.wav (45675884 bytes)
Depeche Mode - Little 15.ape (22299092 bytes)
Depeche Mode - Little 15.frog (22182411 bytes)
- Depeche Mode - The Things You Said.wav (42837020 bytes)
Depeche Mode - The Things You Said.ape (21479016 bytes)
Depeche Mode - The Things You Said.frog (21382759 bytes)
- Depeche Mode - Strangelove.wav (53943164 bytes)
Depeche Mode - Strangelove.ape (29087544 bytes)
Depeche Mode - Strangelove.frog (28807726 bytes)

.......and OptimFrog is a very very young app.
However i'm a happy flac (much more reliable than Monkey) user:
-8 -V is my sign

Bye, dB

Well, I've done some tests regarding 24 and 32bit samples:

Monkey's Audio, LPAC and WavPack managed to successfully compress 24bit files.

But FLAC, Shorten and Frog failed... :/

None of the programs above managed to compress 32 bit files (tested various types with cool edit; float, packed int, etc...)

Among the lossy encoders; LAME & mppenc did both 24 & 32bit, OggEnc did 32bit but failed on 24bit(!)...

The audio input in ogg is only set up for 8 or 16 bit integer and 32 bit float. It should fail on anything else!

john33

The current flac encoder does not support 32 bits per sample, but the FLAC format does. 24 bit should work fine though, can you provide more details on that?



As far as Sachankara's comparison, one sample does not tell you all that much. If you are a jazz or classical or death metal listener your average results might be quite different. No one has done what I would consider a comprehensive comparison. I think the one on the FLAC page (limited though it is) is the only one that even crosses genres.

But when you look at it, most codecs are within a couple percent on compression. And as long as encoding speed is reasonable it shouldn't be such an issue because you only do it once. If you are talking about archiving your audio for all eternity, in a way that will give you the greatest access to it, with peace of mind that if the format dies you can still get back your data, I don't see how you can be comfortable with a closed format.

If you compress all your stuff with joe-windows-only-codec and the author dies, and Windows XXXXP DLL hell breaks your codec binary, and you have to decode to wav on an old machine and reencode to joe-open-codec, how much time is that going to take you? Probably longer than it would have compressing with slightly slower joe-open-codec in the first place.

Josh

For this test I created two wavs with Cool Edit. The first one was saved as "24-bit packed int (type 1 - 24bit)" and the second as "24-bit packed int (type 1 - 20bit)".

I tested flac 1.0.2. When I tried to encode the first one it gave me "ERROR: unsupported bits per sample 24".
The second gave "ERROR: unsupported bits per sample 20"...

This doesn't help you so much if you can't compile the source on
your operating system and you do not find out why. Or it uses
special tools to prepare compiling. For a lot of software packages
it is not a problem to spend some days until the project compiles
and the programs are working.

Open source is not a full guaranty for your files. Maintainers are
still needed, otherwise you have the same problem as with closed
source projects. Source compiles but crashs on 64 bit CPUs or
128 bit CPUs. On Linux 5.0.34. You're ******.

You can have exactly the same problems. May be a guru
cares about your program and maintains it after the original
author "died", but this can happen or not.

Take a fresh installed Linux which never saw a FLAC developer
before and then try to compile the current FLAC source.
A good way to test portability is to compile the source
on alien computers by aliens.

Other problem. If you doing exactly the things in lines 9...15
of the online help (--help) you get corrupted files on Windows.
This means the online help describes how to cook files.
Please correct this problem as soon as possible. This bug is fixed
in LPAC (1.36), Shorten (3.4) and some other programs.

And this bug is a problem introduced by C, not by Windows,
Mac OS, VMS, ...

Currently all lossless coders (except LPAC) do a very bad job
on PCM files with missing codes. This can be:

- 20 bit PCM data
- DAT longplay PCM
- 16 bit PCM coming from µLaw or A-Law
- 9...15 bit, 17...23 bit PCM
- PCM which was amplified/attenuated again after 16 bit quantization (this is a very dirty thing, but common on current main stream CDs)

The worst is 9 bit PCM and 8 bit PCM in a 16 bit PCM file.
Here it can happens that a FLAC or Monkey's Audio file is
2 or 3 times larger than LPAC. For 20 bit the difference is
typically between 15 and 20%.

WavPack does optimally compress 17-23 bit PCM data, and will optimally compress 20-bit PCM whether or not it is indicated as such in the header. Older version would handle 9-15 bit PCM, but I found that virtually no program would even read such files, so this was dropped at some point.

I have considered recognizing missing codes because I have also seen them in commercial CDs, but it can be time consuming (I would be surprised if LPAC does this in its fast mode) and usually doesn't gain anything. Also, if amplification is done with dither (which it should be!) then missing codes won't be generated at all.

There is a new beta version of WavPack available that has a new high mode available in both lossless and hybrid modes that achieves compression more comparable to LPAC and MAC. For lossless, the old high mode has become the default and the old default has become the fast mode. There were also two bugs fixed (one involving very long filenames and one with decoding WavPack files earlier than version 3.0). The new modes are not yet supported by the winamp plugin and the docs are not updated, but this should all be done shortly. If anyone is interested it is available here:

www.wavpack.com/wp394b.zip

Cheers…

That is very true.

I once tried compiling Mppdec on BeOS (Not that I would use it, but Frank mentioned in his page that he was looking for decoder binaries, and I wanted to help).

I downloaded all possible source packages (at that time there were many), and tried to compile them all. With or without ASM.

Unfortunately, GCC started flooding me with errors. I tried to see if I could fix them, but to no avail.

I gave up.

Of course, now, there is a BeOS decoder. But maybe there wouldn't be. (BeOS is officialy dead, and Be Inc. is out of business)

So, open source is, indeed, not a guarantee that your can use the program on every plataform.

Regards;

Roberto.

Btw: I couldn't compile FAAD on BeOS either. But I don't care anymore, my FreeBe loopback file is gone for good.

Jeesh, if you're gonna make an argument in favor of open source / codecs and against windows-only stuff, at least make it a good one...

Good point. The issue isn't about open/closed code, the issue is about open/closed file formats. Do I mind that almost all the Windows ZIP compressors are closed source? No, because I have the file format specification, so no matter what system I use in the future, I can write a decoder.

In many open source projects, the source itself doubles as the file format specification. This isn't ideal - certainly not as good as a seperate specification, but is light years better than a closed file format which can't be decoded without reverse engineering.

For archival purposes, you need to rule out any closed file formats. You also need to be suspicious of formats with no formal documentation, only source code (the source code may be buggy, or otherwise badly written). The specification is much, much more important than the actual performance of the encoder/decoder.

FLAC is has good documentation of some parts of the format (the streaming layer in particular), and adequate documentation of others (through the source code). This is currently the best you can do. Monkey's Audio, LPAC, and the fifteen million other undocumented file formats, are found to be unsatisfactory at the very first step.

Thus, if you're looking for an archival lossless file format, FLAC is currently your best option.

I'm a little bit confused about something and please bear with as I haven't done much lossless encoding (though I have d/l'ed several albums in .ape format and was pleased with how fast and easy it was to decompress 'em so I could encode to mpc). So my question is why the different compression settings that were mentioned? If it's lossless it's all going to sound the same no matter whether you use regular, high or very high compression right?So why would it be recommended that I not encode to Money Audio using very high compression to get the smallest file possible?

YOU can, but I can't.
What good it does if I have all the format specs, but can't write a line of code, and no one else (specially programmers) is interested in the format?

In this aspect (I.e: formats for people that don't have programming skills), I would rather stick with some format that has become mainstream, or backed up by a large corporation (Like DWG, by AutoDesk) rather than some open and rarely used counterpart. (Obs: OpenDWG hasn't completely reverse-engineered DWG yet, so it's not really an alternative)



Again, I don't agree. It's like wishing the world was made only by ISO formats, with excellent docs and specs. It simply isn't like that, and, frequently, you have to stick with closed formats to do archival. If not music archival, Image archival, Document archival, General data archival...

It's a case of being sensible enough to choose what best suits your needs, choose a format that won't likely die, and hope that it doesn't happens. If a format dies, you aren't a good programmer, and no one else uses that format, than there's nothing you can do.

Regards;

Roberto.

Where did you read that?

I didn't find it in the thread.

Anyway, I don't agree. I believe you should always use the best compression in Lossless files.

QUOTE

Well, I like Robin Whittle's page on lossless compression. It's quite a good comparision (Although a bit outdated, and Flac isn't compared to others), with various different musical genres. And Rkau is a clear winner, with better results in 10 out of 12 tests (9 out of 11, of you want to leave out pink noise)

And that leads to a question: If Rkau is better than the others, why isn't everyone using it?

It's not just because it is closed, or it's slow. The general people simply don't care about what's best (And don't care about closed/open source).
They will use what's easier for them. And, at this moment, the easier is Monkey.

Regards;

Roberto.

Then I can pay someone to write a decoder. Given full documentation of a file format, it wouldn't be that difficult, particularly if you were only looking to decode files.


This is inapplicable. FLAC is not rarely used, and there is no monopoly or traditional supplier of other lossless compression formats.


I quite agree - and the needs for archival are, first and foremost, that you will be able to retrieve your data. Companies come and go very frequently, and the overwhelmingly popular closed formats of one generation (wordstar, wordperfect), become obsolete, archaic, and eventually unreadable, unless there is information out there about the file formats.

Another comparison between MAC, LPAC, FLAC and WavPack: http://home.wanadoo.nl/~w.speek/comparison.htm