Which is the best lossless codec?, Discussion thread Options

[kotrtim]

1. LOWER COMPRESSION BUT HIGHER CPU USAGE
2. NO COMPRESSION MODE TO SELECT

EDIT : But compared with Monkey's Audio, WMA still use more power.

WMA = 34.0MB = ~13% CPU Usage
Monkey's Audio - High = 33.7MB = ~7% CPU Usage

This post has been edited by kotrtim: Nov 26 2004, 06:17

[rjamorim]

Thanks. I added "unefficient" to WMA's cons.

I'm not sure unability to select compression mode is necessarily a bad thing. It surely makes up for it in ease of use.

So, I don't think it's a factor worth taking into consideration. For the same reason, one could put "too many compression modes" in FLAC's cons.

[kotrtim]

roberto , i'm so sorry
wma compression is a bit better than flac-8, but not much

I've encoded a sample 5:39
and this is what i get
original size = 57.1 MB
wma lossless = 34.0 MB
Monkey's Audio - High = 33.7 MB
FLAC-8 = 34.8 MB
FLAC-5 = 34.9 MB
REAL Lossless = 35.2 MB
Apple Lossless = 35.1 MB

but when I playback with foobar2000
WMA = 932kbps
FLAC = 860kbps
?????is this a bug?

this is the first time i test REAL,no kidding, REAL Lossless encodes very fast, comparable to FLAC-5, deocoding use less than 2% CPU usage with real player (P4 1.4GHz)

besides pro and cons list, i think this topic
should also have a comparison list, so that the reader can compare
and pick their desired codec

this is my proporsal

1 = Excellent/Superb
2 = Very Good/Very Fast
3 = Medium
4 = Slow/Poor
5 = Very bad/Very Slow

A Codec
Encoding Speed = 2
Decoding Speed = 1

Average Compression ratio = 55%

Error Handling = 3
Seeking = 1
Tagging = 1

Hardware Support = 3
Software Support = 1

ID Tag = Vorbis Comment

Container = OGG

Recommended Usage = Playback/archieve

Hybrid = Yes
Streaming = No
Open Source = Yes
Multichannel = No
High Resolution = No
OS Support = Windows Only

This post has been edited by kotrtim: Nov 26 2004, 07:34

[sshd]

1. inefficient not unefficient

2. Neither FLAC nor Monkey (MAC) are error robust. A single bit error in a MAC file will render the file undecodeable. A broken FLAC file will be playable but probably cause error in player. FLAC and MAC have built-in error checking not tolerance. Error checking has nothing to do with "robustness" IMHO.

3. Missing items on pro/cons list: Tagging system

[BoraBora]

Another lossless comparison (in french, but the table is clear enough whatever the reader's language):

Guruboolez tests

Maybe add an item "Still in development/Not developed anymore" ?

[rjamorim]

I've encoded a sample 5:39
but when I playback with foobar2000
WMA = 932kbps
FLAC = 860kbps
?????is this a bug?

Guess so...

this is the first time i test REAL,no kidding, REAL Lossless encodes very fast, comparable to FLAC-5, deocoding use less than 2% CPU usage with real player (P4 1.4GHz)

That's interesting. Thanks for mentioning that.
I just added an entry for Real Audio.

besides pro and cons list, i think this topic
should also have a comparison list, so that the reader can compare
and pick their desired codec

this is my proporsal

1 = Excellent/Superb
2 = Very Good/Very Fast
3 = Medium
4 = Slow/Poor
5 = Very bad/Very Slow

A Codec
Encoding Speed = 2
Decoding Speed = 1

Average Compression ratio = 55%

Error Handling = 3
Seeking = 1
Tagging = 1

Hardware Support = 3
Software Support = 1

ID Tag = Vorbis Comment

Container = OGG

Recommended Usage = Playback/archieve

Hybrid = Yes
Streaming = No
Open Source = Yes
Multichannel = No
High Resolution = No
OS Support = Windows Only

Interesting idea. I'll try to come up with a table comparing them.

1. inefficient not unefficient

Oops. Thanks for correcting.

2. Neither FLAC nor Monkey (MAC) are error robust. A single bit error in a MAC file will render the file undecodeable. A broken FLAC file will be playable but probably cause error in player. FLAC and MAC have built-in error checking not tolerance. Error checking has nothing to do with "robustness" IMHO.

Well, I think I remember someone once reporting that new versions of Monkey's Audio would skip errors. Won't be able to find that post though...

And my idea was precisely to mention formats that would skip errors, opposed to formats that would break the playback from the error onward completely, like WavPack3, LPAC and so on.

Maybe I should change the wording?

3. Missing items on pro/cons list: Tagging system

Added, thanks.

Another lossless comparison (in french, but the table is clear enough whatever the reader's language):

Guruboolez tests

Nice, just added the URL.

Maybe add an item "Still in development/Not developed anymore" ?

Hrm... I think it's kinda hard to draw the line there. Where does a format stops being developed? Shorten, as a format, stopped being developed years ago. But given its popularity tools for the format and playback plugins are still being developed, so I wouldn't say the format is really dead.

I guess that would only really apply to RKau (which I probably won't even add to this list) and, maybe, LPAC.

Regards;

Roberto.

This post has been edited by rjamorim: Nov 26 2004, 13:14

[Garf]

<zealot>

I see no reason to put very fast decoding with FLAC and fast decoding with Wavpack when Wavpack actually decodes faster in the modes with the same compression.

Wavpack supports floats, which AFAIK FLAC does not. Optimfrog also supports floats, dont know about the others.

Also, Wavpack is supported in Nero.

</zealot>

[rjamorim]

<zealot>I see no reason to put very fast decoding with FLAC and fast decoding with Wavpack when Wavpack actually decodes faster in the modes with the same compression.

Good point. I changed that.

Wavpack supports floats, which AFAIK FLAC does not. Optimfrog also supports floats, dont know about the others.

Well, I think that falls into high resolution audio support. I don't think that is a popular enough feature to justify being mentioned, but I wouldn't know...

Also, Wavpack is supported in Nero.</zealot>

Ah, indeed. Thanks for reminding me.



I just added a color-coded table with features and whatnot.

Regards;

Roberto.

[guruboolez]

2. Neither FLAC nor Monkey (MAC) are error robust. A single bit error in a MAC file will render the file undecodeable.

I've tested recently with MAC 3.99 (artificial corruption through hex. editor): files are perfectly playable with foobar2000. A small part was missing, the console poped-up in order to inform me about a CRC error. No player crash, no abrupt sound or ending.
~Three years ago, I was able to do the same thing with Winamp MAC plug-in.

[kurtnoise]

Some notes about RALF :

I made some quick tests with Helix Producer to see the behaviours of this lossless codec with some others...Why Helix Producer : because there are 3 encoding modes whereas in RealPlayer it's only one. So, according to my results I'll be much less enthousiast like kotrtim...

The method that I used is the same as Speek. For the Real Audio Lossless, I extracted my wave files with EAC and then encoding with Helix Producer. Decoding has made by RealPlayer 10.

I created some tables to observe different results (it's in French, but I think it's readable.) BTW, some Hints : vitesse d'encodage == Encoding Speed ; vitesse de décodage == Decoding Speed ; Ratio de Compression == Compression Ratio ; Taille == Size.


Global Results:




Sample 1: (En Quarantaine_Miossec (extract of "1964" album) / Duration = 2''29 ) // Hint : french pop rock




Sample 2: (Sample 2 : Rock el Casbah_Rachid Taha (extract of "Tékitoi ?" album) / Duration = 4''33.) // Hint : rock.




Sample 3: (Mouth's Cradle_Björk (extract of "Medúlla" album) / Duration = 4''01.) Hint : electro.




Sample 4: (Symphony n°4 (Sehr Behäglich)_Gustave Malher / Duration = 1''52.) // Hint : classic.




Tests made on Pentium 3 800 Mhz & 512 Mo RAM. I hope that help somebody.

[guruboolez]

I forgot: nice initiative Roberto.
Two comments:
- LA is missing. Deliberate choice?
- I suggest an additional line, with "extra-future" including all things "that is a popular enough feature to justify being mentioned" like:
high-resolution audio, md5 hash, self-extract module, various containers compatibility, etc...

[kotrtim]

according to the player itself (Itunes n Real)
Apple Lossless encode at ~18X
Real Lossless encode at ~12X

If Real is "VERY FAST" Apple shouldn't be "AVERAGE"

To be more accurate, someone has to use a stop watch? and encode the same uncompressed file to measure the speed........ That needs lot of time

[rjamorim]

I've tested recently with MAC 3.99 (artificial corruption through hex. editor): files are perfectly playable with foobar2000. A small part was missing, the console poped-up in order to inform me about a CRC error. No player crash, no abrupt sound or ending.
~Three years ago, I was able to do the same thing with Winamp MAC plug-in.

Hrm... the plot deepens

OK then, I put Monkey's as error robust again.

Some notes about RALF :

I made some quick tests with Helix Producer to see the behaviours of this lossless codec with some others...Why Helix Producer : because there are 3 encoding modes whereas in RealPlayer it's only one. So, according to my results I'll be much less enthousiast like kotrtim...

The method that I used is the same as Speek. For the Real Audio Lossless, I extracted my wave files with EAC and then encoding with Helix Producer. Decoding has made by RealPlayer 10.

I created some tables to observe different results (it's in French, but I think it's readable.) BTW, some Hints : vitesse d'encodage == Encoding Speed ; vitesse de décodage == Decoding Speed ; Ratio de Compression == Compression Ratio ; Taille == Size.
Tests made on Pentium 3 800 Mhz & 512 Mo RAM.  I hope that help somebody.

It will be very useful for me, thanks. And no worries about french, je peut comprendre un petit peu

I forgot: nice initiative Roberto.

Thanks

Two comments:
- LA is missing. Deliberate choice?

No. As I wrote near the end of the comparision:
"Formats I need help from forum members about pros and cons: TTA, LPAC, LA..."

I didn't add LA because of lack of information on it. But I guess i'll add it now with whatever I know about it, and count on users to help me fill the gaps.

- I suggest an additional line, with "extra-future" including all things "that is a popular enough feature to justify being mentioned" like:
high-resolution audio, md5 hash, self-extract module, various containers compatibility, etc...

Good idea. Maybe something that doesn't really justify as pros and cons, like container support, but still might be interesting to some people. I'll work on it.

according to the player itself (Itunes n Real)
Apple Lossless encode at ~18X
Real Lossless encode at ~12X

If Real is "VERY FAST" Apple shouldn't be "AVERAGE"

To be more accurate, someone has to use a stop watch? and encode the same uncompressed file to measure the speed........ That needs lot of time

Haha, dude, you said yourself Real is very fast

this is the first time i test REAL,no kidding, REAL Lossless encodes very fast

Since Hans never got around to testing Real on his comparision, I couldn't rely on him to provide speed values. So I relied on your data :B

[guruboolez]

Another suggestion: what about "adaptability" of each format? In other word, the possibility for the user to choose a better (for himself) compromise between ration/encoding-decoding time. Some people are not interested by the defaut setting of their favorite lossless encoder, but are using different profile : in order to reach an ultra-fast decoding, or to obtain the very best encoding ratios. Monkey Audio for exemple allow impressive ratio, with still acceptable decoding/encoding speed. WavPack allows very-fast decoding process, acceptable ratio but very slow encoding speed (asymetrical mode).

Some format are very malleable, and therefore very different for different users (MAC, OptimFROG, WavPack). Some other format are very restrictive: you must accept the choice made by the developer (TTA, WMA, ALAC). This flexibility is in my opinion something very precious. The current table is a bit simplifying, and it masks some of possible features of these flexible audio formats.

This post has been edited by guruboolez: Nov 26 2004, 18:53

[rjamorim]

Some format are very malleable, and therefore very different for different users (MAC, OptimFROG, WavPack). Some other format are very restrictive: you must accept the choice made by the developer (TTA, WMA, ALAC). This flexibility is in my opinion something very precious. The current table is a bit simplifying, and it masks some of possible features of these flexible audio formats.

OK, that makes sense. I added a "Flexibility" entry in the table. Codecs with 4 or more different settings (which actually make some difference, unlike FLAC which outputs pretty much the same results after -4) are ranked very good. LA only has two modes, so it's ranked average. The formats with only one mode are ranked bad.

I ranked Real Audio bad since you can't reach two of the three settings on Real Player, you need Producer for that...

[guruboolez]

I would say "none" rather than "bad" flexibility. The lack of flexibility is not necessary a bad thing, and sometimes makes sense (alac for exemple: hardware decoding/battery life as main target).

[HansHeijden]

Since Hans never got around to testing Real on his comparision, I couldn't rely on him to provide speed values. So I relied on your data :B

Indeed back in May I wanted to test Real as well, and initally had several problems getting the software to function correctly. With the help of Karl that was solved ultimately. But then it appeared decoding required the "plus" version of RealPlayer 10. Only free solution for decoding was the Helix software. All these complications made me decide to put Real lossless on hold, so it can mature and become free. (See edit below)

FWIW, I do however still have the 7-album ("all") result, for compression only:
high mode: 56.94% at 3.2x speed
medium mode: 56.98% at 4.2 speed
low mode: 59.36% at 10.5 speed

Great to see that all the other important properties of lossless compressors are becoming gathered here, that's too much, and too changeable for me to deal with!

Edit: After reading again some emails with Karl, I recall some more things. In the end I used the Helix software for both compress and decompress. It was free and only required registration. But the files produced some audible skips during playback in RealPlayer, and certain error messages during decompression with Helix.

This post has been edited by HansHeijden: Nov 26 2004, 21:14

[kotrtim]

Haha, dude, you said yourself Real is very fast w00t.gif

for me its very fast man, LAME 3.96 mp3 preset stansdard only encode at ~4X
MPC at ~6X, Monkey's Normal at ~8X, that's why 12X to me is considered very fast

EDIT: Mplayer Linux, Xine Player Linux can play both WMA pro and Lossless

This post has been edited by kotrtim: Nov 27 2004, 04:52

[rjamorim]

Edit: After reading again some emails with Karl, I recall some more things. In the end I used the Helix software for both compress and decompress. It was free and only required registration. But the files produced some audible skips during playback in RealPlayer, and certain error messages during decompression with Helix.

I see...

Your results will be valuable neverthless. Thank-you for them.

EDIT: Mplayer Linux, Xine Player Linux can play both WMA pro and Lossless

I know, but I would rather list only official support, and not hackish windows emulation that will only work on x86 Linux.


I'm travelling right now, and depending on cybercoffees without MS Excel, so I probably won't update the table until I return, around December 10th.

Thank-you for all your support so far.

Regards;

Roberto.

[jcoalson]

this table is nice, good to finally have a sticky topic, hopefully it will cut down on the redundant questions.

for the 'hardware support', 'good' for ALAC seems like a stretch as it is only supported in one device (ipod); FLAC devices are in the teens or maybe 20s now

http://flac.sourceforge.net/links.html#hardware

2. Neither FLAC nor Monkey (MAC) are error robust. A single bit error in a MAC file will render the file undecodeable. A broken FLAC file will be playable but probably cause error in player. FLAC and MAC have built-in error checking not tolerance. Error checking has nothing to do with "robustness" IMHO.

FLAC is error tolerant; damage is limited to corrupted frames. whether any particular player chooses to recover from errors or stop with with a warning is an implementation detail. I believe it's the same with wavpack.

Josh

[sshd]

2. Neither FLAC nor Monkey (MAC) are error robust. A single bit error in a MAC file will render the file undecodeable. A broken FLAC file will be playable but probably cause error in player. FLAC and MAC have built-in error checking not tolerance. Error checking has nothing to do with "robustness" IMHO.

FLAC is error tolerant; damage is limited to corrupted frames. whether any particular player chooses to recover from errors or stop with with a warning is an implementation detail. I believe it's the same with wavpack.

Error tolerance usually means you can damage some bits and recover them later - i.e. RAID1, RAID5, par2, ...

[sehested]

for the 'hardware support', 'good' for ALAC seems like a stretch as it is only supported in one device (ipod); FLAC devices are in the teens or maybe 20s now

ALAC is also used by Airport Extreme with AirTunes. Furthermore iPods comes in different models, even different manufacturers (HP).

[Zurman]

How about an overall notation system?
3 points for very good/fast
2 for good/yes/fast
1 for average
0 for bad/no/slow

[BoraBora]

About OS support: I understand what it means, but newbies could misinterpret this as "I can play this format on my OS" which is not exactly the same thing. Correct me if I'm wrong but WavPack can't yet be read on Linux. And probably not yet on MacOS either, I suppose.

About hardware support: from a mass-market point of view, I think no lossless codec can earn a "good" score. Players bought in millions in the whole world are portable players, car players and standalone DVD players. Everything else is still a niche market (home streaming etc.). ALAC and Flac have somewhat good support when you compare them to other lossless codecs (which have none ) but from a consumer point of view are a very limiting factor in hardware choice. Including all codecs, lossy and lossless would go like this:

MP3 = Very Good
Ogg Vorbis = Good
ALAC and Flac = Average or bad (I'd say "bad" myself)

Of course, this is only valid from a mass-market point of view.

This post has been edited by BoraBora: Nov 28 2004, 23:51

[WaldoMonster]

Nice work.

I'm missing native ReplayGain in the table.
I know that FLAC support this feature.