Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Latest comparison of popular lossless compressors (Read 10045 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Latest comparison of popular lossless compressors

Hi, inspired by the lossless compressors comparison on true-audio.com, I'd like to do an updated version of that, and present the results on my website.
I plan to do this on a running basis, meaning, that whenever a new version of whatever compressor comes out, the results will be added to the comparison.

Now, to start, I'd like some suggestions what kind of music or whatever signals I should use as test set.

TAU used six different albums, all of them were CD rips, which I think is OK for the general user, but does not include other sampling rates (for instance direct A/D conversion by the sound device) or other types of signals (for instance speech, other type of signals used for technical or scientific applications, etc.).
I'd like some suggestions for music, speech, and other signals. If you can, you could point me to the files containing the signals I should do the compression tests on (no copyrighted material, if possible).

I did some preliminary tests, but I'd like to do that on a wider range of signals and then compile the results with rrdtool or something. Otherwise, I'm stuck with using CD rips and some files I found on the internet myself, where the origin is not really clear...

Latest comparison of popular lossless compressors

Reply #1 –
Quote from: polemon on
Hi, inspired by the lossless compressors comparison on true-audio.com, I'd like to do an updated version of that, and present the results on my website.
I plan to do this on a running basis, meaning, that whenever a new version of whatever compressor comes out, the results will be added to the comparison.

Now, to start, I'd like some suggestions what kind of music or whatever signals I should use as test set.

TAU used six different albums, all of them were CD rips, which I think is OK for the general user, but does not include other sampling rates (for instance direct A/D conversion by the sound device) or other types of signals (for instance speech, other type of signals used for technical or scientific applications, etc.).
I'd like some suggestions for music, speech, and other signals. If you can, you could point me to the files containing the signals I should do the compression tests on (no copyrighted material, if possible).

I did some preliminary tests, but I'd like to do that on a wider range of signals and then compile the results with rrdtool or something. Otherwise, I'm stuck with using CD rips and some files I found on the internet myself, where the origin is not really clear...

Now everybody who benchmarks audio creates own test set...I think it would be good to have a public one, so results of different benchmarks are really comparable. I believe that after enough searching we'd be able to find enough free music to create a good data set.
Squeez Chart is a good starting point, it's data is arbitrary and hardly representative of anything, but at least some of the samples are available.

Latest comparison of popular lossless compressors

Reply #2 –
From SqueezeChart web page:
Quote
Better compression of lossless digital audio may be achieved by changing from Linear PCM to newer coding technologies such as Sony's DSD.

I very much doubt it.

Latest comparison of popular lossless compressors

Reply #3 –
I've done my first benchmark. The software I wrote to do those benchmarks automatically, has a bug when computing the total playing time, but the rest is almost ready to be published.

For the first test I used just one ripped CD.

This is what the software has as output:
Code: [Select]
Playing time: 01:08:08 (4088)

             size        compression time   decompression time   compression factor

WAV:         728474748   00:00.000          00:00.000            1

FLAC (0):    571962015   00:58.006          00:53.643            0.7851500914346
FLAC (1):    547779202   01:01.950          00:58.514            0.75195359002341
FLAC (2):    545713763   01:12.050          00:58.105            0.74911829750892
FLAC (3):    547647024   01:06.640          00:53.399            0.75177214516158
FLAC (4):    521124620   01:22.850          00:59.429            0.71536401423759
FLAC (5):    519090667   01:47.210          00:56.900            0.71257194353702
FLAC (6):    518429045   01:49.610          01:00.040            0.71166234165745
FLAC (7):    517908583   04:30.130          00:57.408            0.71094925997352
FLAC (8):    517061226   07:32.910          01:00.420            0.70978606659952

WV (f):      524145004   01:23.140          01:34.090            0.71951018952822
WV (normal): 515684302   01:46.650          01:36.280            0.70789592009303
WV (h):      510651950   02:15.730          02:06.540            0.70098785359681
WV (hh):     508619550   03:02.150          02:47.560            0.69819791474776

TTA:         514173563   01:43.750          01:57.800            0.70582208156377


Version Info:

FLAC 1.2.1
WavPack 4.60.1
TTAenc 3.4.1

Once I get RRDtool to produce nice plots from that date, they'll be much more eye-pleasing.

I think this first test is quite interesting in some parts, as it reveals, that TTA is slower in decompression than compression.
It seems that other formats than FLAC make smaller files and have faster compression, but FLAC is still unbeaten when it comes to decompressing the data.

Latest comparison of popular lossless compressors

Reply #4 –
So, from your test it looks like FLAC decompression time doesn't always 'scale' with the compression ratio. Is this expected or was it more due to the testing not being perfect (perhaps the CPU was being used by other processes at times)?

Latest comparison of popular lossless compressors

Reply #5 –
Are you going to include other formats? e.g. TAK, ALAC

Latest comparison of popular lossless compressors

Reply #6 –
Are you going to use a variety of material? Metal usually compresses less than pop for example.

Latest comparison of popular lossless compressors

Reply #7 –
Quote
Metal usually compresses less than pop for example.
Sounds real fun. Even if there is some correlation then it must be just negligible.

Latest comparison of popular lossless compressors

Reply #8 –
Quote from: Brand on
So, from your test it looks like FLAC decompression time doesn't always 'scale' with the compression ratio. Is this expected or was it more due to the testing not being perfect (perhaps the CPU was being used by other processes at times)?
The CPU is a rather slow Intel Atom at 1.6GHz single core with Hyper Threading. I decided to use a slow CPU, so algorithmic impact becomes more clearly.
The CPU was used by other background processes, of course, but none that would impact the compression much.

Quote from: dv1989 on
Are you going to include other formats? e.g. TAK, ALAC
Depends. If Linux builds of those compressors are available, yes.

Quote from: probedb on
Are you going to use a variety of material? Metal usually compresses less than pop for example.
That's the whole point of this thread in the first place. I was asking for suggestions what signals I should use for testing. I'm using one CD rip right now. because compression and decompression is fast enough to develop the benchmark software. Otherwise I'd have to wait ages just to run one test.

Latest comparison of popular lossless compressors

Reply #9 –
Quote from: polemon on
That's the whole point of this thread in the first place. I was asking for suggestions what signals I should use for testing. I'm using one CD rip right now. because compression and decompression is fast enough to develop the benchmark software. Otherwise I'd have to wait ages just to run one test.


Extreme metal will give you a good test, I'd recommend something like Emperor, Nile or maybe The Berzerker

Latest comparison of popular lossless compressors

Reply #10 –
Quote from: probedb on
Extreme metal will give you a good test, I'd recommend something like Emperor, Nile or maybe The Berzerker


Please give the names of songs or albums, so I can see if I can get them.

Also, in case someone stumbles upon some interesting signals, please let me know where to get them.

I'm planning on attaching a turntable to my mixer and encode some audio directly with 48KHz sampling rate as well.

Latest comparison of popular lossless compressors

Reply #11 –
The chart and table here may give you some additional ideas, re. grouping kinds of music (not by genre but more by orchestration) and providing ReplayGain values. IMO a carefully selected sample set allows one to get instructive results without a huge sample size.

C.
PC = TAK + LossyWAV  ::  Portable = Opus (130)

Latest comparison of popular lossless compressors

Reply #12 –
Quote from: Entrase on
Quote
Metal usually compresses less than pop for example.
Sounds real fun. Even if there is some correlation then it must be just negligible.


Dunno about pop, but there's a strong correlation between loudness and compression ratio, which means f.e. that classical music is more compressible than metal and older recordings compress better than newer ones. I would be surprised if there weren't other correlating factors that make different genres compress differently.

Latest comparison of popular lossless compressors

Reply #13 –
Quote from: _m²_ on
I would be surprised if there weren't other correlating factors that make different genres compress differently.
Solo harpsichord music compresses worse than solo piano due to more treble content, so yes, there definitely are several audible factors that affect the compression ratio.