Lossless Compression Test, FLAC - WAVPACK - TAKC Options

[A_Man_Eating_Duc...]

I've have been running a few compression tests over the last few days to compare the difference between the vanilla switches of Flac, Wavpack and Takc.

Encoders used

Flac v1.2.1
Wavpack v4.41.0
Takc v1.03
MAC v4.01

For these results 332 full albums were encoded at each setting and then tallied up in excel.

Here are the graphs





Albums encoded

CODE

311 - 311
A - Hi-Fi Serious
A New Found Glory - Catalyst
A New Found Glory - Nothing Gold Can Stay
A Perfect Circle - Mer De Noms
A Perfect Circle - Thirteenth Step
AC-DC - Back In Black
AFI - Sing The Sorrow
Alice In Chains - Facelift
Alice In Chains - Unplugged
Amici Forever - Defined
Anthrax - Persistence Of Time
Anthrax - Sound Of White Noise
Anthrax - We've Come For You All
Arctic Monkeys - Favourite Worst Nightmare
Audioslave - Revelations
Autopilot Off - Make A Sound
Bad Religion - Against The Grain
Bad Religion - New Maps Of Hell
Bad Religion - No Control
Bad Religion - Stranger Than Fiction
Bad Religion - Tested
Bad Religion - The Gray Race
Beck - Odelay
Bee Gees - The Very Best Of The
Belle & Sebastian - The Boy With The Arab Strap
Belle & Sebastian - Tigermilk
Ben Folds - Ben Folds Live
Ben Folds - Rockin' The Suburbs
Ben Folds Five - The Unauthorized Biography Of Reinhold Messner
Ben Folds Five - Whatever And Ever Amen
Bic Runga - Beautiful Collision
Bic Runga - Birds
Billy Joel - Greatest Hits Disc 1
Billy Joel - Greatest Hits Disc 2
Billy Talent - II
Blink-182 - Dude Ranch
Blink-182 - Take Off Your Pants And Jacket
Bob Mould - Bob Mould
Bodyjar - How It Works
Bodyjar - No Touch Red
Breaking Benjamin - We Are Not Alone
Bush - Sixteen Stone
Cake - Comfort Eagle
Cake - Fashion Nugget
Cake - Motorcade Of Generosity
Cake - Prolonging The Magic
Cat Stevens - The Very Best Of Cat Stevens
Chicago - Heart Of Chicago 1967 - 1997
Coldplay - Parachutes
Counting Crows - Across A Wire - MTV Live from the 10 Spot Disc 1
Counting Crows - Across A Wire - MTV Live from the 10 Spot Disc 2
Counting Crows - August And Everything After
Counting Crows - Recovering The Satellites
Crowded House - Recurring Dream The Very Best Of
Dave Dobbyn - Lament For The Numb
Dave Dobbyn - Loyal
Dave Dobbyn - The Islander
Dave Dobbyn - Twist
Def Leppard - Hysteria
Dinosaur Jr - Beyond
Dire Straits - Sultans Of Swing - The Very Best Of
Elliott Smith - XO
Elton John - Greatest Hits 1970 - 2002 Disc 1
Elton John - Greatest Hits 1970 - 2002 Disc 2
Eminem - The Eminem Show
Eminem - The Slim Shady LP
Eric Clapton - One More Car, One More Rider - Disc 1
Eric Clapton - One More Car, One More Rider - Disc 2
Eric Clapton - Unplugged
Evanescence - Fallen
Extreme - Pornograffitti
Faith No More - Album Of The Year
Faith No More - Angel Dust
Faith No More - Free Concert In The Park
Faith No More - King For A Day, Fool For A Lifetime
Faith No More - Live At The Brixton Academy
Faith No More - The Real Thing
Fat Freddys Drop - Based On A True Story
Fatboy Slim - You've Come A Long Way, Baby
Fear Factory - Digimortal
Fear Factory - Obsolete
Fear Factory - Remanufacture
Finger Eleven - Finger Eleven
Finley Quaye - Maverick A Strike
Finley Quaye - Much More Than Much Love
Foo Fighters - Foo Fighters
Foo Fighters - The Colour And The Shape
Foo Fighters - There Is Nothing Left To Lose
Frank Zappa - Ship Arriving Too Late To Save A Drowning Witch
Fun Lovin' Criminals - 100 Percent Colombian
Fun Lovin' Criminals - 100% Colombian
Fun Lovin' Criminals - Come Find Yourself
Fun Lovin' Criminals - Loco
Fun Lovin' Criminals - Mimosa
Funeral For A Friend - Casually Dressed & Deep In Conversat
Funeral For A Friend - Seven Ways To Scream Your Name
Funeral For A Friend - Tales Don't Tell Themselves
Gary Moore - Still Got The Blues
George Harrison - The Best Of George Harrison
Good Charlotte - Good Morning Revival
Green Day - American Idiot
Green Day - Dookie
Green Day - Insomniac
Green Day - Nimrod
Green Day - Shenanigans
Green Day - Warning
Grinspoon - Thrills, Kills + Sunday Pills
Groove Armada - Lovebox
Guns N' Roses - Appetite For Destruction
Head Like A Hole - Double Your Strength Improve Your Health And Lengthen Your Life
Hoobastank - The Reason
Hootie & The Blowfish - Cracked Rear View
Husker Du - The Living End
Incubus - Make Yourself
Incubus - Morning View
Iron Maiden - A Real Live One
Iron Maiden - Brave New World
Iron Maiden - Fear Of The Dark
Iron Maiden - Piece Of Mind
Iron Maiden - Rock In Rio Disc 1
Iron Maiden - Rock In Rio Disc 2
Iron Maiden - Rock In Rio
Iron Maiden - Seventh Son Of A Seventh Son
Jack Johnson - In Between Dreams
Jeff Wayne - The War Of The Worlds Disc 1
Jeff Wayne - The War Of The Worlds Disc 2
Jimmy Eat World - Chase This Light
Jimmy Eat World - Jimmy Eat World
Joe Satriani - The Extremist
John Butler Trio - Grand National
John Farnham - Anthology 1
John Lennon - Imagine
John Mayer - Continuum
John Mayer - Heavier Things
John Mayer - Room For Squares
Lagwagon - Blaze
Lagwagon - Double Plaidinum
Lagwagon - Duh
Lagwagon - Hoss
Lagwagon - Let's Talk About Feelings
Lagwagon - Let's Talk About Leftovers
Lagwagon - Trashed
Less Than Jake - Hello Rockview
Less Than Jake - Losing Streak
Less Than Jake - Pezcore
Linkin Park - Hybrid Theory
Linkin Park - Meteora
Linkin Park - Minutes To Midnight
Live - Mental Jewelry
Live - The Distance To Here
Living Colour - Time's Up
Living Colour - Vivid
Long Beach Dub Allstars - Right Back
Lostprophets - Liberation Transmission
Lostprophets - Start Something
Mad Caddies - Keep It Going
Me First And The Gimme Gimmes - Have A Ball
Michael Jackson - Off The Wall
Michael Jackson - Thriller
Millencolin - For Monkeys
Millencolin - Home From Home
Millencolin - Life On A Plate
Millencolin - Pennybridge Pioneers
Millencolin - The Melancholy Collection
Misfits - American Psycho
Misfits - Famous Monsters
Moby - Play
Mr. Bungle - Disco Volante
Mr. Bungle - Mr. Bungle
Muse - Absolution
Muse - Black Holes And Revelations
Muse - Hullabaloo Disc 1
Muse - Hullabaloo Disc 2
Muse - Origin Of Symmetry
Muse - Showbiz
My Chemical Romance - The Black Parade
Nirvana - Bleach
Nirvana - Incesticide
Nirvana - Unplugged In New York
No Fun At All - No Straight Angles
No Fun At All - Out Of Bounds
No Fun At All - State Of Flow
No Fun At All - The Big Knockover
No Fun At All - Throw It In
No Use For A Name - Hard Rock Bottom
No Use For A Name - Leche Con Carne
No Use For A Name - Making Friends
Nofx - 45 Or 46 Songs That Weren't Good Enough To Go On Our Other Records Disc 1
Nofx - 45 Or 46 Songs That Weren't Good Enough To Go On Our Other Records Disc 2 Catching Zzz's
Nofx - Heavy Petting Zoo
Nofx - I Heard They Suck Live
Nofx - Liberal Animation
Nofx - Pump Up The Valuum
Nofx - Punk In Drublic
Nofx - Ribbed
Nofx - S&M Airlines
Nofx - So Long And Thanks For All The Shoes
Nofx - The Decline
Nofx - The Longest Line
Nofx - The War On Errorism
Nofx - White Trash, Two Heebs, And A Bean
One Dollar Short - Eight Days Away
Ozzy Osbourne - Bark At The Moon
Ozzy Osbourne - Black Rain
Ozzy Osbourne - Blizzard Of Ozz
Ozzy Osbourne - No More Tears
Ozzy Osbourne - Ozzmosis
Ozzy Osbourne - The Ozzman Cometh Disc 1
Ozzy Osbourne - The Ozzman Cometh Disc 2
Pacifier - Pacifier
Pacifier - Weapons Of Mass Destruction
Papa Roach - Getting Away With Murder
Papa Roach - The Paramour Sessions
Paul Potts - One Chance
Pearl Jam - Pearl Jam
Pearl Jam - Rearviewmirror
Pennywise - A Word From The Wise
Pennywise - About Time
Pennywise - From The Ashes
Pennywise - Full Circle
Pennywise - Land Of The Free
Pennywise - Straight Ahead
Pennywise - Unknown Road
Phil Collins - ...But Seriously
Phil Collins - ...Hits
Pink Floyd - Live 8
Pink Floyd - The Dark Side Of The Moon
Pink Floyd - The Wall Disc 1
Pink Floyd - The Wall Disc 2
Pink Floyd - Wish You Were Here
Placebo - Without You I'm Nothing
Queen - Greatest Hits
Queens Of The Stone Age - Queens Of The Stone Age
Radiohead - OK Computer
Rage Against The Machine - Evil Empire
Rage Against The Machine - Rage Against The Machine
Red Hot Chili Peppers - Blood Sugar Sex Magik
Red Hot Chili Peppers - Californication
Reel Big Fish - Cheer Up
Reel Big Fish - Why Do They Rock So Hard
Regurgitator - Unit
Russell Watson - Reprise
Saliva - Survival Of The Sickest
Salmonella Dub - Inside The Dub Plates
Salmonella Dub - Killervision
Seether - Karma And Effect
Shihad - Killjoy
Shihad - Shihad
Shihad - The General Electric
Silverchair - Diorama
Silverchair - Freak Show
Silverchair - Frogstomp
Silverchair - Young Modern
Simon & Garfunkel - The Definitive Collection
Skid Row - Skid Row
Skunk Anansie - Paranoid & Sunburnt
Skunk Anansie - Post Orgasmic Chill
Skunk Anansie - Stoosh
Smashing Pumpkins - Adore
Smashing Pumpkins - Mellon Collie And The Infinite Sadness - Dawn To Dusk Disc 1
Smashing Pumpkins - Mellon Collie And The Infinite Sadness - Twilight To Starlight Disc 2
Smashing Pumpkins - Pisces Iscariot
Smashing Pumpkins - Siamese Dream
Soundgarden - Superunknown
Spiderbait - Grand Slam
Spiderbait - Ivy And The Big Apples
Staind - 14 Shades Of Grey
Stevie Wonder - Talking Book
Stevie Wonder - The Definitive Collection Disc 1
Stevie Wonder - The Definitive Collection Disc 2
Story Of The Year - In The Wake Of Determination
Sugar Ray - Floored
Sum 41 - All Killer No Filler
Sum 41 - Chuck
Sum 41 - Does This Look Infected
Supergrass - Supergrass
Supergroove - Backspacer
Supergroove - Traction
Switchfoot - Nothing Is Sound
System Of A Down - Toxicity
Tenacious D - Tenacious D
The Beatles - Help
The Beatles - Let It Be... Naked
The Beatles - Let It Be
The Beatles - Magical Mystery Tour
The Beatles - One
The Beatles - Revolver
The Beatles - Rubber Soul
The Beatles - Sgt. Pepper's Lonely Hearts Club Band
The Beatles - Yellow Submarine
The Black Crowes - Shake Your Money Maker
The Darkness - Permission To Land
The Doobie Brothers - Best Of The Doobies
The Fratellis - Costello Music
The Killers - Hot Fuss
The Living End - Modern Artillery
The Living End - Roll On
The Living End - State Of Emergency
The Living End - The Living End
The Mighty Mighty Bosstones - Lets Face It
The Offspring - Americana
The Offspring - Ignition
The Offspring - Ixnay On The Hombre
The Offspring - Smash
The Police - Greatest Hits
The Reverend Horton Heat - It's Martini Time
The Rolling Stones - Forty Licks Disc 1
The Rolling Stones - Forty Licks Disc 2
The Strokes - Is This It
The Vandals - Hitler Bad, Vandals Good
The Vandals - Look What I Almost Stepped In
The Vandals - Oi! To The World
The Verve - This Is Music (The Singles 92-98)
The Verve - Urban Hymns
Tool - Aenima
Tool - Salival
Travis - The Invisible Band
Unwritten Law - Elva
Unwritten Law - Unwritten Law
Van Morrison - The Best Of Van Morrison
Various Artists - Natural Born Killers Soundtrack
Various Artists - Simply The Best Singers
Various Artists - The Blues Brothers - Music From The Soundtrack
Vicious Hary Mary - Orchestra Phantasma
Weezer - Maladroit
Weezer - Pinkerton
Weezer - The Blue Album
Weezer - The Green Album
Whitesnake - Greatest Hits
Yellowcard - Ocean Avenue
Yellowcard - Paper Walls

EDIT: Fixed graph sizes
EDIT 2: added Monkey's audio, fixed graph colors.
EDIT 3: added Monkey's audio Insane

This post has been edited by A_Man_Eating_Duck: Feb 9 2008, 22:16

[greynol]

That's a lot of data!

You can even see the steady down-trend in the file sizes.

This post has been edited by greynol: Feb 6 2008, 10:57

[A_Man_Eating_Duc...]

i know, my pc has been chugging away for 3 days solid. Thank god it didn't crash throughout the whole process.

[greynol]

I've been using TAK -p3 since it was introduced as TAK -p2 and have been very happy with the performance. Before that I was using Monkey's Audio -c3000. I wonder what the charts would have looked like with MAC in it.

[A_Man_Eating_Duc...]

i was going to add MAC as well but i couldn't for the life of me figure out how to batch it since it doesn't accept wildcards.

This post has been edited by A_Man_Eating_Duck: Feb 6 2008, 10:59

[greynol]

MAC can't accept wildcards? What are you using to do your encoding? The command line version over at rarewares works pretty well. Even does redirection to stdin or -out with some applications.

[A_Man_Eating_Duc...]

MAC won't accept *.* as input.


I just make a batch file and checks for a .wav file and if found encode it and then use sweep to run it through each directory.

[greynol]

In just a few lines you could probably create a batch that will recurse subdirectories for you, if that's what you need.

[A_Man_Eating_Duc...]

thats what i use Sweep for.

I'll use monkeys audio frontend and in a few days add MAC to the graphs.

It's going to take 4 hours for the first run on FAST mode.

[greynol]

Encoding speed between MAC -c1000 and flac -5 are comparable.
-c3000 is about the same as WavPack -hh.
-c4000 is about the same as TAK -p5.

-c5000 isn't worth your time.

Thanks for including them!

[halb27]

It would be great to see especially MAC -c4000 in comparison to TAK.

[Synthetic Soul]

Out of interest, my tests put those settings in exactly the same order (although I don't have FLAC -1 to -4).

Can I ask, was there a specific reason for your test? Are you currently looking to swap codecs? Are you surprised at all with any of the findings?

Even though I have pointed out that FLAC and TAK are incomparable at the same compression rate at least a few times before, it brings it home to me even more seeing them so separate in your chart. It also shows that WavPack has a good variety of settings.

[trev]

to me those graphs don't mean much at all without respective encoding times (and possibly decoding times).

it's always a balance of compression Vs speed. you can't judge on just compression alone.

[jetpower]

You're a winer trev. This is many albums tested so statistical error is minimal. Great stuff I'd say. And trev if you want to know encoding & decoding speed, check this on your CPU to have meaningful results. You shouldn't rely too much on results on different hardware.

[A_Man_Eating_Duc...]

Can I ask, was there a specific reason for your test?

i just wanted to see the effect of each encoder on a large music collection. You always hear people saying that A compresses heaps better (they always exaggerate) than B so I thought I would do my own test.

Are you currently looking to swap codecs?

I was thinking about switching from my FLAC -6 to Wavpack, but now seeing the result i may switch to TAK -2

Are you surprised at all with any of the findings?

Actually yes i am with the FLAC -5 to FLAC -7, i can't believe how close the compression ratios are. There is only a 84.05MB savings going from -5 to -7. (which is a saving 0.25MB's per album)

to me those graphs don't mean much at all without respective encoding times (and possibly decoding times).

it's always a balance of compression Vs speed. you can't judge on just compression alone.

I know, Synthetic Soul's website has all the info you need about that. I didn't want to reinvent the whole wheel just the spokes.
http://www.synthetic-soul.co.uk/comparison/lossless/

This post has been edited by A_Man_Eating_Duck: Feb 6 2008, 21:13

[Nick.C]

to me those graphs don't mean much at all without respective encoding times (and possibly decoding times).

it's always a balance of compression Vs speed. you can't judge on just compression alone.

How many times do you encode the raw audio data? Most people do it once (then periodically when a new codec with better features / compression / etc. comes along ). Therefore, although coding time may be of incidental importance, it's not the main one.

Decoding time is paramount, if you equate time to decode processor effort required to battery life on a portable device of some sort.

[greynol]

How often you decode depends on why you're using lossless compression. For archival purposes, you may decode only once or twice; maybe never at all.

[Nick.C]

How often you decode depends on why you're using lossless compression. For archival purposes, you may decode only once or twice; maybe never at all.

By decode, I mean play...

[greynol]

They may never be played either.

Besides, there are environments where decode speed between the listed contenders will make no difference when it comes to playback.

This post has been edited by greynol: Feb 6 2008, 21:47

[trev]

How many times do you encode the raw audio data? Most people do it once (then periodically when a new codec with better features / compression / etc. comes along ). Therefore, although coding time may be of incidental importance, it's not the main one.

well not for me. i encode 1 or 2 hours worth every week, sometimes more.

my point is that many of the results are similar, very similar, and if x takes 4 times as long as y then to some people it's not much point saving a few MB at the cost of a few hours.

i wasn't flaming or anything, just no-one had mentioned speed yet and i think it's very important.

[TBeck]

I've have been running a few compression tests over the last few days to compare the difference between the vanilla switches of Flac, Wavpack and Takc.
...
For these results 332 full albums were encoded at each setting and then tallied up in excel.

Very interesting. Thank you!

This is many albums tested so statistical error is minimal.

That's wrong...

You need a big N and a representative file set. If you don't know better (about relevant factors), you have to choose your samples randomly. A persons selection determined by his taste (systematic selection) is rarely representative.

Different genres of music will produce different results.

This sample set seems to have similar properties as the files used for Synthetic Souls's comparison.

For TAK this means:

The files don't benefit from the high predictor orders of the presets p3 to p5. The advantage of p5 over p2 is only about 0.5 percent.

It's different for Josh Coalson's FLAC comparison:

Here the advantage of p5 over p2 is about 1 percent.

:
I've been using TAK -p3 since it was introduced as TAK -p2 and have been very happy with the performance. Before that I was using Monkey's Audio -c3000. I wonder what the charts would have looked like with MAC in it.

I bet the result would be similar to Synthetic Souls's comparison: The Compression of TAK's strongest setting will be about on par with Monkey's High.

Again, this is different for the FLAC comparison: There TAK performs better than Money's Extra...

My advice: When choosing a lossless compressor, perform your own comparison with a representative selection of your files. Depending on your taste, the results may be quite different from other (public) comparisons.

TAK seems to perform best on classical music, jazz and speech but not so good (compared to Monkey's Audio "Extra" and "Insane") on files of high volume and/or affected by dynamic compression ("Loudness War").

This post has been edited by TBeck: Feb 7 2008, 07:22

[jetpower]

This is many albums tested so statistical error is minimal.

That's wrong...

You need a big N and a representative file set. If you don't know better (about relevant factors), you have to choose your samples randomly. A persons selection determined by his taste (systematic selection) is rarely representative.

I agree. I should have written more. What I meant was that large number of albums minimizes weird results in e.g. one of them.
For me personally, Man's Eating Duck album selection is, I think, closer to what I'd compress than an all-round album pool (rock music > all:)), so it is more valuable this way.

Different genres of music will produce different results.

For TAK this means:

The files don't benefit from the high predictor orders of the presets p3 to p5. The advantage of p5 over p2 is only about 0.5 percent.

It's different for Josh Coalson's FLAC comparison:

Here the advantage of p5 over p2 is about 1 percent.

Could this mean you are considering music genre presets (rock, metal, jazz, classical) in future versions of TAK for encoding albums?

[A_Man_Eating_Duc...]

here we go, MAC is now added.

I'm running an Insane encode now, just to see the results. it's going to take 30 hours to complete so i'll update the graph once done

This post has been edited by A_Man_Eating_Duck: Feb 8 2008, 06:12

[buktore]

I'm quite surprise to see MAC -High show better compression than TAK -5.

Then I look at your file and I think I understand now, most of it are rock. (guess)

With quieter music, TAK -5 have considerably higher compression than MAC -High according to my experience. But with nosier music + some jazz, MAC -High have a tiny bit better compression rate than TAK -5.

[A_Man_Eating_Duc...]

With quieter music, TAK -5 have considerably higher compression than MAC -High according to my experience. But with nosier music + some jazz, MAC -High have a tiny bit better compression rate than TAK -5.

How much does "considerably higher" mean?