I've just made a blind test and compared results of various codecs at 64kbps. Sorry for not adding more, as it already took me numerous hours to complete the test.

test made by using PCabx and abchr tools. Test samples (10 to 20 seconds) are ripped from random CDs of various genres. They are: Nightwish-Wishmaster; Depeche Mode - Dream On; Dead Can Dance - Shadow Magnet; Beethoven - overture.

The funny thing is that only Beethoven was able to survive after such perversions With Ogg it was very close to original. The rest was always clear 10/10.

The comparison results are:
Lame ap 128>OGG>lame ap 96>WMA8 at 64kbps>WMA9 at 64 > WMA9 at VBR q50.

Weird, WMA9 was a complete crap I thought it should sound better.
But at the same time OGG Vorbis was very close to lame at 128kbps and WMA8 at 64 was near lame ap 96.

Sure, I know about a test done by ff123. Just wanted to make my own and test new WMA codec.

I think, that the only format, which could be told about, that it's somewhat like twice smaller, than MP3 but with the same quality is Vorbis. It's far from perfect at 64K, but it's really nice at such a low bitrate.

lame ap 128>ogg 64 you mean?

You should have included MP3Pro, it's a top competitor at this bitrate.

I would like to see raw results as well.

"Should I stay or should I go?" [tm]

I would give WMA9 CBR (2pass) a go, I have found VBR to be bad with WMA9.

Sure. Sorry, I've forgotten to type Ogg's bitrate.


Yes. I know. I did some tests few months ago and MP3pro sounded nearly perfect, 'till I moved to live recording (with applauds), where all the codecs failed (I don't count MPC, haven't tested it too much yet).
From the other side I wanted to test two WMAs vs. Vorbis most of all =) There's one big thing against MP3pro: a lack of good free encoders: I must either use Thomson demo or download bloated apps.
I'll try to test it next time. With AAC and MPC.


I think, they are not interesting. With WinABX I got 10/10 with mosst of the samples. From the other side Vorbis has given me 5/10 (with WinABX) with Beethoven sample, which was easy to encode (not much Hi-freq signal) and my headphones are crap
From te other side the comparison between formats was made with ????? by a simplest scheme: sample1 is better, than sample 2. Sample two is worse, than three and so on. So by making quite a large number of comarisons I've got my results.

Try WMA9Pro Q25.

The Pro codec is way better than the Std one, prolly because MS could finally break that damned backwards compatibility and upgrade the codec. WMA9Pro Q25 gives roughly 60-70kbit depending on what type of music, should fit for your 64k test.

Agreed, the 128 pro actually better than LAME -ap128

Funny!
standard at 128 k cuts off at ~18 kHz
Pro cuts off at ~16 kHz?

Anyone noticed this?

I just encoded some Fatboy Slim song (electronica) in WMA9Pro 128kbit and the cutoff is at 18khz according to the Soundforge Spectrum Analysis.

Wh-wh-what? WMA9 Pro is better than mp3 @ 128 now?

We MUST retest, to make sure Ogg is still the god, and find out how WMA is progressing.

We will not be Borg'd! We must be Xiph.org'd.

Ruairi

long way to be god, OGG now still suffers from STEREO COLLAPSE even at 128 kbit/s

MP3 & WMAPro pretty much better stereo field than OGG

Of course OGG sound better

Of course...

Could you please provide some ABX results to support that claim? (I have played around with Vorbis 1.0 -q 4 many times, and I have never noticed anything wrong with the stereo image. There is this treble boost that makes the encodes quite easy to ABX, but that's a different matter.)

Oh, and while you're at it, please test Garf's GT3 encoder, too.

I'm not saying that your findings are impossible, but without any objective test results, your claims are... just that, unverified claims, no more.

Well I can hear stereo collapsing with Ogg at less than 128 kb/s - with 41_30sec.wav or velvet.wav it is easily noticeable, without a need for ABX-ing.

The issue is - what is the best trade off between stereo image and other artifacts? IMO, I don't like stereo problems at any cause - I would rather listen to less-bandwidth source but with normal stereo image, but this is just a matter of personal taste, of course

Just started a new 64K test. It's a damned hard to make ABX comparison, when your abchr tool is not playing (damn!) music. I don't know, what the hell's with it... There's either a problem with WAWs or something wrong with decoded files.
So, here are some interesting draft results:
All the codecs can easily be divided into 2 big groups:
Group one, 31337. These are WMA9pro at q25 (24 bit encoding), MP3pro (both CBR and VBR) and OGG Vorbis
The second one is WMA8, even at 80kbps, AAC VBR "tape" and MPC "thumb" with MP3 at ap 96.
As it was expected, MPC is a complete crap on these bitrates. (The same goes to AAC, WMA8 and MP3).
WMA9pro, unlike WMA8 or "simple" WMA9 was a really nice thing to test

Some draft ABX data:
WMA9pro(q25) vs. MP3pro (VBR 50% in MMJB): 3/7
WMA9pro vs. OGG Vorbis: 4/9 (another sample)
WMA9pro vs. OGG Vorbis: 7/9
MP3pro VBRvs. OGG: 5/6

Even though the difference in files was ABXable, it was quite hard to find different fragments, where I could make comparison, but not just guest.

Other close sounding samples (no ABX test were done yet):
WMA8 at 80kbps and AAC "tape"
MP3 ap 96 and AAC "tape"

I was heavily surprised by one result: in two tests WMA8 at 80kbps went AFTER (sic!) WMA8 at 64kbps. Should check it trice in ABX again Just they sounded different and I liked 64K version more.

Don't forget, that these are still intermediate results and much more tests should be done.
I'm gonna only leave WMA9pro, Vorbis and MP3pro (should I use VBR?) to reduce test time as it takes hours to make comparisons of 9 different formats.

BTW, is anyone interested in final test results?
Feel free to post your own ABX results.

[edit]useless quote removed [/edit]

The thing what bugs me at WMA9Pro is that you can push it far below 96kbit using Quality VBR, but the minimum CBR bitrate being 128kbit. Then again, I'm suspecting a new codec different to WMA9Std.

There's a known problem when using abchr with some wav files which don't report the length properly. Usually this is the result of a particular decoder. One solution is to open the wav in a sound editor, edit a small bit from either the beginning or end of the file, and save back to wav. Another solution (so I'm told) is to resave to an ACM wav without the need to edit.

ff123

Well, guys. The test was successfully finished (great thanks to ff123 for comment, I've just "reencoded" all the WAV samples again to WAV PCM and now it works OK). Here are my results and comments.

WMA9p~>OGG~>MP3pro2=MP3Pro>MP3>MPC>AAC>WMA8 (see comments below!)

where

WMA8: WMA8 64K
MP3pro: MMJB MP3 vbr 50%
MP3pro2: MMJB CBR 64k
MP3: lame 3.90.2 --alt-preset 96
AAC: psytel 2.15 quality VBR tape
MPC: mpc thumb preset
OGG: ogg vorbis -q0
WMA9p: WMA9pro Q25 44KHz 24 bit stereo

First three codecs seems to be almost equal and need much more testing, than mine. Sometimes WMA was better, sometimes Vorbis, Sometimes MP3pro... tastes differ

Here are ABX results for some similar samples (often the more total trials, the less is difference):

Original vs. MP3pro: 5/5 pval=0.031
MP3pro vs. WMA9p: 5/5 pval=0.031
MP3pro vs. MP3pro2: 2/5 pval=0.813 (not surprising)
MPC vs. AAC: 3/5 pval=0.500
MPC vs. WMA8: 7/8 pval=0.035
WMA9p vs. OGG: 6/9 pval=0.254
WMA9p vs. MP3pro2: 7/9 pval=0.090
OGG vs. MP3pro2: 5/5 pval=0.031

On pop music sample with much drums (hihats) I liked MPC more, than MP3pro. Just there was a different kind of distortions: MP3 had more hi-freq signal, but smth ugly below it. MPC dealed worse with high frequencies, but it was more accurate with the rest.

Coming back to first three...
WMA9 vs. MP3pro vs. OGG
They all sounded different from the original, staying close one to another. In this test I usually rated higher samples with better high frequencies, as the rest was usually encoded OK (excluding, maybe, some fragments of "chewed" MP3/AAC samples and too metallic WMA8). I don't want to start a holy war here: all WMA9, OGG and MP3pro are relatively good and your test results may slightly differ from mine. Just I liked WMA9pro (can't believe I'm saying that) slightly better, than the rest

Good luck!

(Hope to start 128K test this weekend)