I was listening to Pet Shop Boys - In The Night from album (1986) Disco (The Remix Album) in mp3 -V4 and noticed the sound was not clean. So I figured it was time to compare Lame v3.98, Nero v1.1.34.2 and Ogg aotuv20080728 (supossedly the best version available) in the 100-190 kbps range. Every year or so I compare the most recent versions available.

I focussed on the 11.6-13 sec range but there are a lot of spots suitable for ABXing. Here go my ABX results.

Lame v3.98 produced 163 kbps output.


As Nero mostly often needs less bits to get a similar result to Lame I choosed to ABX Nero q0.40 which produced 138 kbps output.


AS Nero q0.40 was also easy to ABX I ABXed Lame 3.98 (163 kbps) against Nero q0.45 (170 kbps) to see if I could match both lossy files and draw a conclusion about quality.


On this sample and at these bitrates Nero clearly wins on quality as the distortions are much smaller than with lame3.98.

I got dissapointed about Ogg at q4 (133 kbps) and q5 (174 kbps) because it was too easy to ABX, the distortions were clearly worse than with Lame and Nero. I've tried some other songs (Prince, DJ Tiësto) and must conclude that Ogg Vorbis can't keep the pace of quality improvement of Lame and Nero.

Edit: Sample can be found here.
Edit2: Output bitrates mentioned in my posts are averages of the entire song, not of the 27 secs sample.

The relation of Nero vs. Lame won't change, but do you mind trying a Lame 3.98 very high quality setting like -V0 or --abr 270?

I think mp3 users are aware of trading top playback compatibility against inferior quality per given bitrate compared to other codecs in critical situations, but with today's players' capacities using very high bitrate is an option for many users.

My ABX results focussing on range 11.7-12.8 secs:

Lame v3.98 abr 200 which resulted in a 194 kbps average output. This one wasn't difficult.


Lame v3.98 V0 which resulted in a 278 kbps average output. This one wasn't very difficult to ABX which surprised me.


Since I had a post in mind that a specific sample encoded in V2 did better than V0 I tried to ABX V2 and got no clear results as well as with abr 270. I took a break because I noticed in previous ABX sessions that hearing gets tired as well as my brain. When I tried again several hours later I got this:

Lame v3.98 abr 270 which resulted in a 264 kbps average output. This one was hard, these results I got after a third trial reset.

Then I tried again Lame V2 (222 kbps output) and after 5 trial resets I gave up. Maybe I'll try again when I get up with a fresh mind.

Finally I listened to both V0 (278 kbps) and abr 270 (264kbps) and abx-ed both lossy files 8/10 5.5% but my actual goal on this one was to compare distortion and artifacts (nearly not present). Clearly File A sounded better and that was the abr 270 sample.

When you say trial reset, do you mean re-performances of the tests? You might consider posting the results of the additional trials that you performed - they are required for the results to be meaningful.

Why do the number of trials in your tests vary? 11, 7, 18, 10, 13, 18.

I exit the ABX tool and try again later. With hard samples at high bitrates normally I have to train/warm up my ears and spot the artifact which can differ with different encoding parameters.

I stop abx-ing of very tough samples when guessing probability is below 1% because of hearing fatigue risk when continuing. Where to put the limit? For the purpose of the ability to distinguish files I think it's better ending abx-test based on guessing probability than a fixed number of trials.

Pio wrote an excellent piece which explains ABX testing methodology.

Rule 3 (parts A and B) are particularly important here.

Thanks for the link, I haven't seen that post before nor the linked ones inside it.

Rule 3 is nice but a lot can be discussed about it because every coin has two sides. Leaving a complete discussion for an other thread I just want to mention that the consistency and rigor of this rule break by allowing training sessions. In the absence of a notary there will always be doubts.

I usually do not hide the results to shorten the time spent but reading other posts linked I see the weakness of finishing after a low enough p value. When treating tough samples hiding results would just lead to spending a lot more of time and better results (higher success/failure ratio). The actual problem is one never knows if he/she is in top shape, not even Usain Bolt will do the 100 metres every day below 10 secs (and abx-ing is not a competition).

There's always a tradeoff.

I have this album and will try to ABX your sample.
LAME 3.98.2

First Trial:



Second Trial:



I thought I heard something, but then, it's V4, pretty transparent to me.
I guess V0 keeps being my comfort zone since I don't encode tracks at their limit of transparency.

Have you tried with the official ogg/vorbis 1.2.0? I have listened to some of the samples suggested at http://www.hydrogenaudio.org/forums/index....=64673&st=0, and for some of the samples I can't hear a difference between the original and mp3 at 128 - but I think that the difference is very clear i the example with "Vangelis___Chariots_of_Fire__edit_.flac". With ogg/vorbis at 128 I can't hear a difference.

So I would like to know what I should listen for - especially when using ogg/vorbis.

Regards,
Morten

.

Alexxander, could you please describe the Aotuv q5 distortions which 'were clearly worse than with Lame and Nero'?

Also, can anybody with good hearing capabilities confirm Alexxander's conclusion about the quality inferiority of the latest OGG Aotuv at q4-q5 to Lame 3.98 and Nero Digital?

Oh my, I give it a try. In this sample I ABX on the synthesizer that's making the dominant tune (is on foreground) in the first 17 secs of the song and also on the pacemaking cymbalsound (some acompanying high frequency artifact). Let me make clear that lame3.98 -V4 (163kbps) sounds horrible but aoTuVb5.6 q5.0 is close to transparant to me, as opposed to q4.0.

Thank you, Alexander, I wish I had ears half as good as yours. All my attempts to ABX your Pet Shop Boys sample encoded even with Aotuv q2 ended complete fiasco.

Some time ago I switched from Lame MP3 V0 to OGG Aotuv q7 for my music collection which I play on my HTPC through a very decent Hi-Fi hardware. I chose the OGG format because many test results and people opinions on this forum had made me believe in its advantage over MP3 quality wise while being free from some problems such as gapless playback.

Regards,
Misha

That's a bit of a streched based upon only a few samples.



He would need to post some samples first. I don't know exactly what he is hearing, but in terms of using Vorbis which I encode with I have rarely heard some samples with audible distortion. Albeit they do exist, but should not be a problem especially if you are encoding with -q 6 or even 7 for that matter.

I'm sorry but I didn't see an ABX test for OGG...

A quick one from yesterday I saved, those from august I haven't anymore as from time to time I empty my Temp folder. But if any ABX of other specific setting is wanted I could give it a try.


Song average is 127kbps

Yes, could be true. This is why I recently asked in an other post how to setup and do myself an ABC/HR test. Once the results of the now running mp3 Listening Test are known I will get the best mp3 encoder and do some ABC/HR against ogg and nero aac. I'm thinking about using the same samples as the running Listening Test but I have a mixed feeling about testing problematic samples against less complex ones


Flac is in this post.

On -V4 I abxed the intro tsk-tsk.. 8/8 - some typical imprecise sound like preecho.
Then at 19secs preecho on snare 8/8.

-V3 was too much effort: same spots 4/8 then 5/8

Seriously, statements like this would require an intensive multi-format listening test with many samples. The last one I've seen, Vorbis defeated LAME in nearly all samples. But that was for classical only. And the current encoders haven't been tested.