IPB

Welcome Guest ( Log In | Register )

6 Pages V  < 1 2 3 4 5 > »   
Reply to this topicStart new topic
MP3 320 kbs vs Redbook ripped to WAV, Positive ABX test result with foobar on classical material (Ravel)
greynol
post Sep 18 2013, 19:12
Post #51





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (UltimateMusicSnob @ Sep 18 2013, 10:47) *
Does it rise to the level of becoming something that has to (or at least, potentially could) be addressed in the MP3 algorithms themselves?

I don't think so. Those interested in transparency will use lossless or perhaps a more modern lossy format. Once you are at the level of near-transparency, ignoring killer samples/hard to encode instruments (harpsichord), due to broken masking or otherwise, the differences are so subtle that they'll likely never be noticed except during a critical listening test where you have the lossless version quickly accessible.

QUOTE (UltimateMusicSnob @ Sep 18 2013, 10:47) *
Mix and mastering engineers of course are already well aware that their products will be played back on sub-optimal systems, and have dedicated techniques and 'bad' reference monitors just for that purpose, to see how robust the mix is in different contexts (nothing is robust to cheap OR expensive earbuds, IMO, but oh well...).

I don't have much faith in sound "engineers" when it comes to their ability to assess objective sound quality to the point that I must question their assessment of subjective sound quality. Perhaps, for me, it has been spoiled by just a few nut-jobs; oh well.

This post has been edited by greynol: Sep 19 2013, 12:56


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
UltimateMusicSno...
post Sep 18 2013, 22:02
Post #52





Group: Members
Posts: 55
Joined: 17-September 13
Member No.: 110129



To drag this back toward 'Listening Tests', I mostly do my foobar trials with closed-back headphones on. It's a purely circumstantial choice, as I have children running around the house, not particularly quiet A/C switching on and off, plenty of ambient and incidental noise generally. Without isolation, I doubt I could concentrate, let alone test.

There are probably some differences in outcomes due to testing with headphones w/no crossfeed instead of speakers. I'm assuming speakers would be more difficult, given that all other factors (including some commensurable measure of "accuracy" or "precision" in the headphone or speaker quality) were the same, but perhaps that's not the case.

In my noisy environment, it's either use the headphones or don't bother. Even with that, a good number of my tests are of the type, Run 5 rounds, fix two bowls of cereal, do 5 more.

Go to the top of the page
+Quote Post
Canar
post Sep 18 2013, 22:21
Post #53





Group: Super Moderator
Posts: 3327
Joined: 26-July 02
From: princegeorge.ca
Member No.: 2796



QUOTE (UltimateMusicSnob @ Sep 18 2013, 14:02) *
In my noisy environment, it's either use the headphones or don't bother. Even with that, a good number of my tests are of the type, Run 5 rounds, fix two bowls of cereal, do 5 more.
This methodology, far from being suboptimal, probably also keeps you from a significant amount of fatigue. smile.gif

It's often recommended around these parts to take breaks during listening tests. You've stumbled upon a best practice by accident!

This post has been edited by Canar: Sep 18 2013, 22:22


--------------------
∑:<
Go to the top of the page
+Quote Post
Wombat
post Sep 18 2013, 22:56
Post #54





Group: Members
Posts: 950
Joined: 7-October 01
Member No.: 235



QUOTE (Canar @ Sep 18 2013, 23:21) *
You've stumbled upon a best practice by accident!

I don't think it was solely by accident. It seems UltimateMusicSnob has quiete a bit of experience and tried it the correct way from posting results over posting samples to being still polite while being offended very unpolitely by me..
My excuse for that but lately there come more and more pinheads to forums only to claim all kinds of extraordinary things they don't take in question. Leave alone prove something with a valid testing process.
Go to the top of the page
+Quote Post
TomasPin
post Sep 19 2013, 00:31
Post #55





Group: Members
Posts: 168
Joined: 5-June 13
From: Argentina
Member No.: 108508



QUOTE (Wombat @ Sep 18 2013, 18:56) *
My excuse for that but lately there come more and more pinheads to forums only to claim all kinds of extraordinary things they don't take in question. Leave alone prove something with a valid testing process.



laugh.gif

I agree it's nice to help someone who tries to do things right. Some people just can't be helped (I wonder many times why they bother posting...) but it's certainly not the case here, where the OP is showing not only to be willing to do what we kindly tell him to, but also a very knowledgeable person. Kudos to him.

On topic,
QUOTE
I mostly do my foobar trials with closed-back headphones on

It's generally asserted that you can detect encoding artifacts easier by using headphones, even more if their frequency response is not even, as was previously said. IRC most developers of lossy codecs fine-tune their algorithms with headphones.

Even so, at 320kbps you should not be hearing a difference either way. Interesting case we got here...

QUOTE
There are probably some differences in outcomes due to testing with headphones w/no crossfeed instead of speakers.

There may be but not necessarily so, it depends on which set of speakers and which headphones were used in that test and how true to the source each of them are.

This post has been edited by TomasPin: Sep 19 2013, 00:33


--------------------
A man and his music: http://tubular.net/
Go to the top of the page
+Quote Post
gib
post Sep 19 2013, 01:26
Post #56





Group: Members
Posts: 227
Joined: 20-January 03
From: A Tropical Isle
Member No.: 4640



QUOTE (TomasPin @ Sep 18 2013, 13:31) *
Even so, at 320kbps you should not be hearing a difference either way. Interesting case we got here...

Let's not go that far with the absolutes (bolding mine). It's quite rare to hear a difference at such bitrates, but it does happen.

QUOTE (halb27 @ Sep 17 2013, 20:17) *
If this is the case lame3100l --bCVBR 316 may well improve the situation.

halb's lame3100 has been mentioned several times in this thread. Do you plan on trying it out, UltimateMusicSnob?
Go to the top of the page
+Quote Post
UltimateMusicSno...
post Sep 19 2013, 01:57
Post #57





Group: Members
Posts: 55
Joined: 17-September 13
Member No.: 110129



QUOTE (gib @ Sep 18 2013, 19:26) *
QUOTE (TomasPin @ Sep 18 2013, 13:31) *
Even so, at 320kbps you should not be hearing a difference either way. Interesting case we got here...

Let's not go that far with the absolutes (bolding mine). It's quite rare to hear a difference at such bitrates, but it does happen.

QUOTE (halb27 @ Sep 17 2013, 20:17) *
If this is the case lame3100l --bCVBR 316 may well improve the situation.

halb's lame3100 has been mentioned several times in this thread. Do you plan on trying it out, UltimateMusicSnob?

Absolutely, yes. I've done SF 10, LAME at 320 and at -V0. Frankly, each one requires listening for different characteristics
to discern a difference. 3100 is next.
Go to the top of the page
+Quote Post
TomasPin
post Sep 19 2013, 02:27
Post #58





Group: Members
Posts: 168
Joined: 5-June 13
From: Argentina
Member No.: 108508



QUOTE (gib @ Sep 18 2013, 21:26) *
QUOTE (TomasPin @ Sep 18 2013, 13:31) *
Even so, at 320kbps you should not be hearing a difference either way. Interesting case we got here...

Let's not go that far with the absolutes (bolding mine). It's quite rare to hear a difference at such bitrates, but it does happen.

Yes, you're right. My apologies. crying.gif


--------------------
A man and his music: http://tubular.net/
Go to the top of the page
+Quote Post
testyou
post Sep 19 2013, 03:27
Post #59





Group: Members
Posts: 90
Joined: 24-September 10
Member No.: 84113



QUOTE (TomasPin @ Sep 18 2013, 16:31) *
at 320kbps you should not be hearing a difference either way

I took you to mean: "I think that 320kbps should be sufficient to achieve transparency"? (And when it fails it is likely due to a shortcoming of the encoder/codec.)
Go to the top of the page
+Quote Post
TomasPin
post Sep 19 2013, 03:44
Post #60





Group: Members
Posts: 168
Joined: 5-June 13
From: Argentina
Member No.: 108508



QUOTE (testyou @ Sep 18 2013, 23:27) *
QUOTE (TomasPin @ Sep 18 2013, 16:31) *
at 320kbps you should not be hearing a difference either way

I took you to mean: "I think that 320kbps should be sufficient to achieve transparency"? (And when it fails it is likely due to a shortcoming of the encoder/codec.)

I meant that, yes, but I agree that the phrase should not is a bit extreme and I was not very clear. Never mind. smile.gif

EDIT: Likely due to a shortcoming of the encoder, but possibly because of the person listening and his hearing.

This post has been edited by TomasPin: Sep 19 2013, 03:57


--------------------
A man and his music: http://tubular.net/
Go to the top of the page
+Quote Post
greynol
post Sep 19 2013, 03:44
Post #61





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



There are people who can routinely discern 320 from lossless on non-killer samples.


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
TomasPin
post Sep 19 2013, 03:53
Post #62





Group: Members
Posts: 168
Joined: 5-June 13
From: Argentina
Member No.: 108508



QUOTE (greynol @ Sep 18 2013, 23:44) *
There are people who can routinely discern 320 from lossless on non-killer samples.

And I know that. Poor people... laugh.gif


--------------------
A man and his music: http://tubular.net/
Go to the top of the page
+Quote Post
UltimateMusicSno...
post Sep 19 2013, 03:59
Post #63





Group: Members
Posts: 55
Joined: 17-September 13
Member No.: 110129



Okay, this was LAME 3100 at V0. This version reminded me most of the Sound Forge encoding I used in the first example, although the bass weighting was slightly different (less as compared to SF10).

CODE
foo_abx 1.3.4 report
foobar2000 v1.2.8
2013/09/18 21:40:27

File A: C:\Users\KiarkAudio\Documents\Ravel Listening Tests\Ravel_Test_File_2_short.wav
File B: C:\Users\KiarkAudio\Documents\Ravel Listening Tests\Ravel_Test_File_5.mp3

21:40:27 : Test started.
21:40:54 : 01/01 50.0%
21:41:12 : 02/02 25.0%
21:41:41 : 03/03 12.5%
21:42:03 : 04/04 6.3%
21:42:28 : 05/05 3.1%
21:44:26 : 06/06 1.6%
21:45:26 : 07/07 0.8%
21:52:23 : 08/08 0.4%
21:53:16 : 08/09 2.0%
21:54:01 : 08/10 5.5%
21:54:18 : 09/11 3.3%
21:54:45 : 10/12 1.9%
21:54:58 : Test finished.

----------
Total: 10/12 (1.9%)
Go to the top of the page
+Quote Post
UltimateMusicSno...
post Sep 19 2013, 04:22
Post #64





Group: Members
Posts: 55
Joined: 17-September 13
Member No.: 110129



QUOTE (greynol @ Sep 18 2013, 21:44) *
There are people who can routinely discern 320 from lossless on non-killer samples.

Another question on nomenclature: a "killer sample" is one which has some feature that produces a relatively obvious artifact? The comments I've seen about "lead sample" here and there give me this impression.

The distinction I'm wondering about is between a sample that some can positive-ABX just because they hear a quality difference of *some* kind, and a second sample (possibly the "killer") in which some artifact pops out because it contains some sonic characteristic which causes something more like a breakdown of the effectiveness of the MP3 algorithm being used?

The Ravel, for example, is very low level, but low levels can be found in thousands of recordings--I wouldn't think of this as a killer sample, then. Or am I missing the meaning altogether?
Go to the top of the page
+Quote Post
gib
post Sep 19 2013, 07:11
Post #65





Group: Members
Posts: 227
Joined: 20-January 03
From: A Tropical Isle
Member No.: 4640



QUOTE (UltimateMusicSnob @ Sep 18 2013, 16:59) *
Okay, this was LAME 3100 at V0. This version reminded me most of the Sound Forge encoding I used in the first example, although the bass weighting was slightly different (less as compared to SF10).

I just gave ABXing this clip a shot. At -V5 I think I was on to something, but after 5 trials or so I seemed to lose it. Maybe if I'd taken a break I'd have done better. Regardless, I think I can safely say ABXing this clip at any sort of high bitrate is out of my league.

I don't meant to pester you into another test, but with halb's lame3100l, perhaps you should try the settings he himself recommended earlier in this thread: --bCVBR 316
Go to the top of the page
+Quote Post
UltimateMusicSno...
post Sep 19 2013, 16:11
Post #66





Group: Members
Posts: 55
Joined: 17-September 13
Member No.: 110129



QUOTE (gib @ Sep 19 2013, 01:11) *
QUOTE (UltimateMusicSnob @ Sep 18 2013, 16:59) *
Okay, this was LAME 3100 at V0. This version reminded me most of the Sound Forge encoding I used in the first example, although the bass weighting was slightly different (less as compared to SF10).

I just gave ABXing this clip a shot. At -V5 I think I was on to something, but after 5 trials or so I seemed to lose it. Maybe if I'd taken a break I'd have done better. Regardless, I think I can safely say ABXing this clip at any sort of high bitrate is out of my league.

I don't meant to pester you into another test, but with halb's lame3100l, perhaps you should try the settings he himself recommended earlier in this thread: --bCVBR 316
Thanks for the reminder. I'll do that setting next.
Go to the top of the page
+Quote Post
halb27
post Sep 19 2013, 16:36
Post #67





Group: Members
Posts: 2414
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



Thank you for testing Lame 3100 (Lame 3.100a2?, lame3100l?) -V0. It's really remarkable that you got at 5/5 within just two minutes.
But if the issue is really due to low volume, -V0 is hardly the solution, no matter used with Lame 3.100a2 or lame3100l. I'm looking forward to your --bCVBR 316 result.

This post has been edited by halb27: Sep 19 2013, 16:41


--------------------
lame3100m --bCVBR 300
Go to the top of the page
+Quote Post
UltimateMusicSno...
post Sep 19 2013, 17:47
Post #68





Group: Members
Posts: 55
Joined: 17-September 13
Member No.: 110129



QUOTE (halb27 @ Sep 19 2013, 10:36) *
Thank you for testing Lame 3100 (Lame 3.100a2?, lame3100l?) -V0. It's really remarkable that you got at 5/5 within just two minutes.
But if the issue is really due to low volume, -V0 is hardly the solution, no matter used with Lame 3.100a2 or lame3100l. I'm looking forward to your --bCVBR 316 result.

That was LAME 3100L. Actually 3100l, but then it's hard to make out what the last character is--I had to hunt around in Command Prompt to find the directory ZIP just created, thought I had a capital I there.

For the target market and usage of MP3 generally, I'm coming to the opinion that the sort of ABX-ing I'm doing is mostly beside the point.

I hunt for audible differences in order to get these results. They are subtle, and they do take some hunting. Once I find the audible distinctions, completing the ABX with positive result can go quickly. Not always--depends on what I'm listening for--bass can take longer than treble, for example.
Aside from demonstrating that MP3 is not *perfectly* transparent for *all* listeners under excruciatingly detailed inspection--which is in fact my research question--it's difficult for me to see how these results could advance the state of the art at all. All I've got at the end is a 'black swan', per John Stuart Mill, falsifying a universal proposition.
Go to the top of the page
+Quote Post
halb27
post Sep 19 2013, 18:42
Post #69





Group: Members
Posts: 2414
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



IMO your listening test is valuable, especially as it's about 'normal' music, not special electronic stuff or harpsichord or other specials where it is known that mp3 can behave way below perfection.
It's also valuable for people to get a personal attitude towards your results and what they mean to them that you're adding comments like 'the difference is subtle'.
Thanks again.
BTW I also make the difference between listening tests and practical listening situations. I'm paranoid enough not to accept issues which could be audible in practical listening situations even when chance for that is very low. But there are issues that are audible to me in ABX tests when comparing carefully against the original (the harp40_1 sample for instance) but which are so subtle that to me they can be ignored for practical listening situations.

This post has been edited by halb27: Sep 19 2013, 18:51


--------------------
lame3100m --bCVBR 300
Go to the top of the page
+Quote Post
greynol
post Sep 19 2013, 18:50
Post #70





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (halb27 @ Sep 19 2013, 10:42) *
especially as it's about 'normal' music

...albeit poorly recorded 'normal' music.


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
IgorC
post Sep 19 2013, 19:27
Post #71





Group: Members
Posts: 1506
Joined: 3-January 05
From: Argentina, Bs As
Member No.: 18803



QUOTE (testyou @ Sep 18 2013, 23:27) *
QUOTE (TomasPin @ Sep 18 2013, 16:31) *
at 320kbps you should not be hearing a difference either way

I took you to mean: "I think that 320kbps should be sufficient to achieve transparency"? (And when it fails it is likely due to a shortcoming of the encoder/codec.)

I would say it's a shortcoming of a whole MP3 format. Just try electronic music with sharp transients. There was sample of electronic music, "EIG". It will never be transparent for MP3 320 kbps. Any MP3 encoder will have some pre-echo on such samples because the short block is still quite big (192) comparing to other formats (commonly 120 or 128). That's why AAC/Opus/Vobis do better job at ~192 kbps than MP3 at 320 kbps (considering high quality encoders for all formats).

P.S. Link to eig sample http://www.hydrogenaudio.org/forums/index....showtopic=49601

This post has been edited by IgorC: Sep 19 2013, 19:39
Go to the top of the page
+Quote Post
greynol
post Sep 19 2013, 19:43
Post #72





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (UltimateMusicSnob @ Sep 18 2013, 20:22) *
a "killer sample" is one which has some feature that produces a relatively obvious artifact?

Yes, they are samples that expose an encoders weaknesses; though they aren't necessarily so obvious that everyone can identify the artifacts.

QUOTE (UltimateMusicSnob @ Sep 18 2013, 20:22) *
The comments I've seen about "lead sample" here and there give me this impression.

I have no idea what a "lead sample" is.

QUOTE (UltimateMusicSnob @ Sep 18 2013, 20:22) *
(possibly the "killer") in which some artifact pops out because it contains some sonic characteristic which causes something more like a breakdown of the effectiveness of the MP3 algorithm being used?

Correct.

QUOTE (UltimateMusicSnob @ Sep 18 2013, 20:22) *
The Ravel, for example, is very low level, but low levels can be found in thousands of recordings--I wouldn't think of this as a killer sample, then. Or am I missing the meaning altogether?

I honestly don't know if this can be categorized as a killer sample. Is the entire recording this low, or does it reach full-scale (or within a couple of dB or full-scalle)? Someone coughing next to the microphone doesn't count. Honestly, this is a horrible recording. I'm not at all surprised that a lossy encoder is having a hard time making a determination of what constitutes signal.

In the end I think it's a matter of whether it's getting the noise right. I don't agree with the idea that there is a difference in tonal balance.


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
halb27
post Sep 19 2013, 19:44
Post #73





Group: Members
Posts: 2414
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



QUOTE (IgorC @ Sep 19 2013, 20:27) *
.. There was sample of electronic music, "EIG". ..

While you're totally right that mp3 isn't perfect at electronic stuff like this I think mp3 can be very acceptable even with this. eig is one of those killer samples that made me create lame3100l, and in this case it doesn't take highest bitrate settings to make it very acceptable IMO. Did you try say lame3100l -V0 on eig?

This post has been edited by halb27: Sep 19 2013, 19:45


--------------------
lame3100m --bCVBR 300
Go to the top of the page
+Quote Post
IgorC
post Sep 19 2013, 19:56
Post #74





Group: Members
Posts: 1506
Joined: 3-January 05
From: Argentina, Bs As
Member No.: 18803



QUOTE (halb27 @ Sep 19 2013, 15:44) *
QUOTE (IgorC @ Sep 19 2013, 20:27) *
.. There was sample of electronic music, "EIG". ..

While you're totally right that mp3 isn't perfect at electronic stuff like this I think mp3 can be very acceptable even with this. eig is one of those killer samples that made me create lame3100l, and in this case it doesn't take highest bitrate settings to make it very acceptable IMO. Did you try say lame3100l -V0 on eig?


habl27,

It was a several years. 3.97, 3.98, 3.99, all your extensions 3.99/3.100x..... A bit better handling but still the issue is here.
All these years LAME devs have made all possible to make it better. And it's got better but it's still there.

You can do an extraodrinady job but You can't overcome a limitation of format. No.

Go to the top of the page
+Quote Post
UltimateMusicSno...
post Sep 19 2013, 20:03
Post #75





Group: Members
Posts: 55
Joined: 17-September 13
Member No.: 110129



QUOTE (greynol @ Sep 19 2013, 12:50) *
QUOTE (halb27 @ Sep 19 2013, 10:42) *
especially as it's about 'normal' music

...albeit poorly recorded 'normal' music.

A live orchestral performance in the hall, captured to a recording, no matter how good the mic's or the engineer, always sounds 'canned' and 'distant' to me, and no amount of extra mic's or later EQ-ing has ever fixed that in my experience. A full Romantic orchestra can peak 120db or even higher (a lone operatic soprano can do nearly as well), so the dynamic range is extraordinary, and classical musicians are loath to give *anything* up to compression.

This clip starts out extraordinarily low, but in the climaxes later on it reaches to its maximum mastered level of slightly over -6db (-5.7xx db, if I recall). Is there some other sense in which this clip is badly recorded?
Go to the top of the page
+Quote Post

6 Pages V  < 1 2 3 4 5 > » 
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 19th April 2014 - 10:52