Help - Search - Members - Calendar
Full Version: How do we establish "better/best"?
Hydrogenaudio Forums > Hydrogenaudio Forum > Scientific Discussion
Pages: 1, 2
hellokeith
I am fairly sure that out of 100 people, most (if not all) would say a 44/16/stereo wav file of a contemporary/popular/easily-recognizable song does sound "better" than the same file at 11/8/mono.

But how is this established?

How is it that we know (or at least think we know) that in our car, music played on FM does sound signficantly better than music played on AM?

How is it that I can hear a song for the first time and make a subjective decision that 1) the playback equipment sounds bad or 2) the recording/mastering quality of the song is bad or 3) both?
Digga
giving a general short answer to a general question:

the fact that something for someone is/sounds/smells/(...) 'better' is because the person compares A to B and decides that because of factor a1, a2, etc A is better than B.
this lies in the nature of 'better', it's a comparison (better than, something cannot just be better per se. it is always different to something in regards to quality.

if somebody (P) would be aware of a certain input for the first time, let's say a (human) voice, then at first there is no better or worse. (the person might like it or may not because of similar non-voice stimuli that it compares it with but let's put that aside for now). then, when P gets to know some other voices from different people of different tones from the same person, etc, P will be able to compare them and decides what sounds better or worse.

let's further assume that compressed audio should reflect the reality as we see (or better hear) it most accurately to be 'good'. i.e. we compare compressed voices with non compressed voices and detect differences.
most of the time the compressed voice is labeled as worse, as it falls out of the pattern of voices we hear every day.

QUOTE
How is it that I can hear a song for the first time and make a subjective decision (about quality)
thesis:
- because you compare it to other auditive input of the category 'music' you know and decides: this one does not not reflect the (subjective) normal reality (i.e. other voices or music made and perceived directly by other humans or by yourself).
different = potentially bad.

this is just a quick sum-up what I think about this and could/should be more elaborate and better structured than as it is now but I hope you get the idea.
gaillard
I think it can also be done with only science. The human ear is a machine and can be understood given the time. Ill make an analogy to our eyes. There is a limit to how small of a pixel a person can see. it has been measured. this is for a given distance mind you. but once an object is so small that you can not distinguish it from another it with different features (like different shapes) then it doesn't matter quite what shape it is does it? we havn't reached it yet but once pixels are small enough there will be no reason to go smaller. same with audio.

there might not be a resolution to the ear since its anologe but there is a resolution to our brain. our eyes can see very small but our brain can't tell the difference.

whatever the specs are they can be measured and will be correct but people will debate them none the less.
Firon
Human beings can't really hear above ~20KHz or so (with RARE exceptions), so I'd say we've already reached the limits of audio. The dynamic range you can actually hear depends on how quiet the room is, though above a certain SPL, sound will damage your ears, so that sets an upper limit of what you should hear.
gaillard
but how much resolution? this is hard because its really the brain... not our ears as they are not digital. its like records. its analog, it can capture things so minut that digital can never match that, but if our brain has a limit how minut it can tell the difference between then it could.
Firon
Resolution of what? Audio has a lot of aspects.
And digital can capture things -exactly- (not approximately, but exactly) with the right parameters, especially in regards to audio. That "analog can capture things so minute that digital can't" argument has been refuted a lot.
greynol
http://www.hydrogenaudio.org/forums/index....showtopic=40134
http://www.hydrogenaudio.org/forums/index....showtopic=47827
Here we go again, lol.

And just to stir the pot a little, practically speaking, quantization error does prevent capturing things exactly.

But to let people know where I stand (I won't pretend that this makes a difference), I believe that 44.1/16 is more than adequate for a final product.
gaillard
an anolog system of anything has infinite resolution compared to any non quantum computing. Take for example a needle on a pivot. One end of the needle is pushing on some deformable material, clay for instance. if you where to run that clay surface by the needle at a set rate and then wiggled the other end of the needle sporadically it would get a bunch of impressions. now in digital how do you store those impression levels? you would need an infinite number of values.

this is why calculus was invented. to be able to do math with infinite things. but you can not use calculus do things machanical that are infinite, because in a given amount of time how are you going to run an infinite number of calculations?
cabbagerat
QUOTE(gaillard @ Sep 3 2006, 18:29) *

but how much resolution? this is hard because its really the brain... not our ears as they are not digital. its like records. its analog, it can capture things so minut that digital can never match that, but if our brain has a limit how minut it can tell the difference between then it could.
Please stop trolling. If you really believe this then you need to do some background reading (maybe learn what the terms SNR and quantization noise mean, for a start) before you start a debate on it.
QUOTE
an anolog system of anything has infinite resolution compared to any non quantum computing. Take for example a needle on a pivot. One end of the needle is pushing on some deformable material, clay for instance. if you where to run that clay surface by the needle at a set rate and then wiggled the other end of the needle sporadically it would get a bunch of impressions. now in digital how do you store those impression levels? you would need an infinite number of values.
I'll give you the benefit of the doubt and assume that you are merely ignorant and not a troll.

Analog electronic systems do not have infinite resolution. Small changes in voltage are swamped by noise generated in all the conductors and semiconductors in the system. This noise is measured with a Signal to Noise Ratio (SNR) - when this is low enough the signal is effectively lost. Noise is the limiting factor in both analog and digital audio systems.

QUOTE
this is why calculus was invented. to be able to do math with infinite things. but you can not use calculus do things machanical that are infinite, because in a given amount of time how are you going to run an infinite number of calculations?
Eh? What exactly are you suggesting here?
pepoluan
The only way to determine better/best, IMO, is ABC/HR.
gaillard
QUOTE(cabbagerat @ Sep 4 2006, 05:17) *

QUOTE(gaillard @ Sep 3 2006, 18:29) *

but how much resolution? this is hard because its really the brain... not our ears as they are not digital. its like records. its analog, it can capture things so minut that digital can never match that, but if our brain has a limit how minut it can tell the difference between then it could.
Please stop trolling. If you really believe this then you need to do some background reading (maybe learn what the terms SNR and quantization noise mean, for a start) before you start a debate on it.
QUOTE
an anolog system of anything has infinite resolution compared to any non quantum computing. Take for example a needle on a pivot. One end of the needle is pushing on some deformable material, clay for instance. if you where to run that clay surface by the needle at a set rate and then wiggled the other end of the needle sporadically it would get a bunch of impressions. now in digital how do you store those impression levels? you would need an infinite number of values.
I'll give you the benefit of the doubt and assume that you are merely ignorant and not a troll.

Analog electronic systems do not have infinite resolution. Small changes in voltage are swamped by noise generated in all the conductors and semiconductors in the system. This noise is measured with a Signal to Noise Ratio (SNR) - when this is low enough the signal is effectively lost. Noise is the limiting factor in both analog and digital audio systems.

QUOTE
this is why calculus was invented. to be able to do math with infinite things. but you can not use calculus do things machanical that are infinite, because in a given amount of time how are you going to run an infinite number of calculations?
Eh? What exactly are you suggesting here?




Did i say anything about an electrical system? i am merely talking about how to store something that is analog, just how to represent it. I don't appreciate all the insults.
Woodinville
QUOTE(gaillard @ Sep 3 2006, 17:34) *
there might not be a resolution to the ear since its anologe but there is a resolution to our brain. our eyes can see very small but our brain can't tell the difference.


There most certainly is a "resolution" limit to any analog, and there is certainly a well-established absolute threshold of hearing, one that is very close to, i.e. just above, the actual noise level due to molecules/atoms in the atmosphere bouncing off the eardrum (literally, yes, that's what "air pressure" is).

Furthermore, the ear reports its transduction to the brain in the form of pulse-position modulation, with quite limited time resolution, and not as good frequency resolution as you might expect. 30dB at a given level is about the best SNR you can expect from that part of the system, and that's well documented in the literature, too. Yes, Scharf says 40dB for tone masking noise in high barks, but that's because the barks are too wide at high frequencies, and you should use a decent ERB scale instead. smile.gif

The best interaural time resolution reported anywhere is in the neighborhood of 5 microseconds, and more reports are in the 10-40 microsecond range (but how you test is very, very important and touchy here).

So, there are some fundamental limits known very well.

White noise at roughly 6dB SPL is the same level as the atmospheric noise level at your ear. Going much below that (for white noise) is pretty meaningless. -5.5 dB is the best reported "noise masking noise" report in the literature that I use. More to the point, absolute threshold must take into account frequency and ERB/critical bandwidth, and it is just, barely, concievably possible that a completely unimpaired listener in the quietest possible room MIGHT JUST be able to hear that white noise in the 1kHz range near the ear canal resonance. Maybe.

So that 6dB SPL white noise limit looks pretty solid. We can take that as a decent lower level for anything ever required.

Let's go up 20 bits from that, we're at 126dB. That's louder than most any electronic reproduction system can manage, short or long term (I'm not talking about cars turned into pressure chambers for "loud stereo" testing here).

Even at 16 bits we get to 102dB. That's up there.

For 24 bits, we get to 150dB SPL. That's "instant damage to hearing" range.

Let's not even talk about 32 bit uniform PCM. That's military.

QUOTE(gaillard @ Sep 4 2006, 10:50) *
Did i say anything about an electrical system? i am merely talking about how to store something that is analog, just how to represent it. I don't appreciate all the insults.



You need to slow down right now and realize that Quantum Mechanics, the size and charge of electrons, the mass of atoms, etc, puts a low level on any kind of analog system. These limits are not exceedling small, they are not something to be waved away, we can detect one photon with our eyes, and the noise level of the atmosphere I've addressed in my previous post.

Any form of analog capture of sound suffers from the issue of atmospheric noise. Sound does not exist, ever, in any form, WITHOUT said noise, it can not, as sound is defined as short-term variations in air pressure. If you have any air, you have noise.

Said level is easily detectable with instruments, and intrudes into all real systems, no matter what kind of analog they use for their storage, be it a time-continuous, frequency-continuous analog, or a discrete-time, discrete-level analog of the original signal.

QUOTE(pepoluan @ Sep 4 2006, 06:11) *

The only way to determine better/best, IMO, is ABC/HR.


This is a separate question, and a very difficult one.

"Best" for one person may not be "best" for another. "Best" can mean "what the listener prefers", or "the most accurate" or presumably something else determined by the listener at the time of listening. If it's some kinds of especially "interesting" sound, "silence" may be the prefered sound.

But "best" is preference, and in the present world, only preference.

One can say "best capture of this point" or something like that, but until we capture an entire soundfield and reproduce it, there's nothing like an analytical "best" to be had.

"Best" is preference, really. The problem with the OP is that it is philosphically interesting more than it is technically interesting.
hellokeith
QUOTE(Woodinville @ Sep 7 2006, 22:20) *
"Best" is preference, really. The problem with the OP is that it is philosphically interesting more than it is technically interesting.


In my experience, a vast majority (if not all) automobile listeners prefer music played on CD vs FM, and on FM vs AM. This would appear to be technically measureable, as there are technical differences between CD playback and FM broadcasts & AM broadcasts. So I disagree that it is relegated to impractical philosophy. But I also think it is more than just statistical sciences, which discount human preference as arbitrary or illogical.
Woodinville
QUOTE(hellokeith @ Sep 7 2006, 23:19) *
In my experience, a vast majority (if not all) automobile listeners prefer music played on CD vs FM, and on FM vs AM. This would appear to be technically measureable, as there are technical differences between CD playback and FM broadcasts & AM broadcasts. So I disagree that it is relegated to impractical philosophy. But I also think it is more than just statistical sciences, which discount human preference as arbitrary or illogical.



You show that two measurements may correlate. No doubt.

Correlation, of course, does not imply causation. But, of course, it can.

Indeed, higher quality is undoubtedly a factor. However, I think that singling out the "technical differences" has to be justified. I agree that some quality, convenience, and durability issues are certainly involved, but that it would require a lot more work to determine what is actually happening.

As to "impractical philosophy", please show me where I said "impractical philosophy". All philosophy is not impractical, in fact this very discussion shows its import in this very subject, I would suggest. Finally, I do not agree that "statistical sciences" discount preference as arbitrary or illogical. Statistics makes a measure of a preference, it neither validates nor discounts it.
pepoluan
QUOTE(Woodinville @ Sep 8 2006, 10:20) *
QUOTE(pepoluan @ Sep 4 2006, 06:11) *
The only way to determine better/best, IMO, is ABC/HR.
"Best" for one person may not be "best" for another. "Best" can mean "what the listener prefers", or "the most accurate" or presumably something else determined by the listener at the time of listening. If it's some kinds of especially "interesting" sound, "silence" may be the prefered sound.
Exactly. Which is why ABC/HR is the only way. Suppose you got 2 waveform files, using different encoding methods. Play them inside ABC/HR. Give a score to 1 track, another score to the other track. Find out which is the best according to your score.

My solution still stands smile.gif

Note: I say ABC/HR not ABX there. Comprende? wink.gif
Woodinville
QUOTE(pepoluan @ Sep 8 2006, 11:03) *

QUOTE(Woodinville @ Sep 8 2006, 10:20) *
QUOTE(pepoluan @ Sep 4 2006, 06:11) *
The only way to determine better/best, IMO, is ABC/HR.
"Best" for one person may not be "best" for another. "Best" can mean "what the listener prefers", or "the most accurate" or presumably something else determined by the listener at the time of listening. If it's some kinds of especially "interesting" sound, "silence" may be the prefered sound.
Exactly. Which is why ABC/HR is the only way. Suppose you got 2 waveform files, using different encoding methods. Play them inside ABC/HR. Give a score to 1 track, another score to the other track. Find out which is the best according to your score.

My solution still stands smile.gif

Note: I say ABC/HR not ABX there. Comprende? wink.gif


Why do you need the hidden reference? You're asking "which of these two is better" not "which one sounds more like the original".

You can determine, for ONE person, what is "best" at a given time and place. I assert that's about all you can do in most tests.

If you test many people, you may get a variety of answers as to which is "best". Sometimes you'll get a concensus, very often you'll get several clusters of "best" that correspond to different kinds of preference.
pepoluan
QUOTE(Woodinville @ Sep 9 2006, 06:02) *
Why do you need the hidden reference? You're asking "which of these two is better" not "which one sounds more like the original".
Let me quote the description from ff123's ABC/HR page:
QUOTE
Each group (there are eight groups) consists of two files: the reference file and a sample. The listener doesn't know which is which because the program has shuffled them within the group.

Now, if you want to know what's better, you can use just one group, assign (arbitrarily) one file as the reference, and the other file as the sample. You won't know! It's blind. Run the test, assign a score to both files, the one with the highest score wins.

Even guruboolez himself had, in a few (rare) occasion, mistook the sample for the reference, and assign a score to the reference instead... so it's an honest-to-goodness blind and scored test.

QUOTE(Woodinville @ Sep 9 2006, 06:02) *
You can determine, for ONE person, what is "best" at a given time and place. I assert that's about all you can do in most tests.

If you test many people, you may get a variety of answers as to which is "best". Sometimes you'll get a concensus, very often you'll get several clusters of "best" that correspond to different kinds of preference.
I did not refute you on this. In fact, I agree with your position. "Best" for one is different from "best" for another.

I was only pointing out the tool by which one can determine which is "best" for him/her.
jlt
QUOTE
The only way to determine better/best, IMO, is ABC/HR.

QUOTE
Which is why ABC/HR is the only way. Suppose you got 2 waveform files, using different encoding methods. Play them inside ABC/HR. Give a score to 1 track, another score to the other track. Find out which is the best according to your score.

My solution still stands

QUOTE
I did not refute you on this. In fact, I agree with your position. "Best" for one is different from "best" for another.

I was only pointing out the tool by which one can determine which is "best" for him/her.
seems confuse and create a paradox.

you mean that someone really feel that "this" is best and using the "tool" for comparisons he find the "real" result.
if this "real" result is against his taste,what should he do?
he change his taste of what is best or trash the scientific result?

what the scientific result is prooving for his taste about what is "best/better"?



edit
typos
kjoonlee
QUOTE(hellokeith @ Sep 4 2006, 05:20) *

I am fairly sure that out of 100 people, most (if not all) would say a 44/16/stereo wav file of a contemporary/popular/easily-recognizable song does sound "better" than the same file at 11/8/mono.

But how is this established?

How is it that we know (or at least think we know) that in our car, music played on FM does sound signficantly better than music played on AM?

How is it that I can hear a song for the first time and make a subjective decision that 1) the playback equipment sounds bad or 2) the recording/mastering quality of the song is bad or 3) both?

44kHz recordings are more likely to be transparent (indistinguishable from live music).
16bit recordings are more likely to be transparent as well.
Stereo is nice because people have two ears.

You can compare it subjectively with live/uncompressed music; if you notice something unnatural in one method but not in another, then you can establish what's better.
pepoluan
QUOTE(jlt @ Sep 9 2006, 11:29) *
seems confuse and create a paradox.

you mean that someone really feel that "this" is best and using the "tool" for comparisons he find the "real" result.
if this "real" result is against his taste,what should he do?
he change his taste of what is best or trash the scientific result?

what the scientific result is prooving for his taste about what is "best/better"?
What paradox?

I'm sorry to say this, but you seem to have not the slightest grasp of what ABC/HR is.

It is not an automated tool to tell you which one is better.

It is simply a comparator & scorer tool. It will present both audio files, you play them manually, and you score them manually, fully according to your taste. If you find the left one is better, give it a higher score. And vice versa.

It should be easy to deduce, based on how this tool works, that whichever scores better means that that one is better than the other, according to your liking

When the test concludes, ABC/HR will then 'uncloak' what the left and the right samples are. You just compare your score, and realize, "Oh, L has better score than R. That means, L is better for me than R."

Simple, no?

The beauty of ABC/HR is the blind scoring method it imposes; there is no such thing as codec bias etc.

For instance, you may be (unconsciously) preferring codec X instead of Z, because in your mind (or based on hearsay) Z introduces more white noise than X, X only having slight pre-echo. So you set out to prove it with ABC/HR.

As you run the ABC/HR session, you evidenly found out that L is more enjoyable to you than R. You can't really pin down the reason why, but you definitely like L better than R. So you conclude the test. Surprise, surprise, L is Z, R is X.

Based on ABC/HR you definitely prefer L than R. When the hidden samples are revealed, it becomes evident that you definitely prefer Z than X. To your ear, when tested blindly, Z sounds better than X.

ABC/HR is a great tool to dispel all codec bias.
jlt
QUOTE
It is not an automated tool ..
yes,i know.

QUOTE
Simple, no?
after your explanations.....yes! wink.gif

QUOTE
What paradox?
mine.
my questions was not only about ABC/HR but for the whole topic.

QUOTE
fully according to your taste
ok but taste is personal(individual)!
is easy to deduce but don't means that what is "better" is the "best" for my(our) taste.
as "best" is individual taste and "better" is the scientific result,we trash the scientific result or unsure.gif change the mind as(if) "best/better" are differents?
worse: if we change the mind means that we don't have personal taste...(sorry,my poor english can't explain better than this)

ps:thanks for your complete and clever explantions about ABC/HR pepoluan! smile.gif

...but the "paradox" arise again(if you understood what i mean)
pepoluan
Ahhh, okay jlt, I got what you meant smile.gif

Well, yes, there's a problem of my individual taste against the public conception/taste.

In that kind of conflict, there's no other way but to gather a statistics, i.e. we compare Z and X (as in my example above) across, say, the 30'000 member of HA. We gather the scores for Z and X, and do some statistical calculations (I'm not really good at statistics, so please forgive me) for standard deviation, median, etc. etc. etc.

... of course the best we can come up with is something 'diplomatic' wink.gif like:

QUOTE
According to our survey, there's 95% confidence in saying that more than 70% of HA members prefer Z over X (with a score of 4.5 against 3.2)

Of course the same survey also implies that almost 30% of HA members (with 95% confidence) prefer X over Z ... huh.gif ... but at least the clear majority (i.e. > 66%) prefers Z over X.

It's a bit like clothing situation... my favorite day-to-day wear is black polo shirt over black denims with black hiking shoes... and I have 95% confidence that less than 5% of the general population prefer that kind of color combination... laugh.gif

Ahh, statistics. There's lies, there's damn lies, and there's statistics. (Paraphrased from Benjamin D'Israeli)
jlt
QUOTE
and I have 95% confidence that less than 5% of the general population prefer that kind of color combination...
ah!ah!ah!... lol
you're too much.

you wrote the word explaining the meaning of "paradox": is conflict! (lol,my poor english really bore me because now came one new paradox(or conflict?):what is "better" and what is "best"?! conflict or paradox?! tongue.gif )

thanks pepoluan,your last 'diplomatic' post show how the 'statistics' works.
I'm not really good at statistics too because the "ghost of paradox"(or conflict?) back "alive" sometimes. lol

smile.gif
Hollunder
For me better works by a simply comparison, if something A is better than something B in all aspects it can be considered better than A.
This is unlikely to happen, so A could be better than B in some aspects, so A is better then B at [ some aspects] but can't be considered better as a whole.
So far it's theoretically possible to be objective.

Here we have some aspects where B is better then A, some where A is better than B or both are equal in some of these. We need to weigh those aspects to conclude which one is better, and this is of course subjective.


To best: I hate it.
It often implies absolutism that is seldom valid.
To make a valid 'best' statement you usually need to limit it in several ways.
So one can state that something is best at a given time, out of the [comparable things] one knows, and most likely only in his opinion.
If someone simply states that something is the best, he usually automatically implies at least those three limitations, which could make it a valid statement, the problem is that no one else knows what he automatically implied or not implied.
Uhm, that was more the communication-problem... anyway, I think one can't validly state that something is the best without limiting in several ways.
Woodinville
QUOTE(pepoluan @ Sep 8 2006, 16:40) *
I was only pointing out the tool by which one can determine which is "best" for him/her.


And, I'm pointing out that having a 'reference' in a pure preference test may be a mistake.

In general, in any test, you don't give the subject something will confuse them.
hellokeith
Lemme put it a different way:

I show all of you 3 photos. You have never before seen the 3 photos. Each photo is of the same scene, a high resolution image taken recently of the International Space Station with Earth below. The difference of the photos is that one is 16 colors, one is 256 colors, and one is 32 bit color.

I find it difficult to grasp there would be no concensus (if not absolute concensus) that the 32 bit color image is *best*. What I'm hypothesizing is that there is some genetic logic in all of us, which means preference is not necessary arbitrary nor infinitely diverse. Who doesn't like a cold cup of water or a nicely cooked potato?

I think people get too caught up in music taste to understand there are some common mechanics which *can* be stated as fact, like "all other things being equal, CD playback is better than FM radio broadcast".
Mike Giacomelli
QUOTE(gaillard @ Sep 4 2006, 10:50) *

QUOTE(cabbagerat @ Sep 4 2006, 05:17) *

QUOTE(gaillard @ Sep 3 2006, 18:29) *

but how much resolution? this is hard because its really the brain... not our ears as they are not digital. its like records. its analog, it can capture things so minut that digital can never match that, but if our brain has a limit how minut it can tell the difference between then it could.
Please stop trolling. If you really believe this then you need to do some background reading (maybe learn what the terms SNR and quantization noise mean, for a start) before you start a debate on it.
QUOTE
an anolog system of anything has infinite resolution compared to any non quantum computing. Take for example a needle on a pivot. One end of the needle is pushing on some deformable material, clay for instance. if you where to run that clay surface by the needle at a set rate and then wiggled the other end of the needle sporadically it would get a bunch of impressions. now in digital how do you store those impression levels? you would need an infinite number of values.
I'll give you the benefit of the doubt and assume that you are merely ignorant and not a troll.

Analog electronic systems do not have infinite resolution. Small changes in voltage are swamped by noise generated in all the conductors and semiconductors in the system. This noise is measured with a Signal to Noise Ratio (SNR) - when this is low enough the signal is effectively lost. Noise is the limiting factor in both analog and digital audio systems.

QUOTE
this is why calculus was invented. to be able to do math with infinite things. but you can not use calculus do things machanical that are infinite, because in a given amount of time how are you going to run an infinite number of calculations?
Eh? What exactly are you suggesting here?




Did i say anything about an electrical system? i am merely talking about how to store something that is analog, just how to represent it. I don't appreciate all the insults.


The issue here is that you are confusing the theory of an anlog signal (which cannot exist in reality) with actual analog signals, which are only rough approximatations of this ideal (much in the sense that PCM is an approximation of the ideal digital signal with infinate bit depth). True analog depends on having infinate levels to choose from. Basically it assumes that all of reality is continuous. Clearly this is not the case because air is made of molecules, energy comes in quanta, and electric fields have discrete electrons in them.

Actually both PCM and analog have the same fundimental limiation, and thats that neither can store infinately much information in a finite space due to those pesky laws of thermodynamics. The difference is in the approach they take to addressing this problem. Since PCM is generally more robust and easier to work with, its more popular these days, although the two are largely equivilent.

This limitation is even more fundimental then this however, It comes up in a lot of places. For instance, if analog signals like you're thinking of actually existed, we could build Hyper Turing Machines and use them to invalidate the Church Turing Thesis, which is one of the underling premises of computer science. Unfortunately, so far, it appears that our universe cannot accomidate such things.
jlt
QUOTE
I'm pointing out that having a 'reference' in a pure preference test may be a mistake.

how can have a mistake in what we prefer? scientific proove against the preference?
it takes me again to what i wrote about preference(taste) :
QUOTE
taste is personal(individual)!
is easy to deduce but don't means that what is "better" is the "best" for my(our) taste.
as "best" is individual taste and "better" is the scientific result,we trash the scientific result or change the mind as(if) "best/better" are differents?
worse: if we change the mind means that we don't have personal taste...

....really worse: what is the advantage of abandon(left) the preference to the scientific proove if they are differents?
i'm sure that we all choose for the preference.....right?

@ hellokeith

great post! wink.gif

@ Mike Giacomelli
i'm still reading your post again and again because
QUOTE
The issue here is that you are confusing the theory of an anlog signal (which cannot exist in reality)
but we can ...lol
what's the issue if we commit mistakes and confusions in what don't exist?
cabbagerat
QUOTE(Mike Giacomelli @ Sep 9 2006, 12:01) *

This limitation is even more fundimental then this however, It comes up in a lot of places. For instance, if analog signals like you're thinking of actually existed, we could build Hyper Turing Machines and use them to invalidate the Church Turing Thesis, which is one of the underling premises of computer science. Unfortunately, so far, it appears that our universe cannot accomidate such things.
This conjecture, which seems to be a good candidate for being a fundamental law of the universe, comes up in all sorts of strange places. Computability theory is extremely interesting - with profound implications for the future of technology. For example, were it not for relativity and it's effects, it would be possible to construct a mechanical oracle, which would calculate the exact value of the halting probability in finite time. This (or the analog hypercomputer you describe) would really mix things up in physics, computer science and mathematics.
KikeG
QUOTE(jlt @ Sep 9 2006, 22:01) *

QUOTE
I'm pointing out that having a 'reference' in a pure preference test may be a mistake.

how can have a mistake in what we prefer? scientific proove against the preference?


ABC/HR is not a test for just preference. Is for testing similarity to the reference, and just that. It can be a test for preference just in case your absolute goal and preference is always for fidelity to the reference, no matter the reference. But then we are establishing in advance what has to be considered better. In case of lossy codecs this is usually the goal, but this can't be said to be always the case. What if you test a DSP that enhances a poor sounding reference? A non-processed clip would sound exactly as the reference, so would get the highest rating in the test, but the processed clip would probably sound better to your ears, and should get a lower rating. But sounds better. So ABC/HR would just complicate things.

In a test for just personal preference, without any other considerations, you would just need two clips. You would listen to them, and then say which one you prefer. Without any conditions of what has to be considered better. Just your taste.
jlt
QUOTE
ABC/HR is not a test for just preference.Is for testing similarity to the reference, and just that. It can be a test for preference just in case your absolute goal and preference is always for fidelity to the reference, no matter the reference.
So ABC/HR would just complicate things.
humm..seems confuse but i understood.
ABC/HR can be used for tests if the preference is always for fidelity but would just complicate things.
ok.

QUOTE
In a test for just personal preference, without any other considerations, you would just need two clips. You would listen to them, and then say which one you prefer. Without any conditions of what has to be considered better. Just your taste.
all right,is clever.

my pleasure "meet you" in the forum,thanks for all clarifications.
smile.gif
pepoluan
QUOTE(jlt @ Sep 9 2006, 17:23) *
QUOTE
and I have 95% confidence that less than 5% of the general population prefer that kind of color combination...
ah!ah!ah!... lol
you're too much.

you wrote the word explaining the meaning of "paradox": is conflict! (lol,my poor english really bore me because now came one new paradox(or conflict?):what is "better" and what is "best"?! conflict or paradox?! tongue.gif )

thanks pepoluan,your last 'diplomatic' post show how the 'statistics' works.
I'm not really good at statistics too because the "ghost of paradox"(or conflict?) back "alive" sometimes. lol

smile.gif
Um, I know what the definition of "paradox" is; what I want to know in my posting was, which part of your statement does your use of "paradox" applies to?

Which is why I asked, "What paradox?" Or in a more formal wording: "You stated that there was I a paradox; I did not see where. Could you point out where is the paradox?"

QUOTE(Woodinville @ Sep 10 2006, 01:59) *
QUOTE(pepoluan @ Sep 8 2006, 16:40) *
I was only pointing out the tool by which one can determine which is "best" for him/her.
And, I'm pointing out that having a 'reference' in a pure preference test may be a mistake.
Gee, I see you are also ignorant of the "/HR" part. It stands for "Hidden Reference", which means that the reference is hidden, i.e. the listener has no visual cue whatsoever which one is called the reference; in this case, the ABC/HR tool will only perform as comparator and subjective scorer, allowing you to give score to both the "Left" sample and the "Right" sample (i.e. based on their location on the computer screen).

QUOTE(KikeG @ Sep 11 2006, 15:30) *
ABC/HR is not a test for just preference. Is for testing similarity to the reference, and just that.
That is what happens when you launch a full-fledged ABC/HR session with 1 reference sample and > 1 test samples.

When you use only 2 samples, assign one as the pseudo-reference, and the other as the test; when the test run, there will be only 2 columns, labeled "Left" and "Right". You have no way to determine, before the test ends, which one is which, unless you do a memory dump and analyze the internal variables of the tool.


How I wish people before posting will at least check out how the ABC/HR tool works; I am not debating the principle of the tool.


Put it this way: The ideal tool to punch a nail into a wood is arguably a hammer. But in lack of a hammer, I wouldn't hesitate to use a biggish, solid rock, despite the obvious difference in the functions of a hammer and a rock.
Mike Giacomelli
QUOTE(pepoluan @ Sep 11 2006, 07:08) *

QUOTE(KikeG @ Sep 11 2006, 15:30) *
ABC/HR is not a test for just preference. Is for testing similarity to the reference, and just that.
That is what happens when you launch a full-fledged ABC/HR session with 1 reference sample and > 1 test samples.

When you use only 2 samples, assign one as the pseudo-reference, and the other as the test; when the test run, there will be only 2 columns, labeled "Left" and "Right". You have no way to determine, before the test ends, which one is which, unless you do a memory dump and analyze the internal variables of the tool.


How I wish people before posting will at least check out how the ABC/HR tool works; I am not debating the principle of the tool.




I'm fairly certain you didn't read Kikeg's post. You seemed to repeat everything he said and then accuse him of not saying it.
KikeG
In case of just comparing two samples, both ABXY and ABC/HR could be employed to do a test for preference, though they were not designed for that precise task. You would have to avoid listening to A&B in ABXY or to the reference in ABC/HR.

In case of testing several clips ABC/HR would make things quite complicated. For that task, the old tool WavRate from Ivan could be handy. It's similar to ABC/HR but without needing to rate a hidden reference for every clip at test. It has a single reference that can be easily avoided.
pepoluan
QUOTE(Mike Giacomelli @ Sep 11 2006, 21:13) *
I'm fairly certain you didn't read Kikeg's post. You seemed to repeat everything he said and then accuse him of not saying it.
Did I do that?

Ah, sorry then, my bad. Apologies to you, KikeG.
gaillard
QUOTE(Mike Giacomelli @ Sep 9 2006, 16:01) *



The issue here is that you are confusing the theory of an anlog signal (which cannot exist in reality) with actual analog signals, which are only rough approximatations of this ideal (much in the sense that PCM is an approximation of the ideal digital signal with infinate bit depth). True analog depends on having infinate levels to choose from. Basically it assumes that all of reality is continuous. Clearly this is not the case because air is made of molecules, energy comes in quanta, and electric fields have discrete electrons in them.

Actually both PCM and analog have the same fundimental limiation, and thats that neither can store infinately much information in a finite space due to those pesky laws of thermodynamics. The difference is in the approach they take to addressing this problem. Since PCM is generally more robust and easier to work with, its more popular these days, although the two are largely equivilent.

This limitation is even more fundimental then this however, It comes up in a lot of places. For instance, if analog signals like you're thinking of actually existed, we could build Hyper Turing Machines and use them to invalidate the Church Turing Thesis, which is one of the underling premises of computer science. Unfortunately, so far, it appears that our universe cannot accomidate such things.



Which is smaller, molecules in the air of some wave or 16 bits of the same wave? Not to mention molecules are not just static points they change and influence their neighbors and such, how can one just ignore all of that?
pepoluan
QUOTE(gaillard @ Sep 11 2006, 22:56) *
QUOTE(Mike Giacomelli @ Sep 9 2006, 16:01) *
The issue here is that you are confusing the theory of an anlog signal (which cannot exist in reality) with actual analog signals, which are only rough approximatations of this ideal (much in the sense that PCM is an approximation of the ideal digital signal with infinate bit depth). True analog depends on having infinate levels to choose from. Basically it assumes that all of reality is continuous. Clearly this is not the case because air is made of molecules, energy comes in quanta, and electric fields have discrete electrons in them.
Which is smaller, molecules in the air of some wave or 16 bits of the same wave? Not to mention molecules are not just static points they change and influence their neighbors and such, how can one just ignore all of that?
Unfortunately all this pointless debate about analog waves finally have to pass the highly-variable neuro-path between the ear and the brain... which IIRC is a roughly-quantized sampler.
cabbagerat
QUOTE(gaillard @ Sep 11 2006, 07:56) *

Which is smaller, molecules in the air of some wave or 16 bits of the same wave? Not to mention molecules are not just static points they change and influence their neighbors and such, how can one just ignore all of that?
Huh? What are you trying to imply?

Bits don't really have a size (but there is a limit on how small they can be), and sound waves have length, but their physical size depends on the surface that radiated them and other factors. I don't really see the point of your argument.
jlt
@ pepoluan
blink.gif
take it easy tongue.gif
read with atention please:
QUOTE
you wrote the word explaining the meaning of "paradox": is conflict!

QUOTE
Um, I know what the definition of "paradox" is; what I want to know in my posting was, which part of your statement does your use of "paradox" applies to?
i wrote that paradox means conflict are equivalent words and i not accuse something in your post,i don't apply the word to your post but only about the meaning.
i was clear and i'm trying to be clear here again.
few more lines later i thank you...seems that you really misunderstood what i wrote:
QUOTE
thanks pepoluan,your last 'diplomatic' post show how the 'statistics' works


wait....now i need to read the remainder and stop to crying.gif cry before answer. (lol)

...
QUOTE
Gee, I see you are also ignorant of the "/HR" part.
for sure and not only about "/HR" part.
i don't know "everything about all".my mind is not blocked with false proud and remains opened to learn.
who think that "knows to much" or always have "wise feelings" have mind blocked...and can be sick.it's not my case.
i learn more doing questions than trying to show more than i know( hey,this last phrase can be used in rock or metal...repeat 3 times...)

..the remainder of your post is another lesson and i thank you again.
remeber,
forum is to change informations, (i)we are here to learn and when possible,take what we know as gift to the others.

regards.
Woodinville
QUOTE(gaillard @ Sep 11 2006, 08:56) *


Which is smaller, molecules in the air of some wave or 16 bits of the same wave? Not to mention molecules are not just static points they change and influence their neighbors and such, how can one just ignore all of that?


Perhaps you weren't watching, but it's easily shown that the atmospheric noise over the area of an eardrum is roughly a white noise level in the 20Hz to 20kHz range on the order of 6db SPL (standard psychoacoustic reference).

Now, what you mean by "16 bits of the same wave" is not clear to me, since you have not specified a relationship of quantizer level to reproduction level. Likewise, what are you talking about "are not just static points", we're talking measured noise level here. Isn't something like "measured noise level" the way to verify a calculation? (n.b. the calculation works out very nicely in the real world)

The level of atmospheric noise is not under a great deal of dispute among those who have looked at the mechanism and studied what comes out of a low-noise microphone. Have you some data to offer us here?

QUOTE(pepoluan @ Sep 11 2006, 09:01) *

Unfortunately all this pointless debate about analog waves finally have to pass the highly-variable neuro-path between the ear and the brain... which IIRC is a roughly-quantized sampler.



It would be better to call it a fairly rougly quantized pulse-position-modulation system with many parallel channels.

But your point does hold. Inside any given critical bandwidth at a given signal level, something on the order of 30dB SNR is about what you get.

QUOTE(pepoluan @ Sep 11 2006, 07:08) *
Gee, I see you are also ignorant of the "/HR" part. It stands for "Hidden Reference", which means that the reference is hidden, i.e. the listener has no visual cue whatsoever which one is called the reference; in this case, the ABC/HR tool will only perform as comparator and subjective scorer, allowing you to give score to both the "Left" sample and the "Right" sample (i.e. based on their location on the computer screen).


Let's you and I understand something, I am one of the people who DESIGNED the ABC/hr test.

A is a reference. It is always the reference. It is nothing but the reference. The fact that there is ALSO a hidden reference is irrelevant. The reference is always, and I repeat, always known to the listener. All the time, always. If it's not, then it's not an ABC/hr test.

You can not get away from the simple fact that A is always the reference. The reference is always available. In a pure test of preference this is wrong.

Consider:

I make a 24 bit signal that sounds good. Then I reduce the level by 10dB, and put a Q=20 peak 10dB high at 2kHz in this signal.

I make that the reference.

Now, I run a test with A being that, one of B or C being that, and the other one being a refiltered signal that has a Q=20 ZERO of -10dB directly over the peak, followed by a 10dB gain. In short, except for losing about 2 bits at the bottom end of the signal, we're back to the good-sounding signal.

We run this in ABC/hr. What happens? The FIXED signal, the one that everone would prefer, is now "very different" and will always score lower than the broken signal.

In other words, you do not test preference with ABC/hr.

It won't work right.

That's not what it's for.
pepoluan
There's something designed for something, yet it is usable for something else. Like my analogy of hammer vs stone...

Ah, but I digress. I'm tired of this pointless debate. rolleyes.gif

Anyways, to return to the crux of the matter: The only way to determine what's best between 2 alternatives... is to blindly compare & score them with whatever tool you have that does allow blind comparison & scoring, and accept the score as truth based on your subjective preference. If your ABC/HR tool allows you to not listen to the reference... then don't listen to the reference and listen to the L & R samples only. Discipline yourself to not listen to the reference.

There, that's my final statement.

Barring any personal attack wink.gif I don't think I have more to add here biggrin.gif

Stay cool, people smile.gif

/me = over & out.

wink.gif
ff123
QUOTE(Woodinville @ Sep 11 2006, 11:04) *


Let's you and I understand something, I am one of the people who DESIGNED the ABC/hr test.



I wonder if I've "talked" with you in another venue? Anyway, everyone seems to be saying the same thing here, that ABC/hr was not designed as a pure preference test.

The book I consult for subjective testing says that the duo-trio test, as they term it, is a difference, not a preference test, so strictly speaking the grading scale should not be there.

But it seems to work with the grading scale regardless.

ff123
Woodinville
QUOTE(pepoluan @ Sep 11 2006, 11:14) *
Discipline yourself to not listen to the reference.



Why not just use direct (double-blind) A/B comparison? Goodness!


QUOTE(ff123 @ Sep 11 2006, 11:58) *
The book I consult for subjective testing says that the duo-trio test, as they term it, is a difference, not a preference test, so strictly speaking the grading scale should not be there.

But it seems to work with the grading scale regardless.

ff123


Uh, but the grading scale is not a preference scale, it is a difference scale. While I don't have the reference in my hand, there are quite some work that shows that difference tends to be more stable than preference, including scaling the difference.

Consider the example of the 10dB bump/removed bump I gave above. With a difference scale, the subject has no conflict, it sounds different. Yes, its' better, so it won't be annoying, or maybe the user will decide that it is annoyingly different than the reference, but it IS different, and clearly so.

Consider now if we used an MOS scale for both. The listener would face a nasty dichotomy.
ff123
QUOTE(Woodinville @ Sep 11 2006, 12:05) *

QUOTE(pepoluan @ Sep 11 2006, 11:14) *
Discipline yourself to not listen to the reference.

Why not just use direct (double-blind) A/B comparison? Goodness!


Because the (free) software is not readily available, probably. There has been some demand though for a MUSHRA-style format, where the comparisons don't force you to distinguish from the reference individually. Unfortunately, I won't be the one to write any other software in my limited free time.

QUOTE

Uh, but the grading scale is not a preference scale, it is a difference scale. While I don't have the reference in my hand, there are quite some work that shows that difference tends to be more stable than preference, including scaling the difference.

Consider the example of the 10dB bump/removed bump I gave above. With a difference scale, the subject has no conflict, it sounds different. Yes, its' better, so it won't be annoying, or maybe the user will decide that it is annoyingly different than the reference, but it IS different, and clearly so.

Consider now if we used an MOS scale for both. The listener would face a nasty dichotomy.


I see your point. The text I consulted did not indicate that a grading scale could be used. I'll have to go back and see why that is. Could be a hole in what is otherwise a fairly comprehensive book.

ff123
BradPDX
There are likely only 2 scientific methods to use here, given the type of data involved.

One is to simply measure signals as in the "days of old" - e.g., analog hi-fi days - and determine which one is statistically closer to the original. This can be presented as criteria of % distortion, noise, phase, etc. For many years this was a chief marketing tool of audio manufacturers, arguing that an amplifier with 0.001% distortion was inherently "better" than one with 0.01% distortion.

While technically valid, this does say anything about the wildly sensitive, selective, non-linear "customer" for these products, the human ear. Ergo ABX or ABC/HR tests. These do not judge quality, but differences and quite possibly preferences of the "customer".

The ear/brain system is very fickle and varies widely from one person to another - the "hardware" may not matter as much as the "software", or ultimate perceiver of the signals in question. Here our science boils down to statistics and nothing more. If I test 100 individuals and find that there is no statistical difference they can perceive between 128kbps MP3s and FLACs of the same material, is 128k MP3 "just as good"? Yes, for these statistical purposes, it is. Does this mean that the participants have damaged hardware? Probably not. It is probably the software.

In this case, only tests from the first method will reliably show that one signal is "better" than the other.

As an engineer (and musician/producer), it is my belief that sound reproduction for music is hampered by many issues that are not at all addressed by these types of tests and measurements. It is something of an "elephant in the room" point of view, I suppose; I assert that the entire method of recording and delivering 2 channel audio is so flawed that "realism" is out of the realm of possibility for the great majority of recordings (surely a topic in itself!). I do not buy any arguments about expensive cables or amplifier "burn-in", I believe the problems are far from subtle and that esoteric playback systems are generally a waste of money thrown at the wrong area. There, now you know my bias blink.gif

Does this mean that listening the recorded music isn't enjoyable? Of course not - I am listening right now. But convinced that the performance is "real"? No way. From that point of view, the goal of various audio delivery systems in the current environment is to provide an aesthetically enjoyable experience, not a faithful one per se.

So the short answer: we can't establish "better/best" in most cases except 1) in extreme cases, and 2) statistically.
jlt
QUOTE
Does this mean that listening the recorded music isn't enjoyable? Of course not - I am listening right now. But convinced that the performance is "real"? No way. From that point of view, the goal of various audio delivery systems in the current environment is to provide an aesthetically enjoyable experience, not a faithful one per se.
BINGO!

QUOTE
So the short answer: we can't establish "better/best" in most cases except 1) in extreme cases, and 2) statistically.
perfect.
i'm not an engineer (and musician/producer) like you, i'm only technician/home musician and this is exactly what i was trying to post(but was affraid) and my crap english can't express the right words like yours.
you broke the "paradox" that i was "talking" (i think that you understand me after read my posts)

my best regards!
cool.gif
gaillard
wow guys, you can take some things and run with them thats for sure.

I am not talking of the eardrum or any darn electronics here. just simple (or not simple) pattern in nature verse it exactly REPRESENTED digitally. its not possible without infinite resolution (resolution has a broader definition too...)

thus my original example...
greynol
QUOTE(gaillard @ Sep 11 2006, 15:23) *

I am not talking of the eardrum or any darn electronics here. just simple (or not simple) pattern in nature verse it exactly REPRESENTED digitally. its not possible without infinite resolution (resolution has a broader definition too...)

Are you sure quantization of reality requires infinite resolution?

Since you're considering a broader definition of resolution, shouldn't you also consider your definition of reality?
hellokeith
QUOTE(BradPDX @ Sep 11 2006, 14:24) *
From that point of view, the goal of various audio delivery systems in the current environment is to provide an aesthetically enjoyable experience, not a faithful one per se.

So the short answer: we can't establish "better/best" in most cases except 1) in extreme cases, and 2) statistically.


Very nicely layed out post, Brad.

Regarding fidelity in reproduction, since very few musical pieces are performed by all members of a band/group simultaneously, there really isn't an analagous way to reproduce that. And some might even argue that "surround sound" or three dimensional sound is not realistic, since few audience members ever sit in the middle of the stage while the group is performing live.

What I would like to see, is a multichannel audio format which allows us to mix the musical components as we please, and save that mixing in the form of metadata. Then there could be whole new genre's by digital dj's, without the piracy concerns inherent in our current limited digital audio formats.
Woodinville
QUOTE(gaillard @ Sep 11 2006, 15:23) *
I am not talking of the eardrum or any darn electronics here. just simple (or not simple) pattern in nature verse it exactly REPRESENTED digitally. its not possible without infinite resolution (resolution has a broader definition too...)

thus my original example...


So, if you're not willing to consider the eardrum, you're not willing to accept the pattern in nature.

There is, regardless of your belief system, no such thing as "exact" representation in any kind of analog, be it sampled and quantized, or quasi-continuous in time and amplitude.

In the real world there is no exact.
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2008 Invision Power Services, Inc.