Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Very high bitrate listening test (~320-400kbps) (Read 29570 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Very high bitrate listening test (~320-400kbps)

Reply #25 –
Waste of time, effort and resources.

Quote from: memomai on
I'd like to compare them in encoding speed


You don't need a listening test to do that. Just do some encoding batches and post the timings.

Quote
problem samples


On most of the tested configurations and encoders, they will be a handful, if that much. I can only think of maybe Kalifornia and some of that horrid noise Dibrom listens to.

Quote
artefacts


-> 0

Quote
and transcoding efficiency


sthayashi started a transcoding listening test in late 2003. He's still waiting for results.

Very high bitrate listening test (~320-400kbps)

Reply #26 –
Quote from: Porcupine on
As I wrote in my previous post, there are still rare samples that are quite untransparent at 320 kbps and even higher, with all these encoders. It is just that they are rare, you must search for a specific type of signal..

This, and what you said previously proves that a listening test (which compares the performance of different codecs) is not useful at these bit rates. Problem samples are only useful for tuning a codec (including settings). Because every codec has it own problem samples, you can not sensibly rate the codecs against each other with typical problem samples only.

What you propose would be more like a problem sample result list. A list with notes how the codecs handle them (at a certain bit rate).

edit: after finishing this post it turned out to be basically what 2Bdecided already said
In theory, there is no difference between theory and practice. In practice there is.
 

Very high bitrate listening test (~320-400kbps)

Reply #27 –
Don't forget that any difference between codecs up in that range, even on problem samples, is bound to be very tiny. In fact, I wouldn't be be surprised if most of the rankings felll between a 4.9 and 5. Thus, your effect sizes would be so small as to be statistically insignificant, even when using a p value of .05....and you'd need a very high N to even approach a barely-acceptable degree of power.

In short, I doubt you'd find anything with any certainty, especially considering how small the differences between codecs have been found to be at 128kbps (let alone 300-400kbps!).

Very high bitrate listening test (~320-400kbps)

Reply #28 –
It's all explained in my canned response, really.

Quote
Why a listening test at high bitrates (192kbps) wouldn't work:

1) Most samples would have already reached transparency at that bitrate.
And choosing only problem samples would make the test less significant
since you wouldn't be testing a broad range of musical styles.

2) Only a handful of golden ears would be able to reliably ABX the
samples, and even after ABXing they would hardly give scores lower
than 4.5

3) Since all scores would be around 4.5, the error margins would be big
enough to make all codecs tied. To avoid that, you would need to have
hundreds of participants, in an attempt to bring the error margins down.

4) You would have a hard time finding hundreds of golden ears, and even
more, hundreds of golden ears willing to participate, because the test
would be very fatiguing and frustrating, due to the difficulty of ABXing.

5) At the end, even if you managed to bring the error margins down, the
codecs would be ranked so close that you wouldn't be able to produce a
decent conclusion. All codecs would seem (or be) tied to each other.


Memomai claims the test is not meant to test transparency, but at the very first post, he says he wants to "compare them in <...> artifacts", which amounts to testing who has more artifacts and who has less - that is, who is closer to transparency.

Very high bitrate listening test (~320-400kbps)

Reply #29 –
memomai, eig.wv is floating around on these hydrogenaudio boards somewhere. I just did a quick google search and a sample turned up for download. That's not where I got my sample from but it's probably the same sample.

Very high bitrate listening test (~320-400kbps)

Reply #30 –
And I thought the definition of transparency means "MOST people cannot tell the difference if it's the original or not".

Porcupine, I'll look for it, thanks!

I think I'll do the test on my own like I'm thinking about how to do it if nobody here can give me a useful advice. I don't know if the results will be released on HA, cause it might be a violation against a rule on HA.
FB2K,APE&LAME

Very high bitrate listening test (~320-400kbps)

Reply #31 –
A collection of known bad samples for various codecs at extremely high bitrate (>= 300 kbps) as was proposed would be welcome I think and would be a help for chosing a codec for those people who are willing to use very high bitrate.

It will be hard to draw conclusions from this collection cause samples may be rare to non-existing for specific codecs - be it due to the fact of lacking public experience.
But anyway it's better than having experience scattered around, and maybe some new experience will come up.
lame3995o -Q1.7 --lowpass 17

Very high bitrate listening test (~320-400kbps)

Reply #32 –
To be fair, ff123 still hosts a collection of samples IIRC, and HA used to host one (years ago). Whatever happened to that?

There is/was a very out of date one on the lame website too.

Cheers,
David.

Very high bitrate listening test (~320-400kbps)

Reply #33 –
Quote from: memomai on
And I thought the definition of transparency means "MOST people cannot tell the difference if it's the original or not".


Transparency is in the ears of the listener. Most songs are transparent for me with Lame around 128 (and probably lower), but the same doesn't apply to Guruboolez.

It doesn't make much sense to generalize transparency to a group of people because it is case-by-case dependant.

Very high bitrate listening test (~320-400kbps)

Reply #34 –
.
Out of curiosity: the other day I posted both 320 Kb/s and .WAV files on my blog:

www.cdnav.com

(Oct 13th 2007 post)

I was absolutely unable to discern any differences...

[Much] better yet: grab a free copy of Audacity and Foobar 2000 and rip your own favorite CD into the above formats - AXB your hearts out!

Andrew D.


.

Very high bitrate listening test (~320-400kbps)

Reply #35 –
Does anyone 'out there' know of a [statistically valid / properly conducted and documented], double-blind listening evaluation that attempts to discern an audibly distinctive difference between 320CBR MP3, OOG or what have you, and 'classic' Redbook CD?

I find a load of em' using Google and the Hydrogen Audio search engine, at every bitrate - except - 320CBR.  I know, I know, many will yell at me that there is no difference, so why do it... however, just work with me here... have you ever participated in, or read about just such an evaluation at - specifically - 320CBR?

Appreciate your time,

Andrew D.

(PS: I've done several such evaluations... I cannot tell one apart from the other so far).

www.cdnav.com

Very high bitrate listening test (~320-400kbps)

Reply #36 –
A listening test above 320kbit would be interesting, but not for normal listening cases. Such a test should more specialize on transcoding and possible other postprocessing. Guru once did such a test for himself.... there were some quite interesting results..... on the other hand, it may very well be the case that only guru and a handful of other people are capable to do something like that - even with transcoding - so statistical relevance over a larger audience would be nonexistent as well again..... meh, okay i as well chime in on the punchline "it wouldn't work".
I am arrogant and I can afford it because I deliver.