Multiformat Listening Test @ 128 kbps - OPEN Options

[naylor83]

Doesn't telling everyone where the artifacts are kinda defeat the point of a listening test?

Well, I had listened at the beginning and I couldn't hear a thing, so it certainly didn't help me

Ok. I'll stop asking.

[naylor83]

Doesn't telling everyone where the artifacts are kinda defeat the point of a listening test?

The whole procedure of a listening test kind of defeats itself if you think like this. Training yourself to hear artifacts also shouldn't be done then, because you then might recognize new artifacts you missed before, which would change the notation of the codecs. I think the important thing is whether you can hear artifacts are not. Without knowing what to listen for and preferably also in which parts of the music, I am pretty sure practically everybody would have to rate all codecs as 5.0 in this test.

Well, specifying the exact position and sample where an artifact is noticable is a bit unfair IMHO.

But I think sTisTi has a point all the same. If I can hear it, I can hear it. If I can't - as it turned out in this case - (I can't even single out the low anchor for Carbonelli...) then I can't. All the knowledge in the world probably couldn't help me find that artifact, and hence the sample to me is a 5.0.

[Sebastian Mares]

That is true, too.

[naylor83]

That is true, too.

And you have to remember that the people taking a test like this probably vary a lot when it comes to training and skill, so the results can never be seen as representative for say 'average listeners' or 'pedantic listeners' - it's most likely a mix of all kinds.

[rjamorim]

Jesus Christ!

Discussing artifacts ist verboten. Please stop it at once, people!

This post has been edited by rjamorim: Dec 6 2005, 23:07

[Sebastian Mares]

Edit: Roberto was faster.

This post has been edited by Sebastian Mares: Dec 6 2005, 23:12

[Sebastian Mares]

That is true, too.

And you have to remember that the people taking a test like this probably vary a lot when it comes to training and skill, so the results can never be seen as representative for say 'average listeners' or 'pedantic listeners' - it's most likely a mix of all kinds.

One thing I didn't consider - when person X says that sample Y has an artifact at postition 123, then person Z is going to keep testing and testing sample Y because he knows that it has an artifact.

[caligae]

One thing I didn't consider - when person X says that sample Y has an artifact at postition 123, then person Z is going to keep testing and testing sample Y because he knows that it has an artifact.

But this would only be unfair if that specific artifact is only spotable with encoder A, but encoder B has noticable artifacts at different positions. If this is not the case it would be an improvement for the test results IMHO -- because you get real ratings instead of all 5.0s.

[rjamorim]

If this is not the case it would be an improvement for the test results IMHO -- because you get real ratings instead of all 5.0s.

You also get artificial results, as people might listen again and again, paying much more attention than they normally would.

[eisa01]

My fear is that I'll only be able to figure out what the low anchor is

Have only tested two samples, and I've only managed to find one version that had artifacts.

This post has been edited by eisa01: Dec 7 2005, 01:54

[markanini]

So far I've done three samples and sure it's difficult to spot artefacts on other than the low anchor but it's not totally impossible. A quiet enviroment and adjusting the volume moderately loud for each sample helps. The hard parts is probably that you need to be in a focused state of mind and you'll have to listen to all the samples more than a few times to spot the artefacts.

Just a few observations I made. Not that I've done many listening teste before...

[Oliver]

My fear is that I'll only be able to figure out what the low anchor is

Hi eisa01,

I have the same fear! I plan to use the practice files, in the hope that they will improve my ability to hear the problems.

Oliver

Practice files http://ff123.net/64test/64kbsPractice.zip

[Dibrom]

It has already been requested twice that people stop discussing their experience of the listening test itself while the still is still being carried out.

Please respect that, otherwise it is likely that this thread will be closed so as to maintain test integrity.

Really, until the test is finished, the only things that should be discussed in this thread, in my opinion (Sebastian can add something here if he doesn't agree), are metaissues such as discussion about obtaining or distributing the files, announcing the test at other places, or discussion about problems with the abc/hr program.

If you can't hear any differences, fine. That is just as useful of a result for the purposes of the test. Please don't ask other people to tell you if they hear differences or to describe their testing experience.

By the way, if Sebastian requests it, I will remove previous posts discussing any of this from the current thread.

[Shade[ST]]

I found an issue with the delivered ABC/HR program :

DISCLAIMER - DO NOT READ THIS IF YOU ARE PERFORMING THE OPEN TEST - IT MAY INFLUENCE YOUR RESULTS.

PM me for details.

This post has been edited by Shade[ST]: Dec 7 2005, 19:43

[ErikS]

I found an issue with the delivered ABC/HR program :
[...]hear a click[...]

Make sure you're using JRE v1.5. If you haven't upgraded, clicks in the sound is an issue which is already known. Perhaps there should have been a warning or something when abc/hr falls back on older JRE... well well...

[Shade[ST]]

I am using the latest JRE. I reformatted two days ago.

[naylor83]

I am using the latest JRE. I reformatted two days ago.

Me too.

[naylor83]

If this is not the case it would be an improvement for the test results IMHO -- because you get real ratings instead of all 5.0s.

You also get artificial results, as people might listen again and again, paying much more attention than they normally would.

But then again, that is true for any listening tests. Or do you suggest we just listen to every sample once, as if we were only 'enjoying the music', and then rate everything at 5.0?

However, I will respect Sebastians request not to ask/answer questions like that. And nobody has gone against him since he stated his opinion on the matter.

[Sebastian Mares]

If this is not the case it would be an improvement for the test results IMHO -- because you get real ratings instead of all 5.0s.

You also get artificial results, as people might listen again and again, paying much more attention than they normally would.

But then again, that is true for any listening tests. Or do you suggest we just listen to every sample once, as if we were only 'enjoying the music', and then rate everything at 5.0?

However, I will respect Sebastians request not to ask/answer questions like that. And nobody has gone against him since he stated his opinion on the matter.

No, it's OK to re-listen to the song as many times as you want.
However, when someone says that sample XYZ has an artifact, you are not going to let go until you spotted it because you know there is an artifact there.

[Lyx]

But then again, that is true for any listening tests. Or do you suggest we just listen to every sample once, as if we were only 'enjoying the music', and then rate everything at 5.0?

This is not a black/white issue. Some "artificial" training is necessary - but the line has to be drawn somewhere. And you now know where it is.

- Lyx

[plonk420]

RareWares seems to be sending .TORRENT files as wrong MIME type. browser normally opens them to download or launch (and trust me, i use a LOT of torrent sites ;). RareWares causes my firefox to display them as text files...

[SpacemanSpiff0x0]

Holy crap, I just finished the test and I must say, I am really impressed by the quality of these samples. It's phenominal.

I have one question and I will not in any way be aggrevated if it is not answered (for some test reason).

The question I have is, what were the specific settings for the Nero encoded files. Was it one of the default LC profiles. Perhaps VBR/Stereo - Transparent [110-150]? Or was it a custom setting?

Thanks.

[Shade[ST]]

I have one question and I will not in any way be aggrevated if it is not answered (for some test reason).

You can read the whole discussion about the choice of samples in the pre-test thread. just search for it.

[caligae]

I found an issue with the delivered ABC/HR program :

I don't think you can avoid this (except every listener being supervised by some authority during the test). Even if you use strong encryption, the sound has to go to the soundcard unencrypted sooner or later. It's just like with DRM.

Yesterday I could easily abx a file without even the sound turned on within a few seconds.

So we just have to trust that no one wants to ruin the test by sending in manipulated results. Since the encoders and settings are also available, it would even be possible to favor/bash a specific codec.

[Shade[ST]]

I don't think you can avoid this (except every listener being supervised by some authority during the test). Even if you use strong encryption, the sound has to go to the soundcard unencrypted sooner or later. It's just like with DRM.

It would be possible to implement a disk cache flushing mechanism that would make sure that both accesses take the same time. Or, on opposite levels, a cache mechanism that would make sure that no delays are heard when playing back sound. A sort of safety, like, don't start playing until all files are fully cached (at least the beginning of them, maybe 1-2 seconds, praps)