My friend and I are doing a science fair project on lossy compression. (we are high schoolers if you care) The purpose of this project is to show people that 128kbps is not cd quality despite what the media tells them. This test is concieved as an update to rjamorim's extension test using newer versions of the nero aac and ogg encoders. (We'd like to thank rjamorim for providing a link to the test samples and for the inspiration )

We are more interested in the abx results of your tests than on the 5 point rating that each format is given because it is less discretionary, (there either is a difference or isn't) but if someone more knowledgeable could provide a little bit more info about why most testers seem to prefer the 5 point scale we could change it.

We are running a parallel 64 kbps test among people at our school, but because they are not audiophiles and can barely notice a difference at this setting we do not feel they contribute meaningfull to the 128kbps test.

We have already prepared all of the files (encoded and then decoded to wav) and test files for use in ABC-HR. To take the test go to Music.Black2d.com. Once we have finished collating the data all results will be available on the site.

Thank you in advance for any contributions you can provide.

Look here:
http://www.hydrogenaudio.org/show.php/showtopic/17563/st/0
There is an mp3 listening test at 128kbs. Maybe you and your friend could join?

1.- That test ended monday at 00:00 hrs

2.- soltari1 wants to test different encoders (you know, different formats) against a source reference. He is not trying to test encoders directly against themselves to pick a winner.

Such a test would be very exhausting, IMHO, because you would have to use the same people to test every encoder, so the test would be more valid to prove the point you seem to want to make: that no encoder is "CD-quality" (transparent) at that bitrate. And I venture to say that some samples WILL be transparent to some people, so you would need to use more than one sample to prove your point. Have you found people to sit through 10-12 tests? (5-6 encoders, 1-2 samples) If you have, please BY ALL MEANS go ahead, and don't forget to let us know your results.

That is why Roberto's system is better (and even that one seems to be exhausting, from what I have read)

Just a note to say that you probably won't get too many testers once they find out the download is 220 MB!

ff123

BTW, The 5 point scale is used to answer the questions: "How do the codecs compare against each other and how do they compare against the reference?" If you get enough people and they're sensitive enough, you might be able to statistically show for a given group of people, both that 1) the codecs are rated significantly different from the reference, and 2) the codecs can be distinguished from each other (one is better than another).

So the 5 point scale can yield more information than just the ABX method. However, in order for the statistics to work, you require quite a few people, and they need to be sensitive to artifacting as well. That's a tall order at 128 kbit/s.

ABX is simpler because there's only one question to answer: "Does a given codec sound different from the reference?" There's no rating involved and even one person can yield statistically significant results.

If it would help we could offer just the test samples and the encoder settings and versions we used to make the files. I think that we will prally stick to reporting mainly the abx test results because we do not have enough people for a statistically accurate 5 point rating. If we get more testers we could change that. If it is too exhausting to do all the samples we would appreciate it if you do one or two. Obviously we'd prefer more, but any results help.