IPB

Welcome Guest ( Log In | Register )

8 Pages V  « < 3 4 5 6 7 > »   
Closed TopicStart new topic
Public, Multiformat Listening Test @ 64 kbps - OPEN
guruboolez
post Aug 7 2007, 12:25
Post #101





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



To illustrate the debate about noise and clicks I uploaded a new harpsichord sample (recorded in the 60') with much more mechanical clickety-clack :
http://www.megaupload.com/?d=WEJXUM18

I also have a fresh and expensive opera live recording with several stage noise (feet or object on the parquet floor) which were so close to ripping artefacts that I assimilated them first to mastering issues. I also have an excellent opera in Hybrid-SACD with weird pops on one track I can't explain. And to finish, a new CD re-issue (licensed by a label to a second one) with a ripping click noise: the CD is new, pressed but with a flaw typical of amateur ripping...


I don't think we should include samples that weren't correctly ripped unless with explicitely want to test how will react different encoders to a common issue. Better use a good rip of interesting samples rather than poor ones. But those clicks are far to be unsignificant: millions peoples have the same ripping artifacts on their library and the exemples I listed above are showing that even EAC-maniac can have the same or similar ones.
Go to the top of the page
+Quote Post
muaddib
post Aug 8 2007, 12:54
Post #102





Group: Developer
Posts: 398
Joined: 14-October 01
Member No.: 289



12.8. is near... How is the test going? How many results are expected?
Go to the top of the page
+Quote Post
naylor83
post Aug 8 2007, 15:35
Post #103





Group: Members
Posts: 204
Joined: 19-June 05
From: Uppsala, Sweden
Member No.: 22842



Will it be possible to get the key file for the result files when the test has finished? It would be interesting to compare your own results with the averages.


--------------------
davidnaylor.org

Vorbis Q4, please. AoTuv b5, preferably.
Go to the top of the page
+Quote Post
Alex B
post Aug 8 2007, 16:14
Post #104





Group: Members
Posts: 1303
Joined: 14-September 05
From: Helsinki, Finland
Member No.: 24472



QUOTE (naylor83 @ Aug 8 2007, 17:35) *
Will it be possible to get the key file for the result files when the test has finished? It would be interesting to compare your own results with the averages.

Normally it has been published after the test has finished. Also, the complete test data must be publicly available for anyone who wants to verify the results or further analyze the data.

BTW, I mailed my result files today.


--------------------
http://listening-tests.freetzi.com
Go to the top of the page
+Quote Post
Sebastian Mares
post Aug 8 2007, 18:16
Post #105





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (naylor83 @ Aug 8 2007, 16:35) *
Will it be possible to get the key file for the result files when the test has finished?


Absolutely, yes! smile.gif

I received both results, BTW. So far, I haven't been able to sort the results, but I hope to do so tomorrow. After that, I can also tell you which samples have a higher priority. What I can tell is that I have 22 mails in my inbox. biggrin.gif


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
naylor83
post Aug 8 2007, 19:18
Post #106





Group: Members
Posts: 204
Joined: 19-June 05
From: Uppsala, Sweden
Member No.: 22842



It will be very interesting to see the results. I felt I was finding more serious artifacts in the lower-numbered samples, ~1-5. Will be interesting to see if other people were doing the same.

This post has been edited by naylor83: Aug 8 2007, 19:21


--------------------
davidnaylor.org

Vorbis Q4, please. AoTuv b5, preferably.
Go to the top of the page
+Quote Post
Sebastian Mares
post Aug 9 2007, 23:32
Post #107





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



Sample 8 is the only one with less than 10 results. For the other samples, here is the number of (valid) results so far:

Sample 01: 18
Sample 02: 18
Sample 03: 16
Sample 04: 17
Sample 05: 15
Sample 06: 14
Sample 07: 11
Sample 08: 9
Sample 09: 12
Sample 10: 10
Sample 11: 11
Sample 12: 11
Sample 13: 13
Sample 14: 11
Sample 15: 10
Sample 16: 10
Sample 17: 12
Sample 18: 11

Unfortunately, there were several invalid results with ranked references which I am going to ignore.


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
naylor83
post Aug 10 2007, 00:02
Post #108





Group: Members
Posts: 204
Joined: 19-June 05
From: Uppsala, Sweden
Member No.: 22842



QUOTE (Sebastian Mares @ Aug 10 2007, 00:32) *
Unfortunately, there were several invalid results with ranked references which I am going to ignore.


Oops. Hope none where mine smile.gif


--------------------
davidnaylor.org

Vorbis Q4, please. AoTuv b5, preferably.
Go to the top of the page
+Quote Post
Whelkman
post Aug 10 2007, 00:30
Post #109





Group: Members
Posts: 91
Joined: 22-January 02
Member No.: 1111



QUOTE (Sebastian Mares @ Aug 9 2007, 18:32) *
Unfortunately, there were several invalid results with ranked references which I am going to ignore.

Can you describe invalidities? I read ABC/HR's documentation, but it is out of date. I got through five rounds of samples or so before I understood the behavior of the ABX "Testing Mode" but went back and corrected or reran previous trials (good thing I wrote notes for each sample). I should be okay now, but one never knows.

EDIT: I see that I've downloaded the Java version yet documentation points to the Win32 version, so it's not out of date as much as mildly incompatible.

This post has been edited by Whelkman: Aug 10 2007, 01:51
Go to the top of the page
+Quote Post
Sebastian Mares
post Aug 10 2007, 08:31
Post #110





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (Whelkman @ Aug 10 2007, 01:30) *
Can you describe invalidities?


As I said, results with ranked references.


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
muaddib
post Aug 10 2007, 09:45
Post #111





Group: Developer
Posts: 398
Joined: 14-October 01
Member No.: 289



QUOTE (Whelkman @ Aug 10 2007, 01:30) *
QUOTE (Sebastian Mares @ Aug 9 2007, 18:32) *
Unfortunately, there were several invalid results with ranked references which I am going to ignore.
Can you describe invalidities? I read ABC/HR's documentation, but it is out of date. I got through five rounds of samples or so before I understood the behavior of the ABX "Testing Mode" but went back and corrected or reran previous trials (good thing I wrote notes for each sample). I should be okay now, but one never knows.EDIT: I see that I've downloaded the Java version yet documentation points to the Win32 version, so it's not out of date as much as mildly incompatible.

Whenever you are not sure whether you can hear differences or not, you should do ABX in testing mode. When you do enough correct trials (5 out of 5, or 7 out of 8) then reference is grayed out and you can only move slider for coded sample (you can only rank coded sample). Before doing ABX in testing mode it is advisable first to do it in training mode until you are sure that you can hear differences.

IMO whenever you can not successfully do ABX in testing mode, you should give 5 to an encoder.

And one more thing. Come on people please do a bit more on samples 7-18! smile.gif At least sample 7 is very easy even for high anchor.
Go to the top of the page
+Quote Post
Whelkman
post Aug 10 2007, 19:25
Post #112





Group: Members
Posts: 91
Joined: 22-January 02
Member No.: 1111



QUOTE (muaddib @ Aug 10 2007, 04:45) *
Whenever you are not sure whether you can hear differences or not, you should do ABX in testing mode. When you do enough correct trials (5 out of 5, or 7 out of 8) then reference is grayed out and you can only move slider for coded sample (you can only rank coded sample). Before doing ABX in testing mode it is advisable first to do it in training mode until you are sure that you can hear differences.

IMO whenever you can not successfully do ABX in testing mode, you should give 5 to an encoder.

Great summary. This information would be helpful to newcomers were it included somewhere in the test summary or linked elsewhere.
Go to the top of the page
+Quote Post
beto
post Aug 10 2007, 19:26
Post #113





Group: Members (Donating)
Posts: 713
Joined: 8-July 04
From: Sao Paulo
Member No.: 15173



QUOTE (muaddib @ Aug 10 2007, 05:45) *
IMO whenever you can not successfully do ABX in testing mode, you should give 5 to an encoder.


That is precisely what I do. I have no problem in giving 5 to an encode. If I cannot spot any obvious difference after listening to the sample two or three times it gets a 5 rating. wink.gif


--------------------
http://volutabro.blogspot.com
Go to the top of the page
+Quote Post
naylor83
post Aug 10 2007, 21:23
Post #114





Group: Members
Posts: 204
Joined: 19-June 05
From: Uppsala, Sweden
Member No.: 22842



QUOTE (beto @ Aug 10 2007, 20:26) *
That is precisely what I do. I have no problem in giving 5 to an encode. If I cannot spot any obvious difference after listening to the sample two or three times it gets a 5 rating. wink.gif


I try a bit harder than that, because I like to know that I have all the detail in the music that I can possibly hear when I dish out a big 5.0.


--------------------
davidnaylor.org

Vorbis Q4, please. AoTuv b5, preferably.
Go to the top of the page
+Quote Post
woody_woodward
post Aug 11 2007, 00:18
Post #115





Group: Members
Posts: 348
Joined: 21-August 02
Member No.: 3138



Editorial comment:
Your testing procedure is just too complicated for a senior citizen old codger like me. Too many installs, too many directories, to many apps... Too much.... If anyone comes up with a simple web based user interface count me in....


Note to moderators: If my comment seems overly harsh just delete it.

Woody
Go to the top of the page
+Quote Post
Martin F.
post Aug 11 2007, 00:37
Post #116





Group: Members
Posts: 125
Joined: 15-July 06
From: Germany
Member No.: 32930



I didnít realize there were 18 samples Ö At first I only found sample 18 from the torrent mentioned in post #6, later I found the other samples but thought they were from earlier listening tests blush.gif


--------------------
FLAC.
Go to the top of the page
+Quote Post
Sebastian Mares
post Aug 11 2007, 00:40
Post #117





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



Well, all you need to do is simply download the ABC/HR ZIP file provided on the test page, unpack it to a folder of choice, then download the samples you want to test, unpack them in the ABC/HR folder, run the corresponding DecodeXX.bat files from the "bin" directory and that's it. All you need to do then is to start ABC/HR, load the corresponding "*.ecf" file, test, save as "*.erf" and send me the file. That's not too complicated. wink.gif

QUOTE (Martin F. @ Aug 11 2007, 01:37) *
I didnít realize there were 18 samples Ö At first I only found sample 18 from the torrent mentioned in post #6, later I found the other samples but thought they were from earlier listening tests blush.gif


Well, both the TXT and HTML readmes list all 18 samples - both as torrent and HTTP downloads.


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
Martin F.
post Aug 11 2007, 00:45
Post #118





Group: Members
Posts: 125
Joined: 15-July 06
From: Germany
Member No.: 32930



QUOTE (Sebastian Mares @ Aug 11 2007, 01:40) *
Well, both the TXT and HTML readmes list all 18 samples - both as torrent and HTTP downloads.


I was a bit confused … the readme says "Place the sample package ("SampleXX.zip") in the same folder as ABC/HR and uncompress it." I thought #18 was the "latest version" biggrin.gif

This post has been edited by Martin F.: Aug 11 2007, 00:47


--------------------
FLAC.
Go to the top of the page
+Quote Post
ff123
post Aug 11 2007, 01:09
Post #119


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



QUOTE (Sebastian Mares @ Aug 10 2007, 16:40) *
Well, all you need to do is simply download the ABC/HR ZIP file provided on the test page, unpack it to a folder of choice, then download the samples you want to test, unpack them in the ABC/HR folder, run the corresponding DecodeXX.bat files from the "bin" directory and that's it. All you need to do then is to start ABC/HR, load the corresponding "*.ecf" file, test, save as "*.erf" and send me the file. That's not too complicated. wink.gif

QUOTE (Martin F. @ Aug 11 2007, 01:37) *

I didn’t realize there were 18 samples … At first I only found sample 18 from the torrent mentioned in post #6, later I found the other samples but thought they were from earlier listening tests blush.gif


Well, both the TXT and HTML readmes list all 18 samples - both as torrent and HTTP downloads.


It still takes a bit of work to get started on these tests, although it's not difficult. I can think of other ways to do this. For example: have the java app start from a click on a webpage. The application could have another dialog box specifically made for such tests which lists the samples the test administrator has selected, has buttons for downloading the compressed files for each sample (either by torrent or http), and has another button for starting the test once the download has completed (the app would automatically take care of decompressing/decoding everything). The only thing you would need to specify is where to download the compressed file to, and where to load it from when you want to start/continue the test. Instead of emailing the .erf file, you could choose to submit it directly by pressing a button (perhaps with IP logging to identify possible multiple submissions from the same person).

The only downside to this is people may not like the abchr application to connect to the internet, because then there are always questions about what it does when it connects.
Go to the top of the page
+Quote Post
Whelkman
post Aug 11 2007, 03:18
Post #120





Group: Members
Posts: 91
Joined: 22-January 02
Member No.: 1111



QUOTE (ff123 @ Aug 10 2007, 20:09) *
The only downside to this is people may not like the abchr application to connect to the internet, because then there are always questions about what it does when it connects.

The people who care about their the nature of their Internet traffic probably aren't the same who want the most convenient experience possible. I assume the application would still work manually for those inclined to run it thusly.
Go to the top of the page
+Quote Post
kwanbis
post Aug 11 2007, 04:40
Post #121





Group: Developer (Donating)
Posts: 2353
Joined: 28-June 02
From: Argentina
Member No.: 2425



why can't an installer with ALL ready be made? (not that i care, but for others like the prev poster)


--------------------
MAREO: http://www.webearce.com.ar
Go to the top of the page
+Quote Post
Sebastian Mares
post Aug 11 2007, 08:44
Post #122





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



Someone would have to write an installer in Java or something that runs on Windows, Linux and Mac.


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
kdo
post Aug 11 2007, 09:15
Post #123





Group: Members (Donating)
Posts: 304
Joined: 18-April 02
From: Russia
Member No.: 1812



I've got two comments about the current abc/hr java version:

1) in ABX mode there are keyboard shortcuts Q,W,E,D to play/stop samples -- this is VERY helpful.
However, in the ABC/HR mode, the same shortcuts are available but which sample do they refer to? That is, when I press W or E - which sample of the 5 it is going to play? Had to click the buttons with the mouse to make sure it's playing the ones I wanted.

2) on my laptop the playback via the java sound engine skips every so often. Setting high priority in the task manager (windows2000) doesn't help.
Go to the top of the page
+Quote Post
Sebastian Mares
post Aug 11 2007, 09:47
Post #124





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



Well, I cannot help in those cases since I didn't write the program, sorry. Also, there is no better thing we can use that is also available for multiple platforms.

Anyways, today is the last day you can test and submit results. I will close the test tomorrow at around 12 PM German / Central European Time (GMT +1 with daylight saving time). Once the test is closed, no results will be accepted, so please don't wait until the very last minute.

Important! If you decide to start testing now, please focus on samples 7+ since they are the ones with the least results. This only applies to people who start testing now - if you already tested samples 1 to 6 and your results are pending, submit them, of course. Again, if you start testing now, please do so for samples 7 and higher - do not start testing now while choosing samples 1 to 6!

This post has been edited by Sebastian Mares: Aug 11 2007, 09:50


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
beto
post Aug 12 2007, 01:58
Post #125





Group: Members (Donating)
Posts: 713
Joined: 8-July 04
From: Sao Paulo
Member No.: 15173



QUOTE (naylor83 @ Aug 10 2007, 17:23) *
I try a bit harder than that, because I like to know that I have all the detail in the music that I can possibly hear when I dish out a big 5.0.


That of course makes sense. Maybe I am just not patient enough. tongue.gif


--------------------
http://volutabro.blogspot.com
Go to the top of the page
+Quote Post

8 Pages V  « < 3 4 5 6 7 > » 
Closed TopicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 20th April 2014 - 11:56