IPB

Welcome Guest ( Log In | Register )

8 Pages V  < 1 2 3 4 5 > »   
Closed TopicStart new topic
Public MP3 Listening Test @ 128 kbps - CLOSED
Sebastian Mares
post Oct 26 2008, 23:03
Post #51





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (jmartis @ Oct 26 2008, 22:19) *
I downloaded ABC/HR, I also downloaded a sample package "Sample07.zip". I ran Sample07.bat, then executed abchr.jar, clicked Open ABC/HR config, loaded Sample07.ecf.

The problem: Clicking play on any sample doesn't produce any sound.


From the presentation page:

QUOTE
Please note: if you don't hear any sound when using ABC/HR, simply change the device from ABC/HR's settings. In some cases, ABC/HR might not default to the primary audio driver.


QUOTE (jmartis @ Oct 26 2008, 22:19) *
This sample package happened to be the eig sample


Impossible - eig isn't even used in this test. blink.gif


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
halb27
post Oct 26 2008, 23:04
Post #52





Group: Members
Posts: 2414
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



My sample07 is Suzanne Vega (Tom's Diner) as well.


--------------------
lame3100m --bCVBR 300
Go to the top of the page
+Quote Post
Sebastian Mares
post Oct 26 2008, 23:08
Post #53





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



Just downloaded Sample07 from both mirrors and it's definitely Tom's Diner. As I mentioned already, eig isn't even featured in this test.

The samples included are (alphabetically sorted):
atrain
Castanets_Original
chariotsoffire
fatboy_30sec
finalfantasy
Hypnotize
inthenight
Layla
linchpin
macabre
sfbay
TomsDiner
velvet
Waiting

Edit: Wondering why IPB sent me 3 mails for the same reply from Bodhi.

This post has been edited by Sebastian Mares: Oct 26 2008, 23:09


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
lvqcl
post Oct 26 2008, 23:41
Post #54





Group: Developer
Posts: 3209
Joined: 2-December 07
Member No.: 49183



QUOTE (jmartis @ Oct 27 2008, 01:19) *
Problem #2. This sample package happened to be the eig sample, which produces easily noticable artifacts. Just out of curiosity, I tried to encode the sample with FHG surround encoder "-br 0 -m 4 -q 1 -vbri -ofl" and compared my encoding to the test samples. It sounded different from all of them, and it shouldn't.


I tried to encode samples #2 and #12 myself. My files were bit-identical to downloaded ones. (By the way, l3enc encodes raw files, not wavs). And sample 7 is definitely "Tom's Diner", not "Everything is green".
Go to the top of the page
+Quote Post
/mnt
post Oct 27 2008, 02:06
Post #55





Group: Members
Posts: 696
Joined: 22-April 06
Member No.: 29877



My Sample 7 file is Tom's Diner anyway and not eig. I have to admit i fail to ABX Tom's Diner on the non low anchor encodes (l3ence is very easy to ABX though). And now for some reason after i tried to ABX Tom's Diner. i need to watch a episode of Seinfeld laugh.gif.

This post has been edited by /mnt: Oct 27 2008, 02:09


--------------------
"I never thought I'd see this much candy in one mission!"
Go to the top of the page
+Quote Post
Neasden
post Oct 27 2008, 02:59
Post #56





Group: Banned
Posts: 185
Joined: 1-July 08
Member No.: 55148



If one encoder SURPRISES you then this definitely should not be LAME, because if it were it, it wouldn't be a surprise, would it?

This post has been edited by Neasden: Oct 27 2008, 03:00
Go to the top of the page
+Quote Post
halb27
post Oct 27 2008, 06:53
Post #57





Group: Members
Posts: 2414
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



QUOTE (/mnt @ Oct 27 2008, 03:06) *
... I have to admit i fail to ABX Tom's Diner on the non low anchor encodes ...

I have to admit that there are many samples where only the very obvious low anchor has an issue for me. Shows that even 128 kbps mp3 usually is fine (to me), and it's personal properties to which kind of artifact we are sensitive.

This post has been edited by halb27: Oct 27 2008, 10:06


--------------------
lame3100m --bCVBR 300
Go to the top of the page
+Quote Post
melomaniac
post Oct 27 2008, 10:44
Post #58





Group: Members
Posts: 43
Joined: 1-August 08
From: Brussels
Member No.: 56565



QUOTE (halb27 @ Oct 27 2008, 07:53) *
I have to admit that there are many samples where only the very obvious low anchor has an issue for me. Shows that even 128 kbps mp3 usually is fine (to me).

Me too.

QUOTE
[...] and it's personal properties to which kind of artifact we are sensitive.

I’m more sensitive to warbling, hi-hat chirping and warped ride cymbal (I play the drums) and also distorted voices or trumpets but not so much to pre-echo.

P.S. : I really liked the Vangelis sample, interesting one IMO.

This post has been edited by melomaniac: Nov 2 2008, 12:44
Go to the top of the page
+Quote Post
Alex B
post Oct 27 2008, 11:12
Post #59





Group: Members
Posts: 1303
Joined: 14-September 05
From: Helsinki, Finland
Member No.: 24472



QUOTE (/mnt @ Oct 27 2008, 04:06) *
I have to admit i fail to ABX Tom's Diner on the non low anchor encodes (l3ence is very easy to ABX though).


That's surprising. There's another Tom's Diner sample that can be easily distinguished.

IMHO, it has obvious problems and ABXing is not necessary. The four other samples are closer to the original, but not fully transparent.

I wonder if you simply tried to hear other kinds of artifacts and didn't notice this problem because it is different.

Would you mind testing the two samples I uploaded here: http://www.hydrogenaudio.org/forums/index....showtopic=66815


--------------------
http://listening-tests.freetzi.com
Go to the top of the page
+Quote Post
/mnt
post Oct 27 2008, 11:37
Post #60





Group: Members
Posts: 696
Joined: 22-April 06
Member No.: 29877



QUOTE (Alex B @ Oct 27 2008, 11:12) *
QUOTE (/mnt @ Oct 27 2008, 04:06) *
I have to admit i fail to ABX Tom's Diner on the non low anchor encodes (l3ence is very easy to ABX though).


That's surprising. There's another Tom's Diner sample that can be easily distinguished.

IMHO, it has obvious problems and ABXing is not necessary. The four other samples are closer to the original, but not fully transparent.

I wonder if you simply tried to hear other kinds of artifacts and didn't notice this problem because it is different.

Would you mind testing the two samples I uploaded here: http://www.hydrogenaudio.org/forums/index....showtopic=66815


I have found some issues with that sample. The mp3 version has a skratchy artifact on the vocals, abit like fatboy at V5.

CODE
foo_abx 1.3.3 report
foobar2000 v0.9.6 beta 1
2008/10/27 10:30:58

File A: C:\Downloads\sample07_source.flac
File B: C:\Downloads\sample07_mp3.flac

10:30:58 : Test started.
10:31:08 : 01/01 50.0%
10:31:11 : 02/02 25.0%
10:31:16 : 03/03 12.5%
10:31:19 : 04/04 6.3%
10:31:22 : 05/05 3.1%
10:31:30 : 06/06 1.6%
10:31:37 : 07/07 0.8%
10:31:42 : 08/08 0.4%
10:31:48 : 09/09 0.2%
10:31:53 : 10/10 0.1%
10:32:00 : 11/11 0.0%
10:32:05 : 12/12 0.0%
10:32:11 : Test finished.

----------
Total: 12/12 (0.0%)


This post has been edited by /mnt: Oct 27 2008, 11:38


--------------------
"I never thought I'd see this much candy in one mission!"
Go to the top of the page
+Quote Post
jmartis
post Oct 27 2008, 11:49
Post #61





Group: Members
Posts: 381
Joined: 9-April 06
From: Czech Republic
Member No.: 29311



QUOTE (Sebastian Mares @ Oct 27 2008, 00:08) *
Just downloaded Sample07 from both mirrors and it's definitely Tom's Diner. As I mentioned already, eig isn't even featured in this test.

OK, sorry then, I must have gotten a wrong (old) sample... Thanks for the help.
Go to the top of the page
+Quote Post
Alex B
post Oct 27 2008, 11:56
Post #62





Group: Members
Posts: 1303
Joined: 14-September 05
From: Helsinki, Finland
Member No.: 24472



QUOTE (/mnt @ Oct 27 2008, 13:37) *
I have found some issues with that sample. The mp3 version has a skratchy artifact on the vocals, abit like fatboy at V5.


The sample is a direct clip from one of the test samples (and it is not from the low anchor).


--------------------
http://listening-tests.freetzi.com
Go to the top of the page
+Quote Post
melomaniac
post Oct 27 2008, 12:05
Post #63





Group: Members
Posts: 43
Joined: 1-August 08
From: Brussels
Member No.: 56565



QUOTE (Alex B @ Oct 27 2008, 12:56) *
QUOTE (/mnt @ Oct 27 2008, 13:37) *
I have found some issues with that sample. The mp3 version has a skratchy artifact on the vocals, abit like fatboy at V5.


The sample is a direct clip from one of the test samples (and it is not from the low anchor).

I've just made an ABC/HR test on sample07 and I can only spot the one you're talking about right now (like /mnt, no problem ABX it) and the low anchor, but not the other four.
Go to the top of the page
+Quote Post
/mnt
post Oct 27 2008, 12:17
Post #64





Group: Members
Posts: 696
Joined: 22-April 06
Member No.: 29877



QUOTE (Alex B @ Oct 27 2008, 11:56) *
QUOTE (/mnt @ Oct 27 2008, 13:37) *
I have found some issues with that sample. The mp3 version has a skratchy artifact on the vocals, abit like fatboy at V5.


The sample is a direct clip from one of the test samples (and it is not from the low anchor).


That was embarrassing smile.gif, i did not know what was really wrong with that sample, i knew its imfamus for tuning. When i was trying to ABX Tom's Diner last week; I thought all the non low anchor samples sounded ok.

This post has been edited by /mnt: Oct 27 2008, 12:18


--------------------
"I never thought I'd see this much candy in one mission!"
Go to the top of the page
+Quote Post
Alex B
post Oct 27 2008, 12:19
Post #65





Group: Members
Posts: 1303
Joined: 14-September 05
From: Helsinki, Finland
Member No.: 24472



Normally we should not discuss about the test samples before the test is over, but I think this is a special case. The problem is quite obvious and it's worrying if someone cannot spot it.

So please be careful when starting a new test sample. Try to clear your mind from the previously listened samples and the possibly different problems you may have spotted. It might be a good idea to have a short break and do something completely different before opening the next test configuration file.

This post has been edited by Alex B: Oct 27 2008, 12:24


--------------------
http://listening-tests.freetzi.com
Go to the top of the page
+Quote Post
halb27
post Oct 27 2008, 12:41
Post #66





Group: Members
Posts: 2414
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



In case I know what to listen for (and I'm sensitive for the kind of problem with the sample) in many cases it's rather easy to ABX the differences.
With an unknown sample it's a lot more difficult. It's also a question of how much time is used on the sample. I remember having considered the harp40_1 Helix ~200 kbps result perfect, whereas after another day of intensive listening tests with other encoders at even 320 kbps I could easily distinguish all the mp3 results from the original. We are able to learn. But is hard learning significant? We do not listen to music this way when enjoying it.
So I think if we have given care with the listening test (while avoiding spending hours on a sample) we should consider an encoder sample transparent if we can't hear a difference. We should accept the fact that this may be just our (maybe temporary) inability to hear the issue. We shouldn't feel challenged too much in finding differences. A little bit of a challenge is necessary to do a good job, but we shouldn't go very far. After all it's about perfectness of encoders under real life conditions, especially at 128 kbps.

This post has been edited by halb27: Oct 27 2008, 12:48


--------------------
lame3100m --bCVBR 300
Go to the top of the page
+Quote Post
Lyx
post Oct 27 2008, 12:56
Post #67





Group: Members
Posts: 3353
Joined: 6-July 03
From: Sachsen (DE)
Member No.: 7609



Adding to halb's comments:

This may also be relevant to testing-bias. In tests like these, we focus on short samples, with each of them typically representing one single issue. But the purpose of doing that isn't knowing how transparent those test-samples sound like - the purpose is extrapolating the scores to generic encoder quality (so, music in general). As we know, tuning a lossy encoder also is a matter of balance - what improves one issue, may worsen other issues. It is therefore very important, that the test-samples cover a wide range of "scenarios" evenly. Else, the following could happen: testers focus on specific issues and get accustomed to noticing them. The encoder then gets tuned to improve those cases, while unintentionally creating new different problems... but since the testers are focussed on the biased sample pool, the regression stays unnoticed. As visible in this thread, this may happen even if the testers also listen to music with those regressions, but don't recognize them because of being focussed on looking for issues related to the biased sample pool.
Go to the top of the page
+Quote Post
Alex B
post Oct 27 2008, 13:31
Post #68





Group: Members
Posts: 1303
Joined: 14-September 05
From: Helsinki, Finland
Member No.: 24472



Many of us already knew that a MP3 VBR test @ 128 kbps would be quite difficult for the testers. I'd say that all contenders are usable for casual listening. Spotting the small differencies and occasional obvious artifacts needs concentration.

Particularly in this test we should not too easily let the contenders to pass any of the tests "as transparent" if we want to get any meaningful test results. Otherwise the test would be unnecessary. We could just say that VBR MP3 @ 128 kbps is good enough for casual listening and there is no matter which encoder is used.

QUOTE (halb27 @ Oct 27 2008, 14:41) *
So I think if we have given care with the listening test (while avoiding spending hours on a sample) we should consider an encoder sample transparent if we can't hear a difference.

I think at least half an hour would be a reasonable time for each test configuration. That would give only 5 minutes on average for each encoder. It is useful to try several different short passages when trying to ABX them and that takes some time.

I spent about an hour with Tom's Diner. The four near transparent encoders were difficult to ABX. I have yet to decide if I give them all the same rating or if I try to put them in some order.

This post has been edited by Alex B: Oct 27 2008, 13:34


--------------------
http://listening-tests.freetzi.com
Go to the top of the page
+Quote Post
halb27
post Oct 27 2008, 14:37
Post #69





Group: Members
Posts: 2414
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



QUOTE (Alex B @ Oct 27 2008, 14:31) *
Particularly in this test we should not too easily let the contenders to pass any of the tests "as transparent" if we want to get any meaningful test results. Otherwise the test would be unnecessary. ... I think at least half an hour would be a reasonable time for each test configuration. ...

Sure we shouldn't let the contenders pass too easily, and about half an hour per sample with all 6 encoder results sounds reasonable to me. In case there are suspicious spots 5 minutes per encoder result quickly pass by. But I admit in those cases where I didn't even have a suspicion I didn't spend half an hour.

BTW I'm just through with the first 9 samples, and my primary concern is with having a really silent home when I'm testing and being not pressed to finish a test because of other things waiting for me. So usually I manage to go through 1 or 2 tests per day.


--------------------
lame3100m --bCVBR 300
Go to the top of the page
+Quote Post
Bodhi
post Oct 28 2008, 11:13
Post #70





Group: Members
Posts: 261
Joined: 10-June 06
Member No.: 31712



QUOTE (Sebastian Mares @ Oct 26 2008, 22:36) *
Patience, young padawan... biggrin.gif

crying.gif
Go to the top of the page
+Quote Post
halb27
post Oct 28 2008, 20:41
Post #71





Group: Members
Posts: 2414
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



sample12: for a change a sample with 4 encoders having obvious issues. What a relief!


--------------------
lame3100m --bCVBR 300
Go to the top of the page
+Quote Post
/mnt
post Oct 28 2008, 22:08
Post #72





Group: Members
Posts: 696
Joined: 22-April 06
Member No.: 29877



QUOTE (halb27 @ Oct 28 2008, 20:41) *
sample12: for a change a sample with 4 encoders having obvious issues. What a relief!

Thats fatboy, alot of codecs struggle at the vocals on this sample and very easy to ABX.


--------------------
"I never thought I'd see this much candy in one mission!"
Go to the top of the page
+Quote Post
Alexxander
post Oct 29 2008, 12:12
Post #73





Group: Members
Posts: 456
Joined: 15-November 04
Member No.: 18143



Just sent my results. I had a nice and sometimes hard time spotting problems of this nice selection of samples. Curious to see that a few samples were easy to ABX on speakers (no matter what encoder excluding low-anchor). But generally spoken using headphones is a must.

I want to thank those who made possible this mp3 listening test and especially Sebastian.
Go to the top of the page
+Quote Post
lexor
post Oct 29 2008, 14:55
Post #74





Group: Members
Posts: 216
Joined: 20-July 03
Member No.: 7896



Having trouble running the test here. I've downloaded the zip with abc-hr and java (from the link in the readme). The directory structure is preserved, I can unarchive samples no problem using the appropriate .bat file. Then it breaks.

I get this at start up.
CODE
D:\Program Files\Java\jre1.5.0_15\bin>java -jar E:\ABC-HR\abchr.jar
java.io.FileNotFoundException: clicodecs.cfg (The system cannot find the file sp
ecified)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(Unknown Source)
        at abchr.settings.DecoderSettings.<init>(Unknown Source)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)

        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
        at java.lang.reflect.Constructor.newInstance(Unknown Source)
        at java.lang.Class.newInstance0(Unknown Source)
        at java.lang.Class.newInstance(Unknown Source)
        at abchr.gui.ProjectFrame.<clinit>(Unknown Source)
        at abchr.gui.Main.main(Unknown Source)


and this, when I press play.
CODE
Exception in thread "AWT-EventQueue-0" java.lang.IllegalArgumentException: Line
unsupported: interface SourceDataLine supporting format PCM_SIGNED 44100.0 Hz, 1
6 bit, stereo, 4 bytes/frame, little-endian, and buffers of 88200 to 88200 bytes

        at com.sun.media.sound.PortMixer.getLine(Unknown Source)
        at abchr.audio.PlaybackThread.play(Unknown Source)
        at abchr.gui.PlayAction.actionPerformed(Unknown Source)
        at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
        at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
        at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
        at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
        at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(Unknown Source)
        at java.awt.AWTEventMulticaster.mouseReleased(Unknown Source)
        at java.awt.Component.processMouseEvent(Unknown Source)
        at javax.swing.JComponent.processMouseEvent(Unknown Source)
        at java.awt.Component.processEvent(Unknown Source)
        at java.awt.Container.processEvent(Unknown Source)
        at java.awt.Component.dispatchEventImpl(Unknown Source)
        at java.awt.Container.dispatchEventImpl(Unknown Source)
        at java.awt.Component.dispatchEvent(Unknown Source)
        at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
        at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
        at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
        at java.awt.Container.dispatchEventImpl(Unknown Source)
        at java.awt.Window.dispatchEventImpl(Unknown Source)
        at java.awt.Component.dispatchEvent(Unknown Source)
        at java.awt.EventQueue.dispatchEvent(Unknown Source)
        at java.awt.EventDispatchThread.pumpOneEventForHierarchy(Unknown Source)

        at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
        at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
        at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
        at java.awt.EventDispatchThread.run(Unknown Source)


What should I do?

This post has been edited by lexor: Oct 29 2008, 14:55


--------------------
The Plan Within Plans
Go to the top of the page
+Quote Post
Sebastian Mares
post Oct 29 2008, 15:01
Post #75





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



I think I know what the problem is... What happens if you do this:

cmd
E:
cd \ABC-HR
D:\Program Files\Java\jre1.5.0_15\bin\java.exe -jar abchr.jar


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post

8 Pages V  < 1 2 3 4 5 > » 
Closed TopicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 18th April 2014 - 22:20