IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
Unsure if I am conducting my ABX Comparator tests properly/first timer, [moved from fb2k General]
juscor
post Jun 10 2013, 19:50
Post #1





Group: Members
Posts: 8
Joined: 9-June 13
Member No.: 108562



I've been using this plugin to try and determine what bitrate is transparent for me using the new opus codec. The problem is that I'm scoring PERFECTLY on my tests even at as high of a bitrate as 256.blink.gif While the song does the become clearer and more detailed the higher the bitrate climbs, one very subtle difference always remains that allows me to distinguish between the two and skew the results in my favor.

The volume of the lossy version in comparison to the lossless one seems to be ever so slightly lower? It's not that the song is quieter, it just sounds less powerful in delivery. huh.gif At first I though the problem was with the codec given how young it is, so I encoded another version in ogg vorbis at -q 5 (hydrogenaudio wiki reports this quality setting as transparent). Still, I got basically the same results (only 1 wrong after repeating the test three times).

My hearing simply CANNOT be this finely tuned, so what am I doing wrong? Perhaps there's a setting I've overlooked? I tried applying ReplayGain but the comparator seems to ignore it. Maybe it's the fact that the songs' peaks are different? This is my first time ever running these tests so I'd really appreciate it if somebody can provide some clarification. Thank you!! cool.gif

This post has been edited by juscor: Jun 10 2013, 20:09
Go to the top of the page
+Quote Post
ChronoSphere
post Jun 10 2013, 22:19
Post #2





Group: Members
Posts: 399
Joined: 11-March 07
Member No.: 41384



Do you take a lossless song -> convert it to lossy -> load both to abx without dsp/replaygain? In that case the volume should be the same. I haven't noticed any volume variation in my tests. You also might have good ears, too. I know that for me mp3 starts getting transparent around v2, but I still can distinguish between them even if it's extremely hard.

For example, here is v3
CODE
foo_abx 1.3.4 report
foobar2000 v1.2.6
2013/06/08 19:01:55

File A: 03 ゆうまお - 蒼空にくちづけたら.flac
File B: 03 蒼空にくちづけたら_v3.mp3

19:01:55 : Test started.
19:02:40 : 01/01 50.0%
19:03:09 : 02/02 25.0%
19:03:21 : 03/03 12.5%
19:04:00 : 04/04 6.3%
19:04:42 : 05/05 3.1%
19:05:02 : 06/06 1.6%
19:05:22 : 07/07 0.8%
19:05:43 : 08/08 0.4%
19:05:54 : 09/09 0.2%
19:06:16 : 10/10 0.1%
19:06:19 : Test finished.

----------
Total: 10/10 (0.1%)


And here's v0
CODE
foo_abx 1.3.4 report
foobar2000 v1.2.6
2013/06/08 19:20:26

File A: 03 蒼空にくちづけたら.flac
File B: 03 蒼空にくちづけたら_v0.mp3

19:20:26 : Test started.
19:21:02 : 01/01 50.0%
19:21:12 : 01/02 75.0%
19:21:34 : 02/03 50.0%
19:21:47 : 03/04 31.3%
19:21:57 : 03/05 50.0%
19:22:09 : 03/06 65.6%
19:22:29 : 03/07 77.3%
19:22:58 : 04/08 63.7%
19:23:13 : 05/09 50.0%
19:23:24 : 06/10 37.7%
19:23:44 : 07/11 27.4%
19:24:09 : 07/12 38.7%
19:24:12 : Test finished.

----------
Total: 7/12 (38.7%)


Point is, just because a codec is generally transparent for people doesn't mean it is for you.

This post has been edited by ChronoSphere: Jun 10 2013, 22:20
Go to the top of the page
+Quote Post
aztec_mystic
post Jun 10 2013, 22:24
Post #3





Group: Members
Posts: 93
Joined: 28-March 13
Member No.: 107425



QUOTE (ChronoSphere @ Jun 10 2013, 23:19) *
Do you take a lossless song -> convert it to lossy -> load both to abx without dsp/replaygain? In that case the volume should be the same.

Really?
Go to the top of the page
+Quote Post
ChronoSphere
post Jun 10 2013, 22:32
Post #4





Group: Members
Posts: 399
Joined: 11-March 07
Member No.: 41384



Yes, unless you applied replay gain while converting to lossy.
Even if the lossless track has replay gain, it will be transcoded to wav without applying the value by the abx plugin.
Go to the top of the page
+Quote Post
juscor
post Jun 10 2013, 22:54
Post #5





Group: Members
Posts: 8
Joined: 9-June 13
Member No.: 108562



QUOTE (ChronoSphere @ Jun 10 2013, 23:19) *
Do you take a lossless song -> convert it to lossy -> load both to abx without dsp/replaygain? In that case the volume should be the same. I haven't noticed any volume variation in my tests. You also might have good ears, too. I know that for me mp3 starts getting transparent around v2, but I still can distinguish between them even if it's extremely hard.


I use a lossless source and then convert it to lossy. Following that I load both into abx.
This happens with or without replaygain, but i'll run another test just be sure.

Here are my previous results:

Opus 256
CODE
foo_abx 1.3.4 report
foobar2000 v1.2.6
2013/06/10 11:12:33

File A: C:\Users\XXXXX\Music\Cuetools Coverted\Fear Factory\1995 - Demanufacture (Digipak)\01. Demanufacture.flac
File B: L:\Libraries\Music\Fear Factory - Demanufacture (Digipak)\Fear Factory - Demanufacture.opus

11:12:33 : Test started.
11:13:13 : 01/01 50.0%
11:13:21 : 02/02 25.0%
11:13:29 : 03/03 12.5%
11:13:40 : 04/04 6.3%
11:13:49 : 05/05 3.1%
11:14:02 : 06/06 1.6%
11:14:12 : 07/07 0.8%
11:14:30 : 08/08 0.4%
11:14:38 : 09/09 0.2%
11:15:17 : 10/10 0.1%
11:15:25 : 11/11 0.0%
11:15:44 : 12/12 0.0%
11:15:50 : 13/13 0.0%
11:16:11 : 14/14 0.0%
11:16:25 : 15/15 0.0%
11:16:35 : 16/16 0.0%
11:16:46 : 17/17 0.0%
11:17:15 : 18/18 0.0%
11:17:23 : 19/19 0.0%
11:17:32 : 20/20 0.0%
11:18:06 : Test finished.

----------
Total: 20/20 (0.0%)


Ogg Vorbis -q 5
CODE
foo_abx 1.3.4 report
foobar2000 v1.2.6
2013/06/10 10:07:38

File A: C:\Users\XXXXX\Music\Cuetools Coverted\Fear Factory\1995 - Demanufacture (Digipak)\01. Demanufacture.flac
File B: L:\Libraries\Music\Fear Factory - Demanufacture (Digipak)\Fear Factory - Demanufacture.ogg

10:07:38 : Test started.
10:09:07 : 01/01 50.0%
10:09:18 : 02/02 25.0%
10:09:33 : 03/03 12.5%
10:10:57 : 04/04 6.3%
10:11:10 : 04/05 18.8%
10:11:20 : 05/06 10.9%
10:11:34 : 06/07 6.3%
10:11:48 : 07/08 3.5%
10:11:58 : 07/09 9.0%
10:12:13 : 08/10 5.5%
10:12:21 : 09/11 3.3%
10:12:55 : Test finished.

----------
Total: 9/11 (3.3%)


I didn't go to 20 on the vorbis test because at the time I wasn't sure if it was the codec or not.



--------------------------UPDATE----------------------------

So I ran the test again after highlighting both tracks and selecting "remove ReplayGain information from files" and ran the ABX test. This time it felt more difficult, though I could still detect a slight difference in how "powerful" the samples sounded. Maybe the reason why I was able to detect the nuances more easily before was because foobar uses album gain by default (I remember reading that somewhere), so perhaps when I converted to lossy, that information might have been carried over. If what chronosphere says about how abx treats replaygain values is true, then this could be an explanation. Anyway, here are my results with opus at 256 and no ReplayGain:

CODE
foo_abx 1.3.4 report
foobar2000 v1.2.6
2013/06/10 15:00:27

File A: C:\Users\XXXXX\Music\Cuetools Coverted\Fear Factory\1995 - Demanufacture (Digipak)\01. Demanufacture.flac
File B: L:\Libraries\Music\Fear Factory - Demanufacture (Digipak)\Fear Factory - Demanufacture.opus

15:00:27 : Test started.
15:01:06 : 01/01 50.0%
15:01:19 : 02/02 25.0%
15:01:38 : 03/03 12.5%
15:01:57 : 03/04 31.3%
15:02:06 : 04/05 18.8%
15:02:22 : 05/06 10.9%
15:02:31 : 06/07 6.3%
15:02:41 : 07/08 3.5%
15:03:06 : 08/09 2.0%
15:03:51 : 09/10 1.1%
15:04:07 : 10/11 0.6%
15:04:27 : 10/12 1.9%
15:04:43 : 11/13 1.1%
15:05:33 : 12/14 0.6%
15:05:42 : 13/15 0.4%
15:05:54 : 14/16 0.2%
15:06:27 : 15/17 0.1%
15:06:40 : 16/18 0.1%
15:07:04 : 17/19 0.0%
15:07:27 : 18/20 0.0%
15:08:08 : Test finished.


I am open to running another test with higher bitrates or another codec like vorbis if any of you think it's necessary or are interested.

This post has been edited by juscor: Jun 10 2013, 23:33
Go to the top of the page
+Quote Post
lvqcl
post Jun 11 2013, 02:22
Post #6





Group: Developer
Posts: 3208
Joined: 2-December 07
Member No.: 49183



Fear Factory is not something that is easy to encode. Also, I suspect that flac and opus files have different samplerates - maybe this also affects ABXing.

QUOTE (juscor @ Jun 11 2013, 01:54) *
I am open to running another test with higher bitrates or another codec like vorbis if any of you think it's necessary or are interested.

Do you use official Vorbis or AoTuV? Try latest AoTuV -q 7.

Other very good codecs: Musepack (quality 6), Apple AAC (via qaac; tvbr 100), or Winamp FhG AAC (quality 5).

This post has been edited by lvqcl: Jun 11 2013, 02:38
Go to the top of the page
+Quote Post
juscor
post Jun 11 2013, 04:40
Post #7





Group: Members
Posts: 8
Joined: 9-June 13
Member No.: 108562



QUOTE (lvqcl @ Jun 11 2013, 03:22) *
Fear Factory is not something that is easy to encode. Also, I suspect that flac and opus files have different samplerates - maybe this also affects ABXing.

Sounds reasonable. It was mostly in the sections where all the instruments were active at once that I could detect the differences. I highly doubt with something less "complex" that I would be able to make the same distinctions.

QUOTE (lvqcl @ Jun 11 2013, 03:22) *
Do you use official Vorbis or AoTuV? Try latest AoTuV -q 7.

Other very good codecs: Musepack (quality 6), Apple AAC (via qaac; tvbr 100), or Winamp FhG AAC (quality 5).

I use the latest AoTuV encoder with the LancerMod. I only tested it at -q 5 though. I'll try -q 7 as well as the other codecs you recommended tomorrow morning and update the thread accordingly.
Go to the top of the page
+Quote Post
Kohlrabi
post Jun 11 2013, 05:25
Post #8





Group: Super Moderator
Posts: 953
Joined: 12-March 05
From: Kiel, Germany
Member No.: 20561



QUOTE (juscor @ Jun 11 2013, 05:40) *
I use the latest AoTuV encoder with the LancerMod.
Are there AoTuV/Lancer encoders newer than from 2006? I didn't find any. Nevertheless even a Vorbis encoder that old should perform reasonably enough. Maybe you just encountered a problematic track, or you are susceptible to Vorbis and CELT artifacts. Have you tried any other tracks?


--------------------
Audiophiles live in constant fear of jitter.
Go to the top of the page
+Quote Post
lvqcl
post Jun 11 2013, 09:17
Post #9





Group: Developer
Posts: 3208
Joined: 2-December 07
Member No.: 49183



QUOTE (Kohlrabi @ Jun 11 2013, 08:25) *
Are there AoTuV/Lancer encoders newer than from 2006?

http://www.rarewares.org/ogg-oggenc.php

Also my compiles (they're without FLAC reader and SRC resampler): ver 5.7; ver 6.03.
Go to the top of the page
+Quote Post
Gainless
post Jun 11 2013, 13:42
Post #10





Group: Members
Posts: 169
Joined: 28-October 11
Member No.: 94764



Which version of Opus do you use?
Would be also good to upload the sample you used here.
Go to the top of the page
+Quote Post
juscor
post Jun 11 2013, 19:08
Post #11





Group: Members
Posts: 8
Joined: 9-June 13
Member No.: 108562



QUOTE (Gainless @ Jun 11 2013, 14:42) *
Which version of Opus do you use?

https://people.xiph.org/~greg/opus-tools_58d80ab.zip
It's the second 1.1 pre-beta build linked in this thread by jmvalin: http://www.hydrogenaudio.org/forums/index....howtopic=101061

I'll try and upload my samples. Might take a little while since my connection is unstable.


Anyway, I tested vorbis at -q 7 and MPC at -q 6. It was definitely more difficult to discriminate between the two samples with vorbis, and even moreso with MPC (it had more "oomph" to it). I had to sit there for a while listening to the samples before starting my tests just to get a feel for them. I ended up scoring lower with MPC, and I'd say it's extremely near transparent, if not transparent already. I tested it 3 times more than vorbis just to be sure that I was actually perceiving it as more difficult.

ogg vorbis -q7 results:
CODE

foo_abx 1.3.4 report 1
foobar2000 v1.2.6
2013/06/11 08:27:36

File A: C:\Users\XXXXX\Music\Cuetools Coverted\Fear Factory\1995 - Demanufacture (Digipak)\01. Demanufacture.flac
File B: L:\Libraries\Music\Fear Factory - Demanufacture (Digipak)\Fear Factory - Demanufacture.ogg

08:27:36 : Test started.
08:28:09 : 01/01 50.0%
08:28:20 : 02/02 25.0%
08:28:27 : 03/03 12.5%
08:28:34 : 04/04 6.3%
08:28:42 : 05/05 3.1%
08:29:17 : 05/06 10.9%
08:29:23 : 06/07 6.3%
08:29:32 : 06/08 14.5%
08:29:39 : 07/09 9.0%
08:29:47 : 07/10 17.2%
08:30:03 : 08/11 11.3%
08:30:06 : Test finished.

----------
Total: 8/11 (11.3%)


foo_abx 1.3.4 report 2
foobar2000 v1.2.6
2013/06/11 08:31:07

File A: C:\Users\XXXXX\Music\Cuetools Coverted\Fear Factory\1995 - Demanufacture (Digipak)\01. Demanufacture.flac
File B: L:\Libraries\Music\Fear Factory - Demanufacture (Digipak)\Fear Factory - Demanufacture.ogg

08:31:07 : Test started.
08:32:03 : 01/01 50.0%
08:32:24 : 02/02 25.0%
08:32:40 : 03/03 12.5%
08:32:49 : 04/04 6.3%
08:32:56 : 04/05 18.8%
08:33:03 : 05/06 10.9%
08:33:23 : 06/07 6.3%
08:33:37 : 07/08 3.5%
08:33:44 : 08/09 2.0%
08:34:43 : 09/10 1.1%
08:35:14 : 10/11 0.6%
08:35:41 : Test finished.

----------
Total: 10/11 (0.6%)



foo_abx 1.3.4 report 3
foobar2000 v1.2.6
2013/06/10 10:14:47

File A: C:\Users\XXXXX\Music\Cuetools Coverted\Fear Factory\1995 - Demanufacture (Digipak)\01. Demanufacture.flac
File B: L:\Libraries\Music\Fear Factory - Demanufacture (Digipak)\Fear Factory - Demanufacture.ogg

10:14:47 : Test started.
10:15:38 : 01/01 50.0%
10:15:49 : 02/02 25.0%
10:16:05 : 03/03 12.5%
10:16:22 : 03/04 31.3%
10:16:33 : 04/05 18.8%
10:16:55 : 04/06 34.4%
10:17:05 : 05/07 22.7%
10:17:22 : 06/08 14.5%
10:17:38 : 07/09 9.0%
10:18:47 : 08/10 5.5%
10:19:07 : 09/11 3.3%
10:22:05 : Test finished.

----------
Total: 9/11 (3.3%)




MPC -q 7 Results
CODE
foo_abx 1.3.4 report 1
foobar2000 v1.2.6
2013/06/11 08:53:44

File A: C:\Users\XXXXX\Music\Cuetools Coverted\Fear Factory\1995 - Demanufacture (Digipak)\01. Demanufacture.flac
File B: L:\Libraries\Music\Fear Factory - Demanufacture (Digipak)\Fear Factory - Demanufacture.mpc

08:53:44 : Test started.
08:57:54 : 01/01 50.0%
08:58:01 : 01/02 75.0%
08:58:11 : 02/03 50.0%
08:58:28 : 03/04 31.3%
08:58:46 : 04/05 18.8%
08:59:12 : 05/06 10.9%
08:59:23 : 06/07 6.3%
08:59:38 : 06/08 14.5%
08:59:50 : 07/09 9.0%
09:00:11 : 08/10 5.5%
09:00:58 : 08/11 11.3%
09:01:04 : Test finished.

----------
Total: 8/11 (11.3%)


foo_abx 1.3.4 report 2
foobar2000 v1.2.6
2013/06/11 09:54:48

File A: C:\Users\XXXXX\Music\Cuetools Coverted\Fear Factory\1995 - Demanufacture (Digipak)\01. Demanufacture.flac
File B: L:\Libraries\Music\Fear Factory - Demanufacture (Digipak)\Fear Factory - Demanufacture.mpc

09:54:48 : Test started.
09:55:14 : 01/01 50.0%
09:55:21 : 02/02 25.0%
09:55:44 : 02/03 50.0%
09:56:11 : 03/04 31.3%
09:56:30 : 03/05 50.0%
09:57:37 : 04/06 34.4%
09:58:06 : 05/07 22.7%
09:58:17 : 06/08 14.5%
09:58:44 : 06/09 25.4%
09:59:45 : 07/10 17.2%
10:00:24 : 08/11 11.3%
10:00:31 : Test finished.

----------
Total: 8/11 (11.3%)


foo_abx 1.3.4 report 3
foobar2000 v1.2.6
2013/06/11 09:42:28

File A: C:\Users\XXXXX\Music\Cuetools Coverted\Fear Factory\1995 - Demanufacture (Digipak)\01. Demanufacture.flac
File B: L:\Libraries\Music\Fear Factory - Demanufacture (Digipak)\Fear Factory - Demanufacture.mpc

09:42:28 : Test started.
09:43:12 : 01/01 50.0%
09:45:30 : 02/02 25.0%
09:45:38 : 02/03 50.0%
09:46:04 : 02/04 68.8%
09:46:22 : 03/05 50.0%
09:46:36 : 03/06 65.6%
09:47:18 : 04/07 50.0%
09:47:46 : 05/08 36.3%
09:48:01 : 06/09 25.4%
09:48:11 : 07/10 17.2%
09:48:22 : 08/11 11.3%
09:48:34 : Trial reset.
09:48:57 : 01/01 50.0%
09:49:17 : 01/02 75.0%
09:49:43 : 02/03 50.0%
09:50:38 : 03/04 31.3%
09:51:19 : 03/05 50.0%
09:52:24 : 04/06 34.4%
09:52:50 : 05/07 22.7%
09:53:03 : 06/08 14.5%
09:53:22 : 07/09 9.0%
09:53:37 : 08/10 5.5%
09:54:04 : 09/11 3.3%
09:54:06 : Test finished.

----------
Total: 17/22 (0.8%)



foo_abx 1.3.4 report 4
foobar2000 v1.2.6
2013/06/11 10:01:32

File A: C:\Users\XXXXX\Music\Cuetools Coverted\Fear Factory\1995 - Demanufacture (Digipak)\01. Demanufacture.flac
File B: L:\Libraries\Music\Fear Factory - Demanufacture (Digipak)\Fear Factory - Demanufacture.mpc

10:01:32 : Test started.
10:04:18 : 01/01 50.0%
10:04:46 : 02/02 25.0%
10:05:08 : 03/03 12.5%
10:05:19 : 03/04 31.3%
10:05:57 : 04/05 18.8%
10:06:08 : 05/06 10.9%
10:06:22 : 06/07 6.3%
10:06:39 : 07/08 3.5%
10:06:51 : 08/09 2.0%
10:07:52 : 09/10 1.1%
10:08:11 : 09/11 3.3%
10:08:18 : Test finished.

----------
Total: 9/11 (3.3%)


This post has been edited by juscor: Jun 11 2013, 19:17
Go to the top of the page
+Quote Post
db1989
post Jun 11 2013, 19:30
Post #12





Group: Super Moderator
Posts: 5141
Joined: 23-June 06
Member No.: 32180



Good to hear as it will be interesting for others to be able to test/analyse this. Do please ensure that any sample uploaded or linked here is ≤ 30 seconds in length.
Go to the top of the page
+Quote Post
juscor
post Jun 11 2013, 20:34
Post #13





Group: Members
Posts: 8
Joined: 9-June 13
Member No.: 108562



Are there any opus splitters out yet? Trying to find one to make the sample but it seems they don't exist. Should I just convert it to wav and cut it that way instead?
Go to the top of the page
+Quote Post
TomasPin
post Jun 11 2013, 21:09
Post #14





Group: Members
Posts: 167
Joined: 5-June 13
From: Argentina
Member No.: 108508



QUOTE (juscor @ Jun 11 2013, 16:34) *
Should I just convert it to wav and cut it that way instead?

There should be no problem in doing so.
(As long as the audio editor you're using handles the conversion right, and most do)

Edit: maybe you should avoid dithering the output. I know Audacity for instance does that by default and it is not desirable if converting to the same bit depth and sample rate as the source, as it (basically) ads noise which could interfere in the ABX process.

This post has been edited by TomasPin: Jun 11 2013, 21:13


--------------------
A man and his music: http://tubular.net/
Go to the top of the page
+Quote Post
pdq
post Jun 11 2013, 22:18
Post #15





Group: Members
Posts: 3305
Joined: 1-September 05
From: SE Pennsylvania
Member No.: 24233



It would be simpler to just post the (edited) flac file and let other do their own encoding.

How sure are you that this is truly lossless? If at any time it had been lossy encoded then that could cause problems with subsequent lossy encoding.
Go to the top of the page
+Quote Post
Gainless
post Jun 11 2013, 22:50
Post #16





Group: Members
Posts: 169
Joined: 28-October 11
Member No.: 94764



QUOTE (pdq @ Jun 11 2013, 23:18) *
How sure are you that this is truly lossless? If at any time it had been lossy encoded then that could cause problems with subsequent lossy encoding.

Well, if there aren't obvious lowpasses or a reference to compare the peaks we can't be 100% sure anyway.

@OP
Just download Audacity and disable the dither at the "Quality" settings for cutting.

This post has been edited by Gainless: Jun 11 2013, 22:54
Go to the top of the page
+Quote Post
pdq
post Jun 11 2013, 22:57
Post #17





Group: Members
Posts: 3305
Joined: 1-September 05
From: SE Pennsylvania
Member No.: 24233



QUOTE (aztec_mystic @ Jun 10 2013, 17:24) *
QUOTE (ChronoSphere @ Jun 10 2013, 23:19) *
Do you take a lossless song -> convert it to lossy -> load both to abx without dsp/replaygain? In that case the volume should be the same.

Really?

Actually, lame applies a scale factor by default for lower bitrates.
Go to the top of the page
+Quote Post
aztec_mystic
post Jun 11 2013, 23:40
Post #18





Group: Members
Posts: 93
Joined: 28-March 13
Member No.: 107425



QUOTE (pdq @ Jun 11 2013, 23:57) *
Actually, lame applies a scale factor by default for lower bitrates.

Thanks for explaining. At this point, I figured I must've missed something.
Go to the top of the page
+Quote Post
ChronoSphere
post Jun 12 2013, 11:54
Post #19





Group: Members
Posts: 399
Joined: 11-March 07
Member No.: 41384



Question: am I interpreting my results (see above) correctly? As in, mp3 v0 is still not completely transparent to me. Or should I count 38% guessing as transparent?

QUOTE (pdq @ Jun 11 2013, 23:57) *
QUOTE (aztec_mystic @ Jun 10 2013, 17:24) *
QUOTE (ChronoSphere @ Jun 10 2013, 23:19) *
Do you take a lossless song -> convert it to lossy -> load both to abx without dsp/replaygain? In that case the volume should be the same.

Really?

Actually, lame applies a scale factor by default for lower bitrates.
Ah, that would explain why it's so much easier to distinguish then.
Go to the top of the page
+Quote Post
halb27
post Jun 12 2013, 12:03
Post #20





Group: Members
Posts: 2414
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



Yes, 38% guessing means nothing at all.
You should get at 5% guessing or below, and you should decide in advance whether you will do 5 trials (you need a 5/5 result then), or 8 trial (you need a 7/8 result then), or whatever you like to do.


--------------------
lame3100m --bCVBR 300
Go to the top of the page
+Quote Post
db1989
post Jun 12 2013, 12:58
Post #21





Group: Super Moderator
Posts: 5141
Joined: 23-June 06
Member No.: 32180



To look at it from a statistical/experimental perspective: A value of 5% corresponds to a p-value/significance level of 0.05, a generally accepted standard for determining whether observed effects/trends can be statistically distinguished from chance/noise. The p-value represents the probability that the observed result could have occurred purely by chance, i.e. could have occurred under the null hypothesis. The null hypothesis posits that the treatment in question (lossy compression, administration of medication, etc.) has no effect. Thus, by obtaining a p-value below a reasonable significance level, one can suggest with confidence that the null hypothesis is false and therefore that the treatment does have an effect.
Go to the top of the page
+Quote Post
juscor
post Jun 12 2013, 19:08
Post #22





Group: Members
Posts: 8
Joined: 9-June 13
Member No.: 108562



QUOTE (pdq @ Jun 11 2013, 22:18) *
It would be simpler to just post the (edited) flac file and let other do their own encoding.

How sure are you that this is truly lossless? If at any time it had been lossy encoded then that could cause problems with subsequent lossy encoding.

Unless I have false memories, I am certain that this is a lossless encoding ripped directly from CD source.
Plus, I've heard lossy--->losslesss--->lossy encodes before and this definitely doesn't sound like one of them.


QUOTE (Gainless @ Jun 11 2013, 22:50) *
Well, if there aren't obvious lowpasses or a reference to compare the peaks we can't be 100% sure anyway.

@OP
Just download Audacity and disable the dither at the "Quality" settings for cutting.

I ended up using WavSplitter since I've never used audacity and this looked much simpler.


I'll upload the original, uncut lossless version alongside the samples if anybody wants to experiment and encode their own versions. I'm new to this site, so if this violates any TOS, let me know and I will take down the link. I'm uploading as I type this so the links should be up today. It's just going painfully slow.
Go to the top of the page
+Quote Post
db1989
post Jun 12 2013, 19:23
Post #23





Group: Super Moderator
Posts: 5141
Joined: 23-June 06
Member No.: 32180



QUOTE (juscor @ Jun 12 2013, 19:08) *
I'll upload the original, uncut lossless version alongside the samples if anybody wants to experiment and encode their own versions. I'm new to this site, so if this violates any TOS, let me know and I will take down the link.
Take it down? You canít post it here in the first place. Rather than wondering aloud whether doing so would violate any rules, you could go and read them. You were asked to do so during registration, even. All of that aside, I already saved you the toil by posting earlier in this thread what the rules clearly state:
QUOTE (db1989 @ Jun 11 2013, 19:30) *
Do please ensure that any sample uploaded or linked here is ≤ 30 seconds in length.

If your goal is to provide a lossless version for others to use, what exactly is stopping you from trimming that to 30 seconds beforehand?
Go to the top of the page
+Quote Post
juscor
post Jun 12 2013, 21:07
Post #24





Group: Members
Posts: 8
Joined: 9-June 13
Member No.: 108562



QUOTE (db1989 @ Jun 12 2013, 20:23) *
QUOTE (juscor @ Jun 12 2013, 19:08) *
I'll upload the original, uncut lossless version alongside the samples if anybody wants to experiment and encode their own versions. I'm new to this site, so if this violates any TOS, let me know and I will take down the link.
Take it down? You can’t post it here in the first place. Rather than wondering aloud whether doing so would violate any rules, you could go and read them. You were asked to do so during registration, even. All of that aside, I already saved you the toil by posting earlier in this thread what the rules clearly state:
QUOTE (db1989 @ Jun 11 2013, 19:30) *
Do please ensure that any sample uploaded or linked here is ≤ 30 seconds in length.

If your goal is to provide a lossless version for others to use, what exactly is stopping you from trimming that to 30 seconds beforehand?


My apologies, I admit I did not read the TOS for which I have no excuse. I also forgot about your ≤ 30 seconds rule post when I read the other poster's suggestion about uploading the song. I read too quickly and thought others might have wanted to experiment with other portions of the song is all.

Excluding the song the in its entirety, here are the samples as promised. They are 20 seconds in length and in wav:
Lossless
MPC -q 6
Opus 256
Vorbis -q 7

I'll upload the other bitrate versions i tested tonight. I also tested Vorbis at -q 5, and Opus at 160, 144, 128, 96, 80, and 64 if you'd rather encode at these bitrates now using the lossless sample I provided.

NOTE: As I mentioned earlier, the difference between the lossy and lossless version is very subtle (lossy sounds slightly less powerful). Before starting the test, I recommend listening to both very closely to get a feeling for them. Once you notice (if you do) what I'm talking about, it becomes easier to discriminate between the encodes. I also recommend going in with fresh ears since it's hard to concentrate and pick up the differences otherwise.

One more thing, if you don't like heavy metal with harsh vocals, then I advise against listening to my sample. Listening to it over and over again will quickly get on your nerves. Hell, I like the song and it still vexed me. laugh.gif

This post has been edited by juscor: Jun 12 2013, 21:25
Go to the top of the page
+Quote Post
juscor
post Jun 13 2013, 18:22
Post #25





Group: Members
Posts: 8
Joined: 9-June 13
Member No.: 108562



Just tested a slow paced, rather simple rap song at MPC -q 6. 8/11 was my result:
CODE
foo_abx 1.3.4 report
foobar2000 v1.2.6
2013/06/13 08:43:18

File A: L:\Libraries\Music\Gravediggaz - Niggamortis\08 - 1-800 Suicide.flac
File B: L:\Libraries\Music\Gravediggaz - 1-800 Suicide.mpc

08:43:18 : Test started.
08:47:59 : 01/01 50.0%
08:48:07 : 02/02 25.0%
08:48:19 : 02/03 50.0%
08:49:16 : 03/04 31.3%
08:49:40 : 04/05 18.8%
08:49:56 : 04/06 34.4%
08:50:06 : 04/07 50.0%
08:50:16 : 05/08 36.3%
08:51:35 : 06/09 25.4%
08:52:25 : 07/10 17.2%
08:53:55 : 08/11 11.3%
08:54:29 : Test finished.

----------
Total: 8/11 (11.3%)


I think for all intents and purposes, MPC at this quality setting is transparent to me. Given this test result as well as my previous ones, I'll be sticking to this codec for now. I doubt Vorbis or Opus will perform better for me. Also, looking at opus bitrates, it seems that its vbr isn't quality based, but ABR based. I'll give this codec some time to mature before I make the switch.

This post has been edited by juscor: Jun 13 2013, 18:25
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 17th April 2014 - 20:36