Hydrogenaudio Forums > mpc, mp3, ogg ABX @ ~180k

Hydrogenaudio Forums > Hydrogenaudio Forum > Listening Tests

Gecko

Aug 30 2005, 04:44

Hi,

Guruboolez inspired me to do a little testing.

Sample: "Faraway Project - Radiate (Mystery Islands Remix)" taken from "Trancemaster 4006".

Mirror provided by Defsac
Rapidshare

Codecs and settings:
LAME 3.97alpha11 @ -V2 --vbr-new
Musepack 1.15v alpha @ --quality 5
OggEnc v2.6 (Lancer 20050709 based on aoTuV b4) @ -q 6

Soundcard: Terratec Aureon Space with Prodigy drivers (using integrated headphone amp)
Headphones: Beyerdynamic DT 880
Software: Foobar ABX component

The files I performed the ABX test on were encodes of the whole song, not just the small sample I posted. I started with the MPC version and had difficulties. After some more or less unsuccessful trials, I raised the volume a little above comfortable and the artifact became clear. MP3 and OGG tests were solely performed at increased volume. I allways start out with 8 trials. Then I examine the results and decide to do 8 more or quit. On the mpc sample I later decided to add 5 extra rounds to increase relevancy.

The critical section is just at the beginning of the song. The Bass drum, a few hi-hats and some "fuzz". The fuzz sounds something like "zzzz duh duh". It is on those "duh" sounds that the encodes all have more sibilance or a more fuzzy high end.

It is worth noting that in the original there is almost no energy above 16kHz. The hi-hats, which you would expect to go above 16kHz, are perhaps a loop which has been lowpassed at 16kHz. The "duh duh" however goes all the way up to 22kHz.

Since all codecs express similar sibilance, I wonder if it might be an artifact of lowpassing the signal?

Results:
mpc: 13/16 + 5/5 = 18/21
ogg: 8/8
mp3: 8/8

ABX logs:

CODE

foo_abx v1.2 report
foobar2000 v0.8.3
2005/08/23 13:25:07

File A: file://D:\temp\my samples 2005\Faraway Project - Radiate (Mystery Islands Remix).wav
File B: file://D:\temp\my samples 2005\mpc\Faraway Project - Radiate (Mystery Islands Remix).mpc

13:25:20 : Test started.
13:26:39 : 01/01  50.0%
13:26:57 : 01/02  75.0%
13:27:21 : 02/03  50.0%
13:27:31 : 02/04  68.8%
13:27:53 : 03/05  50.0%
13:28:06 : 03/06  65.6%
13:28:37 : 04/07  50.0%
13:28:51 : 05/08  36.3%
13:29:14 : 06/09  25.4%
13:29:33 : 07/10  17.2%
13:30:14 : 08/11  11.3%
13:30:43 : 09/12  7.3%
13:31:05 : 10/13  4.6%
13:31:21 : 11/14  2.9%
13:31:27 : 12/15  1.8%
13:31:47 : 13/16  1.1%
13:31:54 : Test finished.

 ---------- 
Total: 13/16 (1.1%)


foo_abx v1.2 report
foobar2000 v0.8.3
2005/08/30 12:11:27

File A: file://D:\temp\my samples 2005\Faraway Project - Radiate (Mystery Islands Remix).wav
File B: file://D:\temp\my samples 2005\mpc\Faraway Project - Radiate (Mystery Islands Remix).mpc

12:11:41 : Test started.
12:11:52 : 01/01  50.0%
12:12:14 : 02/02  25.0%
12:12:28 : 03/03  12.5%
12:12:47 : 04/04  6.3%
12:12:57 : 05/05  3.1%
12:12:58 : Test finished.

 ---------- 
Total: 5/5 (3.1%)


foo_abx v1.2 report
foobar2000 v0.8.3
2005/08/30 12:06:52

File A: file://D:\temp\my samples 2005\ogg\Faraway Project - Radiate (Mystery Islands Remix).ogg
File B: file://D:\temp\my samples 2005\Faraway Project - Radiate (Mystery Islands Remix).wav

12:07:07 : Test started.
12:07:52 : 01/01  50.0%
12:08:02 : 02/02  25.0%
12:08:40 : 03/03  12.5%
12:08:52 : 04/04  6.3%
12:09:01 : 05/05  3.1%
12:09:09 : 06/06  1.6%
12:09:24 : 07/07  0.8%
12:09:37 : 08/08  0.4%
12:09:43 : Test finished.

 ---------- 
Total: 8/8 (0.4%)


foo_abx v1.2 report
foobar2000 v0.8.3
2005/08/23 20:44:06

File A: file://D:\temp\my samples 2005\Faraway Project - Radiate (Mystery Islands Remix).wav
File B: file://D:\temp\my samples 2005\mp3\Faraway Project - Radiate (Mystery Islands Remix).mp3

20:44:23 : Test started.
20:46:24 : 01/01  50.0%
20:46:48 : 02/02  25.0%
20:47:21 : 03/03  12.5%
20:47:39 : 04/04  6.3%
20:48:18 : 05/05  3.1%
20:52:08 : 06/06  1.6%
20:52:24 : 07/07  0.8%
20:52:54 : 08/08  0.4%
20:52:56 : Test finished.

 ---------- 
Total: 8/8 (0.4%)

guruboolez

Aug 30 2005, 05:06

Thanks for sharing results

Do you have any preference for one encoding? Your test doesn't tell it but only that none of these three encodings are transparent to you.

P.S. I can't download the file. Could someone confirm?

Gecko

Aug 30 2005, 05:26

QUOTE(guruboolez @ Aug 30 2005, 01:06 PM)

Thanks for sharing results

Do you have any preference for one encoding? Your test doesn't tell it but only that none of these three encodings are transparent to you.

I wanted to do that but hesitated. For one I would have to ABX each of the encodes against one another (or do abc/hr) to establish a difference. I was surprised that the artifact was very similar for all encodes, so I guess ABXing them would be very difficult. I was also surprised that no encoder would handle this transparently, being different in psymodel and architecture and what not. See my comment on the lowpassing. As you can see from the logs, ABXing went pretty fast for all of them. I couldn't say one was harder than the other. Mpc maybe was a little harder, but that was also the first encoder I tried, so others had the benefit of training. Apart from the fact that I am probably biased towards mpc.

Without blind testing I am unable to give a preference.

QUOTE

P.S. I can't download the file. Could someone confirm?

In theory, when you click the link above, you get this table with "FREE" and "PREMIUM" buttons at the bottom. Click "Free". Then you will see a countdown near the bottom of the page (usually 20secs) which gives you time to admire the ads. After that a download link appears.

rjamorim

Aug 30 2005, 08:03

What were the bitrates?

Gecko

Aug 30 2005, 12:11

Sorry, forgot those.

Bitrates for the whole song (as reported by foobar)
mpc: 171
ogg: 197
mp3: 203

So mpc fails like all the others, but does so most efficiently.

Triza

Aug 30 2005, 15:35

Glad for your upload. I am constantly looking for killer samples for Vorbis, which is my chosen lossy format.

It is a funny sample.

Just for fun I set my player to play it repeatedly and there is no glich in it when it restarts I would not be able to tell if this is a long boring track or just playing repeatedly. Well done. I bet you use this as a base for some mix.

Thanks,

Triza

Gecko

Aug 30 2005, 16:22

QUOTE(Triza @ Aug 30 2005, 11:35 PM)

Just for fun I set my player to play it repeatedly and there is no glich in it when it restarts I would not be able to tell if this is a long boring track or just playing repeatedly. Well done. I bet you use this as a base for some mix.

Having DJ aspirations, one tends to cut samples beat and bar accurate. However there are still 122 samples of silence at the beginning.

Olive

Sep 3 2005, 06:44

QUOTE(Gecko @ Aug 30 2005, 07:26 AM)

For one I would have to ABX each of the encodes against one another

lol guys I'm inexperienced at this, until what point should I go in order to get relevant results? I'm asking because ABX'ing against the original was relatively fun and quick, but MP3 V2 vs Vorbis q6 is getting tough! I'm at 12/18 (probability that you were guessing: 11.9%) after more than 50 minutes!
(should mention I'm not at home, with a laptop and inexpensive Planctronics headphones- will provide more details when I'm done)

Defsac

Sep 3 2005, 06:54

Sent you a PM re hosting (rapidshare only lets me download the first 3 seconds of the file for some reason).

Gecko

Sep 3 2005, 07:00

QUOTE(Olive @ Sep 3 2005, 02:44 PM)

lol guys I'm inexperienced at this, until what point should I go in order to get relevant results? I'm asking because ABX'ing against the original was relatively fun and quick, but MP3 V2 vs Vorbis q6 is getting tough! I'm at 12/18 (probability that you were guessing: 11.9%) after 70 minutes!
(should mention I'm not at home, with a laptop and inexpensive Planctronics headphones- will provide more details when I'm done)

If you don't hide the results, then you need to fix the number of trials beforehand. You can't just stop when you've reached the desired probability.

Try this: run abx and just randomly choose X is A or X is B. At some point you will get results that look relevant, but you weren't even listening to the samples.

Olive

Sep 3 2005, 07:27

I had in mind getting below 1% prob of guessing whatever how many trials but yes I guess you're right I can't just stop when I'm satisified with the result

Gecko

Sep 3 2005, 07:46

QUOTE(Defsac @ Sep 3 2005, 02:54 PM)

Sent you a PM re hosting (rapidshare only lets me download the first 3 seconds of the file for some reason).

OK, thanks, link added to first post.
http://www.crapshack.com/files/FarawayProj...emix)_clip.flac

Seymour

Sep 17 2005, 10:40

Gecko

QUOTE

it might be an artifact of lowpassing the signal?

I see no lowpassing with Lancer on the spectral graph... Am I blind?

Well, I've done some my own tests with this sample coded by Musepack 1.15v --standard and Vorbis Lancer aoTuV beta 4 -q6. It was very hard to ABX Musepack (as it was always for me

)... I've decided to hide results and to make 16 tryouts (I beg my pardon for one more tryout with Musepack).

CODE

foo_abx v1.2 report
foobar2000 v0.8.3
2005/09/17 20:15:03

File A: file://E:\My Received Files\FarawayProject-Radiate(MysteryIslandsRemix)_clip.flac
File B: file://E:\My Received Files\FarawayProject-Radiate(MysteryIslandsRemix)_clip.flac.OGG

20:15:03 : Test started.
20:15:26 : 01/01 50.0%
20:15:33 : 02/02 25.0%
20:15:41 : 02/03 50.0%
20:16:09 : 03/04 31.3%
20:16:21 : 04/05 18.8%
20:16:31 : 05/06 10.9%
20:16:49 : 06/07 6.3%
20:16:56 : 07/08 3.5%
20:17:03 : 08/09 2.0%
20:17:25 : 09/10 1.1%
20:17:39 : 09/11 3.3%
20:18:12 : 10/12 1.9%
20:18:20 : 11/13 1.1%
20:18:27 : 11/14 2.9%
20:18:54 : 12/15 1.8%
20:19:49 : 13/16 1.1%
20:20:02 : Test finished.

----------
Total: 13/16 (1.1%)

This is Vorbis test. Bitrate is 198 kbps.

CODE

foo_abx v1.2 report
foobar2000 v0.8.3
2005/09/17 20:20:12

File A: file://E:\My Received Files\FarawayProject-Radiate(MysteryIslandsRemix)_clip.flac
File B: file://E:\My Received Files\FarawayProject-Radiate(MysteryIslandsRemix)_clip.flac.MPC

20:20:12 : Test started.
20:20:35 : 01/01 50.0%
20:21:48 : 01/02 75.0%
20:21:56 : 01/03 87.5%
20:22:31 : 02/04 68.8%
20:22:37 : 02/05 81.3%
20:22:53 : 02/06 89.1%
20:23:01 : 02/07 93.8%
20:23:18 : 03/08 85.5%
20:23:32 : 04/09 74.6%
20:23:58 : 05/10 62.3%
20:24:04 : 05/11 72.6%
20:24:12 : 05/12 80.6%
20:24:19 : 06/13 70.9%
20:24:28 : 07/14 60.5%
20:24:35 : 08/15 50.0%
20:24:41 : 09/16 40.2%
20:24:48 : 10/17 31.5%
20:24:59 : Test finished.

----------
Total: 10/17 (31.5%)

And this is Musepack at 193 kbps.

Even if I sense a difference, it is hard and my ears very quick get tired...

KikeG

Sep 17 2005, 12:24

QUOTE(Olive @ Sep 3 2005, 02:27 PM)

I had in mind getting below 1% prob of guessing whatever how many trials but yes I guess you're right I can't just stop when I'm satisified with the result

That would work too up to 100 trials or so.

This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.