IPB

Welcome Guest ( Log In | Register )

4 Pages V  < 1 2 3 4 >  
Reply to this topicStart new topic
ITU-R BS.1387 Analysis Tool finally available, under GPL!
Speek
post Jan 5 2002, 02:37
Post #26





Group: Members
Posts: 394
Joined: 31-October 01
Member No.: 386



What is the offset for FhG. Fastenc (CoolEdit Pro filter)? With the test Ivan described I think it's about 96 (that's 192 for EAQUAL) when decode with LAME, but when I put an higher offset number in EAQUAL the ODG gets better. Wouldn't it be logical that the offset that gives best ODG is the right offset :confused:
Go to the top of the page
+Quote Post
Ardax
post Jan 5 2002, 04:05
Post #27





Group: Members
Posts: 233
Joined: 3-December 01
Member No.: 578



Here's some more ODG values for Speek's samples, except one, because it's smarter than me. smile.gif

(Speek, I can't decode sade_sweetest_taboo.pac. LPAC says that it's not a valid LPAC file. Did I pick the wrong decoder?)

These tests were done with lame 3.91 using --alt-preset 128 and oggenc rc3 -q 4.00. This was to test files that were nominally targeting 128 kbps, since that's the file size I want, and I don't want to futz with command lines until they're exactly 128 kbps (although that's useful information too). I just want to cram in the command line that gives me the best quality near 128 kbps. Quick and easy.

lame 3.91 --alt-preset 128
fools.wav : 126.4 kbps : ODG -0.9395
ftb_samp.wav : 121.1 kbps : ODG -1.3005
iron.wav : 126.9 kbps : ODG -0.8146

oggenc rc3 -q 4.00
fools.wav : 121.9 kbps : ODG -0.8032
ftb_samp.wav : 131.0 kbps : ODG -0.8292
iron.wav : 123.6 kbps : ODG -0.8476

Interesting results, to look at. Oggenc does a pretty good job at looking "consistent." I think I'm going to go play some Civ III now. biggrin.gif
Go to the top of the page
+Quote Post
Xiphiod
post Jan 5 2002, 04:25
Post #28





Group: Members
Posts: 5
Joined: 10-October 01
Member No.: 260



Here's some real world numbers with ogg:

Drowning Pool - Bodies
GT2 192.0 -.36184*
5 160.2 -.3340
5.5 180.8 -.0975
6 201.2 .0290

Primus - Mrs. Blaileen
5 145.0 -.3014
5.5 161.8 -.0781
6 177.7 .0332

Propellerheads - Bang On!
5 152.0 -.3271
5.5 170.0 -.0932
6 187.0 .0182

*Garf Tuned RC2 @ 160
Go to the top of the page
+Quote Post
CiTay
post Jan 5 2002, 04:27
Post #29


Administrator


Group: Admin
Posts: 2376
Joined: 22-September 01
Member No.: 3



Hmm, why do you use the "sade__sweetest_taboo" sample anyways? It was a sample i sent to Roel about a year ago, because of some lowpass issues. This sample isn't really a "hard" sample for encoders... or was it that what you wanted, an everyday-sample?
Go to the top of the page
+Quote Post
ff123
post Jan 5 2002, 05:36
Post #30


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



I tested my dogies.wav encodes using the analysis tool, and here are the results:

ODG

xing: -1.35
wma: -1.46
lame: -1.24
ogg: -1.48
aac: -1.15
mpc: -0.73

The listeners rated the files as follows:

xing: -2.61
wma: -1.77
lame: -1.75
ogg: -1.63
aac: -0.85
mpc: -0.74

In other words, xing was much worse than wma/lame/ogg, which in turn was much worse than aac/mpc. The analysis tool does not seem to rate the files the same way.

BTW, I went through a lot of trouble when I first made the WAV sample to time align all the files, and I just now double-checked to make sure that the samples were all aligned. The ogg file was shifted by 1 sample (23 microseconds), so I shifted it into alignment, but the numeric results didn't change.

I wouldn't necessarily declare results from this program infallible yet smile.gif

ff123
Go to the top of the page
+Quote Post
dosdan
post Jan 5 2002, 10:20
Post #31





Group: Members
Posts: 18
Joined: 8-October 01
Member No.: 246



To those using the w32 LAME 3.90.2 binary. I've noticed that
the decoded.wav generated by "lame --decode test.mp3 decoded.mp3" while not having a offset at the front may have added trailing silence so that the size of decoded.wav is slightly bigger than test.wav

This will cause incorrect ODGs particularly on smaller test files say 1MB. I've been improving accuracy by removing the trailing silence in CoolEdit. The extra size varies from 100 bytes to 3KB so far. So check those decoded wav file sizes.
Go to the top of the page
+Quote Post
Speek
post Jan 5 2002, 11:39
Post #32





Group: Members
Posts: 394
Joined: 31-October 01
Member No.: 386



QUOTE
Speek, I can't decode sade_sweetest_taboo.pac. LPAC says that it's not a valid LPAC file. Did I pick the wrong decoder?

I see, the file got truncated with uploading. I'm uploading it again now.

QUOTE
Hmm, why do you use the "sade__sweetest_taboo" sample anyways? It was a sample i sent to Roel about a year ago, because of some lowpass issues. This sample isn't really a "hard" sample for encoders... or was it that what you wanted, an everyday-sample?

So that's were it came from! I was just picking some of the longer testfiles I have. Most are from the LAME test samples webpage, but I saw that this one wasn't. In the readme of EAQUAL it says that a test sample should be at least 10 to 20 sec., but the longer the better. Many test samples are to short.

QUOTE
I wouldn't necessarily declare results from this program infallible yet smile.gif


No, I wouldn't to. I'm just getting some data. I don't know were it leads to. One of the things I'm thinking about is: Ogg seems to get the best ODG scores. Is it really the best encoder, or does it something better than other encoders that EAQUAL pays much attention to? Maybe Ogg does something else worse than some other encoders, but EAQUAL doesn't pay much attention to that factor? Just open questions, I don't have a clue smile.gif
Go to the top of the page
+Quote Post
Ivan Dimkovic
post Jan 5 2002, 11:57
Post #33


Nero MPEG4 developer


Group: Developer
Posts: 1466
Joined: 22-September 01
Member No.: 8



It is using 1024 point FFT window for the analysis - so it could miss some pre-echo events now and then.

It is ranking psytel velvet.aac (encoded with 2.01) better than Dolby Professional AAC encoding, and at least to my ears Dolby sounds better (not too much, but I can hear the difference) smile.gif
Go to the top of the page
+Quote Post
Alexander Lerch
post Jan 5 2002, 13:00
Post #34


zplane.development Compaact! developer


Group: Developer
Posts: 65
Joined: 4-January 02
Member No.: 918



Hi all.

Hope you enjoy EAQUAL.
Sorry about the confusion about the offset, it was a bug and now it is fixed (and available at sourceforge). Now the offset is really counted in samples. If you use the old version of EAQUAL, you have to multiply the number of samples with the number of channels.

Alexander


--------------------
zplane.development
http://www.zplane.de
Go to the top of the page
+Quote Post
Alexander Lerch
post Jan 5 2002, 13:02
Post #35


zplane.development Compaact! developer


Group: Developer
Posts: 65
Joined: 4-January 02
Member No.: 918



Hi!

You have to use any -scale switches very carefully. The algorithm is stable for small level differences, but it will suck for big level differences.
Scaling can be very useful because many decoders tend to clip full scale audio output - in this case the test signal would be distorted. Here you should use scaling to avoid wrong ratings with clipped audio signals.

Alexander


--------------------
zplane.development
http://www.zplane.de
Go to the top of the page
+Quote Post
Jan S.
post Jan 5 2002, 13:02
Post #36





Group: Admin
Posts: 2548
Joined: 26-September 01
From: Denmark
Member No.: 21



QUOTE
of the things I'm thinking about is: Ogg seems to get the best ODG scores.


Hmmm... that's not what I see. From the results on your site it looks like mpc -xtreme is the best.


Jan.
Go to the top of the page
+Quote Post
Alexander Lerch
post Jan 5 2002, 13:05
Post #37


zplane.development Compaact! developer


Group: Developer
Posts: 65
Joined: 4-January 02
Member No.: 918



My last message now smile.gif

If you must use EAQUAL for codec ratings (which is not really its intention), please keep in mind that you have to average over _many_ audio files to get reasonable results - and with good reasons you may doubt those ratings too. You have to average results over least 20, 50 or 100 signals I would guess.

Alexander


--------------------
zplane.development
http://www.zplane.de
Go to the top of the page
+Quote Post
Speek
post Jan 5 2002, 13:53
Post #38





Group: Members
Posts: 394
Joined: 31-October 01
Member No.: 386



QUOTE
Originally posted by Jansemanden
 

Hmmm... that's not what I see. From the results on your site it looks like mpc -xtreme is the best.


Jan.


Look again smile.gif Most times Ogg Vorbis -q 6 gets a better score than MPC -xtreme. The times MPC gets a better score it has a much higher bitrate.

BTW. a positive ODG is very good according to Alexander Lerch (he is the author of EAQUAL).

BTW2. I'm not saying Ogg is better than MPC, I'm just saying it gets a better score from EAQUAL on the samples I tried.
Go to the top of the page
+Quote Post
Ivan Dimkovic
post Jan 5 2002, 13:57
Post #39


Nero MPEG4 developer


Group: Developer
Posts: 1466
Joined: 22-September 01
Member No.: 8



Hehe... competition is now having to tune psymodels of their encoders to match EAQUAL values smile.gif hehehe (just kidding)

Did anybody run EAQUAL on WMA? I know it sucks, but just for a quick check smile.gif
Go to the top of the page
+Quote Post
Speek
post Jan 5 2002, 14:13
Post #40





Group: Members
Posts: 394
Joined: 31-October 01
Member No.: 386



QUOTE
Originally posted by Ivan Dimkovic
Hehe... competition is now having to tune psymodels of their encoders to match EAQUAL values smile.gif hehehe (just kidding)

Did anybody run EAQUAL on WMA? I know it sucks, but just for a quick check smile.gif


What bitrate would you like?
Go to the top of the page
+Quote Post
Ivan Dimkovic
post Jan 5 2002, 14:37
Post #41


Nero MPEG4 developer


Group: Developer
Posts: 1466
Joined: 22-September 01
Member No.: 8



96 and 128, but WMA is a VBR codec - it sometimes gives too large bitrate, greater than nominal...
Go to the top of the page
+Quote Post
Speek
post Jan 5 2002, 15:31
Post #42





Group: Members
Posts: 394
Joined: 31-October 01
Member No.: 386



I've justed added WMA8 results for 128 kb/s. It doesn't suck to hard according to EAQUAL, but the bitrates are a bit higher than the others.
Go to the top of the page
+Quote Post
Ivan Dimkovic
post Jan 5 2002, 15:37
Post #43


Nero MPEG4 developer


Group: Developer
Posts: 1466
Joined: 22-September 01
Member No.: 8



QUOTE
Originally posted by Speek
I've justed added WMA8 results for 128 kb/s. It doesn't suck to hard according to EAQUAL, but the bitrates are a bit higher than the others.


Well, try psytel aacenc with -noshort switch (disables short blocks) - you'll notice that it also performs "good" but the results are unacceptable from the pre-echo point of view smile.gif
Go to the top of the page
+Quote Post
Skeeve242
post Jan 6 2002, 02:28
Post #44





Group: Members
Posts: 8
Joined: 27-December 01
Member No.: 778



If you use EAC's "Process File" function and remove leading & trailing silence, would that eliminate the offset problem for EAQUAL?
Offset will be the same for all alligned files?


bye, Skeeve
Go to the top of the page
+Quote Post
dosdan
post Jan 6 2002, 04:25
Post #45





Group: Members
Posts: 18
Joined: 8-October 01
Member No.: 246



QUOTE
Originally posted by Skeeve242
If you use EAC's "Process File" function and remove leading & trailing silence, would that eliminate the offset problem for EAQUAL?
Offset will be the same for all alligned files?


The offset in question is not caused by leading or trailing digital silence in the original (ripped) wave file. The offset is caused when it converted to an MP3 and then reconverted back to a wav. MP3 can have overlap of info between frames e.g. when you decode the first frame of a MP3 you can't be sure you have all the data for the very start of the music info. I believe there is feame overlap both in encoding and decoding but if you know what this delay is you can remove it. If you use LAME to decode a LAME encoded file these constant values are known to the program and it can compensate when decoding i.e.

skipping initial 1105 samples (encoder+decoder delay)


Not sure about the trailing digital silence that I'm seeing from the decoded file. This was not on the original and has been added after decoding so EAC could never strip it. The amount of added digital silence varies so it is not a constant number of samples to be removed.
Go to the top of the page
+Quote Post
dosdan
post Jan 6 2002, 04:33
Post #46





Group: Members
Posts: 18
Joined: 8-October 01
Member No.: 246



QUOTE
Originally posted by Skeeve242
If you use EAC's "Process File" function and remove leading & trailing silence, would that eliminate the offset problem for EAQUAL?
Offset will be the same for all alligned files?


The offset in question is not caused by leading or trailing digital silence in the original (ripped) wave file. The offset is caused when it converted to an MP3 and then reconverted back to a wav. MP3 can have overlap of info between frames i.e when you decode the first frame of a MP3 you can't be sure you have all the data for the start of the data stream. I believe there is overlap both in encoding and decoding but if you know what this delay is you can remove it. If you use LAME to decode a LAME encoded file these constant values are known to the program and it can compensate when decoding i.e.

skipping initial 1105 samples (encoder+decoder delay)


Not sure about the trailing digital silence that I'm seeing from the decoded file. This was not on the original and has been added after decoding so EAC could never strip it. The amount of added digital silence varies so it is not a constant number of samples to be removed.
Go to the top of the page
+Quote Post
Speek
post Jan 6 2002, 14:16
Post #47





Group: Members
Posts: 394
Joined: 31-October 01
Member No.: 386



I've added some more results to my EAQUAL page:
http://home.wanadoo.nl/~w.speek/eaqual.htm
Go to the top of the page
+Quote Post
cd-rw.org
post Jan 6 2002, 14:24
Post #48





Group: Members
Posts: 176
Joined: 5-October 01
Member No.: 217



Just out of curiosity, could someone also give --r3mix a shot.


--------------------
http://www.bitburners.com - We Burn a Bit
Go to the top of the page
+Quote Post
Alexander Lerch
post Jan 6 2002, 16:30
Post #49


zplane.development Compaact! developer


Group: Developer
Posts: 65
Joined: 4-January 02
Member No.: 918



QUOTE
Originally posted by ff123
I tested my dogies.wav encodes using the analysis tool, and here are the results:

In other words, xing was much worse than wma/lame/ogg, which in turn was much worse than aac/mpc.  The analysis tool does not seem to rate the files the same way.

BTW, I went through a lot of trouble when I first made the WAV sample to time align all the files, and I just now double-checked to make sure that the samples were all aligned.  The ogg file was shifted by 1 sample (23 microseconds), so I shifted it into alignment, but the numeric results didn't change.

I wouldn't necessarily declare results from this program infallible yet smile.gif

ff123


You have to keep in mind, that it is difficult already to compare results from different listening tests, even if they were realized ITU-R BS.1116-compliant.
Perhaps you should think of EAQUAL as _one_ listener whose results are not depending on his mood and his concentration, who never gets tired and whose results are repeatable, though only his opinion.
BTW, what are the confidence intervals for your listeners?

The difference of one sample has indeed very little influence on the result. If you want to interpret this in the frequency domain, the big difference will only be there for frequencies near Nyquist. These frequencies will not influence the result much.

And it's true, no model can be infallible smile.gif

Alexander


--------------------
zplane.development
http://www.zplane.de
Go to the top of the page
+Quote Post
ff123
post Jan 6 2002, 17:23
Post #50


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



The results of the dogies listening test can be found at:

http://ff123.net/dogies/dogies_plots.html

With an experiment-wise confidence of 95%, MPC and AAC were deemed better than the others, and LAME/WMA8/OGG were deemed better than XING. N = 15.

The Fisher's protected LSD is known to not be "protected" if the number of samples exceeds 3, but I verified via resampling analysis that the results are indeed as given above with 95% confidence.

In other words, the listeners ranked the various codecs on this sample with clear and statistically significant preference.

ff123
Go to the top of the page
+Quote Post

4 Pages V  < 1 2 3 4 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 17th April 2014 - 23:04