Help - Search - Members - Calendar
Full Version: Vorbis Listening Test
Hydrogenaudio Forums > Lossy Audio Compression > Ogg Vorbis > Ogg Vorbis - General
Pages: 1, 2, 3, 4
harashin
I made bitrate tables from my own several albums include various styles of music. -q4 should be fair to the 128kbps test.
aotuv_tables.htm
QuantumKnot
I've modified the oggenc source to use only a '.' (dot, fullstop) in the quality levels, should we decide to use fractional quality values.

[Link removed]

Test to see if it gives the same output as aoyumi's binary. I ran into a few errors when compiling his source and made a few rectifications to fix this.

EDIT: Found a bug in the quality parsing...
EDIT 2: Fixed the quality parsing bug
QuantumKnot
For those who want better pre-echo handling in aoTuV, I've merged only the pre-echo tunings (q 2 to 5) from QKTune beta 3.2 with aoTuV.

[Link removed]
harashin
QUOTE(QuantumKnot @ Apr 11 2004, 09:19 AM)
Test to see if it gives the same output as aoyumi's binary.  I ran into a few errors when compiling his source and made a few rectifications to fix this.

They seem to produce different files here. I'm not sure if the difference is audible though.
user posted image
QuantumKnot
QUOTE(harashin @ Apr 11 2004, 11:03 AM)
QUOTE(QuantumKnot @ Apr 11 2004, 09:19 AM)
Test to see if it gives the same output as aoyumi's binary.  I ran into a few errors when compiling his source and made a few rectifications to fix this.

They seem to produce different files here. I'm not sure if the difference is audible though.

Oh no ohmy.gif I used VC7 to compile.

The changes I made to Aoyumi's original code are:

In mapping0.c, mapping0_forward() function:

CODE
//int nc_db[n/2];
 int *nc_db = (int *)calloc(n/2, sizeof(int));


Since n is not constant, static declaration of that array won't work, so I did a dynamic allocation.

Also, there was a variable declaration in the middle of a block which VC7 complained about, so I moved it to the beginning of the block.

sad.gif

EDIT: I've emailed Aoyumi my patched oggenc.c file so that he can do the compile himself.
guruboolez
Using different compilers isn't sufficient to obtain encoders that produces different output files? IIRC, there were always OBJECTIVE difference between lame release (Mitiok vs Dibrom vs John33), though this difference was never audible. Am I wrong? I've no experience at all on software compiling...
Aoyumi
Locale fix binary upload was built and carried out using "oggenc.c" which QuantumKnot corrected. A binary can be downloaded from test page.

Moreover, the difference of a binary is a difference of a compiler. I am using GCC. And an optimization option is also a standard range (default).

Furthermore, since it was a thing that it cannot compile well by VC, correction was added and re-uploaded to the source code. someone check whether this moves normally? (I do not have VC) The binary which builds this source code and is obtained brings the same encoding result as a front thing. smile.gif
bond
QUOTE(QuantumKnot @ Apr 11 2004, 01:56 AM)
For those who want better pre-echo handling in aoTuV, I've merged only the pre-echo tunings (q 2 to 5) from QKTune beta 3.2 with aoTuV.

http://www.rarewares.org/quantumknot/oggencaqk.exe

great! can we use this one for the listening test?
Garf
I'd guess that would need at least retesting the bitrate and doing the listening test all over :-/
bond
QUOTE(Garf @ Apr 11 2004, 10:30 AM)
I'd guess that would need at least retesting the bitrate and doing the listening test all over :-/

why? could this hurt quality?
Garf
QUOTE(bond @ Apr 11 2004, 11:34 AM)
QUOTE(Garf @ Apr 11 2004, 10:30 AM)
I'd guess that would need at least retesting the bitrate and doing the listening test all over :-/

why? could this hurt quality?

Any tuning brings the risk of an unexpected sideeffect...
Latexxx
Isn't it enough to compare this new thing to original version and check which one is better?
Garf
QUOTE(Latexxx @ Apr 11 2004, 11:43 AM)
Isn't it enough to compare this new thing to original version and check which one is better?

Yes, a listening test in other words. You also need to check whether it doesnt change the average bitrate (and possibly use new setting to encode with), which is exactly what I already said in my first post.
Aoyumi
I am Sorry, There was a serious mistake for locale fix version and source code.
Now, it is substituted for the normal thing.
If some people already downloaded, I will ask you to eliminate the older one.

QUOTE
why? could this hurt quality?

Bad influence will come out of hack of the stereo contained in QKTune depending on the case. It is clear at the low bit rate.
If it is the transplant of only pre-echo and tuning, it should succeed in general (however, it cannot be guaranteed).
sad.gif
guruboolez
I've few lossless albums on my computers. I've compared the bitrate on two different ones:

- cello concertos from Joseph Haydn: highly tonal, fews attacks. Conclusion: both encoders produces exactly the same bitrate (according to foobar2000).

- works for mandolins from Antonio Vivaldi: more sharp attacks. Conclusion: bitrate of AoTuV+QK is clearly higher, up to 11 kbps on two track. Difference on album: +4-5 kbps.

It will be interesting to compare the bitrate with metal or rock discs, mith more percussive instruments.

(P.S. I didn't look for quality)
maikmerten
QUOTE(guruboolez @ Apr 11 2004, 11:56 AM)
It will be interesting to compare the bitrate with metal or rock discs, mith more percussive instruments.

CODE

Artist: Dream Theater
Album: Images And Words
Year: 1992
Genre: Progressive Metal

1.0.1 -q 4,25 | aoTuV -q 4 | aQK -q 4

129,6  128,1  137,7
134,1  132,9  141,2
130,4  128,4  140,8
128,2  127,1  134,8
132,9  130,6  145,1
131,7  129,3  142,7
120,6  119,6  122,9
132,7  130,6  144,5

Artist: Iron Maiden
Album: Brave New World
Year: 2000
Genre: Heavy Metal

1.0.1 -q 4,25 | aoTuV -q 4 | aQK -q 4

129,2  126,5  141,1
128,3  126,2  138,5
130,3  127,8  140,8
132,3  129,8  143,7
130,9  129,1  141,2
129,0  127,2  139,0
128,9  126,4  140,0
130,7  129,4  139,8
131,3  129,0  141,9
131,1  128,6  140,8

Artist: Judas Priest
Album: Rocka Rolla
Year: 1974
Genre: (early) Heavy Metal

1.0.1 -q 4,25 | aoTuV -q 4 | aQK -q 4

128,2  125,8  135,1
131,6  129,2  143,5
126,7  123,8  129,2
128,7  127,0  135,0
122,4  119,8  121,7
130,4  127,3  140,0
125,5  123,1  129,5
127,5  125,1  134,8
128,0  125,5  135,0
123,1  120,2  133,4
121,3  120,6  129,0


aQK = aoTuV + QK

Bitrate is a bit too high for a ~128 kbps test IMO.
guruboolez
maikmerten> thanks a lot. This is confirming my suspicions.
The "problem" with Garf/QK tuning, it's the gap existing between tonal music and percussive music. There's usually a small difference in bitrate between classical and other kind of music (~8...10 kbps, something like that on average). With QK tuning, the difference is much higher for the same quality setting. It's not a problem in real life, but for this test, it's a serious one. If we lower the setting in order to match 128 kbps, it will lower for sure the quality for tonal samples, like BachS1007 for exemple (because Garf/QK tuning have no quality effect with this kind of music). In other word, with AoTuV+QK we will probably gain quality on some (percussive) samples, but we will also lose (audibly?) quality on other. Really problematic.
A solution would be to keep -q4, and accept the bitrate difference. But...

...I'm really far from sharing the opinion of people asking for bitrate exact match on the tested samples, but I know that the discussion always appears after the test. Here is the bitrate table of both encoders, on the 12 samples used for the AAC test

CODE


AoTQK    AoTuV

162      136
147      127
134      117
137      126
121      118
131      126
123      119
135      129
152      137
170      138
165      138
143      135

AVR      AVR
143.33   128.83


There is virtually no discussion possible for vorbis AoTuV (really close to 128 kbps). I fear that some people will complain about methodology if AoTuV+QK will be the vorbis competitor.


EDIT: clarifications (I hope so).
guruboolez
Other thing: my version of AoTuV+QK oggenc is broken. I've encoded samples with -q3,5 and -q3,7, and bitrate was terribly low (100 kbps on average). Quality is simply awful. -q4 is working perfectly, but at lower bitrate, there's a big problem.

EXEMPLE (extreme): velvet.wav

-q4 = 165 kbps
-q3,99 = 84 kbps
Latexxx
QUOTE(guruboolez @ Apr 11 2004, 03:17 PM)
Other thing: my version of AoTuV+QK oggenc is broken. I've encoded samples with -q3,5 and -q3,7, and bitrate was terribly low (100 kbps on average). Quality is simply awful. -q4 is working perfectly, but at lower bitrate, there's a big problem.

EXEMPLE (extreme): velvet.wav

-q4 = 165 kbps
-q3,99 = 84 kbps

Did you test using the ./, hacked version? If you did, try -q3.99 .
guruboolez
I don't see a second uploaded version of AoTuV+QK. Am I wrong?
Big_Berny
Hi,
I saw the bitratebug too:

Die Ärzte - Unrockbar (Aotuv-QK):
-q4 ---> bitrate: 134.4kbps (Encoder shows quality 4.000000)
-3.99 ---> bitrate: 116.4kbps (Encoder shows quality 3.990000)
-3,99 ---> bitrate: 96.1kbps (Encoder shows quality 3.000000)

This shows that the encoder doesn't support "," but it's normal that the difference between 4 and 3.99 is so high? Does q4 uses other optimations than q3.99?

EDIT: OUPSSSS, I think I did something wrong. Wait some minutes...
EDIT2: No! My results were true and I just saw, that the q3.99 sounds very crazy! I think there is a bug in the encoder!

Big_Berny
guruboolez
QUOTE(Big_Berny @ Apr 11 2004, 08:09 PM)
This shows that the encoder doesn't support "," but it's normal that the difference between 4 and 3.99 is so high? Does q4 uses other optimations than q3.99?

It's clearly a bug. Whatever the bitrate value is (100 or 130 kbps), the quality of 3.99/3,99 is horrible (lowpass is alterning between 18 Khz and 2 Khz !!!).
Big_Berny
Also the oggenc-aotuv is broken!

q4 ---> bitrate: 126 (sounds ok)
q3.99 ---> bitrate: 117 (sounds very strange!)

No I did no ABX! But this bug is very audible also for my bad ears!

Big_Berny
p0l1m0rph1c
Here, it doesn't make any difference (the . or , ).

Carlos Paredes - Movimento Perpétuo
q4 -> bitrate: 174 kbps
q3.99 -> bitrate: 110.8 kbps
q3,99 -> bitrate: 110.8 kbps

Still, there's clearly an issue with q3.99

EDIT: and what an issue! It's a serious problem, very, very audible.
Big_Berny
I tested aoyoume's reference-encoder! Fortunatly there is no bug! (Only that the "," doesn't work here...)
But q3.99 is a little bit bigger than q4 (128kbits vs. 126kbits)...

I would say that roberto should use the reference-encoder with q4...
Big_Berny
rjamorim
Oh, my... sad.gif
maikmerten
QUOTE(rjamorim @ Apr 11 2004, 08:01 PM)
Oh, my... sad.gif

I seriously hope you won´t have a nervous breakdown wink.gif








finally: My 100th post.
guruboolez
QUOTE(rjamorim @ Apr 11 2004, 09:01 PM)
Oh, my... sad.gif

Don't worry. It's probably a small mistake, easy-to-fix.
Anyway, there's no evidence that AoTuV+QK should be use. We must test it before, and find the ideal bitrate setting (of course, if you prefer take the responsability of using an untested version of vorbis, you could do it).

In my opinion, AoTuV beta 2 is a wise choice. For two reasons:
- -q4 could be use without problem. 128 kbps seems to be reached an average, for daily use as well for the short samples selected for the next test. The QK modified version is more problematic (opting for a quality setting inferior to -q4/128 would probably be questionned by some people, as weel as big 128 kbps overrun).

- AoTuV beta 2 was tested, and approved by four persons. It's not a strong collective approbation, but it's better than nothing. AoTuV+QK didn't compete. Bugs are always possible. It's the work of two indepedant developers. An hybrid encoder. Epistemologicaly speaking, it's very problematic to use something like that, even if the output quality is possibly better than AoTuV. If we found a bug after the test, the whole seriousness of the listening test might be ruined.

On the other side, both AoTuV beta2 and AoTuV+QK have a very short life expectancy. In few weeks, something like AoTuV beta 3 will be released. Vorbis 1.1 (major update) too... therefore, the conclusions of the next multiformat test about vorbis will quickly be invalidate. Whatever the choice of the codec for the test.
QuantumKnot
Gosh, I think I've created a stir.

Firstly, can I say that the aoTuV + QKTune was just an experimental encoder and wasn't meant to be included in the multiformat test. I guess I should have stated that but I did mention it in my earlier reply to guruboolez' questions about including pre-echo tunings into aoTuV and how it wouldn't be able to make the multiformat test.

Quoting myself:
QUOTE
Yes, it should be quite easy to add pre-echo tunings to aoTuV though it is not going to be able to make the multiformat test.


I thought I'd give it a try for those interested but it's still too early to use it. Also, it only includes changes to values that affect pre-echo. There have been no point stereo changes.

Secondly, Aoyumi found the bug in my "rectified" source which is causing the problem, I think. Therefore I suggest that people use the binaries on his test page rather than the ones I made.

Just for clarification, please use the binaries at

http://www.geocities.jp/aoyoume/aotuv/test

Aoyumi included the locale fix I emailed him so this version only uses a '.' (dot, fullstop). Again, please make sure through testing of his binary (not mine) that the locale fix is not giving any strange bugs

Sorry for the confusion.
QuantumKnot
QUOTE(Aoyumi @ Apr 11 2004, 06:43 PM)
Furthermore, since it was a thing that it cannot compile well by VC, correction was added and re-uploaded to the source code. someone check whether this moves normally? (I do not have VC)

Just to confirm, the source code from your test page now compiles flawlessly in VC smile.gif
nyaochi
I'm very sorry I could not contribute to this listening test because I had been busy recently. sad.gif I read aoTuV 20040402 was chosen for the upcoming multi-codecs listening test. I'm in favor of the decision too. smile.gif Although I had finished half of the test, I threw it away and began again another test, including oggencaqk -q4.

Here's my listening result:
user posted image
Garf
QUOTE(guruboolez @ Apr 11 2004, 11:18 PM)
In few weeks, something like AoTuV beta 3 will be released. Vorbis 1.1 (major update) too... therefore, the conclusions of the next multiformat test about vorbis will quickly be invalidate.

Unless it's "Vorbis weeks' and 'Vorbis quick' smile.gif
guruboolez
smile.gif
Aoyumi is nevertheless faster, and he updates his encoder more often.
maikmerten
QUOTE(Garf @ Apr 12 2004, 01:20 PM)
Unless it's "Vorbis weeks' and 'Vorbis quick' smile.gif

"God - give me patience. NOW!" tongue.gif
rjamorim
Oh, my, what a mess.

I'm about to start doing bitrate deviation tests and encoding. As I understand it, I am supposed to be using this binary:
http://www.geocities.jp/aoyoume/aotuv/ogge...x_20040402m.zip

If I'm wrong, please shout ASAP. And God help us all...
bond
QUOTE(rjamorim @ Apr 12 2004, 04:06 PM)
I'm about to start doing bitrate deviation tests and encoding. As I understand it, I am supposed to be using this binary:
http://www.geocities.jp/aoyoume/aotuv/ogge...x_20040402m.zip

so guys thats important now! we want vorbis to behave at the best possible way, right?

so everyone following this discussion closely get your nuts together, forget all the unnecessary discussions and tell rjamorim if he uses the right binary now!!!



rjamorim,
the version you linked to handles the ./, problem correctly
-q4,25 output 129kbps
-q4 output 123kbps on a small test here
detokaal
Yes.
QuantumKnot
QUOTE(guruboolez @ Apr 12 2004, 11:29 PM)
smile.gif
Aoyumi is nevertheless faster, and he updates his encoder more often.

Plus we should leave open the possibility that aoTuV may still sound better than the upcoming Vorbis "1.1". smile.gif I've had a quick scan over Aoyumi's modifications and they are quite interesting.

Nyaochi:
Don't worry. This test did come up very suddenly. And thanks for the results too. Another thumbs down for 1.0.1 smile.gif

Roberto:
Yes, that is the binary that uses a . (dot) only. And since it's a compile by anyone except me, it's gonna work fine. laugh.gif
MadXviD
QUOTE(QuantumKnot @ Apr 12 2004, 07:32 PM)
And since it's a compile by anyone except me, it's gonna work fine. laugh.gif

Although I don't use Vorbis myself, I really appreciate what you've done in the past months so please don't say those things!.

What you're doing here is unvaluable and don't forget you're human, cheer up and keep providing us your good Vorbis work.

edit: though I have to admit that was funny laugh.gif .
ff123
QUOTE(QuantumKnot @ Apr 12 2004, 03:32 PM)
Plus we should leave open the possibility that aoTuV may still sound better than the upcoming Vorbis "1.1".  smile.gif  I've had a quick scan over Aoyumi's modifications and they are quite interesting.

I still don't quite understand the argument that the official Xiph vorbis implementation is supposed to be a "reference" encoder and that's why it may not fold in improvements.

Doesn't it hurt Vorbis overall to be fractioned into competing implementations? So now there needs to be a "recommended binary" for Vorbis encoders, which might even vary by bitrate? Doesn't seem optimal to me.

ff123
bond
hm now that rjamorim postponed the test we can maybe test and prepare a merged version

maybe a merged version, using the best parts from all tunings, would give the best results...
what do you guys think?
maikmerten
QUOTE(ff123 @ Apr 14 2004, 05:42 PM)
Doesn't it hurt Vorbis overall to be fractioned into competing implementations?  So now there needs to be a "recommended binary" for Vorbis encoders, which might even vary by bitrate?  Doesn't seem optimal to me.

I can see your point.

Vorbis 1.0.2 is in CVS - just needs to be released AFAIK. Vorbis 1.1 will certainly take some time. There is probably enough time for Vorbis 1.0.3 - perhaps someone being in contact with xiph.org could ask if this may be a "community powered" version using some 3rd-party tuning.

This tuning has to meet some requirements:

- It should sound better (of course)
- it should not inflate bitrates

Benefits for xiph.org:

- new version "for free"
- prove Vorbis is open
- pressure for Vorbis 1.1 is lowered

Just my 2 cents...
QuantumKnot
QUOTE(ff123 @ Apr 15 2004, 03:42 AM)
Doesn't it hurt Vorbis overall to be fractioned into competing implementations?  So now there needs to be a "recommended binary" for Vorbis encoders, which might even vary by bitrate?  Doesn't seem optimal to me.

ff123

I guess without competition, there can be no progress. smile.gif

I already pointed to the possibility of a forking of Vorbis in a previous thread but the point is that the various Vorbis tunings still produce compatible files. They can be played on any standard Vorbis decoder. The only things we have changed are constants affecting various things that determine quality (that are there to be modified). Incompatibility between the various versions is detrimental but we are not seeing that with Vorbis.

The only concern one has is that everyone is free to modify the Vorbis source code however they feel like and are not obliged to release it as open-source. Hence it is possible for Vorbis-variants or forks to appear which are incompatible with Xiph Vorbis, yet still carry the "built on Vorbis" tag, and perhaps include patented and proprietary modifications. Very hypothetical at this stage but forking is still a possibility.
dev0
Forking the format or forking the encoder?
I don't think forking the format would be beneficial to anybody.
QuantumKnot
QUOTE(bond @ Apr 15 2004, 05:04 AM)
hm now that rjamorim postponed the test we can maybe test and prepare a merged version

maybe a merged version, using the best parts from all tunings, would give the best results...
what do you guys think?

I think aoTuV is as good as it gets. Nyaochi's results on aoTuV+QKTune don't indicate any sizable improvements which dispels the hope that the pre-echo issues that guruboolez raised could be solved by a simple merging. smile.gif
guruboolez
About pre-echo: I compared AoTuV against 1.01 CVS on some samples (like mandolin), and AoTuV souned sharper. Can't say if it's really pre-echo improvements, but the noise reduction seems to affect in a positive way the sharpness feeling (at least, on some samples).
Anyway, improvements are really nice. At mid/low bitrate (~96 kbps), vorbis AoTuV is probably the best encoding solution I ever heard (better for my taste than he-aac, at least -again- on some samples).
Nice work smile.gif
QuantumKnot
Excellent. I'm gonna re-rip my copy of Vivaldi's Four Seasons using aoTuV. biggrin.gif

And since I use CDex, I made these DLLs from aoTuV which can be downloaded from:

http://www.rarewares.org/quantumknot/aoTuV-dlls.zip

in case anyone is interested in using it. smile.gif
maikmerten
I have build a complete set of aoTuV RPMs for RedHat 9... please let me know if anybody is interested.
maikmerten
QUOTE(dev0 @ Apr 14 2004, 11:47 PM)
Forking the format or forking the encoder?
I don't think forking the format would be beneficial to anybody.

Forking the encoder. There would be no benefit from changing the bitstream format IMO (it´s already flexible enough).
maroonmike
QUOTE
Excellent. I'm gonna re-rip my copy of Vivaldi's Four Seasons using aoTuV


Sorry, but I am a little confused... huh.gif

At the risk of jumping the gun, what 'q' settings should be used for aoTuV vs GT3b2? (like q0-q4 = aoTuv; > q4 = GT3b2)

Also, is a merger between these two encoders being proposed???
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2008 Invision Power Services, Inc.