I have done a few tests between Vorbis 1.1 and Vorbis aoTuVb3 at around 64kbps, to possibly help with selection between the two in the upcoming multiformat 64kbps test.

First test: kraftwerk



Here both encoded samples are very easy to ABX. It is very difficult to ABX between the two encoded samples, but aoTuVb3 seemed to add a bit of extra noise (very very small amount) a few seconds into the sample.

Second test: SinceAlways



A bit more interesting here. aoTuVb3 seems to be easier to ABX vs the original. Easier to ABX between the two encoded samples as they have artifacts in difference places. 1.1 has larger HF boost on the crash cymbals, but aoTuVb3 has strange "warped" sound on some of the hi-hats.

Overall, Vorbis 1.1 seems to be winning.

More tests to come...

Good to read that. I thought Aotuv was only really tuned for higher bitrates? That seems to confirm it, but I am still a little surprised.

Aoyumi worked on -q0...-q-1 and created -q-2 preset. AoTuV can't be tuned for high bitrate only

Hmm yeah good point. Merci vilmal min Hèrr, or however they say it in Elsässich.

The reason I was asking was that certain codecs can be tuned for lower bitrates: HE-AAC, mp3PRO, or the early days of Vorbis... or does HE-AAC actually perform better at high bitrates too ( < 192 Kbps average)?

SBR is only good for bitrates up to 96 kbps. LC AAC wins it at higher bitrates.

No (not with current encoders).



Latexxx was faster.

I am glad that you started this thread Phreakazoid and hope that other tests will follow soon, so that the best Vorbis encoder will be used for the test.

While testing should focus on 1.1 and AoTuV3, you might want to give Archer a try, but only if you have time.

I've done another test, this time with Archer RC4 in the mix. One of the listed features of the aoTuVb3 release was improved pre-echo handling, so I've used the castanets sample as a test of this behaviour.

Results:



All of the encoded samples sounded pretty terrible (as was expected), but it seems that aoTuVb3 and Archer RC4 both improved a little over 1.1. In the Vorbis 1.1 sample, there was a bit of extra noise on the castanet rattle, compared to aoTuVb3 and Archer RC4.

For this test... equal winners are aoTuVb3 and Archer RC4.

I announced new aoTuV pre-beta4.
It should solve the problem which is conspicuous by aoTuV beta3.
Please test.

http://www.geocities.jp/aoyoume/aotuv/test.html

aoTuV pre-beta4 [20050328] release.

I corrected disorder of a small sound in specific conditions.

http://www.geocities.jp/aoyoume/aotuv/test.html

I released the newest version of aoTuV pre-beta4 [20050403]. Unless I or other someone find a big problem, there will not be formal beta4 and a big difference.

Thanks for your effort Aoyumi!

Tests with this version are of course welcome.

Would it be possible to get the source in order to compile it on linux? I guess it's not a problem as the source of the experimental version is provided on the site.

Thanks!

The reason I do not upload a source code is that it is not adjusting more than q1 with having scribbled the comment.

About the experiment version, the version to which I open the source code is performing minimum tests and minimum checks.

please wait.

Again, I would like to ask everyone who is interested to conduce more Vorbis listening tests. The 64 kbps multi-format test will most probably start late April or early May. Tiger is supposed to come out on April 29th and with it the Apple HE-AAC encoder.

Just noticed a new AoTuv pre-beta4 has been released : http://www.geocities.jp/aoyoume/aotuv/test.html. With the source

I uploaded oggenc(win32) of pre-beta4 now.
It includes the new tuning by all the bit rates.

Anything about the date of the final release of Aotuv Beta 4, Aoyumi?

When is going to be a "beta-stable"?

And just for curiosity, it have really big important tunings? like the tunings from 1.0.1 to the beta2... for being the next vorbis release (1.2.0?).

And thanks for you great work, I really (and many others) appreciate it.

The release time of beta4 is dependent on the degree of advance of adjustment and a test.
I want to release by the beginning in May.


Yes, it is a change important for me as a user at least. New code and tunings are included in beta 3/4. Although beta3 solved some problems, there was a case where the problem of channel coupling was highlighted as a result. And beta4 was made in order to solve the problem of beta3.
For example, beta3 makes noise control more dynamic, in order to reduce pre-echo. It affected control of channel coupling, therefore the problem was produced. By correction of this channel coupling processing, those problems are solved in general.

About an official vorbis release, nothing can be said from me. For the moment, I am not involving.

[EDIT] TYPO

Aoyumi:
As far as I know, Monty loves your job, he seems to rely on you to keep the house during the time he will code for vorbis II nowadays ... last time I saw him he was already planning to test the beta3 new low bitrates mode ... so there is no doubt he will swallow beta4 with 1.1 post svn if ABXing test are good ... specially if there is a 64kbps listening test & aob4 wins ... not only Monty, but other Xiph dev seems to push Monty to swallow aotuv quick ... anyway we'll remind him of you if he ever forgets

Who says that AoTuV4 will be tested? I am still waiting for some ABX tests to decide which Vorbis encoder to be featured.

So far, I have three test results: AoTuV won once, Xiph twice.

Sebastian Mares:
yes indeed ... I was just planning the best that can happen ... no one says aob4 will win the overall test too, if selected

I'm currently testing xiph 1.1.0 vs archer RC4 vs aoTuv PB4 to help decide which one should be used in the upcoming 64kbps tests.
I just wanted to post a preliminary report after 18 of 37 samples. puh this takes time
I used foobar to extract lossless sample clips to wav, and decode all vorbis clips back to wav. "foobar2000 v0.8.3 unicode version Compiled on: Jun 28 2004"
Gear: Sennheiser HD600 driven by asus P4P800 mainboard onboard audio (analog devices soundmax)

So far, I've only found 2 samples where I could abx the difference between vorbis tunings/versions.

First one is "Autechre - Gantz Graf (02)"
http://moozooh.nm.ru/samples_for_ha/ae_gg02.ape

ABX Results:
Original vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\aoTuV_pre_b4_20050412\2.wav.wav
8 out of 8, pval = 0.004
Original vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\OggEnc_SSE_20050320ArcherRC4\2.wav
8 out of 8, pval = 0.004
Original vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\oggdropXPdV1.8.3-generic\2.wav
8 out of 8, pval = 0.004
C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\aoTuV_pre_b4_20050412\2.wav.wav vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\OggEnc_SSE_20050320ArcherRC4\2.wav
8 out of 8, pval = 0.004
C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\aoTuV_pre_b4_20050412\2.wav.wav vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\oggdropXPdV1.8.3-generic\2.wav
8 out of 8, pval = 0.004

easy to abx. transient smear, stereo colapse. xiph/archer got alot of "treble boost" effect. aotuv pb4 much closer to the original. xiph and archer sounds identical.


The other one is "Deep_Purple___When_A_Blind_Man_Cries"
http://www.hydrogenaudio.org/forums/index....pe=post&id=1428

ABX Results:
Original vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\aoTuV_pre_b4_20050412\10.wav.wav
16 out of 16, pval < 0.001
Original vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\oggdropXPdV1.8.3-generic\10.wav
17 out of 17, pval < 0.001
Original vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\OggEnc_SSE_20050320ArcherRC4\10.wav
16 out of 16, pval < 0.001
C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\aoTuV_pre_b4_20050412\10.wav.wav vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\oggdropXPdV1.8.3-generic\10.wav
23 out of 32, pval = 0.010
C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\aoTuV_pre_b4_20050412\10.wav.wav vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\OggEnc_SSE_20050320ArcherRC4\10.wav
25 out of 32, pval = 0.001
C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\oggdropXPdV1.8.3-generic\10.wav vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\OggEnc_SSE_20050320ArcherRC4\10.wav
12 out of 12, pval < 0.001

only slight difference between xiph and archer, but I prefer xiph. tiny bit less flange effect. 12 out of 12 might seem it was easy, but it was very hard to abx.
It could be that the flanging was not in sync, and made it possible for me to tell them apart. overall sound was very close.
aotuvPB4 worst of them. Much easier to abx between aotuv and xiph/archer than between xiph and archer.
Got some spells of listening fatigue, that's why they go up to 32tests.

I would say the win by aoTuv PB4 on the autechre transient test was bigger than the win xiph 1.1.0 got on the deep purple track.

It's obvious to me that my ears are more sensitive to some effects than others. I hope more people can do tests, and atleast do tests on these two tracks to verify my findings.
I'll post again when I've completed all the 37 sample clips

Thanks for your efforts pusle, very much appreciated.

I did some comparison between 1.1 and aoTuV pb4, which was very hard for me. I wanted to use samples employed by the former 64Kbps listening test, but I could not find the conspicuous difference between 1.1 and aoTuV pb4. So I gave up using these samples and switched to my daily listening samples. Then I found the following samples from several albums that have differences among the two encoders.

"Take Me Out", "Franz Ferdinand" by Franz Ferdinand [sample]


"Mint", "Even So" by Bonnie Pink [sample]


"Genjitsu ni oite", "Kyoiku" by Tokyo Jihen [sample]


"Fuyu no sanatorium", "Hajime Chitose" by Hajime Chitose [sample]


My impression is that it's safe to use aoTuV pb4 for the upcoming listening test because: I didn't notice any severe regression (although 1.1 is slightly better than pb4 for Mint sample) during the quick evaluation; the improvement for "Genjitsu ni oite" sample is quite positive; and Aoyumi himself, who conducted the continuous effort for the low-bitrate range, recommends pb4 over beta 3 according to his note.

I have now listened to 37 samples + the 4 samples in the post above by nyaochi. My main conclusion is that my hearing isn't that good

Most of the tracks was abx'able vs the original, but hearing differences between xiph and archer was close to impossible.
Of the 37 samples I could only abx between xiph and aotuvPB4 on 2 samples.
I thought I could hear differences for like 6-7 of them, but not abx able below a usable margin.
Mostly get it right the first 5-6 tries then I might just have been kidding myself or getting listening fatigue.

I agree with nyaochi that this is very hard. For his 4 samples I could only abx the mint one. But my conclusion is the opposite of his.
It comes down to preference in artifacts, but I think aoTuv pb4 was closest to the original in overall "sound".

The main info I want to get across is this:
if you hate the treble boost effect, and listen to alot of transients go for aoTuvbp4
but then you get more stereo wobble and swoshy sound effects in the background.
xiph seems to preserve less stereo, but more stable sound with less stereo wobble/swoosh. Worse on some types of transients.
However the differences are so small it borders on academic for the most part in my opinion.

What DOES stand out is the typical vorbis artifact of tonal vibrato effect. Which after debate in another thread seems to be linked to how vorbis handles stereo.
This effect completely dwarfs everything else to my ears, and is readily apparent on almost all tracks where stereo is used heavily, like live recordings etc.
the pre-echo , noise modulation, treble boost and background swosh are not annoying to me at this bitrate. The vibrato/wobble effect is , big time.
I could hear no difference between the candidates with regard to this effect.
I repeat my conclusion from another thread where one track was encoded as dual mono with same bitrate and sounded better than normal vorbis:
it is here where the potential for improvements in vorbis really lies and where people doing tunings should put their effort.

Now, if I had to choose one of them I'd choose aoTuv pb4, as it for the most part maintains the "sound" from the original slightly better to my ears.
But since my ears apparently isn't that golden, I wish some guru's might take some time off to test a few samples before the listening test starts

I've just done a quick retest of the SinceAlways sample, using vorbis 1.1, aoTuvb3 and this time adding aoTuV pb4 20050412. Here are the results:



Interestingly, I noticed different artifacts to the last time I tested this sample in this thread. aoTuV pb4 sounded the best of the three, this time around. It sounded less coarse and less stereo collapse, compared to the other two.