IPB

Welcome Guest ( Log In | Register )

4 Pages V  < 1 2 3 4 >  
Reply to this topicStart new topic
CELT 0.9.1 is out!, Help wanted
IgorC
post Feb 5 2011, 02:50
Post #51





Group: Members
Posts: 1506
Joined: 3-January 05
From: Argentina, Bs As
Member No.: 18803



0.11.0 has been released. Good news.

I wonder that there is still a little interest in CELT on this forum.
Apparently it has already better quality than Vorbis and it's open source alternative to HE-AAC/LC-AAC.
It can be good for internet radio, video streaming, archiving the audio with high quality as well.

This post has been edited by IgorC: Feb 5 2011, 02:54
Go to the top of the page
+Quote Post
sauvage78
post Feb 5 2011, 04:54
Post #52





Group: Members
Posts: 677
Joined: 4-May 08
Member No.: 53282



Personnaly I have some interest in it, but I need a win32 command line encoder/decoder to test it my way. I am moderately interested in testing how it competes nowadays against aotuv & nero aac in a self-made single test, but I am not interested to help for development in the long run... specially if it doesn't compete well & specially on already encoded wavs which are not my own selection of killer samples. (In short I am interested in a very selfish test serving my own needs & I don't care what others may think of my test). Anyway I am not in a hurry to test it because if ever it would beat aotuv (which would mean it has improved a LOT since V0.52 ... which it certainly has considering how many updates there were since V0.52) then it will be frustrating to have all the aotuv zealots on my back for teaching them that Vorbis would likely be dead ... I wish that CELT would be the new messiah but having seen in the past how musepack/vorbis/aotuv development has stalled when their developer interest slowed down, I suspect that, sooner or later (hopefully later), CELT will follow the same fate, so honestly I am much more interested by an open source aac equivalent to x264 developed by a team, which I think would be more future proof, then I will ever be by a xiph codec developed by a loner (no matter how clever he is). My only problem with aac is its non-native & non-robust gapless support via tricky deletable tags. I like the robust vorbis gapless support & I think CELT might inherit this which is why I still have some interest in it. AAC already perfectly suits my needs for blu-ray audio part encoding because gaplessness is not needed there.
If ever CELT beats aotuv then we will end up with a better-then-vorbis codec with no [f2k-cuetools-eac3to-mkv-5.1-rockbox] support, which would be an annoying (but hopefully temporary) situation.

All this put together made that if I do have some interest in CELT, I always plan to test it ... later. I use flac for CD & nero aac for blu-ray & CELT alone is unlikey to change this. My small interest for CELT is due to 2 possibility both with unlikely probability:
1: I run out of HDD space because my 5 HDD sata (the 6th one is for SDD) are full, then I will need CELT. (Only if I don't favor lossywav)
2: Webm becomes better than x264, then I will need CELT. (... but I don't believe in Santa Claus anymore)

As you see CELT beeing trapped in the family of Xiph codecs, & this family of codec being non-competitive (x264 beat webm/theora, & nero aac beats aotuv), even if CELT alone would become the perfect lossy codec (the quality of nero aac with the robust gaplessness of vorbis) that doesn't mean it would be enougth for me to use it, because CELT is "bonded by blood" to a sub-optimal family of video codecs. (Using it would mean mixing mpeg with xiph in mkv which I tend to avoid.)

I know it may sound discouraging, but I only explain all that for you to understand why I didn't anwser to your request for me to test CELT one month ago.
Just like lossywav, I like CELT on paper, but I just don't use it in real life actually.

Sorry for not answering your pm directly a month ago (jmvalin), I was too lazy & short of time to write this down. Furthermore I was fearing that you wouldn't like much my opinion about the actual state of lossy codecs.

To sum up according to me, CELT is in a very paradoxal situation, where it might have become a real challenger for aotuv & nero aac, but because of its missing features/missing support/non-mpeg status, even if it would be the best codec around, I would likely not use it ... well at last not untill an eco-system was built around it.

I don't want to be the only guy on earth knowing that CELT is (maybe) the best pure lossy codec around, so I just don't test it to avoid the headache ... I know it's a vicious circle, but it's not my problem, I leave the frustration to ... IgorC [no you don't need to thks me, man wink.gif]

This post has been edited by sauvage78: Feb 5 2011, 05:20


--------------------
CDImage+CUE
Secure [Low/C2/AR(2)]
Flac -4
Go to the top of the page
+Quote Post
jmvalin
post Feb 5 2011, 05:20
Post #53


Xiph.org Speex developer


Group: Developer
Posts: 473
Joined: 21-August 02
Member No.: 3134



A few things I'd like to say here about CELT to answer some of questions and address some misconceptions. First, CELT has gone a long, long way since 0.5.2. In fact, there were more changes since 0.5.2 than there were from the beginning to 0.5.2. Among the things that changed is that CELT now supports longer frame sizes than you tested earlier (20 ms vs 5 ms) though the smaller frame size is still supported. Another difference is that it now supports VBR. On top of that, there's been a huge number of other changes. I'm sure if you tested again, the results you would get would be a lot different than with 0.5.2. I recommend you listen to the samples on Monty's CELT demo page (if you hear clicks in the samples when played from the web page, it's a browser bug and you should download them directly), which has some comparisons with Vorbis and AAC. The 32 kb/s and 48 kb/s samples aren't that useful (below the sweet spot), but the 64 kb/s ones should give a good idea of the quality. Of course, CELT also scales up to 510 kb/s for stereo. Also, if you look at IgorC's results, you'll see that CELT actually beats Vorbis aotuv on most samples, though of course the results vary from one sample to another.

As for you comment about "As you see CELT beeing trapped in the family of Xiph codecs", you should know that CELT is actually one half of the Opus that is currently being standardized at the IETF. The other half is Skype's SILK codec. So practically speaking, CELT is likely to see very wide deployment, starting with Skype. As I've pointed out before, CELT's main focus is *interactive* audio, which means it can give very good audio quality for real-time applications (unlike MP3, Vorbis and HE-AAC). To achieve that we've had to do some sacrifices, but even despite that, the quality is competitive with that of the AAC family. And the same codec can actually work from 6 kb/s (speech) to 510 kb/s, rather than having different profiles for different bit-rates and applications (HEv1, HEv2, LC, LD, ELD). So being standardized at the IETF means we're no longer talking about just a single developer or organization.

As for the tools, don't worry, that's the next thing on the list once the bit-stream is frozen. Which brings up another point. The bit-stream is currently in soft-freeze, so there's a short window of time where the codec is in a pretty good state, but we can still fix any issue that would be discovered before the bit-stream is officially frozen. So I definitely recommend you try CELT again. If binaries are a problem, I may be able to get someone to make Win32 ones. Alternatively, I can provide you with encoded samples.
Go to the top of the page
+Quote Post
sauvage78
post Feb 5 2011, 06:20
Post #54





Group: Members
Posts: 677
Joined: 4-May 08
Member No.: 53282



One of the reason why I wait for a CLI encoder/decoder to test CELT is because I have a pack of around 50 supposed killer samples that were uploaded by other ABXers on HA & that I need to test in order to know which one I can or cannot ABX personnaly. So actually I cannot really tell you what I will test or not (other than the samples I used in my first test). Before I would even start testing CELT there would be a first round of samples selection. Most likely I will test these 50 samples on aotuv 96-128Kbps & select those which are abnormally bad for the bitrate, only then will I test those samples on CELT. In the end I will only publish result for those I am 100% sure that I can ABX (A handfull). I am not the guy trying to ABX randomly selected average music on average bitrate, IMHO it is useless. I will stress the codec to the maximum.

Also I don't want to be tied to your release timing, in the past I already tried to help lossywav development & I stopped because each time I published some ABX results Nick.C instantly published a new lossywav version & it was a neverending game. (It was interesting in the beginning but it is tiring & I get bored in the end).
Nowaday I don't want to help any lossy codec for development as actually I simply don't use lossy anymore (except Nero aac for blu-ray).
If you release a CLI encoder/decoder, sooner or later, I will very likely test CELT by curiosity, but not because you asked me. This will happen naturally just because it is interesting to test how all lossy codecs evolved every year (or ... two years nowadays). So this is also one of the reason why I didn't test CELT again so far, both nero aac & aotuv development have been almost frozen since my test (there were only very minor update), & I think that except for CELT my old test results are still overall valid, which is why I haven't updated it. A multi-codec test with lot of code move is much more exciting than a single codec test, currently the lack of updates from competitors doesn't help to motivate me testing CELT.

Sorry, I don't really care if it would be the right time to test CELT just because there will be a bitstream freeze. Also I cannot promise that I will do it instantly as you releases binaries (in the past already I promised some listening test to Nick.C that I never did). ABXing is a boring task (specially if you don't use the codec yourself) & I will do it when I fell in the mood for it. All I can say is that if you would have regulary released binaries it's a long time I would already have re-tested it.

Sorry if it sound annoying, ABXing is just way more enjoyable for me, if it's independant & free.


--------------------
CDImage+CUE
Secure [Low/C2/AR(2)]
Flac -4
Go to the top of the page
+Quote Post
IgorC
post Feb 5 2011, 06:34
Post #55





Group: Members
Posts: 1506
Joined: 3-January 05
From: Argentina, Bs As
Member No.: 18803



QUOTE (sauvage78 @ Feb 5 2011, 02:20) *
Sorry if it sound annoying, ABXing is just way more enjoyable for me, if it's independant & free.


I can't disagree.
One should test on his own music. The music that you listen and only you can provide some interesting feedback on the music that you know how to listen.

Go to the top of the page
+Quote Post
[JAZ]
post Feb 5 2011, 11:40
Post #56





Group: Members
Posts: 1706
Joined: 24-June 02
From: Catalunya(Spain)
Member No.: 2383



@sauvage:

I don't understand you.
First, you put doubts on how much it has improved since 0.5.2, when there have been several threads with samples to test the progression.
Next, you say that you don't care, because you would like to test with your own samples, and for that you need an encoder/decoder.
Then, jmvalin corrects some misconceptions that you expressed and explains why you should expect a clear improvement since 0.5.2, if you cared.
And then, you insist that you don't have interest in it, because the codec has been a moving target, not like aotuv or nero aac.

Sincerely, I was going to get the source and try to compile either with mingw or visual studio (whatever was easier, since i haven't seen the sources yet), but your comments sort of discouraged me to do so.

You know.. its sort of going to a forum that talks about a beta of a program, and saying that you hate betas and don't trust the developers.
Go to the top of the page
+Quote Post
jmvalin
post Feb 5 2011, 12:53
Post #57


Xiph.org Speex developer


Group: Developer
Posts: 473
Joined: 21-August 02
Member No.: 3134



QUOTE (sauvage78 @ Feb 5 2011, 00:20) *
Sorry if it sound annoying, ABXing is just way more enjoyable for me, if it's independant & free.


Well, you can do it any time you like. What I'm saying is that if it happens within the next month or so, then you can *also* have a potential impact on the codec. Though in fact even after, you could at least impact the encoder. IgorC's earlier testing was in that regard very very useful. Between him, Monty and a few others, I was able (in about 2-3 months) to increase the quality from clearly inferior to Vorbis to better than Vorbis on many/most samples.
Go to the top of the page
+Quote Post
sauvage78
post Feb 5 2011, 13:28
Post #58





Group: Members
Posts: 677
Joined: 4-May 08
Member No.: 53282



QUOTE
you put doubts on how much it has improved since 0.5.2
? I have no doubt it has improved a lot, the problem is more did it improve so much that it is now on par with well established competitors ? ... sorry when it come to ABXing, the only guy I trust more than myself is /mnt (because I often agree with him on bad samples, & when I don't obviously ... it means he is more skilled than me), not that I don't trust other ABXer, but when you start ABXing for yourself, except for finding new killer samples easyly, you don't need the opinion of others anymore. IIRC, it happened that I already disagreed with IgorC about the relative quality of aotuv vs. nero aac in the past, so I take his results with caution, not that I would cast a shadow on him, but IIRC using a different methodology & different samples, he ended with a different conclusion than me (using other samples & methodology). Maybe he was right & maybe I would have agreed with him if I would have run the exact same test. Maybe he is more sensitive than me on some kind of artefact, I don't know. All I know is that I trust myself, as pretentious as it may sound, simply because it's me who will listen to my own encodes, & not the ABXer with golden ears which tells me that this codec is good or bad.

QUOTE
because the codec has been a moving target
? I don't care about the codec being a moving target, as long as you don't request me to run after it.

... well in case I wasn't clear, I am not some kind of philantropic guy wasting my time testing codecs just for fun, if I test codecs that's because I expect some personnal benefits from using them !!! In the past I have spent some time testing experimental codecs more or less as an audio enthousiast would do, just for fun. Years ago I tested CELT out of pure curiosity (back then it happened that it was very bad but it was very experimental so in fact I didn't even considered it as a competitor), but for various reasons this time is over for me. Not that I don't want to do ABXing anymore, but it has to be worth the effort. I have no doubt that CELT did improve a LOT (specially has I know how bad it was), but the overall state of lossy codecs is rather depressing IMHO as most well established codecs have stagnated since my old test. So what I am trying to explain is that actually my personnal gain for testing it is low as I am unlikely to use it (I admit that nowadays I am more a mpeg guy, not that I particulary like patents but sadly the quality of x264 & nero AAC speaks for itself, & I like the safety of the industry support to mpeg, I also admit that I completely changed my mind on this topic as when I started ABXing (Edit: Before I started) I was a vorbis fan). In the end, even when it may have seemed that I was randomly playing at ABXing experimental codecs, I had IMHO a big personnal gain from testing so many different lossy codecs: I have learned exactly what I expect from a lossy codec & I expect so much that sadly none of them fully pleases me & ... I use lossless. But don't think it's about quality, the quality of nero aac q0.55 (~192Kbps VBR) is enought for my taste: I don't use lossy because no lossy codecs has my 3 requirements: (1:Transparent Quality)+(2:Native Robust Gaplessness)+(3:Future Proof: Big Norm). Only lossywav can almost fullfill all my (quality+features) needs, but it's twice the HDD space ...

So to come back to CELT, yes I do have some interest in testing it, the biggest question for me being how close/far it will be from aotuv & if it is slowly friendly killing vorbis in the shadow ? Answering this question alone is interesting, even if the other codecs didn't evolve, but whatever is the answer in the end I may not use CELT personnaly so I don't care if I test it here & now. Maybe Vorbis will, but I won't die if I don't test CELT soon. I known I will re-test it sooner or later anyway, my goals & my timing are just not the same as jmvalin, he would like me to test CELT before he froze the bitstream, but I expect to test it slowly within 1 or 2 months after he releases a windows binarie. I just don't want to spoil my fun.

Anyway I wanted to apologize for not answering his invitation a month ago, which was cowardice.

This post has been edited by sauvage78: Feb 5 2011, 13:59


--------------------
CDImage+CUE
Secure [Low/C2/AR(2)]
Flac -4
Go to the top of the page
+Quote Post
jmvalin
post Feb 5 2011, 15:18
Post #59


Xiph.org Speex developer


Group: Developer
Posts: 473
Joined: 21-August 02
Member No.: 3134



If you want to see how far CELT has come, there's no need to argue for hours about is. Here's CELT at 64 kb/s and here's Vorbis at 64 kb/s. The original is here.
Go to the top of the page
+Quote Post
IgorC
post Feb 5 2011, 18:39
Post #60





Group: Members
Posts: 1506
Joined: 3-January 05
From: Argentina, Bs As
Member No.: 18803



QUOTE (jmvalin @ Feb 5 2011, 01:20) *
If binaries are a problem, I may be able to get someone to make Win32 ones. Alternatively, I can provide you with encoded samples.

If there will be binaries than I will can do an extensive test on many samples and make detailed conclusions as performance per genres, different kind of artifacts, stereo image, soundstage, comparison with other codecs. Or you can suggest what aspects are need to be tested.

If you think that celt is ready for this kind of test then binaries are good. However if you still need some results for particular samples than you can provide these samples as well.


PS. It might also be true that comparison between CELT and Vobris, AAC at higher bitrates (128 kbps and above) can be misleading. CELT has new superior technics but Vorbis and AAC encoders are very well tuned during the years.

This post has been edited by IgorC: Feb 5 2011, 19:17
Go to the top of the page
+Quote Post
list
post Feb 6 2011, 19:25
Post #61





Group: Members
Posts: 27
Joined: 7-September 10
From: Argentina
Member No.: 83667



any future improve towards 32kbps range?
Go to the top of the page
+Quote Post
jmvalin
post Feb 7 2011, 04:00
Post #62


Xiph.org Speex developer


Group: Developer
Posts: 473
Joined: 21-August 02
Member No.: 3134



QUOTE (list @ Feb 6 2011, 13:25) *
any future improve towards 32kbps range?


It has improved a lot and will continue to improve. That being said, I doubt it'll reach what I consider acceptable quality level. For the record, I'm not aware of any codec that produces acceptable stereo music quality at 32 kb/s. The closest may be HE-AAC v2, but I still find the quality to be way too low to be acceptable.
Go to the top of the page
+Quote Post
NullC
post Feb 7 2011, 05:42
Post #63





Group: Developer
Posts: 200
Joined: 8-July 03
Member No.: 7653



QUOTE (IgorC @ Feb 5 2011, 09:39) *
PS. It might also be true that comparison between CELT and Vobris, AAC at higher bitrates (128 kbps and above) can be misleading. CELT has new superior technics but Vorbis and AAC encoders are very well tuned during the years.


You can say that. For example, the CELT encoder doesn't have an explicit psy-model. The format is designed so that one isn't strictly necessary, but a good one could be very helpful for VBR rate selection, right now all the VBR really does for CELT is improve transients. We haven't included an explicit psy-model because we've been focused on things that can't be done after the bitstream is final. The encoder also doesn't perform any analysis which would increase latency or which would be too computationally expensive for embedded devices, e.g. it doesn't use anything more than 2.5ms lookahead and it doesn't switch frame sizes on the fly (except the binary short/long decision). But future and alternative encoders can and will.

So there should be plenty of room for future improvements remaining.

Though I'm not so confident at comparisons with AAC and Vorbis at higher rates for another reason: This is not at all what CELT was designed to do. We built it for very low-latency, so that you could have awesome quality teleconferencing and telephony, remote music, low bandwidth digital microphones and headsets, etc. By some happy accident it's looks like its almost competitive with the best popular high latency codecs at least at some bitrates. But most implementations these codecs have the benefit of well tuned VBR engines, so on high rates with killer samples I expect that they'll do much better than CELT.

But you can not use AAC (non-LD) or Vorbis for telephony, it's a non-starter. They're useless for this. Among the codecs that can be used for low latency CELT is very good. In strict hard-CBR mode (no-bitres) CELT is very good, etc.

If you care about audio quality, then you should care about CELT not because it's going to be the best hifi codec for storing your music collection (These days you're crazy if you're not just using lossless for that purposes) but because you're going to find it all around you for other purposes— embedded in telephones and teleconferencing systems and in VoIP apps, used for wireless audio systems, in video games for realtime audio as well as sound effects, etc.

In any case, since people asked... I create a win32 build of the 0.11.0 encoder and decoder tools: https://people.xiph.org/~greg/celt/celt011-win32.zip

I would encourage testing at 48kHz rather than 44.1kHz because 48kHz is our officially supported and tuned sampling rate. The other rates are offered on a best effort basis for custom applications which have particular timing requirements. I'd certainly like to hear reports about other rates if you test them, but I can't promise that they work as well as 48kHz.

This post has been edited by NullC: Feb 7 2011, 05:45
Go to the top of the page
+Quote Post
jmvalin
post Feb 7 2011, 05:53
Post #64


Xiph.org Speex developer


Group: Developer
Posts: 473
Joined: 21-August 02
Member No.: 3134



QUOTE (NullC @ Feb 6 2011, 23:42) *
I would encourage testing at 48kHz rather than 44.1kHz because 48kHz is our officially supported and tuned sampling rate. The other rates are offered on a best effort basis for custom applications which have particular timing requirements. I'd certainly like to hear reports about other rates if you test them, but I can't promise that they work as well as 48kHz.


I'd like to add that the recommended frame size is 960 samples for 48 kHz sampling rate (i.e. 20 ms). That's going to be the default for the Opus codec. That being said, CELT will also support 44.1 kHz and other frame sizes.
Go to the top of the page
+Quote Post
list
post Feb 7 2011, 12:52
Post #65





Group: Members
Posts: 27
Joined: 7-September 10
From: Argentina
Member No.: 83667



I found some samples in which @64 kbps,48 khz , CELT is giving much worse results than aotuv/nero.
like this one Bachpsichord.wv
I used a 0.10 celt binary i found. Maybe it's not the last one, but you can take a look anyway.
Go to the top of the page
+Quote Post
jmvalin
post Feb 7 2011, 13:03
Post #66


Xiph.org Speex developer


Group: Developer
Posts: 473
Joined: 21-August 02
Member No.: 3134



QUOTE (list @ Feb 7 2011, 06:52) *
I found some samples in which @64 kbps,48 khz , CELT is giving much worse results than aotuv/nero.
like this one Bachpsichord.wv
I used a 0.10 celt binary i found. Maybe it's not the last one, but you can take a look anyway.


Yes, that's a known "killer sample" for CELT. As I said, CELT now sounds better than Vorbis on many/most samples, but certainly not all of them. File samples like Bachpsichord, this is mostly due to some trade-offs we had to do to get low latency.
Go to the top of the page
+Quote Post
NullC
post Feb 7 2011, 16:30
Post #67





Group: Developer
Posts: 200
Joined: 8-July 03
Member No.: 7653



QUOTE (jmvalin @ Feb 7 2011, 04:03) *
QUOTE (list @ Feb 7 2011, 06:52) *
I found some samples in which @64 kbps,48 khz , CELT is giving much worse results than aotuv/nero.
like this one Bachpsichord.wv
I used a 0.10 celt binary i found. Maybe it's not the last one, but you can take a look anyway.


Yes, that's a known "killer sample" for CELT. As I said, CELT now sounds better than Vorbis on many/most samples, but certainly not all of them. File samples like Bachpsichord, this is mostly due to some trade-offs we had to do to get low latency.


For the very tone rich samples that do poorly (mostly due to CELT's small transform size) I'm very interested in knowing if anyone encounters any that can't be fixed by throwing bitrate at them. So long as the codec goes transparent at some semi-reasonable rate for these samples then VBR can eventually be made to cover it. I don't expect anything to be found that doesn't eventually sound good, but if something is then having the opportunity to fix it in the bitstream would be very helpful.

Go to the top of the page
+Quote Post
IgorC
post Feb 9 2011, 10:26
Post #68





Group: Members
Posts: 1506
Joined: 3-January 05
From: Argentina, Bs As
Member No.: 18803



I have a job to do until 10th of March. After that I will have a plenty time to explore CELT.
Go to the top of the page
+Quote Post
IgorC
post Feb 13 2011, 21:09
Post #69





Group: Members
Posts: 1506
Joined: 3-January 05
From: Argentina, Bs As
Member No.: 18803



Very short test on difficult eig sample

Big surprise is that CELT handles it very well huh.gif . Much better than HE-AAC and aotuv Vorbis last test beta

Results for 64 kbps:
CELT 4.0
Vorbis 3.5
Apple HE-AAC 2.5
Nero HE-AAC 2.0

Vorbis's bitrate went from nominal -q0 (~64 kbps) to 96 kbps while CELT performs better without desviation from established bitrate (65.393 kbps)
Nero also ends up at 77 kbps. But it doesn't help.

CODE
ABC/HR Version 1.1 beta 2, 18 June 2004
Testname:

1L = D:\Samples\celt011\eig\Nero 64 kbps q025 eig.wav
2L = D:\Samples\celt011\eig\Vobis beta.wav
3L = D:\Samples\celt011\eig\Apple CVBR 64 kbps High eig.wav
4R = D:\Samples\celt011\eig\eig_celt64.wav

---------------------------------------
General Comments:

---------------------------------------
1L File: D:\Samples\celt011\eig\Nero 64 kbps q025 eig.wav
1L Rating: 2.0
1L Comment: clicking
---------------------------------------
2L File: D:\Samples\celt011\eig\Vobis beta.wav
2L Rating: 3.5
2L Comment:
---------------------------------------
3L File: D:\Samples\celt011\eig\Apple CVBR 64 kbps High eig.wav
3L Rating: 2.5
3L Comment: clicking
---------------------------------------
4R File: D:\Samples\celt011\eig\eig_celt64.wav
4R Rating: 4.0
4R Comment:
---------------------------------------
ABX Results:

Tested with headphones Sennheiser HD 650 and audio interface E-MU Tracker Pre USB 2.0

Log and test files h*tp://www.mediafire.com/?fdk216h5s89b121

This post has been edited by IgorC: Feb 13 2011, 21:30
Go to the top of the page
+Quote Post
IgorC
post Feb 13 2011, 23:37
Post #70





Group: Members
Posts: 1506
Joined: 3-January 05
From: Argentina, Bs As
Member No.: 18803



Single result for one random sample.
Test of scalability of quality with bitrate.

Results:
kbps - mark:
64 - 2.0
96 - 3.5
128 - 4.3-4.5
192 - 5.0 (transparent)

It scales good and proof my rough rule of thumb that every +32 kbps diffscore is lower by factor of 2.

Log and test files www.mediafire.com/?e8ldh4urm3ymnn7

This post has been edited by IgorC: Feb 13 2011, 23:54
Go to the top of the page
+Quote Post
IgorC
post Feb 14 2011, 16:32
Post #71





Group: Members
Posts: 1506
Joined: 3-January 05
From: Argentina, Bs As
Member No.: 18803



The first sample that wasn't transparent at 192 kbps (maybe at higher bitrate too). Linchpin
The issue with this sample is that CELT adds some colored background noise to rhythm guitar. It's more obvious at lower bitrates. I didn't compare to other encoders because of lack of concentration at that point but I remember that it was a difficult sample always.

All previous and this sample were encoded and decoded with following settings:
CODE
celtenc --bitrate 192 --comp 10 -V --stereo l.wav l_192.oga
celtdec l_192.oga l_192_celt.wav


All samples were encoded at 44.1 kHz. I know that 48 khz would be preferable but I have very little time for resampling right now.

ABX log:
CODE
foo_abx 1.3.4 report
foobar2000 v1.1.2
2011/02/14 04:45:31

File A: D:\Samples\linchpin\Linchpin__Edit_.flac
File B: D:\Samples\celt011\l_192_celt.wav

04:45:31 : Test started.
04:46:20 : 01/01  50.0%
04:46:33 : 01/02  75.0%
04:46:45 : 02/03  50.0%
04:46:53 : 02/04  68.8%
04:47:06 : 03/05  50.0%
04:47:29 : 04/06  34.4%
04:47:50 : 05/07  22.7%
04:47:55 : 06/08  14.5%
04:48:15 : 07/09  9.0%
04:50:16 : 07/10  17.2%
04:50:34 : 08/11  11.3%
04:50:47 : 08/12  19.4%
04:51:09 : 09/13  13.3%
04:51:51 : 10/14  9.0%
04:52:31 : 11/15  5.9%
04:52:54 : 12/16  3.8%
04:53:01 : Test finished.

----------
Total: 12/16 (3.8%)
Go to the top of the page
+Quote Post
IgorC
post Feb 14 2011, 22:21
Post #72





Group: Members
Posts: 1506
Joined: 3-January 05
From: Argentina, Bs As
Member No.: 18803



CELT performs not enough well and worse than Vorbis on creuza
First of all I thought it was not optimal sample rate (44.1 kHz). Then I resampled it to 48 kHz (foobar built-in resampler with high quality mode) but strangely CELT performed even worse.

Logs and encoded files www.mediafire.com/?obhkrhhcoypcdbo

Big picture. CELT has no problem with electronic and rock music and it preserves the stereo and transients well.
Problem for CELT is sample like creuza, bittersweet (from my previous posts). These are tonality samples.
I report only where CELT performs suboptimal. On samples like Closer to God, Linchpin, Since Always, Girl, Waiting it performs at least on par and actually better than Vobis or AAC in the most of the cases.

This post has been edited by IgorC: Feb 14 2011, 22:39
Go to the top of the page
+Quote Post
jmvalin
post Feb 15 2011, 02:04
Post #73


Xiph.org Speex developer


Group: Developer
Posts: 473
Joined: 21-August 02
Member No.: 3134



QUOTE (IgorC @ Dec 10 2010, 13:01) *
How does CELT scale with higher bitrate 80-128 kbps?
HE-AAC has good quality/size trade at 48-64 kbps but already no advantage over LC-AAC at 80 kbps. While Vorbis vice versa.


CELT is actually designed to scale (i.e. keep improving) all the way up to ~510 kb/s in stereo. On the low side, it should go a bit lower than Vorbis, but not quite as low as HE-AAC.
Go to the top of the page
+Quote Post
IgorC
post Mar 15 2011, 16:02
Post #74





Group: Members
Posts: 1506
Joined: 3-January 05
From: Argentina, Bs As
Member No.: 18803



~67 kbps test:
CELT 0.11.2 (complexity 10, bitrate 67.5 kbps, stereo, other settings by default)
Apple HE-AAC (constrained VBR 64 kbps, high quality: --high) (real bitrate ~67 kbps)
Vorbis Aotuv 6.02b q0.1

Setup: headphones Sennheiser HD 650, audio interface EMU Tracker Pre USB.

All files for CELT were resampled to 48 kHz by foobar resampler (PPHS, Ultra Mode enabled) as CELT is optimized for 48 kHz right now.

Bitrate verification on 10 albums:
CELT - 66.3 kbps
HE-AAC – 66.5 kbps
Vorbis – 66.5 kbps

Average score for 30 samples:
CELT – 3.36
HE-AAC – 2.91
Vorbis – 2.60

Excel table with results
mediafire.com/?qv2xa6sa8a9qa8v

Samples and logs.
mediafire.com/?lq21f6bz57v3d1d

The next test is at 96 kbps and then 128.
Go to the top of the page
+Quote Post
NullC
post Mar 16 2011, 13:27
Post #75





Group: Developer
Posts: 200
Joined: 8-July 03
Member No.: 7653



QUOTE (IgorC @ Mar 15 2011, 07:02) *
~67 kbps test:
CELT 0.11.2 (complexity 10, bitrate 67.5 kbps, stereo, other settings by default)
Apple HE-AAC (constrained VBR 64 kbps, high quality: --high) (real bitrate ~67 kbps)
Vorbis Aotuv 6.02b q0.1


Hey— interesting findings!

The results for CELT vs the others are statistically significant according to bootstrap.py (not the the averages weren't clear enough)

I'm looking forward to seeing your other tests.

$ ./bootstrap.py -s 1000000 -p 40000 --blocked --compare-all res.txt
bootstrap.py v1.0 2011-02-03
Copyright © 2011 Gian-Carlo Pascutto <gcp@sjeng.org>
License Affero GPL version 3 or later <http://www.gnu.org/licenses/agpl.html>

Reading from: res.txt
Read 3 treatments, 30 samples => 3 comparisons
Means:
CELT HE-AAC Vorbis
3.360 2.907 2.597

Unadjusted p-values:
HE-AAC Vorbis
CELT 0.012* 0.000*
HE-AAC - 0.130

HE-AAC is worse than CELT (p=0.012)
Vorbis is worse than CELT (p=0.000)

p-values adjusted for multiple comparison:
HE-AAC Vorbis
CELT 0.024* 0.000*
HE-AAC - 0.131

HE-AAC is worse than CELT (p=0.024)
Vorbis is worse than CELT (p=0.000)
Go to the top of the page
+Quote Post

4 Pages V  < 1 2 3 4 >
Reply to this topicStart new topic
2 User(s) are reading this topic (2 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 17th April 2014 - 14:09