IPB

Welcome Guest ( Log In | Register )

4 Pages V  < 1 2 3 4 >  
Reply to this topicStart new topic
What's the best AAC encoder today?, Status as of 2013
Kamedo2
post Sep 5 2013, 18:52
Post #26





Group: Members
Posts: 168
Joined: 16-November 12
From: Kyoto, Japan
Member No.: 104567



Although average score of Apple TVBR is slighty poorer than CVBR with no significance in the 2011 public test and my 2012 private test, http://www.hydrogenaudio.org/forums/index....showtopic=97913
I'm guessing TVBR is better, with not so much confidence. Both in these two tests, TVBR is slightly poor in score, and substantially less in bitrate. If you look at my bitrate-quality graph, 96kbps area in my test, maybe you can understand what I'm trying to say.
Go to the top of the page
+Quote Post
C.R.Helmrich
post Sep 5 2013, 20:32
Post #27





Group: Developer
Posts: 681
Joined: 6-December 08
From: Erlangen Germany
Member No.: 64012



QUOTE (eahm @ Sep 5 2013, 17:23) *
Chris,
I've never used the word Frauenhofer. Quote who misspelled not just the last comment.

Alright, NePaC and a number of other visitors of this forum except eahm: Fraunhofer (not Frauenhofer). No need to get upset, eahm.

QUOTE
I don't really care what Fraunhofer uses and what it's called, I was listing the option and Fraunhofer doesn't have any True VBR switch.

It doesn't have a Constrained VBR switch, either. What's your point?

By the way, I know that my encoder lost against Apple CVBR in 2011, as Kamedo2 explained (thanks for educating people about the p-values).

Chris

This post has been edited by C.R.Helmrich: Sep 5 2013, 20:35


--------------------
If I don't reply to your reply, it means I agree with you.
Go to the top of the page
+Quote Post
testyou
post Sep 5 2013, 21:11
Post #28





Group: Members
Posts: 90
Joined: 24-September 10
Member No.: 84113



QUOTE (C.R.Helmrich @ Sep 5 2013, 12:32) *
[Fraunhofer] doesn't have a Constrained VBR switch

So Fraunhofer doesn't name their switches according to functionality. What's your point?


Just to be sure, am I interpreting the graph correctly?

This post has been edited by testyou: Sep 5 2013, 21:40
Go to the top of the page
+Quote Post
Kamedo2
post Sep 6 2013, 06:57
Post #29





Group: Members
Posts: 168
Joined: 16-November 12
From: Kyoto, Japan
Member No.: 104567



QUOTE (testyou @ Sep 6 2013, 05:11) *
Just to be sure, am I interpreting the graph correctly?


It's hard to determine insignificance from a graph, when it's from a typical listening test that have both difficult and easy samples.
The significance is stated in the original data, saying:
CODE
CVBR is better than Nero (p=0.000)
TVBR is better than Nero (p=0.000)
FhG is better than Nero (p=0.000)
FhG is worse than CVBR (p=0.005)
CT is better than Nero (p=0.000)
CT is worse than CVBR (p=0.000)
CT is worse than TVBR (p=0.000)
CT is worse than FhG (p=0.000)
low_anchor is worse than Nero (p=0.000)
low_anchor is worse than CVBR (p=0.000)
low_anchor is worse than TVBR (p=0.000)
low_anchor is worse than FhG (p=0.000)
low_anchor is worse than CT (p=0.000)

http://listening-tests.hydrogenaudio.org/i...-a/results.html
It's from the bootstrap analysis. It's adjusted for multiple comparison, so it's very likely that the 13 significance statements above is all correct.
Go to the top of the page
+Quote Post
C.R.Helmrich
post Sep 6 2013, 09:45
Post #30





Group: Developer
Posts: 681
Joined: 6-December 08
From: Erlangen Germany
Member No.: 64012



QUOTE (testyou @ Sep 5 2013, 22:11) *
QUOTE (C.R.Helmrich @ Sep 5 2013, 12:32) *
[Fraunhofer] doesn't have a Constrained VBR switch

So Fraunhofer doesn't name their switches according to functionality. What's your point?

Not sure what you mean. Let's take an example. If I encode loud rock music or even pure noise with a target VBR rate of x kbps, I get close to 1.2x or even 1.3x kbps. If I encode a sine sweep instead, I get maybe 0.6x or 0.7x kbps. See also the bit-rate table of the 2011 test. That's not nearly as constrained as Apple's CVBR, right? So why should I call the mode "Constrained VBR", then? Does nero distinguish between two VBR modes? I see only VBR, ABR, and CBR.

If you guys really need to put another word in front of "VBR", then call it "True VBR", and write, "Fraunhofer supports True VBR". Thank you very much.

QUOTE
Just to be sure, am I interpreting the graph correctly?

Thanks for thinking CVBR vs. FhG is insignificant, but no. A rule-of-thumb which I use all the time and which works perfectly here is:
  • If the confidence bars of one coder overlap with at least the mean score of another coder, then there is no significant quality difference between those coders.
  • If the confidence bars of one coder do not overlap with the mean score of another coder, then there is a small but significant quality difference.
  • If the confidence bars of two coders do not overlap at all, then there is a large, significant quality difference between the coders.

Chris

This post has been edited by C.R.Helmrich: Sep 6 2013, 09:49


--------------------
If I don't reply to your reply, it means I agree with you.
Go to the top of the page
+Quote Post
nu774
post Sep 6 2013, 11:52
Post #31





Group: Developer
Posts: 476
Joined: 22-November 10
From: Japan
Member No.: 85902



This result says that QT TVBR is not significantly better than FhG but QT CVBR is, and no significant difference between TVBR and CVBR.
How should I interpret this?

When I hear the word "better", I imagine of something like ordered relations between them.
However the above means that:
CODE
QT TVBR = QT CVBR and QT TVBR = FhG and QT CVBR > FhG

So apparently their relations are not transitive, and the result cannot be considered to form an ordered set.
Or is it possible to consider that there actually is a hidden order and I can derive the result QT CVBR > QT TVBR (or QT TVBR > FhG), which was not directly drawn by statistical comparison but can be indirectly deduced from, assuming transitive relation?
Go to the top of the page
+Quote Post
Kamedo2
post Sep 6 2013, 13:42
Post #32





Group: Members
Posts: 168
Joined: 16-November 12
From: Kyoto, Japan
Member No.: 104567




Is b better than a? We can't be sure. In this test, it was. But humans are whimsical, it can be luck. After more extensive tests with more samples, things may turn out to be a > b. If we announce a < b and after a while we change it to a > b, we'll lose creditability. In that case, we should say there are no significant differences, or ab. Likewise, bc, cd, de, ef, fg. But we can say a < g. It's not likely that a and g having the same average score. a having more score is even less likely.

Go to the top of the page
+Quote Post
nu774
post Sep 6 2013, 14:00
Post #33





Group: Developer
Posts: 476
Joined: 22-November 10
From: Japan
Member No.: 85902



It seems that I was silently assuming that when a < g and a ≒b and b ≒ g then it cannot be b < a.
But if I understand you correctly, that's a wrong assumption.
Thanks for explanations smile.gif
Go to the top of the page
+Quote Post
Destroid
post Sep 6 2013, 21:55
Post #34





Group: Members
Posts: 544
Joined: 4-June 02
Member No.: 2220



Thanks, Kamedo2. That chart and explanation actually made me realize I was also (mis)interpreting the final scores of the very test under discussion, of which I often refer to. Now if I was going to get medieval about "best" AAC codec I would have to consider my type of material and look up the results of individual samples in that test to see how each encoder fared. But... surprise! That just happens to be the case with the bitrate that the test was run under. Maybe the gaps between encoders widens or narrows under different bitrates. It's a good time to rely on one's own ears, I would say.

sidenote: I got to using the QAAC quite a bit lately ( at --tvbr 50), the main reason is the tagging is very much compatible with my player. (C.R.Helmrich- maybe you could look into this for Fhgaacenc when you have extra time, or some other tag-related question comes up? I really like the codec you're involved with smile.gif )


--------------------
"Something bothering you, Mister Spock?"
Go to the top of the page
+Quote Post
IgorC
post Sep 7 2013, 16:58
Post #35





Group: Members
Posts: 1506
Joined: 3-January 05
From: Argentina, Bs As
Member No.: 18803



Some observations.
Let’s compare http://listening-tests.hydrogenaudio.org/s...8-1/results.htm and
http://listening-tests.hydrogenaudio.org/i...-a/results.html
Itunes AAC encoder was used as high anchor and scored 4.69 in first one but 4.391 in last one. Why is this difference? Sure, these are two different tests. However there were people saying "Man, I could provide a results in previous HA tests at low bitrates like 48-64 kbps, but not for higher bitrates because it’s hard as codecs are already very good at that point"
Some part of these people hasn’t participated in the last AAC 2011’ test. The overall scores would be a bit higher and the difference between competitors would be less if these people could provide results as they would rank pretty high. I’m confident that FhG and Apple would end with > 4.50 scores and insignificant difference like MP3 encoders here http://listening-tests.hydrogenaudio.org/s...8-1/results.htm . I believe that previous tests of Sebastian Mares are a bit more permissive. That’s not worse or better, it’s just a different approach with a different circumstances.
And hell, in my experience, testing AAC encoders at 96 kbps is actually harder than MP3 at 128 kbps.
Taking a bigger picture across a different bitrates, FhG has the most optimal HE-AAC v1/v2 encoder at 32-64 kbps respectively (according to my internal personal tests), and fine, it was Nš2 at 96 kbps , slightly (I would say very slightly) behind of Apple encoder in the last HA public test. But these two encoders have an excellent quality at 128 kbps (and higher).

This post has been edited by IgorC: Sep 7 2013, 17:07
Go to the top of the page
+Quote Post
halb27
post Sep 7 2013, 23:35
Post #36





Group: Members
Posts: 2414
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



Two different liststening tests are exactly this: they're different: different samples, different listeners, different listening situations when the same listener. So different results are a matter of fact, though.they are averaged out a bit because several listeners and several samples are involved. Most critical for the result is sample selection.

This post has been edited by halb27: Sep 7 2013, 23:39


--------------------
lame3100m --bCVBR 300
Go to the top of the page
+Quote Post
mrgou
post Sep 8 2013, 20:36
Post #37





Group: Members
Posts: 64
Joined: 23-March 05
Member No.: 20859



Has there been any improvement in the encoder between iTunes 10 & 11? I think the new iTunes interface sucks, but if the encoder has been improved, it might be worth it. Or could the Application Support component from v11 be installed along with v10?
Go to the top of the page
+Quote Post
Garf
post Sep 8 2013, 21:11
Post #38


Server Admin


Group: Admin
Posts: 4853
Joined: 24-September 01
Member No.: 13



QUOTE (halb27 @ Sep 8 2013, 00:35) *
Two different liststening tests are exactly this: they're different: different samples, different listeners, different listening situations when the same listener. So different results are a matter of fact, though.they are averaged out a bit because several listeners and several samples are involved. Most critical for the result is sample selection.


The whole point of the construction of such tests is what the results should be repeatable. Of course the individual subjective ratings will fluctuate, but the averages, and the conclusions after taking into account statistical variance, shouldn't.

This will fail if there are bad biases in the test. It's not unreasonable to assume listeners in our tests are more picky than average, pushing scores down, for example. This affects conclusions about transparency but not so much relative codec ratings.

We try to get the sample selection diverse and varied enough so that it's representative. Another test with a representative but different sample set should produce compatible results. If it does not, our sample selection isn't good enough and needs to be improved.

So I'll state again: different results (that are statistically significant) are very much not a matter of fact. They're an indication something is broken.
Go to the top of the page
+Quote Post
TomasPin
post Sep 8 2013, 22:53
Post #39





Group: Members
Posts: 168
Joined: 5-June 13
From: Argentina
Member No.: 108508



QUOTE (mrgou @ Sep 8 2013, 16:36) *
Has there been any improvement in the encoder between iTunes 10 & 11?

For that matter you should look up which CoreAudioToolbox version came with the installer of each iTunes version. Looking here, and knowing the latest one is 7.9.8.3 (if I recall correctly), there may be some improvements. Whether those are relevant for you is an entirely different matter.

(If you can find some place where they list each corresponding releases I'll thank you kindly; I was unsuccessful at that.)

This post has been edited by TomasPin: Sep 8 2013, 22:54


--------------------
A man and his music: http://tubular.net/
Go to the top of the page
+Quote Post
halb27
post Sep 8 2013, 23:20
Post #40





Group: Members
Posts: 2414
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



QUOTE (Garf @ Sep 8 2013, 22:11) *
...This will fail if there are bad biases in the test. It's not unreasonable to assume listeners in our tests are more picky than average, pushing scores down, for example. This affects conclusions about transparency but not so much relative codec ratings.
...So I'll state again: different results (that are statistically significant) are very much not a matter of fact. They're an indication something is broken.

I agree if it's about relative codec ratings. I don't if it is about absolute scores of a codec.


--------------------
lame3100m --bCVBR 300
Go to the top of the page
+Quote Post
eahm
post Sep 8 2013, 23:21
Post #41





Group: Members
Posts: 882
Joined: 11-February 12
Member No.: 97076



7.9.8.3 is exactly like 7.9.8.2 but I don't remember if 7.9.8.2 is like 7.9.8.1. At this point, wait until Tuesday for the new iTunes then start the test.

This post has been edited by eahm: Sep 8 2013, 23:22
Go to the top of the page
+Quote Post
Pulstar
post Sep 9 2013, 05:59
Post #42





Group: Members
Posts: 35
Joined: 26-January 10
Member No.: 77550



Can I safely use FDK within Handbrake? I encode many vids at 32kbps AAC. I am miffed that qaac doesn't have v2 of HE-AAC, but too late to rectify my mistake (hopefully it isn't a large one).
Go to the top of the page
+Quote Post
Pulstar
post Sep 9 2013, 07:57
Post #43





Group: Members
Posts: 35
Joined: 26-January 10
Member No.: 77550



No edit button? Anyway I did a quick test of an M4A of Vangelis' Alpha which I ripped earlier, and apparently the cut-off using Fraunhofer's v2 of HE-AAC (file 2.m4a) is higher than qaac (1.m4a). I guess I'll go with the former for lower-bit encodes.



This post has been edited by Pulstar: Sep 9 2013, 07:58
Go to the top of the page
+Quote Post
LithosZA
post Sep 9 2013, 08:43
Post #44





Group: Members
Posts: 181
Joined: 26-February 11
Member No.: 88525



The cut-off looks nearly the same to me. Which one sounds less annoying?
Go to the top of the page
+Quote Post
Pulstar
post Sep 9 2013, 09:15
Post #45





Group: Members
Posts: 35
Joined: 26-January 10
Member No.: 77550



I'll have to do a personal ABX test from FLAC sources to really determine which is best lol. Btw isn't anything approaching +16kHz better for the human ear?
Go to the top of the page
+Quote Post
db1989
post Sep 9 2013, 11:43
Post #46





Group: Super Moderator
Posts: 5147
Joined: 23-June 06
Member No.: 32180



QUOTE (Pulstar @ Sep 9 2013, 09:15) *
anything approaching +16kHz
What is this supposed to mean? Semantically, it is hopelessly unclear.

QUOTE
better for the human ear?
Are you insinuating that files with frequencies above 16 kHz sound better or otherwise audibly detectable when compared to files lowpassed around that point, and/or are healthier for the senses? If either, evidence will be needed, personal double-blind listening tests or medical citations respectively.
Go to the top of the page
+Quote Post
mrgou
post Sep 9 2013, 12:27
Post #47





Group: Members
Posts: 64
Joined: 23-March 05
Member No.: 20859



QUOTE (TomasPin @ Sep 8 2013, 23:53) *
For that matter you should look up which CoreAudioToolbox version came with the installer of each iTunes version. Looking here, and knowing the latest one is 7.9.8.3 (if I recall correctly), there may be some improvements. Whether those are relevant for you is an entirely different matter.

I checked 10.7, and it comes with 7.9.8.1. The latest (11.0.5) comes with 7.9.8.3, so I'm not sure it would include major improvements. I'll try and see how iTunes 10 behaves if I substitute the DLL with the latest one.

QUOTE
(If you can find some place where they list each corresponding releases I'll thank you kindly; I was unsuccessful at that.)

I couldn't find anything either. Shouldn't be too hard downloading and inspecting older versions. I'll try and see if I can put that together and post it on the forum.
Go to the top of the page
+Quote Post
mrgou
post Sep 9 2013, 18:44
Post #48





Group: Members
Posts: 64
Joined: 23-March 05
Member No.: 20859



I extracted the files CoreAudioToolbox.dll and CoreFoundation.dll and put them next to qaac.exe on a computer where iTunes isn't installed. It worked fine.

Also, the same wave file encoded on this PC with CoreAudioToolbox from iTunes 11.0.5 is the same size as that encoded on a PC with iTunes 10.7, so the encoder appears to be the same.

I mapped some of the recent versions of iTunes with the version of CoreAudioToolbox that comes with it:

CODE
iTunes    CoreAudioToolbox
10.5            7.9.7.8
10.6            7.9.7.9
10.6.3        7.9.7.9
10.7            7.9.8.1
11.0            7.9.8.2
11.0.1        7.9.8.2
11.0.2        7.9.8.2
11.0.3        7.9.8.3
11.0.4        7.9.8.3
11.0.5        7.9.8.3


Is that of sufficient interest to be posted on its own?
Go to the top of the page
+Quote Post
eahm
post Sep 9 2013, 19:17
Post #49





Group: Members
Posts: 882
Joined: 11-February 12
Member No.: 97076



QUOTE (mrgou @ Sep 9 2013, 10:44) *
I extracted the files CoreAudioToolbox.dll and CoreFoundation.dll and put them next to qaac.exe on a computer where iTunes isn't installed. It worked fine.

mrgou, have you seen makeportable? https://sites.google.com/site/qaacpage/cabi...rects=0&d=1

This post has been edited by eahm: Sep 9 2013, 19:32
Go to the top of the page
+Quote Post
saratoga
post Sep 9 2013, 19:55
Post #50





Group: Members
Posts: 4718
Joined: 2-September 02
Member No.: 3264



QUOTE (Pulstar @ Sep 9 2013, 04:15) *
I'll have to do a personal ABX test from FLAC sources to really determine which is best lol. Btw isn't anything approaching +16kHz better for the human ear?


AAC-HE doesn't really encoder higher frequencies anyway, so you're just looking at harmonic extension of the lower frequencies. Hence, I wouldn't worry much about what the stuff above 11khz looks like unless you're using regular AAC.
Go to the top of the page
+Quote Post

4 Pages V  < 1 2 3 4 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 19th April 2014 - 09:07