Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Multiformat 128 kbps Listening Test (Read 276050 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Multiformat 128 kbps Listening Test

Reply #600 –
Quote
Quote
Quote
That's the spirit, eh?


Yes, that's the spirit since you and others threw this thread in the gutter doing some of the dirtiest tricks imaginable.
[a href="index.php?act=findpost&pid=345714"][{POST_SNAPBACK}][/a]


Hmm.. speaking of "some of the dirtiest tricks imaginable," I seem to recall an old saying about a pot and kettle....

Back on topic, I think that Gambit actually has a point.  More on that...

Quote
Quote
Quote
Seriously, I asked people politely not to reply. Few moments later, I can already see some replies. What's wrong with following what I said? Also, as you can see from the last two similies, it was not really meant the way I wrote it.
[a href="index.php?act=findpost&pid=345716"][{POST_SNAPBACK}][/a]

You can't seriously expect people not wanna discuss that...
[a href="index.php?act=findpost&pid=345719"][{POST_SNAPBACK}][/a]


But I am not interested in that at this time. Everything I asked for was that people simply give their vote for one of the four options. That't is.

Edit: I don't want replies because I KNOW that thread is going to end like this one if I allow replies.
[a href="index.php?act=findpost&pid=345719"][{POST_SNAPBACK}][/a]


As Gambit already pointed out, you can't honestly expect people to not respond on a discussion board.  It simply isn't going to happen that way.

And despite whether you (or me, or any of the other participants of the discussion) like it, this test has become controversial.  Although apparently Roberto -- and probably a handful of other people in the thread -- seem to think it is the fault of people like Gambit and I that this thread has been that way; this simply isn't true.  It's a reflection of the problematic way in which this test proposal was initially "prepared" and what kind of exceptations various parties seemed to have had, coupled with multiple apparent communication failures.

Regardless, since it is controversial, you're likely to get continued debate up until the very end and simply requesting it probably won't do anything.  No offense Sebastian, really, but I don't think you were prepared to run a test of this sort on this scale, or knew what to expect from the ensuing debate, and it shows.  It's good to see you haven't given up, but getting upset at people wanting to debate the issues isn't going to help either.

Since there's been too much discussion to turn back and rethink some of the principles behind the whole thing and start fresh, what can be done to move forward at this point?

1.  It seems the main problem still plauging the test is what format to use for the final choice, and how to use it.  Since there's still disagreement, and since some don't want to have another discussion about it (i.e., not wanting a discussion but only poll), then assuming you want the appearance of consensus, I think I'd agree with JohnV that maybe it's better to just leave it out and come back later.  This possibly brings a couple benefits: a) Disagreement over 2-pass issues can be avoided for now and possibly thought out in a less rushed manner for next time, which brings b) less worry about obtaining samples, and c) more likelyhood of participation and decent results since the overall test will be less tiring.  I admit I am disappointed with this idea simply from the prospect of WMA not being tested, especially given the feeling I've had recently, prompted by certain discussions, that perhaps WMA isn't being treated objectively on the boards here.  But there has to be a compromise somewhere.

2.  If 1 is followed, then a more thorough subsequent test can be held with codecs like ATRAC, MPC, WMA, and WMA Pro, perhaps with some sort of high scoring reference format from the first test.  That'd probably make for a more managable and interesting test set.  One benefit is that the next time around, lessons learned from this discussion would be more readily applicable.  Another is that a quick followup would likely retain some of the momentum of the first test, which could only be a good thing.  Perhaps it could even garner more interest than the first.  Finally, more people would likely be happy with the overall result of both tests since there would be no need to leave out some of the formats that are being excluded right now.  This eliminates the voting problem.

Considering that, I disagree with Sebastian that following 2 would be a waste of resources.  The result could be almost as valuable as the initial test, and the pacing could really end up to be beneficial on multiple accounts.
[a href="index.php?act=findpost&pid=345764"][{POST_SNAPBACK}][/a]


I never said that my organization was perfect and I admit there were some errors on my side like not defining the exact purpose of the test from the beginning. I also try to learn so that next time, this can be avoided.
However, you can't blame me for everything either. It's not my fault that people started bitching about LAME developers or started insulting Guru. I also don't find it very friendly that Gambit continued to insult me when I decided something he didn't agree with. I doubt you would like to go to school and have the maths teacher insult you because you don't know how to determin the maxiumum and minumum points of a function.
Regarding the poll, I wanted to open the poll to obtain some results quickly without starting a debate like this one. We already ran out of time which is partially my fault. But I didn't imagine that WMA is so problematic and that's why it took so much time. Another reason why we ran out of time is that people consequently failed to understand some points I made clear, like that we don't want to test popular settings, but popular formats at their best settings. Then we spent several pages debatting about a low anchor which has absolutely no relevance to this test. The low anchor should sound bad, that's it.
Anyways, have to jet to school now.

Multiformat 128 kbps Listening Test

Reply #601 –
Quote
Quote
Quote
"Stay" I don't have - but I could get it (a FLAC) at allofmp3 if no-one else has it...
[a href="index.php?act=findpost&pid=344565"][{POST_SNAPBACK}][/a]

I have it, but I probably won't be able to upload it until Sunday. Hopefully that won't be a problem.
[a href="index.php?act=findpost&pid=344597"][{POST_SNAPBACK}][/a]


Any news on this?
[a href="index.php?act=findpost&pid=345646"][{POST_SNAPBACK}][/a]

Sorry for the long delay. It turns out I only have the radio edit, not the long version. I could still provide this version if you want it...
Over thinking, over analyzing separates the body from the mind.

Multiformat 128 kbps Listening Test

Reply #602 –
I'm a bit confused: Is the poll still open? The first post of this thread mentions that WMA Std will be used.

(I'm sorry if I missed a post that clears this up)
"We cannot win against obsession. They care, we don't. They win."

Multiformat 128 kbps Listening Test

Reply #603 –
The poll is open. Just keep the discussion here, please.

Poll: The Fifth Element


[span style='font-size:8pt;line-height:100%']P.S./Edit:
- Sebastian hasn't changed the first post (yet) after opening the poll.
- In my opinion it's obvious why he wanted to keep the discussion in one place.
- I've found this discussion normal. It would have been odd if no debate had happened. Though, perhaps some of the replies were not focused enough and based on proper argumentation.[/span]

[span style='font-size:7pt;line-height:100%']Edit 2: typo[/span]

Multiformat 128 kbps Listening Test

Reply #604 –
Quote
It's not my fault that people started bitching about LAME developers or started insulting Guru. I also don't find it very friendly that Gambit continued to insult me when I decided something he didn't agree with.
[a href="index.php?act=findpost&pid=345840"][{POST_SNAPBACK}][/a]

Well done young padawan, Roberto teached you well...  If it takes for me to be the dick of this thread to make the test better, then so be it. But I would point out the flaws I see to anybody, it's nothing personal. And would you rather discuss those things before the test starts with us or get them thrown in your face afterwards by somebody else and hence jeopardize the validity of the results?

And regarding the poll... As of now, there are 97 votes. Judging from previous tests, not even half of those people will participate. So you have people voting that I like to call (yes, I love that word ) "clueless". Because they are interested in the results, which is fine, but  they don't realize the difficulty of the test with another contestant. So the results of the poll from people who will actually participate in the test might be completely different.

Multiformat 128 kbps Listening Test

Reply #605 –
Quote
And regarding the poll... As of now, there are 97 votes.  Judging from previous tests, not even half of those people will participate.[a href="index.php?act=findpost&pid=345896"][{POST_SNAPBACK}][/a]

Another poll?

- Sure! I love testing.
- Not sure yet, but probably.
- Seriously doubt, but you never know.
- No. It sucks.

Multiformat 128 kbps Listening Test

Reply #606 –
Quote
And regarding the poll... As of now, there are 97 votes.  Judging from previous tests, not even half of those people will participate.[a href="index.php?act=findpost&pid=345896"][{POST_SNAPBACK}][/a]


Well, there is no way of limiting the "5th element poll" to people who will participate fully in the test, so there's not much to be done about that.

I can't see any harm in hearing what people on these forums in general would want to include in the test.

Edit: To clarify, I mean that "I can't see any harm in hearing which of those options people on these forums would want to include in the test."
davidnaylor.org

Multiformat 128 kbps Listening Test

Reply #607 –
Quote
I'm a bit confused: Is the poll still open? The first post of this thread mentions that WMA Std will be used.

(I'm sorry if I missed a post that clears this up)
[a href="index.php?act=findpost&pid=345889"][{POST_SNAPBACK}][/a]


The poll is still open, yes.

Quote
Quote
It's not my fault that people started bitching about LAME developers or started insulting Guru. I also don't find it very friendly that Gambit continued to insult me when I decided something he didn't agree with.
[a href="index.php?act=findpost&pid=345840"][{POST_SNAPBACK}][/a]

Well done young padawan, Roberto teached you well...  If it takes for me to be the dick of this thread to make the test better, then so be it. But I would point out the flaws I see to anybody, it's nothing personal. And would you rather discuss those things before the test starts with us or get them thrown in your face afterwards by somebody else and hence jeopardize the validity of the results?
[a href="index.php?act=findpost&pid=345896"][{POST_SNAPBACK}][/a]


WTF are you talking about? Do you want to say that Roberto influenced me to think you're a dick? Sorry to disappoint you, but I found that myself.
And pointing out flaws looks different that starting to bitch and flame IMO. You simply came into this thread and started writing how clueless we all are and how God-like you are. I had to ask you and Dibrom multiple times to provide some constructive critism and not only "this is shit", "you are the most clueless guy in this thread", etc.

Quote
And regarding the poll... As of now, there are 97 votes. Judging from previous tests, not even half of those people will participate. So you have people voting that I like to call (yes, I love that word ) "clueless". Because they are interested in the results, which is fine, but  they don't realize the difficulty of the test with another contestant. So the results of the poll from people who will actually participate in the test might be completely different.
[a href="index.php?act=findpost&pid=345896"][{POST_SNAPBACK}][/a]


You had no problem with 5 competitors as long as the format YOU wanted to be included (WMA Standard) was tested. Now that that's not possible, you had to find something to bitch about.

Multiformat 128 kbps Listening Test

Reply #608 –
Quote
Quote
Quote
Quote
"Stay" I don't have - but I could get it (a FLAC) at allofmp3 if no-one else has it...
[a href="index.php?act=findpost&pid=344565"][{POST_SNAPBACK}][/a]

I have it, but I probably won't be able to upload it until Sunday. Hopefully that won't be a problem.
[a href="index.php?act=findpost&pid=344597"][{POST_SNAPBACK}][/a]


Any news on this?
[a href="index.php?act=findpost&pid=345646"][{POST_SNAPBACK}][/a]

Sorry for the long delay. It turns out I only have the radio edit, not the long version. I could still provide this version if you want it...
[a href="index.php?act=findpost&pid=345873"][{POST_SNAPBACK}][/a]


I have to check if the radio edit contains the portion I want.

Multiformat 128 kbps Listening Test

Reply #609 –
Come on guys. Chill. There have been a million misunderstandnings so far. Just forget it all.

We just want to take part in a listening test here.
davidnaylor.org

Multiformat 128 kbps Listening Test

Reply #610 –
Quote
Because they are interested in the results, which is fine, but  they don't realize the difficulty of the test with another contestant. So the results of the poll from people who will actually participate in the test might be completely different.
[{POST_SNAPBACK}][/a]

And did you realized the difficulty when you posted your first messages in the topic? You're now requesting four competitors, but you were first in favour of both wma and wmapro ([a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38723&view=findpost&p=341454]Nov 12 2005[/url]). Does it mean that you expected first to remove one of the three major competitors (LAME, Vorbis, AAC)?

Note that it's really difficult to follow your thoughts:

• « I definitely would like to see both WMA Standard and Pro included (...) can say whatever you want about WMA, but it at least had gapless support way before some of the other codecs had it (...) And I'm very much interested in the differences between WMA Standard and Pro. »
http://www.hydrogenaudio.org/forums/index....ndpost&p=341454

• « (...) and I think we agreed that it's useless to test WMA Pro »
http://www.hydrogenaudio.org/forums/index....ndpost&p=345495

• « #@%$& piece of WMA crap...  »
http://www.hydrogenaudio.org/forums/index....ndpost&p=345527

Very interesting first, then useless and finally piece of crap  And then you wonder about the lack of organisation in the debate?

EDIT: mainly typo

Multiformat 128 kbps Listening Test

Reply #611 –
Quote
Anyways, regarding MPC, what's the point in testing a dead format? No HW support (maybe with the Rockbox firmware, but that's used by geeks only ), uncertain patent status, painfully slow development (if any), no multi-channel, no high sampling rates, slow seeking, cannot be used in movies...
By the way, you can't even say that the format is the best since Vorbis beat it last time and I am sure AAC slowly reaches its level, too (if it hasn't already).
[a href="index.php?act=findpost&pid=343203"][{POST_SNAPBACK}][/a]


well, now it is clear, why mpc wasn't chosen/considered for the test.
if the conductor is prejudiced...

HW support:
Rockbox -> iriver and more in future probably
but already way more important:
PocketPCs, PPC, PDA
^^ widespread hardware

But this hardware issues cannot be the guideline for HA, to test codecs !
example: Consider Vorbis:
hardware support ?
I cannot go to the local shops and buy a vorbis player. Vorbis is mainly iriver & PPC only also !
The considered WMA pro: hardware support ?!

We cannot and should not decide about hardware here,
here we can test the most modern codecs, even if they are not in use by Joe Average.

To the other "arguments" regarding mpc:
Klemm has opened source, shouldn't be patent issues anymore. The situation is comparable to vorbis somehow, if a patent lawyer wants to find something, he will, as the codecs deal all with music compression

slow development of mpc:
This cannot be an argument. This is a sign, that mpc reached as lossy encoder the limits of development. And for our tests, mpc gets another advantage, because you can use either 1.14 or 1.15(v) as high or medium-high anchor to connect old tests with new tests of the other codecs with newer versions.

The most annoying and even wrong assumptions are following 2:
"Vorbis beat mpc last time ?!"
When happened that ?
IIRC, MPC & Vorbis were tied at top of those 2 multiformat 128k tests,
the pre-previous time MPC was a little bit higher rated, but Vorbis inside the statistical margin, and the last test vice versa,
ie. Vorbis a little bit higher, but both in the statistical overlap.
That means, MPC & Vorbis are both the cream.
And especially for these results, MPC should be considered again to compete against the other formats, as then you have a anchor to watch the improvements (or not!) of the other formats.
"I am sure AAC slowly reaches its level, too (if it hasn't already)."
oh, we don't need listening tests anymore, instead we ask for the feeling of Sebastian, he will tell us, which is the ranking

Summary:
A conductor shouldn't be prejudiced against certain formats.
A test is about comparing formats, not about deselecting formats prior the test.
We shoudn't care about hardware support, the development goes towards more memory and more cpu capacity. By design of HA, we should deal with possible advanced formats, what the industry does later, we cannot influence much now.
The technics will reveal itself, and therefor, it is up to us, to test without prejudices.

As Dibrom mentioned, and wich occured always during the exorbitant long discussion,
the test purposes haven#t been very clear or convincing, formats for portable players, vbr formats at best possible settings.
portable player: here you introduce already grey shades, unclearness. PPC is portable, but excluded ?
irivers with rockbox are excluded ?
Both exlcusions are unfair, as both are the future and present !
This shows, we should stay open minded and simply concentarte on testing.
Of course,
too many formats in 1 test row, are bad.
But then there should be a roadmap to test some other formats.

Independent from "portable" formats at best settings, I mentioned earlier, that a ranking of common formats with common settings would be very very interesting, to show, how big are the differences to the best of the cream.

especially, to argue with Joe Average at other discussion baords eg., or in real life, eg. even with industry.

About testing:
We should carry out a test to get a ranking with the common formats with most used settings,
and that contains wma at q50, ca. 104k, doesn't it ?
because our new vbr codecs are all advanced and wma is vbr also, and certain codecs claim to have "CD-quality" already at 64k, and vbr can get good quality at lower qualities than old common 128k cbr encodings, it makes  a lot sense and our testing probably easier to setup, and easier to carry out out for the testers, if the target quality of the test is not 128 anymore, but ca. 104.
And I don't tolerate an answer like from Busemann before to this idea, who said "no" without reasoning, because I gave a reason.

Multiformat 128 kbps Listening Test

Reply #612 –
"A conductor shouldn't be prejudiced against certain formats."

True. But easier said than done.

"A test is about comparing formats, not about deselecting formats prior the test."

True. But you can only include so many encoders in one test. Some kind of selection has to be made. Since Sebastian is conducting the test, he has the final word. If you don't like it - conduct a parallell test of your own.

"We shoudn't care about hardware support, the development goes towards more memory and more cpu capacity."

Don't really understand what you mean. Anyway. This is an open forum. Sebastian can conduct whatever test he likes.

"Of course, too many formats in 1 test row, are bad.
But then there should be a roadmap to test some other formats."

Go ahead and make one.

"Independent from "portable" formats at best settings, I mentioned earlier, that a ranking of common formats with common settings would be very very interesting, to show, how big are the differences to the best of the cream. especially, to argue with Joe Average at other discussion baords eg., or in real life, eg. even with industry."

Sure, I'd like to see that too. Set it up why don't you.

"About testing:
We should carry out a test to get a ranking with the common formats with most used settings, and that contains wma at q50, ca. 104k, doesn't it ?"

Yes. But that's another test in that case. (At least judging from the current poll outcome.)

"because our new vbr codecs are all advanced and wma is vbr also, and certain codecs claim to have "CD-quality" already at 64k, and vbr can get good quality at lower qualities than old common 128k cbr encodings, it makes  a lot sense and our testing probably easier to setup, and easier to carry out out for the testers, if the target quality of the test is not 128 anymore, but ca. 104."

I agree, a test at lower bitrate would probably be easier for the tester, and maybe more interesting to some. However, this test has been decided for the 128 kbps range.

"And I don't tolerate an answer like from Busemann before to this idea, who said "no" without reasoning, because I gave a reason."

True, his answer wasn't exactly helpful.
davidnaylor.org

Multiformat 128 kbps Listening Test

Reply #613 –
Yes, I knew the weak spots of my writing above, you need to make choices & decisions, i wrote it above.
But I am somehow disappointed, which reasons Sebastian finally wrote about his decision to exclude mpc. Those cannot stand longer without being corrected, eg. in his 1st post to the poll about the 5th format, he links to his flawed arguments contra mpc.

I appreciate his efforts to conduct a 128k test, but the even sometimes hateful discussions here, have shown, that something here is not optimal.

Any test preparation needs a lot of time and efforts, from the guys, who setup the test, and from participants.
At least, a test should give some meaningful results.
of course, everybody can setup any test like he wanna do, but in case, the test goes wrong, he doesn't need to wonder, if there is criticismn or even no participation.

And because these tests even at "low" 128k vbr require efforts (thanks to format developments, don't forget the devs!),
it would be very wise, to still change now eg. the targeted bitrate down to something, like 104k.
As the test should only be about the very popular and commercial formats MP3/Lame, AAC, Vorbis, then WMA standard belongs in adaquate manner to the test ?
(you might notice here, that I don't argue for mpc in this test setup anymore, which i did in past for mpc as known anchor to get comparison with the previous tests in past, but now i argue clearly to include wma standard to this next test, if it should be about popular formats)


So, as solution it wouldn't hurt anybody personally involved here, to adjust the target bitrate of all competitors down to something like 104k ?
or are technical reasons against, any unseen new difficulties, like previously with 128k and wma-problem ?

Another alternative solution comes to my mind, if the primary goal should still be popular formats at 128k, but with wma-standard ?
--> test at 128k abr/cbr
This would give an interesting ranking/result, if wma standard offers 128k abr or cbr. Could the other competitors aac, Vorbis, (lame no problem), be adjusted to 128k abr/cbr ?
iirc, Vorbis might give trouble with this alternative test, a pure vbr mode only ?
If a format has only a pure vbr mode, it gets advantage over the others which are forced to abr/cbr.



Another alternative:
exclusion of wma-standard and test at 128k vbr.
But then the goal of testing popular portable formats is missed clearly.


let#s discuss the other poll alternatives:

None [ 20 ]  [19.23%]
WMA Professional, VBR Q50 [ 48 ]  [46.15%]
WMA Standard, VBR Q50 [ 14 ]  [13.46%]
MusePack [ 22 ]  [21.15%]



WMA Professional, VBR Q50
would reach the 128k target, but makes testing wma pro sense in the goal of this test ?
I don't think so, which portables, or hardware can play wma pro besides PC ?
Near to none ?



WMA Standard, VBR Q50
average at 104k,
2 possibilities to test this:

1. adjust target bitrate of the complete test down to ca. 104k.
Any disadvantages ?
I don#t see important ones, I think, all vbr formats mentioned here for the test, should compete more or less well even at 104k vbr. Very interesting format test for portable hardware.

2. test wma-stzandard at q50 together with the others at 128k.
disadvantage: should be clearly unfair test for wma-50, the test results regarding wma-50 won't have a value, wma-50 would be kind of low anchor, maybe it is so good, that.... it is on par with the others, that would be positive surprise, but at bitrates from 100-130k you cannot expect wonders, as all encoders are tuned meanwhile well.


Musepack:
Musepack at q4 at ca. 128k vbr is a very interesting competitor always,
but it isn't very popular, though there is some more or less hardware support, portable.
Advantage to include musepack:
it would give an anchor to compare the results and developemnts of the new format versions to previous 128k tests.

Summary:
We see, that all mentioned solutions have advantages and disadvantages.
The only solution, which might work well with wma-standard, is lowering target bitrate.

Multiformat 128 kbps Listening Test

Reply #614 –
Quote
But this hardware issues cannot be the guideline for HA, to test codecs !
example: Consider Vorbis:
hardware support ?
I cannot go to the local shops and buy a vorbis player. Vorbis is mainly iriver & PPC only also !
[{POST_SNAPBACK}][/a]

[a href="http://wiki.xiph.org/index.php/PortablePlayers]http://wiki.xiph.org/index.php/PortablePlayers[/url]

Quote
The considered WMA pro: hardware support ?!

WMAPro is a possibility since it appears that WMA can't be easily take into consideration. There's a vacant place and it could only be taken by a non-hardware-compatible format.


Quote
To the other "arguments" regarding mpc:
Klemm has opened source, shouldn't be patent issues anymore. The situation is comparable to vorbis somehow, if a patent lawyer wants to find something, he will, as the codecs deal all with music compression

Open-source is not a criterion of this test. MPC is very different from Vorbis, simply because it sucks at low bitrate. Good performance at low bitrate is apparently a must to become popular. MPC is for a niche. Open-source or not.

Quote
slow development of mpc:
This cannot be an argument. This is a sign, that mpc reached as lossy encoder the limits of development.

It's wrong. According to Klemm, there were many point which needed to be developed:
- SV8
- intensity stereo
- tuning of PNS
- problems with low volume sample
If development has end, it's simply because Klemm don't work anymore on the codec and that no developer has skill enough to work deeply on the codec itself.

Quote
The most annoying and even wrong assumptions are following 2:
"Vorbis beat mpc last time ?!"
When happened that ?
IIRC, MPC & Vorbis were tied at top of those 2 multiformat 128k tests,
the pre-previous time MPC was a little bit higher rated, but Vorbis inside the statistical margin, and the last test vice versa,
ie. Vorbis a little bit higher, but both in the statistical overlap.
That means, MPC & Vorbis are both the cream.

WMAPro was tied with AAC, Vorbis and MPC on the first listening test. But it wasn't a reason to include it again for the second one.
MPC is good at such bitrate (not always). Nobody is against this idea now.

Quote
Summary:
A conductor shouldn't be prejudiced against certain formats.

True. But as far as I can see it, Sebastian opened a poll to decide between MPC and WMAPro. His prejudice towards MPC doesn't affect the final decision.
Quote
A test is about comparing formats, not about deselecting formats prior the test.

Without selection, 15 or more encoders could pretend to be tested.
I did such tests this summer. 12 or 13 encoders were included. I did preliminary pools to make a choice. It's the only solution if you want to test all "valid" encoders and settings. But it's not possible in the current configuration (collective test). Therefore, we must "deselect formats prior the test".

Quote
We shoudn't care about hardware support, the development goes towards more memory and more cpu capacity. By design of HA, we should deal with possible advanced formats, what the industry does later, we cannot influence much now.
The technics will reveal itself, and therefor, it is up to us, to test without prejudices.

The "advanced format" is one criterion among others. I proposed it here. There's no reason to favour it over one another. And I'm not sure that "the design of HA" is a valid argument. The same argument was used in the past by people blaming Roberto (and others) for testing 128 kbps instead of 200 kbps.

Quote
As Dibrom mentioned, and wich occured always during the exorbitant long discussion,
the test purposes haven#t been very clear or convincing, formats for portable players, vbr formats at best possible settings.

They were quickly decided. Not on the beginning. But once decided, there's no point to debate about it again -- excepted when a serious issue is revealed (the problem with WMA 2-pass is one of them explaining why the portable argument, which was the major one util yesterday, can't be followed anymore. Hence the poll to let people make their choice between MPC and WMAPro).

Quote
PPC is portable, but excluded ?
irivers with rockbox are excluded ?

A PPC is a small computer. Rockbox is just a hack. The industry is not supporting MPC.

Multiformat 128 kbps Listening Test

Reply #615 –
Quote
So, as solution it wouldn't hurt anybody personally involved here, to adjust the target bitrate of all competitors down to something like 104k ?
or are technical reasons against, any unseen new difficulties, like previously with 128k and wma-problem ?[a href="index.php?act=findpost&pid=345935"][{POST_SNAPBACK}][/a]

Two answers:
- we must be sure than iTunes VBR and LAME VBR could also be set to reach 104 kbps. I doubt so...
- the purpose of the test was to evaluate encoders at 128 kbps. 135 kbps is close to this value; 104 is really far (closer to 96 kbps than 128).
I have nothing against the idea of testing encoders at ~100 kbps; but it would also be nice to see tests at ~130 kbps, with or without WMA.

Quote
Another alternative solution comes to my mind, if the primary goal should still be popular formats at 128k, but with wma-standard ?
--> test at 128k abr/cbr
This would give an interesting ranking/result, if wma standard offers 128k abr or cbr. Could the other competitors aac, Vorbis, (lame no problem), be adjusted to 128k abr/cbr ?

Forcing ABR/CBR is not very interesting. With format like Vorbis it has no sense. It's also going against latest development of encoders (LAME VBR, iTunes VBR).


Quote
Another alternative:
exclusion of wma-standard and test at 128k vbr.
But then the goal of testing popular portable formats is missed clearly.

Obviously! The portable criterion can't be invoked anymore if WMA is missing. We need to use another main criterion. High Quality VBR as example. Or why not High Quality VBR encoders still in development?  There's only four formats corresponding to this criterion: AAC, MP3, Vorbis, WMAPro. Then, no debate anymore

Multiformat 128 kbps Listening Test

Reply #616 –
Quote
Quote
PPC is portable, but excluded ?
irivers with rockbox are excluded ?

A PPC is a small computer. Rockbox is just a hack. The industry is not supporting MPC.
[a href="index.php?act=findpost&pid=345936"][{POST_SNAPBACK}][/a]



I have taken most of your arguments above already as basis to my writing previously, there is nothing new.
But I comment again, i repeat me, on the hardware arguments.
A explained above, that hardware & industry reasons have a relative weak & questionable, debateable basis.  wouldn't consider rockboy as hack, iirc, it is legally, freely programmed, no use of industries patents/firmware.
by coincidence, I know somebody from England well, who was participating at rockbox dev., who previously developed also very successful at DVD-player firmware (<.. that partially deserves more to be called hacking in some respects...).
by design, a dvd-player is quite close to PC, flashing firmwares is supported by industry for widespread players for updates eg.
So, hardware developemnt is a process in progress.
And the limits between hardware players and PC or Pocket-PCs are vanishing,
all in all, it is useless for us, to discuss this.
The consumer decides, if he buys proprietary products as ipod, or general usb stick, or HD-based player, or pocketPC.
Are we here supporters and testers in the name of the industry ?
I don't think so, no.
Please read my ideas above, what can be done, to get an interesting test of popular formats.
I think, testing wma-pro is at last priority, if you test wma-pro, you can include also mpc without any arguing.
Priority:
The question is still, and i have shown possible solutions, of howto include and test wma-standard finally.


edit-addon:

well! If the goal of the test is rewritten, to test only vbr formats at 128k in active development, then you can exclude wma and mpc, but as then the test competitors are down to 4, it is no rpob, to include mpc again as anchor to get comaprison with previous tests, exactly to value the progress of new lame, new vorbis, new aac.

Multiformat 128 kbps Listening Test

Reply #617 –
Quote
But I am somehow disappointed, which reasons Sebastian finally wrote about his decision to exclude mpc. Those cannot stand longer without being corrected, eg. in his 1st post to the poll about the 5th format, he links to his flawed arguments contra mpc.
[a href="index.php?act=findpost&pid=345935"][{POST_SNAPBACK}][/a]


What is flawed about them?

Portable --> If you count PPC as argument, why not count Notebooks, too? They are portable. And then, Monkey's Audio, VQF, WavPack, LPAC... they're all portable.

Patent --> Being open source doesn't mean that it's patent-free. MPC is based on MP2 which uses patented technology.

What other arguments are flawed?

Multiformat 128 kbps Listening Test

Reply #618 –
Quote
well! If the goal of the test is rewritten, to test only vbr formats at 128k in active development, then you can exclude wma and mpc, but as then the test competitors are down to 4, it is no rpob, to include mpc again as anchor to get comaprison with previous tests, exactly to value the progress of new lame, new vorbis, new aac.
[a href="index.php?act=findpost&pid=345940"][{POST_SNAPBACK}][/a]


Why should WMA be excluded then? Both WMA Standard and WMA Professional provide VBR encoding and are being developed. The only problem with WMA Standard is that the "pure" VBR mode produces bitrates which are too high or too low for this test. I don't see a problem with WMA Professional, though.

Multiformat 128 kbps Listening Test

Reply #619 –
Sebastian, you are absolutely right, you have given answers yourself,

indeed, me and many other people probably use notebooks as portable hardware with mpc, wavpack etc.
Read my thoughts above, then you find, what is wrong about your mpc assumptions, but that won't probably help for the test, no usage.
But if you read, you will find, that I think, the general idea of your test, is it worth, otherwise, I woudn't write here so much.
And you will notice, hopefully, that I don#t go into the endless debates like others here did.

You will find, that I think, that testing popular formats, (important: popular!, instead writing or claiming of industrial hardware portable formats), is necessary.

So, I would try to make a fair comparison of Lame-mp3, Vorbis, AAC & wma-standard,
at fair bitrates,
ie., testing 128k of vbr is of course interesting, but as wma-standard cannot be compared here,
the logical conclusion is:
try to find another bitrate, lower than 128k, which is produceable by all competitors.





edit-addon:


Citing the Vorbis Portable Player page  :

Portable Digital Assistants (PDAs)
PDAs are also cable of operating as portable music players using available software applications.

So, pda's (= PPC) are considered to be portable player even by Vorbis wiki, I see no problem here, and so mpc can be played on pda also.
But as reasoned above, it isn't worth to discuss the hardware/industry issues, the limits between hardware players and Pcs are vanishing.

Multiformat 128 kbps Listening Test

Reply #620 –
I see no sense in testing MPC again. It has been tested at this bitrate and didn't change much in ages. Also the bitrate is not very likely to be used with MPC in my opinion.

Multiformat 128 kbps Listening Test

Reply #621 –
Well, as you can all see, I included MPC to the poll. If MPC wins, I have no problem testing it. I excluded it from the test until now because I gave priority to WMA Standard since it was more popular. However, now that WMA Std. doesn't work the way I (and I guess others, too) thought, we can test MPC.

Multiformat 128 kbps Listening Test

Reply #622 –
Quote
Quote
well! If the goal of the test is rewritten, to test only vbr formats at 128k in active development, then you can exclude wma and mpc, but as then the test competitors are down to 4, it is no rpob, to include mpc again as anchor to get comaprison with previous tests, exactly to value the progress of new lame, new vorbis, new aac.
[a href="index.php?act=findpost&pid=345940"][{POST_SNAPBACK}][/a]


Why should WMA be excluded then? Both WMA Standard and WMA Professional provide VBR encoding and are being developed. The only problem with WMA Standard is that the "pure" VBR mode produces bitrates which are too high or too low for this test. I don't see a problem with WMA Professional, though.
[a href="index.php?act=findpost&pid=345945"][{POST_SNAPBACK}][/a]


the 128k vbr is the point.
it is now fact, that wma standard cannot be tested well at 128k vbr.
And wma-pro offers 128k vbr, but is a weird, not popular (or hardware supported portable) format, as well as mpc and probably many other formats.
See my other suggestions,, solutions to include wma-standard.
Citing my ideas as single sentences will not reveal the complete picture.


ilikedirtthe2nd Posted Today, 03:23 PM
  I see no sense in testing MPC again. It has been tested at this bitrate and didn't change much in ages. Also the bitrate is not very likely to be used with MPC in my opinion.
exactly that is advatage of mpc, usage as comparable anchor to past tests, to reveal the developemnt of new lame, new aac, new vorbis.
If somebody points to another comparable anchor format to past tests, we can use that instead, the idea is not about mpc, but about anchor for cross-comparisons.

You can imagine, that mpc was developed a lot at q4, because Klemms personal transparency level with mpc at his equipment during developing was below q5, something with q4.xx , iirc.
So, the dev. himself will have carried out a lot of tests around q4, so it was not that surprising that mpc was at top together with vorbis in those previous 128k tests.


Sebastian Mares Posted Today, 03:23 PM
  Well, as you can all see, I included MPC to the poll. If MPC wins, I have no problem testing it. I excluded it from the test until now because I gave priority to WMA Standard since it was more popular. However, now that WMA Std. doesn't work the way I (and I guess others, too) thought, we can test MPC.

And again, we agree on testing wma-standard is of higher priority than including mpc as comparable anchor or testing it again.
So, i wrote a lot above, to convince you guys, to change the test setup a little bit, to test wma-standard together with lame, aac, vorbis.
The poll shows still, that mpc got ca. 25%, that mpc is still of interest at ha, why not, as mpc is still good quality.

Multiformat 128 kbps Listening Test

Reply #623 –
Quote
However, you can't blame me for everything either. It's not my fault that people started bitching about LAME developers or started insulting Guru.


Heh.  I can only speak for myself, but I didn't insult Guru.  I don't remember Gambit insulting him either, but maybe I'm wrong.  However, I certainly do remember Guru resorting to peronsal attacks (and I recall that it was at this point that the split thread took the biggest turn for the worst -- this was shortly after I tried to end it peacefully by the way) on me in the argument multiple times.  Seems you got it backwards there, bub.

Of course I don't expect certain folk to remember it so inconveniently...

Quote
I also don't find it very friendly that Gambit continued to insult me when I decided something he didn't agree with.


It was rude of Gambit to make the comment that he did.  But on the other hand, he was making an observation based on actual behavior.  Given all that has transpired in this thread and now this, I can't say that what he said was wrong (sorry), even if it wasn't very tactful to make it a public observation.

Quote
I doubt you would like to go to school and have the maths teacher insult you because you don't know how to determin the maxiumum and minumum points of a function.
[a href="index.php?act=findpost&pid=345840"][{POST_SNAPBACK}][/a]


I don't think it was even really about the math (for the most part), which just says another thing about communication...

Quote
WTF are you talking about? Do you want to say that Roberto influenced me to think you're a dick? Sorry to disappoint you, but I found that myself.
And pointing out flaws looks different that starting to bitch and flame IMO.
You simply came into this thread and started writing how clueless we all are and how God-like you are. I had to ask you and Dibrom multiple times to provide some constructive critism and not only "this is shit", "you are the most clueless guy in this thread", etc.


Ah, yes, of course.  Another one of those "I like criticism, but only when it's not critical, and only when I don't take it personally."

For your information, the original criticisms were constructive (I notice you didn't even respond to my "constructive criticisms" in my last post -- how's that for selective reading?).  They became unconstructive once someone decided to take them personally and use them as a launching point to rail on with his tirades about how damn evil HA and myself are, and how there are insidious conspiracies against LAME lurking in dark shadows and the hearts of HA admins alike.  Or something like that, but probably worse, and definitely much scarier.

I notice that in both of your complaints, you didn't once fault this person, yet place the blame on Gambit and I.  Again, how convenient for you!

You say you aren't influenced?  Hrmm.. could have fooled me.  You're being influenced by someone's fantasy, whether it's simply yours or someone elses.

*sigh* I'm really surprised to see this sort of nonsense coming from you, after all that has been gone through to end earlier problems in this thread.  As test coordinator, I'd have thought you'd want to keep this kind of thing out of the discussion, and keep the discussion on topic.

But anyway, I guess it's illustrative. You and your buddies have made much of a point by now that you only want debate on your terms, within very specific constraints (mostly translating to: "don't question anything relating to the test" ), otherwise you get very upset.  That's not the way that HA works (granting exception for forum rules), and certainly shouldn't be the way a supposedly democractic process in test preparation should transpire.  It's simply not worth it to me to continue to bother with this thread under those considerations, so I suppose you won't have to look forward to any more "unconstructive" criticisms from me at any rate.  In hind sight, it was quite stupid of me to ever optimistically come back for seconds to begin with....

Multiformat 128 kbps Listening Test

Reply #624 –
Sebastian, may I make a suggestion:

Why don't you start over again, in a new thread. We can leave all of this behind us. You can clearly state in the first post what the purpose/goal of the test is, and any discussion can go on from there.
davidnaylor.org