Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Transcode Test is now OPEN! (Read 17747 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Transcode Test is now OPEN!

Reply #25 –
Quote
I believe your batch files have a small problem with the last line, they don't wavegain anything (at least in my system)  

Even if wavgain isn't doing anything, I don't think the batch files need to change.  I checked the average rms levels of the left and right channels separately for each transcode plus the reference in one of the samples, and the volumes were the same within a tenth of a dB.

ff123

Transcode Test is now OPEN!

Reply #26 –
I'm averaging more updates than microsoft at this point. 
Guess what I just did?

I wonder if I'm too scatterbrained to conduct this test.

Transcode Test is now OPEN!

Reply #27 –
Damn. I just found out that i don't have time to participate this test. But this war doesn't need one man...

Transcode Test is now OPEN!

Reply #28 –
Quote
Damn. I just found out that i don't have time to participate this test. But this war doesn't need one man...

Like I said, I can extend the test if need be.  Right now, I have ZERO results, so if you need more time, I can easily extend the test end.

Transcode Test is now OPEN!

Reply #29 –
till when would you like to extend it?

Transcode Test is now OPEN!

Reply #30 –
Quote
Quote
Damn. I just found out that i don't have time to participate this test. But this war doesn't need one man...

Like I said, I can extend the test if need be.  Right now, I have ZERO results, so if you need more time, I can easily extend the test end.

I realy don't know next time when can i get my computer connected to internet after tomorrow. I'm moving now in new place and i just made contrackt with new ISP. They say that normaly it takes 2-5 weeks until connection is open...
But maybay i can download test pakage now and someday try to send results from somewhere. I realy like to put my share to this test, but don't whait for me.

Transcode Test is now OPEN!

Reply #31 –
Quote
till when would you like to extend it?

Probably 12:00AM Dec. 1st.  That gives over a month of time on very difficult material.  After that, I'll either have plenty of results, or I'll call the test a failure due to lack of results.  I choose that date specifically, because that's when I will have returned from the US holiday, Thanksgiving.  Granted, that may be TOO much time, but I don't want to deal with the results immediately prior to my holiday.

Transcode Test is now OPEN!

Reply #32 –
Quote
Quote
till when would you like to extend it?

Probably 12:00AM Dec. 1st.  That gives over a month of time on very difficult material.  After that, I'll either have plenty of results, or I'll call the test a failure due to lack of results.  I choose that date specifically, because that's when I will have returned from the US holiday, Thanksgiving.  Granted, that may be TOO much time, but I don't want to deal with the results immediately prior to my holiday.

Based on the likely amount of ABXing for each test group that will be required for this test (as discussed earlier), I'd also vote for an extension if it wouldn't cause you any schedule conflicts, personal or otherwise.

Also, as of now I'm not positive that I'd have time to complete and submit even one set of test results by the original end date, but if you extended to Dec. 1st, then I'm sure I could participate.  My interest is high, but my time is somewhat constricted here lately.

And thanks for offering to accommodate those of us who may need more time!

Transcode Test is now OPEN!

Reply #33 –
Quote
if it wouldn't cause you any schedule conflicts, personal or otherwise.

I was going to conduce a Vocodec test by the end of this month, but now I'll probably reschedule it to January.

Transcode Test is now OPEN!

Reply #34 –
Quote
Based on the likely amount of ABXing for each test group that will be required for this test (as discussed earlier), I'd also vote for an extension if it wouldn't cause you any schedule conflicts, personal or otherwise.

Also, as of now I'm not positive that I'd have time to complete and submit even one set of test results by the original end date, but if you extended to Dec. 1st, then I'm sure I could participate.  My interest is high, but my time is somewhat constricted here lately.

And thanks for offering to accommodate those of us who may need more time!

Roberto and Phong are the only ones I'm worried about in terms of scheduling conflicts.  They're the ones with listening tests in the pipeline and the further I push back the test end date, the more they have to wait.

Personally, I'd be perfectly happy letting you guys take as long as you want, but there needs to be some obvious test end date so that Roberto and Phong can get their respective tests rolling.

Transcode Test is now OPEN!

Reply #35 –
I am confused about something... According to your test page, 5 codecs/settings are being tested (MPC @ -q5 & -q7, vorbis -q6, lame -aps, and wavpack lossy -320) against a direct encode to --preset medium, right?
Then why are there samples xxx_1.mp3 through xxx_6.mp3 PLUS an xxx.mp3 and six test per sample? Does this mean that one of the samples is actually another reference (direct encode)? If so, I think that is MEAN plus completely unnecessary; since that means six VERY difficult (or rather IMPOSSIBLE  ) extra tests to prove what? That you can't ABX something against itself? I think its of no use either to discard people who take the test sloppily or can't follow instructions. Those will simply show up with ranked references or lack of ABX results. For those of us that are putting some effort to ABX the samples, I think it would be an utter waste of our time to put us to test something for as long as our patience lasts (which can be quite a while for some) just to get 5.0s.
Or is it just that there is a sixth codec being tested that you forgot to note down on the page? I kinda hope so.

Transcode Test is now OPEN!

Reply #36 –
I thought I mentioned that somewhere, but I guess I haven't.  There is a 6th sample that's just a transcode of preset medium to itself several times.  The point behind this is to give at least one sample that should be audibly different.  A lower anchor.  That's the crap setting that you guys should be able to ABX in about 2 minutes (counting setup time).  If you CAN'T identify the crap sample, then you might want to reconsider taking this test.

Transcode Test is now OPEN!

Reply #37 –
Quote
Like I said, I can extend the test if need be.  Right now, I have ZERO results, so if you need more time, I can easily extend the test end.

Well, if you didn't before, you have some now  ....although 1 test on one sample isn’t really statistically valid, now is it?

Transcode Test is now OPEN!

Reply #38 –
Quote
Well, if you didn't before, you have some now   ....although 1 test on one sample isn?t really statistically valid, now is it?

friedman.exe determines whether the results are statistically valid or not. But I would reckon that at least 10 results per sample are needed.

Transcode Test is now OPEN!

Reply #39 –
This test would be a lot easier with different samples.....I've done BlueMonday, VirtualReality and Waiting so far with pretty good results, but samples such as Talula sound pretty good at --alt-preset cbr 112 to my ears.....

edit: sthayashi - could we have an update on how many results you have so far?

[strongbad]A message to the peoples[/strongbad] : Come on! We need results to, ah, "seal the deal." - participate, it's hard, but I want to see this test succeed!

Transcode Test is now OPEN!

Reply #40 –
Quote
This test would be a lot easier with different samples.....I've done BlueMonday, VirtualReality and Waiting so far with pretty good results, but samples such as Talula sound pretty good at --alt-preset cbr 112 to my ears.....

edit: sthayashi - could we have an update on how many results you have so far?

You bet.  I have 3 results.  I wish I had more.  Hopefully I will after this next weekend.

I figured I would give people at least two weekends to try, but if there is enough demand, like I stated before, I'll extend the test until the end of the month.

Transcode Test is now OPEN!

Reply #41 –
Finally got around to downloading the samples...  Two of the .sh files have the wrong filenames in them (01 and 04).

Also, has anyone ever gotten wavegain to compile under linux?  I suppose the volume differences are neglegible here...
I am *expanding!*  It is so much *squishy* to *smell* you!  *Campers* are the best!  I have *anticipation* and then what?  Better parties in *the middle* for sure.
http://www.phong.org/

Transcode Test is now OPEN!

Reply #42 –
Quote
Finally got around to downloading the samples...  Two of the .sh files have the wrong filenames in them (01 and 04).

To quote a ninjai character, FAAAAHHHK.  I knew about it and thought I had fixed it with one of the updates (in fact I'm damn near positive I did, so I wonder WTF is up).

The start.zip file has ONCE again been updated.
Quote
Also, has anyone ever gotten wavegain to compile under linux?  I suppose the volume differences are neglegible here...

IIRC, some people managed to successfully compile it under OSX, so I'd imagine that Linux wouldn't be that difficult.
Thanks for participating.

Transcode Test is now OPEN!

Reply #43 –
@sthayashi
I hope you are still considering going to the end of the month. I'm tight for time here at the moment.

Den. >_<

Transcode Test is now OPEN!

Reply #44 –
Quote
@sthayashi
I hope you are still considering going to the end of the month. I'm tight for time here at the moment.

Since a few regulars have asked for an extension, I shall grant them one.  The test end date has been extended to Nov 30th.
 

Transcode Test is now OPEN!

Reply #45 –
Due to lack of interest and few results, the transcode test is now officially closed.

Feel free to contact me if you want to be tested with your own parameters (i.e AAC->mp3 or what have you).

Transcode Test is now OPEN!

Reply #46 –
Oh, man. That's too bad

Transcode Test is now OPEN!

Reply #47 –
I'm soo sorry.  I haven't had time and had only gotten a couple of the samples done and didn't realize the deadline was the 30th.  I hope you weren't like one or two result sets away from having enough.
I am *expanding!*  It is so much *squishy* to *smell* you!  *Campers* are the best!  I have *anticipation* and then what?  Better parties in *the middle* for sure.
http://www.phong.org/

Transcode Test is now OPEN!

Reply #48 –
I'm really sorry Steve.  I had pushed myself to participate in this test, but my workload increased about threefold, and I never had enough time available.  My time online and my participation everywhere else dropped a lot too in the past few weeks.  The holidays get like this every year, with projects at work having converging deadlines, more people on vacation and fewer of us to pick up slack.

If you keep the links to the test packages working, then could people run the tests as they get time (more likely after the holidays)?  Or without a time restriction would it be a pain to manage?

Transcode Test is now OPEN!

Reply #49 –
Well, I need to update the webpage, which I'll do when I return home from work.  As is, I already have the files available on my site, but I was worried about exceeding my bandwidth limit.  I guess that was an unfounded fear.

If you have a results file (or several results), please send them to me.  I will respond to you and let you know what you could and could not hear.

I guess I shouldn't say that the test is cancelled, exactly, but the official end date is now undefined, and I won't publicly release results until I have enough to be statistically significant.  Given the number of results I have now, it's not clear if I will ever have enough.

If you do contribute your own results, I'll tell you what you got and how it compares to others.  At least that way, people will still have some incentive to participate.  I will make some public generalizations about what I do have:
a) Every codec was successfully ABX'ed, to a limited degree (worst case was a pval of 0.006)
b) Half the results was what I expected.  The other half was not.