OK this is old stuff but I haven't posted here yet so, I'll give it a shot.

I am conducting a Video Quality Assessment project based on a blind test of a group of people. You can see the announcement and more info about it here and here .

There is already a list of 67 people that are taking part. However a large portion of them hasn't replied to my e-mails yet for unknown reasons. It could be because I have delayed this a bit longer that it was meant to or because they are on holidays?

Anyways, if anyone would like to take part, please reply here or drop me an e-mail so that I can reopen the application form.

The test is quite like ff123's one with the slight difference that this is about video and not audio and the software will not play all the samples simultaneously

Any questions, please let me know, some of this might be slightly obscure so give me a chance to explain.

All a tester has to do is download some test clips and say how good they look?

Do you have a list of encoders/settings and test clips that you are using? I didn't see a link to the test specifics from the forum posts, but perhaps I didn't look hard enough.

It sounds interesting to me. There's even more bickering about which format is best for video than there is for audio

Yeah it is almost impossible to find people who agree 100% on a codec and settings.

I thought that RV9 was the best, followed by XviD, followed very closely by DivX.
Someone disagree ?

i prefer uncompressed avi



mike

Well, more or less yes. There will be specific software accompanied that makes sure you don't cheat (at some reasonable level, there is no way to be 100% sure) and that all users view the videos under almost the same conditions.

A list of codecs taking part in this test is:
1. DivX 5.02 Pro
2. DivX 3.11
3. XviD
4. On2 VP3
5. Streambox ACT-LT2
6. Apple MPEG-4
7. LSX MPEG-4
8. Sorenson SV3
9. Indeo 5.11

The list might change *slightly*

Unfortunately the Real SDK licence does not allow you to make a player that can play Real media AND AVI or whatever. I haven't tried contacting them though and see if they can make an exception. We'll see.

I am still looking for test clips, there are a lot of candidates in my hard drive
As for the settings, it depends on the clip and bitrate (various bitrates will be test).

Let me know if you want to join


@ Pio 2001:
I would put them in a different order but it's not significant. Many users of HA would spit on MP3Pro but ff123's test showed they actually liked it. Go figure.

Perhaps you might be interested in my objective color comparison application (gabriel.mp3-tech.org).

And yes I know it's only objective (ie not final judge), and no that is not just a set of eqm-like basic stuff)

I am working on the Objective side of that test too. It's gonna take me a bit longer than the subjective though so they will be published seperately.

What is your test based on? The docs are in French.

Well yes, unfortunately the docs are in french.

If you download teh win32 executable, there should be a little english text file provided with it...

It is based on several objective criterions. Those criterions are then applied on neighbourhood of pixels and not directly on pixels values (big difference compared to psnr that could be heavily influenced by a single pixel value change)

If you have a fast connection, you might want to check the slides, even if they are in french

Tell me if you need more specific details...
Also one bad point is that this was coded about 2 years ago under a linux platform. But using todays compilers, there seems to be big problems to compile the code.... (as an example, the win32 compilation is limited to quite small portions of a picture)

Blah. I'd go with HuffYUV then.

This will be quite difficult to do properly, as to mask which video codec is which it will be necessary to decompress and then recompress with some lossless codec (like hufyuv).

If you want to cheat there are several ways to do it and I am not willing to spend time to force people not to cheat.

Even if I do spend time to implement protections, they can always do something about it, so what would be the point?

Huffman coding is not uncompressed video.

Frank - dont be anal about it! We know Huffyuv isnt uncompressed, its lossless!

Ruairi

I'd be interested in participating. So divx 4 wouldn't be tested? Seems more valuable to have that than, say, vp3 - which some comparisons I've seen reveal to be extremely poor /w regards to detail.

I hope the tests include some anime footage, water footage, and quick camera pans of public in concerts.

I encode mainly from vhs tape, have tried divx 311a, divx 502p xvid, mpeg1 and mpeg2 (tmpeg). divx can get good but some persistent noise is always in, even in high bitrates.. a noise that stays static even if the video changes (like if you had your monitor dirty). Its not present in the others. divx311a can be tricky as it likes to drop frames (nandub used to fix that), and mpeg is good but need bitrate. Found that mpeg1 352x240 at 2000 kbps or higher looks good. mpeg2 needs too much more to be of any use (svcd levels tend to get too blocky for my tastes).

test will probably be a 2pass encoding to get the desired size? (at least that would make sense)

what u need in test clips imho:

1st clip
-scenes with lots of movements combined with long static scenes
(to see how bits are distributed, from what i saw so far divx 5.02 pro suffers in simple scenes just becaouse 2 much of bits is used for movement and contra seems to be true for xvid & divx3.11)

2nd clip (probably part of some trailer)
-hard one with fade in, fade out, dissolves, pans, zooms, action, ect
(lets see how they fall apart)

3rd
-normal film section mostly static
(to see the real 'sharpness' of the codec)

4th
-a tv and/or high quality video grab (to see whats happening at 4:3)


they should be encoded with two different average bitrates, maybe something like a. simulation of 1cd rip and b. bestpossible but not insane.


cough, just thinking out loud

Alrighty then.

http://www.everwicked.com/form/limited.php

Go there and apply, you will recieve instructions shortly.

Cheers