Ugly red in lossy video/image compression, General question Options

[phong]

This is a general theoretical question, not about any particular codec, for people who know more about this kind of compression than me. Why do saturated reds and oranges often look so bad (moreso than other colors) in compressed video? I've seen it in mpeg1, mpeg2, mpeg4, jpeg, etc. Saturated reds often bleed alot, or look blocky or pixelated around the edges. Sometimes there is a weird crosshatch/checkerboard pattern in large red areas (I see this a lot on my TiVo). A picture of a bright orange traffic cone taken with a digital camera often stands out as having really bleedy or pixelated edges. I'm not just talking about the bleeding you get from an oversaturated CRT.

So, the question is, Why?

[i4004]

human eyes are most sensitive to color red.
also, the problem is accentuated by 420 colorspace(that most codecs use) in general, and a low-resolution plenty of video these days has.

saturated red can't look good on (say) 320x240 because color resolution for that res. is only 160x120.

one more thing is problematic: if chroma is point-upsampled upon playback it won't look very good, and plenty of decoders do just that.
for illustration of that, load your clip to vdub and play with "chroma smoother" internal filter to see how could it look if better upsampler was used.

[smok3]

from http://world.std.com/~mmcirvin/bluesky/eyes.html :

The retina has three types of cones, light receptors that have sensitivity to different ranges of light frequencies. There are "red" receptors that are most sensitive to low frequencies, "green" receptors that are most sensitive to middle frequencies, and "blue" receptors that are most sensitive to high frequencies.

short story:
at the end one can use less info/resolution for storing red, more for storing green and again a little less for blue to get a 'decent looking' picture.

-----

you can start with the digital camera sensor for example, this is easy as you can think in RGB terms:
http://www.siliconimaging.com/Images/a_Bay...ern_Diagram.gif
(half of the info will be green, another half = red + blue)

[Gabriel]

There are several possible explanations:

*Most codecs are using subsampled chroma planes (4:2:0). You then have a few possible errors:
#chroma downsampling in the encoder: if you are not using a proper method, you can mess it up
#chroma upsampling in the decoder: a proper method is also needed
#chroma sampling position: as the chroma plane is downsampled, it is important to know what are the positions of the chroma points to be able to upsample correctly upon decoding. This is often ignored

*YUV colorspace:
Many codecs are using the YUV colorspace. If you look at this space (http://en.wikipedia.org/wiki/Image:YUV_UV_plane.jpg), you can see that orange is located in a corner. It means that the transition from orange to other colors is likely to require some rather big values to be expressed. As higher values usually mean a higher number of bits and/or bigger quantization, this is likely to cause artifacts on orange borders if you save too much bits there.

[i4004]

gabriel just repeated what i said (with addition of thing about "location of orange" that doesn't hold any water), and smok3 totally misunderstood a thing he quotted: they talk about light's spectrum frequencies.
same bandwidth is devoted to chroma components(u and v, which can ,in turn, be converted to rgb for display) in yv12, but humans just "better see reds".


red is evolutionary more important because it's more important to see blood, then to see trees or sky.

[smok3]

i4004: red is perceped faster, but needs less detail than green to be understood, so thats how it is sampled from the beginning (now thats not perfect in most cases of course, thats why the artifacts can be seen.)

the ccd sensor is a very good example and it is kinda similar to the chroma sampling used in different codecs (some ppl that deal with digital photography will use special sharpening methods to sharpen red more than other channels, especially if the specific image contains lots of red stuff - otherwise the image may look like it has some sort of wrong DOF).
p.s. check how the sigma sensors are done on the other hand (keyword:foveon)

YUV
since most of the Y component is usually 'green' you can expect that RGB calculation for display will get the most detailed info about green as well from all the components (at least thats how i understand it.)

[i4004]

1-retina has 3 types of cones. and there is no disproprotion in number of cones for these 3 types, right?
so how did you derive the notion that green needs most bandwidth?
again, you have misunderstood what that article talks about. it talks about how different cones are sensitive to differnett em frequency.
nothing more.
from a standpoint of human vision what does
"red is perceped faster, but needs less detail than green to be understood"
mean? we need the same "in-eye resolution" for red in order to percept it like other colors (and this is the fact)...the brain does the rest (accentuattes sensation of red relative to other colors)

2-http://www.siliconimaging.com/RGB%20Bayer.htm
you are taking this on a wrong level: the problem is not(!) stuff happening in camera, but problem arises when things get converted to yv12 and too low resolution is used (hello mpeg4 world).
i mean, OMG we would all be very happy if we could all use bayer(or any other sensore type) with no further chroma subsampling and on it's native resolutions: chroma subsampling is the problem here.
also, you are aware that bayer is not the only pattern used, right?

3-most of y component is...grayscale...heh..
uv carrie the colors.

4-how will you prove codecs devote less bandwidth to red?
please do, if you can.
or just tinker with vdub chroma smoother, as i suggested (for example on some vcd with reds)

[smok3]

1. i didnt derive anything, will comment this later if you really wish...

2. bubble, bubble, bubble.

3. Y = 0.299R + 0.587G + 0.114B
what does this tell you?

4. are you serious?

[i4004]

>3. Y = 0.299R + 0.587G + 0.114B
>what does this tell you?

this tells me how to get y from rgb components.
it doesn't tell me much more.

amount of one color in color-mix doesn't say it is more detailed.

also, you are aware that y alone can't be used to derive 3 colors, i hope.
this format mainly exists because of historic reasons of compatibility with old equipment (ie to separate y from chroma was invented to stay compatible with b/w tv-sets.)

edit/ smok3 edited hsi post, and so do i:

1. i didnt derive anything, will comment this later if you really wish...

well said, you didn't derive anything.

2. bubble, bubble, bubble.

you see a problem with uncompressed image sensor is sending to compressor?
WOW!
how's that for tinkering with electronic gadgets (to reach the image prior to compression) and simillar!
it's "babble", btw.

4. are you serious?

very serious. you said codecs devote less bandwidth to red, and i like you to prove this.
if you knew working principles of uv chroma components, you would of never said it:
gabriel linked this
http://en.wikipedia.org/wiki/Image:YUV_UV_plane.jpg
uv describe vectors that in turn describe colors, so how can it be color-red-aware at all? all colors will get same bandwidth in such uv plane.

ps.
foveon is also a new technology. how did we do cameras before this, and what portion didn't you like?
for example, what didn't you like on analog tv? it didn't have any of these color problems, because it's not working with pixels, and it's not aggresively subsampling chroma to spare bandwidth.

This post has been edited by i4004: Nov 16 2005, 14:36

[Gabriel]

gabriel just repeated what i said (with addition of thing about "location of orange" that doesn't hold any water),

Sorry, it seems that my explanation was not clear enough.
If you consider a set of time or spacial values and transform them into the frequency space, and higher contrast between the initial values will produce more high freq content in the frequency space.
Now, let's quantize this. The higher the quantization step for high freqs will be, the higher smearing in the original time or spacial domain will occur.
As a last step, lets have a look a orange. If is totally excentred in the UV plane. Of course, in a given interval, an extreme value is more likely to have a big difference from another one than a medium value. Orange is an extreme value in UV, and so is likely to have a big difference from another color, and this will cause a transient in the color space. With high freqs quantization, transients are likely to introduce smearing.

[i4004]

green and blue are also "excentred in the UV plane". and yet they're not so problematic (at first sight).

and red is more of a problem than orange.

and red is not at the edge at all.

heh..perhaps i need to make some illustration to make you see how b and g are equally problematic (on lores yv12 stuff)..once you take a closer look.

This post has been edited by i4004: Nov 16 2005, 14:42

[smok3]

i4004:

4. what happens when you feed xvid with 4:4:4? (yes it gets less red allready on input, what it does internally, dont know, read the source code if you really need to know.)

1. wikipedia:
The human eye contains photoreceptor cells called cones which normally respond most to yellowish-green (long wavelength or L), green (medium or M), and bluish-violet (short or S) light (peak wavelengths of 564 nm, 534 nm, and 420 nm respectively). The difference in the signals received from the three kinds allows the brain to perceive a wide gamut of different colors.
so no, its not red, mkay?

anyway, you seem to be only very good in twisting words, so iam no longer in this thread (bored, sorry).

[i4004]

uh, you're still not getting it.
the structure of that sentence is not
"The human eye contains photoreceptor cells called cones which normally respond most to yellowish-green (long wavelength or L), then followed by green (medium or M), and then followed by bluish-violet (short or S) light (peak wavelengths of 564 nm, 534 nm, and 420 nm respectively). ie receptors are tuned to yellowish green the best"

IT IS NOT.
read it again. it is essentially same sentence as you misinterpreted in your first post in this thread.

what they said is this:
"photoreceptors have best sensitivity to wavelengths of 564 nm, 534 nm, and 420 nm"
all 3 of them. have same sensitivity. to our eyes.
but how do you judge the eyes without the brain (this is getting pretty philosophical..heh).

here is what i have said. human eyes are (physically) equally sensitive to all 3 colors (same amount of color receptors), but human brain(which interprets it) is not. brain says red is most important.
now, can you prove me wrong?
(don't make it seem like we talk about SF so you can't prove me wrong on any of the stuff we're discussing, as i surely proved you wrong. just give me an example where "yellowish-green" mattered more than red. from your life. ).

i find it fascinating that you will say "its not red" almost as if you don't have eyes to judge with.
i myself like color orange the best, but i'm perfectly aware that red is what will get most attention.
red shoes, red cars, red lips, red blood.
not orange shoes, orange cars, orange lips or oranges.
or "yellowish-green" which is rather undefined.
how much percentage of yellow and how much of green?

as for xvid: if you feed xvid with 444, it will subsample the color to 420 yv12, which is a derivation of yuv colorspace with chroma reduced by factor 2 in both vertical and horizontal direction.
the chroma is reduced by mere halving the number of chroma pixels, NOT by reducing bandwith of some of the chroma components (which is, as i said, impossible to do here).
this is equally not SF, and codecs working principles are well understood(they must be, in order to apply them).

do you want some illustrations?
why not do it yourself?
ie, make some CG image with red green and blue dots or patterns...on (say) black backround. make it big.
now encode to xvid.
now downsize image by 2: encode to xvid again.
repeat
etc. untill you come to 160x120 or so.

now i usually don't mind stubborness in people, but i mind them not remembering how pc is excellent way to make experiments yourself.
no need for big discussion if all of us can experiment with our machines.
and machines can't get more versatile than pc.

if you like, i'll do it for you. i already have a pattern that i used to illustrate colorspace issues etc.
i'll just post few xvid clips and you'll get it.
but as i said, you can do it yourself. you can load images to vdub easilly.
it's better if you have that "aha" moment. i already had it plenty of times.
<wink>

to end this theoretical mumbo-jumbo(or "blah,blah" as you said..hehe) i just wanna quote how scharfi said it in very simple way:
"Also they are using a point sampling algorithm to convert YV12 to YUY2 which creates this ugly jaggyness."
http://md2.boerde.de/scharfi/

also, see this
http://forum.doom9.org/showthread.php?p=547849#post547849
he used blue there....he could of used green too...offcourse, red would be most eaisest to spot jagginess on. because of our eyes.

so please do few experiments (i mentioned vdub chroma smoother and progressive downsampling&encoding to xvid) and let's see what you come up with.

btw. i should also note that all stuff i said here goes for progressive footage! if we wanaa talk about interlaced yv12(of mpeg2 and mpeg4) we will end up on CUE and ICP issues, but that would take us too far.
these are also a bit more complicated.
(tivo (and any other laced mpeg2 system) will have that on the edges)

[i4004]

as words don't seem to help much here, here's a small illustration what can be done via proper chroma upsampling(on playback):
http://i4004.net/i4004/chroma%20on%20lores...0on%20lores.rar

feel free to zoom-in.

[phong]

Thanks guys, this is great. This question has been in the back of my mind for a while now, and I seem to have posted it in exactly the right place.