You don't want Ogg sound, you want Vorbis sound.
Ogg is a container format, like AVI, Quicktime, etc.
Vorbis is an audio format, usually wrapped in Ogg.
Some time ago, people tried to put Vorbis audio inside AVIs. This would have worked, but for deficiencies in MS's implementation of AVI. As a consequence, AVIs with Vorbis audio are basically unplayable in Windows (although they may work if you don't seek or pause).
Because the AVI file format is so limited, people have been searching for replacements. There's the well designed but patented to hell MP4 format (which will become a commercial standard soon), the Ogg container format (which is very good at containing for streaming, but not so good for some other uses), and a scheme called MCF currently in development specifically designed to be a Movie Container Format.
http://www.doom9.org/ogg.htm is a very good guide to encoding into Ogg (or .OGM, as the Windows people seem to like to call OGg Movies).