Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Format wars, why particular codecs win (Read 5940 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Format wars, why particular codecs win

Dear Hydrogen Audio people,

While reading a very neat thread by A_Man_Eating_Duck on lossless compression :

http://www.hydrogenaudio.org/forums/index....showtopic=61054

some interesting things were brought up as to why various formats were popular.
Both lossless and lossy.

I don't really want to address the comparison of file sizes, I want to address from a non audiophile perspective why I think some formats "won" the format wars and why I personally chose the formats I prefer. So new thread.

-=-

I am not an audiophile, I have some hearing impediments that do not prevent me from thoroughly enjoying quality music but do undoubtedly prevent me from hearing some of the subtle nuances that make some music great. To me, with very few exceptions, oggenc with q3 is transparent. Stock old version of libvorbis as packaged by RHEL. I suspect most here need a little higher than that to not hear differences. At least with stock libvorbis 1.2.3.

I'm also not that much into musician fanboyism. I like Bob Dylan, I always have. I did not know until earlier this week that the picture of Bob Dylan on his debut album had been reversed. Apparently that is rather common knowledge among most Dylan fans.

So I think I have a slightly different perspective than most here on things like why certain formats are preferred.

My personal preferences : flac for lossless and ogg vorbis for lossy. I'm not a xiph fanboy though, I prefer h.264/avc over theora - I still don't know about WebM, I'll wait and see, but anything video I currently do I use h.264/avc. For speech I prefer Ogg Vorbis over speex.

Mentioned in that thread:

Quote
trev said:
to me those graphs don't mean much at all without respective encoding times (and possibly decoding times).

it's always a balance of compression Vs speed. you can't judge on just compression alone.


Encoding time doesn't really matter to me. That only matters to people encoding large volumes of music at the same time. For me, encoding is something that can be taking place while I'm doing something else, either on my computer or elsewhere, and does not require my attention, so whether it take 5 minutes or 15 minutes to encode a CD is not relevant.

Ripping - I do all ripping from Linux using cdparanoia. I rip each CD 3 times into 3 different directories, and then calculate md5sum of the resulting waves. If there is no difference, I assume the rip really was lossless.

On some older CDs (especially bought used) that are messed up, it sometimes takes doing this process several times before I get enough matching md5sums that I'm confident. Most CDs all three rips are identical. Encoding time doesn't matter to me, even the slowest encoders are faster than ripping a CD 3 times so my personal paranoia that cdparanoia didn't get a perfect rip can be satisfied.

Decoding time doesn't matter to me. I don't play flac. If oggenc -q3 is transparent to me, what's the point in playing lossless? Every now and then when new version of libvorbis finds its way on my system, I do mass re-transcode. That's easy to script and I can let that run without paying attention, so again, decode time doesn't matter to me.

Filesize does. Well, use to, I currently have terrabytes of unformated space on my logical volume but historically I had to have part of my flac collection on DVD-R.

So why do I choose flac instead of (insert codec here)?

Three primary reasons:

A) I use Linux as my primary operating system and it comes bundled with every distribution out there. It is easy to install on OS X and Winblows too.

B) Ogg Vorbis is my lossy codec of choice and it is cake to transcode from flac to Ogg Vorbis. oggenc reads flac native and copies all the meta-information over (er, except album art, not sure what's up with that but I don't really care) exactly how it is in the flac file.

C) Bob Dylan. I have never seen him in concert but I've heard plenty. Yes, that is my one technically illegal activity, but it is restricted to Bob Dylan bootlegs and never his commercial stuff, never other artists, and I don't keep them around, I'm not a collector. But I really like to hear them. They usually are traded by collectors though and collectors have for whatever reason chosen flac (even though I'd personally prefer to save bandwidth and grab lossy).

Why bootleggers have chosen flac, I don't really know, only thing I can think of is that they tend to be hard core music fans and hard core music fans that understand how to get a recording onto disc and sound good are probably somewhat intelligent and people who are somewhat intelligent are more likely to use FOSS because of the modification that can be done to meet their needs. Even if they don't code, it's easy to apply a patch and recompile. Also, the philosophy of personal privacy that is prevalent among FOSS advocates lends itself well to those who want to do something technically illegal.

That's a lot of conjecture on my part, but I think that's why bootleggers have generally chosen flac. It's the choice of those creating the concert sets that get distributed that then results in the leeches such as myself using the format they chose.

Also from thread:
Quote
user said:
Somehow i want to appeal to the Lossless crowd, to think about customers power.
If we want in sooner or later future more (cheap/priceworthy) standalone and portable devices, which play Lossless, we should maybe concentrate on 1 preferred and used Lossless format. And, that is somehow obviously FLAC with already good acceptance by commercial industry.
If the industry has an easy decision to take, which Loslsess format to implement, they will be easier to convince, to just do it.
If the Lossless format usage is way more distributed, let's say: flac, wavpack, apple Lossless, ape, tak all used in similar percentages by end-users, industry will have difficulties to decide themselves, to implement any of Lossless. Or they take the "wrong" format, maybe 'Orange'-Lossless, which we wouldn't really want ?!
So, let's make FLAC the most used format, as it is, so we will get more and even more hardware support. portables, stand-alone HiFi, car-HiFi.


I don't think any lossless format will ever make it as common in general consumer market. Even with ever increasing disk sizes, people want that disc space for other things. Videos, games, etc.

That post was made in 2008 so I can't fault him too much but another issue is cloud storage and portable devices. Everybody and their brother it seems has smart phones, tablets, other devices that they largely use to play their music, with their music frequently stored on a cloud. The cost of bandwidth and data transfer times make lossless economically not feasible for cloud based storage of your digital music. The bill at the end of the month would simply just be too big. I don't understand why iTunes and Amazon etc. use such high bitrates in their 99 cent lossy offerings as it is, that just wastes bandwidth when the music is served over 3g networks with the customer paying for their data usage.

lossless is not for the typical consumer. It's for audiophiles, collectors, and people like me (not really either) who simply like to be able to re-transcode to newer lossy w/o needing to re-rip.

Now on to lossy -

Quote
CoyoteSmith said:
mp3 won the lossy war because consumers chose it, flac is the consumer choice for lossless so when drive size continues to rise there will be much more interest in it, i know people that say defensively "there's no way you can hear a difference between LAME mp3 and flac" but as drive size rise, they'll be answering themselves "well i have the drive room, so why not?"


Quote
simonh said:
I'm not so sure CoyoteSmith. Mp3 won the war because there was no real competition. Now there is competition, most consumers aren't interested. Quite right too. Thanks to Lame (no thanks to Franhofer), mp3 is still holding its own competing with the latest codecs.

There are some people on this forum who have abx'd enough samples to not trust a lossy codec, but they are a tiny minority. Consumers don't abx. Consumers don't know or care about lossless. I only know a couple of people who have even heard of lossless audio codecs. I don't know anyone who knows what replaygain is. Personally, I'd much prefer to see replaygain/gapless hardware support for mp3 than lossless support. Thats just me though.


I think the reason mp3 "won" the lossy war was because of napster and winamp.
mp3 was the format of choice for pirates. The only other real choices at the time, if I remember, were WMA and real. I don't remember winamp playing WMA, I don't think it did back then, but winamp was a hell of a lot better than either Windows Media Player or Real Player, so winamp is what people used to play their music and mp3 worked well with winamp so mp3 is what was traded around the napster network.

I was using MacOS 8 back then, I remember there being extensions you could just drag tracks from the CD to the desktop and they would encode to mp3 on the fly. I couldn't encode to either WMA or Real. When I got into Linux (1998) again, mp3 was something I could encode to but WMA and Real were not.

That's why mp3 "won" - it was ubiquitous and you could play the songs in WinAmp and xmms.

I no longer prefer mp3, I use Ogg Vorbis now.
Why do I use Ogg Vorbis?

A) Vorbis is the only quality FOSS codec that can have their encoder distributed without patent royalties. That means with any Linux distro it just works. With sox it just works. With audacity it just works. With FireFox and midori and opera it just works. Yes, I have purchased the fluendo plugins so I can still play mp3 in anything gstreamer, but I would have build a gstreamer plugin to encode and I would have to rebuild sox and I would have to build support for audacity etc. - all I can do, but I don't have to with Vorbis, it just works. I suppose that won't be an issue after 2017 and is not an issue for people using Windows Media or QuickTime based media applications.

B) Tagging. Tagging via Vorbis Comments is just so much nicer than it is with ID3vX. UTF8 works so I don't have to worry about setting locale to something else to tag foreign characters. I'm not limited with genre. I can use multiple genre to make playlist generation so much easier. Maybe I want songs that were both pop and one hit wonders. I can search for ogg files that are tagged as both those genres. etc. The freedom I have with Vorbis Comments is just so much more than it ever was with ID3vX in mp3.

I like Shakira, especially her Spanish stuff. I ripped her CDs in iTunes to mp3 way back when iTunes was new. I tagged them with the accents etc. where appropriate. I transfered them to my iPod mini. They did not display correctly. I did find I could get them to display correctly if I encoded / tagged with lame as long as I was careful to set my character encoding in the shell first (leaving it UTF8 in Linux gave same problem iTunes did) - what was causing the issue was simple lack of UTF8 support in ID3. That's not a problem with Vorbis Comments, it's got UTF8 support so I don't have to worry about changing my character encoding when using certain characters.

C) There is no C. Having it just work on the operating systems I primarily use and superior tagging, yup, that is pretty much it. OK, maybe there is a C. It seems the FOSS nature of Vorbis is allowing it to pick up popularity in the gaming world. Will that, combined with its use in WebM, allow it to dethrone mp3 as the more common lossy format?

-=-=-=-=-=-

What is a non audiophile doing hanging around HA?
Well, I am a geek, and this place is well known in the geek world as THE place where intelligent audio discussion takes place.

I'm currently working on a community social network project that hopes to be friendly to local community bands, including a digital distribution channel for their music.

We will be using html5 audio to allow people to hear their music. That mean we need it encoded in both Ogg Vorbis and MP3. But we have concerns about MP3, notably the cost of the patents. With respect to lame, I don't think we have to worry, I think we would only need license if we were distributing lame itself. I don't think we need to license it to compile it and use it.

But putting the mp3 in html5 audio is broadcast and that may actually prevent us from being able to accommodate Safari and IE users (unless they have the right xiph plugin). It looks like we might be OK because it looks like they won't come after us until we are grossing a certain amount, but grossing that amount does not equate to netting enough to be able to afford it. We may experiment with only being ogg and seeing how the community reacts. That also would save money on the CDN.

That's a reason I personally hope Ogg Vorbis picks up. If Microsoft and Apple shipped with decoder plugins, we wouldn't even need to worry about the extra expense (CDN space and potentially license fees) that mp3 will cost us. I suspect we'll have to do mp3 though.

But that's why this non audiophile is here, if we have any chance of success I need to know more about this stuff. Do it right or not at all.

Format wars, why particular codecs win

Reply #1 –
Quote from: AliceWonder on
Ripping - I do all ripping from Linux using cdparanoia. I rip each CD 3 times into 3 different directories, and then calculate md5sum of the resulting waves. If there is no difference, I assume the rip really was lossless.


Just curious as to why you don't do ripping using something that can use ACCURATERIP database (like EAC or dbpoweramp).  With your work flow, you're ripping the the SAME disk with the SAME drive. So errors could slip through.  Alternatively, with ACCURATERIP data, you'd be comparing your rip to other people who have ripped the same disk. So when you get a match (of even "1", but in some cases hundreds of matches), you are confirming that someone else with a DIFFERENT copy of the same disk, using a different disk drive, got the exact same data as you. That is an infinitely more powerful check on having a good rip, than your comparison of 3 rips on the same machine.  Anyhow, just curious.

Quote from: AliceWonder on
What is a non audiophile doing hanging around HA?


I'd say that the majority use of the term audiophile around these parts has a negative connotation. The word audiophile has been ruined by all the audiophools out there. But HA is refreshing as it avoids most of this sort of nonsense.

Format wars, why particular codecs win

Reply #2 –
Quote from: garym on
Just curious as to why you don't do ripping using something that can use ACCURATERIP database (like EAC or dbpoweramp).


Last I checked, EAC and I believe dbpoweramp were Windows only. I don't "do wine" - I have in the past, it's always disappointing.

Quote
With your work flow, you're ripping the the SAME disk with the SAME drive. So errors could slip through.


Yes, but they would have to be bit for bit identical errors to slip through.

Quote
Alternatively, with ACCURATERIP data, you'd be comparing your rip to other people who have ripped the same disk. So when you get a match (of even "1", but in some cases hundreds of matches), you are confirming that someone else with a DIFFERENT copy of the same disk, using a different disk drive, got the exact same data as you. That is an infinitely more powerful check on having a good rip, than your comparison of 3 rips on the same machine.  Anyhow, just curious.


I may code something up to check that database.

Quote from: AliceWonder on
I'd say that the majority use of the term audiophile around these parts has a negative connotation. The world audiophile has been ruined by all the audiophools out there. But HA is refreshing as it avoids most of this sort of nonsense.


To me, an audiophile is someone who deeply loves music and the audio technology that brings it to them, both software and hardware.

Format wars, why particular codecs win

Reply #3 –
Quote from: garym on
Quote from: AliceWonder on
Ripping - I do all ripping from Linux using cdparanoia. I rip each CD 3 times into 3 different directories, and then calculate md5sum of the resulting waves. If there is no difference, I assume the rip really was lossless.


Just curious as to why you don't do ripping using something that can use ACCURATERIP database (like EAC or dbpoweramp).  With your work flow, you're ripping the the SAME disk with the SAME drive. So errors could slip through.  Alternatively, with ACCURATERIP data, you'd be comparing your rip to other people who have ripped the same disk. So when you get a match (of even "1", but in some cases hundreds of matches), you are confirming that someone else with a DIFFERENT copy of the same disk, using a different disk drive, got the exact same data as you. That is an infinitely more powerful check on having a good rip, than your comparison of 3 rips on the same machine.  Anyhow, just curious.



Ahh, forgot about linux. these are win programs. But are there no linux programs that can access the ACCURATERIP database when ripping?

Format wars, why particular codecs win

Reply #4 –
Apropos your mentioning Ogg Vorbis for playing over the internet, you should research Opus, which has streaming and suchlike high amongst its various goals and is proving highly capable and competitive even in this relatively early phase of its existence; it may well end up being the successor to Vorbis, not only in spirit (function) but  specifically as a part of Xiph/Ogg.

Format wars, why particular codecs win

Reply #6 –
Quote from: AliceWonder on
Ripping - I do all ripping from Linux using cdparanoia. I rip each CD 3 times into 3 different directories, and then calculate md5sum of the resulting waves. If there is no difference, I assume the rip really was lossless.


Rubyripper does that automatically.

Quote from: garym on
But are there no linux programs that can access the ACCURATERIP database when ripping?


Morituri supports AccurateRip.

Format wars, why particular codecs win

Reply #7 –
Quote from: AliceWonder on
Quote from: garym on
Just curious as to why you don't do ripping using something that can use ACCURATERIP database (like EAC or dbpoweramp).


Last I checked, EAC and I believe dbpoweramp were Windows only. I don't "do wine" - I have in the past, it's always disappointing.


EAC has long worked through WINE without problem or disappointment.

EDIT:  Skamp has good alternatives above.

Quote from: AliceWonder on
Quote from: garym on
With your work flow, you're ripping the the SAME disk with the SAME drive. So errors could slip through.


Yes, but they would have to be bit for bit identical errors to slip through.


Which is very likely.  Random errors are unlikely in ripping.  Systematic errors are what you need to worry about, and your methodology does little (to none) in the way of systematic error prevention.

One would expect the encountered errors to be bit-identical.
Creature of habit.

Format wars, why particular codecs win

Reply #8 –
You don't use windows, but for those that do and want to confirm *existing* rips, there is now a way to do that with foobar2000 (file verify w/accuraterip) and PerfectTunes (beta), a new product by Spoon, developer of dbpa, etc.

Format wars, why particular codecs win

Reply #9 –
I laptop (Thinkpad T520) does run Windows 7 - but I don't want to use it for media ripping.
I do have foobar2000 on it though

Format wars, why particular codecs win

Reply #11 –
Quote from: AliceWonder on
What is a non audiophile doing hanging around HA?

au·di·o·phile [aw-dee-uh-fahyl]
noun
a person who is especially interested in high-fidelity sound reproduction.

Who doesn't?

Format wars, why particular codecs win

Reply #12 –
And a case could be made that the levels of effort and knowledge regarding audio ripping, formats, organising, etc. demonstrated in the OP might be admissible as qualifications for the term “audiophile”!  . . . at least in a positive sense, as opposed to the various negative connotations bestowed upon it by less objective people who use it as a badge.

Format wars, why particular codecs win

Reply #13 –
There was just recently a thread about the lossless 'format war' over at http://www.hydrogenaudio.org/forums/index....showtopic=95670 . The 'lossy' format war differs in that one is stuck with old files unless one has a lossless original.  That gives whoever ruled the ground first, a stronger upper hand.


Here's some milestones according to Wikipedia:

1993: the MP3 ISO standard
1994: mp3's show up on the internet
1995: RealAudio
1997: AAC is ISO-standardized. Winamp is launched, with MP3 support. And mp3.com starts up.
1998: Fraunhofer announces plans to charge for MP3 use, prompting development of competition:
1999: http://www.microsoft.com/en-us/news/press/.../WM4Lnchpr.aspx
2000: first Ogg Vorbis release
2001: the MP4 container format (ISO-standardized 2003) -- was there really any other way to tag AAC files before that one?


Now if anyone had the power to turn the tides in 1999, it would have to be Microsoft. True, Microsoft did a few less successful moves with WMA; the ridiculed claims that at 64 kb/s it was superior to MP3@128 and practically at CD quality ... this was in 1999; also, WMA looked to be a platform introduced to replace MP3s by DRM in a time where everyone hated the unplayable fake CDs. But if Microsoft couldn't fight off MP3 in 1999, then that's a pretty good sign that it was already too late. I'd guess that nobody could take down MP3 back then.

Except Fraunhofer, of course. And maybe Microsoft's (and Monty@Xiph's) timing even saved the MP3 format from being killed by royalty charges. 


Was AAC too late? I'd say it was.  It was the MPEG, with Fraunhofer on board.  They designed something to improve upon and replace their own baby, and failed. But again, was there really anything but tagless raw AAC in 1997?


So what is left to explain, I think, is that RealAudio failed.  Pretty much everyone hated RealPlayer.  It was privacy-infringing to “Don't be evil, be even worse” extent.  And it was used for a plugin for content that the users weren't supposed to get their hands on, and had to fight Flash.

Format wars, why particular codecs win

Reply #14 –
Morituri is by Thomas Vander Stichele - I seem to recall him being one of the GStreamer hackers, a good guy too.

Format wars, why particular codecs win

Reply #15 –
Quote
With respect to lame, I don't think we have to worry, I think we would only need license if we were distributing lame itself. I don't think we need to license it to compile it and use it.
Technically, I believe you are supposed to have a license to encode MP3.    I'm pretty sure they have never gone after individuals, but if you are going to distribute, I'd recommend a licensed MP3 encoder just to be safe.  (Winamp Pro and the non-free versions of dBpoweramp include fully-licensed MP3 endcoders.)

As far as distributing MP3s, MP:3licensing.com says:
Quote
No license is needed for non-commercial activities...  or for entities with associated gross annual revenue of less than $100,000 USD.
 

Format wars, why particular codecs win

Reply #16 –
I'm basically in about the same boat as Alice.

I use Linux and BSD on everything I own so any audio in the house is going to be played in those environments, and FLAC and Vorbis "just work" in Linux.

I'm not a super stickler for tagging awesomeness, and really only care that tags mostly work. Vorbis comments or ID3, fine by me.

One area of difference is that I DO play FLAC files at home. I simply find no need to maintain another lossy copy of my archive to play on my LAN.. At 1Gbps on the home LAN a raw PCM stream would be playable many, many times over let alone a compressed FLAC file.

I couldn't care much less about encoding time. I encode FLAC at -8 and it still outruns by CD reader by a healthy margin.

I stick to Vorbis for portable audio where storage space is an issue. I obviously don't have super ears because to me Vorbis is REALLY hard to ABX at q2. So I can encode at q2, roughly 96Kbps on average, and fit far more of my collection in a limited storage device than if I used LAME, which in my experience reaches the same quality at -V4.

On devices that only support mp3, I always rely on LAME at -V4 to -V2, depending on the storage space available on the device.

Format wars, why particular codecs win

Reply #17 –
Quote from: yourlord on
I use Linux and BSD on everything I own

[...]

On devices that only support mp3, I always rely on LAME at -V4 to -V2, depending on the storage space available on the device.


Thought of http://khenriks.github.com/mp3fs/ ?

Format wars, why particular codecs win

Reply #18 –
Nope, but an interesting thing to add to that would be the ability to specify what format to transcode to on a mount point..

Also, I don't know if anyone here has played with KDE much, but it's file manager (Dolphin) has a pretty awesome feature. When you put in an audio CD and try to open the volume in Dolphin it gives you a lot more than you'd expect.

Here is the description from the manual for it:

"You now notice that Dolphin is proposing a WAV file for each track, plus:

    a CDA folder containing indexing information in the usual CDA format.
    a Whole CD folder, containing one file for each format (.cda, .flac, .mp3, .ogg, .wav) holding all the tracks
    a FLAC folder, containing the tracks encoded into FLAC format (lossless information format)
    an Information folder containing the CDDB informations
    an MP3 folder, containing all the tracks in MP3 format
    a Ogg Vorbis folder, containing the tracks encoded in OGG format

You then just have to copy the folder of your choice, in your preferred format to obtain the relative encoded version of your CD! "