I've been encoding a lot of files lately with Lame, and instead of trashing the wav files afterwards I'm going to compress them with one of the available lossless encoders, either FLAC, LPAC, or WavPack. I've tried all 3 on well over 16 gigs of wav files. FLAC and WavPack both shrunk that 16 gigs down to 10 gigs on the hightest compression ratios. With LPAC, I got the size down to about 15.8 gigs, or roughly about 200 mb smaller.

If space is the only consideration, the smaller the file size, the better. However, being the perfectionist nut that I am, that is not enough. When I look ahead, I think about other things, like which of these encoders is open source, which is not, which seems to be more actively developed, etc. FLAC would seem to offer the best option for this line of thinking. I also dig WavPack's lossless and lossy modes, and the new beta is definately sweet. But at the same time, I don't want to encode all my files with a beta encoder...

I'd appreciate your thoughts on the matter, as I know a lot of you here have probably pondered the same thing sometime in the recent past.

Cheers,

Jason

Have you tried monkeys audio?
Should have best compression.


http://www.monkeysaudio.com/


I just lpac because mpc can encode directly from that.
So I rip to lpac and encode later to mpc.
But that's not an important thing for you I guess.


Jan.

You're looking to archive your audio files by losslessly encoding them. So, you'll want to be able to *decompress* them years into the future, possibly using a different OS to the one you are currently using.

For these purposes, anything which does not have either an open source decoder or a detailed file specification is useless. This rules out LPAC and Monkeys Audio.

Your only two options are Shorten and FLAC. Shorten has the advantage of being used by some real life applications (there's a Shorten based audio file sharing network out there). FLAC has the advantage of being actively maintained, and having better compression ratios than Shorten.

You're right about that. FLAC is probably the best bet for my purposes. I'll look into Shorten and get some benchmark numbers to compare to what I've already tried. Indeed, I could need access to these files later on Linux and I'd be totally screwed if I only had a windows decoder.

Any other comments are welcome...I appreciate your responses.

Thanks,

Jason

Of course, if you had to you could always reinstall win98 or equivalent in a few years just to decompress your archive to .wav.

I wouldn''t worry too much about future support and development... if your archive was large enough, the space savings by using the highest compression would give you room to also archive the OS and software needed in the future to decompress.

Use what's best right now, within reason of course. If things change dramatically later, simply decompress and recompress using the then best available compressor. It's lossless afterall.

LPAC has a Linux decoder....You can use that too.

LPAC has a 'Linux' and 'Solaris' version. I imagine that these are Linux x86 and Solaris SPARC. Useless if I were using Linux ARM or Solaris x86 or NetBSD PPC or HURD (in some hypothetical distant future... ).

The point is that - if you are archiving something, you need to know about the file format, so that if necessary, you, or a coding guru friend of yours, could write a decoder for the files. Closed file formats are just an all round bad idea.

I agree 100%. There's absolutely NO reason to change to open source lossless compressor because of the fear closed source coder may become useless sometime in the future. That's rediculous.

With lossy encoding there could be concern that support will end if the decoder is closed source. But with lossless coder you just install older/another OS and decode-recompress without any quality loss...if there's ever need for that.

I agree, but there are still other advantages to open source file formats, depending on what you want to do. If you're a coder yourself, it might be nice to be able to write yourself little utilities (or Perl scripts) to mess with the files (or get statistics on them, or whatever you want to do). Of course that's only one consideration, and depends on what you personally are interested in.

Not ridiculous at all. What's going to happen to the historians of the future when they try to research current trends, only to find that there aren't even specifications for the format the data they want is in?

So, you can use a lossless encoder that produces files that *anyone* regardless of processor type or OS can read, or you could use a lossless encoder which you can't even guarantee will still work for *you* tomorrow.

LOL. Lets say I want to use Monkey's Audio now. If somebody wants my Monkey's Audio files and can't play them, that's his problem to decode-recompress, that's not a lot to ask. I want the best compression now.

How the heck Monkey's Audio could possibly become useless in a day? And it's almost sure that something better than MA will come up in the future, so there's regardless what you use now a strong possibility that you're gonna recompress again.
And with lossless it takes just a bit time, no loss of quality.

So it's absolutely BS to try to frighten people with closed source issue in this case. Monkey's is out there in the Internet and isn't simply gonna disapper like that, probably never.

I'm a bit on the fence with this issue. I use Monkeys Audio, it does have the best compression. But I also respect and usually prefer to use open source software in general. Matt says he's going to open his source one of these days - soon hopefully.

True. But that wasn't my point. Firstly, you're being very selfish if you are 'sharing' files using Monkeys Audio, given that it is a completely Windows based system. If it's only for your own use, then of course you can do what you like (as long as you don't mind staying in Windows).

Secondly, you don't know what the decoder and encoder *do*. Hypothetically, the encoder and decoder could both refuse to work after April 2002, or after the 1000th processed file. They could be set up to store information about you and upload it on a certain date. Without disassembling the code, you can't be sure that these won't happen. This is not scaremongering -- an inherent limitation of closed file formats and closed programs is that you have to trust the providers, not only to not screw up *now*, but not screw up at all in the future as well.


This has been said by many people in many different areas -- and it happens very rarely. I reserve my right not to believe a single word of what he says until 'one of these days' arrives.

So you also believe that evil Matt "the satanic Monkey" has written trojan procedures etc. into his code, and it will stop working when the next eclipse of the sun happens?

You can express your concerns to Matt and every MA user at MA forum:
http://www.monkeysaudio.com/cgi-bin/YaBB/Y...i?board=general

Not to start a flame war here, and I'll start off by saying that I do believe in Open Source (but I don't necessarily believe that closed source is always bad), but I'd have to say that this ideology is a bit flawed IMO.

Now, lets think about this for a minute. Do you actually inspect every line of code in all the open source projects that you use? I know that I don't. I don't have the time to do this, and I don't necessarily have the knowledge to do it effectively in every case. If someone wants to hide a trojan somewhere, it wouldn't be particularly difficult to hide it in open source over closed source, depending on the situation. Sure... the likelyhood of it being discovered in the Open Source project is much higher than that of the closed source project, but if you look at how many people actually contribute to Open Source audio projects.. the number is not significant IMO so such a scenario could easily go unnoticed in either case.

What I'm saying is basically that IMO this argument holds very little water. Sure.. it's a possibility, but there's also a possibility that a simple bug could exist in an Open Source project that erases all your data. Really.. its not impossible

Ok.. enough of my ranting

Man, I did not exactly expect this kind of feedback...but I welcome the open discussion. I didn't intend this to be an open source vs. closed source issue, but rather was wondering more about how people weigh their choices when it comes to using lossless compression. I'm not listening to my lossless compressed files, so the quality of their sound is not very important to me...however, the compression ratio is, and as well the possible lifespan of the compression technology being used to create the archives.

I get more of a warm fuzzy feeling from open source than I do closed, if not for anything but that fact that if the main developer of the project goes awol or just decides he's not going to do any further development, anyone else can pick it up and keep the development going. Even on closed source projects, the main developer may be eager to hand the code off to someone else to keep it alive, but this doesn't happen the majority of the time. Just IMO, take it for what it's worth...I don't code any audio related applications, but I have done a lot of work on the Linux front, all open source...and at the same time, at the office I work mostly in an all Win32 environment writing a ton of closed source VC++. I do that because it pays the bills, but I love the open source market so much more.

Jason

Matt - the "evil satanic Monkey"... ROFLOL Good one John!
Matt's actually a pretty cool guy.

This really quickly becomes a matter of personal choice once you get down to the last best contestants.

Monkey's has the best compression. If you need cross-platform though, it's out. flac is the best if you need really wide cross-platform support, and lpac & flac are pretty close if you're only in Windows, Linux, and Solaris x86. mpc's direct lpac support is cool, but it doesn't seem to be really different than flac -d foo.flac | mpc -o foo.mpc. Unless you're in Windows.

Personally, I use flac, because I want the linux support, and using open-source gives me a warm fuzzy feeling. If Monkey goes open source and cross-platform and is faster and better, then I'll switch to it.

As far as inspecting every line of software that you compile, well...
Ken Thompson said it better than I could....

Sigh.. why don't we just wait for Ogg Squish?

Using winrar at setting "good" and multimedia compression "on"...
I get 24% compression....


And I think winrared files will still be usable in 100 years.. Am I correct?


Mikez

Oh, and I totally agree with Jon Ingram.....

Openup your source or you can not be trusted. That's just very simple. That's why no boady trusts microsoft to do their most demanding jobs..



Mikez

I certainly see the value in Open Source (and I support it myself), but I believe there is a point where advocacy can go beyond reason. To me, for the most part software is about functionality, not (usually) about philosophy -- I will use the software that works best if I am able to and there are no major downsides such as legal issues or something else which actively (not "theoretically") prevents me from doing so. This "best" software doesn't always happen to be Open Source software, so when I see people pull the "well it's closed so who knows.. maybe this person is evil and wants to take over the world with his audio encoder, etc, etc" I just can't help but find it a bit flawed.

People can argue all day the merits of Open Source (and some point are very valid), but to those which say there is no merit in closed source software which happens to work really well (APE, MPC, and thats just in the audio field).. I'd say that's a bit of an ignorant approach.

That being said.. for software projects like Monkey's Audio and MPC, I see no reason they shouldn't be Open Source, considering how they are not commercial, but that's not up to me. That doesn't mean I'm going to ignore them though. Functionality is more important to me than licensing, because at the end of the day I want to actually use something that works rather than advocate for something which may not necessarily work (or at least work well enough).

Well, we don't want to wait 5 years

Seriously, FLAC has full Ogg support, so a new Squish may never come.

Monty's argument for FLAC (or Shorten): there are no decoders for the other formats that work on his computer (a PowerPC)

I would like to see MA open sourced, and it's nice that Matt talked about it, but I doubt it's going to happen.

--
GCP

Very nice point, with which I agree 100%

I've had a quabble with RMS over this, with him basically demanding that I remove important functionality from my program because it worked via non-free library.

He made very good points for doing so, but in the end I prefer not to screw the part of my users that don't care about freedom (would be like 95%) and just want to use my program. Forcing freedom onto people does not make them free.

As a compromise, I made two versions. One worse that is totally Free and another that is better but does not come with source.

Anyone can decide for himself how much freedom he wants.

--
GCP

I agree with you - the best software isn't always (indeed - is rather rarely ) open source. My beef wasn't with closed source programs, but with closed source programs together with proprietary *file formats*.

Perhaps I get a little too dogmatic when I write past 1am, sorry about that , but I'm not a leet 12 year old Microsoft basher.

I don't like seeing closed file formats get so prevalent that major effort has to be spent reverse engineering them, particularly when solutions without any of these problems already exist. It's a similar thing in compression, watching in horror at all these Windows people using closed or improperly documented formats, or in word processing, watching the Word DOC format becoming the 'industry standard'.



MPC is in a slightly different category -- you might not be able to *encode* in the future, but you will at least be able to *decode*. I would currently say that there *is* no merit *for me* in APE at the moment (suppose I wrote a better lossless compressor, which was only available for OS/2 - how much merit would that have?) , although it has the potential to have a lot -- it does after all compress much better than the alternatives.

I agree. I'm a BIG fan of open source but you can't punish yourself by limiting yourself ONLY to open source. For example, I don't use my Linux box for many of my computing needs. Because of functionality issues. And I have been following the FreeDOS project for a couple of years now. In a few years it MIGHT be as functional as MS DOS was TEN YEARS AGO.

Just a personal comment: open source is different from free software.

I believe in open source, but not in free software.

Open source just means that you can see the source code. To my mind, this is a key point for open knowledge.
On the other hand, free software is a philosophical thing where you believe that every software should be totally free (both in price and freedom), and reject "non-free" software.


As an example, Lame is open source, but not free software.

Huh? Who do I send my check to???

Lame is only free in term of price, but not freedom.

Sorry for getting horribly offtopic, but here is another thing that I've been pondering about.

Consider that, for example, for such and such reasons MPC developers all switch to Windows systems and nobody really works on Linux anymore.

Now they add feature X, which isn't supported by my Linux decoder of choice (the XMMS plugin).

Who's going to add it? How many people can make such plugins and grind down the MPC source code?

Same example, but XMMS is deprecated and a new app gets to be the standard Unix player, with a different plugin system. Of course, due to changes in the the Linux OSS layer, XMMS no longer works.

Another example:

Say that all Vorbis developers get sued for whatever, and corporate America being what it is, they all get locked up in jail.

Then, Intel/AMD's 64-bit CPU comes out at $1 a piece, everybody buys one. And you try to get your Ogg decoder compiled on it.

But it won't work. There's an incompatibility between the Ogg source and the new compiler for that CPU.

Who's going to fix it for you? Everybody in the know is in jail.

I realize this may all sound farfetched, but my point is that you should not really count on being able to work with those old files unless someone is actually actively supporting it.

I know a company that is currently massively screwed in a similar way. They were selling DOS apps. Hey, it was DOS, but the app was very good in what it did. But MS totally dropped DOS support in Windows XP. They may have the source to their app, their still screwed.

--
GCP

A point which I'd like to make is that open source is in no way a guarantee for open knowledge. Patents will stop that. (MS 'shared' source anyone?)

Free source OTOH (at least the GPL) will ensure that you can use the knowledge without fear of patents.

(This is also the reason why LAME being GPL is 'questionable', to say the least. As far as I understand you can only use it for 'educational purposes', i.e. to learn or experiment with MP3 encoding)

--
GCP

(This is also the reason why LAME being GPL is 'questionable', to say the least. As far as I understand you can only use it for 'educational purposes', i.e. to learn or experiment with MP3 encoding)


Hey, that's all I use LAME for...don't know nothin' bout playin music...

You're right, and so Lame is LGPL.
Remember that the lgpl only covers the source code, in the same way a copyright protect a book.
Lgpl doesn't say anything about the algorithms, in the same way a copyright on a book doesn't cover the ideas exposed into the book.

So the lgpl protect the way the algorithms are written, but absolutely not the algorithms themselves.

And yes, open source gives you access to knowledge. But it doens't means that you'll be able to use this knowledge freely.
Lame source code offers knowledge, but you're right, you can't use this knowledge freely.