TAK 1.0 - Final release of the new lossless codec, offers superior performance Options

[TBeck]

TAK 1.0 - Final release

It's been a long time...

About

TAK is a lossless audio compressor, similar to FLAC, WavPack and Monkey's Audio. On average, lossless compression reduces the file size to about 50 percent of the original size; however the compression can vary between approximately 30 and 70 percent, depending on the file. Decompression restores a bit identical copy of the original audio data (that's why we call it lossless).

My goal was to develop a compressor which combines good compression with optimal decoding speeds. On average, the current implementation should match the compression efficiency of Monkey's Audio High, while achieving decompression speeds similar to FLAC.

Features

- Good compression. The strongest mode (Extra) is on par with Monkey's Audio High and OptimFrog Normal. TAK's fastest mode (Turbo) easily outperforms FLAC's strongest mode. This classification is based upon the evaluation of hundreds of files of different genres; it may not be the case for every file that you compress.
- Fast compression speed. Currently I know of no other compressor which compresses faster than TAK's Turbo and Fast modes, while providing similar compression ratios.
- Very fast decompression speed. The decoding speeds achieved are similar to, and often surpassing, those achieved by FLAC.
- Support for any common audio format (not fully implemented).
- Streaming support. An info frame, which contains all the information required to decompress a file, is inserted into the compressed audio data every 2 seconds.
- Error tolerance. A single bit error will never affect more than 250 ms of the audio data, as the compressed data is being stored in independent frames no more than that duration. The decoder is able to decompress even badly damaged files, by removing corrupt frames or replacing them with silence.
- Error detection. Each single frame is protected by a 24-bit checksum (CRC).
- Fast and sample accurate seeking. The file header contains a seektable with seekpoints (at 1 second intervals). In the absence of the seektable seeking can still be achieved by using the sync codes of the frame headers and the (optional) relative offset values contained in each info frame, which are pointing to the previous and next info frame.
- Meta data. A flexible and extensible structure in which to embed non audio data.

Changes

Improved:

- Synthetic Soul was so kind to correct my bad english in the ReadMe.

Fixed:

- Decoding with the command line version: if you specified wildcards for the file selection and the source directory contained only 1 TAK file, the decoder always threw the message 'File already exists'. Even worse: using the overwrite option in this situation led to deletion of the compressed source file!

- Usually the decoder ignores any data appended to the file end of the compressed file (for instance APEv2 tags). But it failed, if the file size was an integer multiple of the frame size (in samples).

Both bugs affected only the decoder, therefore there is no need to re-encode TAK files created with beta 2. Files created with earlier versions than beta 2 should be replaced.

Thank you

Thank you to all the members who have supported the development of TAK!

Firstly, you have provided encouragement to build yet another lossless audio compressor. Otherwise I would have dropped the whole project!

You helped to evaluate and optimize TAK. There have been 13 releases over a period of 8 months before the alpha version, and all of them have been evaluated by you!

Special thanks to the following members (in alphabetical order):

Destroid
Josef Pohm
Synthetic Soul

Download

The final can be downloaded from rarewares.org (Thanks to rjamorim!):

TAK 1.0 (Final)

Future

Some important features are missing in this release. Please find below my current to-do list:

- Plugins for other applications, such as Winamp or foobar2000 (a Winamp plugin is already close to completion).
- An SDK for other developers.
- Internal tagging.
- Unicode support.
- Piping support.
- MD5 audio checksums for verification and identification.
- A german version.
- Embedded cue sheets.
- Embedded cover art.
- Multi channel audio.
- Even more speed and better compression.

This post has been edited by TBeck: Jan 30 2007, 05:18

[cabbagerat]

Shade[ST] (welcome back!) raises an interesting point. Do any of the zip programs have strong encryption? I think WinZip may. You could always zip everything up, encrypt it, and store it online somewhere.

Zip the file, then encrypt the archive with a passphrase using GPG. WinZip does supports good (AES based, afaik) encryption, but the format is non-standard and may change in the future, rendering the backup useless. The OpenPGP standard is open, meaning finding a program to open your backup in the future should be trivial.

[MaB_fr]

Shade[ST] (welcome back!) raises an interesting point. Do any of the zip programs have strong encryption? I think WinZip may. You could always zip everything up, encrypt it, and store it online somewhere.

Zip the file, then encrypt the archive with a passphrase using GPG. WinZip does supports good (AES based, afaik) encryption, but the format is non-standard and may change in the future, rendering the backup useless. The OpenPGP standard is open, meaning finding a program to open your backup in the future should be trivial.

I'm sorry, i did not consult the old TAK/Yalac forum thread, so some point may be inacurate...

As a developper, my advice would be : open the source now with an explicit "not documented" sign on it.
Forbid anyone to modify, copy partially or fork it (by adding these specific order in each file near the copyright and the date).

In my opinion, keeping it closed at the time of release is a bad move. For many reasons.
At first, you end up with scepticism from people like me, who'd like to just take a peek at the code, just to be sure that the benefits claim are true and to be able to see the perks...
Also, you take the risk of fading from your own velocity (are you able to keep the pace with your users needs (in the codec department the needs are wide)...they will grow exponentially as you release the features...will you ?).
And last, are you accepting to shrink the range of users you want to target. Some of the developpers out there will very kindly port your soft in any language/plateform they see fit. It only depends of your willing, and you can adopt any politic you see as the good one (elect your own developpers on merit, choose some skilled friend, do a loterie, make a three month release cycle, plan "on-going" port, choose your birthday as the Java port release date, etc...).

I think, the important thing here is to make choice and communicate on it. A codec is an important piece of software. In our "media age", i may be the root of many uses. HydrogenAudio present some of the cutting-edge users in our current "audio world". Messing up with them is not something you should overlook, i would say...

I'm sorry to be the bad guy here, it may be a great advance in audio codec, but nowadays, we can't take anything for granted before reversing it (at least in software).

If you prefer to go closed, say it now. It's your choice, but you must be clear with all of your users.
If you go on radar now, you will endure the damage for a long time...

Sorry for the bad vibes

May the byte be with you...

MaB_fr

This post has been edited by MaB_fr: Jan 29 2007, 10:19

[gib]

At first, you end up with scepticism from people like me, who'd like to just take a peek at the code, just to be sure that the benefits claim are true and to be able to see the perks...

Now I'm not a programmer, so I might be way off base here. But why do you need to look at the source code to alleviate your skepticism when TAK has been heavily tested by many people and, more importantly, has been publically available for anyone to try for the last 3 iterations (2 betas and the final)? I can certainly understand preferring open source software, and this post is in no way a comment on whether TAK should be open or closed, but the notion that you need to look at the source code "to be sure that the benefits claim are true" strikes me as nonsensical.

[pepoluan]

At first, you end up with scepticism from people like me, who'd like to just take a peek at the code, just to be sure that the benefits claim are true and to be able to see the perks...

Now I'm not a programmer, so I might be way off base here. But why do you need to look at the source code to alleviate your skepticism when TAK has been heavily tested by many people and, more importantly, has been publically available for anyone to try for the last 3 iterations (2 betas and the final)? I can certainly understand preferring open source software, and this post is in no way a comment on whether TAK should be open or closed, but the notion that you need to look at the source code "to be sure that the benefits claim are true" strikes me as nonsensical.

I agree. Even looking at the source code is no way to determine if a complex algorithm (which I believe TAK is) works. The only way to know if it works or not is to test.

Thomas has provided the binaries. All you need to "... be sure that the benefits claimed are true ..." is to compress a WAV file using the binaries, and time it with whatever means you have. Then decompress it, again timing it. Then compare the result of the decompression with the original.

I am very sorry to say that some open source advocates in some way resemble audiophooles.

[...Just Elliott]

At first, you end up with scepticism from people like me, who'd like to just take a peek at the code, just to be sure that the benefits claim are true and to be able to see the perks...

Now I'm not a programmer, so I might be way off base here. But why do you need to look at the source code to alleviate your skepticism when TAK has been heavily tested by many people and, more importantly, has been publically available for anyone to try for the last 3 iterations (2 betas and the final)? I can certainly understand preferring open source software, and this post is in no way a comment on whether TAK should be open or closed, but the notion that you need to look at the source code "to be sure that the benefits claim are true" strikes me as nonsensical.

I agree. Even looking at the source code is no way to determine if a complex algorithm (which I believe TAK is) works. The only way to know if it works or not is to test.

Thomas has provided the binaries. All you need to "... be sure that the benefits claimed are true ..." is to compress a WAV file using the binaries, and time it with whatever means you have. Then decompress it, again timing it. Then compare the result of the decompression with the original.

I am very sorry to say that some open source advocates in some way resemble audiophooles.

For a codec, source code is a very important matter.

Now, if you made a convrter or player - fine, that doesn't matter.

If you're making a codec which people may rely on, you need to guarantee that:

- they will be able to use it on any platform with interest, or make it work themselves
- they will be able to verify the algorithm will always be 100% flawlessly lossless
- etc

Right now, it's a windows only closed source codec. This offers none of those benefits, and at least one person (me) is unable to use it or listen to TAK files at all.

[TBeck]

If you're making a codec which people may rely on, you need to guarantee that:

- they will be able to use it on any platform with interest, or make it work themselves
- they will be able to verify the algorithm will always be 100% flawlessly lossless
- etc

Right now, it's a windows only closed source codec. This offers none of those benefits, and at least one person (me) is unable to use it or listen to TAK files at all.

I am aware of this point of view, but repetition will not change anything.

I am already spending all (more than i should) of my free time for TAK developement. There simply aren't any resources left. The neccessary source code conversion can start, when TAK's feature set is quite complete and i have time again.

While i respect, that some people insist on the source code, i can not agree to all of their arguments.

"windows only" sounds very limiting, but given it's huge user base i don't have to be worried about lack of potential users before the source code release.

- they will be able to verify the algorithm will always be 100% flawlessly lossless

Only in theory.

The source code is quite complex. Someone would have to be very knowledgeable and spend very much time to find errors. Would you guarantee that this will happen? If the source code has been released, will you wait until some expert has checked it before using the codec? How do you know, if a trustable expert has checked it?

There are open source codecs available since years, and the developers are still finding bugs. Obviously source code availability can not provide you any guarantee, that the code is bug free.

This post has been edited by TBeck: Jan 31 2007, 19:53

[rjamorim]

"windows only" sounds very limiting, but given it's huge user base i don't have to be worried about lack of potential users before the source code release.

Besides, linux whiners can use wine.

[SebastianG]

Besides, linux whiners can use wine.

wine is nice but I've yet to figure out how to pass PCM data via a pipe or fifo to a "wine"-ed encoder. I'd like to avoid intermediate .WAV files for transcoding. Doesn't seem to work with wine. (I only tried it with Nero's aac encoder)

This post has been edited by SebastianG: Jan 31 2007, 22:52