TAK 1.0 - Final release

It's been a long time...

About

TAK is a lossless audio compressor, similar to FLAC, WavPack and Monkey's Audio. On average, lossless compression reduces the file size to about 50 percent of the original size; however the compression can vary between approximately 30 and 70 percent, depending on the file. Decompression restores a bit identical copy of the original audio data (that's why we call it lossless).

My goal was to develop a compressor which combines good compression with optimal decoding speeds. On average, the current implementation should match the compression efficiency of Monkey's Audio High, while achieving decompression speeds similar to FLAC.

Features

- Good compression. The strongest mode (Extra) is on par with Monkey's Audio High and OptimFrog Normal. TAK's fastest mode (Turbo) easily outperforms FLAC's strongest mode. This classification is based upon the evaluation of hundreds of files of different genres; it may not be the case for every file that you compress.
- Fast compression speed. Currently I know of no other compressor which compresses faster than TAK's Turbo and Fast modes, while providing similar compression ratios.
- Very fast decompression speed. The decoding speeds achieved are similar to, and often surpassing, those achieved by FLAC.
- Support for any common audio format (not fully implemented).
- Streaming support. An info frame, which contains all the information required to decompress a file, is inserted into the compressed audio data every 2 seconds.
- Error tolerance. A single bit error will never affect more than 250 ms of the audio data, as the compressed data is being stored in independent frames no more than that duration. The decoder is able to decompress even badly damaged files, by removing corrupt frames or replacing them with silence.
- Error detection. Each single frame is protected by a 24-bit checksum (CRC).
- Fast and sample accurate seeking. The file header contains a seektable with seekpoints (at 1 second intervals). In the absence of the seektable seeking can still be achieved by using the sync codes of the frame headers and the (optional) relative offset values contained in each info frame, which are pointing to the previous and next info frame.
- Meta data. A flexible and extensible structure in which to embed non audio data.

Changes

Improved:

- Synthetic Soul was so kind to correct my bad english in the ReadMe.

Fixed:

- Decoding with the command line version: if you specified wildcards for the file selection and the source directory contained only 1 TAK file, the decoder always threw the message 'File already exists'. Even worse: using the overwrite option in this situation led to deletion of the compressed source file!

- Usually the decoder ignores any data appended to the file end of the compressed file (for instance APEv2 tags). But it failed, if the file size was an integer multiple of the frame size (in samples).

Both bugs affected only the decoder, therefore there is no need to re-encode TAK files created with beta 2. Files created with earlier versions than beta 2 should be replaced.

Thank you

Thank you to all the members who have supported the development of TAK!

Firstly, you have provided encouragement to build yet another lossless audio compressor. Otherwise I would have dropped the whole project!

You helped to evaluate and optimize TAK. There have been 13 releases over a period of 8 months before the alpha version, and all of them have been evaluated by you!

Special thanks to the following members (in alphabetical order):

Destroid
Josef Pohm
Synthetic Soul

Download

The final can be downloaded from rarewares.org (Thanks to rjamorim!):

TAK 1.0 (Final)

Future

Some important features are missing in this release. Please find below my current to-do list:

- Plugins for other applications, such as Winamp or foobar2000 (a Winamp plugin is already close to completion).
- An SDK for other developers.
- Internal tagging.
- Unicode support.
- Piping support.
- MD5 audio checksums for verification and identification.
- A german version.
- Embedded cue sheets.
- Embedded cover art.
- Multi channel audio.
- Even more speed and better compression.

I just wanted to say congratulations. It's been exciting following the development of TAK from your very first post on this messageboard all the way up to this v1.0 release.

I also want to say thank you. TAK looks very impressive and I plan to migrate my lossless archive over to TAK in the near future. Specifically, whenever a foobar plugin is made. No hurry, though. I'm a patient fellow.

What is TAK?

What are the advantages over rivals?

askoff, you're kidding, right?

Did you really miss all these threads? (and all the buzz generated by this new codec?)

http://www.hydrogenaudio.org/forums/index....c=43179&hl=
http://www.hydrogenaudio.org/forums/index....c=43289&hl=
http://www.hydrogenaudio.org/forums/index....c=43493&hl=
http://www.hydrogenaudio.org/forums/index....c=43494&hl=
http://www.hydrogenaudio.org/forums/index....c=50958&hl=
http://www.hydrogenaudio.org/forums/index....c=51565&hl=

The main advantage of TAK is the fact that it offers compression ratios comparable to Monkey's Audio with decoding speeds comparable to FLAC. In a way, the best of both worlds. Like many other HA members, I'm very excited with this codec.

TBeck, congratulations on this release!

Just read a bit here at the forum

Thomas, congratulations to you. TAK gained much interest here and sure looks very promising incredibly good!

I can't believe that it's been almost ten months since you first mentioned your new codec... time sure does fly!

Congratulations on getting to this point, and good luck for the future. Oh, and thank you! Thanks for sharing with the community, and for keeping me occupied with the alpha and beta testing - it's been enjoyable.

Edit: perhaps askoff knows Tak by it's "codename", Yalac?

Edit 2: It may be worth pointing new users to these posts:

• Case's Tag updated to recognise Tak files
• foosion's TAK Decoder Stub for foobar2000 (tag (not play) Tak files with foobar)

Fine!

This release is definitely a break for me, something has changed. Surely i will work further much on TAK, but this exciting creation phase is over, the baby is there!


BTW: I forgot to write you, that i have deceided for 'TAK' instead of 'Tak'. Probably a bit surprising but when i read the title of the readme, 'Tak' looked a bit lost. Well, some insight into my very elaborated deceision making process...

Thomas

I'm sorry if I hurt someones feelings for asking so simple question, but I guess I'm not onlyone who doesn't know what it is. And if I have managed so far without it I'm quite reluctant to use lots of time to search what it is.

Not mine.


I thought it was ok to post it into 'Lossless / Other Codecs' without a description, because here are so many posts regarding Tak. If i had posted it in the news section, i would have added a description. One reason why i haven't is my bad english...

Thomas

Edit: I will copy the feature list from the ReadMe into the initial post.

Uh, what about unix users?

I'm glad Marie Curie didn't have the same attitude...

Those of us with a more amenable disposition are clearly quite enthused though.

In my understanding the intention is that TAK will be ported from Delphi to C at a later date, during which time an SDK, source, and no doubt *nix release will become available. This is no doubt simplifying things and perhaps totally inaccurate, I'm sure Thomas will correct me when he is back on. In essence though, you have no option at this time, but have not been forgotten. It is still early days (currently no method for playback for any OS!).

Edit: A relevant topic, although not answering your question directly.

freepascal supports Linux, FreeBSD, Mac OS X/Darwin, Mac OS classic, DOS, Win32, OS/2, Netware (libc and classic) and MorphOS ...

Do we have a media player that can play Tak? Hopefully, foobar2000 will have one soon.

Good job on the release and I look forward to testing it out.

Glad to see that people read my posts.

No, no playback! Thomas is working on a Winamp plugin at the moment.

Just like to add my congratulations!! TAK is a big step forward for lossless compression.

Very good kwanbis, that's the point! And FreePascal was intended to be sourcecode compatible with Borland's Turbo Pascal and Delphi V7. Maybe TAK can be compiled with it?

btw, Pascal is a very nice language. Learning to write human readable code with Pascal is much easier than with C++

Excellent release. I hope the source will be released soon also.

I also agree with kwanbis and Sunhillow.

Kinda offtopic but... in fact I'm trying to make a Pascal wrapper for foobar2000's API. Then from there I am planning to make 2 foobar2000 plugins, one a visualization plugin, and the other a gapless UI plugin.

I hate C/C++

I was going to leave this for Thomas to answer, as he is the only one who can properly answer this, but I 'm bored, and this thread seems to be going down a singular path.

It is my understanding that the key point here is that Thomas wants to redevelop the code structure before doing anything else. This possibly includes even a foobar component.

If TAK is compiled for *nix as is then users will also expect a method to play back the files. Then Mac users will feel left, then Solaris users... All this detracts from the idea of getting the source into a form that makes it easier for the majority of third parties to utilise (SDK, libraries), and that can be released to the public.

In essence, by spending time porting and developing player components now Thomas would not be spending his time developing the source to be released for others to do with as they please (differing OS'es and various media player components). It is a long-term gain, not a short-term.

Thank you Neil. This is nearly correct. My plan for the further development wasn't too clear before the final release, because i was too involved into the preparation of the release. I needed a bit of mental distance to feature and optimization issues.

To understand my current plans it is important to be aware of TAK's situation after the final release:

1) TAK is out, it has some nice features (especially the speed/compression proportion) which may attract a fair amout of people. But they will soon find out, that some important features are missing: Player support and internal tagging. Because i want to avoid that people are loosing interest because of those limitation, it's very important for me to add those features as soon as possible.

2) If the source code was available, other developers could work on support for other applications. But it isn't and it will take a long time to bring it into an adequate form. TAK development can not be suspended until a source code release.

It's also important, that the preparation of a source code release is definitely no fun for me, it's very much of mostly boaring work. I will not spend the next 2 or 3 months on this work.

BTW: I don't want to discuss what 'adequate form' means, i definitely will not release any source code i am not feeling comfortable with! While i appreciate earlier offers to help me with the source code, i am convinced, that i have to do it by myself (but i will ask for help on specific issues). If other developers would perform the source clean up and conversion from Delphi to C, i would have to check, if they did it right. Given TAK's complexity, this would also be very time consuming and boaring.

My current plan:

1) The missing features have to be added by me.

2) I just started the work on a SDK. It will consist of a binary library, which other developers may use to write plugins for other applications. It will grow step by step. The first release will only provide decoding funktions for (seekable) Tak streams. I will use my existing Delphi code for this.

3) Source code conversion will be performed in small (bearable) portions, when i have time. Absolutely no promises about a release date!

Like Neil said, a WinAmp plugin is in the work. Possibly it's already good enough for an alpha release. But i want to use the plugin as a test platform for my first SDK version. Therefore i want to wait with a release until the SDK is ready. Urgent user demand may change this...

I hope, it's a bit clearer now. I really wish i would be better in writing english!

Thomas

Great to see that TAK have reach version 1.0, as there is a market for more lossless codecs.
But I am really sad to see that your current to-do list don't mention releasing binaries for non-Windows OS'.

This part of Neil's post is absolutely correct! While i myself will work on a binary SDK for windows users, support for other OS' probably will have to wait until the source code release!

Exception: If i have time and should find out, that it's easy to compile TAK with Free Pascal, i may provide a command line version for other OS'.


Please see above.

Sorry. I should work faster...

Thomas,

Congratulations and thanks for your efforts. I have been following your efforts since the start but never posted. I think you have pushed the boundaries of lossless audio technology and for that you should be commended. I can't wait to start using TAK in foobar and my hopefully-soon-to-exist HTPC!

Ok, I understand. Support for non-Windows OS will have to wait until you release the source code.
I just got the impression from kwanbis' post that the current source code could be compile for most OS, using freepascal.

I know that this is not you highest priority, but hope it will at least make it into your to-do list.

Since TAK is really interesting for all those people who want to archive their CDs my high-priority-list would be:
- Internal tagging.
- Embedded cue sheets.
- Embedded cover art.

Wow great codec...I like FLAC but hates how it takes forever to encode it if you want decent compression...if this gains some more support (like: plugins, being used by more than just HA users etc.) this will become my (and many others) lossless codec of choice

Congratulations Thomas on the final release. I've been silently following the development of this codec and I must say it is one of the most exciting things that emerged on HA lately. Very impressive work and thank you for sharing it. I think it was on April Fool's Day that TAK was first mentioned and most people took it for a joke. That fact alone should make it clear how impressive TAK is.

Regards

Upon further reflection it occured to me that, due to my non-fancy way of archives my CDs, I can really start using TAK right now. I don't really need a foobar plugin (though it would be nice), and I don't care about any embedded stuff or the like.

I bought a few CDs yesterday. When I get around to ripping them this weekend I'll be archiving them in TAK format. I don't know if anyone else has begun using TAK for true archiving rather than merely testing, but if I happen to be the first, I'll be proud.

hmmm. i can't use TAK due to its buggy WAV handling. everytime i create a .wav file from foobar2000, TAK complains that it can't read the file...

Strange, all the WAVEs that I have used for testing were created by foobar (from WavPack archives).

Can you elaborate?

Bear in mind, as detailed in the readme, that the current implementation can only process "Sample rates from 8000 to 96000 Hz. Bit depths of 8, 16 or 24. Channels: Mono or Stereo.".

the wav file that i made came from an mp3(wanted to test out TAK's prefilter). but for some reason, TAK doesn't like it.

what's also weird is that if i use LAME to decode the mp3, the wav file will play nicely with TAK.

Perhaps you could take a look at the WAVE file in foobar ("Properties" > "Properties" tab) and report the details. You can highlight all the info using Ctrl+A, and copy all info including names using Ctrl+Shift+C, e.g.:

File Name : 06.wav
File Path : D:\FLAC Corpus\source\06.wav
Subsong Index : 0
File Size : 98 031 404 bytes
Last Modified : 2006-05-03 19:19:14
Duration : 9:15.733 (24507840 samples)
Sample Rate : 44100 Hz
Channels : 2
Bits Per Sample : 16
Bitrate : 1411 kbps
Codec : PCM
Encoding : lossless

I wonder whether you are creating 32 bit WAVE files from foobar.

As you probably can imagine, i am absolutelyl interested to clear this up!

I am a bit surprised because of the huge count of files (from the tests i know it has to be far more than 1000, one secret tester compressed and verified more than 60 GB without any trouble!) that have been tested without problems.

The only problem i can remember was the 2 GB file size issue and this has been fixed.

There must be something very special with this file. I really would appreciate, if you could provide the information Synthetic Soul asked for. This could be helpful.

And what exactly is the error message?


Thomas

I only know of two conditions, which possibly have not been tested often:

1) 8-bit Mono files.

2) Wave files with non audio meta data appended to the end.

Is it possible, that Foobar writes additional meta data to the file? Possibly the content of the ID3 tag?

Could someone please answer this question?

foobar2000 did it (optional writing of APEv2 tags at the end of the file) in the past, but this feature was removed since 0.9

For good reasons i asume...

Thank you!

Then this is one possible reason for the failure, although TAK'a ability to read trailing meta data has been tested (a bit)...

I think i just found a bug in foobar2000 by accident...


the wav file is

and the original mp3 file is



i think the only thing which causes that is mp3packer since i used it on the mp3 file.

also as i said, LAME makes a good wav file so no worries about that.

also, just so i'm clear, i'm using the latest version of foobar2000.

edit:also, the original mp3 file is not 32-bit. it's a 16-bit file with 2-channels

Thank you very much for the fast reply!

This lightens me enormously!

Thomas

As Sintetic Soul suggested, you are creating a 32bit floating point wave, not an 8/6/24bit integer wave. In other words, it is not mp3packer, nor a bug in foobar2000.

Simply change the "preferred bit depth" in foobar preferences ( tools- converter , and then at the bottom ).


"Also", mp3 has no bitdepth. only PCM (wave) does.

To make it clear: Mangix' problems had not been caused by a bug in TAK, but by his Foobar configuration. Foobar created a 32-bit floating point wave file instead of the standard 16-bit integer format.

Rock on Thomas! This is really a revolution in lossless audio. I'm glad all the testing is done and the format is finally public. Though when such breakthroughs happen, you should always have a copy of your code with someone you trust (IMO), in case something bad happens to your computer, yourself, etc. (Not that I wish to worry you, of course)

Should i take care for this before visting my dentist in about 3 hours?

Only if he has criminal friends who know where you live...

Shade[ST] (welcome back!) raises an interesting point. Do any of the zip programs have strong encryption? I think WinZip may. You could always zip everything up, encrypt it, and store it online somewhere.

yes

edit: Good Morning Synthetic

Zip the file, then encrypt the archive with a passphrase using GPG. WinZip does supports good (AES based, afaik) encryption, but the format is non-standard and may change in the future, rendering the backup useless. The OpenPGP standard is open, meaning finding a program to open your backup in the future should be trivial.

I'm sorry, i did not consult the old TAK/Yalac forum thread, so some point may be inacurate...

As a developper, my advice would be : open the source now with an explicit "not documented" sign on it.
Forbid anyone to modify, copy partially or fork it (by adding these specific order in each file near the copyright and the date).

In my opinion, keeping it closed at the time of release is a bad move. For many reasons.
At first, you end up with scepticism from people like me, who'd like to just take a peek at the code, just to be sure that the benefits claim are true and to be able to see the perks...
Also, you take the risk of fading from your own velocity (are you able to keep the pace with your users needs (in the codec department the needs are wide)...they will grow exponentially as you release the features...will you ?).
And last, are you accepting to shrink the range of users you want to target. Some of the developpers out there will very kindly port your soft in any language/plateform they see fit. It only depends of your willing, and you can adopt any politic you see as the good one (elect your own developpers on merit, choose some skilled friend, do a loterie, make a three month release cycle, plan "on-going" port, choose your birthday as the Java port release date, etc...).

I think, the important thing here is to make choice and communicate on it. A codec is an important piece of software. In our "media age", i may be the root of many uses. HydrogenAudio present some of the cutting-edge users in our current "audio world". Messing up with them is not something you should overlook, i would say...

I'm sorry to be the bad guy here, it may be a great advance in audio codec, but nowadays, we can't take anything for granted before reversing it (at least in software).

If you prefer to go closed, say it now. It's your choice, but you must be clear with all of your users.
If you go on radar now, you will endure the damage for a long time...

Sorry for the bad vibes

May the byte be with you...

MaB_fr

Now I'm not a programmer, so I might be way off base here. But why do you need to look at the source code to alleviate your skepticism when TAK has been heavily tested by many people and, more importantly, has been publically available for anyone to try for the last 3 iterations (2 betas and the final)? I can certainly understand preferring open source software, and this post is in no way a comment on whether TAK should be open or closed, but the notion that you need to look at the source code "to be sure that the benefits claim are true" strikes me as nonsensical.

He came from nowhere at the fool's day last year and announced that he's intend to create a new lossless codec that have almost all virtues of current ones. I guess a lot of people here thought it's just a joke. Now here it is, a mature product.

Thanks TBeck for all your effort. I'll try it when some of the major players support the playback.

Danke, Anke!

A bit of reading would have saved you and me time. I am really boared from having to answer the same questions over and over again. Earlier i thought, a faq could help, but it wouldn't: if someone isn't willing to look into some recent posts, how should he find a faq?


I couldn't say it better!


Thank you for telling me what to do. I hope, it was ok to answer this question in the previous page of this thread, or was this too late?


It's perfectly ok to have a different opinion, but please don't waste my time!

Thomas