Beta release of TAK 1.1.1 ((T)om's lossless (A)udio (K)ompressor)

It consists of:

- TAK Applications 1.1.1 Beta
- Winamp plugin 1.1.1 Beta
- Decoding library 1.1.1 Beta
- TAK Software Development Kit 1.1.1 Beta

The final release of the SDK will additionally contain some source code for the TAK container.

Download:

Click to view attachment
After the beta test phase this archive will be removed.

What's new

New Features:

- In very rare cases the presets -p3 and -p4 could compress much worse than the lower presets. A new filter in the encoder will nearly eliminate this annoying effect. It can also increase the average compression by a tiny (<= 0.05 percent) amount. Because of the filter, encoding with -p3 and -p4 will be a bit slower.
- Creation and verification of MD5 checksums of the raw audio data. The file info command can show you the MD5.
- Option to lower the process priority. Nice for background processing.

Improvements:

- Up to 9 KB smaller binaries. Although i have removed a lot of the assembler optimizations, the speed is still very close to the previous version.
- Further clean up of the Code.

Modifications:

- Support for seek tables removed. The new version will neither add seek tables to newly encoded files nor use seek tables contained in files created with older program versions. Important: Seeking in files without seek table is only supported since V1.1.0. Please update the WinAmp plugin and/or the decoding library for full seeking support in media players.
- There is a new metadata object which contains position and size of the last frame in the stream. This info is useful for seeking and tag detection.

Known issues:

- If you use pipe decoding and the application reading the pipe is beeing terminated before the whole file has been read, TAKC may get into an endless loop and has to be manually killed with the task manager. I don't think this is a big issue but i will try to fix it in one of the next versions. BTW: Big thanks to shnutils for testing the pipe decoding!
- There seem to be some compatibility issues with pipe decoding to some other applications ("crc1632.exe" has been reported). I will try to fix it in the next release.

Beta testing

The beta version has already gone through extensive testing performed by my automatic scripts. But especially because of the many changes for 1.1.1 rare bugs are still possible (as always...). Please try the beta release and report any bugs in this thread.

I would also be happy about tests of compression efficiency and speed. Because the final release will have identical performance (there may be a speed variation of 1 to 2 percent because of different code alignment of another build), it does make sense to test the beta.

Thanks for testing and have fun

Thomas

Thanks Thomas!

Thanks for your hardly work!
Could you change the low priority process function into that I could specific what level I want in the next version?
I find a little question in the 1.1.1beta version, the encoded file by 1.1.1beta version is be identified as 1.1.0 in foobar2000.I use foobar2000 0.9.6.3 beta1 and foo_input_tak.dll 0.4.2, maybe foo_input_tak.dll need to rebuild for this new version?

I don't really like to. It would require more explainations in the documentation and i like to keep it as simple as possible. Furthermore the meaning of the levels is likely to be different between different operating systems.

But if you don't feel comfortable with my level-choice (it's IDLE_PRIORITY_CLASS), feel free to tell me.


Sorry, my fault. I hadn't changed the codec version constant in the applications. Thanks for the report!

I don't think it's neccessary to release a fixed beta now?

Thomas

TBeck I am really impressed with TAK but I am reluctant to use it because of it being closed source. I read some time ago that you were planning to license the source under a permissive license once the code was cleaned. Is there any progress regarding that?

aliumalik +1

I still follow tak development, but I will use flac as long as tak is closed.
For pure compression/storage, I prefer to buy a new HDD than using a closed codec.
I only wish I could enjoy tak's speed/compression improvement over flac.

Try again.

Sigh...
Every time there's a new TAK topic, someone has to bring this up...
Thomas will release the source when he thinks it's the right time, this was discussed in EVERY single TAK topic.
This is getting a little old don't you think?

... and everytime there is a new release it is not the right time ... sorry I have been following tak since the time it was called yalac so I have been waiting a long time ... I have been more than patient ... I recall a time when monkey audio developper was telling that he would maybe release the code ... specially as third party software developpers were asking him to do so ... it never happened ... (the source is available but the license is ... weird)

always telling: "later" is a clue that tak doesn't share the way of thinking of free software ... who care if it's bugged as long as it is instantly fixed ... free software developpers release opensourced code specially to find bugs ... the more eyes see the code, the less buggy is the code ...

years ago Tom had an excuse ... he was not releasing the code because he wanted to write a research/scientific article about his new technology & be the first to publish...
at time this was a very valid excuse to me ...

I never saw the article, I never saw the code ...

I am not crying for the code, I am only pointing the lack of honesty ... if Tak is closed source software, so be it, but plz tell it openly ... so that I don't long for more & that I can pass my way when I see a new Tak release.

Besides porting to other platforms, I doubt there will be much more improvements aside from Thomas himself. You can see that the main development of both WavPack and FLAC are still from the original authors.
OSS is good, but it's not like there's a ton of people that have the knowledge/interest in audio encoding just waiting to get their hands on a new code.

Thank you for adding Low Priority option, TBack
Now, I'm waiting patiently for 1.1.1 to be finalized, and then I'm going to migrate to TAK.

Educate me. Why are you reluctant to use TAK in its current form/release?
Are there people waiting in the wings wanting to improve it? Should I worry about some backdoor/malware? Some other reason?

Just curious, because personally, I can't read code well enough for it to matter to me.

Code being free generally ensures longevity, because although you (and me) don't write code, almost certainly someone who can, can pick it up, keep it up to date, port it to other, newer systems, etc. In 20 years from now it's quite possible Thomas will have lost interest and we no longer use Windows on x86 machines. If the source is open, anyone who can code and has an interest can port it to whatever system he choses so that you can keep decoding your music (and encoding).

Indeed, it's very useful to have the source in the collective subconscious, and I'd be a bit happier to if it was open (and I hope that it will be in the not too far future). But fortunately TAK is a lossless codec, so its demise and the need to convert music encoded with it won't cause data loss like it would in case of a lossy one So it's pretty safe, even if it's closed at the moment.
So I'm waiting patiently.

Thank you for supporting me. But i think, it's totally ok to ask those questions, if someone is really interested into using TAK. It's different, if they are only beeing asked for some general ideological reasons.

While i don't want to criticize personal subjective preferences (it's a matter of taste i am also affected by in different fields...), there have been some statements i have to comment:

Despite beeing a bit harsh, this is useful for me. I think you are right, that it's time for me to make a definite statement about an open source release of TAK. I will do this in a separate post within the next days.


I don't think there have been many bugs in TAK releases. There can be minor issues with the beta releases, but please take into account, that i don't have any testers outside of hydrogen. At the time i release a beta, nobody else has tried the new version.

I am sure even the most prominent open source projects are obtaining bug reports following a beta release.

I can't empirically deceide, if open source projects are suffering from less bugs than commercial projects (for instance Internet Explorer vs. Firefox). I have the feeling, it's true for some of them. But i think, there are of other important reasons to be taken into account:

- Because open source developers are working on the things they like, they will be far more motivated and feel much more responsible (identification) for the results of their work.
- Open source releases are not subjected to the same time constraints as many commercial projects, where the software often has to be released at any price.

And the same is usually true for the developers of Freeware...


I already said, that i see the need for a general statement about an open source release of TAK. Therefore i understand your demand for a definite answer. Where i can't follow you is the harshness... Did i do you any harm because i haven't released the source code yet? Did it affect your live in an important way?

There simply is something you would like to have, but not in it's current form. I think, that's live...


Thank you. Just one thing to add: My internet page about TAK is part of my presentation as self-employeed software developer. What do you think would happen, if i published malware???

Thomas

I think people have given most of the reasons why I want it to be OSS. For me personally
1) Cross platform usability. I use both linux and windows and most of my music is stored externally so I need for it to be playable on both platforms. Most linux players cant/won't include support for for such formats because a) they have a conflicting license or b) they are shipped as a binary blob which causes other problems. In such a scenario you have to resort to reverse engineered decoders which might not be legal and considering that the TAK community is so small someone might not even care to reverse engineer it.
2) Future development. In the (improbable) scenario that TBeck stops development on TAK someone will be able to provide at least playback support on new platforms. I know that not many developers would get involved in the development process but at least someone would be able to provide playback support/small fixes etc in the future IF development on TAK is stopped.
3) Market adoption. I know this is a far fetched idea but if TAK is kept closed source it might not even see the small hardware adoption FLAC has right now. I would like to at least have a possibility of it getting implemented in some hardware.

There are other small details which make users like me reluctant. Don't get me wrong TAK is a great codec and the reason so many people are nagging regarding this issue is because they want to use it without any apprehensions. Most of the people here continually shift between file formats and it might not seem a big deal to them but for the "encode and forget" user base such little things can be problematic. In the end while TAK compresses better than FLAC and it is indeed an excellent product it does not offer the peace of mind FLAC does.

PS: If TBeck decides to license it under an OSS license I would suggest dual licensing under GPL/LGPL. Thanks for all your work and contribution to the lossless community TBeck whichever way you decide to go.

Could you please try, if the low priority setting fits your needs?


Thank you, that's very clear and thoughtful.

And there is no contradiction from me.

But i have to take some other factors into account, when thinking about an open source release. I will post about it as soon as i find some time to write it down.

Thomas

I already tried this beta, and it's exactly what I need Thank you again.

By the way, I wonder whether TAK will report if there's decoding errors, like, decoded files are not the same as original files (MD5 mismatch)?
For instance WAVPACK, it will show decoded file's MD5 and original file's MD5 when decode. FLAC will tell if there's MD5 mismatch and will not decode those files. Since I've never have corrupted TAK files before I wouldn't know how TAK will report in such situation.

And.. how about files that I encoded before MD5 option was implemented (TAK 1.0.4 and earlier)... if in the future, my files become corrupted, can TAK tell me whether those files are corrupted or should I re-encode using newer versions?

TAK already stores a 24-bit checksum for each and every frame, so this should not serve as a reason to re-encode.

Thanks, I just need a confirmation for peace of mind.

I understand. I went through a bit of paranoia over transcoding lossless formats in the past, which is why I remember this.

At last, I'm back home! I tried your new decoding library with foobar, and its speed seems to be pretty much the same level as 1.1.0, in spite of the removed asm routines. Tested on a Prescott P4 (workplace) and a Conroe Core2 (home). We've got a Brisbane X2 too, but it's just another K8 what you've already tested anyway

So how does this compare to lossless formats like flac? does it encode smaller files?

Yes, it does and with even less encoding time than FLAC.
See this comparison page for more details.

This is actually quite dissapointing. I tried to add TAK support to CUETools using pipe encoding/decoding, but encountered this problem too.
As container format is still a mystery, i have to start the decoder just to find out the basic information about the file, such as sample rate and length.
I abort decoding after reading the wave header, but takc.exe often stays running, and consuming CPU resources.
Would be nice if 1.1.1 didn't have this problem. Also would be nice to know enough about TAK container format at least to be able to determine
basic audio stream properties without starting a decoder. I saw a RIFF header in the files, is it always present? How do i find out where it starts?
One more request, could the decoder be enhanced to work from pipe to pipe, i.e. "takc.exe - -"? This is required to be able to play/transcode TAK
streams directly from a zip archive or network stream without creating (large) temporary files on disc.
Using .dll library is not an option for me, because 1) it only provides decoding, but i also need an encoder. 2) i need an x64 version, but sdk only contains x86.

By the way, did you ever think about using some existing well-documented and well-implemented container format, like ogg(flac)?
I really don't understand why each and every codec tends to have a new container for it.
Reusing a well-known format would make it a lot easier for developers to support a new codec in their applications.

This was suggested back in the days:
http://www.hydrogenaudio.org/forums/index....st&p=409080

Thank you.


1) Wrong. FLAC is not using their own container. It's using ogg container.
2) Other developers can't use the current TAK container format due to the total absense of documentation and source code for it.
We could have been able to deal with TAK container for 2.5 years now if the decision to use existing format was taken back then.
We'll see if the promised publication of some container-related code would help, and how long will it take for developers to use it.
3) We cannot 'use only the codec and wrap the data into a container of our choice' because all we have right now is a command line
encoder/decoder which obviously cannot support container of our choice, and a decoder library which also cannot support container
of our choice - it doesn't have a frame-oriented API.

There are many ways to make a successfull codec. The easy ones are
1) To be a big corporation which can force people to use this or that
2) To make a good open-source portable codec with decent documentation
Imho, if you want TAK to be successfull, you will need to make it as open and as portable as you can.
Using an open container format and making a small cross-platform frame encoder/decoder sdk might have been a giant step towards this.

Sorry if i sound harsh. Don't mean to offend and i really appreciate your efforts.
That's just one of my bad habits to point out other people's mistakes.
That's much more fun than making my own mistakes

Why are you quoting something 2½ years old? You do realise that this issue has been raised a hundred times before, and you are saying nothing new?

Oh, and read this:

http://flac.sourceforge.net/faq.html#general__native_vs_ogg

Because i wasn't here 2½ years ago

I have just added TAK support to CUETools and came to this thread to share my impressions. Didn't really care if they are old or new I think feedback is a great thing.

Concerning FLAC - i stand corrected.

If I criticise you now for something you did when you were 10 (I'm assuming you're not 10), that's NOT helpful "feedback".


If you know it's a bad habit, but you do it anyway, then you do mean to offend.


That's a weird way to say "sorry I was offensive, irrelevant and then wrong".

C.

I'm mostly concerned about current TAK state, not it's history.
I just wrote what difficulties did i have when i tried to add TAK support to CUETools, and i think that's helpful feedback.
Other developers might face same difficulties trying to do the same.
My sense of humor might be wierd, but i was joking of course when i said
that pointing out other people's mistakes is a bad habit.
Some people find it bad, but i personaly think that's a good one.
I don't want to get personal here, i only wanted to discuss the codec and the SDK.
Don't see how it can be offensive, irrelevant and wrong.
But if noone is interested, that's ok. Not that it's very important to me.

TBeck wrote:



Very good :-) The only project that doesn't bloat more and more with every new version ?



Useless. LGPL can always be switched to GPL (but not vice versa), so "GPL/LGPL" is exactly equal to "LGPL". See also below.




WRONG.

FLAC has both it's own container (from the beginning) and can be used inside the OGG container also (later added). Thus FLAC can be used as audio codec for OGG Theora and Dirac video.

WAVPACK AFAIK has only it's own one, so it can't be used for video now, although a WAVPACK mapping for OGG could be created, if there was interest for such.

VORBIS has no own container and fully depends from OGG (maybe not that good design ...).

TAK has only it's own container and is closed source.



Agree. Creating a TAK mapping into the OGG container and adding it into Xiph's official codec list at the occasion of open sourcing would be a good idea. Of course the license should be XIPH/BSD then, rather than GPL.

Please read, a few posts above yours:


What's the point when Gregory has already acknowledged his error? Was Vorbis even under discussion?

On a side note, WavPack is fully compliant with the Matroska container (MKV/MKA): http://www.matroska.org/technical/specs/codecid/index.html.

I had, sorry.



Also this one.



There is definitely a point, I summarized the (potentially) free audio codecs (including Vorbis), and pointed the usage inside video files.



Or not "compliant" all, just a mapping proposal exists.



The follow up link is broken: http://www.matroska.org/technical/specs/codecid/wavpack.html

To finalize my evil point from above, if TAK gets open sourced and an OGG mapping, it could be maybe supported in Firefox 3.5 or 4.0 (3.1 supports Vorbis + Theora only)

Nope, compliant it is, outdated page and broken link nonetheless.

Preparing the final release

Sorry for the delay. I had much less time to work on TAK than i expected when releasing the beta.

The lack of time also affected my capability to reply to the posts in this thread and to some emails i received.

Now i am preparing the final release.

Please report any bugs or misbehaviour you may have encountered with the beta.

Thomas

TAK V1.1.1 final has been released.