Content

In this thread the testers should post comparisons of the preset modes of Yalac (Working name for "Yet another lossless audio compressor") with other lossless compressors.

The results for specific variations of individual encoder options should go into the thread " Yalac – Evaluation and optimization".

Guidelines

Please post the exact version number of the compressors. Compression ratio should be specified in percent of the uncompressed file size. Speed figures should be specified as multiple of real time (duration of the test files). A specification of your test system, especially the CPU, would be helpful.

Open end: Feel free to add more results later.

What happened

Bad timing of my introduction (April 1.) forced an early publication of an evaluation release of Yalac, to prove, that it really works. This are results of 8 forum members, who where so kind to test the experimental release for me.

Many thanks!

Goals

Yalac should finally achieve compression ratios on par with Monkey's Audio High. Decoding speed should be at least two times higher than Monkey and never be significantly lower than with FLAC (possibly Yalac will later be integrated into FLAC).

My next steps

The first results i have received from the testers show me some weaknesses of the encoder. That's a good thing, because that means, that there definitely is a chance to increase the compression efficiency! Same is true for the speed; especially the decoder is not fully optimized yet. But it will take some time, before i will come up with an optimized release.

Links to 24 bit files i have used

I think, they are hard to find.



Or make a better Google search. foosion shows you how: http://www.hydrogenaudio.org/forums/index....ndpost&p=381702

My results can be found at the following address:

http://synthetic-soul.co.uk/comparison/lossless/

My results seem to bear that out completely:



A little more info in the main testing thread.

I had issues with the inability for Yalac to accept mono files and other files with RIFF header issues. I am most interested in default/normal settings.



edit: System = A64 3000+ 512MB Win2Ksp4

That is quite impressive how you have your site set up soul, so very easy to read and access the info within.

Comparison now includes Yalac Fastest.

• Summary of encoders
• "Yalac Fastest" individual file stats

Thank you. It made sense to me to store the data in a relational database rather than a spreadsheet or other flat file.

That said, I do mean to add the abilty to download any view in CSV format, for viewing in Excel. Maybe tomorrow.

These are benchmarks with Yalac 0.03 with two albums (EAC CDImage file). The first album is rock recording that was remastered and represents modern, heavily compressed albums. The second is an un-remasted pop/rock album.

Notice that the behavior of each codec on different types albums affect both compression ratio and processing time:




Yalac already has great performance and ratio. It would be interesting to see if the encoding time can be further optimized

edit: added LA scores, not interesting since it usually gets the best ratio, is extremely slow and it crashed
edit2: added TTA scores

Many thanks!

I will work on the speed of the HIGH preset later. I would like it to be only two times slower than NORMAL.

Thomas

Test #3 - LP recorded to cassette, includes vinyl pops, tape hiss, distortion from the A/D conversion process, etc.:


Test #4 - spoken word, dialogue, mono, recorded directly to digital medium - no transfer

edit: added TTA scores

Destroid's results prove that YALAC is the best asymetric lossless audio codec to date.

Congrats, Thomas!

Keep up the hard work. And tell us if your house is still in one piece.

The most obvious thing I'm overlooking is testing with MPEG-4 ALS, which has many confusing parameters. I do not know the equivilent command line for fast/normal/high settings. And I'm not sure if either of the two existing MPEG-4 ALS binaries is representative of the codec at this time.

Otherwise, yes. Yalac delivers MA compression ratio with FLAC decompression speed.

Garf posted some suggestions for "presets" here, and there are some other suggestions in that thread. Also, in the main Yalac thread, Joseph Pohm suggested a setting which I used here, and Thomas responded with some settings that he had used here.

When I tested there were three binaries: the reference software, the bug-fixed version, and Garf's optimised version. On checking the MP4ALS page now it seems a newer version will be released in the next few days.

As you say, whether testing against a reference encoder is relevant at all is another matter. I didn't put too much emphasis on MP4ALS as I am more interested in testing Yalac against current popular codecs.

I'm still waiting for something to beat LA's compression ratio...I guess it would take a pretty substantial breakthrough in lossless compression before that happens.

This is exactly what I approximated. I will look into MP4ALS benchmarks from these threads. It appears that MP4ALS is better at multichannel. Not all lossless encoders are outfitted for such applications.

I agree. And that's one reason, why i was very interested into the results of your test of mono files. They seem to comfirm my own findings: The difference between YALAC and other better performing compressors is smaller with mono than with stereo files. My channel decorrelation obviously needs some improvement.

Many thanks for your tests!

Thomas

I'm really impressed, the fast compression ratio is almost as good as MA on normal! If the encoding speed could be improved a little on high, then YALAC would even be more amazing.

Thanks for testing it Destroid, and thanks for working on this TBeck, this format might just rock the lossless world

High will be improved! One simple way to achieve this would be the deactivation of one encoder options, which is most responsible for the speed decrease from Normal to high, but usually gives only a small improvement of compression ratio.

Another possibility would be the optimization of the slow encoder option. I allready have one prototype, which performs about 70 percent better speedwise.

Thomas

Wow.

My first observations in monural audio is your codec is very competitive. And for your information, I was running binary comparisons of the output from Yalac to the original and the source file, and there were no differences.

Very well done.

Hello, its my 1st post here.
Have you guys know this audio lossless software? (http://www.true-audio.com/)
it compresses very good too, better than flac and also has a winamp plugin.
it would be interesting to test it and compare it with Yalac in these listings.

Following my comparison using 16bit 44.1KHz stereo tracks I had started preparing for a comparison using the same tracks converted to mono, and also 48KHz and 24 bit.

Is it worth me continuing with this?

I am also wary that you, Thomas, are currently in the process of changing things around a fair bit. I wonder whether I might do better to hold off until you release these changes?

That said, I am, of course, happy to help if you would like more variety in your results. I just don't want to be wasting my time really, if you feel that you already have enough data for the current run.

Of course, the developer Alexander Djourik (ald) is a member of Hydrogen Audio.

TTA was considered in previous tests by guruboolez (http://guruboolez.free.fr/lossless/), speek (http://members.home.nl/w.speek/comparison.htm) and Hans Heijden (http://web.inter.nl.net/users/hvdh/lossless/lossless.htm)

i think most stereo decorrelation is in the predictors. if you do something
simple as ms-decorrelation + predictors the results are always suboptimal.

Very nice!

But good that you have asked:

1) More mono tests aren't necessary for my purposes. It's good that one tester has confirmed my own findings.

2) 48 Khz vs. 44 Khz shouldn't change much.

3) If i convert 16 to 24 bit files with Cooledit, it only does shift the original lower bits 8 bits up and fills the free space (the new lower 8 bits) with 0. If you try such files with YALAC, you will only find, that YALAC doesn't look for this special case (an option sometimes called 'Look for wasted bits') and will perform very bad compared to other compressors. The implementation of such a check is easy but not done yet.

Real 24 bit files would be far more interesting.

I surely will ask for more specific tests, if there are improvements, that are worth the effort.

This should not stop testing! Other files can always give surprising results, which give me hints for further improvements.

One important note:

I tend to write too much about my actual improvements. But they are only prototypes! It will take a considerable amount of time to optimize and debug them! There will be no considerably improved releases within the next weeks! Speedups are a different matter, they may come fast.

I tend to forget, how long it did take, to built the actual YALAC.

Thomas

is there a reason not to test it along with Yalac?

Not especially. Is there a compelling reason to include it though?

Personally I chose to compare Yalac with the popular codecs, at relevant settings.

It doesn't seem that TTA offers anything spectacular to warrant it being included. It appears to be no real competition in compression or decoding speed.

I don't mind adding a run to my comparison, it would be little trouble.

TBeck,

Your compressor is actually a BOMB. Right now, it offers the best possible compromise between compression ratio and decoding speed. Thank you for being so awesome.

Out of interest on my part TTA is now included when specifying All=1

http://synthetic-soul.co.uk/comparison/los...index.asp?All=1

Is there any way you can add some columns?

.. and

Yes, that would be possible.

Another member previously suggested that I include some rating based on compression and speed, in an attempt to rate an en-/decoder further. I know Joseph Pohm uses such a calculation (I've seen his impressive comparison results), but I'm not sure of the algorithm.

I'm a little limited for width on the table. I would have to drop a column or two to accommodate more data.

I'll try to take a look tomorrow though.

Is the second calculation standard, i.e.: is that a well-known method to obtain a rating for an encoder?

Edit: I may actually include TTA in the core results, as it does hold one record; that of fastest compressor.

That second method is evaluating the compresser's algorithmic efficiency, more or less.

The first one looks good, but I can't see how the log helps at all in the second one. If the encoder was in fact logarithmic, it should be x*log(compression rate) / compression ratio, where x is a coefficient. 100 would work for x. If you would like a more detailed explanation as to why compression ratio shouldn't be in the log, I could explain.

Therefore, I would suggest:

Good idea. Now make it

On second thought, since none of the codecs work logarithmically but rather asymptotically, then it wouldn't work. Sorry.

Well, OK, if anyone can suggest an algorithm that would provide a useful rating then I will be happy to add it.

You could always use my "Download to CSV" feature to test using my data in Excel...

I don't have much time at the moment (I've quit my job and need to learn PHP/Postgre before I start my new one) so I need to be certain that any time I do allocate is being spent wisely.

Current Progress

you may call it Tom's diary...

For the time beeing i have given up the idea to look for bigger (> 0.3 percent) increases of the compression efficiency of the higher presets. There are promising new methods, but they have to be evaluated which will take a long time (maybe months).

For now it seems a better approach to optimize the existing methods:

- Squeeze the last bits out of the algorithms.
- Make them faster.
- Clean up the source code.
- Constitute better presets that are well balanced for speed-compression ratio.

First results:

Preset FASTEST is gone. The new FAST preset achieves the same speed on my system and does this with only a small decrease in compression efficiency (maximum 25 percent of the difference between old FAST and FASTEST).

Next goals:

Make preset HIGH faster.

I would be glad, if one or two of the testers, who allready have posted results, would check the new FAST-Preset against the old one.

A new version for all testers will be released, when more presets are tuned or i am unsure about the general effect (my test corpus isn't representative) of specific optimizations. The current version may not be too stable and besides FAST anything might change soon.

Thomas

Moderation: Removed unnecessary quote

This may sound like a stupid question, but where would I download it? I would be very interested to test some stuff on it...

No stupid question!

But it's a closed test now, because there are enough testers to help me with my current optimizations. If one of the testers should leave, you were welcome to participate. I put your name on my list.

Thanks for your interest

Thomas

At last, the first set of monural instrument tracks Notice how this type of material favors all lossless encoders.

It's safe to exclude LA for obvious reasons, but also because of the binary's odd behavior in my script.

TTA was benchmarked (and TTA was added to my previous tests in this thread). TTA is comparable to MAC fast (-c1000).

Current Progress (V0.05)

In my last post i asked for testers for a partial update with an optimized FAST preset. But the general optimization goes faster than expected, hence the next full release for any tester will come soon and an intermediate release wouldn't make much sense.

Done:

- Preset FASTEST is gone. The new FAST preset achieves the same speed on my system and does this with only a small decrease in compression efficiency (maximum 25 percent of the difference between old FAST and FASTEST).

- Preset HIGH is now 40 percent faster on my system. INSANE should benefit even more from the optimizations, but you know, that it isn't really useful.

- Removed 'Apply Window' from preset HIGH, because it sometimes hurts enormously. Because of some other slight optimizations HIGH should not perform worse than before.

To do (for V.0.05):

- Speed optimization of Frame Partition Calculator resolution 256, which is much slower than 128.

- Look for more speed optimizations.


If someone should remember some older post: My home is still intact. They digged for world war II bombs but obviously only found some rotten metal...


Thomas

Cool, thanks for the update Thomas.

I thought I should reply to let you know that I, and I'm sure many other members, do find these updates of interest.

Oh, and I'm glad that your home is still in one piece.

V0.05 is done

Changes:

- Preset FASTEST is gone. The new FAST preset achieves the same speed on my system and does this with only a small decrease in compression efficiency (maximum 25 percent of the difference between old FAST and FASTEST).
- Preset HIGH is 40 percent faster on my system.
- Preset HIGH activates the option "Bitcoder / Optimize choice".
- Preset INSANE uses frame partition resolution 256 and the new encoder option "BitCoder / Optimize Partition".
- Modification of the window function used by "Apply Window". Might help with some problem files...
- Selection of a preset doesn't affect the debug options in the encoder options dialog anymore.
- Select the new debug option "No output" to encode without generating files.
- Redesign of the dialogue options for frame partitioning.
- Error in BitCoder fixed, which sometimes generated errors in the file stream.
- Working data of the decoder was not properly aligned. The fix can improve decoding performance on some systems.
- Clean up of source code. Could give new errors...
- File format changed!

Not done:

- Speed optimization of Frame Partition Calculator resolution 256, which is much slower than 128. Unfortunately my new algorithm didn't work well.

Release:

I hope to send the new version to the following testers within the next hours:

Destroid
guruboolez
Josef Pohm
Shade[ST]
Synthetic Soul

Only reason for this selection: I haven't heard from the other testers within the last 10 days and i don't want to fill their mail box with new versions they possibly currently don't need.

Any of them can sent me an email anytime, and i will sent the current version.

What could be tested:

The performance of the new implementations of the presets FAST, HIGH and INSANE might have changed.

Apply window should hurt less on the rare problem files.

Plans for V 0.06:

- Implementation of a simple command line version.
- Possibly some further assembler optimization to speed up especially preset HIGH. All optimizations in V0.05 were algorithmic.
- More clean ups of my source code. Not really interesting for the tests.


Thomas

I have a question, related to the fact that I haven't post any result these last days. Is it possible, or not too hard or too long, to get a CLI encoder?
The reason I'm asking for that is simple:

My collection of (classical music) files is losslessly ~4GB; once uncompressed to work with YALAC, it's 10 GB. And once encoded with YALAC, I have to deal with 14 GB (10 GB of wav file and 4 of .yaa ones). I'm in vacation, with my laptop, but I can't get such free space on my HDD. Even my desktop computer is currently short in free space.
With a CLI encoder testing would be easier (direct lossless -> yalac transcoding).

N.B. the slowness of my laptop harddrive, and the level of fragmentation, is making speed comparison absolutely unreliable at the moment.

N.B. a "test" tool (i.e. decoding the stream but without file writing, thus with limited disk access) should also significantly increase the speed measurement accuracy.


EDIT:
just noticed that:


Cool!

It's good to see that you're progressing rapidly, TBeck. It's very exciting to see progress on this. And I'm really glad that your house didn't get blown up too!

Keep it up!

By the way, I see you have some goals for your 0.06 release. Maybe you're not yet prepared to answer this, but what all do you want to have done before releasing a public, stable version of your codec? (One that can be addid into the HA Lossless wiki)

Yes!

I am for this as well. Why? Because my script is dumb-smart and doesn't need to be monitored. Any testers that are interested cam PM me. It's main proficiencies are that it works on only files that start with an underscore, outputs files with double-underscore and deletes the output files starting with double-underscore. It outputs the time/filesize to screen with simple commands which can be logged to file with the redirect option. And now it works with LA using a work-around

myea, sorry.. but between increased workload at uni, and seeing the depressingly professionally done results of Synthetic (), and knowing guru is also helping testing (knowing he also tests classical music specifically), it's a bit hard to figure out what i might still add to the mix that isn't represented already ;(

still, the CLI in the next version might make it a bit easier for me to just batch-test stuff to see if i can find odd behavior in random tracks, so who knows )

The speed up in high mode is there.


This time I threw at Yalac the most difficult CD I own (AlbumRG scan = -12.24dB) to see if any compression changes were present with the increased encoding speed.


edit: clarification - RG scan, changes were not applied
edit2: other compressors' scores added

Many thanks for your immediate testing! I am always very curious after a release...

Good to see the overall performance at about the levels i would have expected. Why old Fast is faster than new Fast is a tough question. Maybe the bigger frame size of old fast is better for your system. Possibly disk IO is faster when the data is beeing written in bigger chunks. Or it's a strange issue with the CPU cache.

The public release should add at least those features:

- Integrity checking of individual frames and error resistance: Possibility to skip damaged frames.
- Seektables for fast seeking.
- Inclusion of additional RIFF-info of the source wave file to reconstruct not only the original audio data but the whole source data.
- Some tagging support.

I will think about the details later.

Hello Thomas,

Do you think you could slightly interrupt your optimizations to make a command line mode? I don't think this should be programmatically complicated, but it would be very useful for batch series of tests...

Thanks,
Tristan.

Thanks for asking so careful...

Ok, i will try a minimalistic command line version:

- Specification of encode/decode.
- Specification of a single file or any files (*) of an directory (how should this be specified?).
- Specification of a preset.
- Switch protocol on/off.

Would this be sufficient for the tests?

I wanted to add more options, e.g. the specification of any encoder option (far more than accessible by the dialog) within external script files. But i should make the testing easier as soon as possible.

That should be fine.. if no filename is listed, maybe you could make it automatically check for a --all switch or something like that.

Most of us will probably be using Synthetic Soul's batch file to do our testing (because, for sure, he WILL release one.. He likes to make batch files so much!), and that will be per-file, anyways.


Here is (maybe) the text you could put in the docs.


The extra c on yalacc is for command-line (that way, gui-inclined people can still use the gui, if they want to)
Also, I suggest using "log" instead of "protocol" -- it's more relevant to the english language.