Yalac - Need some testing, (Yet another lossless audio comprossor) Options

[TBeck]

My work on an quick and dirty evaluation release (for some prove of concept) is nearly done.

But there is at least one important bug left, which affects one single file of my test corpus and generates an error in the last 8 samples of the file. Unfortunately i have no idea what the reason is. From my experience bugs that can not be found in 2 or 3 hours tend to be really hard to find.

I would be glad, if one or two interested forum members would like to test this buggy version with some files. Possibly i could find a pattern in the bug, if it manifests with other files too.

It would be optimal, if the testers would have experience in the testing of lossless compressors. Yalac doesn't always reconstruct the original header of the wave file, so a validation of the decompressed audio data isn't just a simple file compare yet. If no other tools are available, the raw audio data would have to be extracted from the wav file for a comparison.

That surely would be some work.

Thomas

[Mark0]

It would be optimal, if the testers would have experience in the testing of lossless compressors. Yalac doesn't always reconstruct the original header of the wave file, so a validation of the decompressed audio data isn't just a simple file compare yet. If no other tools are available, the raw audio data would have to be extracted from the wav file for a comparison.

Maybe this tool I made some times ago to do just that may prove useful after all!
Link: RIFFStrip

Bye!

[TBeck]

It would be optimal, if the testers would have experience in the testing of lossless compressors. Yalac doesn't always reconstruct the original header of the wave file, so a validation of the decompressed audio data isn't just a simple file compare yet. If no other tools are available, the raw audio data would have to be extracted from the wav file for a comparison.

Maybe this tool I made some times ago to do just that may prove useful after all!
Link: RIFFStrip

Bye!

Many Thanks! It's really helpful.

I did try it on my test files and it worked great.

Thomas

[Destroid]

Maybe this tool I made some times ago to do just that may prove useful after all!
Link: RIFFStrip

Bye!

[/quote]

Yes, perfect! Thanks!

Regards,

[Synthetic Soul]

Thanks Mark0. Looks like we are setup to test?

[TBeck]

Thanks Mark0.  Looks like we are setup to test?

Can i send you an archive with the program via email?

Thomas

[Synthetic Soul]

I have PM'ed my email address.

Many thanks for the offer. I hope I can be of help.

[Synthetic Soul]

BTW, to those that are still unbelievers, I have performed a very small test:

CODE

Codec                Comp (%)  Encode (s)  Decode (s)
=====================================================
OptimFROG Default       54.27     220.384     168.732
Monkey's Audio High     54.41      91.846     121.350
YALAC High              54.86     459.560      68.517
YALAC Normal            55.25     105.680      64.090
WavPack -h              55.69      84.262      82.198
WavPack Default         57.44      55.592      66.549
FLAC -8                 57.56     293.061      60.252
FLAC Default            57.91      82.402      68.17

This was performed with just five files. Also, it was performed while I was (supposed to be) working; I have foobar playing, and various other apps taking resources. Decompressed files bit-compare with original waves in foobar. All apps are the latest versions, timings with the command line apps were taken using TimeThis. YALAC is GUI-only at the moment, so this test has relied on the timings reported by the app itself.

So, I wouldn't pay too much attention to the figures (I'm not even going to bother publishing my PC's spec), but this is confirmation that we may well have a serious contender here.

I hope to perform a more diverse and accurate test during this week.

Edit: Added OptimFROG and WavPack -h. Added text about apps and timings.
Edit 2: Added FLAC Default

It looks like YALAC is high compression with fast decoding. With that in mind, does anyone have any suggested encoder settings to test it against? Monkey's Audio Extra High is obviously on my list, and probably Wavpack -x. I really can't be bothered to test Insane-type settings (OptimFrog bestnew; YALAC Insane; Monkey's Audio Insane; WavPack -hx6), as I just don't see these things as useful. I'll leave that to someone else with more patience.

This post has been edited by Synthetic Soul: Apr 4 2006, 15:32

[TBeck]

So, I wouldn't pay too much information to the figures (I'm not even going to bother publishing my PC's spec), but this is confirmation that we may well have a serious contender here.

Many...Many... Many thanks!

This lightens me enormously!

Thomas

[Josef Pohm]

It looks like YALAC is high compression with fast decoding.  With that in mind, does anyone have any suggested encoder settings to test it against? 

Current high compression with fast decoding champion, in my opinion, is, by far and with no doubt, Garf optimized version of MP4ALS reference encoder, with "-a -o32" switch.
Its compression ratio should be in the range of OFR-Normal and APE-High, while decoding speed should be a little less than three times faster than the first one and a little less than twice faster than the second... (by the way it encodes quite fast as well).

I would do it myself but I won't be back home until the weekend. You could also try MP4ALS reference encoder (which shows a slightly better compression ratio), but, in my tests, it decodes around 10% slower than Garf modified version.

And whether YALAC beats MP4ALS or not, I want to say to Thomas that, so far, it surely looks like he did a very good work!

Edit: Then I had doubts... and finally I changed my mind, as you can see here. Sorry for spreading inaccurate informations.

This post has been edited by Josef Pohm: Sep 27 2006, 13:24

[Garf]

You could also try MP4ALS reference encoder (which shows a slightly better compression ratio), but, in my tests, it decodes around 10% slower than Garf modified version.

Hmm, I produced a version with equal or better compression than the reference code, but I'm not sure if I ever uploaded that Without more information about patents or proper embedding in MP4 I didn't have the impression much would happen around MP4 ALS yet.

This post has been edited by Garf: Apr 4 2006, 18:15

[Synthetic Soul]

Current high compression with fast decoding champion, in my opinion, is, by far and with no doubt, Garf optimized version of MP4ALS reference encoder, with "-a -o32" switch.
...
You could also try MP4ALS reference encoder (which shows a slightly better compression ratio), but, in my tests, it decodes around 10% slower than Garf modified version.

Can anyone point me in the right direction?

Thomas, are you waiting for more volunteers, or have you already distributed YALAC to anyone else?

I guess the server's going down soon...

[Shade[ST]]

I'm a volunteer. tristan -dot- dumas -dot- bonnier -at- gmail -period- com

[TBeck]

First test is closed now (means not further participiants).

I will sent the evaluation version to anyone, who has asked for it in this post or via personal mail within the next day. Should be about 5 to 7 testers then, which seems to be enough for this first reality check test.

I'm very interested in the next results.

Many thanks to the testers

Thomas

[TBeck]

And whether YALAC beats MP4ALS or not, I want to say to Thomas that, so far, it surely looks like he did a very good work!

Thanks!

I did a comparison of the asymmetric mode (-7) of the reference implementation of mpeg4 als with yalac on my test corpus. I did vary some parameters.

CODE

-a    Adaptive order = on
-b    BGMC = on
-g5   Block switching level = max
-l    LSBcheck = on
-n    Frame length
-o    Filter order

-7 should translate to -a -b -g5 -l -n20480 -o1023.

Parameter variations tested:

CODE

Mpeg4
 7    = -a -b -g5 -l -n20480 -o1023         (-7)
 A    = -a -b -g5 -l -n20480 -o256          (Predictor num reduced to 256)
 B    = -a -b -g5 -l -n4608  -o256  -r-1    (FrameSize = 4608, ClosedFrame)
 C    = -a    -g5 -l -n4608  -o256  -r-1    (Rice encoding instead of BGMC arithmetic)

Results:

CODE

          yalac       Monkey 3.99               MPEG4 Als
Pred:       256    |  Norm    High    Extra  |  7       A       B       C      |
-------------------+-------------------------+---------------------------------+
Song_02     47.80  |  48.77   48.00   47.28  |  47.52   47.68   47.87   48.44  |
Song_04     32.58  |  34.07   33.58   32.35  |  31.60   31.83   32.16   32.81  |
Song_06     33.24  |  34.22   33.77   33.09  |  32.90   33.06   33.31   33.88  |
Song_08     44.46  |  46.06   44.81   43.59  |  43.72   43.77   43.99   44.56  |
Song_10     55.94  |  56.29   55.95   54.97  |  55.34   55.41   55.59   56.17  |
Song_12     53.26  |  54.17   53.04   51.99  |  52.71   52.72   52.80   53.29  |
Song_14     48.44  |  49.02   48.65   47.76  |  48.95   48.99   49.08   49.64  |
Song_16     73.79  |  73.78   73.70   73.44  |  74.23   74.23   74.76   74.23  |
-------------------+-------------------------+---------------------------------+
            47.34  |  48.32   47.62   46.70  |  46.96   47.05   47.23   47.80  |
-------------------+-------------------------+---------------------------------+
Bach_01     54.36  |  62.55   60.05   55.42  |  52.06   54.03   54.58   55.19  |
Bartok_01   48.13  |  50.52   49.17   47.74  |  47.23   47.37   47.59   48.17  |
Debussy_01  25.96  |  26.82   26.48   26.12  |  25.22   25.29   25.54   26.19  |
Mahler_01   47.62  |  47.89   47.73   47.09  |  46.77   47.24   47.48   47.81  |
Speech_01   30.09  |  32.09   31.81   31.78  |  31.35   31.35   31.39   31.84  |
-------------------+-------------------------+---------------------------------+
            41.28  |  --.--   --.--   --.--  |  40.49   41.02   41.29   41.81  |
-------------------+-------------------------+---------------------------------+

This results are a bit outdated now. Yalac now uses up to 384 predictors and a Frame size of 11025 Samples. i should repeat this test with the corresponding parameter set for Mpeg4. But it shoudn't make a big difference, because very very few files benefit from more than 256 predictors.

Thomas

[TBeck]

Thomas, are you waiting for more volunteers, or have you already distributed YALAC to anyone else?

Oh, i did miss this one! No more volunteers needed. Time to wait for the results and then for an analysis.


Thomas

[TBeck]

What i myself would like to do next:

1) Extend my simple Wave-IO code to allow testing of 24 Bit samples and possibly higher sampling rates (evaluation version is limited to 48 KHz).

2) Provide a switch to disable the use of MMX-Instructions. I'm really interested into the effect of this on other processors than my good old Pentium 3.

I hope to complete this within the next two days and then will sent it to the testers.

No, i don't want to join the 'have a new version every day'-club. But i think, that especially the option to test 24-Bit Samples is needed to judge the general usefulness of yalac.

Thomas

[Shade[ST]]

Sturdiness test :
(Changes are in encoded file, before decoding)

Using a 12000 Hz wav file, changed 3F at position 0xd1600 to 4F

(1) changed 3F at position 0xd1600 to 4F : decompression succeeded, stream syncs to original.
(2) Randomly changed 5 bits : Stream decoder failure

Resulting file for (1) : no message pops up on bitcompare. It seems files must be identical.
(2) generates a 48 kbyte WAV file. Which doesnt play.

[TBeck]

Sturdiness test :
(Changes are in encoded file, before decoding)

Using a 12000 Hz wav file, changed 3F at position 0xd1600 to 4F

(1) changed 3F at position 0xd1600 to 4F : decompression succeeded, stream syncs to original.
(2) Randomly changed 5 bits : Stream decoder failure

Resulting file for (1) : no message pops up on bitcompare.  It seems files must be identical.
(2) generates a 48 kbyte WAV file.  Which doesnt play.

Interesting.

I possibly didn't write, that there actually are no check sums included into the format. It's a totally raw data stream (frames are byte-aligned) without any additional info like checksums or seektables. If they are added (i will do it), i would expect a maximum loss in compression efficiency of about 0.02 percent if i use 16 Bit CRC sums and 32 Bit seektable entries for each frame of 16 bit, 44 KHz stereo data.

The actual presets always use frame sizes of 250 ms. Because all frames are independently encoded, a single error in the data stream would cause a maximum loss of 250 ms of audio data. I didn't evaluate the possibility to reconstruct the affected frame yet.

Thanks for the info

Thomas

[TBeck]

Because i am always talking about my secret test corpus:

The first section contains CD-Ripps of the beginning of the following songs:

CODE

Song_02  Dire Straits "(Forgot the title)", 1:19
Song_04  Bruce Cockburn (Cover) "Red ships take off...", 1:27
Song_06  King Crimson "Lady of the dancing waters", 1:29
Song_08  Peter Gabriel "Mercy Street", 3:10
Song_10  Thin Lizzy "Whisky in the jar",  1:19
Song_12  Tracy Chapman "Mountains o' things",  1:02
Song_14  Bruce Cockburn (Cover) "Silver wheels",  1:20
Song_16  Kunze "Dein ist mein ganzes Herz",  1:12

The second section contains files from http://www.rarewares.org/test_samples/:

CODE

Bach_01    Bachpsichord
Bartok_01  Bartok_strings2
Debussy_01 Debussy
Mahler_01  Mahler
Speech_01  female_speech

I think, i have to clarify in which aspects i found this selection to be representative. I did test many more songs and found, that some of them were most useful for my evaluation purposes, because they showed differences between mine and other encoders or because they were most sensitive to the encoder parameters i wanted to optimize. For exapmle the Song_xx-Files are ordered by the degree they could benefit from an increase of the prediction order: Song_02 uses up to 384 predictors while Song_16 doesn't significanttly benefit from more than 16 predictors.

So these files should cover the whole spectrum of differences in signal characteristics, that affect my encoder.

But that doesn't mean, that they are representative for some typical Audio collection!

I myself forgot this fact. But the first test results in this thread brought it back to my mind, because the performance advantage of yalac was significantly lower as in my tests.

Possibly the files with high predictor orders are over represented in my test corpus.

Thomas

[Shade[ST]]

In response to your comment, does that mean that YAA (sounds german, no? ) will always encode at least 250 ms of audio data? Can it process smaller files? Also, is there any way you might consider making that time frame toggleable? I've tried every parameter available to me, and collected all of the statistical data that I could..

Here is an excel file with my analysies.

http://startrooper.free.fr/ha/YAA.xls

[TBeck]

In response to your comment, does that mean that YAA (sounds german, no? ) will always encode at least 250 ms of audio data?  Can it process smaller files?  Also, is there any way you might consider making that time frame toggleable?  I've tried every parameter available to me, and collected all of the statistical data that I could..

Here is an excel file with my analysies.

http://startrooper.free.fr/ha/YAA.xls

Should YAA be the free acronym, whe were so desperately looking for?

Yalac should be able to compress smaller files, because it allready has to generate one smaller frame, that contains the last samples of the file, that usually are less than the standard frame size.

The frame size can be varied. It's one of the many encoder options, which actually can not be accessed via the simple GUI. Maximum frame size is 250 ms. I don't like higher values, becaus this would lower the speed of the encoder and of random seeking. And my test showed no significant increase in compression efficiency with bigger frames.

The lower bound isn't defined yet (maybe 50 ms). Smaller frame sizes can be useful, if you use low predictor numbers. They can give higher compressing speeds without a significant loss in compression ratio.

Now i will look for an excel viewer...

Many thanks!

Thomas

[Shade[ST]]

If you prefer, I can make an image (png)

In maybe 7-10 hours (when I wake up)

[TBeck]

If you prefer, I can make an image (png)

In maybe 7-10 hours (when I wake up)

Thanks, but i think, that excel is a very common format to store such results. I will need this viewer sooner or later.

Good night

Thomas

[TBeck]

Important note for the testers:

For this test it now seems to be sufficient, if you evaluate the presets Normal and High and maybe Fast. There is abolutely no urgent need to vary the specific Parameter settings! If one should have allreday done it, it would be neverteless interesting.

The first results, that i have received, could be summarized as follows:

1) My test corpus isn't too representative. The differences between High and Insane modes that show up in the testers data are smaller than the results of my test corpus. It generally looks as if the higher predictor orders that are applied by High and Insane usually don't give that much of an advantage.

2) The speed performance heavily depends on the CPU of the testing system! That was surprising for me. Worst one is an AMD Duron with 800 MHz, where the decompressor speed only was half of FLAC. This is surely caused by the very small second level cache of this Duron: 64 KB vs. 256 KB on my Pentium 3.

From the posts in the threads regarding yalac i would think that yalac would be most useable (could fill a gap between existing compressors), if it has the following features:

1) Compression efficiency on par with Monkey's Audio High.

2) Decoding speed on par or better than FLAC:

One further condition, possibly not too obvious:

3) Because i would like to publish the source (for instance to be used within FLAC) sooner or later, it would be advantegous, if the algorithm complexity (in terms of lines of source code) wasn't to high. That would make the adoption by other developers easier.

So this are my plans for a next test:

1) Give the testers access to the encoder options, that are hidden behind the presets.

2) Let them test the possible combinations for compression and speed efficiency.

Then it's my turn to deceide, which options could be removed, to speed up the codec without a significant decrease in compression performance.

After this is done, i can optimize the remaining options for speed.

I possibly would tolerate a loss in compression efficiency of about 0.2 to 0.3 percent, if this would make things really faster.

There is still one feature of the codec disabled, which gives about 0.35 percent better compression without significantly lowering the speed. This could be activated, to compensate the loss caused by the removal of some other slow options.

What's your opinion?

Thomas