My work on an quick and dirty evaluation release (for some prove of concept) is nearly done.

But there is at least one important bug left, which affects one single file of my test corpus and generates an error in the last 8 samples of the file. Unfortunately i have no idea what the reason is. From my experience bugs that can not be found in 2 or 3 hours tend to be really hard to find.

I would be glad, if one or two interested forum members would like to test this buggy version with some files. Possibly i could find a pattern in the bug, if it manifests with other files too.

It would be optimal, if the testers would have experience in the testing of lossless compressors. Yalac doesn't always reconstruct the original header of the wave file, so a validation of the decompressed audio data isn't just a simple file compare yet. If no other tools are available, the raw audio data would have to be extracted from the wav file for a comparison.

That surely would be some work.

Thomas

Maybe this tool I made some times ago to do just that may prove useful after all!
Link: RIFFStrip

Bye!

Many Thanks! It's really helpful.

I did try it on my test files and it worked great.

Thomas

Maybe this tool I made some times ago to do just that may prove useful after all!
Link: RIFFStrip

Bye!

[/quote]

Yes, perfect! Thanks!

Regards,

Thanks Mark0. Looks like we are setup to test?

Can i send you an archive with the program via email?

Thomas

I have PM'ed my email address.

Many thanks for the offer. I hope I can be of help.

BTW, to those that are still unbelievers, I have performed a very small test:


This was performed with just five files. Also, it was performed while I was (supposed to be) working; I have foobar playing, and various other apps taking resources. Decompressed files bit-compare with original waves in foobar. All apps are the latest versions, timings with the command line apps were taken using TimeThis. YALAC is GUI-only at the moment, so this test has relied on the timings reported by the app itself.

So, I wouldn't pay too much attention to the figures (I'm not even going to bother publishing my PC's spec), but this is confirmation that we may well have a serious contender here.

I hope to perform a more diverse and accurate test during this week.

Edit: Added OptimFROG and WavPack -h. Added text about apps and timings.
Edit 2: Added FLAC Default

It looks like YALAC is high compression with fast decoding. With that in mind, does anyone have any suggested encoder settings to test it against? Monkey's Audio Extra High is obviously on my list, and probably Wavpack -x. I really can't be bothered to test Insane-type settings (OptimFrog bestnew; YALAC Insane; Monkey's Audio Insane; WavPack -hx6), as I just don't see these things as useful. I'll leave that to someone else with more patience.

Many...Many... Many thanks!

This lightens me enormously!

Thomas

Current high compression with fast decoding champion, in my opinion, is, by far and with no doubt, Garf optimized version of MP4ALS reference encoder, with "-a -o32" switch.
Its compression ratio should be in the range of OFR-Normal and APE-High, while decoding speed should be a little less than three times faster than the first one and a little less than twice faster than the second... (by the way it encodes quite fast as well).

I would do it myself but I won't be back home until the weekend. You could also try MP4ALS reference encoder (which shows a slightly better compression ratio), but, in my tests, it decodes around 10% slower than Garf modified version.

And whether YALAC beats MP4ALS or not, I want to say to Thomas that, so far, it surely looks like he did a very good work!

Edit: Then I had doubts... and finally I changed my mind, as you can see here. Sorry for spreading inaccurate informations.

Hmm, I produced a version with equal or better compression than the reference code, but I'm not sure if I ever uploaded that Without more information about patents or proper embedding in MP4 I didn't have the impression much would happen around MP4 ALS yet.

Can anyone point me in the right direction?

Thomas, are you waiting for more volunteers, or have you already distributed YALAC to anyone else?

I guess the server's going down soon...

I'm a volunteer. tristan -dot- dumas -dot- bonnier -at- gmail -period- com

First test is closed now (means not further participiants).

I will sent the evaluation version to anyone, who has asked for it in this post or via personal mail within the next day. Should be about 5 to 7 testers then, which seems to be enough for this first reality check test.

I'm very interested in the next results.

Many thanks to the testers

Thomas

Thanks!

I did a comparison of the asymmetric mode (-7) of the reference implementation of mpeg4 als with yalac on my test corpus. I did vary some parameters.




-7 should translate to -a -b -g5 -l -n20480 -o1023.

Parameter variations tested:



Results:




This results are a bit outdated now. Yalac now uses up to 384 predictors and a Frame size of 11025 Samples. i should repeat this test with the corresponding parameter set for Mpeg4. But it shoudn't make a big difference, because very very few files benefit from more than 256 predictors.

Thomas

Oh, i did miss this one! No more volunteers needed. Time to wait for the results and then for an analysis.


Thomas

What i myself would like to do next:

1) Extend my simple Wave-IO code to allow testing of 24 Bit samples and possibly higher sampling rates (evaluation version is limited to 48 KHz).

2) Provide a switch to disable the use of MMX-Instructions. I'm really interested into the effect of this on other processors than my good old Pentium 3.

I hope to complete this within the next two days and then will sent it to the testers.

No, i don't want to join the 'have a new version every day'-club. But i think, that especially the option to test 24-Bit Samples is needed to judge the general usefulness of yalac.

Thomas

Sturdiness test :
(Changes are in encoded file, before decoding)

Using a 12000 Hz wav file, changed 3F at position 0xd1600 to 4F

(1) changed 3F at position 0xd1600 to 4F : decompression succeeded, stream syncs to original.
(2) Randomly changed 5 bits : Stream decoder failure

Resulting file for (1) : no message pops up on bitcompare. It seems files must be identical.
(2) generates a 48 kbyte WAV file. Which doesnt play.

Interesting.

I possibly didn't write, that there actually are no check sums included into the format. It's a totally raw data stream (frames are byte-aligned) without any additional info like checksums or seektables. If they are added (i will do it), i would expect a maximum loss in compression efficiency of about 0.02 percent if i use 16 Bit CRC sums and 32 Bit seektable entries for each frame of 16 bit, 44 KHz stereo data.

The actual presets always use frame sizes of 250 ms. Because all frames are independently encoded, a single error in the data stream would cause a maximum loss of 250 ms of audio data. I didn't evaluate the possibility to reconstruct the affected frame yet.

Thanks for the info

Thomas

Because i am always talking about my secret test corpus:

The first section contains CD-Ripps of the beginning of the following songs:



The second section contains files from http://www.rarewares.org/test_samples/:




I think, i have to clarify in which aspects i found this selection to be representative. I did test many more songs and found, that some of them were most useful for my evaluation purposes, because they showed differences between mine and other encoders or because they were most sensitive to the encoder parameters i wanted to optimize. For exapmle the Song_xx-Files are ordered by the degree they could benefit from an increase of the prediction order: Song_02 uses up to 384 predictors while Song_16 doesn't significanttly benefit from more than 16 predictors.

So these files should cover the whole spectrum of differences in signal characteristics, that affect my encoder.

But that doesn't mean, that they are representative for some typical Audio collection!

I myself forgot this fact. But the first test results in this thread brought it back to my mind, because the performance advantage of yalac was significantly lower as in my tests.

Possibly the files with high predictor orders are over represented in my test corpus.

Thomas

In response to your comment, does that mean that YAA (sounds german, no? ) will always encode at least 250 ms of audio data? Can it process smaller files? Also, is there any way you might consider making that time frame toggleable? I've tried every parameter available to me, and collected all of the statistical data that I could..

Here is an excel file with my analysies.

http://startrooper.free.fr/ha/YAA.xls

Should YAA be the free acronym, whe were so desperately looking for?

Yalac should be able to compress smaller files, because it allready has to generate one smaller frame, that contains the last samples of the file, that usually are less than the standard frame size.

The frame size can be varied. It's one of the many encoder options, which actually can not be accessed via the simple GUI. Maximum frame size is 250 ms. I don't like higher values, becaus this would lower the speed of the encoder and of random seeking. And my test showed no significant increase in compression efficiency with bigger frames.

The lower bound isn't defined yet (maybe 50 ms). Smaller frame sizes can be useful, if you use low predictor numbers. They can give higher compressing speeds without a significant loss in compression ratio.

Now i will look for an excel viewer...

Many thanks!

Thomas

If you prefer, I can make an image (png)

In maybe 7-10 hours (when I wake up)

Thanks, but i think, that excel is a very common format to store such results. I will need this viewer sooner or later.

Good night

Thomas

Important note for the testers:

For this test it now seems to be sufficient, if you evaluate the presets Normal and High and maybe Fast. There is abolutely no urgent need to vary the specific Parameter settings! If one should have allreday done it, it would be neverteless interesting.

The first results, that i have received, could be summarized as follows:

1) My test corpus isn't too representative. The differences between High and Insane modes that show up in the testers data are smaller than the results of my test corpus. It generally looks as if the higher predictor orders that are applied by High and Insane usually don't give that much of an advantage.

2) The speed performance heavily depends on the CPU of the testing system! That was surprising for me. Worst one is an AMD Duron with 800 MHz, where the decompressor speed only was half of FLAC. This is surely caused by the very small second level cache of this Duron: 64 KB vs. 256 KB on my Pentium 3.

From the posts in the threads regarding yalac i would think that yalac would be most useable (could fill a gap between existing compressors), if it has the following features:

1) Compression efficiency on par with Monkey's Audio High.

2) Decoding speed on par or better than FLAC:

One further condition, possibly not too obvious:

3) Because i would like to publish the source (for instance to be used within FLAC) sooner or later, it would be advantegous, if the algorithm complexity (in terms of lines of source code) wasn't to high. That would make the adoption by other developers easier.

So this are my plans for a next test:

1) Give the testers access to the encoder options, that are hidden behind the presets.

2) Let them test the possible combinations for compression and speed efficiency.

Then it's my turn to deceide, which options could be removed, to speed up the codec without a significant decrease in compression performance.

After this is done, i can optimize the remaining options for speed.

I possibly would tolerate a loss in compression efficiency of about 0.2 to 0.3 percent, if this would make things really faster.

There is still one feature of the codec disabled, which gives about 0.35 percent better compression without significantly lowering the speed. This could be activated, to compensate the loss caused by the removal of some other slow options.

What's your opinion?

Thomas

Wow TBeck... you seem to know what you are doing. I don't think the world needs another lossless encoder all too badly, but it looks like you're on the right track to providing a better compression/decompression speed ratio than most other encoders out there; this is probably the most important criteria for me, so I'm certainly intrigued. I look forward to seeing what subsequent versions are capable of.

As for the name, before deciding on anything, you should check out http://www.filext.com/ for the extension that you propose to use.

YAA is safe for now.

Also, perhaps you're not quite there yet, but what do you propose to use for tagging? And how would your codec compare to others in terms of features? Seekable? Streamable? Support for multi-channel audio? Support for widely varying sampling rates? (reference http://wiki.hydrogenaudio.org/index.php?ti...less_comparison for a comparison between other codecs)

I think having many configurable options is key to having a widespread format. Look at lame and vorbis encoders -- of course, they have simple presets (-V XX, or -Q XX), but you can always tune settings even more. In the case of a lossless compressor, of course, this has less prevalence, but nevertheless, I think you should leave all parameters in, while making some only active if a debug switch is toggled.

Additionally, if you need help with documentation, I am fluent in french and english, and can read pascal and C++.

I might get you other samples tested soon. I'll keep you posted,
Tristan.
----

From the software readme :

As of yet, supported Formats: Only Wave, 16 Bits per Sample, Stereo, up to 48 KHz.
This limitation is due to my unfinished file-io-library. The compresor itself supports 8 - 28 Bit, 8 - 192 KHz.
---
Another point, also : I have a few ideas about codec implementation, that I'm not sure how you use.
Do you check for recurring patterns in the audio? Do you compare channels and use M/S encoding (One of my samples was mono, but only compressed to 43%..)
I'm very interested in knowing how the codec works.

I am not totally sure, what i will do next (after the tests). But my actual preference would be:

1) Make a simple public release of the codec. It will not contain any sophisticated features besides (fast) seeking, error checking and possibly tagging. I think this would be the minimum features to make it a bit usable. Support for 8, 16, 24 bit and sampling rates up to 192 KHz is allreday implemented into the codec but not in my file io library.

2) Look for a good way to publish a paper about my codec design. I would like to get some reputation...

3) Optimize the public release and fix the bugs the users will report.

4) When everything seems quite stable (and i possibly had enough fun...), release the source code.

But things can change. My first post about my compressor (on April 1) was absolutely spontaneous. So i was in no way prepared.

Thomas


Have some trouble with the forum software. Replies to other posts are beeing added to this one. Will better wait some time.

A little testing done by gURuBoOleZZ on classical music (AMD Duron 800) :



Original post (in French) http://forum.hardware.fr/forum2.php?config...nojs=0#t1058206

I promised a new version for the testers within 1 or 2 days. May take a bit longer.

Done:

1) Support for 24 Bit and samplingrates up to 96 KHz.

2) Encoder options dialog now gives access to far more options that were hidden behind the parameter determination mode settings.

To do:

3) Explaination of the new settings. I don't want "dumb" testers, who don't know what they are doing. Wouldn't be fair. Given my limited english skills, this will take some time.

4) Enabling of the protocol function, which writes internal encoder diagnostics and statistics to a text file.

Thomas

Where i can download?

you can't yet.. i'm fairly sure it'll come soon (within weeks), though

anyway, Aquamarine makes for awkward abbreviations (AAC, anyone? ), but you could always call it something horribly presumptuous, like (for instance) Mozart's Choice? (seeing how it's his birthyear )

Evaluation Release 0.02 for the testers

New features of this second and probably last release for this test:

1) Support for 24 Bit and samplingrates up to 96 KHz.

2) Encoder options dialog now gives access to far more options that were hidden behind the parameter determination mode settings.

3) Switch to disable the use of MMX-Optimizations.

4) Switch to enable the protocol function, which writes internal encoder diagnostics and statistics to a text file.

5) Explaination of the new settings.

What i would like the testers to do

1) Repeat the test with your files with Normal and High-Presets and with the protocol function enabled. Send me the protocol files. You will have to save or rename the protocol file between runs, because each run overwrites the previous results.

Updated!!! Please read post "What i really would like the testers to do..."

2) Possibly check some 24 Bit files. Not to urgent.

There is no need to systematically test the new options! If you would like to play with them, feel free to do so. But i don't think, that these results should be listed in the final public result of this test, because there would be too many possible combinations and therefore huge unreadable result tables.

Main reason for the unlocking of the individual options are the strange results of one tester. I will ask him via EMail, to check some specific settings.

Some words about speed

I didn't do any optimization of the encoder or decoder, although i was very tempted to do. Nevertheless there will be changes of the speed results.

This is caused by some limitation of my compiler. For optimum performance, some parts of the code have to be aligned to memory adresses, which are an integer multiple of 16. But my compiler doesn't do this. So every time i change something –maybe the GUI- code is beeing moved around and speed varies within a range of about 5 – 10 %. This is totally random.

Close the test

After the new test runs i have asked for, we should bring this first reality check test to an end. We possibly should discuss the best way to publish the results here:

1) Every tester posts his results in his individual format.
2) I post them in some standard format and ask the testers for confirmation.

I think 1) would be ok.

My tester list

I will send the new version to any tester on my list within the next hours:

benski
boombaard
Destroid
guruboolez
Josef Pohm
Shade[ST]
Skymmer
Synthetic Soul

Please notify me, if i forgot one. There is a bit confusion between forum and email identies on my side. Sorry...

Again, many many thanks to the testers!

Thomas

Important note for the testers!

The encoder option 'Apply window' definitely hurts on some files! It is beeing automatically activated with the presets High and Insane. This regards to V0.01 and V0.02.

One tester reported files which compressed over 3 percent worse with this option activated! I could verify this on some test file, he sent me.

V0.02 allows disabling of this option. You should possibly disable it, especially when High or Insane are giving you worse results than Estimate.

I'm not sure, what goes wrong. Need some time for an analysis.

But it shows, how useful this evaluation test will be for the optimization of my encoder.

Thomas

Yeah, really useful!

Possibly some good news.

Here some results for the critical Test file:



But now something really interesting. I allready wrote about one inactive encoder option, which on my test corpus gives about 0.35 better compression. Let's see, how this baby performs on this critical file. General setting is NORMAL + Optimize Quantization.



Looks promising.

Hard to predict, on how many files this will help, but this definitely should be evaluated.


Thomas

What i really would like the testers to do...

Sorry for another change!

But i didn't have enough time, to analyze enough incoming data early enough. This will be the last change.

It seems, as if the activation of the frame partition validator, which is part of HIGH and INSANE modes, not only gives a too small increase in compression efficiency in ralation to it's speed penality, but on average even hurts!

Hence i would like to redifine the Presets as follows:



If you would like to do some more tests, those should be the most interesting settings.

Again sorry. I have to do to many things simultaneously. And i didn't expect so many results so different to my test corpus.

Thomas

Stop the confusion!

You possibly can see from my last posts, that i am getting a bit confused from the results of the testers, that are sometimes so different from my test corpus.

I think, i should perform some deeper analysis of the surprising findings, before i ask the testers for specific evaluations. Otherwise there would be a chance, that the testers waste their time.

What i would like to do now:

1) Close the official test within the next days.

2) Open a new thread 1, where testers should post comparisons of the presets of yalac with other compressors. This possibly would be the most interesting for other forum readers.

3) Open a new thread 2, where testers can post about the effects of special encoder parameter variations and crazy findings. This would be useful for my next encoder improvements. And this specific tests can go on.

What do you think?

One further question: Is it ok, to use this forum for discussion with the testers?

Thomas

Would the preset values be the standard preset buttons of 0.02, or the presets as you list above (i.e. HIGH = NORMAL + Optimize Quantization + 256 Predictors)?

I am nearing the end of a 50 track comparison between Yalac and a few other encoders, and am just beginning to test Yalac. Can I continue as per your instructions above (i.e.: adapted values for Extra and Insane)? I'm keen to see my test completed, and was hoping to undertake the Yalac processing tonight.

I would use the original (standard) presets, beacuse probably most testing done before my confusing posts might have used them.

Thomas

New ULTRA FAST preset

Would you think, it could be useful to test some new ULTRA FAST mode? My latest version (0.03, not released) allows the use of smaller Frames, which makes the FAST preset about 50 % faster at encoding without significantly affecting compression efficiency.

Maybe it's not really useful, because encoder speed could allreday be limited by disk IO at the FAST preset.

Not urgent. If one should like to test it, it could be added to the results later.

Thomas

This could be interesting for streaming purposes. If you add it, I'll test it. (if you build it, they will come!)

If they come, i am happy

Ok, i will try to complete it until tomorrow.

But i have to correct me. The speed advantage seems to be CPU dependend. Another machine gave me only 30 %.

But there is potential. The fast presets are suffering from the fact, that there still is many unoptimized and debug code left, which doesn't contribute much to the calculation time of the modes with high predictor orders, but slows the fast modes down.

Thomas

I have published the results from my encoder comparison on the Comparison Thread.

I now hope to perform some more varied tests with 24bit files; mono files; and bespoke encoder settings.

I may wait for the command line version before I undertake anything else too strenuous though. I guess this depends whether I can free up some time (which is in short supply at the moment), or how long it will be before a command line version can be released.

NB: I also hope to document my test system, as I spent a reasonable while creating some batch and Windows Script files to automate the process of running encoding and decoding tests on a test corpus and entering those results into a relational database (to a degree that it just takes a couple of double-clicks). I think this system could be of use to some other people, but I don't really know how others approach this; I may have simply re-invented the wheel, or over-complicated things!

Thanks for the results!

Yes, please wait for the command line version. I know, that testing is quite uncomfortable with the odd GUI.

And i myself tend to forget, that i called this test "reality check". It has not necessary to be too detailed yet.

But you may get some other impression because i can't stop myself posting my new ideas, findings and optimizations every day! I'm a bit out of control... Working on that now...

It think that your test setup could be helpful for others. No, i won't tell you about my primitive way...

Do you have a tool, that automatically compares the files sizes to get the compression ratio?

Thomas

Wow! Really great and useful presentation! Many many thanks again! Must be some very useful testing tools.

Thomas

No problem. I may well continue testing before you release a command line version. I'm not sure I can stop myself.

It's hardly surprising. It seems as though you have something here that puts you in the big league of lossless codecs. I'm sure you must have put in a lot of work to get to this point.

The ASP does that at the moment. Just the raw data (durations in seconds and file sizes in bytes) is recorded in the database. I have a table with a record for each source file that records the source file size (and duration), and a link table that links a file to an encoder setting, which records the file size of the encoded file (and the en-/decode times).

I used a simple batch file to list the filesize of the sources files, and the batch file that encodes the files records the times and also the generated file size. The decode batch file also records the times. A Windows Script file then scrapes the encode/decode times and file sizes from these text files and inserts them into the database (which is MS Access for ease - I also thought that I may let people download it).

I really am glad to be of help.

Are there any other views that you think would be useful? It would be very easy for me to knock up another page to display the info in another way, if you think that some more useful stats could be detailed.

Tomorrow i will sent a new version with the new FASTEST preset to the testers. That could be interesting.

What could be helpful for analysis would be a test run of the wole file set with the protocol function enabled for NORMAL and HIGH. You have to move or rename the protocol file between the runs, because it's always been overwritten.

I would suspect, that the files use low predictor numbers. That's usually the case, if HIGH is only so little bettter than NORMAL. And really strange that HIGH is faster on decoding than FAST!

Yes, my test corpus isn't representative. I have just downloaded the whole test file set from rarewares.org, to get more variety.

Thomas

Can you make the protocol (log) file append outputs? That would make it easier. You could even add a seperator each time you write to it, like a 30 char long line of '.' or '*' or '-', to make reading back entries faster.

Good idea!

Done:

- Option to append protocol
- Store encoder options into protocol
- Store results into protocol
- Justify result columns

Probably a dumb question but does this support .cue's? Can you email me a copy to hand.of.omega, it's a @gmail.com address.