Help - Search - Members - Calendar
Full Version: Yalac - Comparisons
Hydrogenaudio Forums > Lossless Audio Compression > Lossless / Other Codecs
Pages: 1, 2, 3, 4, 5, 6, 7, 8
TBeck
May 17 2006, 20:52
QUOTE
' date='May 17 2006, 06:58' post='393177']
I think people looking at this codec are looking at speed also.
...
As such, I would make both modifications. Make fast faster by 15%, and make a (new) fastest preset, faster than what we currently have.

QUOTE (Destroid @ May 17 2006, 21:21) *
I agree the speed aspect is a Yalac specialty.

Yes, without the speed it would be quite useless. Then i would prefer Monkey or OptimFrog for maximum compression.

Hence speed will be my highest priority. I will not implement compression ratio improvements that significantly reduce decoding speed. It's a different matter at the encoding side.

QUOTE
So, in regards to speed:

- Machines like mine using ATA100 I usually max-out at about 82x decoding speed for 16bit 44KHz stereo
- The fastest encoding speed I clocked was about 98x with my outdated codec
- Yalac decodes from all modes at about the maximum ATA performance
- any speed enhancements for YALAC Fast(est) encoding should take into account the limitations of average HDD performance, otherwise the performance boost will not be noticed


Yes, disk-IO is the limit. On my good old pentium III-866 i can achieve higher decoding speeds than you with your fast machines, if i turn off the decoder output (i should give public access to this option for evaluation purposes...).

QUOTE
Given that, I'd love to see the speed increase or a new fastest preset.

For your information, Thomas, the archival encoder I use achieved a speed of 97.83x with a ratio 47.94%, while Yalac 0.06 Fast clocked at 68.11x with an impressive 46.14% ratio with "free" super-fast decompression speed. Good work!

Thanks to you and thanks to ShadeST for your work and the encouragement!

I am not sure, if i can make Yalac that fast. Main reason seems to be, that yalac has to use bigger data blocks than symmetrical encoders for the disk io. Possibly things will change if i implement asynchronous file io, which unfortunately is not beeing supported by my Windows 98...

And to achieve the maximum encoding speed for FASTEST, i would have to build a special variant of my encoder. Currently FASTEST would do much work unneccesarily twice. My encoder has not been designed for those ultra fast modes.

Thomas
foosion
May 18 2006, 00:28
QUOTE (TBeck @ May 17 2006, 21:52) *
Possibly things will change if i implement asynchronous file io, which unfortunately is not beeing supported by my Windows 98...
Asynchronous I/O would be harder to integrate into a future (cross-platform) library which I hope you plan to create. smile.gif Such a library should let the caller handle I/O operations instead of directly trying to open a file itself. In the simplest case, the caller would feed chunks of data to the library to encode or decode data linearly. Obviously, things get a little more complicated, if you want to support playback in audio players where it is desirable to be able to seek.
I am no expert regarding audio codec libraries (meaning I haven't used many), but I think Monkey's Audio has a nice library interface.
TBeck
May 18 2006, 00:54
QUOTE (foosion @ May 18 2006, 01:28) *
Asynchronous I/O would be harder to integrate into a future (cross-platform) library which I hope you plan to create. smile.gif Such a library should let the caller handle I/O operations instead of directly trying to open a file itself. In the simplest case, the caller would feed chunks of data to the library to encode or decode data linearly.

Good point. But asynchronous io would only be an option. I may define an abstract interface for file io, which has to be implemented by the user and can use asynchronus io if available.
TBeck
May 18 2006, 14:28
Current Progress (V0.07)

Done

- New option for the channel decorrelation to detect bigger lags between channels. Without this option files with lags outside of the search range of the standard decorrelation algorithm can not use channel decorrelation. It's usually some "All or nothing" case: Either it helps compression or not. Don't expect too much, on my test corpus it only helps about 1 of 20 files. And it slows encoding down.

- Evaluation of other possibilities to improve the channel decorrelation. Most of them didn't work well. Some of them look promising but have to be further evaluated (the file format is perepared now for their later use). And some would help but considerably slow down both encoding and decoding. That's not acceptable.

- New preset FASTEST, which is about 50 percent faster than FAST on my system and compresses about 0.9 percent worse than FAST on my test corpus.

To do (for V0.07)

- Clean up of my new channel decorrelation code.

- Speed optimization of the new channel lag detection. Possibly i will wait for test results of the new algorithm, before i optimize it. If it doesn't help compression a bit, its optimization has no priority.

- Implementation of an extra path within my codec to avoid unnecessary processing when using preset FASTEST. May give another 25 % speed up.

- Possibly new options for the variation of some parameters of the disk io to tune the performance on individual systems.
Synthetic Soul
May 19 2006, 10:24
Firstly, thanks for the update Thomas. As I said before, I, and I'm sure the other testers, find your reports very interesting. I can't believe that you are squeezing even more speed out, let alone a 50% increase!

Now, the reason for my post:

QUOTE (TBeck @ May 17 2006, 20:52) *
QUOTE (Destroid @ May 17 2006, 21:21) *
So, in regards to speed:

- Machines like mine using ATA100 I usually max-out at about 82x decoding speed for 16bit 44KHz stereo
- The fastest encoding speed I clocked was about 98x with my outdated codec
- Yalac decodes from all modes at about the maximum ATA performance
- any speed enhancements for YALAC Fast(est) encoding should take into account the limitations of average HDD performance, otherwise the performance boost will not be noticed
Yes, disk-IO is the limit. On my good old pentium III-866 i can achieve higher decoding speeds than you with your fast machines, if i turn off the decoder output (i should give public access to this option for evaluation purposes...).
Joseph Pohm and I have been discussing this issue for the past three days.

Joseph has very eloquently highlighted some anomolies in my results, due to IO limitations. He has some superb charts which articulate the issues I am encountering with speeds over 60x or so. My max is around 80x, while his is 120x.

We are currently looking at Timer as a replacement for TimeThis, which I have used previously. Joseph's initial tests suggest that Timer can accurately report CPU time only (what it calls "Process Time", as opposed to its "Global Time", which is the figure that TimeThis reports). This would be very useful for me to record times unaffected by IO. In fact, I could scrape both CPU (Processed) and CPU+IO (Global) times from the report, for a comparison...

While it is likely that we will pursue this further, we thought that others may find this useful information now. Although no-one else is currently using TimeThis, the times reported by Yalac itself suffer the same fate, and will therefore be affected by IO latency also. Therefore, if you are interested in seeing results unaffected by IO latency, but can't wait for Thomas to provide a switch to turn off decoding output, you may want to take a look at Timer.
TBeck
May 19 2006, 12:15
QUOTE (Synthetic Soul @ May 19 2006, 11:24) *
Firstly, thanks for the update Thomas. As I said before, I, and I'm sure the other testers, find your reports very interesting. I can't believe that you are squeezing even more speed out, let alone a 50% increase!

Fine. I like to post my reports. One reason: It forces me to define goals for the next release. Otherwise there is always a fair chance, that i am loosing myself in little structured evaluations of my daily ideas.

For the speed increase: I did achieve 50 percent, but the next release will only bring 20 percent for FAST. I will explain the reasons in another post.
QUOTE
Joseph has very eloquently highlighted some anomolies in my results, due to IO limitations. He has some superb charts which articulate the issues I am encountering with speeds over 60x or so. My max is around 80x, while his is 120x.

Yes, Joseph has many superb charts. The Html-reports he is sending me are seldom smaller than 500 KB...
QUOTE
Joseph's initial tests suggest that Timer can accurately report CPU time only (what it calls "Process Time", as opposed to its "Global Time", which is the figure that TimeThis reports). This would be very useful for me to record times unaffected by IO. In fact, I could scrape both CPU (Processed) and CPU+IO (Global) times from the report, for a comparison...

The comparison of the pure processing time would be very intersting for speed evaluations of my code optimizations. And it would give a hint for what to expect if faster pc's or hard disks become available in the future

But it possibly wouldn't be optimal for general comparisons of different compressors. On my system the frame size of my encoder significantly affects encoder and decoder performance. The frame size here equals the block size of the disk io's. On my system 100 ms would be the optimum for speed, but 250 ms are providing maximum compression. This is caused by the way may encoder works. Other -possibly symmetrical- encoders may be able to process smaller frames and therefore could use the optimum of 100 ms on my system. Hence their possible speed advantage would be caused by the design of the encoder and shoud be taken into account for fairness (even if this is an disadvantage for Yalac...).

My 2 cents...
Synthetic Soul
May 19 2006, 12:54
QUOTE (TBeck @ May 19 2006, 12:15) *
Yes, Joseph has many superb charts. The Html-reports he is sending me are seldom smaller than 500 KB...
His dedication to the art is phenominal. I have been trying to pursuade him to post more, as I believe you have. I don't think I can actually imagine the amount of data that you must be sent following a new release. smile.gif

QUOTE (TBeck @ May 19 2006, 12:15) *
The comparison of the pure processing time would be very intersting for speed evaluations of my code optimizations. And it would give a hint for what to expect if faster pc's or hard disks become available in the future
I am intending to perform some runs this weekend, targetting the faster decoders, to see how the figures all tie up. I have some data from Jospeh to compare with. My slower decoders are faster than his, as my machine is faster, but currently he gets better speeds with the faster decoders, by minimising the IO latency. Hopefully I can use his figures, accounting for the difference in PCs, to make a comparison. I'll be sure to post here.

QUOTE (TBeck @ May 19 2006, 12:15) *
My 2 cents...
I must admit that I'm not sure that I understand the paragraph. You state that "it possibly wouldn't be optimal for general comparisons of different compressors" but conclude that it "shoud be taken into account for fairness". I hope I have not taken those quotes completely out of context. unsure.gif Do you basically mean that Yalac may not fair so well if IO latency is taken out of the equation?

Joseph and I have discussed the issue that IO latency will be a "real world" issue for most users, when decoding to WAVE. However, when decoding to RAM (playback) it will not be a factor. It also seems to me to be a perverting influence for test conditions, unless I could somehow provide data that allowed you to detimine the degree to which the IO latency was affecting the speed.

There's another $0.02 for the pot. wink.gif
Shade[ST]
May 19 2006, 15:45
If I use YALAC, I will be using it for playback and transcoding purposes (in the eventuality of a foobar2000 playback plugin). As such, I really don't care much for IO speeds. Don't waste too much time, if you don't need to wink.gif
Synthetic Soul
May 19 2006, 16:23
But this is it. The way I understand it we are currently analysing IO speeds.

To analyse the speed to decode, for transcoding or playback, we need to look at the time taken in memory only, and not the memory plus IO time (which is what TimeThis and Yalac.exe will report).

IMHO both rates are of great interest, but until now I was blissfully unaware of the difference.
TBeck
May 19 2006, 19:42
QUOTE (Synthetic Soul @ May 19 2006, 13:54) *
I must admit that I'm not sure that I understand the paragraph. You state that "it possibly wouldn't be optimal for general comparisons of different compressors" but conclude that it "shoud be taken into account for fairness". I hope I have not taken those quotes completely out of context. unsure.gif Do you basically mean that Yalac may not fair so well if IO latency is taken out of the equation?

No. What i wanted to say: If smaller frame sizes speed up disk io, then the ability of other encoders to use smaller frame sizes (than yalac) is a strength of them, which shoudn't be neglected, because it woud be relevant for the practical use. Hence general comparisons of encoder speed should include disk-io. If yalac should perform better without disk-io, that would be interesting but not too important for the performance under real world conditions. (I am voting against a test setup, which would possibly favour yalac...).

I hope it's more clear now. Have to learn some better english...
Synthetic Soul
May 19 2006, 20:06
Thanks for the clarification Thomas. Your English is infinately better than my German.

From what I understand though, CPU-only timings are also relevant in "real world" scenarios, when decoding to memory while playing, rather than decoding to file (WAVE). Therefore, both timings are of interest: CPU-only/decoding to memory, and CPU+IO/decoding to file.

Thankfully Timer will report both.

I may change my database so that both timings can be reported. I would certainly like to, but time is an issue as always. I thought I had concluded all my little projects, but it always seems there's another just around the corner. I suppose I should stop peering around corners so often...
TBeck
May 19 2006, 20:13
QUOTE (Synthetic Soul @ May 19 2006, 21:06) *
From what I understand though, CPU-only timings are also relevant in "real world" scenarios, when decoding to memory while playing, rather than decoding to file (WAVE). Therefore, both timings are of interest: CPU-only/decoding to memory, and CPU+IO/decoding to file.

I forgot about this.
TBeck
May 19 2006, 21:17
Current Progress (V0.07)

In my last report i was talking about a speed advantage of 50 percent for the new preset FASTEST over FAST. But i did deceide against it.

I did play around with some parameters to speed up FAST. I could achieve more than 50 percent more speed with an compression penality of about 1 percent. But one single parameter variation gave me 20 percent speed up with a penality of only 0.1 percent. Obviously a far better speed-compression ratio.

Therefore i dropped the FASTEST preset. V0.07 will instead contain a 20 percent faster FAST preset.

Then i looked at the other presets. I always had the feeling, that they are not very well balanced. After many evaluations of existing test data and new parameter variations i came up with a new configuration of the presets:

- Presets: FAST, NORMAL, HIGH, EXTRA, INSANE.
- INSANE sets any accessible parameter to the maximum to evaluate the currently possible maximum compression (Joseph Pohm may call it I2...).
- With the exception of INSANE, any preset should be only two times slower than it's predecessor. I find this simple rule more user friendly.
- Therefore NORMAL and HIGH needed a speed up. HIGH is now about 80 percent faster than before.
- EXTRA now uses the parameter configuration of the old HIGH preset with the addition of the new improvements of the stereo decorrelation.

To make it clear: I didn't perform any code optimizations. The new preset configuration is only based upon new selections of the underlying encoder parameters.

On my test corpus the new presets are considerably faster than the old ones with a compression penality of about 0.1 percent. I hope that your evaluation of V0.07 will confirm this. Otherwise i may have to perform some more fine tuning.

I am not sure, what more i will do for V0.07. Possibly i will drop some other plans for now (for instance the possibility to vary disk-io parameters), because there is allreday enough to be evaluated.
Synthetic Soul
May 22 2006, 10:08
Questions regarding release date and license have been moved to Yalac: Miscellaneous Questions.

FYI, we now have:
TBeck
May 25 2006, 01:12
V0.07 is done

Changes:

Compression algorithms:

- New option for the channel decorrelation to detect bigger lags between channels. Without this option files with lags outside of the search range of the standard decorrelation algorithm can not use channel decorrelation. Don't expect too much, on my test corpus it only helps about 1 of 20 files. And it slows encoding and -in case of a bigger lag- even decoding down. Sometimes this option slightly (less than 0.05 percent) decreases compression efficiency.

Presets:

- Reconfiguration of the existing presets and inclusion of the new preset EXTRA.
- With the exception of INSANE, any preset should be only two times slower than it's predecessor. I find this simple rule more user friendly.
- INSANE sets any accessible parameter to the maximum to evaluate the currently possible maximum compression (Joseph Pohm may call it I2...).
- EXTRA now uses the parameter configuration of the old HIGH preset with the addition of the new improvements of the stereo decorrelation.
- Speedup of the presets FAST (20 %), NORMAL (35 %) and HIGH (60 %). The compression efficiency should be only 0.1 percent worse than before.

Internals:

- New way to scale sample values down for the reduced precision arithmetic of my 16 Bit DSP. Can slightly change compression ratio of individual files.

GUI:

- Debug option "No Output" of the GUI-Version now disables generation of output files for both encoding and decoding.

Command line:

- Specify the new switch "-debug1" to disable file output when encoding or decoding.

- New test cases to evaluate the new channel decorrelation option "Extra lag":
CODE
-c0 = Off
-c1 =  8
-c2 = 16
-c3 = 32
-c4 = 64


Other:

- Clean up of source code. Could give new errors...
- File format changed!

Release:

I hope to send the new version to the following testers within the next 24 hours:

Destroid
Josef Pohm
Shade[ST]
Synthetic Soul

Only reason for this selection: I haven't heard from the other testers within the last 10 days and i don't want to fill their mail box with new versions they possibly currently don't need.

Any of them can sent me an email anytime, and i will sent the current version.

What should be tested:

- Comparison with V0.06: Speed and compression performance of presets FAST, NORMAL, HIGH and EXTRA.
- If the new preset EXTRA should perform better than HIGH of V0.06, then it would make sense to further evaluate the new channel decorrelation option "Extra lag". You may try preset EXTRA with bigger lags: -c3 or c4 for 32 / 64. The protocol file contains a new section with a distribution of the lags the encoder has used.
- A verification of the decoded files makes most sense for preset EXTRA. There is no need to verify the output of the other presets.

Plans for V 0.08:

- I am always working on the optimization of some new filter algorithm, which can significantly improve compression. Probably the next version will contain a first suboptimal implementation.

- Some complex modifications of my frame partitioning may provide a small increase of compression efficiency. I am not quite sure, if it is worth the considerable amount of work.

Thomas
Destroid
May 25 2006, 02:46
QUOTE (TBeck @ May 25 2006, 00:12) *
What should be tested:

- Comparison with V0.06: Speed and compression performance of presets FAST, NORMAL, HIGH and EXTRA.
- If the new preset EXTRA should perform better than HIGH of V0.06, then it would make sense to further evaluate the new channel decorrelation option "Extra lag". You may try preset EXTRA with bigger lags: -c3 or c4 for 32 / 64. The protocol file contains a new section with a distribution of the lags the encoder has used.
- A verification of the decoded files makes most sense for preset EXTRA. There is no need to verify the output of the other presets.


I am writing the batch file for a full-scale test/comparison now for 0.06 vs. 0.07 and all the other major compressors smile.gif I am going assume that EXTRA will be tested four times using -c[n] variants. If it's worth bothering, would testing INSANE with -c3 and -c4 be of interest?

And yes, I'll be adding FC /b verification for the EXTRA settings.
TBeck
May 25 2006, 03:09
QUOTE (Destroid @ May 25 2006, 03:46) *
I am writing the batch file for a full-scale test/comparison now for 0.06 vs. 0.07 and all the other major compressors smile.gif I am going assume that EXTRA will be tested four times using -c[n] variants. If it's worth bothering, would testing INSANE with -c3 and -c4 be of interest?

Wow! That's very exciting!

If -c3 and -c4 don't help EXTRA, there would be little to expect for INSANE. But if you can do it automatically...
Destroid
May 25 2006, 10:56
Another album comparison benchmark smile.gif This album is rock and multiple types of instruments (acoustic and electric guitar, electric bass, woodwinds, drums) with monologue and chorus vocals.
CODE
King Missile - Mystical S**t/Fluting on the Hump 697,725,548 bytes (65:55)
===========================================================================
name/params Ratio EncTime/CPU% DecTime/CPU%
--------------------- ------ ------------ ------------
Yalacc 0.06 -p0 55.04% 76.54x / 76% 92.42x / 55%
Yalacc 0.07 -p0 55.16% 84.01x / 72% 93.40x / 56%
MAC 4.01 beta2 -c1000 55.56% 66.31x / 96% 51.70x / 85%
FLAC 1.1.2 --fast 62.60% 86.09x / 68% 83.15x / 52%
WavPack 4.3 -f 58.03% 90.62x / 74% 87.16x / 60%
OFR 4.520 --mode fast 54.80% 24.70x / 87% 34.41x / 98%
--------------------- ------ ----------- ------------
Yalacc 0.06 -p1 54.73% 33.27x / 94% 87.80x / 60%
Yalacc 0.07 -p1 54.76% 43.49x / 90% 84.83x / 60%
MAC 4.01 beta2 -c2000 54.60% 49.61x / 96% 42.71x / 92%
FLAC 1.1.2 (default) 58.12% 59.22x / 83% 85.89x / 57%
WavPack 4.3 (default) 57.15% 77.69x / 79% 80.53x / 67%
OFR 4.520 (default) 54.10% 17.88x / 91% 24.76x / 99%
MP4ALS RM17 (default) 56.45% 29.76x / 95% 55.96x / 63%
LA 0.4 normal 53.05% 6.20x / 99% 7.72x / 99%
--------------------- ------ ----------- ------------
Yalacc 0.06 -p2 54.57% 11.92x / 99% 83.29x / 61%
Yalacc 0.07 -p2 54.63% 17.59x / 98% 82.40x / 64%
MAC 4.01 beta2 -c3000 54.32% 43.41x / 98% 38.13x / 93%
MAC 4.01 beta2 -c4000 53.90% 23.97x / 99% 23.40x / 98%
FLAC 1.1.2 --best 57.95% 12.12x / 99% 88.97x / 60%
WavPack 4.3 -h 55.27% 52.10x / 89% 66.47x / 87%
OFR 4.520 --best 53.84% 4.85x / 95% 6.57x / 99%
MP4ALS RM17 -7 55.07% 0.99x / 99% 12.57x / 96%
LA 0.4 high 52.89% 4.61x / 99% 5.45x / 99%
--------------------- ------ ----------- ------------
Yalacc 0.06 -p3 54.51% 4.47x / 99% 84.66x / 62%
Yalacc 0.07 -p3 * 54.57% 11.58x / 99% 84.20x / 63% (FC /b = OK)
Yalacc 0.07 -p3 -c3 * 54.57% 11.43x / 99% 84.00x / 62% (FC /b = OK)
Yalacc 0.07 -p3 -c4 * 54.57% 11.16x / 99% 84.04x / 62% (FC /b = OK)
Yalacc 0.07 -p4 54.51% 3.36x / 99% 85.72x / 63%

* Denotes modes with files compared after encoding and decoding



No noticable difference in EXTRA using -c3 and -c4 except for encoding times, so I didn't bother using the switches on INSANE mode. Perhaps different material would make a noticable difference. As noted above, the EXTRA mode had no problems with file integrity as proven by file comparison.

There are some nice improvements in performance here. Awesome smile.gif

Oh yeah wink.gif -- System A64 3000+, 512MB, Caviar 80GB, Win2K
Synthetic Soul
May 26 2006, 23:08
http://synthetic-soul.co.uk/comparison/yalac/

As before, all Yalac runs (0.02-0.07) can be viewed by using http://synthetic-soul.co.uk/comparison/yalac/?all=1.

NB: This table uses Timer's Global Time, which equates to the time reported by TimeThis (CPU+IO). I do have the Process times recorded as well, but I need to get some time to create version 2 of my system to report them. If I can't find time soon I may just upload a/some suplimentary spreadsheet/s.

My scripts now automatically check the MD5 on decode against the source MD5, and all hashes match.
TBeck
May 27 2006, 00:26
Many thanks again to Destroid and Synthetic Soul!

For me the reconfiguration of the presets looks advantageous. There are some nice speed improvements for FAST, NORMAL and HIGH and the reduction in compression efficiency is on average less than 0.1 percent. I would like to keep the new presets. What do you think?

The extended search range of the stereo decorrelation (preset EXTRA) seems useless. Nothing really new for me: About 95 percent of my work on optimizations of the compression efficiency were useless. But you have to try it before you know it... I will probably remove the extra lag option from the encoder. Fortunately there is another currently hidden option aimed to some different purpose within the frame decorrelation which can replace the extra lag option to some extent, if it should be really needed.

I am currently working on the new prefilter option. Earlier (when applied to yalac 0.01) it gave 0.35 percent better compression, but now only 0.2 percent on average... Maybe that some other optimizations of yalac are allready providing some of the benefits of the prefilter. On the other hand there are some files which compress more than 4 percent better with the filter. Possibly there is some room for improvements left.

It seems, as if i am near to the maximum compression my codec design can achieve. In this case i would call the 0.2 percent improvement significant...

That doesn't mean, that there will be no more improvements. For instance the frame partition calculator isn't fully optimized yet.
Mo0zOoH
May 27 2006, 02:57
QUOTE (TBeck @ May 27 2006, 03:26) *
I am currently working on the new prefilter option. Earlier (when applied to yalac 0.01) it gave 0.35 percent better compression, but now only 0.2 percent on average... Maybe that some other optimizations of yalac are allready providing some of the benefits of the prefilter. On the other hand there are some files which compress more than 4 percent better with the filter. Possibly there is some room for improvements left.

It seems, as if i am near to the maximum compression my codec design can achieve. In this case i would call the 0.2 percent improvement significant...

That doesn't mean, that there will be no more improvements. For instance the frame partition calculator isn't fully optimized yet.

TBeck, are you a good wizard or something? biggrin.gif
I will definitely start using your codec right after the fb2k plugin comes out!
Synthetic Soul
May 27 2006, 09:33
Thomas, as per previous request, 0.07 values have now replaced 0.06 values in the multi-encoder comparison:

http://www.synthetic-soul.co.uk/comparison/lossless/

As previously, non-standard settings (-cX) can be viewed by appending ?all=1 (and removed by using ?all=0):

http://www.synthetic-soul.co.uk/comparison/lossless/?all=1


Personally, I like the way that the new presets have a good spread of encoding speed (roughly halfing, except for insane which just goes all out). Compression does not alter drastically, so you can pick a preset according to your speed requirements, and know that it won't make a horrendous difference, and that the difference will be exponentially relative to the speed benefit. It seems a very tidy way of doing things, and easy for a user to find the best trade-off.
dethis
May 27 2006, 10:35
Hi Thomas. You are realy a wizard

As about the presets, they look perfectly balanced except the 0.7 extra. Looking at SyntheticSoul's and Destroid's results 0.6 HIGH gives better or equal compression compared to 0.7 EXTRA while being 5-10% faster.

Waiting for the FooBar plugin.
Shade[ST]
May 27 2006, 14:48
Yalac fast mode beats them all, when you sort by encoding rate / size ratio, but monkey's normal is faster AND compresses better than yalac... I don't know if anything is to be done with this, but right now fast mode seems fine. Maybe if you could concentrate on the other modes..
Synthetic Soul
May 27 2006, 19:19
QUOTE
' date='May 27 2006, 14:48' post='396419']monkey's normal is faster AND compresses better than yalac...
It decodes twice as slowly though.

I'm not sure that you can compare a preset name so directly. Yalac can't expect to compete in every league.

The difference in compression and encoding rate is minimal. However, trying to equal Monkey's Audio in either value would be at the detriment to the other... unless Thomas can squeeze another 0.3-0.4% compression out, so that he can afford to speed encoding up a little.

Hey, if he can do it then I'll not complain!

I just think that tackling Monkey's Audio head on is unrealistic, especially if you look at Monkey's High surpassing Yalac's Insane compression marginally, and encoding rate substantially.
TBeck
May 28 2006, 22:17
QUOTE (Synthetic Soul @ May 27 2006, 20:19) *
It decodes twice as slowly though.

The ratio my be even better for playback (where disk-io shouldn't matter) or when decoding from memory. I am not sure, if my following estimation for cpu usage of real time playback from memory based upon Destroid's timings and cpu times is valid:
CODE
Yalacc 0.07    -p1    54.76% 43.49x / 90% 84.83x / 60% -> 60 / 84.83 = 0.707 %
MAC 4.01 beta2 -c2000 54.60% 49.61x / 96% 42.71x / 92% -> 92 / 42.71 = 2.154 %

We will have to wait for a Yalac playback plugin to confirm it. Or for someone, who tells me, why this calculation is wrong...
QUOTE
...I'm not sure that you can compare a preset name so directly. Yalac can't expect to compete in every league.

...unless Thomas can squeeze another 0.3-0.4% compression out, so that he can afford to speed encoding up a little.

...I just think that tackling Monkey's Audio head on is unrealistic, especially if you look at Monkey's High surpassing Yalac's Insane compression marginally, and encoding rate substantially.

Very good arguments! I totally agree.
Josef Pohm
May 29 2006, 09:25
QUOTE (TBeck @ May 28 2006, 23:17) *
We will have to wait for a Yalac playback plugin to confirm it. Or for someone, who tells me, why this calculation is wrong...


Just to confirm that your calculation is obviously right, and that I think it could be more readable like this:

CODE
Yalac Normal
Enc: 43,49x @ 90% -> 43,49/0,90 =  48,32x
Dec: 84,83x @ 60% -> 84,83/0,60 = 141,38x

Monkey Normal
Enc: 49,61x @ 96% -> 49,61/0,96 =  51,68x
Dec: 42,71x @ 92% -> 42,71/0,92 =  46,42x


Which shows that Yalac v0.07 Normal encodes slightly slower than Monkey Normal 4.01 but decodes more than three times faster.
I have some results of mine for confirmation:

CODE
               Yalac Normal     Monkey Normal
Encode            36,53x           40,23x
Decode           106,77x           33,05x


Test repeated on another PC:

CODE
               Yalac Normal     Monkey Normal
Encode            28,33x           31,19x
Decode            92,53x           29,22x


Both confirming Yalac v0.07 Normal encodes slightly slower than Monkey Normal 4.01 but decodes more than three times faster.
Synthetic Soul
May 29 2006, 12:06
OK, OK, you guys got me. Considering I've been harping on about CPU-only time constantly in the last week you would have thought I might have picked up on that one... wink.gif

In response to Thomas' post I was actually just in the middle of looking at my Process Time (CPU-only) times, when I spotted that Josef had posted. Here they are anyway:

CODE
Codec             Encode    Decode
==================================
Yalac   Normal    34.11x    97.49x
Monkeys Normal    41.05x    37.96x
Yalac   High      15.07x    91.01x
Monkeys High      36.31x    34.34x

Just a note to say that you can now view Josef's full comparison details here.
TBeck
May 29 2006, 13:45
QUOTE (Synthetic Soul @ May 29 2006, 13:06) *
Just a note to say that you can now view Josef's full comparison details here.

Excellent!

But a bit depressing: Yalac compares worse than on any other sample set i have seen!

I will speed up my work on the PreFilter implementation. Although it gives only about 0.15 to 0.20 percent better compression on my test corpus, i am confident because it were some sample files of joseph which have shown a advantage of several percent (!) with the prefilter turned on...

I hope to release the next version within the next days.
Synthetic Soul
May 29 2006, 19:32
QUOTE (TBeck @ May 29 2006, 13:45) *
But a bit depressing: Yalac compares worse than on any other sample set i have seen!
Yes, I can see how you must be disappointed (even ashamed): only 5th fastest decoder:

CODE
Encoder Setting    Compressed    Decoding
=========================================
Shorten Default       66.238%     132.44x
DaxWav VQH            70.102%     124.80x
DaxWav VQS            69.568%     124.80x
DaxWav VQM            70.008%     119.33x
Yalac Fast            58.527%     114.06x
Flac - Garf -0        66.435%     102.81x

... but look at the compression (and who's 6th fastest)...

Well, if such high standards means you will look to squeezing even more MB out then who am I to correct you. wink.gif
TBeck
May 29 2006, 21:37
QUOTE (Synthetic Soul @ May 29 2006, 20:32) *
QUOTE (TBeck @ May 29 2006, 13:45) *
But a bit depressing: Yalac compares worse than on any other sample set i have seen!
Yes, I can see how you must be disappointed (even ashamed): only 5th fastest decoder:
...
... but look at the compression (and who's 6th fastest)...

Well, if such high standards means you will look to squeezing even more MB out then who am I to correct you. wink.gif

You are right. No, i wasn't fishing for compliments... But they are welcome.

It's my point of view. In the many years of the development of yalac i never cared about FAST modes, i always compared HIGH modes. FAST is the result of the last months interaction with the hydrogen forum members.

And the comparison with LPAC and Mpeg4Als -7 is always most important for me, because they are using basically the same technology as yalac. It's ok to be a bit (up to 0.5 percent) worse, because i had to sacrifice some compression efficiency for the high decoding speed. But the difference is higher in Josefs comparison! That's what depresses me a bit. Ok, it's only one of many test sets, but it isn't ok.

For now i am pinning all my hope on the PreFilter (do you notice the dramatic?).

The four sample files Josef sent me earlier perform very well with the PreFilter:
CODE
Preset High            Prefilter
                       off       on
03_06_03.yaa           64.21 % - 62.24 %
B04_01.yaa             53.76 % - 49.62 %
B04_02.yaa             58.97 % - 54.11 %
C17_01.yaa             66.59 % - 63.16 %

                       60.59 % - 56.83 %


But i don't know, if josef's test set does contain more such Prefilter friendly files...
Synthetic Soul
May 29 2006, 22:38
QUOTE (TBeck @ May 29 2006, 21:37) *
In the many years of the development of yalac i never cared about FAST modes, i always compared HIGH modes. FAST is the result of the last months interaction with the hydrogen forum members.
That must be quite strange for you. I would probably use High if using Yalac, maybe Normal. There isn't a huge variation in the compression rate, and all presets decode very fast. I am happy to go with a relatively slow encode if it will save me a few MB, as I generally start the process going and leave the room. I use WavPack -h as it has decent compression, decent speed, but also very much because of other aspects, like error tolerance, simple inline tagging, etc.

QUOTE (TBeck @ May 29 2006, 21:37) *
And the comparison with LPAC and Mpeg4Als -7 is always most important for me, because they are using basically the same technology as yalac. It's ok to be a bit (up to 0.5 percent) worse, because i had to sacrifice some compression efficiency for the high decoding speed.
Ah, that is interesting to know. I must admit I know little of MP4ALS and nothing of LPAC. I will have to investigate further.

QUOTE (TBeck @ May 29 2006, 21:37) *
The four sample files Josef sent me earlier perform very well with the PreFilter:
Wow. I'm looking forward to checking that on my corpus.
TBeck
May 29 2006, 23:10
QUOTE (Synthetic Soul @ May 29 2006, 23:38) *
...
I use WavPack -h as it has decent compression, decent speed, but also very much because of other aspects, like error tolerance, simple inline tagging, etc.

V 0.08 (which is nearly done) will bring the PreFilter. If there are no problems with this new feature, then my work on V 0.09 will consist of clean ups of my source code and error tolerance.

QUOTE (Synthetic Soul @ May 29 2006, 23:38) *
I must admit I know little of MP4ALS and nothing of LPAC. I will have to investigate further.

Feel free to ask me. I would like to write some short text about the similarities, but you know, english is not a strength of mine. Maybe i will pause the development work on some point and write a german text. One friend of mine will translate it to english.

QUOTE (Synthetic Soul @ May 29 2006, 23:38) *
QUOTE (TBeck @ May 29 2006, 21:37) *
The four sample files Josef sent me earlier perform very well with the PreFilter:
Wow. I'm looking forward to checking that on my corpus.

Don't forget: On my test corpus the PreFilter achieves only 0.15 to 0.20 percent improvement! Josef is always providing me with some very special files which are giving me much fun! I don't know, if you can find them on the free market...

BTW: I have the feeling, that i should make a plan for the next development steps of yalac and possibly post it with some other general info within some new YALAC Faq post... For me it's always advantegous to force myself to think about a better structure of my work. And Yalac surely has raised some expectations and people should know what will be going on. If i know it...
Shade[ST]
May 29 2006, 23:47
I think YALAC developpement has reached another step which you might not have envisioned previously; I am sure many other people follow YALAC development closely, and would be very willing to test it, which is why I suggest you consider testing on a larger scale. That way, you could really see if your optimizations pay off and if you have the same general results everywhere.

Perhaps Synthetic soul could set up a small system on his site to receive feedback from testers, and distribute the executable. If not, I might be able to piece something together, and I can surely help you with the hosting part. I could even give you a ftp account password.

Consider all this well,
Tristan.
TBeck
May 30 2006, 16:43
QUOTE
' date='May 30 2006, 00:47' post='397364']
... which is why I suggest you consider testing on a larger scale. That way, you could really see if your optimizations pay off and if you have the same general results everywhere.

Perhaps Synthetic soul could set up a small system on his site to receive feedback from testers, and distribute the executable. If not, I might be able to piece something together, and I can surely help you with the hosting part. I could even give you a ftp account password.

Many thanks, Tristan! It's quite probable that i will come back to your hosting offer!

But for the extended testing: I am not sure, if it is worth the effort, because more testers means far more results to be evaluated and far more communications and discussions, hence less time left for the development. And new testers wouldn't know about the development history and probably come up with old questions. And i wouldn't know much about their test sets. The active testers have tried the different versions on their test sets and the varying results (between versions) have shown me something about the characteristics of their files.

I know, that it isn't possible, but: If there were only one or two new testers who provided me with such valuable and detailed reports as Josef Pohm, i would spent my whole time on reading and analysing...

Thomas

BTW: I hope that you will try the new version on your test set. If i remember it right, you are having some interesting music within your set, which other test sets are possibily missing...
Synthetic Soul
May 30 2006, 17:07
QUOTE (TBeck @ May 29 2006, 23:10) *
V 0.08 (which is nearly done) will bring the PreFilter. If there are no problems with this new feature, then my work on V 0.09 will consist of clean ups of my source code and error tolerance.
Wow. Sounds like we may be getting close to a beta release.

QUOTE (TBeck @ May 29 2006, 23:10) *
BTW: I have the feeling, that i should make a plan for the next development steps of yalac and possibly post it with some other general info within some new YALAC Faq post... For me it's always advantegous to force myself to think about a better structure of my work. And Yalac surely has raised some expectations and people should know what will be going on. If i know it...
Sounds intriguing. I love reading these things. I used to love reading the diaries of games developers in computer magazines when I was a lad (Andrew Braybrook's Paradroid diary and Jeff Minter's Iridis Alpha (what a game!) diary spring to mind).

QUOTE (Shade[ST @ May 29 2006, 23:47) *
Perhaps Synthetic soul could set up a small system on his site to receive feedback from testers, and distribute the executable. If not, I might be able to piece something together, and I can surely help you with the hosting part. I could even give you a ftp account password.
I'm sorry, but I cannot commit myself any further than I am at the moment (which is really more than I should, but I just can't stop it). If Thomas thinks this would be useful, then please, take it on.
TBeck
May 30 2006, 22:14
V0.08 is done

Changes:

Compression algorithms:

- New PreFilter option. Achieves on average 0.15 to 0.20 percent better compression on my test corpus, up to 4 percent on some special files.
- Extra lag option (introduced in V0.07) for the channel decorrelation removed.
- Channel decorrelation option FULL SEARCH removed.

Presets:

- HIGH now uses the new PreFilter. Expect a speed decrease of about 10 % for encoding. The effect on decoding performance should be minimal.
- EXTRA now uses the new PreFilter and the frame partition search level EXTRA (formerly known as insane). Extra lag option for the channel decorrelation removed. Should be about two times slower than HIGH.
- INSANE now uses the new PreFilter. Channel decorrelation option FULL SEARCH removed. Should be considerably faster now.

INSANE is only a temporary storage for methods, which are far too slow for the compression improvements they achieve. Instead of immediately removing them i keep them within INSANE because i may speed them up later or because i am unsure, if they can help other files more, and want to wait for more test results.

Josef Pohms test results show a significant compression advantage for partition search level EXTRA (formerly known as insane), therefore i deceided to move it into the standard presets. But for the channel decorrelation option FULL SEARCH it was time to say goodbye...

Command line:

- New test cases to evaluate the new PreFilter-option:

-c0 = Off
-c1 = Level Fast
-c2 = Level Normal

Other:

- Clean up of source code. Could give new errors...
- File format changed!

Release:

I hope to send the new version to the following testers within the next 24 hours:

Destroid
Josef Pohm
Shade[ST]
Synthetic Soul

Only reason for this selection: I haven't heard from the other testers within the last 10 days and i don't want to fill their mail box with new versions they possibly currently don't need.

Any of them can sent me an email anytime, and i will sent the current version.

What should be tested:

- Comparison with V0.08: Speed and compression performance of presets HIGH, EXTRA and possibly INSANE.
- HIGH now uses the new PreFilter option with level FAST. You may try -c2 on HIGH, which activates level NORMAL, and check, if it makes any difference.
- A verification of the decoded files makes most sense for preset HIGH or EXTRA (simply choose a preset, which uses the new PreFilter).

Plans for V 0.09:

- Optimizations and corrections of the PreFilter, if neccessary.
- Error tolerance: Detect and skip damaged frames.

Possibly you will have to wait for V0.10 for more error tolerance. After removing the FULL SEARCH option of the channel decorrelation there are some new opportunities for speed improvements, which i may implement in V0.09. And i just have found some interactions of the PreFilter with other encoder methods which have to be evaluated. Maybe it hurts them a bit.
Supacon
May 30 2006, 23:48
Awesome! Glad to hear that you're hard at work.

I can't wait to try a beta version in the near future. I guess you still need to add a few features, like seekability, streamability, tagging, et al, but it sounds like you've got a great algorithm for the most important aspects. It's pretty impressive that you think you can still squeeze some more speed out of your codec too!

Keep it up Thomas!
SebastianG
May 31 2006, 00:26
Hello, TBeck!

I'm wondering what the "PreFilter" actually does and how it is related to the other parts. Could you shed some light on it?

Sebi
TBeck
May 31 2006, 02:17
QUOTE (Supacon @ May 31 2006, 00:48) *
...
It's pretty impressive that you think you can still squeeze some more speed out of your codec too!

Thanks!

But don't expect big speed improvements. Maybe 10 to 20 percent for the presets HIGH and EXTRA.

FAST could be speed up, if i would implement a specialized version of my codec. But this has not a high priority for me.
TBeck
May 31 2006, 03:11
QUOTE (SebastianG @ May 31 2006, 01:26) *
I'm wondering what the "PreFilter" actually does and how it is related to the other parts. Could you shed some light on it?

Sorry, but i don't want to publish details at this point.

As i wrote earlier, i definitely want to benefit from my work in form of some reputation. One way may be the publication of one or two papers.

Some (not directed to you!) may find this a bit selfish or egocentric (earlier you have been a hero, if you published useful freeware, these days you are often a bad guy, if you do not immediately release the source code too...).

What could i reply?

I have been working on Yalac for more than 8 years, always in my free time. To give an estimate: This means about 1.5 years of full work (or of my life time)!

With this in mind, it seems not too easy to give all the work away for nothing...

Very important:

1) This is not directed to you! You only provided me the opportunity to clear some points, which tend to come up now and then.

2) I do not want to start an ideological or philosophical discussion about open source or intellectual property. If necessary, this should be done within a seperate thread. And given my limited english skills, it's very unlikely that i would participate.


What i can say:

Short description from Yalac's readme: "The prefilter cleans the signal a bit, before it is beeing sent to the linear predictor." In other words: It removes some disturbing aspects of the signal, which reduce the quality of the predictor coefficients and hence hurt the compression.

But to be honest: If there is one method within yalac, which i myself do not fully understand yet, then it is the PreFilter! Surely i had reasons to implement it and it works as expected, but sometimes far better than i would have predicted and i will have to evaluate why.

Possibly it compensates some disturbances of the signal which are beeing caused by the scaling needed for my reduced precision (14-Bit) arithmetic. If that's true, then the PreFilter would only be useful for my specific implementation and not for other compressors which are using full scale arithmetics.

Thomas
Shade[ST]
May 31 2006, 03:44
Output is disabled for all encoding settings; SSE is enabled, and all files are transcoded from LAME 3.98a3 -V2 --vbr-new. As such, this doesn't make for a _real-world_ test, but it's quite revelating nonetheless, on different settings, and the quality of the stereo decorellator (indeed better than LAME's)
If you want the bitrates of the original mp3s, I can have them for you. I can also test with a few CD Audio files, if you like, but I doubt the results will be different.
Files are cached before reading
CODE
Default + prefilter
01. Giant Steps.yaa 57.45 % - 20.2 * - 14.220 sec
01. La Vie, L'amour.yaa 47.70 % - 21.3 * - 5.200 sec
01. Le vieux Léon.yaa 36.69 % - 22.8 * - 9.962 sec
02. Parallel Universe.yaa 71.40 % - 23.0 * - 11.740 sec
02. Sa jeunesse.yaa 36.48 % - 23.1 * - 12.362 sec
03. Memories of You.yaa 32.29 % - 23.3 * - 8.253 sec
04. Carl Orff - I. Primo vere - Omnia sol temperat.yaa 39.29 % - 20.3 * - 5.311 sec
04. Celle qui a dit non.yaa 65.00 % - 12.2 * - 23.318 sec
04. Comme Moi (Like Me).yaa 43.11 % - 14.5 * - 12.365 sec
05. Le chemin de la dompe (présentation).yaa 43.90 % - 15.2 * - 4.182 sec
06. Frédéric.yaa 47.80 % - 17.3 * - 11.657 sec
11. Nil (Instrumental).yaa 20.22 % - 23.2 * - 5.905 sec
12. Sergei Sergeyevich Prokofiev - Dance of the knights.yaa 43.06 % - 22.4 * - 14.418 sec
15. Stroker Ace.yaa 54.57 % - 22.1 * - 12.201 sec
17. Carl Orff - III. Cour d'amours - Dies, nox et omnia.yaa 39.24 % - 23.2 * - 5.669 sec
20. Sleighride.yaa 56.57 % - 22.1 * - 8.094 sec

Compression: 47.88 %
Duration: 164.89 sec
Speed: 19.71 * real time

Default + no prefilter :
01. Giant Steps.yaa 57.53 % - 24.2 * - 11.877 sec
01. La Vie, L'amour.yaa 47.54 % - 34.2 * - 3.236 sec
01. Le vieux Léon.yaa 36.61 % - 37.0 * - 6.134 sec
02. Parallel Universe.yaa 71.41 % - 36.6 * - 7.395 sec
02. Sa jeunesse.yaa 36.45 % - 36.6 * - 7.791 sec
03. Memories of You.yaa 32.23 % - 39.6 * - 4.855 sec
04. Carl Orff - I. Primo vere - Omnia sol temperat.yaa 39.25 % - 33.2 * - 3.252 sec
04. Celle qui a dit non.yaa 65.08 % - 29.3 * - 9.714 sec
04. Comme Moi (Like Me).yaa 43.00 % - 33.3 * - 5.392 sec
05. Le chemin de la dompe (présentation).yaa 43.88 % - 33.0 * - 1.927 sec
06. Frédéric.yaa 47.78 % - 31.4 * - 6.443 sec
11. Nil (Instrumental).yaa 20.04 % - 41.0 * - 3.347 sec
12. Sergei Sergeyevich Prokofiev - Dance of the knights.yaa 43.07 % - 34.9 * - 9.234 sec
15. Stroker Ace.yaa 54.47 % - 32.5 * - 8.285 sec
17. Carl Orff - III. Cour d'amours - Dies, nox et omnia.yaa 39.21 % - 14.4 * - 9.118 sec
20. Sleighride.yaa 56.63 % - 18.2 * - 9.813 sec

Compression: 47.85 %
Duration: 107.83 sec
Speed: 30.14 * real time

Default + search level extra :
01. Giant Steps.yaa 57.44 % - 14.0 * - 20.430 sec
01. La Vie, L'amour.yaa 47.48 % - 14.3 * - 7.735 sec
01. Le vieux Léon.yaa 36.56 % - 14.6 * - 15.500 sec
02. Parallel Universe.yaa 71.39 % - 11.0 * - 24.493 sec
02. Sa jeunesse.yaa 36.44 % - 12.0 * - 23.722 sec
03. Memories of You.yaa 32.18 % - 13.7 * - 14.090 sec
04. Carl Orff - I. Primo vere - Omnia sol temperat.yaa 39.24 % - 14.2 * - 7.623 sec
04. Celle qui a dit non.yaa 65.04 % - 11.3 * - 25.109 sec
04. Comme Moi (Like Me).yaa 42.99 % - 13.4 * - 13.403 sec
05. Le chemin de la dompe (présentation).yaa 43.86 % - 13.0 * - 4.898 sec
06. Frédéric.yaa 47.77 % - 13.4 * - 15.114 sec
11. Nil (Instrumental).yaa 19.88 % - 14.0 * - 9.815 sec
12. Sergei Sergeyevich Prokofiev - Dance of the knights.yaa 43.05 % - 12.9 * - 25.004 sec
15. Stroker Ace.yaa 54.45 % - 13.5 * - 19.926 sec
17. Carl Orff - III. Cour d'amours - Dies, nox et omnia.yaa 39.20 % - 14.4 * - 9.105 sec
20. Sleighride.yaa 56.55 % - 10.9 * - 16.407 sec

Compression: 47.81 %
Duration: 252.39 sec
Speed: 12.88 * real time

Default + frame partition 32 :
01. Giant Steps.yaa 57.59 % - 21.7 * - 13.203 sec
01. La Vie, L'amour.yaa 47.57 % - 26.7 * - 4.150 sec
01. Le vieux Léon.yaa 36.70 % - 36.6 * - 6.203 sec
02. Parallel Universe.yaa 71.48 % - 37.6 * - 7.192 sec
02. Sa jeunesse.yaa 36.46 % - 39.5 * - 7.227 sec
03. Memories of You.yaa 32.27 % - 41.8 * - 4.611 sec
04. Carl Orff - I. Primo vere - Omnia sol temperat.yaa 39.25 % - 38.3 * - 2.821 sec
04. Celle qui a dit non.yaa 65.16 % - 31.7 * - 8.968 sec
04. Comme Moi (Like Me).yaa 43.00 % - 34.5 * - 5.204 sec
05. Le chemin de la dompe (présentation).yaa 43.94 % - 36.1 * - 1.765 sec
06. Frédéric.yaa 47.84 % - 35.7 * - 5.659 sec
11. Nil (Instrumental).yaa 20.04 % - 43.6 * - 3.145 sec
12. Sergei Sergeyevich Prokofiev - Dance of the knights.yaa 43.08 % - 38.2 * - 8.442 sec
15. Stroker Ace.yaa 54.60 % - 34.4 * - 7.825 sec
17. Carl Orff - III. Cour d'amours - Dies, nox et omnia.yaa 39.21 % - 39.4 * - 3.338 sec
20. Sleighride.yaa 56.67 % - 35.4 * - 5.052 sec

Compression: 47.90 %
Duration: 94.82 sec
Speed: 34.28 * real time


Default + 64 preds :
01. Giant Steps.yaa 57.79 % - 27.0 * - 10.625 sec
01. La Vie, L'amour.yaa 47.68 % - 41.2 * - 2.684 sec
01. Le vieux Léon.yaa 36.65 % - 45.1 * - 5.037 sec
02. Parallel Universe.yaa 71.42 % - 43.8 * - 6.171 sec
02. Sa jeunesse.yaa 36.58 % - 45.6 * - 6.261 sec
03. Memories of You.yaa 32.25 % - 47.8 * - 4.023 sec
04. Carl Orff - I. Primo vere - Omnia sol temperat.yaa 39.34 % - 42.1 * - 2.568 sec
04. Celle qui a dit non.yaa 65.11 % - 33.7 * - 8.446 sec
04. Comme Moi (Like Me).yaa 43.21 % - 35.5 * - 5.046 sec
05. Le chemin de la dompe (présentation).yaa 43.89 % - 25.4 * - 2.507 sec
06. Frédéric.yaa 47.86 % - 29.4 * - 6.885 sec
11. Nil (Instrumental).yaa 19.96 % - 31.1 * - 4.405 sec
12. Sergei Sergeyevich Prokofiev - Dance of the knights.yaa 43.56 % - 37.1 * - 8.691 sec
15. Stroker Ace.yaa 54.49 % - 29.9 * - 9.002 sec
17. Carl Orff - III. Cour d'amours - Dies, nox et omnia.yaa 39.36 % - 36.7 * - 3.576 sec
20. Sleighride.yaa 56.89 % - 33.7 * - 5.310 sec

Compression: 47.99 %
Duration: 91.25 sec
Speed: 35.62 * real time

Fast :
01. Giant Steps.yaa 57.98 % - 33.0 * - 8.691 sec
01. La Vie, L'amour.yaa 47.94 % - 63.4 * - 1.746 sec
01. Le vieux Léon.yaa 37.05 % - 78.2 * - 2.901 sec
02. Parallel Universe.yaa 71.55 % - 74.9 * - 3.614 sec
02. Sa jeunesse.yaa 37.19 % - 72.5 * - 3.940 sec
03. Memories of You.yaa 32.63 % - 80.5 * - 2.391 sec
04. Carl Orff - I. Primo vere - Omnia sol temperat.yaa 39.80 % - 71.9 * - 1.503 sec
04. Celle qui a dit non.yaa 65.26 % - 36.6 * - 7.766 sec
04. Comme Moi (Like Me).yaa 43.35 % - 49.4 * - 3.629 sec
05. Le chemin de la dompe (présentation).yaa 44.07 % - 50.3 * - 1.265 sec
06. Frédéric.yaa 48.16 % - 65.1 * - 3.107 sec
11. Nil (Instrumental).yaa 20.58 % - 91.2 * - 1.504 sec
12. Sergei Sergeyevich Prokofiev - Dance of the knights.yaa 44.06 % - 76.7 * - 4.204 sec
15. Stroker Ace.yaa 54.67 % - 58.7 * - 4.590 sec
17. Carl Orff - III. Cour d'amours - Dies, nox et omnia.yaa 39.64 % - 85.3 * - 1.541 sec
20. Sleighride.yaa 57.32 % - 51.7 * - 3.466 sec

Compression: 48.31 %
Duration: 55.87 sec
Speed: 58.18 * real time

Fast + 16 preds :
01. Giant Steps.yaa 58.14 % - 31.8 * - 9.012 sec
01. La Vie, L'amour.yaa 48.36 % - 79.3 * - 1.395 sec
01. Le vieux Léon.yaa 37.22 % - 97.1 * - 2.337 sec
02. Parallel Universe.yaa 71.71 % - 77.1 * - 3.509 sec
02. Sa jeunesse.yaa 37.38 % - 99.6 * - 2.868 sec
03. Memories of You.yaa 32.76 % - 106.2 * - 1.812 sec
04. Carl Orff - I. Primo vere - Omnia sol temperat.yaa 40.01 % - 73.7 * - 1.465 sec
04. Celle qui a dit non.yaa 65.50 % - 41.1 * - 6.919 sec
04. Comme Moi (Like Me).yaa 43.59 % - 71.8 * - 2.497 sec
05. Le chemin de la dompe (présentation).yaa 44.38 % - 53.1 * - 1.199 sec
06. Frédéric.yaa 48.39 % - 61.8 * - 3.271 sec
11. Nil (Instrumental).yaa 20.62 % - 95.0 * - 1.443 sec
12. Sergei Sergeyevich Prokofiev - Dance of the knights.yaa 44.23 % - 82.6 * - 3.905 sec
15. Stroker Ace.yaa 54.76 % - 52.1 * - 5.171 sec
17. Carl Orff - III. Cour d'amours - Dies, nox et omnia.yaa 39.78 % - 93.1 * - 1.412 sec
20. Sleighride.yaa 57.76 % - 70.5 * - 2.539 sec

Compression: 48.50 %
Duration: 50.77 sec
Speed: 64.02 * real time

Fast + 16 preds + partition 32 :
01. Giant Steps.yaa 58.20 % - 32.4 * - 8.860 sec
01. La Vie, L'amour.yaa 48.40 % - 71.2 * - 1.554 sec
01. Le vieux Léon.yaa 37.31 % - 94.5 * - 2.401 sec
02. Parallel Universe.yaa 71.80 % - 85.5 * - 3.166 sec
02. Sa jeunesse.yaa 37.38 % - 72.6 * - 3.931 sec
03. Memories of You.yaa 32.78 % - 104.5 * - 1.842 sec
04. Carl Orff - I. Primo vere - Omnia sol temperat.yaa 40.01 % - 84.4 * - 1.280 sec
04. Celle qui a dit non.yaa 65.59 % - 46.2 * - 6.163 sec
04. Comme Moi (Like Me).yaa 43.59 % - 74.6 * - 2.405 sec
05. Le chemin de la dompe (présentation).yaa 44.44 % - 61.6 * - 1.033 sec
06. Frédéric.yaa 48.45 % - 63.5 * - 3.185 sec
11. Nil (Instrumental).yaa 20.62 % - 105.8 * - 1.297 sec
12. Sergei Sergeyevich Prokofiev - Dance of the knights.yaa 44.23 % - 71.6 * - 4.505 sec
15. Stroker Ace.yaa 54.86 % - 59.7 * - 4.516 sec
17. Carl Orff - III. Cour d'amours - Dies, nox et omnia.yaa 39.78 % - 97.7 * - 1.345 sec
20. Sleighride.yaa 57.79 % - 75.5 * - 2.372 sec

Compression: 48.55 %
Duration: 49.87 sec
Speed: 65.18 * real time

Insane :
01. Giant Steps.yaa 56.91 % - 4.4 * - 65.189 sec
01. La Vie, L'amour.yaa 47.30 % - 4.5 * - 24.585 sec
01. Le vieux Léon.yaa 36.27 % - 4.6 * - 49.639 sec
02. Parallel Universe.yaa 71.14 % - 4.6 * - 58.226 sec
02. Sa jeunesse.yaa 36.29 % - 4.4 * - 64.992 sec
03. Memories of You.yaa 32.06 % - 4.6 * - 41.468 sec
04. Carl Orff - I. Primo vere - Omnia sol temperat.yaa 39.14 % - 4.5 * - 23.817 sec
04. Celle qui a dit non.yaa 64.66 % - 4.5 * - 63.896 sec
04. Comme Moi (Like Me).yaa 42.84 % - 4.3 * - 41.489 sec
05. Le chemin de la dompe (présentation).yaa 43.64 % - 4.6 * - 13.877 sec
06. Frédéric.yaa 47.47 % - 4.4 * - 45.504 sec
11. Nil (Instrumental).yaa 19.99 % - 4.4 * - 31.310 sec
12. Sergei Sergeyevich Prokofiev - Dance of the knights.yaa 42.64 % - 4.2 * - 76.918 sec
15. Stroker Ace.yaa 54.26 % - 4.7 * - 57.914 sec
17. Carl Orff - III. Cour d'amours - Dies, nox et omnia.yaa 39.03 % - 4.4 * - 29.740 sec
20. Sleighride.yaa 56.17 % - 4.2 * - 42.857 sec

Compression: 47.55 %
Duration: 731.44 sec
Speed: 4.44 * real time
TBeck
May 31 2006, 03:56
QUOTE
' date='May 31 2006, 04:44' post='397815']
Default + prefilter

Compression: 47.88 %
Duration: 164.89 sec
Speed: 19.71 * real time

Default + no prefilter :

Compression: 47.85 %
Duration: 107.83 sec
Speed: 30.14 * real time


Wow, you are really fast! Many thanks!

Even if you are not providing good news: The prefilter hurts!

Sorry, it will take some time to evaluate the other results. It's a bit too late here...

Could you please try preset EXTRA with PreFilter on/off? No need to hurry...

Thomas
TBeck
May 31 2006, 04:47
QUOTE
' date='May 31 2006, 04:44' post='397815']
Output is disabled for all encoding settings; SSE is enabled, and all files are transcoded from LAME 3.98a3 -V2 --vbr-new. As such, this doesn't make for a _real-world_ test, but it's quite revelating nonetheless, on different settings, and the quality of the stereo decorellator (indeed better than LAME's)
...

Ok, had to take a closer look at it. Maybe more handy this way:
CODE
                    FAST    NORMAL
                            Prefilter
                            Off     On      SL Extr PR 32   64 P    
-------------------------------------------------------------------
01. Giant Steps.y    57.98   57.53   57.45   57.44   57.59   57.79  
01. La Vie, L'amo    47.94   47.54   47.70   47.48   47.57   47.68  
01. Le vieux Léon    37.05   36.61   36.69   36.56   36.70   36.65  
02. Parallel Univ    71.55   71.41   71.40   71.39   71.48   71.42  
02. Sa jeunesse.y    37.19   36.45   36.48   36.44   36.46   36.58  
03. Memories of Y    32.63   32.23   32.29   32.18   32.27   32.25  
04. Carl Orff - I    39.80   39.25   39.29   39.24   39.25   39.34  
04. Celle qui a d    65.26   65.08   65.00   65.04   65.16   65.11  
04. Comme Moi (Li    43.35   43.00   43.11   42.99   43.00   43.21  
05. Le chemin de     44.07   43.88   43.90   43.86   43.94   43.89  
06. Frédéric.yaa     48.16   47.78   47.80   47.77   47.84   47.86  
11. Nil (Instrume    20.58   20.04   20.22   19.88   20.04   19.96  
12. Sergei Sergey    44.06   43.07   43.06   43.05   43.08   43.56  
15. Stroker Ace.y    54.67   54.47   54.57   54.45   54.60   54.49  
17. Carl Orff - I    39.64   39.21   39.24   39.20   39.21   39.36  
20. Sleighride.ya    57.32   56.63   56.57   56.55   56.67   56.89  
-------------------------------------------------------------------
Sum:                 48.31   47.85   47.88   47.81   47.90   47.99  
EncoRate:            58.18   30.14   19.71   12.88   34.28   35.62  
-------------------------------------------------------------------

SL Extr = Frame partition search level Extra
PR 32   = Frame partition resolution 32
64 P    = 64 Predictors


First thoughts:

- The PreFilter hurts here.
- There is no big difference when using only 64 instead of the (default) 128 predictors. Possibly the PreFilter only helps files which are benefiting from higher predictor orders.
- The other variations are not really significant. The reduction of the frame partition resolution doesn't make the preset considerably faster and often the compression advantage of the higher default resolution is bigger than the 0.05 percent here. No urgent reason to change the default resolution.
- The variations (not in this table) of the FAST preset don't give a considerable speed up.

Addendum:

I nearly forgot about my previous post:

"Possibly it compensates some disturbances of the signal which are beeing caused by the scaling needed for my reduced precision (14-Bit) arithmetic. If that's true, then the PreFilter would only be useful for my specific implementation and not for other compressors which are using full scale arithmetics."

If this hypothesis is true, then files with high levels should benefit the most from the PreFilter.
TBeck
May 31 2006, 06:02
QUOTE
' date='May 31 2006, 04:44' post='397815']
Output is disabled for all encoding settings; SSE is enabled, and all files are transcoded from LAME 3.98a3 -V2 --vbr-new. As such, this doesn't make for a _real-world_ test, but it's quite revelating nonetheless, on different settings, and the quality of the stereo decorellator (indeed better than LAME's)

Which PreFilter level did you use, Fast or Normal? Could you please try the other one too?
Shade[ST]
May 31 2006, 11:56
Prefilter was at normal. I'll be posting the Extra / Prefilter soon.

Are fast and normal two completely different algorithms?

Also, maybe the prefilter doesn't always hurt, but in this case mp3 artifacts (which YALAC doesn't expect) make it so...

Peace,
Tristan.
TBeck
May 31 2006, 14:04
QUOTE
' date='May 31 2006, 12:56' post='397899']
Are fast and normal two completely different algorithms?

Also, maybe the prefilter doesn't always hurt, but in this case mp3 artifacts (which YALAC doesn't expect) make it so...

No, Normal only performs a more elaborated check, if the PreFilter would hurt (theoretically...). But both checks are very limited, because i didn't want to slow down encoding.

But...

I did ask Joseph Pohm for a quick check on his sample sets...

...and...

it seems as if there will be a need for some very nice modifications of his comparison!
Josef Pohm
May 31 2006, 18:01
QUOTE (TBeck @ May 31 2006, 15:04) *
I did ask Joseph Pohm for a quick check on his sample sets and it seems as if there will be a need for some very nice modifications of his comparison!


I'll do it very gladly. Here you have my first impression on Yalac v0.08 new feature, PreFilter. Preliminary results, only High mode tested, only compression ratio tested, there are no speed results here. I usually test Yalac on 4 sample set.

CODE
Set A - Primary Sample Set

Yalac v0.08      Preset High - PreFilter Off   57,69%
LPAC v1.40       ExtraHigh                     57,32%
Monkey v4.01     High                          57,31%
MP4ALS Garf      -a -o32                       57,25%
OptimFrog 4.520b Normal                        57,04%
Yalac v0.08      Preset High - PreFilter On    56,46%

Improvement 2,10%, Yalac surpasses all four competitors.

CODE
Set B - BackUp Sample Set

Yalac v0.08      Preset High - PreFilter Off   54,69%
Monkey v4.01     High                          54,00%
LPAC v1.40       ExtraHigh                     53,94%
OptimFrog 4.520b Normal                        53,73%
MP4ALS Garf      -a -o32                       53,54%
Yalac v0.08      Preset High - PreFilter On    52,78%

Improvement 3,49%, Yalac surpasses all four competitors.

CODE
Set C - Yalac-Unfriendly Sample Set

Yalac v0.08      Preset High - PreFilter Off   62,53%
OptimFrog 4.520b Normal                        61,78%
Monkey v4.01     High                          61,67%
LPAC v1.40       ExtraHigh                     60,68%
MP4ALS Garf      -a -o32                       60,46%
Yalac v0.08      Preset High - PreFilter On    60,43%

Improvement 3,35%, Yalac surpasses all four competitors. I'll have to change Sample Set nickname...

CODE
Set D - Yalac-Friendly Sample Set

MP4ALS Garf      -a -o32                       64,40%
LPAC v1.40       ExtraHigh                     64,32%
Monkey v4.01     High                          63,99%
Yalac v0.08      Preset High - PreFilter Off   63,81%
OptimFrog 4.520b Normal                        63,53%
Yalac v0.08      Preset High - PreFilter On    63,00%


Improvement 1,27%. With PreFilter Off Yalac is already better than three competitors out of four. With PreFilter On Yalac surpasses the fourth one as well, and gets very near to OptimFrog High (62,82%).

As a general rule, on all my four sample set, Yalac High compression ratio is now able to stand right between Monkey High and ExtraHigh, as well as between OptimFrog Normal and High. As Thomas pointed out, I can give a positive feedback (actually it's not really me, numbers speak for themselves).
Shade[ST]
Jun 1 2006, 03:52
Extra + Prefilter :
CODE
01. Giant Steps.yaa 56.95 % - 6.2 * - 46.555 sec
01. La Vie, L'amour.yaa 47.33 % - 6.5 * - 16.996 sec
01. Le vieux Léon.yaa 36.33 % - 6.7 * - 33.727 sec
02. Parallel Universe.yaa 71.18 % - 6.8 * - 39.617 sec
02. Sa jeunesse.yaa 36.30 % - 6.5 * - 44.245 sec
03. Memories of You.yaa 32.08 % - 6.9 * - 27.761 sec
04. Carl Orff - I. Primo vere - Omnia sol temperat.yaa 39.15 % - 6.6 * - 16.280 sec
04. Celle qui a dit non.yaa 64.70 % - 6.5 * - 43.630 sec
04. Comme Moi (Like Me).yaa 42.86 % - 6.3 * - 28.539 sec
05. Le chemin de la dompe (présentation).yaa 43.67 % - 6.8 * - 9.327 sec
06. Frédéric.yaa 47.50 % - 6.5 * - 31.049 sec
11. Nil (Instrumental).yaa 20.02 % - 6.5 * - 20.986 sec
12. Sergei Sergeyevich Prokofiev - Dance of the knights.yaa 42.66 % - 5.6 * - 57.195 sec
15. Stroker Ace.yaa 54.30 % - 6.4 * - 42.013 sec
17. Carl Orff - III. Cour d'amours - Dies, nox et omnia.yaa 39.04 % - 6.2 * - 21.053 sec
20. Sleighride.yaa 56.20 % - 5.9 * - 30.155 sec

Compression: 47.58 %
Duration: 509.14 sec
Speed: 6.38 * real time
Extra :
CODE
01. Giant Steps.yaa 57.18 % - 6.3 * - 45.765 sec
01. La Vie, L'amour.yaa 47.40 % - 7.1 * - 15.553 sec
01. Le vieux Léon.yaa 36.36 % - 7.4 * - 30.858 sec
02. Parallel Universe.yaa 71.22 % - 7.8 * - 34.675 sec
02. Sa jeunesse.yaa 36.31 % - 7.2 * - 39.927 sec
03. Memories of You.yaa 32.08 % - 7.8 * - 24.630 sec
04. Carl Orff - I. Primo vere - Omnia sol temperat.yaa 39.11 % - 7.5 * - 14.454 sec
04. Celle qui a dit non.yaa 64.80 % - 7.4 * - 38.459 sec
04. Comme Moi (Like Me).yaa 42.86 % - 7.0 * - 25.534 sec
05. Le chemin de la dompe (présentation).yaa 43.67 % - 7.6 * - 8.404 sec
06. Frédéric.yaa 47.62 % - 7.3 * - 27.794 sec
11. Nil (Instrumental).yaa 19.77 % - 7.4 * - 18.589 sec
12. Sergei Sergeyevich Prokofiev - Dance of the knights.yaa 42.82 % - 6.9 * - 47.060 sec
15. Stroker Ace.yaa 54.30 % - 7.6 * - 35.319 sec
17. Carl Orff - III. Cour d'amours - Dies, nox et omnia.yaa 39.03 % - 6.6 * - 19.822 sec
20. Sleighride.yaa 56.30 % - 6.8 * - 26.276 sec

Compression: 47.64 %
Duration: 453.14 sec
Speed: 7.17 * real time
Synthetic Soul
Jun 1 2006, 10:34
Thomas,

I will run the usual tests at home as always; however, as always, these will take me a few days to complete.

In the meantime I have been running a test here at work on the corpus I used for the FLAC testing I performed (28 files, with a better compression spread than my 50 file corpus I use for my Yalac testing). I can't provide timings, as I have been running the test in low priority. I need some CPU cycles to at least make it look like I'm working... wink.gif

Anyway, here are the compression results:

CODE
Setting    Compressed
=====================
Fast          58.653%
Normal        58.078%
High -c0      57.891%
High          57.638%
High -c2      57.631%
Extra -c0     57.848%
Extra -c1     57.590%
Extra         57.587%
Insane -c0    57.806%
Insane -c1    57.548%
Insane        57.545%

The spreadsheet, which also details the files in the corpus, can be found here. If a file in the corpus is of particular interest it may be worth looking at the overall results of my FLAC test to gain a further insight.

As soon as I have the full results for my corpus/PC at home I'll upload them, but that is likely to be tonight or sometime Friday.

I just thought I may as well make some use of this PC, even if it is only to provide compression values.
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2009 Invision Power Services, Inc.