Yalac - Comparisons, How the evaluation release compares to other compressors Options

[TBeck]

' date='May 2 2006, 05:40 PM' post='388534']

yalacc (filename | --all) [--decode] (-f | [-n] | -h | -i ) [--log]

The extra c on yalacc is for command-line (that way, gui-inclined people can still use the gui, if they want to)
Also, I suggest using "log" instead of "protocol" -- it's more relevant to the english language.

How comes you did steal my latest innovation: "yalacc" !?

The preset options will possibly get a bit modified.

What about overwriting files? Would the GUI-behaviour be ok (for the tests): Overwriting without warning but with added suffix "_d"?

[Shade[ST]]

Maybe add a target directory option, so that the decoded file is in another directory. This shouldn't be necessary, though : if filenames are passed correctly, they can be from another folder, and that will not pose a problem.

Are you seriously mad that I guessed what you would call your command-line YALAC? Sorry

[Destroid]

My $0.02 USD, I would like to avoid the wildcard function in the command-line version and focus on:

YALAC [-f(ast), -h(igh), -d(ecode) {default=normal}] infile [outfile]


The batch file I test with uses FOR loop, but one problem I ran into LA.EXE was not being able to control the output filename, so I had to resort to using a hack.

[Shade[ST]]

In that case, (infile | --all) (compression options) (logging on) -o (outputfile) would be best.

[TBeck]

My $0.02 USD, I would like to avoid the wildcard function in the command-line version and focus on:

YALAC [-f(ast), -h(igh), -d(ecode) {default=normal}] infile [outfile]


The batch file I test with uses FOR loop, but one problem I ran into LA.EXE was not being able to control the output filename, so I had to resort to using a hack.

Yes, because i did read your posts regarding batch processing i allready knew about the need to specify the outputfile.

Would it be ok for now, if the file extensions were fixed?

Would a simple wilcard option hurt?

My current spec looks like this:

CODE

YALACC infile [outfile] [-px -l {default=normal}]

infile might be:

Dir\*        (any file in DIR with the proper extension)
Dir\File     (File FILE in DIR
File         (File FILE in current dir)

outfile may only work with a single file spec.

-p specifies preset:

F or 0 = Fast
N or 1 = Normal (Default)
H or 2 = High
I or 3 = Insane

-l switches the log file generation on.

Quick and dirty...

[Destroid]

Would it be ok for now, if the file extensions were fixed?

Yes.

Would a simple wilcard option hurt?

Sure, I guess I imagined wildcard-handling being tedious to implement (for some programs).

The spec looks good. As mentioned, I use the batch redirect (>filename) to log filesize (ex. DIR __*.yaa) and operation time (TIMETHIS.EXE). Of course, it would be great to save a few minutes using the calculator if the ratio % and realtime x were output to CON and/or logfile.

This post has been edited by Destroid: May 3 2006, 02:00

[Synthetic Soul]

Quick and dirty...

Looks fine to me for using the presets.

I take it it will encode or decode depending on the extension of the source file? Edit: I guess wildcard method will have to default to one or the other (most likely encode), unless the format is actually "DIR\*.wav" or "DIR\*.yaa". (NB: I'm not sure what you guys mean by the file extension being "fixed" so I may be going over old ground here )

I don't see the need to specify a switch for the output filename with that setup.

What about overwriting files? Would the GUI-behaviour be ok (for the tests): Overwriting without warning but with added suffix "_d"?

My personal preference would be that "<name>.yaa" is decoded to "<name>.wav"; if "<name>.wav" exists a prompt appears; and a switch is provided to enable automatic overwriting. However, if we are just talking about a "Quick and dirty..." version for now then the current method is fine by me.

I've been away for a little while but I'm hoping to do some testing with 0.05 soon. I suppose it depends when yalacc.exe will be released; maybe I should wait until then.

This post has been edited by Synthetic Soul: May 3 2006, 10:05

[Synthetic Soul]

The spec looks good. As mentioned, I use the batch redirect (>filename) to log filesize (ex. DIR __*.yaa) and operation time (TIMETHIS.EXE). Of course, it would be great to save a few minutes using the calculator if the ratio % and realtime x were output to CON and/or logfile.

I use "%~z1" in the encoding batch file to output the new file's filesize to a log. Same idea.

The following VBS (Windows Script) file will look at any text files in the same directory for TimeThis' "Elapsed Time" entries and write them (in seconds format) all to a new text file (e.g.: "la.txt" > "la.txt.times.txt"). It's an easy way to scrape TimeThis times from a log created using a redirect. I'm not sure whether it may be of any use, but here it is anyway.

timethis.vbs:

CODE

Dim objFSO, objFolder, objFile
Dim objStream, objOutput, strProcessed

Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objFolder = objFSO.GetFolder("./")

For Each objFile in objFolder.Files
If GetExtension(objFile.Name) = ".txt" Then
Set objStream = objFile.OpenAsTextStream(1)
strProcessed = ProcessFile(objStream.ReadAll)
objStream.Close
If strProcessed > "" Then
Set objOutput = objFSO.OpenTextFile(objFile.Name & ".times.txt", 2, True)
objOutput.Write strProcessed
objOutput.Close
End If
End If
Next

Set objOutput = Nothing
Set objStream = Nothing
Set objFile = Nothing
Set objFSO = Nothing

Function ProcessFile(ByVal strString)
Dim objRegExp, objMatch, colMatches
Dim strLine, i, arrPart, sngSeconds
Set objRegExp = New RegExp
objRegExp.Pattern = "TimeThis : Elapsed Time : (\d+:\d+:\d+\.*\d*)"
objRegExp.IgnoreCase = True
objRegExp.Global = True
Set colMatches = objRegExp.Execute(strString)
For Each objMatch in colMatches
arrPart = Split(objMatch.SubMatches(0), ":")
sngSeconds = (arrPart(0) * 60 * 60) + (arrPart(1) * 60) + arrPart(2)
strLine = strLine & CStr(sngSeconds) & vbCrLf
Next
Set objRegExp = Nothing
ProcessFile = strLine
End Function

Function GetExtension(ByVal strFile)
GetExtension = LCase(Mid(strFile, InstrRev(strFile, ".")))
End Function

[TBeck]

...
I take it it will encode or decode depending on the extension of the source file? Edit: I guess wildcard method will have to default to one or the other (most likely encode), unless the format is actually "DIR\*.wav" or "DIR\*.yaa". (NB: I'm not sure what you guys mean by the file extension being "fixed" so I may be going over old ground here )
...
My personal preference would be that "<name>.yaa" is decoded to "<name>.wav"; if "<name>.wav" exists a prompt appears; and a switch is provided to enable automatic overwriting. However, if we are just talking about a "Quick and dirty..." version for now then the current method is fine by me.

I've been away for a little while but I'm hoping to do some testing with 0.05 soon. I suppose it depends when yalacc.exe will be released; maybe I should wait until then.

This specification is only a fast approach for the tests. It will change for the public release.

CODE

YALACC -mode infile [outfile] [-px -lx -overwrite]

-e          mode encode
-d          mode decode
infile      specify file(s) to be processed
  Dir\*       any file in DIR with the proper extension
  Dir\File    File FILE in DIR
  File        File FILE in current dir
outfile     specify output file or directory, see infile
-px         select encoder preset x
              F or 0 = Fast
              N or 1 = Normal (default)
              H or 2 = High
              I or 3 = Insane
-lx         select log file level x
              0 = no log file (default)
              1 = log compression results
              2 = log compression results and encoder diagnostics
-overwrite  overwrite existing wave files when decoding

In- and output files always have to contain the proper extension ".yaa" or ".wav". Yalac files will always be overwritten without warning. Wave files will be overwritten (again without warning), if the -overwrite switch has been specified. Otherwise the program will stop with an error message.

It's ok to wait until V0.06 for further testing. It's great, that destroid has tested V0.05 and that there were no unexpected results. That's enough for now.

[TBeck]

Current Progress (V0.06)

Sorry if i should talk to much...

Done:

- Preset FAST and NORMAL now are using Apply Window, which gives about 0.05 percent better compression (on my test corpus).
- Other slight improvements can give another 0.05 percent improvement to FAST and NORMAL on most files.
- Selection of Partition calculator resolution 64 gives another 0.05 percent improvement to FAST. It should now achieve the same compression as the (slow) FAST preset of V.04.
- Optimizations of the channel decorrelation make FAST 30 percent and NORMAL 10 percent faster (than V.0.05) on my system (caution: maybe CPU-dependent).
- Command line version is done. In addition to my last spec it let's you overwrite the predictor order of the presets.

To do (for V.0.06):

- More speed optimizations of the channel decorrelation.
- Optimization of the compression efficiency of the channel decorrelation. Some files give 0.50 percent better compression in my evaluation. Average may be about 0.20 (dont know yet).


Maybe i will cancel the optimizations of the channel decorrelation for now and release V0.06 this weekend.


Thomas

[Shade[ST]]

Take your time if you should wish to, but I don't think our testers will mind an intermediary release ;-)

@Synthetic soul : You'll probably see this, so once we get the command-line encoder, could you make a package with batch files and batch file processors to process the logs / data? Then, we could all send our results to you and you could sort them in your database...

That would be really great!


Peace,
Tristan.

[TBeck]

' date='May 5 2006, 02:22 PM' post='389567']
Take your time if you should wish to, but I don't think our testers will mind an intermediary release ;-)

@Synthetic soul : You'll probably see this, so once we get the command-line encoder, could you make a package with batch files and batch file processors to process the logs / data? Then, we could all send our results to you and you could sort them in your database...

I didn't want to stress the testers too much... For me it would be better to test an intermediary release. If some release shouldn't perform well, it is easier to find the reason, if not too much had been changed at once.

I could sent Synthetic soul soon a possibly buggy pre release to check the command line interface with his scripts. It's only an option.

[Synthetic Soul]

I'm not sure what the priority is for testing 0.06. Would we be comparing against other encoders, or is the idea that we simply run Yalac on numerous files using all its presets?

I'm quite happy to pass some batch files on if it will help people test, but I'm not sure the database I used in my initial test is adequate for this round of testing.

The way I see it, if we are simply testing how Yalac performs, we need to test a greater variety of files (mono; 24bit; etc). The database is only really set up to deal with 44.1KHz 16bit stereo files. If you consider what my pages detailed, it would really just be a table to show that fast is faster than normal, and high compresses better than normal. We know that already (well, we can assume).

Don't get me wrong, I think amalgamating the results is a great idea. Joseph Pohm and I have communicated regarding a similar thing. I think I'm being a little short-sighted and just need some help to understand how and what we would all be testing.

[TBeck]

I'm not sure what the priority is for testing 0.06. Would we be comparing against other encoders, or is the idea that we simply run Yalac on numerous files using all its presets?
...
The way I see it, if we are simply testing how Yalac performs, we need to test a greater variety of files (mono; 24bit; etc). The database is only really set up to deal with 44.1KHz 16bit stereo files. If you consider
...
Don't get me wrong, I think amalgamating the results is a great idea. Joseph Pohm and I have communicated regarding a similar thing. I think I'm being a little short-sighted and just need some help to understand how and what we would all be testing.

To be honest, i don't have an elaborated plan for test design beyond my current interests.

I did work on

- mainly speed optimizations
- improvements of my source code quality
- some slight compression improvements (found some opportunities while evaluating my source code).

Hence current tests should check

a) if the speed optimizations are present on different platforms (especially CPU's)
b) if the speed and source code optimizations didn't affect the reliability
c) if the changes made for the slight compression improvements don't hurt on other files.

For this purposes an evaluation based upon the existing test sets and a comparison with previous results would be quite helpful. For a) and b) the tests sets don't have to be really representative.

Most work for V0.07 will probably again consist of speed optimizations of the channel decorrelation. V0.06 brings algorithmic, V0.07 will bring assembler optimizations.

Reason: I want to improve the channel decorrelation in terms of compression efficiency. The current speed optimizations should guarantee, that later increases of the algorithm complexity will not hurt speed too much. And i really like Yalac to be fast...

V0.08 should contain an improved channel decorrelation. Then it will be really useful to test it on heterogenous file sets to tweak it. Mono and 24-Bit files wouldn't be important here.

Besides testing for my development purposes i am surely interested into comparisons with other compressors! Why else should i look for some tenth of a percent better compression...

I hope this shows some direction for possible further testing.

Thomas

P.S.: Some time in the future i will activate my ominous pre filter, which gives up to 4 percent better compression on some rare problem files! But this one needs really much evaluation on big test sets.

This post has been edited by TBeck: May 5 2006, 22:24

[Synthetic Soul]

Hence current tests should check

a) if the speed optimizations are present on different platforms (especially CPU's)
b) if the speed and source code optimizations didn't affect the reliability
c) if the changes made for the slight compression improvements don't hurt on other files.
...
Besides testing for my development purposes i am surely interested into comparisons with other compressors! Why else should i look for some tenth of a percent better compression...

I hope this shows some direction for possible further testing.

Yes, thank you. So in essence we should be testing Yalac 0.06 against previous Yalac tests, to ensure that speed and compression have improved. Also, we should ensure that decompressed files are not corrupt. We can compare against other codecs using the new Yalac results and the previous results for the other codecs (if we are testing on the same corpus).

With this in mind I don't see how all results can be amalgamated into my system, unless testers are prepared to format their previous results into something I can easily import.

[Shade[ST]]

@Synthetic soul : If you make your batch files available, or give us VBScripts or whatever, the few testers that we are can convert our data to your format, and send it to you! It's just a problem of whether you can do something with the data afterwards.

[TBeck]

Most work for V0.07 will probably again consist of speed optimizations of the channel decorrelation. V0.06 brings algorithmic, V0.07 will bring assembler optimizations.

No, V0.06 will bring both. That's the reason, why i didn't send V0.06 to the testers last weekend.

By the way: Preset FAST is now 60 percent faster than V0.05 on my system! This may be the limit, please don't expect more optimizations here. And we have to try, if this improvement will show up on other systems to.

Thomas

[TBeck]

V0.06 is done

Changes:

Compression efficiency and presets:

- Preset FAST and NORMAL now are using Apply Window, which gives about 0.05 percent better compression (on my test corpus).
- Other slight improvements can give another 0.05 percent improvement to FAST and NORMAL on most files.
- Selection of Partition calculator resolution 128 gives another 0.05 percent improvement to FAST. It should now achieve the same compression as the (slow) FAST preset of V.04.

Speed:

- Algorithmic and assembler optimizations of the channel decorrelation.
- Rework of my basic signal processing functions. Dangerous: I didn't touch them for years and i did know why!
- Because my compiler doesn't support proper code alignment, i had to do some strange things to help him out. I hope that there will be no trouble on operating systems with more strict security checks.
- Implementation of a memory manager to avoid cache trashing caused by too many simultaneous uses of the same cache line set.
- Rework of my File-IO. To my surprise this has been most important for the speedup of decoding. Possibly i will have to do some adaptions for specific operating systems.
- Changed floating point precision from double to single. Can sometimes reduce compression efficiency by 0.01 to 0.02 percent.

Probably other systems will benefit far less from my optimizations than my good old pentium 3. Maybe that the optimizations are compensating weaknesses of my CPU that are not present on modern CPUs. Nevertheless i want to provide some data as a baseline:

Speed up of encoding:

Preset FAST 55 %
Preset NORMAL 20 %
Preset HIGH 10 %

Speed up of decoding:

Preset FAST 25 %

Command line:

Command line version is done. Unfortunately the binary is nearly as large as the GUI-Version. Delphi seems to put most of the basic GUI libraries into the binary of a console application, if the code uses exception handling. The initialization of the program may take considerably longer than without the GUI libraries. The compression speed of small audio files may suffer from this overhead.

In addition to my latest specification there is a new command line switch:

-cx Evaluate test ©ase x.

If i will ask the testers to evaluate the effect of internal encoder parameter variations, they can switch between them via this option. For instance: Please try -c1 to -c5.

Other:

- Clean up of source code. Could give new errors...
- File format changed!

Release:

I hope to send the new version to the following testers within the next 24 hours:

Destroid
Josef Pohm
Shade[ST]
Synthetic Soul

Only reason for this selection: I haven't heard from the other testers within the last 10 days and i don't want to fill their mail box with new versions they possibly currently don't need.

Any of them can sent me an email anytime, and i will sent the current version.

What should be tested:

Comparison with V0.05:

- The speed and compression performance of presets FAST, NORMAL and HIGH.
- The decoding speed.

Evaluation of internal encoder options:

I wrote a SSE-Implementation of the Levinson-Durbin-algorithm used for the calculation of the predictor coefficients. It is disabled by default.

You can activate it via command line option -c1 and check it with Preset HIGH. There is definitely no reason to check it with other presets!

On my system this option gives only 1 - 2 % more speed on HIGH. I would like to know, if other systems are getting more out of it, before i will try more SSE optimizations.

Plans for V 0.07:

- Improvement of the compression efficiency of the channel decorrelation.


Thomas

This post has been edited by TBeck: May 12 2006, 17:04

[Shade[ST]]

My exams are now done, I'd be glad to test the encoder on my system.

Could synthetic soul provide batch files to test?

Thanks,

Tristan

[Shade[ST]]

On SSE optimization :

CODE

=== New test ===============================================

Linear Predictor
Predictors: 32
Optimize quantization: Nein
Apply window: Ja

Frames
Duration: 100 ms

Channel decorrelation
Enable: Ja
Full search: Nein

Frame partition calculator
Resolution: 64
Search level: 1 (0 - 3)

Bit coder
Optimize Choice: Nein
Optimize Partition: Nein

Diagnostics
Verify: Nein
No output: Nein
Use MMX: Ja
Use SSE: Ja


=== Bitcoder ===============================================

Resolution
N: 0


=== LPC Predictors =========================================

Number
4 4.31 %
8 9.88 %
12 19.68 %
16 22.57 %
24 17.48 %
32 26.08 %
48 0.00 %
64 0.00 %
80 0.00 %
96 0.00 %
112 0.00 %
128 0.00 %
160 0.00 %
192 0.00 %
224 0.00 %
256 0.00 %
Mean: 19.48 *

Resolution
N: 115927
Minimum: 5.00
Mittelwert: 8.96
Maximum: 11.00
Verteilung
5 0.001 % (1)
6 0.203 % (235)
7 4.246 % (4922)
8 17.197 % (19936)
9 56.678 % (65705)
10 20.745 % (24049)
11 0.931 % (1079)
12 0.000 % ()
13 0.000 % ()
14 0.000 % ()
> 0.000 % ()

ShiftNum (Resolution reduction)
N: 115927
Minimum: 3.00
Mittelwert: 3.00
Maximum: 3.00
Verteilung
0 0.000 % ()
1 0.000 % ()
2 0.000 % ()
3 100.000 % (115927)
4 0.000 % ()
5 0.000 % ()
6 0.000 % ()
7 0.000 % ()
> 0.000 % ()

Resolution / Predictor number Min Mean Max StdDev
4 6.00 8.97 11.00 0.67 Bit
8 5.00 8.69 11.00 0.88 Bit
12 6.00 9.01 11.00 0.73 Bit
16 6.00 9.08 11.00 0.73 Bit
24 6.00 8.97 11.00 0.85 Bit
32 6.00 8.93 11.00 0.75 Bit
48 0.00 0.00 0.00 0.00 Bit
64 0.00 0.00 0.00 0.00 Bit
80 0.00 0.00 0.00 0.00 Bit
96 0.00 0.00 0.00 0.00 Bit
112 0.00 0.00 0.00 0.00 Bit
128 0.00 0.00 0.00 0.00 Bit
160 0.00 0.00 0.00 0.00 Bit
192 0.00 0.00 0.00 0.00 Bit
224 0.00 0.00 0.00 0.00 Bit
256 0.00 0.00 0.00 0.00 Bit

ShiftNum / Predictor number Min Mean Max StdDev
4 3.00 3.00 3.00 0.00 Bit
8 3.00 3.00 3.00 0.00 Bit
12 3.00 3.00 3.00 0.00 Bit
16 3.00 3.00 3.00 0.00 Bit
24 3.00 3.00 3.00 0.00 Bit
32 3.00 3.00 3.00 0.00 Bit
48 0.00 0.00 0.00 0.00 Bit
64 0.00 0.00 0.00 0.00 Bit
80 0.00 0.00 0.00 0.00 Bit
96 0.00 0.00 0.00 0.00 Bit
112 0.00 0.00 0.00 0.00 Bit
128 0.00 0.00 0.00 0.00 Bit
160 0.00 0.00 0.00 0.00 Bit
192 0.00 0.00 0.00 0.00 Bit
224 0.00 0.00 0.00 0.00 Bit
256 0.00 0.00 0.00 0.00 Bit

Bits for whole predictor set Min Mean Max StdDev
4 33 45 53 3 Bit
8 49 76 97 7 Bit
12 74 110 141 8 Bit
16 98 144 181 13 Bit
24 139 196 271 21 Bit
32 182 245 334 23 Bit
48 0 0 0 0 Bit
64 0 0 0 0 Bit
80 0 0 0 0 Bit
96 0 0 0 0 Bit
112 0 0 0 0 Bit
128 0 0 0 0 Bit
160 0 0 0 0 Bit
192 0 0 0 0 Bit
224 0 0 0 0 Bit
256 0 0 0 0 Bit

Bits per predictor Min Mean Max StdDev
4 8.25 11.15 13.25 0.71 Bit
8 6.13 9.52 12.13 0.83 Bit
12 6.17 9.19 11.75 0.71 Bit
16 6.13 9.01 11.31 0.84 Bit
24 5.79 8.15 11.29 0.87 Bit
32 5.69 7.66 10.44 0.73 Bit
48 0.00 0.00 0.00 0.00 Bit
64 0.00 0.00 0.00 0.00 Bit
80 0.00 0.00 0.00 0.00 Bit
96 0.00 0.00 0.00 0.00 Bit
112 0.00 0.00 0.00 0.00 Bit
128 0.00 0.00 0.00 0.00 Bit
160 0.00 0.00 0.00 0.00 Bit
192 0.00 0.00 0.00 0.00 Bit
224 0.00 0.00 0.00 0.00 Bit
256 0.00 0.00 0.00 0.00 Bit

Absolute Sum of coefficients Min Mean Max StdDev
4 8.67 11.62 13.51 0.77 Bit
8 9.09 11.91 14.36 0.88 Bit
12 9.75 12.65 14.85 0.79 Bit
16 10.04 13.13 15.29 0.95 Bit
24 10.44 13.02 16.03 0.98 Bit
32 10.60 13.06 15.81 0.83 Bit
48 0.00 0.00 0.00 0.00 Bit
64 0.00 0.00 0.00 0.00 Bit
80 0.00 0.00 0.00 0.00 Bit
96 0.00 0.00 0.00 0.00 Bit
112 0.00 0.00 0.00 0.00 Bit
128 0.00 0.00 0.00 0.00 Bit
160 0.00 0.00 0.00 0.00 Bit
192 0.00 0.00 0.00 0.00 Bit
224 0.00 0.00 0.00 0.00 Bit
256 0.00 0.00 0.00 0.00 Bit

Absolute Sum of shifted coefficients Min Mean Max StdDev
4 5.67 8.62 10.51 0.77 Bit
8 6.09 8.91 11.36 0.88 Bit
12 6.75 9.65 11.85 0.79 Bit
16 7.04 10.13 12.29 0.95 Bit
24 7.44 10.02 13.03 0.98 Bit
32 7.60 10.06 12.81 0.83 Bit
48 0.00 0.00 0.00 0.00 Bit
64 0.00 0.00 0.00 0.00 Bit
80 0.00 0.00 0.00 0.00 Bit
96 0.00 0.00 0.00 0.00 Bit
112 0.00 0.00 0.00 0.00 Bit
128 0.00 0.00 0.00 0.00 Bit
160 0.00 0.00 0.00 0.00 Bit
192 0.00 0.00 0.00 0.00 Bit
224 0.00 0.00 0.00 0.00 Bit
256 0.00 0.00 0.00 0.00 Bit


=== Frame partition ========================================

SubFrameNum
N: 72252
Minimum: 1.00
Mittelwert: 1.60
Maximum: 3.00
Verteilung
1 61.543 % (44466)
2 16.442 % (11880)
3 22.015 % (15906)
4 0.000 % ()
5 0.000 % ()
> 0.000 % ()


=== Joint Stereo ===========================================

Modes
N: 36126
Minimum: 0.00
Mittelwert: 1.25
Maximum: 2.00
Verteilung
0 21.641 % (7818)
1 31.288 % (11303)
2 47.071 % (17005)
> 0.000 % ()


=== Results ================================================

01. American Idiot.yaa
64.18 % - 29.1 * - 5.987 sec
02. Carl Orff - Fortune plango vulnera.yaa
48.95 % - 41.5 * - 3.855 sec
02. Carl Orff - Veris leta facies; Omnia Sol temperat.yaa
31.18 % - 61.1 * - 5.622 sec
02. Parallel Universe.yaa
59.39 % - 48.8 * - 5.548 sec
02. Wolfgang Amadeus Mozart - O Isis und Osiris (Sarastro - Chor).yaa
41.39 % - 53.2 * - 3.700 sec
03. Who's sorry now.yaa
27.95 % - 48.1 * - 3.782 sec
04. Burke - Johnston - Pennies From Heaven.yaa
55.58 % - 44.5 * - 13.726 sec
04. Carl Orff - I. Primo vere - Omnia sol temperat.yaa
39.80 % - 54.0 * - 2.002 sec
04. Japanese sandman.yaa
22.89 % - 49.2 * - 4.314 sec
05. Koop - Summer Sun.yaa
64.10 % - 17.1 * - 13.122 sec
05. Wolfgang Amadeus Mozart - Rex tremendae majestatis.yaa
44.97 % - 28.3 * - 4.970 sec
06. The Subject Was Faggots.yaa
44.04 % - 25.7 * - 7.470 sec
08. Morgana King - It's De-lovely.yaa
36.95 % - 38.2 * - 3.521 sec
08. Wolfgang Amadeus Mozart - Requiem in D Minor, KV 626, VIII. 08.
Wolfgang Amadeus Mozart - Requiem in D Minor, KV 626, VIII.
Lacrimosa.yaa 40.82 % - 31.5 * - 7.984 sec
11. Nil (Instrumental).yaa
20.58 % - 40.7 * - 3.367 sec
11. Throb.yaa
54.05 % - 45.4 * - 6.019 sec

Compression: 45.44 %
Duration: 95.03 sec
Speed: 38.01 * real time


=== New test ===============================================

Linear Predictor
Predictors: 32
Optimize quantization: Nein
Apply window: Ja

Frames
Duration: 100 ms

Channel decorrelation
Enable: Ja
Full search: Nein

Frame partition calculator
Resolution: 64
Search level: 1 (0 - 3)

Bit coder
Optimize Choice: Nein
Optimize Partition: Nein

Diagnostics
Verify: Nein
No output: Nein
Use MMX: Ja
Use SSE: Nein


=== Bitcoder ===============================================

Resolution
N: 0


=== LPC Predictors =========================================

Number
4 4.31 %
8 9.88 %
12 19.68 %
16 22.57 %
24 17.48 %
32 26.08 %
48 0.00 %
64 0.00 %
80 0.00 %
96 0.00 %
112 0.00 %
128 0.00 %
160 0.00 %
192 0.00 %
224 0.00 %
256 0.00 %
Mean: 19.48 *

Resolution
N: 115927
Minimum: 5.00
Mittelwert: 8.96
Maximum: 11.00
Verteilung
5 0.001 % (1)
6 0.203 % (235)
7 4.246 % (4922)
8 17.197 % (19936)
9 56.678 % (65705)
10 20.745 % (24049)
11 0.931 % (1079)
12 0.000 % ()
13 0.000 % ()
14 0.000 % ()
> 0.000 % ()

ShiftNum (Resolution reduction)
N: 115927
Minimum: 3.00
Mittelwert: 3.00
Maximum: 3.00
Verteilung
0 0.000 % ()
1 0.000 % ()
2 0.000 % ()
3 100.000 % (115927)
4 0.000 % ()
5 0.000 % ()
6 0.000 % ()
7 0.000 % ()
> 0.000 % ()

Resolution / Predictor number Min Mean Max StdDev
4 6.00 8.97 11.00 0.67 Bit
8 5.00 8.69 11.00 0.88 Bit
12 6.00 9.01 11.00 0.73 Bit
16 6.00 9.08 11.00 0.73 Bit
24 6.00 8.97 11.00 0.85 Bit
32 6.00 8.93 11.00 0.75 Bit
48 0.00 0.00 0.00 0.00 Bit
64 0.00 0.00 0.00 0.00 Bit
80 0.00 0.00 0.00 0.00 Bit
96 0.00 0.00 0.00 0.00 Bit
112 0.00 0.00 0.00 0.00 Bit
128 0.00 0.00 0.00 0.00 Bit
160 0.00 0.00 0.00 0.00 Bit
192 0.00 0.00 0.00 0.00 Bit
224 0.00 0.00 0.00 0.00 Bit
256 0.00 0.00 0.00 0.00 Bit

ShiftNum / Predictor number Min Mean Max StdDev
4 3.00 3.00 3.00 0.00 Bit
8 3.00 3.00 3.00 0.00 Bit
12 3.00 3.00 3.00 0.00 Bit
16 3.00 3.00 3.00 0.00 Bit
24 3.00 3.00 3.00 0.00 Bit
32 3.00 3.00 3.00 0.00 Bit
48 0.00 0.00 0.00 0.00 Bit
64 0.00 0.00 0.00 0.00 Bit
80 0.00 0.00 0.00 0.00 Bit
96 0.00 0.00 0.00 0.00 Bit
112 0.00 0.00 0.00 0.00 Bit
128 0.00 0.00 0.00 0.00 Bit
160 0.00 0.00 0.00 0.00 Bit
192 0.00 0.00 0.00 0.00 Bit
224 0.00 0.00 0.00 0.00 Bit
256 0.00 0.00 0.00 0.00 Bit

Bits for whole predictor set Min Mean Max StdDev
4 33 45 53 3 Bit
8 49 76 97 7 Bit
12 74 110 141 8 Bit
16 98 144 181 13 Bit
24 139 196 271 21 Bit
32 182 245 334 23 Bit
48 0 0 0 0 Bit
64 0 0 0 0 Bit
80 0 0 0 0 Bit
96 0 0 0 0 Bit
112 0 0 0 0 Bit
128 0 0 0 0 Bit
160 0 0 0 0 Bit
192 0 0 0 0 Bit
224 0 0 0 0 Bit
256 0 0 0 0 Bit

Bits per predictor Min Mean Max StdDev
4 8.25 11.15 13.25 0.71 Bit
8 6.13 9.52 12.13 0.83 Bit
12 6.17 9.19 11.75 0.71 Bit
16 6.13 9.01 11.31 0.84 Bit
24 5.79 8.15 11.29 0.87 Bit
32 5.69 7.66 10.44 0.73 Bit
48 0.00 0.00 0.00 0.00 Bit
64 0.00 0.00 0.00 0.00 Bit
80 0.00 0.00 0.00 0.00 Bit
96 0.00 0.00 0.00 0.00 Bit
112 0.00 0.00 0.00 0.00 Bit
128 0.00 0.00 0.00 0.00 Bit
160 0.00 0.00 0.00 0.00 Bit
192 0.00 0.00 0.00 0.00 Bit
224 0.00 0.00 0.00 0.00 Bit
256 0.00 0.00 0.00 0.00 Bit

Absolute Sum of coefficients Min Mean Max StdDev
4 8.67 11.62 13.51 0.77 Bit
8 9.09 11.91 14.36 0.88 Bit
12 9.75 12.65 14.85 0.79 Bit
16 10.04 13.13 15.29 0.95 Bit
24 10.44 13.02 16.03 0.98 Bit
32 10.60 13.06 15.81 0.83 Bit
48 0.00 0.00 0.00 0.00 Bit
64 0.00 0.00 0.00 0.00 Bit
80 0.00 0.00 0.00 0.00 Bit
96 0.00 0.00 0.00 0.00 Bit
112 0.00 0.00 0.00 0.00 Bit
128 0.00 0.00 0.00 0.00 Bit
160 0.00 0.00 0.00 0.00 Bit
192 0.00 0.00 0.00 0.00 Bit
224 0.00 0.00 0.00 0.00 Bit
256 0.00 0.00 0.00 0.00 Bit

Absolute Sum of shifted coefficients Min Mean Max StdDev
4 5.67 8.62 10.51 0.77 Bit
8 6.09 8.91 11.36 0.88 Bit
12 6.75 9.65 11.85 0.79 Bit
16 7.04 10.13 12.29 0.95 Bit
24 7.44 10.02 13.03 0.98 Bit
32 7.60 10.06 12.81 0.83 Bit
48 0.00 0.00 0.00 0.00 Bit
64 0.00 0.00 0.00 0.00 Bit
80 0.00 0.00 0.00 0.00 Bit
96 0.00 0.00 0.00 0.00 Bit
112 0.00 0.00 0.00 0.00 Bit
128 0.00 0.00 0.00 0.00 Bit
160 0.00 0.00 0.00 0.00 Bit
192 0.00 0.00 0.00 0.00 Bit
224 0.00 0.00 0.00 0.00 Bit
256 0.00 0.00 0.00 0.00 Bit


=== Frame partition ========================================

SubFrameNum
N: 72252
Minimum: 1.00
Mittelwert: 1.60
Maximum: 3.00
Verteilung
1 61.543 % (44466)
2 16.442 % (11880)
3 22.015 % (15906)
4 0.000 % ()
5 0.000 % ()
> 0.000 % ()


=== Joint Stereo ===========================================

Modes
N: 36126
Minimum: 0.00
Mittelwert: 1.25
Maximum: 2.00
Verteilung
0 21.641 % (7818)
1 31.288 % (11303)
2 47.071 % (17005)
> 0.000 % ()


=== Results ================================================

01. American Idiot.yaa
64.18 % - 39.6 * - 4.403 sec
02. Carl Orff - Fortune plango vulnera.yaa
48.95 % - 34.5 * - 4.641 sec
02. Carl Orff - Veris leta facies; Omnia Sol temperat.yaa
31.18 % - 31.3 * - 10.980 sec
02. Parallel Universe.yaa
59.39 % - 43.7 * - 6.196 sec
02. Wolfgang Amadeus Mozart - O Isis und Osiris (Sarastro - Chor).yaa
41.39 % - 53.7 * - 3.663 sec
03. Who's sorry now.yaa
27.95 % - 60.7 * - 2.996 sec
04. Burke - Johnston - Pennies From Heaven.yaa
55.58 % - 50.5 * - 12.088 sec
04. Carl Orff - I. Primo vere - Omnia sol temperat.yaa
39.80 % - 58.9 * - 1.834 sec
04. Japanese sandman.yaa
22.89 % - 62.3 * - 3.409 sec
05. Koop - Summer Sun.yaa
64.10 % - 38.5 * - 5.836 sec
05. Wolfgang Amadeus Mozart - Rex tremendae majestatis.yaa
44.97 % - 42.3 * - 3.321 sec
06. The Subject Was Faggots.yaa
44.04 % - 50.0 * - 3.836 sec
08. Morgana King - It's De-lovely.yaa
36.95 % - 51.3 * - 2.625 sec
08. Wolfgang Amadeus Mozart - Requiem in D Minor, KV 626, VIII. 08.
Wolfgang Amadeus Mozart - Requiem in D Minor, KV 626, VIII.
Lacrimosa.yaa 40.82 % - 41.7 * - 6.033 sec
11. Nil (Instrumental).yaa
20.58 % - 58.0 * - 2.366 sec
11. Throb.yaa
54.05 % - 44.6 * - 6.136 sec

Compression: 45.44 %
Duration: 80.38 sec
Speed: 44.93 * real time


=== New test ===============================================

Linear Predictor
Predictors: 32
Optimize quantization: Nein
Apply window: Ja

Frames
Duration: 100 ms

Channel decorrelation
Enable: Ja
Full search: Nein

Frame partition calculator
Resolution: 64
Search level: 1 (0 - 3)

Bit coder
Optimize Choice: Nein
Optimize Partition: Nein

Diagnostics
Verify: Nein
No output: Nein
Use MMX: Ja
Use SSE: Ja


=== Bitcoder ===============================================

Resolution
N: 0


=== LPC Predictors =========================================

Number
4 4.31 %
8 9.88 %
12 19.68 %
16 22.57 %
24 17.48 %
32 26.08 %
48 0.00 %
64 0.00 %
80 0.00 %
96 0.00 %
112 0.00 %
128 0.00 %
160 0.00 %
192 0.00 %
224 0.00 %
256 0.00 %
Mean: 19.48 *

Resolution
N: 115927
Minimum: 5.00
Mittelwert: 8.96
Maximum: 11.00
Verteilung
5 0.001 % (1)
6 0.203 % (235)
7 4.246 % (4922)
8 17.197 % (19936)
9 56.678 % (65705)
10 20.745 % (24049)
11 0.931 % (1079)
12 0.000 % ()
13 0.000 % ()
14 0.000 % ()
> 0.000 % ()

ShiftNum (Resolution reduction)
N: 115927
Minimum: 3.00
Mittelwert: 3.00
Maximum: 3.00
Verteilung
0 0.000 % ()
1 0.000 % ()
2 0.000 % ()
3 100.000 % (115927)
4 0.000 % ()
5 0.000 % ()
6 0.000 % ()
7 0.000 % ()
> 0.000 % ()

Resolution / Predictor number Min Mean Max StdDev
4 6.00 8.97 11.00 0.67 Bit
8 5.00 8.69 11.00 0.88 Bit
12 6.00 9.01 11.00 0.73 Bit
16 6.00 9.08 11.00 0.73 Bit
24 6.00 8.97 11.00 0.85 Bit
32 6.00 8.93 11.00 0.75 Bit
48 0.00 0.00 0.00 0.00 Bit
64 0.00 0.00 0.00 0.00 Bit
80 0.00 0.00 0.00 0.00 Bit
96 0.00 0.00 0.00 0.00 Bit
112 0.00 0.00 0.00 0.00 Bit
128 0.00 0.00 0.00 0.00 Bit
160 0.00 0.00 0.00 0.00 Bit
192 0.00 0.00 0.00 0.00 Bit
224 0.00 0.00 0.00 0.00 Bit
256 0.00 0.00 0.00 0.00 Bit

ShiftNum / Predictor number Min Mean Max StdDev
4 3.00 3.00 3.00 0.00 Bit
8 3.00 3.00 3.00 0.00 Bit
12 3.00 3.00 3.00 0.00 Bit
16 3.00 3.00 3.00 0.00 Bit
24 3.00 3.00 3.00 0.00 Bit
32 3.00 3.00 3.00 0.00 Bit
48 0.00 0.00 0.00 0.00 Bit
64 0.00 0.00 0.00 0.00 Bit
80 0.00 0.00 0.00 0.00 Bit
96 0.00 0.00 0.00 0.00 Bit
112 0.00 0.00 0.00 0.00 Bit
128 0.00 0.00 0.00 0.00 Bit
160 0.00 0.00 0.00 0.00 Bit
192 0.00 0.00 0.00 0.00 Bit
224 0.00 0.00 0.00 0.00 Bit
256 0.00 0.00 0.00 0.00 Bit

Bits for whole predictor set Min Mean Max StdDev
4 33 45 53 3 Bit
8 49 76 97 7 Bit
12 74 110 141 8 Bit
16 98 144 181 13 Bit
24 139 196 271 21 Bit
32 182 245 334 23 Bit
48 0 0 0 0 Bit
64 0 0 0 0 Bit
80 0 0 0 0 Bit
96 0 0 0 0 Bit
112 0 0 0 0 Bit
128 0 0 0 0 Bit
160 0 0 0 0 Bit
192 0 0 0 0 Bit
224 0 0 0 0 Bit
256 0 0 0 0 Bit

Bits per predictor Min Mean Max StdDev
4 8.25 11.15 13.25 0.71 Bit
8 6.13 9.52 12.13 0.83 Bit
12 6.17 9.19 11.75 0.71 Bit
16 6.13 9.01 11.31 0.84 Bit
24 5.79 8.15 11.29 0.87 Bit
32 5.69 7.66 10.44 0.73 Bit
48 0.00 0.00 0.00 0.00 Bit
64 0.00 0.00 0.00 0.00 Bit
80 0.00 0.00 0.00 0.00 Bit
96 0.00 0.00 0.00 0.00 Bit
112 0.00 0.00 0.00 0.00 Bit
128 0.00 0.00 0.00 0.00 Bit
160 0.00 0.00 0.00 0.00 Bit
192 0.00 0.00 0.00 0.00 Bit
224 0.00 0.00 0.00 0.00 Bit
256 0.00 0.00 0.00 0.00 Bit

Absolute Sum of coefficients Min Mean Max StdDev
4 8.67 11.62 13.51 0.77 Bit
8 9.09 11.91 14.36 0.88 Bit
12 9.75 12.65 14.85 0.79 Bit
16 10.04 13.13 15.29 0.95 Bit
24 10.44 13.02 16.03 0.98 Bit
32 10.60 13.06 15.81 0.83 Bit
48 0.00 0.00 0.00 0.00 Bit
64 0.00 0.00 0.00 0.00 Bit
80 0.00 0.00 0.00 0.00 Bit
96 0.00 0.00 0.00 0.00 Bit
112 0.00 0.00 0.00 0.00 Bit
128 0.00 0.00 0.00 0.00 Bit
160 0.00 0.00 0.00 0.00 Bit
192 0.00 0.00 0.00 0.00 Bit
224 0.00 0.00 0.00 0.00 Bit
256 0.00 0.00 0.00 0.00 Bit

Absolute Sum of shifted coefficients Min Mean Max StdDev
4 5.67 8.62 10.51 0.77 Bit
8 6.09 8.91 11.36 0.88 Bit
12 6.75 9.65 11.85 0.79 Bit
16 7.04 10.13 12.29 0.95 Bit
24 7.44 10.02 13.03 0.98 Bit
32 7.60 10.06 12.81 0.83 Bit
48 0.00 0.00 0.00 0.00 Bit
64 0.00 0.00 0.00 0.00 Bit
80 0.00 0.00 0.00 0.00 Bit
96 0.00 0.00 0.00 0.00 Bit
112 0.00 0.00 0.00 0.00 Bit
128 0.00 0.00 0.00 0.00 Bit
160 0.00 0.00 0.00 0.00 Bit
192 0.00 0.00 0.00 0.00 Bit
224 0.00 0.00 0.00 0.00 Bit
256 0.00 0.00 0.00 0.00 Bit


=== Frame partition ========================================

SubFrameNum
N: 72252
Minimum: 1.00
Mittelwert: 1.60
Maximum: 3.00
Verteilung
1 61.543 % (44466)
2 16.442 % (11880)
3 22.015 % (15906)
4 0.000 % ()
5 0.000 % ()
> 0.000 % ()


=== Joint Stereo ===========================================

Modes
N: 36126
Minimum: 0.00
Mittelwert: 1.25
Maximum: 2.00
Verteilung
0 21.641 % (7818)
1 31.288 % (11303)
2 47.071 % (17005)
> 0.000 % ()


=== Results ================================================

01. American Idiot.yaa
64.18 % - 57.1 * - 3.055 sec
02. Carl Orff - Fortune plango vulnera.yaa
48.95 % - 51.5 * - 3.109 sec
02. Carl Orff - Veris leta facies; Omnia Sol temperat.yaa
31.18 % - 61.7 * - 5.567 sec
02. Parallel Universe.yaa
59.39 % - 44.9 * - 6.025 sec
02. Wolfgang Amadeus Mozart - O Isis und Osiris (Sarastro - Chor).yaa
41.39 % - 54.5 * - 3.609 sec
03. Who's sorry now.yaa
27.95 % - 62.4 * - 2.913 sec
04. Burke - Johnston - Pennies From Heaven.yaa
55.58 % - 22.9 * - 26.612 sec
04. Carl Orff - I. Primo vere - Omnia sol temperat.yaa
39.80 % - 9.8 * - 11.049 sec
04. Japanese sandman.yaa
22.89 % - 37.1 * - 5.727 sec
05. Koop - Summer Sun.yaa
64.10 % - 22.7 * - 9.888 sec
05. Wolfgang Amadeus Mozart - Rex tremendae majestatis.yaa
44.97 % - 33.4 * - 4.212 sec
06. The Subject Was Faggots.yaa
44.04 % - 46.7 * - 4.111 sec
08. Morgana King - It's De-lovely.yaa
36.95 % - 54.6 * - 2.464 sec
08. Wolfgang Amadeus Mozart - Requiem in D Minor, KV 626, VIII. 08.
Wolfgang Amadeus Mozart - Requiem in D Minor, KV 626, VIII.
Lacrimosa.yaa 40.82 % - 25.0 * - 10.069 sec
11. Nil (Instrumental).yaa
20.58 % - 60.2 * - 2.277 sec
11. Throb.yaa
54.05 % - 29.8 * - 9.163 sec

Compression: 45.44 %
Duration: 109.87 sec
Speed: 32.87 * real time

With FAST mode, it seems SSE slows things down. Further lookout for this type of behavior would be necessary.

[TBeck]

' date='May 13 2006, 01:49' post='391904']
On SSE optimization :
...
With FAST mode, it seems SSE slows things down. Further lookout for this type of behavior would be necessary.

You are really fast!

But please don't post the whole protocol file. Most of the info isn't helpful yet.

Summary of your results:

CODE

Run1: MMX + SSE

Compression:    45.44 %
Speed:          38.01 * real time

Run 2: MMX

Compression:    45.44 %
Speed:          44.93 * real time

Run 3: MMX + SSE

Compression:    45.44 %
Speed:          32.87 * real time

You see the speed difference between Run 1 and Run 3, which use the same settings? And regarding Run 2: I advised to test SSE on Preset HIGH. You did use Preset FAST with 32 Predictors. But SSE will never be used with less than 64 Predictors!

Hence all three test runs have used exactly the same optimizations: Always MMX, never SSE.

The big speed variations are most probably beeing caused by variations of your test systems: Disk caching issues or disturbances by background activity. Under controled conditions all three runs should have achieved about the same speed (maybe within the range of +/- 10 %; on my system i achieve +/- 3 %).

BTW: Activation of the protocol function slows encoding down.

This post has been edited by TBeck: May 13 2006, 01:57

[Shade[ST]]

It seems my disk caching is quite unstable, then. I did hear thrashing sounds between the encoding of two songs. Also, run 1 is command-line; runs 2+3 are GUI.

I am currently testing Insane mode with/without SSE on my corpus (which compresses quite a bit, as you can see..)

Results will be posted in approx 5 min.

[TBeck]

' date='May 13 2006, 02:48' post='391912']
songs. Also, run 1 is command-line; runs 2+3 are GUI.

That's important. There can be speed differences between command line and gui versions.

[Shade[ST]]

These results are actually quite surprising, as they show discrepancies in the SSE optimizations versus compression ratio. It should normally be the same, no?

CODE

Insane (search level 4) + SSE
Compression:    44.65 %
Duration:     1289.32 sec
Speed:           2.80 * real time

Insane (search level 3) + SSE
Compression:    44.66 %
Duration:      955.93 sec
Speed:           3.78 * real time

Insane (search level 3) + No SSE
Compression:    44.65 %
Duration:     1161.98 sec
Speed:           3.11 * real time

This post has been edited by Shade[ST]: May 13 2006, 02:12

[TBeck]

' date='May 13 2006, 03:06' post='391920']
These results are actually quite surprising, as they show discrepancies in the SSE optimizations versus compression ratio. It should normally be the same, no?

I am quite new to SSE... I would have thought, that it gives the same results as single precision arithmetic performed by the floating point unit (FPU). Mabe it differs in the handling of rounding and underflows (i didn't care for the setting of the right SSE-Flags). But 0.01 percent deviation shouldn't be too important. But we should keep an eye on it (?).

Far more interesting for me to see a significant speed benefit for SSE. At least big enough to try a bit more SSE optimizations later.

Thanks

Thomas

P.S. Could you please post your CPU-type? Encoding speed for INSANE is more than two times higher than on my system. Surprising, because the speed of FAST isn't higher than on my system. FAST seems to be very sensitive to disk io performance.

This post has been edited by TBeck: May 13 2006, 02:40