Nick.C
Apr 18 2008, 13:54
Right, I've reassessed the quality presets in light of:
a) discovering the bug in the preparation of the skewing parameters;
b) changing the default number of FFT analyses to 2 lengths for all quality presets;
c) a desire to tighten up the spreading function (same for all quality presets);
d) introducing the floating point quality presets;
e) a desire to increase the quality of the highest quality preset.
So, now the permitted range for quality preset is -0.0000 to -7.0000 (resolution 0.0001).
Corresponding internal settings:
CODE
spreading_function_string : string[29]='22223-22224-12235-12246-12357';
quality_noise_threshold_shifts : array[0..Quality_Presets] of Integer = (-12,-8,-4,0,4,8,12,16);
quality_signal_to_noise_ratio : array[0..Quality_Presets] of Integer = (30,27,24,21,20,19,18,17);
quality_clips_per_channel : array[0..Quality_Presets] of Integer = (0,0,0,1,2,3,3,3);
Resultant bitrates (for my 53 problem sample set):
-0.0: 610kbps; -0.5: 589kbps; -1.0: 566kbps; -1.5: 543kbps; -2.0: 520kbps; -2.5: 496kbps; -3.0: 472kbps; -3.5: 451kbps;
-4.0: 431kbps; -4.5: 412kbps; -5.0: 395kbps; -5.5: 379kbps; -6.0: 364kbps; -6.5: 350kbps; -7.0: 338kbps.
lossyWAV beta v0.9.4 attached to post #1 in this thread.
halb27
Apr 18 2008, 15:34
I encoded my regular sample set (which is a bit typical of what at least I encode usually) with 0.9.4 -3 and got an average bitrate of 417 kbps.
As the new -3 roughly corresponds to the old -2 this is a good result IMO.
I also see sense in shifting the meaning of the quality levels a bit in favor of the top quality edge.
Nick.C
Apr 18 2008, 15:39
QUOTE(halb27 @ Apr 18 2008, 22:34)

I encoded my regular sample set (which is a bit typical of what at least I encode usually) with 0.9.4 -3 and got an average bitrate of 417 kbps.
As the new -3 roughly corresponds to the old -2 this is a good result IMO.
I also see sense in shifting the meaning of the quality levels a bit in favor of the top quality edge.
Excellent!
I've processed my 10 album test set at -1: 520kbps and -7: 300kbps. Over the weekend, I'll process the other integer settings for comparison. I'm listening to Mike Oldfield's Tr3s Lunas at -7 (297kbps) and it's adequate for DAP use (at least).
Nick.C
Apr 19 2008, 07:26
QUOTE(Nick.C @ Apr 18 2008, 22:39)

....I'll process the other integer settings for comparison.
I've finished processing the other integer presets:
FLAC: 854kbps; -0: 573kbps; -1: 520kbps; -2: 467kbps; -3: 417kbps; -4: 376kbps; -5: 343kbps; -6: 318kbps; -7: 300kbps.
-0 results in 2.25GB from 3.35GB original (67.1%); -7 results in 1.17GB from 3.35GB original (35.1%).
halb27
Apr 19 2008, 08:25
I just did this as well for my regular sample set as well as my problem sample set, and the results for the regular music are partially extremely close to yours, Nick:
regular music problem samples
-0 566 kbps 633 kbps
-1 518 kbps 596 kbps
-2 467 kbps 554 kbps
-3 417 kbps 509 kbps
-4 372 kbps 467 kbps
-5 335 kbps 427 kbps
-6 305 kbps 392 kbps
-7 281 kbps 360 kbps
This looks nice IMO.
Nick.C
Apr 19 2008, 14:18
I've finished my FLAC > lossyFLAC transcode at -5a: 298 discs, 3838 tracks, 11d12:56:19, 102GB > 39.5GB, 886kbps > 340kbps.
botface
Apr 20 2008, 05:45
I know that nobody is soliciting opinions but in case anybody's interested .....
I think it's absolutely right to concentrate on getting the original concept implemented for an initial release and adding noise shaping and anything else in a subsequent release. I also think that giving preference to the higher quality settings is a good idea, but then I am more interested in quality than file sizes. For what it's worth I think LossyWAV is now looking like a much more professional, mature product.
I have converted a number of files using 0.9.4. They were a mixture of 16/44.1 and 24/48. All were sourced from vinyl. Not being very comfortable with the command line I used IFLCDrop, which only has "1" as the highest quality option available. Using "1" the 16/44.1 files ended up with bit rates ranging from 505kbps to 525kbps. The 24/48 files had a range of 535kbps to 555kbps - as an aside, I guess this gives credence to the argument that using 24 bits for vinyl is overkill as LossyWAV obviously doesn't think most of the extra bits are storing anything worth keeping. To me these are ideal bit rates as they nicely fill the gap between existing lossy and lossless.
I am in the process of arching my vinyl collection and while I will continue to use lossless for records that are very important to me, I will definetly use LossyWAV for those that are just "nice to have" (especially if someone comes up with a nice friendly GUI front end).
If there is any interest in developing LossyWAV for use ith higher bit depths/sample rates I'd be very happy to help with testing - bearing in mind that I'm not very technical, just a music lover - as I'd like to be able contribute in some way
Nick.C
Apr 20 2008, 10:13
QUOTE(botface @ Apr 20 2008, 12:45)

I know that nobody is soliciting opinions but in case anybody's interested .....
I think it's absolutely right to concentrate on getting the original concept implemented for an initial release and adding noise shaping and anything else in a subsequent release. I also think that giving preference to the higher quality settings is a good idea, but then I am more interested in quality than file sizes. For what it's worth I think LossyWAV is now looking like a much more professional, mature product.
I have converted a number of files using 0.9.4. They were a mixture of 16/44.1 and 24/48. All were sourced from vinyl. Not being very comfortable with the command line I used IFLCDrop, which only has "1" as the highest quality option available. Using "1" the 16/44.1 files ended up with bit rates ranging from 505kbps to 525kbps. The 24/48 files had a range of 535kbps to 555kbps - as an aside, I guess this gives credence to the argument that using 24 bits for vinyl is overkill as LossyWAV obviously doesn't think most of the extra bits are storing anything worth keeping. To me these are ideal bit rates as they nicely fill the gap between existing lossy and lossless.
I am in the process of arching my vinyl collection and while I will continue to use lossless for records that are very important to me, I will definetly use LossyWAV for those that are just "nice to have" (especially if someone comes up with a nice friendly GUI front end).
If there is any interest in developing LossyWAV for use ith higher bit depths/sample rates I'd be very happy to help with testing - bearing in mind that I'm not very technical, just a music lover - as I'd like to be able contribute in some way
We're always looking for opinions / advice / (constructive) criticism, so your comments are very welcome.
As it is, lossyWAV should handle up to 32bit integer samples and there is no limit on sample-rate. Oh, and it will handle up to 8 channels, although this is an arbitrary limit and could be changed (although I can't imagine a scenario that would require it).
Feedback on the quality of the output at higher sample rates (>=96khz) and higher bit-depths (24bit or 32bit) would be very gratefully received!
Nick.C
Apr 20 2008, 13:01
Taking into account what botface was saying about the higher quality presets, I've had (yet) another think about the basis for the presets. If the -nts and -snr values were changed from:
CODE
quality_noise_threshold_shifts : array[0..Quality_Presets] of Integer = (-12,-8,-4,0,4,8,12,16);
quality_signal_to_noise_ratio : array[0..Quality_Presets] of Integer = (30,27,24,21,20,19,18,17);
to
CODE
quality_noise_threshold_shifts : array[0..Quality_Presets] of Integer = (-24,-16,-8,0,4,8,12,16);
quality_signal_to_noise_ratio : array[0..Quality_Presets] of Integer = (39,33,27,21,20,19,18,17);
Then the resultant bitrates (for my 53 problem sample set) would change from:
-0.0: 610kbps; -0.5: 589kbps; -1.0: 566kbps; -1.5: 543kbps; -2.0: 520kbps; -2.5: 496kbps; -3.0: 472kbps;
to
-0.0: 717kbps; -0.5: 686kbps; -1.0: 651kbps; -1.5: 611kbps; -2.0: 555kbps; -2.5: 519kbps; -3.0: 472kbps;
with the remaining presets:
-3.5: 451kbps; -4.0: 431kbps; -4.5: 412kbps; -5.0: 395kbps; -5.5: 379kbps; -6.0: 364kbps; -6.5: 350kbps; -7.0: 338kbps
staying the same.
[edit] Oh, I forgot to say: there's an undocumented parameter, currently called -lowpass (the name should maybe be changed - suggestions please) which changes the upper limit of the frequency range used in the spreading-function / minimum value / average value calculations. Permitted values are in the range 13.5kHz to 20.05kHz. [/edit]
halb27
Apr 20 2008, 14:23
I prefer the old settings, at least as far as -nts is concerned.
Because of the reduced efficiency of the lossless codec due to the small blocksize it happens rather often with non-complex or quiet music when targeting at a lossyFLAC average bitrate of just 380 kbps, that a totally lossless codec yields a lower bitrate than lossyFLAC (that's why I used lossless wavPack instead of lossyFLAC to encode many tracks from my collection which contain classical music or music with few instruments). The higher we choose the bitrate the more often does this happen.
2Bdecided's basic priciple is fulfilled with -nts 0, so for an additional security -nts -4/-8/-12 should be sufficient IMO even for the most cautious people.
I also like the 16 kHz limit for doing the bits-to-remove calculations. Going lower would reduce the risk that bits-to-remove is 0 for the only reason that there's no (or only low volume) HF content around 16 kHz. But it increases the risk of not taking audible HF noise into account. When going higher it's the other way around.
IMO the best compromise is very much what you are doing right now.
Nick.C
Apr 20 2008, 15:13
QUOTE(halb27 @ Apr 20 2008, 21:23)

I prefer the old settings, at least as far as -nts is concerned.
Because of the reduced efficiency of the lossless codec due to the small blocksize it happens rather often with non-complex or quiet music when targeting at a lossyFLAC average bitrate of just 380 kbps, that a totally lossless codec yields a lower bitrate than lossyFLAC (that's why I used lossless wavPack instead of lossyFLAC to encode many tracks from my collection which contain classical music or music with few instruments). The higher we choose the bitrate the more often does this happen.
2Bdecided's basic priciple is fulfilled with -nts 0, so for an additional security -nts -4/-8/-12 should be sufficient IMO even for the most cautious people.
I also like the 16 kHz limit for doing the bits-to-remove calculations. Going lower would reduce the risk that bits-to-remove is 0 for the only reason that there's no (or only low volume) HF content around 16 kHz. But it increases the risk of not taking audible HF noise into account. When going higher it's the other way around.
IMO the best compromise is very much what you are doing right now.
Okay - points taken - reassuring, really.
To satisfy my curiousity, I ran my 53 problem sample set and 10 album test set at what would be quality presets -8, -9 and -10 using the same increments as the existing -3 to -7, i.e. -nts=20,24,28; -snr=16,15,14 respectively, and got:
53 problem sample set: (-7: 338kbps) -8: 318kbps; -9: 303kbps; -10: 292kbps;
10 album test set: (-7: 300kbps) -8: 286kbps; -9: 278kbps; -10: 273kbps;
halb27
Apr 20 2008, 15:28
From just careful listening (especially to the problem samples): is -8/-9/-10 acceptable with respect to the still high quality everybody may expect from a >=270 kbps encoding?
Nick.C
Apr 20 2008, 15:40
QUOTE(halb27 @ Apr 20 2008, 22:28)

From just careful listening (especially to the problem samples): is -8/-9/-10 acceptable with respect to the still high quality everybody may expect from a >=270 kbps encoding?
I think that tomorrow night I'll start a lossyFLAC -10 transcode of my 102GB collection (should take about 6˝hours or so) and I'll listen to it over a couple of days until:
a) I am content that it satisfies the requirement;
or
b) I am not content, therefore transcode to -9 (then -8 if necessary) and repeat the longer term listening test.
Or, maybe I should leave the quality presets alone and use -snr and -nts to "roll-my-own" without pushing lossyWAV too far....?
2Bdecided
Apr 21 2008, 03:47
QUOTE(halb27 @ Apr 20 2008, 21:23)

I prefer the old settings, at least as far as -nts is concerned.
Because of the reduced efficiency of the lossless codec due to the small blocksize it happens rather often with non-complex or quiet music when targeting at a lossyFLAC average bitrate of just 380 kbps, that a totally lossless codec yields a lower bitrate than lossyFLAC.
Do you think it would be worth checking for this automatically?
If found you could increase the blocksize specified to the lossless encoder when encoding the lossyWAV output (e.g. 1024, 2048 etc), and if that didn't help you could just keep the lossless version?
The latter would be really easy if you were transcoding from FLAC (or whatever) lossless anyway (if the lossyWAV program can find the files). Otherwise, I can't see any way of doing the checking other than encoding the file to lossless to see how big it comes out - which is going to double or triple the time spent on lossless encoding! Thoughts?
Cheers,
David.
botface
Apr 21 2008, 10:06
QUOTE(Nick.C @ Apr 20 2008, 19:01)

Taking into account what botface was saying about the higher quality presets, I've had (yet) another think about the basis for the presets. If the -nts and -snr values were changed from:
CODE
quality_noise_threshold_shifts : array[0..Quality_Presets] of Integer = (-12,-8,-4,0,4,8,12,16);
quality_signal_to_noise_ratio : array[0..Quality_Presets] of Integer = (30,27,24,21,20,19,18,17);
to
CODE
quality_noise_threshold_shifts : array[0..Quality_Presets] of Integer = (-24,-16,-8,0,4,8,12,16);
quality_signal_to_noise_ratio : array[0..Quality_Presets] of Integer = (39,33,27,21,20,19,18,17);
Then the resultant bitrates (for my 53 problem sample set) would change from:
-0.0: 610kbps; -0.5: 589kbps; -1.0: 566kbps; -1.5: 543kbps; -2.0: 520kbps; -2.5: 496kbps; -3.0: 472kbps;
to
-0.0: 717kbps; -0.5: 686kbps; -1.0: 651kbps; -1.5: 611kbps; -2.0: 555kbps; -2.5: 519kbps; -3.0: 472kbps;
with the remaining presets:
-3.5: 451kbps; -4.0: 431kbps; -4.5: 412kbps; -5.0: 395kbps; -5.5: 379kbps; -6.0: 364kbps; -6.5: 350kbps; -7.0: 338kbps
staying the same.
[edit] Oh, I forgot to say: there's an undocumented parameter, currently called -lowpass (the name should maybe be changed - suggestions please) which changes the upper limit of the frequency range used in the spreading-function / minimum value / average value calculations. Permitted values are in the range 13.5kHz to 20.05kHz. [/edit]
Nick,
It looks like changing the default -snr and -nts as per your thought will result in bitrates not much lower than lossless, which seems a bit pointless. Anyway, a very cautious person could use higher settings if they want to as it stands.
Re the undocumented feature, a snappy name doesn't spring to mind, but given my interest in higher sample rates could it be useful there? Maybe you could even have different defaults and allowable ranges based on the sample rate of the source file.
Speaking of higher sample rates, I volunteered to do some testing yesterday. It turns out that I can't produce 32 bit integer and since we know that LossyWAV works quite happily with 24 bit is there anything worthwhile I could test? I'd be more than willing to to do some testing on 24/48, 24/64, 24/88.2 and 24/96 if you think it might tell you something you don't already know. If you do want me to do some testing could you suggest the kind of music that's most likely to reveal problems. I'd assume acoustic instruments would be good especially if they had the ability to sustain notes over a long period EG violin, whistle, pipes etc.
GeSomeone
Apr 21 2008, 14:12
QUOTE(Nick.C @ Apr 20 2008, 21:01)

If the -nts and -snr values were changed from:
CODE
quality_noise_threshold_shifts : array[0..Quality_Presets] of Integer = (-12,-8,-4,0,4,8,12,16);
to
CODE
quality_noise_threshold_shifts : array[0..Quality_Presets] of Integer = (-24,-16,-8,0,4,8,12,16);
Funny, I would rather suggest (-6,-4,-2,0,2,4,6,8). Usually there is a sweet spot (like the lowest bit rate with no audible artifacts). If you move away from there extremely, at least theoretically you would be either inefficient or get audible differences.
But as your testing indicates that the "low" quality is still acceptable, I can see the how a gliding scale from lossless to lossy would be filling all the bit rate gaps at the same time
Nick.C
Apr 21 2008, 14:28
QUOTE(halb27 @ Apr 20 2008, 22:28)

From just careful listening (especially to the problem samples): is -8/-9/-10 acceptable with respect to the still high quality everybody may expect from a >=270 kbps encoding?
I managed to listen to my 53 problem sample set at -10 and -9 today - definitely too much hiss at -10 (only for certain samples, however) and it is still noticable (and annoying, although I'm deliberately listening out for it) at -9. I will try to listen to -8 tomorrow.
@Botface: I don't really know which samples / tracks will be best for evaluation purposes - although I would guess that as Gurubooleez has a library of 150 instrumental samples, instrumental will be useful - I think harpsichord is a difficult type to retain transparency for.
@GeSomeone: I'm going to stick with -12..16 (step 4) for the -nts values. The change from integer to float for the quality preset selection has made use selection of intermediate points much simpler rather than manual -snr and -nts selections.
halb27
Apr 22 2008, 01:37
QUOTE(2Bdecided @ Apr 21 2008, 11:47)

QUOTE(halb27 @ Apr 20 2008, 21:23)

I prefer the old settings, at least as far as -nts is concerned.
Because of the reduced efficiency of the lossless codec due to the small blocksize it happens rather often with non-complex or quiet music when targeting at a lossyFLAC average bitrate of just 380 kbps, that a totally lossless codec yields a lower bitrate than lossyFLAC.
Do you think it would be worth checking for this automatically?
If found you could increase the blocksize specified to the lossless encoder when encoding the lossyWAV output (e.g. 1024, 2048 etc), and if that didn't help you could just keep the lossless version?
The latter would be really easy if you were transcoding from FLAC (or whatever) lossless anyway (if the lossyWAV program can find the files). Otherwise, I can't see any way of doing the checking other than encoding the file to lossless to see how big it comes out - which is going to double or triple the time spent on lossless encoding! Thoughts?
Cheers,
David.
I noticed this kind of "issue" when I encoded my entire collection a few months ago. The lossless version in my archive was encoded with TAK then and I just compared the lossyFLAC file size with that of the TAK files.
I then tried lossless FLAC but noticed that in these cases with quiet and/or non-complex music where TAK was very efficient the lossless FLAC -8 efficiency was clearly inferior. So in these cases lossyFLAC suffers not only from the small blocksize but also from the relatively low performance of FLAC. So I used lossless wavPack which I can play on my DAP and which gives efficient results in these cases.
I'm afraid there will be no automatic solution, but IMO it's not really a problem. Most people are expected to transcode from a lossless codec so a lossyFLAC file size increase can be easily seen. Moreover an easy and very straightforward approach can be used, an aproach like "don't care if lossyFLAC bitrate goes up cause it happens rarely" (very adequate for people who usually listen to rock/pop or similar music) or "for solo instrument music and non-complex classical music avoid using lossyFLAC and go straight to an appropriate lossless codec" (adequate for lovers of solo instrument music or similar).
After all these cases say "a lossless codec is very efficient here" more than "lossyFLAC is bad here".
halb27
Apr 22 2008, 02:03
QUOTE(Nick.C @ Apr 21 2008, 22:28)

QUOTE(halb27 @ Apr 20 2008, 22:28)

From just careful listening (especially to the problem samples): is -8/-9/-10 acceptable with respect to the still high quality everybody may expect from a >=270 kbps encoding?
I managed to listen to my 53 problem sample set at -10 and -9 today - definitely too much hiss at -10 (only for certain samples, however) and it is still noticable (and annoying, although I'm deliberately listening out for it) at -9. I will try to listen to -8 tomorrow.
@Botface: I don't really know which samples / tracks will be best for evaluation purposes - although I would guess that as Gurubooleez has a library of 150 instrumental samples, instrumental will be useful - I think harpsichord is a difficult type to retain transparency for.
@GeSomeone: I'm going to stick with -12..16 (step 4) for the -nts values. The change from integer to float for the quality preset selection has made use selection of intermediate points much simpler rather than manual -snr and -nts selections.
There was a time when I thought for a more refined quality setting -nts is what should be used, and as a primary quality setting -1/-2/-3 is what we should have.
But now that we have arrived at these many primary quality settings I think this is the better way to go, but in this case we should concentrate on the primary quality setting and hide details like -nts from the user in the final version (or devide them clearly apart into the advanced options).
From just bitrate/expected quality I like your -0 to -7 settings of 0.9.4 very much (or maybe an aditional -8 in case that's useful).
It's a clear thing (especially when ommitting the a and b appendix which IMO should go into the advanced options as well [as something like -analyses n]).
IMO the default quality should be -3 (in 0.9.4 scale) and we should clearly state that -0 and -1 are useful only as an extremely high quality lossy substitution for a lossless encoding.
Nick.C
Apr 22 2008, 05:24
QUOTE(halb27 @ Apr 22 2008, 09:03)

There was a time when I thought for a more refined quality setting -nts is what should be used, and as a primary quality setting -1/-2/-3 is what we should have.
But now that we have arrived at these many primary quality settings I think this is the better way to go, but in this case we should concentrate on the primary quality setting and hide details like -nts from the user in the final version (or devide them clearly apart into the advanced options).
From just bitrate/expected quality I like your -0 to -7 settings of 0.9.4 very much (or maybe an aditional -8 in case that's useful).
It's a clear thing (especially when ommitting the a and b appendix which IMO should go into the advanced options as well [as something like -analyses n]).
IMO the default quality should be -3 (in 0.9.4 scale) and we should clearly state that -0 and -1 are useful only as an extremely high quality lossy substitution for a lossless encoding.
I will move -nts and -snr to the advanced settings section and implement another advanced setting -analyses <n> per your suggestion, while at the same time removing the a,b or c suffix to the quality preset parameter.
I will also add a -8 parameter for this stage of testing (-nts=20; -snr=16).
lossyWAV beta v0.9.5 attached to post #1 in this thread.
jesseg
Apr 22 2008, 13:39
It's really amazing to see lossyWAV get to where it's at now, and thanks to everyone helping out with this, from the original idea to just the "fans". I really like where it's come to by this point, and thanks to Nick especially of course, and for almost always knowing which "whims" were good to explore, and which ones to hold off on. Without that wisdom, we would still be back before v0.50 =)
I agree with the advanced settings in the help, and it might be a good thing even to leave the advanced stuff complete out of the normal help which comes up with, for instance:
CODE
lossyWAV
lossyWAV -help
and go with something like LAME, and have a
CODE
lossyWAV -longhelp
for the advnaced stuff, that way only one line is needed to mention that there's more help if you use that variable, but it's not crucial that the information is seen for "normal" and "safe" use of lossyWAV.
It also would be nice to get some graphics gurus in here, to do up a right proper logo for this. Perhaps some submissions from the likes of deviant art or the like? I've never used those types of sites much, but there should be a few things out there that would support that kind of thing.
Nick.C
Apr 22 2008, 14:01
QUOTE(jesseg @ Apr 22 2008, 20:39)

It's really amazing to see lossyWAV get to where it's at now, and thanks to everyone helping out with this, from the original idea to just the "fans". I really like where it's come to by this point, and thanks to Nick especially of course, and for almost always knowing which "whims" were good to explore, and which ones to hold off on. Without that wisdom, we would still be back before v0.50 =)
I agree with the advanced settings in the help, and it might be a good thing even to leave the advanced stuff complete out of the normal help which comes up with, for instance:
CODE
lossyWAV
lossyWAV -help
and go with something like LAME, and have a
CODE
lossyWAV -longhelp
for the advnaced stuff, that way only one line is needed to mention that there's more help if you use that variable, but it's not crucial that the information is seen for "normal" and "safe" use of lossyWAV.
It also would be nice to get some graphics gurus in here, to do up a right proper logo for this. Perhaps some submissions from the likes of deviant art or the like? I've never used those types of sites much, but there should be a few things out there that would support that kind of thing.

Where lossyWAV is now is a result of input, feedback and constructive criticism - without any of these, it would not be the utility it is. Thanks are due to all who have bothered to contribute in any way.
I really like the -help and -longhelp parameter suggestions (or maybe combine -help -detail to get the more detailed help

). I'll get busy writing the explanatory text to go behind them and will "hide" the advanced options so that they are not shown when just "lossyWAV" is used as a command line.
I haven't had any success integrating an icon into the Delphi binary - maybe I'm missing something, maybe it's a feature that is missing from the free version of Turbo Delphi.
jesseg
Apr 22 2008, 14:16
As long as the executable has a resource section (yours do already, for version info) then you can use one of several resource editors to import a new .ico into the executable, and they should automatically create and icon group resource too.
Here's two of my favs, both are free:
http://www.wilsonc.demon.co.uk/d10resourceeditor.htmhttp://www.angusj.com/resourcehacker/They are both useful because each of them have their own querks. I recommend using XN whenever you can though. Also usefull is LordPE or PEiD, for rebuilding your executables. That also cuts down on the file size slightly.
And of course there's exe compressors, UPX isn't bad (i use a custom made "uber-brute-force" version which packs each half KB section with multiple types, and then going into previous sections, to figure out what lengths and what compression types will give maximum compression, and yes it's VERY slow, especially with executables over 1MB

)
For an executable under 100kb, I would recommend FSG, but it is picked up as a false alarm by 3 un-popular AV (because their developers are too lazy to write support for unpacking FSG into their AV I guess, of which publicly available source-code already exists for C and Assembly.)
http://programmerstools.org/node/50But yeah, without packing I'm usually able to get lossyWAV down to around 80KB... with the lossyWAV icon i made, which has all 4 sizes. If you stripped it down to just 32x32 size, it could get even smaller.
jesseg
Apr 22 2008, 14:30
Here's an example of one with the icon, and rebuilt.... as well as FSG and my UPX uber-brute. My UPX actually beats FSG on this one, so I no longer suggest using FSG, but UPX, if you pack it at all. See attached.
Nick.C
Apr 22 2008, 14:34
QUOTE(jesseg @ Apr 22 2008, 21:16)

As long as the executable has a resource section (yours do already, for version info) then you can use one of several resource editors to import a new .ico into the executable, and they should automatically create and icon group resource too.
Here's two of my favs, both are free:
http://www.wilsonc.demon.co.uk/d10resourceeditor.htmhttp://www.angusj.com/resourcehacker/They are both useful because each of them have their own querks. I recommend using XN whenever you can though. Also usefull is LordPE or PEiD, for rebuilding your executables. That also cuts down on the file size slightly.
Hehehe.... What fun, adding an icon at this late stage! I downloaded XN Resource Editor and have had a preliminary play about - it's *really* easy. I still like your fading vertical bit cascade with zeroed lsb's at the bottom as a background - not too sure about the font though.
botface
Apr 23 2008, 03:09
With regard to help/advanced help/useability etc I think it makes sense to keep standard options to a minimum. Ideally just a quality level needing to be specified. Not only does that keep things simple it also means that non-techies like me only need to worry about the end result they're looking for and can rest assured that sensible settings have been used by default for all important parameters. Personally, I'd like to see quality settings going from lowest to highest (1 - 9) rather than the other way round, but that's only a personal feeling.
On help specifically, I'd like to see some mention of how the various values of any setting are going to affect the end result. EG :
"-nts <n> set noise_threshold_shift to n dB (-48.0dB<=n<=+36.0dB);
(-ve values reduce bits to remove, +ve values increase)."
is fine as far as it goes but it might be more helpful if it went on to say something like "So, the higher the negative value used the less the liklehood of noise being introduced but at the expense of a higher file size after processing through a lossless codec whereas the higher the positive value ......... etc". Maybe you feel that's over the top for help, if so maybe it could go in the wiki. By the way that's a bit out of date at the moment
halb27
Apr 23 2008, 04:26
I'm also very fond of simplicity and clarity.
I support the suggestion that the advanced options should only be mentioned within something like a longhelp.
Thinking about the advanced options IMO only -nts and -analyses should be used.
It should be mentioned that there is no need to explicitly use -nts or -analyses as these are taken care of by the quality levels.
My suggestion for the -nts <n> description:
-nts 0 yields transparency according to experience.
A negative <n> adds a security margin (and increases file size) which is supposed to be overkill but maybe welcome when lossyWAV is used as a substitute for lossless archiving or similar applications with an extremely high quality demand.
A positive <n> yields a smaller filesize but adds the risk of audible deviations from the original. Due to additional internal precautions however a small <n> like
-nts 4 is expected not to harm transparency.
2Bdecided
Apr 23 2008, 08:50
QUOTE(botface @ Apr 23 2008, 10:09)

Personally, I'd like to see quality settings going from lowest to highest (1 - 9) rather than the other way round
I'd just logged on to suggest exactly this!
Now it's grown from -1 -2 -3 to a non-integer 0-10 scale, I think it might make sense to tie it into a scale that people already understand. The obvious one for me is the one Vorbis uses - Q5 is transparent, lower might not be, higher is overkill / safety margin. Others may have other suggestions.
I apologise for not suggesting this sooner!
Cheers,
David.
Raiden
Apr 23 2008, 09:47
QUOTE(2Bdecided @ Apr 23 2008, 16:50)

non-integer 0-10 scale
Lame uses that scale, too...
Dynamic
Apr 23 2008, 11:26
QUOTE(2Bdecided @ Apr 23 2008, 15:50)

QUOTE(botface @ Apr 23 2008, 10:09)

Personally, I'd like to see quality settings going from lowest to highest (1 - 9) rather than the other way round
I'd just logged on to suggest exactly this!
Now it's grown from -1 -2 -3 to a non-integer 0-10 scale, I think it might make sense to tie it into a scale that people already understand. The obvious one for me is the one Vorbis uses - Q5 is transparent, lower might not be, higher is overkill / safety margin. Others may have other suggestions.
Yes indeed, if you use "q" or "Q" for quality, this seems eminently sensible as -q 5.0 is also the "standard" transparent setting for Musepack, and increasing quality should correspond to an increasing number.
Conversely, LAME VBR (because MP3 isn't necessarily a VBR format) uses -V (not -Q) and here, 0 is the highest quality and bitrate while 9 is the lowest, so people most familiar with LAME might not get the expected behaviour. This discrepancy has always been true of different JPEG image apps, some using discrete settings, some using a "Quality" (0 is worst quality) scale and some using a "Compression" scale (0 is best quality), none of which seemed to correspond very closely to the scale in different apps.
Your original scale for the betas released to date corresponds to the degree of "loss" or "compression" allowed, and oddly enough, with 2 or 3 being equivalent to the "transparent" standard, it corresponds rather closely to LAME's current VBR scale.
Regardless of what you choose, I'd suggest that if you're calling it "quality" it should be a "0 is worse quality than 9" type of scale, and if you're calling is "loss" or "compression" it should be a "0 is better quality than 9" type of scale. Given that "constant quality" is what VBR is all about, my vote is for calling the scale quality and reversing from where you are now.
Happily, if "-q 5.00" in future corresponds to the current -2 or -3 transparent setting, then "-q 0.00" or "-q 1.00" would pretty-much correspond to the current -7 or -8 (one of which you'll decide is the lowest acceptable quality for low-battery portable use).
halb27
Apr 23 2008, 12:09
Though I personally don't care much about whether the quality scale goes up or down I like this idea of having a quality scale analogous to that of Vorbis as Dynamic describes it:
-q 0 = -8
-q 1 = -7
-q 2 = -6
-q 3 = -5
-q 4 = -4
-q 5 = -3
-q 6 = -2
-q 7 = -1
-q 8 = -0
This way I think everybody familiar with vorbis gets an immediate and intuitive feeling about the meaning of the quality setting.
I was thinking about the advanced options again, and with these differentiated quality scales I think we should drop -nts from the user interface for the final version. IMO just -analyses should make it into the advanced options.
Dynamic
Apr 23 2008, 12:44
On a side note, presumably the same quality scale or loss scale as you decide upon for the release version of lossyWAV could be used in any future dedicated hybrid lossy encoder based on the same kind of analysis as lossyWAV (if anybody considers it worth developing - see last paragraph).
For example, if Wavpack or FLAC had a "constant-quality" or "VBR" lossy mode based on the same type of analysis as lossyWAV, then instead of using it to zero the LSBs over a whole block it could be used to define the maximum allowable prediction residual error that should remain in the audio. That could be done by defining the bit-depth of the residuals that get stored (probably the easiest method) or defining the maximum allowable error in the residuals and choosing those residuals in some other way. (The limited bit depth of the predictor, or metadata within the file header or block header might be an indicator of lossy processing, but I guess it's much harder to spot than zeroed LSBs when decoded to PCM, but that's always going to be a problem with other types of lossy, such as MP2, MP3, AAC, Vorbis, ADPCM and the like).
Actually, unless I'm missing something, I guess an encoder like that could retain long block lengths for predictor efficiency but could still use the shorter overlapping FFT analysis windows like lossyWAV to define the allowable uncoded prediction residual error at various times within the block. It might even be possible to continuously (smoothly) vary the allowable error from sample to sample within the block to follow the profile of permissible noise given by the FFT windows that overlap on that sample according to some interpolation or in proportion to the value of each lapping window function centred around each FFT window's centre sample.
Obviously, the predictor's value is dependent on the previous samples, which are now different thanks to the permitted error, and this may worsen the prediction slightly (but this hasn't stopped Wavpack lossy from creating remarkable bitrate reductions with remarkable quality).
Both approaches based on this analysis method hold out great hope for transparent or high quality lossy audio with fairly modest bitrate and relatively low decoding complexity and a closely equivalent quality scale that could show any bitrate savings between methods quite accurately. A correction file for restoring true lossless is compatible with either method (unless you get into serious noise shaping and it becomes too large to use).
LossyWAV certainly delivers most of the possible gain in compression and it is compatible with a number of well-supported lossless codecs completely unchanged (with the option of converting from, say WavPack to FLAC according to support on the playback target without further audio loss), so the possible efficiency advantage of the second approach may be a case of diminishing returns and reduced flexibility regarding re-encoding to another codec. It would be even worse in terms of waiting for decoder support if such an implementation were no longer compatible with existing FLAC or Wavpack decoders, especially those embedded into playback devices.
Nick.C
Apr 23 2008, 13:09
So many posts to take in - with so many valid observations / ideas / comments / etc....
How about:
-0..-8 > -q 10 .. -q 0?
i.e. -0 > -q 10.0; -1 > -q 8.3333; -2 > -q 6.6667; -3 > -q 5.0; -4 > -q 4.0; -5 > -q 3.0; -6 > -q 2.0; -7 > -q 1.0; -8 > -q 0.0?
-snr and -nts could be removed from the user interface in v1.0.0, along with -noclips (perhaps).
halb27
Apr 23 2008, 13:55
That's fine, IMO, and, yes, I forgot about -noclips: I'd like to see -noclips in the advanced options.
Nick.C
Apr 23 2008, 14:39
QUOTE(halb27 @ Apr 23 2008, 20:55)

That's fine, IMO, and, yes, I forgot about -noclips: I'd like to see -noclips in the advanced options.
I'll get to work on beta v0.9.6 tomorrow (I've been installing another RAID card in my server and moving drives about this evening....).
The focus for beta v0.9.6 will be to implement the -q <n> parameter and remove the -<n> parameter, to significantly simplify the basic settings and to introduce the -help and -help -detail parameters / combination to give basic help (beyond that given by running lossyWAV with no parameters) and advanced help (with the advanced settings added).
robert
Apr 23 2008, 15:13
QUOTE(Dynamic @ Apr 23 2008, 19:26)

Conversely, LAME VBR (because MP3 isn't necessarily a VBR format) uses -V (not -Q) and here, 0 is the highest quality and bitrate while 9 is the lowest, so people most familiar with LAME might not get the expected behaviour. This discrepancy has always been true of different JPEG image apps, some using discrete settings, some using a "Quality" (0 is worst quality) scale and some using a "Compression" scale (0 is best quality), none of which seemed to correspond very closely to the scale in different apps.
Your original scale for the betas released to date corresponds to the degree of "loss" or "compression" allowed, and oddly enough, with 2 or 3 being equivalent to the "transparent" standard, it corresponds rather closely to LAME's current VBR scale.
It didn't surprise me to see lossyWav doing it the same way as we did with LAME. We share the idea, that we have to add more noise, to get smaller files or a higher compression ratio. And the question is, how much would you like to distort your input signal.
QUOTE
Regardless of what you choose, I'd suggest that if you're calling it "quality" it should be a "0 is worse quality than 9" type of scale, and if you're calling is "loss" or "compression" it should be a "0 is better quality than 9" type of scale.
So, you would prefer to fly 9th class over 1st class? In school I would prefer to get the note 1 (best) over 6 (worst). I don't think higher quality is associated with higher numbers naturaly, it depends on your social context.
QUOTE
Given that "constant quality" is what VBR is all about, my vote is for calling the scale quality and reversing from where you are now.
Well, by choosing any switch, the user can only degrade quality by increasing the number of bits to remove. Or did I miss a quality enhancement switch?
Don't get me wrong, I'm fine with whatever quality/compression scheme Nick wants lossyWav to have.
halb27
Apr 24 2008, 05:36
Instead of using Vorbis' -q n quality scale we could use Lame's -V n quality scale of course.
It's all a matter of taste.
I personally prefer the Vorbis analogy, not because of the scale direction which doesn't matter to me at all, but because I have a positive feeling towards the correspondence of -3 with -q5 and the corresponding consequences for the other quality settings. Such a -q5 can be considered transparent with a probability extremely close to 1, and from -q6 on there is an ever increasing security margin with a large security margin range to choose from.
With the Lame analogy I see problems. Which -V setting should correspond to -3? It would have to be -V3 or worse qualitywise in order to have our -2 to -0 correspond with higher -V settings.
I feel more comfortable having -3 correspond with Vorbis -q5 than with Lame -V3.
Moreover because of lossyWAV's properties I think it's good to put some emphasis to the extremely high quality settings (useful for high quality lossy archiving for instance).
With Nick's last proposal we have a lot of -q levels which deal with this high end demand (while we still have a lot of -q settings dealing with the lower end).
With the Lame -V levels it wouldn't be like that (or only if we let lossyWAV -3 correspond with something like -V5 which I think isn't very adequate).
robert
Apr 24 2008, 06:02
I don't see the point, why should any lossyWav setting match any Vorbis or LAME setting? If you say lossyWav -3 matches Vorbis -q5, why should I use lossyWav at all? If both are equal in quality, I would choose the smaller files. LossyWav wanted to fill a gap between lossless and other lossy encodings, if you want to pick up the vorbis scale, shouldn't it be from -q5(?) to -q20 then?? And no, I'm not proposing to choose a LAME alike scale, it's just, the lossyWav original settings made sense to me and I don't see any need to change that. Just my two cents.
GeSomeone
Apr 24 2008, 06:43
QUOTE(Nick.C @ Apr 23 2008, 21:09)

-snr and -nts could be removed from the user interface in v1.0.0, along with -noclips (perhaps).
I'd like to keep -nts (in the avanced category of course) it is the most meaningful parameter to tweak (apart from the -q n).
I'm neutral on the -quality vs. -n scale but if you want to change it, now is the time (before the first "final/stable"). It would be nice to know the settings of the integer values of whatever scale is chosen (nmt snr)
BTW is the default still corresponding with -2 ? I suggest to move the default to -3 of -4 (of the old scale).
carpman
Apr 24 2008, 06:54
QUOTE(robert @ Apr 24 2008, 12:02)

I don't see the point, why should any lossyWav setting match any Vorbis or LAME setting? ... LossyWav wanted to fill a gap between lossless and other lossy encodings, if you want to pick up the vorbis scale, shouldn't it be from -q5(?) to -q20 then?? ..... it's just, the lossyWav original settings made sense to me and I don't see any need to change that. Just my two cents.
Agree entirely. That's my two cents.
C.
halb27
Apr 24 2008, 06:58
It's pure emotion, no real reason. lossyWAV -3 quality isn't the same as Vorbis -q5's of course.
To me - maybe only to me - it's like this:
If I'd use Vorbis and struggle for transparency in a robust way while trying not to waste file size I'm fine with -q5 (of course a matter of taste and -q4 or -q6 are candidates for an appropriate setting as well). If I'd want an additional safety margin (maybe for archiving purposes) I'd better use -q6 or higher.
With the Lame -V settings as an alternative there's simply not so much room for various high end settings (also a matter of taste).
Sure the analogies no matter whether it's about Vorbis or Lame have their drawbacks as they may suggest that we get a quality at ~400 kbps that we can have at ~200 kbps or below using Vorbis or Lame.
Despite this my personal preference is still with the Vorbis-like scale, but the many words I've used to try to make that understand are misleading: after all I don't care much about it. I'm also happy with the original lossyWAV scale.
Nick.C
Apr 24 2008, 06:58
QUOTE(GeSomeone @ Apr 24 2008, 13:43)

QUOTE(Nick.C @ Apr 23 2008, 21:09)

-snr and -nts could be removed from the user interface in v1.0.0, along with -noclips (perhaps).
I'd like to keep -nts (in the avanced category of course) it is the most meaningful parameter to tweak (apart from the -q n).
I'm neutral on the -quality vs. -n scale but if you want to change it, now is the time (before the first "final/stable"). It would be nice to know the settings of the integer values of whatever scale is chosen (nmt snr)
I could be persuaded to leave -nts in the advanced options....
[edit] Throughout the development of lossyWAV, -1, -2 and -3 have always been called quality presets. Yes, I agree that 1st class is better than 2nd class, but where does 0th class fit in (as it doesn't exist in normal speech). So, I've gone for a quality-increases-with-value-of-numerical-preset approach, on a scale of 0 to 10. Moving from -1, -2 and -3 to -1 to -7 it seems a logical progression to allow 100,000 quality preset options between -q 0.0 to -q 10.0 with a 0.0001 resolution rather than the original 3. This will allow the user to chose a personal transparency level much more easily than if they had to juggle -nts and -snr manually. Maybe some explanation will need to be added to the wiki with comparisons with previous preset bitrates. [/edit]
I've implemented the -q 0 to 10 quality preset selection and have had a thought. Up until now, the maximum bits-to-remove has been limited to (rms-value-of-all-samples-in-codec-block - 3). I am considering introducing a mechanism which would change the 3 by adding the quality-preset value divided by 4, i.e. at -q 10 subtract 5.5 rather than 3.0. This would increase the output of my 53 problem sample set from 611kbps to 616kbps at -q 10 (-nts -12 -snr 30) and from 472kbps to 482kbps at -q 5 (-nts 0 -snr 21).
halb27
Apr 24 2008, 07:05
QUOTE(Nick.C @ Apr 24 2008, 14:58)

I've implemented the -q 0 to 10 quality preset selection and have had a thought. Up until now, the maximum bits-to-remove has been limited to (rms-value-of-all-samples-in-codec-block - 3). I am considering introducing a mechanism which would change the 3 by adding the quality-preset value divided by 4, i.e. at -q 10 subtract 5.5 rather than 3.0. This would increase the output of my 53 problem sample set from 611kbps to 616kbps at -q 10 (-nts -12 -snr 30) and from 472kbps to 482kbps at -q 5 (-nts 0 -snr 21).
I like the idea.
GeSomeone
Apr 24 2008, 07:15
QUOTE(Nick.C @ Apr 24 2008, 14:58)

I am considering introducing a mechanism which would change the 3 by adding the quality-preset value divided by 4, i.e. at -q 10 subtract 5.5 rather than 3.0.
If I understand correctly, this would only matter for the really quiet parts (or tracks).
Your proposal would increase the bit rates overall (for tracks with quiet passages), how about to even that out by lowering the first constant to (e.g.) 2. -q 0 => 2+0, -5 => 2+1.25, -10 => 2+2.5 ?
Nick.C
Apr 24 2008, 07:22
QUOTE(GeSomeone @ Apr 24 2008, 14:15)

QUOTE(Nick.C @ Apr 24 2008, 14:58)

I am considering introducing a mechanism which would change the 3 by adding the quality-preset value divided by 4, i.e. at -q 10 subtract 5.5 rather than 3.0.
If I understand correctly, this would only matter for the really quiet parts (or tracks).
Your proposal would increase the bit rates overall (for tracks with quiet passages), how about to even that out by lowering the first constant to (e.g.) 2. -q 0 => 2+0, -5 => 2+1.25, -10 => 2+2.5 ?
I'm trying it, but - all of the recent ABX'ing has been done with a minimum of 3 bits kept - so I am reluctant to change the lower limit....
[edit] At -q 0: 2.0 = 306kbps; 2.5 = 310kbps; 2.75 = 313kbps; 3.0 (existing) = 318kbps.
Maybe this could be an advanced option instead, i.e. -minbits <n> would allow the user to add n bits to the minimum_bits_to_keep value at -q 10, n/2 at -q 5, 0 at -q 0, etc. [/edit]
GeSomeone
Apr 24 2008, 07:43
QUOTE(Nick.C @ Apr 24 2008, 15:22)

all of the recent ABX'ing has been done with a minimum of 3 bits kept - so I am reluctant to change the lower limit....
In my example only -q 0-3 would get just slightly lower minimum bits to keep, -q 4 and up would still have at least 3. It was just an idea, introduce more variability, but try not bloat the "default" bit rate without an ABXable reason.. (You can also change the constant to 2.25 or 2.5 if you think it is necessary)
Nick.C
Apr 24 2008, 07:46
QUOTE(GeSomeone @ Apr 24 2008, 14:43)

QUOTE(Nick.C @ Apr 24 2008, 15:22)

all of the recent ABX'ing has been done with a minimum of 3 bits kept - so I am reluctant to change the lower limit....
In my example only -q 0-3 would get just slightly lower minimum bits to keep, -q 4 and up would still have at least 3. It was just an idea, introduce more variability, but try not bloat the "default" bit rate without an ABXable reason.. (You can also change the constant to 2.25 or 2.5 if you think it is necessary)
Or, just allow the user to select a minimum-bits-to-keep between 0 and 8(?), defaulting to 3 for no user input?
2Bdecided
Apr 24 2008, 07:56
I think it does make sense to try to go with something like an existing quality scale, rather than inventing yet another one.
I was going to suggest keeping nts, but I thought that changes from the original algorithm meant that the effects of wild changes to nts were bounded by other parameters - so really, if you want a given effect, you either change all those paramters, or use a quality pre-set that does it for you. So I don't think nts has to stay in a stable release, if the quality pre-sets are well tested.
Cheers,
David.
GeSomeone
Apr 24 2008, 08:07
QUOTE(Nick.C @ Apr 24 2008, 15:46)

Or, just allow the user to select a minimum-bits-to-keep between 0 and 8(?), defaulting to 3 for no user input?
Alright with me, but that is not the same idea i.e. non scaling and an extra advanced setting.
halb27
Apr 24 2008, 10:53
QUOTE(Nick.C @ Apr 24 2008, 15:46)

[Or, just allow the user to select a minimum-bits-to-keep between 0 and 8(?), defaulting to 3 for no user input?
That's ok for me, too.
Now that the encoder has changed a bit I'd like to do another listening test. Because listening tests aren't so much fun I'd like to do this at a time where the encoder is not expected to change again before the final release.
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please
click here.