lossyWAV 1.2.0 Development Thread, Added noise WAV bitdepth reduction method Options

[monoton]

Hello... is there any possibility to force even greater quality loss for lossy wav? Actually I'm yet having a hard time to spot the difference even in the "awful" setting (when using "-q 0" for encoding) though I can spot artifacts on mp3-files. I guess its "noise" (reduction) is a bit different. Generally I'd wonder how the signal would sound like with really low quality settings.

[Nick.C]

The lowest quality setting is actually -q -4 or -A or --awful.

The artifacts are *very* obvious.

[monoton]

Ok.. I used jLossyWav before and that didn't accept negative numbers for quality settings.

Now that I've got the original lossyWav version running I can see the difference. (I would have appreciated some link or hint info in the "use with foobar2000" section of the wiki for some noob like me what to install and set where though, that've saved me an hour or two...)

However, also partly to my defense, even with these settings my first test track wasn't so easy to identify as my other. I've firstly tried it on AC/DC's "Smash N Grab" (Black Ice) and now that I know from the second, orchestral track, where it was clearly audible, I'm quite astonished how good it still sounded!

I think I have two explanations for that: even with -q -4 ACDC was "only" compressed to around 25% of its original size, the orchestral track went to 14% of original size. The other thing is of the nature of these songs: ACDC at making their best to give loudness war more fuel and the orchestral, while still being heavy for an orchestral track, but slight or no compression applied.

[monoton]

Hmm... I tried to create the correction file, but it gives me an error:

"Conversion failed: The encoder has terminated prematurely with code 1; please re-check parameters"
... lossywav - -A -f -correction --silent --stdout ...

is what I'm using with foobar2000. Am I doing something wrong here? It neither works with --correction nor -C (I saw the caps) but works if I omit -C or --correction and not include these parameters.

Thanks for your help in advance!

[Nick.C]

Creation of the correction file is incompatible with piped output (as the two output WAV files require to be written simultaneously).

[halb27]

Thank you Nick, for version 1.1.3e.
I didn't want to do a listening test now, but just compared the results of 1.1.3d -P -V with those of 1.1.3e -P.
I was a bit astonished that the results weren't bit-identical, but looking at the blockwise bits to remove of a few short tracks I saw that there is nearly no difference and the new version was always more defensive (in case of differences).
I can also confirm that using -a n makes things blockwise more defensive with increasing n - as should be.
So everything is fine from this small test.

This post has been edited by halb27: Mar 19 2009, 18:42

[Nick.C]

I made a very small (and conservative) change to the revised spreading function (it increases the resultant FLAC bitrate by less than 1kbps) at the last minute.

[monoton]

Hey there... can you perhaps tell me the parameter setup to get the correction files with foobar2000? I'm quite new to this and I couldn't find it in the documentation or work it out by try and fail yet...

Thanks for your help and all your efforts on this!

[halb27]

Hey there... can you perhaps tell me the parameter setup to get the correction files with foobar2000? I'm quite new to this and I couldn't find it in the documentation or work it out by try and fail yet...

Thanks for your help and all your efforts on this!

You can create a batch file, say lossyFLAC.bat, in your FLAC folder with this contents:

c:\programme\flac\lossyWAV.exe %1 %3 %4 %5 %6 %7 %8 %9 --below --silent
ren "%~N1.lwcdf.wav" "%~N2.lwcdf.wav"
c:\programme\flac\flac.exe -b 512 -l 6 -e -m -r 2 -f --totally-silent "%~N1.lossy.wav" -o"%~N2.flac"
del "%~N1.lossy.wav"

Change the paths of lossyWAV.exe and flac.exe according to your system.

In foobar, Command Line Encoder Settings, you choose this lossyFLAC.bat as your encoder.
As parameters you use %s %d --portable -C (resp. your quality setting).

[Nick.C]

I would add compression of the .lwcdf.wav file if it exists:

CODE

c:\programme\flac\lossyWAV.exe %1 %3 %4 %5 %6 %7 %8 %9 --below --silent
if exist "%~N1.lwcdf.wav" c:\programme\flac\flac.exe -b 512 -l 6 -e -m -r 2 -f --totally-silent --keep-foreign-metadata "%~N1.lwcdf.wav" -o"%~N2.lwcdf.flac"&&del %~N1.lwcdf.wav
c:\programme\flac\flac.exe -b 512 -l 6 -e -m -r 2 -f --totally-silent  --keep-foreign-metadata "%~N1.lossy.wav" -o"%~N2.flac"
del "%~N1.lossy.wav"

[monoton]

Thank you both! That worked well! My knowledge of batch programming is pretty limited But I could work out for most what was meant and added the lwcdf conversion already.

[cBc]

Hello there
I've been using lossyWAV exclusively to create music files for my DAP (rockboxed Sansa e280) and the reduction in filesize while retaining SQ is quite amazing.

I am using this batch file to drop my archived FLACs onto, would someone be so kind as to advise if I have it optimized for use with the latest build 1.1.3e :

@echo off
:repeat
if %1.==. goto end
if exist %1 g:\lossyWAV\flac -d %1 --stdout --silent|g:\lossyWAV\lossywav - --stdout -P --limit 17500 --shaping 0 --stdinname %1|g:\lossyWAV\flac - -b 512 -o "%~dpn1.lossy.flac" --silent && g:\lossyWAV\tag --fromfile %1 "%~dpn1.lossy.flac"
shift
goto repeat
:end


I am sadly not very knowledgeable with batchfiles , and wish to make sure I have the setting correct. Thanks to Nick & all who are helping on this project.

[Nick.C]

Glad that you are happy with lossyWAV.

With respect to your batch file, if you are trying to:

• process using the --portable preset;
• remove shaping and
• increase the calculation range upper limit to 17.5kHz

then, that's what your batch file will do.

Is there any particular reason why you are removing shaping?

This post has been edited by Nick.C: Mar 29 2009, 21:20

[cBc]

Glad that you are happy with lossyWAV.

With respect to your batch file, if you are trying to:

• process using the --portable preset;
• remove shaping and
• increase the calculation range upper limit to 17.5kHz

then, that's what your batch file will do.

Is there any particular reason why you are removing shaping?

hmm..honestly..no. I may have copied some of the parameters from one of the other posts in here. So what I have doesn't sound like what I would need
for best possible SQ using --portable . What would you recommend I change it to?

Thanks again!

[sauvage78]

I dunno if it's of any use but Abfahrt Hinwil (eig) is ABXable with lossyWAV beta 1.1.3e -q 1, the artefact is like a little synthetizer effect in the background. It doesn't really sound bad but it is obvious at -q 1.

foo_abx 1.3.3 report
foobar2000 v0.9.6.4 beta 2
2009/03/30 05:36:02

File A: D:\02- Artefact Only\01- Abfahrt Hinwil (Artefact Only) Lossless.flac
File B: D:\02- Artefact Only\01- Abfahrt Hinwil (Artefact Only).lossy.flac

05:36:02 : Test started.
05:36:30 : 01/01 50.0%
05:36:42 : 02/02 25.0%
05:36:53 : 03/03 12.5%
05:37:04 : 04/04 6.3%
05:37:20 : 05/05 3.1%
05:37:35 : 06/06 1.6%
05:37:57 : 07/07 0.8%
05:38:13 : 08/08 0.4%
05:38:15 : Test finished.

----------
Total: 8/8 (0.4%)

I tried at -q1.5 & -q2, I had 8/8 & 10/12 but the artefact was gone. I was guessing if it was here & I think I was lucky. I would say the artefact is medium at -q1, light at -q1.5, gone at -q2 & up. It seems to confirm my previous test (Old Test) for me the point of transparency of lossywav is near -q 2 based on this two samples. If I would spend hours on it maybe I could ABX -q 2 on this two samples, but I think it's a job for a madman as the result would be based on subliminal feelings more than on rationnal listening.

I report because it's the only killer sample that I tested in my DCT listening test that also affect lossywav. I tested them all at -q1.

Nothing to worry about -P is still safe for me ...

This post has been edited by sauvage78: Mar 30 2009, 05:57

[Nick.C]

@sauvage78 - thanks for the identification of another problem sample and also your ABX'ing to determine where the transparency point is. If you could link to the sample that would be useful.

@cBc: for DAP use, I would suggest simply:

g:\lossyWAV\lossywav - --stdout -P --stdinname %1

[edit] as the lossyWAV related element of the relevant line in your batch file. [/edit]

This post has been edited by Nick.C: Mar 30 2009, 07:29

[sauvage78]

The Abfahrt Hinwil sample is available here Sample. Grab the lossless version of the sample pack (I tested the short version).

The artefact is not the same as DCT codecs, DCT codecs have problem with the little explosions. Lossywav doesn't, it's just an added noise. Test at -q 1 or below cause at -q1.5 the artefact is so low that you will miss it. I think I had some success with my guessing at -q1.5, only because I knew where to suspect a variation thks to -q 1. So far for me lossywav becomes transparent between -q1.5 & -q2. -q2 is a pain to ABX, it's like Nero AAC Q0.55, but even harder.

Edit:
Here is log for -q 1.5, I don't have the log for -q 2.0 as I didn't save it. It is missleading as I only have good results while I couldn't really spot an artefact. What is important to notice in it is that
a small quality increase lead me to double the ABX time for the same result as -q 1. So -q1.5 was already much harder to ABX than -q 1.

foo_abx 1.3.3 report
foobar2000 v0.9.6.4 beta 2
2009/03/30 05:57:37

File A: D:\01- Abfahrt Hinwil (Artefact Only) Lossless.flac
File B: D:\01- Abfahrt Hinwil (Artefact Only).lossy.flac

05:57:37 : Test started.
05:59:18 : 01/01 50.0%
05:59:47 : 02/02 25.0%
06:00:10 : 03/03 12.5%
06:00:27 : 04/04 6.3%
06:00:46 : 05/05 3.1%
06:01:10 : 06/06 1.6%
06:01:38 : 07/07 0.8%
06:02:01 : 08/08 0.4%
06:02:03 : Test finished.

----------
Total: 8/8 (0.4%)

This post has been edited by sauvage78: Mar 30 2009, 08:17

[Nick.C]

Thanks, got them. One thing that you might try (if you have the time) is adding extra analyses at the same quality level (--analyses <n>; 2<=n<=5) to process using additional FFT lengths. My gut feeling is that this will find more of the dips in the noise floor that cause issues with problem samples.

[cBc]

@cBc: for DAP use, I would suggest simply:

g:\lossyWAV\lossywav - --stdout -P --stdinname %1

[edit] as the lossyWAV related element of the relevant line in your batch file. [/edit]

Perfect! Thank you for the help.

[sauvage78]

Just tried on the Therion/Ginnungagap problem sample at -q 0, (I will call it Therion now as it comes from this band/album: Therion - 2001 - Secret of The Runes - 01- Ginnungagap (The Black Hole) (Prologue) & I have the full album). This -a 5 thing is a real improvement, I was instantly able to ABX a positive gain for only a 19ko difference !!! The artefact is still here & very ABXable compared to the original, but if you compare -q 0 with & without -a 5, the artefact is much softer because the added noise is much more stable. From what I heard, this -a 5 thing should be used as default.

foo_abx 1.3.3 report
foobar2000 v0.9.6.4 beta 2
2009/03/31 08:12:35

File A: D:\a00- Therion (Artefact+Context) Lossless -q 0 -a 5.lossy.flac
File B: D:\a00- Therion (Artefact+Context) Lossless -q 0.lossy.flac

08:12:35 : Test started.
08:13:06 : 01/01 50.0%
08:13:24 : 02/02 25.0%
08:13:47 : 03/03 12.5%
08:14:54 : 04/04 6.3%
08:15:14 : 05/05 3.1%
08:16:02 : 06/06 1.6%
08:16:20 : 07/07 0.8%
08:16:52 : 08/08 0.4%
08:16:53 : Test finished.

----------
Total: 8/8 (0.4%)

Same thing for Abfahrt Hinwil, with same settings. The artefact is not more stable here as it is not noisy like the Therion sample, but it's like as if the volume level of the artefact was reduced within the sample. In the end, it's the same: it's much better & easy to ABX for a 4ko difference !!! On this sample it is possible that this -a 5 thing alone lower the transparency point by 0.5 for me. I should test at -q 1 -a 5, to see if the artefact is still here.

foo_abx 1.3.3 report
foobar2000 v0.9.6.4 beta 2
2009/03/31 08:32:32

File A: D:\a01- Abfahrt Hinwil (Artefact Only) -q 0.lossy.flac
File B: D:\a01- Abfahrt Hinwil (Artefact Only) -q 0 -a 5.lossy.flac

08:32:32 : Test started.
08:32:48 : 01/01 50.0%
08:33:06 : 02/02 25.0%
08:33:27 : 03/03 12.5%
08:33:41 : 04/04 6.3%
08:33:56 : 05/05 3.1%
08:34:16 : 06/06 1.6%
08:34:28 : 07/07 0.8%
08:34:48 : 08/08 0.4%
08:34:50 : Test finished.

----------
Total: 8/8 (0.4%)

Edit:
Just tested at Abfahrt Hinwil -q 1 -a 5 vs. -q 1, unfortunatly even reduced the artefact was still here, so I compared -q 1 -a 5 vs. -q 1.5, -q 1.5 is better, so for me the improvement can be evaluated as a 0.25 direct gain within the quality scale. Personnaly I think it's a great gain.

This post has been edited by sauvage78: Mar 31 2009, 08:09

[Nick.C]

Many thanks for the additional ABX testing of the effect of --analyses 5 on these problem samples.

I have been thinking about a modification to the method which reduces the influence of the adjacent codec-blocks on the bits-to-remove for the codec-block being analysed. For 44.1kHz auido, at the moment two 1024 sample FFT analyses are carried out per 512 sample codec-block: -512..511 samples and 0..1023 samples (Samples -512..-1 are in the previous codec-block; 0..511 are the current codec-block and 512..1023 are the next codec-block). This basically takes all of the previous codec-block and the next codec-block into account when analysing the current codec-block. Changing this behaviour so that the overlap outwith the current codec-block is restricted to 50% of the codec-block length rather than 50% of the FFT sample length results in a single analysis: -256 to 768 samples which changes the overall bits-to-remove (some plusses, some minuses compared to the previous method).

This was previously evaluated and discarded but I think too many things were changing together at that time to fully appreciate the impact of the single central 1024 sample FFT analysis.

[halb27]

... Personnaly I think it's a great gain.

Thank you very much for testing and having found -a 5 useful. Looks like I have to change my mind towards the additional analyses.

[Nick.C]

It does make a difference.

However, in addition to the central FFT proposal above I have been thinking (again) about the underlap of the overlapping FFT analyses that are carried out. At present the FFT analyses overlap by 50%, i.e. underlap is also 50%. This behaviour can be modified to reduce to the underlap length to any proportion but sensibly I would limit this to 25%; 12.5%; 6.25%; etc. i.e. length/4; length/8; length/16; etc.

All this does is add extra analyses between those already calculated. This can be performed with or without the single central 1024 sample FFT analysis proposed above.

I will work up beta v1.1.3f with a --centre parameter to enable the single central 1024 sample (or 2 x codec-block length for higher than standard sample rates) FFT analysis and a --underlap <n> parameter where 1<=n<=(yet to be determined) and the underlap length will be FFT length / (2^n).

This should be posted later today.

[sauvage78]

np, I will test lossywav even more in the future if needed, I was away from it as I was playing with blu-ray lately & lossywav is not very suited for AC3 transcoding (& AC3 is crap anyway) ... I thougth for sometimes that I would use vorbis specially as I liked its recent 5.1 improvement, but my listening test disgusted me from vorbis for a long time I think ... It was not the expected affect ... but I want a codec that is free of artefact at high bitrate, open sourced & easyly splitable. Lossywav is the only codec which has all this features. So I am back using it, despite the big bitrate.

Nick:
you pm about testing a -v option (if I recall well), a month ago, I told you that I didn't had the time, if ever this thing still need testing, just explain what you want me to test. I deleted the pm & as I wasn't following the development, I don't even know what it was supposed to do.

Edit: I must say I don't understand a word of the technical discussion, just code it if you think it can help on paper, I'll listen if it works in reality

This post has been edited by sauvage78: Mar 31 2009, 08:44

[Nick.C]

The -v (--varspread) parameter has been integrated with the --oldspread parameter as the revised (and slightly more conservative) spreading function.

Thanks for the offer - but it's already been finalised in v1.1.3e.

I will appreciate greatly any time that you can devote to testing of the --centre and --underlap parameters.