lossyWAV 1.2.0 released, Added noise WAV bitdepth reduction method Options

[Nick.C]

lossyWAV 1.2.0 is released.

lossyWAV is a near lossless audio processor which dynamically reduces the bitdepth of the signal on a block-by-block basis. Bitdepth reduction adds white noise to the processed output.

Changes from 1.1.0c:

• Now compatible with FFTW (requires libfftw3-3.dll to exist on the path);
• Spreading function changed;
• Noise shaping now off by default - can be switched on to automatic or user specified value;
• Introduction of --altpreset parameter which modifies internal presets and lowers upper frequency limit used in determining noise floor to 15.16kHz;
• Major code cleanup.

Download [maintenance release 1.2.0b]:

• Executable  lossyWAV_1.2.0b.zip ( 76.17K ) Number of downloads: 483
  
• Source  lossyWAV_Source_20101122_1.2.0b.zip ( 209.64K ) Number of downloads: 208
  

This post has been edited by Nick.C: Dec 2 2010, 21:48

[Nick.C]

CODE

lossyWAV 1.2.0, Copyright © 2007,2008,2009 Nick Currie. Copyleft.

This program is free software: you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation, either version 3 of the License, or (at your option) any later
version.

This program is distributed in the hope that it will be useful,but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.  See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with
this program.  If not, see <http://www.gnu.org/licenses/>.

Process Description:

lossyWAV is a near lossless audio processor which dynamically reduces the
bitdepth of the signal on a block-by-block basis. Bitdepth reduction adds white
noise to the processed output. The amount of permissible added noise is based
on analysis of the signal levels in the default frequency range 20Hz to 16kHz.

If signals above the upper limiting frequency are at an even lower level, they
can be swamped by the added noise. This is usually inaudible, but the behaviour
can be changed by specifying a different --limit (in the range 10kHz to 20kHz).

For many audio signals there is little content at very high frequencies and
forcing lossyWAV to keep the added noise level lower than the content at these
frequencies can increase the bitrate dramatically for no perceptible benefit.

Usage   : lossyWAV <input wav file> <options>

Example : lossyWAV musicfile.wav

Quality Options:

-I, --insane         highest quality output, suitable for transcoding;
-E, --extreme        high quality output, also suitable for transcoding;
-S, --standard       default quality output, considered to be transparent;
-P, --portable       good quality output for DAP use, not fully transparent.
-Z, --zero           lowest quality preset, probably contains artifacts.

Standard Options:

-C, --correction     write correction file for processed WAV file; default=off.
-f, --force          forcibly over-write output file if it exists; default=off.
-h, --help           display help.
-L, --longhelp       display extended help.
-M, --merge          merge existing lossy.wav and lwcdf.wav files.
-o, --outdir <t>     destination directory for the output file(s).
-v, --version        display the lossyWAV version number.
-w, --writetolog     create (or add to) lossyWAV.log in the output directory.

Advanced Options:

-                    if filename="-" then WAV input is taken from STDIN.
-t, --altpreset      enable alternative preset model which changes default
                     behaviour regarding limit and shaping (if selected).
-a, --analyses       set number of FFT analysis lengths, (3<=n<=5), default=2.
    --blockdist      show distribution of lowest / highest significant bit of
                     input codec-blocks and bit-removed codec-blocks.
-c, --check          check if WAV file has already been processed; default=off.
                     errorlevel=16 if already processed, 0 if not.
-i, --impulse        force use of additional shorter FFT analysis; default=off,
                     automatic for -q 3 and above.
-l, --limit <n>      set upper frequency limit to be used in analyses to n Hz;
                     (10000<=n<=20000), default=16000.
    --linkchannels   Revert to original single bits-to-remove value for all
                     channels rather than channel dependent bits-to-remove.
    --maxclips <n>   set max. number of acceptable clips per channel per codec
                     block. (0<=n<=16) Default = (3,3,3,3,2,1,0,0,0,0,0).
-m, --midside        analyse 2 channel audio for mid/side content.
-q, --quality <n>    quality preset (0.000<=n<=10.000); (10=highest, 0=lowest;
                     default = --standard = 5; --insane = 10; --extreme = 7.5;
                     --portable = 2.5; --zero = 0)
    --sampledist     show distribution of lowest / highest significant bit of
                     input samples and bit-removed samples.
    --scale <n>      scaling factor from WaveGain, etc; (0.0<n<=8.0),default=1.
-s, --shaping [n]    enable fixed noise shaping, automatic if no value input.
                     (0.00<=n<=1.00); automatic = q/10; 0.00 = off, 1.00 = 100%
                     effectiveness, 0.50 = 50%, etc.
    --stdout         write processed WAV output to STDOUT.
    --stdinname <t>  pseudo filename to use when input from STDIN.
-U, --underlap <n>   enable underlap mode to increase number of FFT analyses
                     performed at each FFT length, (2<=n<=8); default=2.
-X, --sortspread     enable sort based spreading; default=off.

System Options:

-B, --below          set process priority to below normal.
-d, --detail         enable detailed bits-to-remove information output mode
    --low            set process priority to low.
-n, --nowarnings     suppress lossyWAV warnings.
-Q, --quiet          significantly reduce screen output.
    --silent         no screen output.

Special thanks go to:

David Robinson       for the publication of his lossyFLAC method, guidance, and
                     the motivation to implement his method as lossyWAV.

Horst Albrecht       for ABX testing, valuable support in tuning the internal
                     presets, constructive criticism and all the feedback.

Sebastian Gesemann   for the noise shaping coefficients and help in using them
                     in the lossyWAV noise shaping implementation.

Matteo Frigo and     for the excellent libfftw3-3.dll contained in the FFTW
Steven G Johnson     distribution (v3.2.1 or v3.2.2).

Mark G Beckett       for the Delphi unit that provides an interface to the
(Univ. of Edinburgh) relevant fftw routines in libfftw3-3.dll.

Don Cross            for the Complex-FFT algorithm originally used.

Link to the hydrogenaudio wiki article

Suggested foobar2000 converter setup:

lossyFLAC:

CODE

Encoder: c:\windows\system32\cmd.exe
Extension: lossy.flac
Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --stdout|c:\"program files"\bin\flac - -b 512 -5 -f -o%d
Format is: lossless or hybrid
Highest BPS mode supported: 24

lossyTAK:

CODE

Encoder: c:\windows\system32\cmd.exe
Extension: lossy.tak
Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --stdout|c:\"program files"\bin\takc -e -p2m -fsl512 -ihs - %d
Format is: lossless or hybrid
Highest BPS mode supported: 24

lossyWV:

CODE

Encoder: c:\windows\system32\cmd.exe
Extension: lossy.wv
Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --stdout|c:\"program files"\bin\wavpack -hm --blocksize=512 --merge-blocks -i - %d
Format is: lossless or hybrid
Highest BPS mode supported: 24

Enclose the element of the path containing spaces within double quotation marks ("), e.g. C:\"Program Files"\directory_where_executable_is\executable_name. This is a Windows limitation.


This post has been edited by Nick.C: Apr 8 2011, 12:51

Reason for edit: Failed attempt at assigning a codebox.

[halb27]

Thank you very much for the new release, Nick.

[shadowking]

Thank you guys.

BTW: i encoded a fiona apple cd and -P -t was slightly larger than old -P. Is this normal ? I thought -t was more aggressive to save a few bits. In any case i am happy with old -P.

[TBeck]

Great work! Thank you.

BTW: i encoded a fiona apple cd and -P -t was slightly larger than old -P. Is this normal ? I thought -t was more aggressive to save a few bits. In any case i am happy with old -P.

The same here with my primary test corpus of 46 (probably problem) files.

[Nick.C]

The quality scale for --altpreset basically extends the range of the default -q 0 to 5 over the range -q -4 to 5. As --portable equates to -q 2.5 in both, -q 2.5 --altpreset (sort of) equates to -q 2.9999 in the default range (there is a bit of a jump at -q 3 as the impulse detection mechanism kicks in....).

This post has been edited by Nick.C: Dec 17 2009, 14:58

[halb27]

Great work! Thank you.

BTW: i encoded a fiona apple cd and -P -t was slightly larger than old -P. Is this normal ? I thought -t was more aggressive to save a few bits. In any case i am happy with old -P.

The same here with my primary test corpus of 46 (probably problem) files.

For problem samples it is very welcome if bitrate goes up in a noticeable way.
For 'normal' samples bitrate of -P should be pretty close to each other when comparing plain vanilla -P against -P --altpreset.
-P --altpreset is more demanding in keeping noise level low in the range up to 15.2 kHz, but doesn't care about noise in the 15.2 ... 16 kHz range which plain vanilla -P does.

This post has been edited by halb27: Dec 17 2009, 21:11

[skamp]

From my point of view as a user, lossyWAV is one of the most innovative audio compression techniques in recent years. The fact that it can (and should) be used with existing lossless codecs is genius. Thank you.

[2Bdecided]

one of the most innovative audio compression techniques in recent years

That's going straight onto my CV!

Can me and Nick call you for references?


(joking!)

Cheers,
David.

[TBeck]

"Honour to whom honour is due"

(I am proud to be able to find answers in my favourite dictionary...)

Joke apart: I regard LossyWav as something really elegant.

This post has been edited by TBeck: Dec 20 2009, 00:42

[zorzescu]

As a title paraphrase of the well known song of Budgie, lossyWAV is the biggest thing since powdered milk

[Steve Forte Rio]

For me "--zero --altpreset" sounds much better (than without altpreset switch), and I don't hear high noise and distortion.

Maybe it would be useful to enable --altpreset by default for high compression modes??

[Nick.C]

--zero --altpreset equates to approximately -q 1.63 --limit 15159 in the default scale. This is due to the scale stretch mentioned above.

[shadowking]

I have some trouble with mp3 transcoding. Last month i compared original V5 mp3 vs lossywav -P V5 mp3 . I abxed 5/5 several times on nearly every sample i tried. The problem wasn't always noticeable but sound very peculiar - like an artifact. Today i tried again with v1.2 -q 2.5, 3.0 and 1.5. I found a passage again and abxed 5/5, the narrowed it down and got 8/8

At -q 4 i didn't abx it. Not sure what is happening or what quality level is needed. With wv lossy 300k range its generaly much harder. I remember Bryant mentioning lossywav noise goes in steps of 6db - could that be problem for transcoding ?

[GeSomeone]

At -q 4 i didn't abx it. Not sure what is happening or what quality level is needed.

maybe you have established that lossyWav -q 4 is needed for your transcodings. Keep in mind that lossyWav was never specifically tuned for transcoding. It is just hoped that it has an advantage to other lossy->lossy transcoding. All depends on how the last encoder handles the added noise...

With lossyWav, due to the tuning that has been done, every release you must again find your "sweetspot" -q value. And now there is also the alternative setting with --altpreset, that has a subtle different range.
I'm not complaining, it's progress .

This post has been edited by GeSomeone: Dec 23 2009, 14:29

[halb27]

Thank you, shadowking, for ABXing.
Not very good news, but valuable experience of lossyWAV behavior.
Do you mind publishing the sample and tell us the bad spot of the sample, shadowking?
I'd like to learn about transcoding quality whether it depends on -V.
Can you ABX the mp3 encoding of say -P --altpreset when you choose a better mp3 quality like -V2?

This post has been edited by halb27: Dec 21 2009, 14:14

[shadowking]

Thank you, shadowking, for ABXing.
Not very good news, but valuable experience of lossyWAV behavior.
Do you mind publishing the sample and tell us the bad spot of the sample, shadowking?
I'd like to learn about transcoding quality whether it depends on -V.
Can you ABX the mp3 encoding of say -P --altpreset when you choose a better mp3 quality like -V2?

Better news: I did more testing yesterday and as per your suggestion, I found out that the main problem was V5 itself - the samples i tested were already ruined by mp3 artifacts. LW didn't add any artifact that wasn't already there though the amplitude of it can change slightly. This is normal.

I tested V4 this time with plain Q1.5, 2.5, 3.5 without NS and things were way better. Q2.5 and over were fine. Q1.5 was abxable in one segment (lame V4 in itself wasn't fully transparent) but q1.5 added a bit of intensity to it. On subsequent trials in was somewhat harder to abx that part. Overall 1.5 was also very good - i test it as a 'low anchor'. Also tested Nero AAC q0.5 with lw Q2.5 and everything seemed transparent.

I'll do more tests soon and report them here.

[Admiral Oggbar]

From the Wiki:

Question: Why create a processor which means that I cannot be sure that a lossless file is truly lossless?
Answer: Unless one creates the lossless file personally, one can never be completely sure that the file is indeed lossless. e.g. If a WAV file is encoded to MP3 and then transcoded to a lossless codec, how can this pre-processing be easily determined?



What kind of circular gibberish is this? That's not an answer. It's false logic, and a poor attempt at justifying the existence of this particular software.

How about an honest answer to that question?

This post has been edited by Admiral Oggbar: Dec 27 2009, 01:30

[singaiya]

What kind of answer do you expect to this question? First, I don't think the answer needs to "justify" lossywav's existence. Second, nobody can ever be exactly sure what processing has taken place before the input, whether a given processor's output is WAV (lossy), WAV (lossless), mp3, or other.

If anything, the premise of the question is faulty; not the answer.

[Nick.C]

What kind of circular gibberish is this? That's not an answer. It's false logic, and a poor attempt at justifying the existence of this particular software.

How about an honest answer to that question?

Which bit of the answer was not "honest" (in your interpretation)?

An early criticism of lossyFLAC / lossyWAV was that lossy output was by design to be contained in a lossless container. "How do we tell the difference?" came the cry.... "Make it yourself." was the response.

There are ways of determining whether audio has been processed by lossyWAV (if you know where and how to look) - I am content that those who use lossyWAV will know which of their files are lossless and which are lossy.

[hellokeith]

An early criticism of lossyFLAC / lossyWAV was that lossy output was by design to be contained in a lossless container. "How do we tell the difference?" came the cry.... "Make it yourself." was the response.

Nick, I'm w/ Singaiya on this one. The point presumes that there may be something "wrong" with processsing/encoding/compression - which is a false pretense - and then doesn't give a thorough explanation to the counter. Personally, I don't think it belongs in the wiki, and I don't think you should even have to answer that question. If someone is that worried about their sources, then they need to go rob mixing studios of the masters.

[halb27]

Better news: ... Q2.5 and over were fine. ...

Thank you, shadowking. Though the --portable quality level does not necessarily have the property that transcoding to mp3 is fine it would have been a little bit disappointing.

[2Bdecided]

Better news: ... Q2.5 and over were fine. ...

Thank you, shadowking. Though the --portable quality level does not necessarily have the property that transcoding to mp3 is fine it would have been a little bit disappointing.

I think Nick's performed a minor miracle!

(Pre-processing with my original implementation changed the already audible artefacts on some samples when encoded to mp3 from much higher "lossyWAV" bitrates than you're seeing now.)

Cheers,
David.

[2E7AH]

This kind of command parameters use, can be frustrating and rejecting:

You can create a batch file, say lossyFLAC.bat, in your FLAC folder with this contents:

c:\programme\flac\lossyWAV.exe %1 %3 %4 %5 %6 %7 %8 %9 --below --silent
ren "%~N1.lwcdf.wav" "%~N2.lwcdf.wav"
c:\programme\flac\flac.exe -b 512 -l 6 -e -m -r 2 -f --totally-silent "%~N1.lossy.wav" -o"%~N2.flac"
del "%~N1.lossy.wav"

Change the paths of lossyWAV.exe and flac.exe according to your system.

In foobar, Command Line Encoder Settings, you choose this lossyFLAC.bat as your encoder.
As parameters you use %s %d --portable -C (resp. your quality setting).

Is this working in reality for someone?

[Nick.C]

Try:

Encoder: C:\windows\system32\cmd.exe
Extension: lossy.flac
Parameters: /d /c c:\"program files"\lossyflac.bat %s %d --standard --correction

I would also suggest that you compress the correction file by adding this line to the bottom of your batch file:

CODE

c:\programme\flac\flac.exe -b 512 -l 6 -e -m -r 2 -f --totally-silent "%~N1.lwcdf.wav" -o"%~N2.lwcdf.flac"
del "%~N1.lwcdf.wav"