lossyWAV 1.3.0 released, Added noise WAV bitdepth reduction method. Options

[Nick.C]

lossyWAV 1.3.0 is released.

lossyWAV is a near lossless audio processor which dynamically reduces the bitdepth of the signal on a block-by-block basis. Bitdepth reduction adds noise to the processed output. The added noise is adaptively shaped by default and can alternatively be fixed noise shaped or white noise depending on command line parameters. When lossyWAV processed output is compressed with certain lossless codecs (FLAC, Wavpack, Tak, LPAC, MPEG-4 ALS and WMA-Lossless) the bitrate of the output file is significantly* reduced compared to the lossless original.

Changes from 1.2.0b:

• Adaptive noise shaping method kindly provided by SebastianG introduced and enabled by default;
• Fixed noise shaping reworked to allow effective shaping of samplerates other than 44.1kHz / 48kHz.
• Revised quality presets;
• Enhanced information output options;
• Major code cleanup.

Download

• Executable lossyWAV_1.3.0.zip
• Source lossyWAV_Source_20110806_1.3.0.zip

* on average, depending on content.

This post has been edited by Nick.C: Aug 10 2011, 21:12

[Nick.C]

CODE

lossyWAV 1.3.0, Copyright (C) 2007-2011 Nick Currie. Copyleft.

This program is free software: you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation, either version 3 of the License, or (at your option) any later
version.

This program is distributed in the hope that it will be useful,but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.  See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with
this program.  If not, see <http://www.gnu.org/licenses/>.

Process Description:

lossyWAV is a near lossless audio processor which dynamically reduces the
bitdepth of the signal on a block-by-block basis. Bitdepth reduction adds noise
to the processed output. The amount of permissible added noise is based on
analysis of the signal levels in the default frequency range 20Hz to 16kHz.

If signals above the upper limiting frequency are at an even lower level, they
can be swamped by the added noise. This is usually inaudible, but the behaviour
can be changed by specifying a different --limit (in the range 10kHz to 20kHz).

For many audio signals there is little content at very high frequencies and
forcing lossyWAV to keep the added noise level lower than the content at these
frequencies can increase the bitrate dramatically for no perceptible benefit.

The noise added by the process is shaped using an adaptive method provided by
Sebastian Gesemann. This method, as implemented in lossyWAV, aims to use the
signal itself as the basis of the filter used for noise shaping. Adaptive noise
shaping is enabled by default.

Usage   : lossyWAV <input wav file> <options>

Example : lossyWAV musicfile.wav

Quality Options:

-q, --quality <t>    where t is one of the following (default = standard):
    I, insane        highest quality output, suitable for transcoding;
    E, extreme       higher quality output, suitable for transcoding;
    H, high          high quality output, suitable for transcoding;
    S, standard      default quality output, considered to be transparent;
    C, economic      intermediate quality output, likely to be transparent;
    P, portable      good quality output for DAP use, may not be transparent;
    X, extraportable lowest quality output, not fully transparent.

Standard Options:

-C, --correction     write correction file for processed WAV file; default=off.
-f, --force          forcibly over-write output file if it exists; default=off.
-h, --help           display help.
-L, --longhelp       display extended help.
-M, --merge          merge existing lossy.wav and lwcdf.wav files.
-o, --outdir <t>     destination directory for the output file(s).
-v, --version        display the lossyWAV version number.
-w, --writetolog     create (or add to) lossyWAV.log in the output directory.

Advanced Options:

-                    take WAV input from STDIN.
-c, --check          check if WAV file has already been processed; default=off.
                     errorlevel=16 if already processed, 0 if not.
-q, --quality <n>    quality preset (-5.0<=n<=10.0); (-5=lowest, 10=highest;
                     default=2.5; I=10; E=7.5; H=5; S=2.5; C=0; P=-2.5; X=-5).
--, --stdout         write WAV output to STDOUT.
    --stdinname <t>  pseudo filename to use when input from STDIN.

Advanced Quality Options:

-A, --adaptive <n/t> modify settings for Sebastian Gesemann's adaptive noise
                     shaping method. takes a parameter to set the order of the
                     FIR filter, (32<=n<=96; default=64; multiple of 8 only);
                     "OFF" to disable adaptive shaping; "NOWARP" to disable
                     default frequency warping;
-a, --analyses <n>   set number of FFT analysis lengths, (2<=n<=6; default=3,
                     i.e. 32, 64 & 1024 samples. n=2, remove 32 sample FFT;
                     n>3 add 512; n>4, add 256; n>6, add 128) nb. FFT lengths.
                     stated are for 44.1/48kHz audio, higher sample rates will
                     automatically increase all FFT lengths as required.
-l, --limit <n>      set upper frequency limit to be used in analyses to n Hz;
                     (10000<=n<=20000; default=16000).
    --linkchannels   revert to original single bits-to-remove value for all
                     channels rather than channel dependent bits-to-remove.
    --maxclips <n>   set max. number of acceptable clips per channel per block;
                     (0<=n<=16; default=3,3,3,3,3,2,2,2,2,2,1,1,1,0,0,0).
-m, --midside        analyse 2 channel audio for mid/side content.
    --nodccorrect    disable DC correction of audio data prior to FFT analysis,
                     default=on; (DC offset calculated per FFT data set).
    --scale <n>      factor to scale audio by; (0.0625<n<=8.0; default=1).
-s, --shaping [n]    enable fixed noise shaping, takes optional parameter [n]
                     to allow user defined shaping proportion (0.0<=n<=1.0),
                     otherwise default to quality setting dependent value.
                     Disables adaptive noise shaping.
    --static <n>     set minimum-bits-to-keep-static to n bits (default=6;
                     7<=n<=28, limited to bits-per-sample - 4).
-U, --underlap <n>   enable underlap mode to increase number of FFT analyses
                     performed at each FFT length, (n = 2, 4 or 8, default=2).

Output Options:

    --bitdist        show distrubution of bits to remove.
    --blockdist      show distribution of lowest / highest significant bit of
                     input codec-blocks and bit-removed codec-blocks.
-d, --detail         enable per block per channel bits-to-remove data display.
-F, --freqdist       enable frequency analysis display of input data.
-H, --histogram      show sample value histogram (input, lossy and correction).
    --longdist       show long frequency distribution data (input/lossy/lwcdf).
    --perchannel     show selected distribution data per channel.
-p, --postanalyse    enable frequency analysis display of output and
                     correction data in addition to input data.
    --sampledist     show distribution of lowest / highest significant bit of
                     input samples and bit-removed samples.
    --spread [full]  show detailed [more detailed] results from the spreading/
                     averaging algorithm.
-W, --width <n>      select width of output options (79<=n<=255).

System Options:

-B, --below          set process priority to below normal.
    --low            set process priority to low.
-N, --nowarnings     suppress lossyWAV warnings.
-Q, --quiet          significantly reduce screen output.
-S, --silent         no screen output.

Special thanks go to:

David Robinson       for the publication of his lossyFLAC method, guidance, and
                     the motivation to implement his method as lossyWAV.

Horst Albrecht       for ABX testing, valuable support in tuning the internal
                     presets, constructive criticism and all the feedback.

Sebastian Gesemann   for the adaptive noise shaping method and the amount of
                     help received in implementing it and also for the basis of
                     the fixed noise shaping method.

Matteo Frigo and     for libfftw3-3.dll contained in the FFTW distribution
Steven G Johnson     (v3.2.1 or v3.2.2).

Mark G Beckett       for the Delphi unit that provides an interface to the
(Univ. of Edinburgh) relevant fftw routines in libfftw3-3.dll.

Don Cross            for the Complex-FFT algorithm originally used.

Link to the hydrogenaudio wiki article

Suggested foobar2000 converter setup:

lossyFLAC:

CODE

Encoder: c:\windows\system32\cmd.exe
Extension: lossy.flac
Parameters: /d /c c:\"program files"\bin\lossywav - --quality standard --silent --stdout|c:\"program files"\bin\flac - -b 512 -5 -f -o%d --ignore-chunk-sizes
Format is: lossless or hybrid
Highest BPS mode supported: 24

lossyTAK:

CODE

Encoder: c:\windows\system32\cmd.exe
Extension: lossy.tak
Parameters: /d /c c:\"program files"\bin\lossywav - --quality standard --silent --stdout|c:\"program files"\bin\takc -e -p2m -fsl512 -ihs - %d
Format is: lossless or hybrid
Highest BPS mode supported: 24

lossyWV:

CODE

Encoder: c:\windows\system32\cmd.exe
Extension: lossy.wv
Parameters: /d /c c:\"program files"\bin\lossywav - --quality standard --silent --stdout|c:\"program files"\bin\wavpack -hm --blocksize=512 --merge-blocks -i - %d
Format is: lossless or hybrid
Highest BPS mode supported: 24

lossyWMALSL*:

CODE

Encoder: c:\windows\system32\cmd.exe
Extension: lossy.wma
Parameters: /d /c c:\"program files"\bin\lossywav - --quality standard --silent --stdout|c:\"program files"\bin\wmaencode.exe - %d --codec lsl --ignorelength
Format is: lossless or hybrid
Highest BPS mode supported: 24

Enclose the element of the path containing spaces within double quotation marks ("), e.g. C:\"Program Files"\directory_where_executable_is\executable_name. This is a Windows limitation.

*: Uses the wmaencoder executable written by lvqcl. Found here.

This post has been edited by Nick.C: Sep 4 2011, 21:20

[Steve Forte Rio]

BRAVO!

[halb27]

Great work! Thank you, Nick.

[TBeck]

Really cool! Thank you!

[punkrockdude]

Nick.C: Do you play FLAC/LossyWAV with your Android HD2? If so, what application do you use to play them? Regards.

[vvneagleone]

Nice work! For what purposes do you think this will be most commonly used?

[Nick.C]

Nick.C: Do you play FLAC/LossyWAV with your Android HD2? If so, what application do you use to play them? Regards.

I use deadbeef - it's a nice simple player.

Nice work! For what purposes do you think this will be most commonly used?

Not really got a firm opinion on that one. Some might use it as a space saving alternative to lossless (say, extreme or insane). Some certainly use it for digitized vinyl. I use it on my DAP of choice (extraportable). lossyWAV sits in its niche - it's not all things to all users - it's just another possible way for users to enjoy audio.

[punkrockdude]

Nick.C: Do you play FLAC/LossyWAV with your Android HD2? If so, what application do you use to play them? Regards.

I use deadbeef - it's a nice simple player.

Thank you. Regards.

[larryfine]

Great! Thank you very much!

[Nick.C]

I've just completed a full music collection transcode FLAC (-5) to lossyWAV (--quality extraportable --maxclips 0|FLAC -5 -b 512 -p 3072) resulting in 307 kbit/s (from 882 kbit/s).

[zerowalker]

Is this a Almost Lossless compression?
As that is what i get it to be after reading this.
I tried it out, but didn´t really get it to work, it just made lossy.wav that was the same size (or a bit larger i think) as the original wave.
Is there a GUI for this, as i am probably failing with commands, as i just tried to drag and drop randomly;P

Other than my stupidity, it seems very interesting, and if it is what i think, i would very likely experiment with it:)!

[bryant]

I tried it out, but didn´t really get it to work, it just made lossy.wav that was the same size (or a bit larger i think) as the original wave.

This is the expected behavior. However, what's not obvious is that the resulting WAV file has redundancies (lots of zeros in the less significant bits) that lossless compressors can take advantage of to get much better compression than they would on the original WAV file.

The sizes that you should be comparing are the original WAV files compressed with FLAC and the lossy.wav files compressed with FLAC.

[zerowalker]

I tried it out, but didn´t really get it to work, it just made lossy.wav that was the same size (or a bit larger i think) as the original wave.

This is the expected behavior. However, what's not obvious is that the resulting WAV file has redundancies (lots of zeros in the less significant bits) that lossless compressors can take advantage of to get much better compression than they would on the original WAV file.

The sizes that you should be comparing are the original WAV files compressed with FLAC and the lossy.wav files compressed with FLAC.

Okay, what settings or command is recommended?
Thank you:)

[Nick.C]

The settings for foobar2000 conversion to lossyFLAC, lossyWV and lossyTak are in post #2.

[punkrockdude]

The settings for foobar2000 conversion to lossyFLAC, lossyWV and lossyTak are in post #2.

I have had a problem with the LossyFLAC command line and had to seperate "-o%d" into "-o %d". Regards.

[Nowings69]

Congratulations

[punkrockdude]

As an intrigued fan of LossyWAV may I ask what you are working on for the next major update or beta version? Regards.

[Anakunda]

HI! I'm curious what listening configuration is loosywaw -q standard targetted. I have achieved average bitrate at 398kbps. It's higher than lame preset insane, is the quality also higher? I guess this is suitable for lower class home hifi audio equipment. If I'm wrong correct me
The auCDtect is pretty fooled saying CDDA-100% for most tracks.
Anyway I'd like to see a public listening tst between loosless format, loosyWAV and MP3@320k. Hopefully someone gets the idea too

This post has been edited by Anakunda: Aug 21 2011, 21:17

[drumliner]

just tried lossywav to create some lossy.flac files and i'm having a strange issue with it, so some help would be most welcome. i'm using it through foobar, having copied the settings directly from this thread. while the encoding takes place as i would expect it to and the size of the resulting files seems appropriate, there's something wrong with the files as they're all reported by various players as being 405'47'' long, no matter what their real length is (except for a 1'' sample that came out supposedly 270' long). players are also unable to seek the files, reported bitrates are obviously abnormally low and audiotester reports the lossyflacs as being truncated at their real length.

[Nick.C]

@punkrockdude: I'm not active developing at the moment - having a bit of time off. Identification and implementation of a psychoacoustic model may take some time....

@Anakunda: I would love to see the results too - but it takes a group with the time and inclination to carry out a listening test....

@drumliner: Apologies - settings updated. There was a change in foobar2000 that required the use of the "ignore-chunk-sizes" parameter in FLAC. I omitted to include the latest settings in post #2.

[drumliner]

thanks Nick. working properly now

[Nick.C]

Added: foobar2000 settings for producing lossyWMALSL. Thanks for developing wmaencoder, lvqcl!

[chrizoo]

I updated from 1.1.0c (which the wiki currently points to) to 1.3.0
and detected that it processes my test file 6 times slower than 1.1.0c (5x instead of 33x). Is this known? What is the reason? Thank you.

[Northpack]

I updated from 1.1.0c (which the wiki currently points to) to 1.3.0
and detected that it processes my test file 6 times slower than 1.1.0c (5x instead of 33x). Is this known? What is the reason? Thank you.

That's because of the adaptive noise shaping introduced in version 1.3.0. It gives a higher quality/bitrate ratio but slows down processing. You can disable it with "--adaptive OFF".