lossyWAV 1.1.0 released., Added noise WAV bit reduction method. Options

[Nick.C]

lossyWAV 1.1.0c is now available for download.

CODE

lossyWAV 1.1.0c, Copyright (C) 2007,2008 Nick Currie. Copyleft.

This program is free software: you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation, either version 3 of the License, or (at your option) any later
version.

This program is distributed in the hope that it will be useful,but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.  See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with
this program.  If not, see <http://www.gnu.org/licenses/>.

Process Description:

lossyWAV adds white noise to the processed output. The amount of added noise is
based on analysis of the signal levels in the frequency range 20Hz to 16kHz.

If signals above the upper limiting frequency are at an even lower level, they
can be swamped by the added noise. This is usually inaudible, but the behaviour
can be changed by specifying a higher --limit (in the range 16kHz to 20kHz).

For many audio signals, there is little content at very high frequencies, and
forcing lossyWAV to keep the added noise level lower than the content at these
frequencies can increase the bitrate dramatically for no perceptible benefit.

Usage   : lossyWAV <input wav file> <options>

Example : lossyWAV musicfile.wav

Quality Options:

-I, --insane        highest quality output, suitable for transcoding;
-E, --extreme       high quality output, also suitable for transcoding;
-S, --standard      default quality output, considered to be transparent;
-P, --portable      good quality output for DAP use. Not considered to be fully
                    transparent, but considered fit for its intended purpose.

Standard Options:

-c, --check         check if WAV file has already been processed; default=off.
                    errorlevel=16 if already processed, 0 if not.
-C, --correction    write correction file for processed WAV file; default=off.
-f, --force         forcibly over-write output file if it exists; default=off.
-h, --help          display help.
-L, --longhelp      display extended help.
-M, --merge         merge existing lossy.wav and lwcdf.wav files.
-o, --outdir <t>    destination directory for the output file(s).
-v, --version       display the lossyWAV version number.

Advanced Options:

-                   if filename="-" then WAV input is taken from STDIN.
    --blockdist     show distribution of lowest significant bit of input
                    codec-blocks and bit-removed codec-blocks.
-D, --dither <n>    enable variable PDF dither of output; default=off;
                    0 = rectangular; 1 = triangular; 0.5 = half way between.
-l, --limit <n>     set upper frequency limit to be used in analyses to n Hz;
                    (16000<=n<=20000), default = 16000.
    --linkchannels  Revert to original single bits-to-remove value for all
                    channels rather than channel dependent bits-to-remove.
-q, --quality <n>   quality preset (10=highest quality, 0=lowest bitrate;
                    default = --standard = 5; --insane = 10; --extreme = 7.5;
                    --portable = 2.5)
    --sampledist    show distribution of lowest significant bit of input
                    samples and bit-removed samples.
    --scale <n>     scaling factor from WaveGain, etc; (0.0<n<=8.0),default=1.0
-s, --shaping <n>   enable fixed noise shaping; (0.00<=n<=1.00); default=q/10;
                    0.00 = off, 1.00 = 100% effectiveness, 0.50 = 50%, etc.
    --stdinname <t> pseudo filename to use when input from STDIN.
    --stdout        write processed WAV output to STDOUT.
-w, --writetolog    create (or append to) lossyWAV.log in the output directory.

System Options:

-B, --below         set process priority to below normal.
-d, --detail        enable detailed bits-to-remove information output mode
    --low           set process priority to low.
-n, --nowarnings    suppress lossyWAV warnings.
-Q, --quiet         significantly reduce screen output.
    --silent        no screen output.

Special thanks:

David Robinson      for the publication of his lossyFLAC method, guidance, and
                    the motivation to implement the method as lossyWAV.
Horst Albrecht      for ABX testing, valuable support in tuning the internal
                    presets, constructive criticism and all the feedback.
Sebastian Gesemann  for the noise shaping coefficients and help in using them
                    in the lossyWAV noise shaping implementation.
Don Cross           for the Complex-FFT algorithm used.

Link to the hydrogenaudio wiki article

Suggested foobar2000 converter setup:

lossyFLAC:

CODE

Encoder: c:\windows\system32\cmd.exe
Extension: lossy.flac
Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --stdout|c:\"program files"\bin\flac - -b 512 -5 -f -o%d
Format is: lossless or hybrid
Highest BPS mode supported: 24

lossyTAK:

CODE

Encoder: c:\windows\system32\cmd.exe
Extension: lossy.tak
Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --stdout|c:\"program files"\bin\takc -e -p2m -fsl512 -ihs - %d
Format is: lossless or hybrid
Highest BPS mode supported: 24

lossyWV:

CODE

Encoder: c:\windows\system32\cmd.exe
Extension: lossy.wv
Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --stdout|c:\"program files"\bin\wavpack -hm --blocksize=512 --merge-blocks -i - %d
Format is: lossless or hybrid
Highest BPS mode supported: 24

Enclose the element of the path containing spaces within double quotation marks ("), e.g. C:\"Program Files"\directory_where_executable_is\executable_name. This is a Windows limitation.
N.B.: lossyWAV 1.2.0 development thread here

This post has been edited by Nick.C: Apr 17 2012, 20:51

Attached File(s)

 lossyWAV_1.1.0c.zip ( 75.56K ) Number of downloads: 787
 lossyWAV_Source_20090430_2145_1.1.0c.zip ( 201.82K ) Number of downloads: 274

 

[Hancoque]

I have three questions:

1. Is noise shaping only used with the portable preset (and below) or also with the standard preset (and maybe higher presets)?
Seems that it's always used with respect to the chosen quality value (q/10).

2. A comparison of ReplayGain values indicates that peak sample values increase with lossyWAV, reaching 1.0 where the originals were below 1.0. May that indicate possibly clipped samples?

3. Can lossyWAV safely be used with 96 kHz material without any disadvantages? Does it depend on whether noise shaping is used or not?

This post has been edited by Hancoque: Jul 25 2008, 12:03

[Nick.C]

I have three questions:

1. Is noise shaping only used with the portable preset (and below) or also with the standard preset (and maybe higher presets)?
Seems that it's always used with respect to the chosen quality value (q/10).

2. A comparison of ReplayGain values indicates that peak sample values increase with lossyWAV, reaching 1.0 where the originals were below 1.0. May that indicate possibly clipped samples?

3. Can lossyWAV safely be used with 96 kHz material without any disadvantages? Does it depend on whether noise shaping is used or not?

1) You can disable shaping with --shaping 0;
2) Samples will sometimes increase due to rounding off lsb's, sometimes decrease - increasing to 32768 will cause some clipping for 16-bit and will be changed to (32767 shr bits-to-remove) shl bits-to-remove. Clips are counted per channel per codec block. If the number of clips exceeds a preset value (see --longhelp) then the bits-to-remove for that channel is reduced by one and the bit removal process is repeated until the number of clips for that channel does not exceed the permitted number or the number of bits removed is zero;
3) Yes - noise shaping is optimised for 44.1kHz and 48kHz - thanks to SebastianG's coefficients - if worried about this noise shaping, use --shaping 0 as above.

[edit] Explanation at #2 [/edit]

This post has been edited by Nick.C: Jul 25 2008, 22:21

[Dynamic]

2) Peak samples will sometimes increase due to rounding off lsb's - this will cause some clipping at +32768 for 16-bit and will be changed to (32767 shr bits-to-remove) shl bits-to-remove;

NickC, I was also interested in this explanation.

Am I right to presume that 'shr' is 'bit-shift to the right' by the number of places following the 'shr' command (discarding the LSBs that shift off the end), and 'shl' is 'bit-shift left', shifting to the left while setting the new LSBs to zero.

I.e. 32767 is expressed in 16-bit signed binary as:

0111 1111 1111 1111

If bits to remove is 5, this clips to what I can also express as:
32768 - (2 ^ bits-to-remove) = 32768 - 2^5 = 32768 - 32 = 32736 =
0111 1111 1110 0000

Rather than do my 2 to the power of 5 calculation, this was done
by shifting the maximum value of 32767 by 5 places to the right,
going to 0000 0011 1111 1111
then back to 5 places to the left, filling LSBs with zeroes:
going to 0111 1111 1110 0000

So (2 ^ bits-to-remove) = 32 is the clipping error, which is 1/1024th of the target signal amplitude in this case, and represents a smaller error than 32 from the original signal (presumably between 16 and 31), where the target bits-to-remove would have generated a rounding error of 15 if it had 17 bits available to round upwards instead of having to round downwards.

In this 5-bit case, this clipping error is equivalent to clipping caused by increasing gain by 0.0085 dB above full scale, which is very low-level, and might reassure other users (e.g. try amplifying a full-scale signal by 0.0085 dB and ABX the clipping distortion, which will exceed the distortion in lossyWAV [edit]when 5 bits are removed, that is[/edit]). The sample error adds energy at about -60dB relative to a full-scale sample in this case, which is only mildly indicative of what scale of event may happen in the frequency domain to which the ear responds.

Things get more complicated with noise shaping in use, though presumably there's a feed-forward of accumulated error (like with error-diffusion dither in imaging) which enables the always-negative clipping adjustment to be offset by greater likelihood of positive shifts in following samples, and presumably the instantaneous clipping is likely to be incorporated into the high frequency end of the shaped noise unless there happen to be numerous successive clipped samples, which naturally means lower frequencies.

This post has been edited by Dynamic: Jul 25 2008, 22:30