lossyWAV 1.1.0 released., Added noise WAV bit reduction method. Options

[Nick.C]

lossyWAV 1.1.0c is now available for download.

CODE

lossyWAV 1.1.0c, Copyright (C) 2007,2008 Nick Currie. Copyleft.

This program is free software: you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation, either version 3 of the License, or (at your option) any later
version.

This program is distributed in the hope that it will be useful,but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.  See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with
this program.  If not, see <http://www.gnu.org/licenses/>.

Process Description:

lossyWAV adds white noise to the processed output. The amount of added noise is
based on analysis of the signal levels in the frequency range 20Hz to 16kHz.

If signals above the upper limiting frequency are at an even lower level, they
can be swamped by the added noise. This is usually inaudible, but the behaviour
can be changed by specifying a higher --limit (in the range 16kHz to 20kHz).

For many audio signals, there is little content at very high frequencies, and
forcing lossyWAV to keep the added noise level lower than the content at these
frequencies can increase the bitrate dramatically for no perceptible benefit.

Usage   : lossyWAV <input wav file> <options>

Example : lossyWAV musicfile.wav

Quality Options:

-I, --insane        highest quality output, suitable for transcoding;
-E, --extreme       high quality output, also suitable for transcoding;
-S, --standard      default quality output, considered to be transparent;
-P, --portable      good quality output for DAP use. Not considered to be fully
                    transparent, but considered fit for its intended purpose.

Standard Options:

-c, --check         check if WAV file has already been processed; default=off.
                    errorlevel=16 if already processed, 0 if not.
-C, --correction    write correction file for processed WAV file; default=off.
-f, --force         forcibly over-write output file if it exists; default=off.
-h, --help          display help.
-L, --longhelp      display extended help.
-M, --merge         merge existing lossy.wav and lwcdf.wav files.
-o, --outdir <t>    destination directory for the output file(s).
-v, --version       display the lossyWAV version number.

Advanced Options:

-                   if filename="-" then WAV input is taken from STDIN.
    --blockdist     show distribution of lowest significant bit of input
                    codec-blocks and bit-removed codec-blocks.
-D, --dither <n>    enable variable PDF dither of output; default=off;
                    0 = rectangular; 1 = triangular; 0.5 = half way between.
-l, --limit <n>     set upper frequency limit to be used in analyses to n Hz;
                    (16000<=n<=20000), default = 16000.
    --linkchannels  Revert to original single bits-to-remove value for all
                    channels rather than channel dependent bits-to-remove.
-q, --quality <n>   quality preset (10=highest quality, 0=lowest bitrate;
                    default = --standard = 5; --insane = 10; --extreme = 7.5;
                    --portable = 2.5)
    --sampledist    show distribution of lowest significant bit of input
                    samples and bit-removed samples.
    --scale <n>     scaling factor from WaveGain, etc; (0.0<n<=8.0),default=1.0
-s, --shaping <n>   enable fixed noise shaping; (0.00<=n<=1.00); default=q/10;
                    0.00 = off, 1.00 = 100% effectiveness, 0.50 = 50%, etc.
    --stdinname <t> pseudo filename to use when input from STDIN.
    --stdout        write processed WAV output to STDOUT.
-w, --writetolog    create (or append to) lossyWAV.log in the output directory.

System Options:

-B, --below         set process priority to below normal.
-d, --detail        enable detailed bits-to-remove information output mode
    --low           set process priority to low.
-n, --nowarnings    suppress lossyWAV warnings.
-Q, --quiet         significantly reduce screen output.
    --silent        no screen output.

Special thanks:

David Robinson      for the publication of his lossyFLAC method, guidance, and
                    the motivation to implement the method as lossyWAV.
Horst Albrecht      for ABX testing, valuable support in tuning the internal
                    presets, constructive criticism and all the feedback.
Sebastian Gesemann  for the noise shaping coefficients and help in using them
                    in the lossyWAV noise shaping implementation.
Don Cross           for the Complex-FFT algorithm used.

Link to the hydrogenaudio wiki article

Suggested foobar2000 converter setup:

lossyFLAC:

CODE

Encoder: c:\windows\system32\cmd.exe
Extension: lossy.flac
Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --stdout|c:\"program files"\bin\flac - -b 512 -5 -f -o%d
Format is: lossless or hybrid
Highest BPS mode supported: 24

lossyTAK:

CODE

Encoder: c:\windows\system32\cmd.exe
Extension: lossy.tak
Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --stdout|c:\"program files"\bin\takc -e -p2m -fsl512 -ihs - %d
Format is: lossless or hybrid
Highest BPS mode supported: 24

lossyWV:

CODE

Encoder: c:\windows\system32\cmd.exe
Extension: lossy.wv
Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --stdout|c:\"program files"\bin\wavpack -hm --blocksize=512 --merge-blocks -i - %d
Format is: lossless or hybrid
Highest BPS mode supported: 24

Enclose the element of the path containing spaces within double quotation marks ("), e.g. C:\"Program Files"\directory_where_executable_is\executable_name. This is a Windows limitation.
N.B.: lossyWAV 1.2.0 development thread here

This post has been edited by Nick.C: Apr 17 2012, 20:51

Attached File(s)

 lossyWAV_1.1.0c.zip ( 75.56K ) Number of downloads: 789
 lossyWAV_Source_20090430_2145_1.1.0c.zip ( 201.82K ) Number of downloads: 276

 

[2Bdecided]

Nick,

1. The FFT size should vary with sample rate, though good luck making that happen easily with all the optimisation you've done!

2. IMO and IIRC, enabling dither shouldn't raise the noise floor much on average - the noise floor should stay the same, but more bits will have to be kept the achieve this.

I don't enable dither either.

Cheers,
David.

[Nick.C]

Nick,

1. The FFT size should vary with sample rate, though good luck making that happen easily with all the optimisation you've done!

2. IMO and IIRC, enabling dither shouldn't raise the noise floor much on average - the noise floor should stay the same, but more bits will have to be kept the achieve this.

I don't enable dither either.

Cheers,
David.

I will modify 1.1.0 to increase the FFT lengths at 69.08kHz, 138.15kHz and 276.3kHz, i.e. 64 to 128 to 256 and 512 samples respectively and correspondingly for the other lengths with a similar increase in codec-block length, 512 to 1024 to 2048 to 4096 samples. This means that the maximum length of FFT will be 8192 samples. [edit] arithmetic failure [/edit]

The spreading functions will not require to be changes as they will be working over approximately the same number of FFT bins after each change, taking into account the increase in sample rate.

I feel that an upper frequency limit of 384kHz is high enough, although I am open to suggestions.

Basically this means that at present I am not confident in the high (i.e. >48kHz) sample rate performance of lossyWAV 1.1.0 and would caution anyone using it at these sample rates against using it for anything other than testing purposes.

On dither, of course you're right - the increase in noise due to dithering will mean that fewer bits can be removed to keep the added noise to the same level that it would have been had to no dither been used used. I will look at creating additional reference threshold constants for the range of dither between --dither 0 (rectangular) to --dither 1 (triangular) to allow dither to be "safely" used.

This post has been edited by Nick.C: Jul 28 2008, 20:15

[Hancoque]

Basically this means that at present I am not confident in the high (i.e. >48kHz) sample rate performance of lossyWAV 1.1.0 and would caution anyone using it at these sample rates against using it for anything other than testing purposes.

Do you refer to lossyWAV in general or to how noise shaping is applied?

My intention is to understand and implement SebastianG's new noise shaping method, but for that I will also have to introduce / find a PSY model of some kind.

I would hope that by using the new noise shaping method some additional bits can be removed for the same apparent quality level of output, thereby further reducing the bitrate.

What would happen if noise shaping is disabled via --shaping 0? Would that be taken into account by *not* removing those "additional bits" then? Otherwise the non-shaped results might be pretty bad in comparison if used with lower quality settings.

I'm also wondering if trading further removal of bits for better noise shaping really yields useful results as both methods seem to cancel each other out:

Removing more bits:
- lower filesize
- more noise

Stronger noise shaping:
- higher filesize
- less (perceived) noise

[Nick.C]

Do you refer to lossyWAV in general or to how noise shaping is applied?

What would happen if noise shaping is disabled via --shaping 0? Would that be taken into account by *not* removing those "additional bits" then? Otherwise the non-shaped results might be pretty bad in comparison if used with lower quality settings.

I'm also wondering if trading further removal of bits for better noise shaping really yields useful results as both methods seem to cancel each other out:

Removing more bits:
- lower filesize
- more noise

Stronger noise shaping:
- higher filesize
- less (perceived) noise

I was referring to lossyWAV in general as the 64/1024 sample fft lengths are fixed at present.
Disabling noise shaping will reduce filesize and increase perceived noise. However the added noise (especially at higher quality presets) should be at or below the existing noise floor.

The noise shaping implementation results in a trade-off between bits removed and filesize. That is why the option remains for the user to disable noise shaping. At --insane the addition of noise shaping only adds a few kbit/s to the FLAC encoded lossyWAV output. The increase in bitrate is substantially more at lower
quality presets.

Modifications for 1.1.0b:

• implementation of increasing fft length for increasing sample rate;
• improved logfile output and --detail output;
• reference threshold constants for rectangular dither and triangular dither have been calculated so added noise should be the same for dither off and any dither level between 0 and 1 - the number of bits-to-remove will however reduce with "increasing" dither.

I expect to post lossyWAV 1.1.0b tonight.