lossyWAV 1.0.0 released., Added noise WAV bit reduction method. Options

[carpman]

A little late, but congratulations Nick.C, halb27 & 2Bdecided!
Nick, I hope you got to run your program (and I assume if you did, it didn't crash, even if you did):

CODE

Procedure Celebrate;
Begin
  Repeat
    Success:=Drink_Beer and not Spill_Beer;
  Until Success=False;
  Goto Bed;
End;

... also thanks to jesseg for FLACdrop.

LossyWAV's future is going to be very interesting; I see it as an excellent format for artists to release their music.

C.

[beto]

I have not followed the development of lossyWAV closely. In my understanding it is a kind of pre-processor that takes out some information from the audio file that is unlikely to be heard anyway. After that you can use your favorite lossless encoder to encode the pre-processed WAV and reach better compression.

How does this compare to Wavpack lossy and Optimfrog Dualstream qualitywise? Is it on pair with those or is it superior/inferior? It is my understanding that lossyWAV uses the same concept as Wavpack or Dualstream (correct me if I am wrong).

[Nick.C]

I have not followed the development of lossyWAV closely. In my understanding it is a kind of pre-processor that takes out some information from the audio file that is unlikely to be heard anyway. After that you can use your favorite lossless encoder to encode the pre-processed WAV and reach better compression.

How does this compare to Wavpack lossy and Optimfrog Dualstream qualitywise? Is it on pair with those or is it superior/inferior? It is my understanding that lossyWAV uses the same concept as Wavpack or Dualstream (correct me if I am wrong).

lossyWAV rounds lower significant bits to zero. This adds white noise to the audio. The level of the added noise has been pre-calculated for each bit removed. The number of bits to remove is determined by processing the results of overlapping FFT analyses of at least 2 different lengths (64 samples and 1024 samples by default). The quality settings take the processed FFT results and make the processor remove more or less bits dependent on internal settings. --quality 5 is generally accepted to be transparent. It should be stated clearly now that lossyWAV is pure variable bitrate: different audio will result in a different lossless encoded bitrate for the same --quality setting (in a similar way that the lossless audio files encoded losslessly may have different bitrates).

I cannot comment on comparison with Wavpack and Optimfrog as to my knowledge no direct comparison has been carried out.

[beto]

...
--quality 5 is generally accepted to be transparent. It should be stated clearly now that lossyWAV is pure variable bitrate: different audio will result in a different lossless encoded bitrate for the same --quality setting (in a similar way that the lossless audio files encoded losslessly may have different bitrates).

I cannot comment on comparison with Wavpack and Optimfrog as to my knowledge no direct comparison has been carried out.

You cannot comment on Wavpack and Optimfrog, yet you claim that --quality 5 of your pre-processor is 'generally accepted to be transparent'. May I ask if any comparison was made to support that claim?

I mean no offense to you or anyone, I am just curious.

[carpman]

You cannot comment on Wavpack and Optimfrog, yet you claim that --quality 5 of your pre-processor is 'generally accepted to be transparent'. May I ask if any comparison was made to support that claim?

I mean no offense to you or anyone, I am just curious.

I can claim an MP3 is transparent without comparing it to Wavpack and/or Optimfrog simply by ABXing it against the original WAV file. What's different here?

C.

[Nick.C]

You cannot comment on Wavpack and Optimfrog, yet you claim that --quality 5 of your pre-processor is 'generally accepted to be transparent'. May I ask if any comparison was made to support that claim?

I mean no offense to you or anyone, I am just curious.

I can claim an MP3 is transparent without comparing it to Wavpack and/or Optimfrog simply by ABXing it against the original WAV file. What's different here?

C.

Carpman is correct. ABX testing has been carried out by individuals with better ears than mine and -q 5 is as I "claimed" generally accepted to be transparent.

Take a look at the "lossyWAV Development" thread in the uploads forum for more details.

[halb27]

... How does this compare to Wavpack lossy and Optimfrog Dualstream qualitywise? Is it on pair with those or is it superior/inferior? It is my understanding that lossyWAV uses the same concept as Wavpack or Dualstream (correct me if I am wrong).

WavPack lossy controls the prediction error, lossyWAV controls the total error. In a sense lossyWAV has the better quality control in at least targeting the 'real' thing. Which does not necessarily mean that lossyWAV has the better quality - at least not at a bitrate which is low for these approaches (~ 300 kbps and below).
Apart from quality considerations where there will always be a lack in finally objective judgement (due to ever limited listening experience), there are practical advantages with lossyWAV as you have a choice of final codecs relevant for playback. FLAC can be used which is considered to be the lossless codec which is available best in the universe of players.

This post has been edited by halb27: May 16 2008, 08:25

[krmathis]

Can you please link me to the source code?
Its released under the GPL license, but I am unable to find it for download.

This post has been edited by krmathis: May 16 2008, 21:04

[Nick.C]

Can you please link me to the source code?
Its released under the GPL license, but I am unable to find it for download.

Download lossyWAV 1.0.0b Source Have fun!

[hödyr]

I have not followed the development and just discovered this now. I read the older threads and the wiki article and did some testing myself, and I have to say, this software is really great . Even the high quality preset sometimes give a tremendous decrease in bitrate. Also I checked out the thread for the next version, and everything (and more) I could think of is already on the 'wishlist'. Keep up the good work

[krmathis]

Download lossyWAV 1.0.0b Source Have fun!

Thanks a lot!
Downloaded, and will see if I can get this thing going on Mac OS X.

[spies]

Downloaded, and will see if I can get this thing going on Mac OS X.

Go, krmathis, Go!

[Nick.C]

Downloaded, and will see if I can get this thing going on Mac OS X.

Go, krmathis, Go!

If you require any of the assembly routines converted back to Delphi, let me know - it should be a relatively simple task - although it will slow down processing fairly dramatically, especially if the FFT unit needs to be converted back.

Also, I could make sure that the remove_bits procedure is modified as per beta 1.0.1, i.e. independent channel bit removal rather than the same bits-to-remove for all channels.

This post has been edited by Nick.C: May 17 2008, 18:44

[french dok]

This seem to be really interesting but one question comes to my mind ?
With this lossy treatment applied to future lossless files aren't we bluring everything ? I mean, we may not be able now to distinguish a real lossless file from an "almost lossless" file, which is finally a lossy one, don't you think?

[halb27]

This seem to be really interesting but one question comes to my mind ?
With this lossy treatment applied to future lossless files aren't we bluring everything ? I mean, we may not be able now to distinguish a real lossless file from an "almost lossless" file, which is finally a lossy one, don't you think?

We had this argument before. The answer is (for FLAC use as an example):
a) the lossyWAV preprocessed FLAC files get the extension .lossy.flac
b) the FLAC metadata carry the information about lossyWAV preprocessing
c) when looking at a lossless codec file you never know what the input was. It might originate from a 192 kbps DRM WMA file from a company offering DRM protected downloads but which you want to play on a non WMA DAP while not giving away quality. Three years after encoding theere's a chance you don't memorize where it came from.

This post has been edited by halb27: May 19 2008, 08:42

[Nick.C]

This seem to be really interesting but one question comes to my mind ?
With this lossy treatment applied to future lossless files aren't we bluring everything ? I mean, we may not be able now to distinguish a real lossless file from an "almost lossless" file, which is finally a lossy one, don't you think?

If you can't distinguish a lossless file from an almost lossless file, is there any problem with having an almost lossless file?

As halb27 said, if you do not create the lossless file yourself, you will never really know whether the audio contained in the lossless container really is lossless.

Yes, lossyWAV blurs the boundary - however if the almost lossless version cannot be ABX'ed from the original then....

For the record, again, I will be using lossyWAV & FLAC to allow me to fit as much of my music as possible onto as big a Compact Flash as I can afford for use on my iPAQ running TCPMP beta v0.81. My FLAC archive will remain complete and unmolested - lossyWAV & FLAC is purely my DAP transcode method of choice.

[french dok]

I see.
Point c) from halb27 is indeed a good remark.

If you can't distinguish a lossless file from an almost lossless file, is there any problem with having an almost lossless file?

I do not use myself flac for my music because I only care about transparency. But I think about people who uses flac for perfect losslessness whatever the transparent result you can get with lossy treatments.

We had this argument before

Oops sorry I didn't know.

BTW, thank you for your answers.

[jesseg]

It should be stated clearly now that lossyWAV is pure variable bitrate

perhaps a better way to word it would be that lossyWAV is variable bit depth. just a thought. again amazing work, and the change in v1.0.1 with the processing of individual channels... brilliant. i can't wait to see how it changes the bitrates (post flac of course, lest i be caught in my own trap), and of course it should be higher quality in some cases.

[tev777]

... set the block size to 512 samples (-b 512 in FLAC) and also to preserve WAV chunks (--keep-foreign-metadata in FLAC)...

Is (-b 512 AND --keep-foreign-metadata) mandatory for this to work with FLAC? You mention that it keeps LossWAV from processing the file a second time, but if I am only running the process once can I leave this out and still get the desired results?

Thank you for your time and effort.

--
E

[Nick.C]

... set the block size to 512 samples (-b 512 in FLAC) and also to preserve WAV chunks (--keep-foreign-metadata in FLAC)...

Is (-b 512 AND --keep-foreign-metadata) mandatory for this to work with FLAC? You mention that it keeps LossWAV from processing the file a second time, but if I am only running the process once can I leave this out and still get the desired results?

Thank you for your time and effort.

--
E

The --keep-foreign-metadata is optional in this case, the -b 512 is not (well, it is, but you would be best not to omit it).

The -b 512 parameter tells FLAC to use a 512 sample block length for encoding. This matches the 512 sample codec-block length in lossyWAV and will result in optimal FLAC encoding in most cases (excepting cases where bits-to-remove does not vary between codec-blocks).

This post has been edited by Nick.C: May 22 2008, 08:11

[Jalan]

is there a more in-depth "how to" for foobar planned (compared to what is in the wiki article, that is)?

it's probably my inability to grasp something fully, but i can replicate the settings shown in the image (with differences in path) and copy+paste+save the batch file and eventually go so far as to rip a test wav and watch the encoder interface run for less than a minute, only to hang for a few seconds afterward and then throw up an error indicating that the encoder halted because it could not find the .wav being used in the process.

on the contrary, getting it to run through XP's command line saw next to no error, aside from clipping issues with the file since i left that parameter out. i don't find myself crawling to the command line often, so i was hoping to rely on foobar to handle the brunt of the work once i could round up files to run the encoder on.

[Nick.C]

is there a more in-depth "how to" for foobar planned (compared to what is in the wiki article, that is)?

it's probably my inability to grasp something fully, but i can replicate the settings shown in the image (with differences in path) and copy+paste+save the batch file and eventually go so far as to rip a test wav and watch the encoder interface run for less than a minute, only to hang for a few seconds afterward and then throw up an error indicating that the encoder halted because it could not find the .wav being used in the process.

on the contrary, getting it to run through XP's command line saw next to no error, aside from clipping issues with the file since i left that parameter out. i don't find myself crawling to the command line often, so i was hoping to rely on foobar to handle the brunt of the work once i could round up files to run the encoder on.

Have you used a path with spaces in it? I ask as this was an early problem I had with foobar2000 using lossyWAV via a batch file. Also, make sure the extension you type in is "lossy.flac" not ".lossy.flac" as the latter will also crash.

Don't be too concerned about clipping (in this case) - even at the lowest quality setting lossyWAV only allows up to a maximum of 3 samples to be "clipped" per channel per codec-block (512 samples in length).

[carpman]

Nick

I think there's a problem with the recommended batch file code for the foobar setup in the LossyWAV wiki:

CODE

@echo off
z:\bin\lossyWAV %1 %3 %4 %5 %6 %7 %8 %9 --below --nowarnings --quiet
z:\bin\flac.exe -5 -f -b 512 "%~N1.lossy.wav" -o"%~N2.flac"
del "%~N1.lossy.wav"

I, like Jalan was having all sorts of difficulty getting lossyWAV to encode via foobar.

Then I noticed that it's calling flac.exe whereas it's calling lossyWAV (.nothing).
I changed it to flac (.nothing) and it worked fine. So maybe this is the problem. Here's the batch file that worked for me.

CODE

@echo off
z:\bin\lossyWAV %1 %3 %4 %5 %6 %7 %8 %9 --below --nowarnings --quiet
z:\bin\flac -5 -f -b 512 "%~N1.lossy.wav" -o"%~N2.flac"
del "%~N1.lossy.wav"

Perhaps if Jalan could try this and see if it works too, then if it does it perhaps it's worth amending the Wiki?

I'm not sure, but is the "no spaces" tip mentioned in the wiki (didn't see it). Something like this might be helpful:

CODE

@echo off
d:\address_with_no_spaces\lossywav\lossyWAV %1 %3 %4 %5 %6 %7 %8 %9 --below --nowarnings --quiet
d:\address_with_no_spaces\lossywav\flac -5 -f -b 512 "%~N1.lossy.wav" -o"%~N2.flac"
del "%~N1.lossy.wav"

C.

This post has been edited by carpman: May 26 2008, 18:42

[Jalan]

Have you used a path with spaces in it? I ask as this was an early problem I had with foobar2000 using lossyWAV via a batch file.

indeed, there were spaces in the path that foobar was attempting to call the batch from.

Perhaps if Jalan could try this and see if it works too, then if it does it perhaps it's worth amending the Wiki?

tried it with the amended path via Nick C's suggestion (no spaces, direct to the main drive) and your removal of the .exe extension to the flac encoder in the batch text and it produced a full file without issue. thanks for making note of it.

This post has been edited by Jalan: May 26 2008, 19:18

[carpman]

tried it with the amended path via Nick C's suggestion (no spaces, direct to the main drive) and your removal of the .exe extension to the flac encoder in the batch text and it produced a full file without issue. thanks for making note of it.

Glad it worked for you.
I'm pretty sure it's the .exe issue, as I tried it with no spaces first and it still didn't work, then got rid of the .exe and it did. And if that's not scientific evidence then my full name isn't Carpman Sherlock Einstein

C.