Help - Search - Members - Calendar
Full Version: lossyWAV 1.1.0 released.
Hydrogenaudio Forums > Lossy Audio Compression > Other Lossy Codecs
Pages: 1, 2, 3
Nick.C
lossyWAV 1.1.0c is now available for download.
CODE
lossyWAV 1.1.0c, Copyright (C) 2007,2008 Nick Currie. Copyleft.

This program is free software: you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation, either version 3 of the License, or (at your option) any later
version.

This program is distributed in the hope that it will be useful,but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.  See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with
this program.  If not, see <http://www.gnu.org/licenses/>.

Process Description:

lossyWAV adds white noise to the processed output. The amount of added noise is
based on analysis of the signal levels in the frequency range 20Hz to 16kHz.

If signals above the upper limiting frequency are at an even lower level, they
can be swamped by the added noise. This is usually inaudible, but the behaviour
can be changed by specifying a higher --limit (in the range 16kHz to 20kHz).

For many audio signals, there is little content at very high frequencies, and
forcing lossyWAV to keep the added noise level lower than the content at these
frequencies can increase the bitrate dramatically for no perceptible benefit.

Usage   : lossyWAV <input wav file> <options>

Example : lossyWAV musicfile.wav

Quality Options:

-I, --insane        highest quality output, suitable for transcoding;
-E, --extreme       high quality output, also suitable for transcoding;
-S, --standard      default quality output, considered to be transparent;
-P, --portable      good quality output for DAP use. Not considered to be fully
                    transparent, but considered fit for its intended purpose.

Standard Options:

-c, --check         check if WAV file has already been processed; default=off.
                    errorlevel=16 if already processed, 0 if not.
-C, --correction    write correction file for processed WAV file; default=off.
-f, --force         forcibly over-write output file if it exists; default=off.
-h, --help          display help.
-L, --longhelp      display extended help.
-M, --merge         merge existing lossy.wav and lwcdf.wav files.
-o, --outdir <t>    destination directory for the output file(s).
-v, --version       display the lossyWAV version number.

Advanced Options:

-                   if filename="-" then WAV input is taken from STDIN.
    --blockdist     show distribution of lowest significant bit of input
                    codec-blocks and bit-removed codec-blocks.
-D, --dither <n>    enable variable PDF dither of output; default=off;
                    0 = rectangular; 1 = triangular; 0.5 = half way between.
-l, --limit <n>     set upper frequency limit to be used in analyses to n Hz;
                    (16000<=n<=20000), default = 16000.
    --linkchannels  Revert to original single bits-to-remove value for all
                    channels rather than channel dependent bits-to-remove.
-q, --quality <n>   quality preset (10=highest quality, 0=lowest bitrate;
                    default = --standard = 5; --insane = 10; --extreme = 7.5;
                    --portable = 2.5)
    --sampledist    show distribution of lowest significant bit of input
                    samples and bit-removed samples.
    --scale <n>     scaling factor from WaveGain, etc; (0.0<n<=8.0),default=1.0
-s, --shaping <n>   enable fixed noise shaping; (0.00<=n<=1.00); default=q/10;
                    0.00 = off, 1.00 = 100% effectiveness, 0.50 = 50%, etc.
    --stdinname <t> pseudo filename to use when input from STDIN.
    --stdout        write processed WAV output to STDOUT.
-w, --writetolog    create (or append to) lossyWAV.log in the output directory.

System Options:

-B, --below         set process priority to below normal.
-d, --detail        enable detailed bits-to-remove information output mode
    --low           set process priority to low.
-n, --nowarnings    suppress lossyWAV warnings.
-Q, --quiet         significantly reduce screen output.
    --silent        no screen output.

Special thanks:

David Robinson      for the publication of his lossyFLAC method, guidance, and
                    the motivation to implement the method as lossyWAV.
Horst Albrecht      for ABX testing, valuable support in tuning the internal
                    presets, constructive criticism and all the feedback.
Sebastian Gesemann  for the noise shaping coefficients and help in using them
                    in the lossyWAV noise shaping implementation.
Don Cross           for the Complex-FFT algorithm used.
Link to the hydrogenaudio wiki article

Suggested foobar2000 converter setup:

lossyFLAC:
CODE
Encoder: c:\windows\system32\cmd.exe
Extension: lossy.flac
Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --stdout|c:\"program files"\bin\flac - -b 512 -5 -f -o%d
Format is: lossless or hybrid
Highest BPS mode supported: 24
lossyTAK:
CODE
Encoder: c:\windows\system32\cmd.exe
Extension: lossy.tak
Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --stdout|c:\"program files"\bin\takc -e -p2m -fsl512 -ihs - %d
Format is: lossless or hybrid
Highest BPS mode supported: 24
lossyWV:
CODE
Encoder: c:\windows\system32\cmd.exe
Extension: lossy.wv
Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --stdout|c:\"program files"\bin\wavpack -hm --blocksize=512 --merge-blocks -i - %d
Format is: lossless or hybrid
Highest BPS mode supported: 24

There is a known problem within foobar2000 (although more likely to do with cmd.exe itself) when running an executable within the cmd.exe command line from a path which includes spaces. The suggested fix for this is to enclose the element of the path which contains spaces within double quotation marks ("), e.g. c:\"program files"\directory_where_executable_is\executable_name

N.B.: lossyWAV 1.2.0 development thread here
botface
QUOTE (Nick.C @ Jul 14 2008, 12:34) *
lossyWAV 1.1.0 is now available for download.

Well done, Nick. Another hassle-free release
Nick.C
QUOTE (botface @ Jul 14 2008, 13:47) *
Well done, Nick. Another hassle-free release
Many thanks - glad to be of service!
Axon
Yayayay! Thanks a lot Nick.

Has there been a comparison of lossless encoder performance when combined with lossyWAV?
Nick.C
QUOTE (Axon @ Jul 14 2008, 19:55) *
Has there been a comparison of lossless encoder performance when combined with lossyWAV?
Erm, no - but it would not be too difficult to set one up.

I would imagine Lame, ogg vorbis and neroAacEnc would be the targets at varying quality levels spanning the range for each, from both lossless FLAC and lossyFLAC (--portable, --standard, --extreme and --insane presets).

So, maybe 3 codecs x 6 levels per codec x (4 quality presets of lossyFLAC + lossless FLAC) = 90 average bitrates for an agreed test set of tracks / samples.

Comments / suggestions please! [edit] Can't count.... [/edit]
carpman
QUOTE (Axon @ Jul 14 2008, 19:55) *
Has there been a comparison of lossless encoder performance when combined with lossyWAV?

Though lossy would also be interesting, perhaps.

C.
Axon
Yeah, I'm mainly curious as to if comparing lossyTAK/lossyFLAC would yield an increase in, um, performance increase relative to a comparison of TAK/FLAC. Certainly some encoders seem to handle white noise better than others, so by reducing the bit depth in lossyWAV's fashion, that impact could be reduced.
Nick.C
QUOTE (carpman @ Jul 14 2008, 20:38) *
QUOTE (Axon @ Jul 14 2008, 19:55) *
Has there been a comparison of lossless encoder performance when combined with lossyWAV?
Though lossy would also be interesting, perhaps.

C.
Apparently I can't read and can't count.... blush.gif

I'll get to work on the lossyFLAC / lossyTAK / lossyWV comparison using recommended FLAC / TAK / Wavpack settings above at -q 0, --portable, --standard, --extreme and --insane.

[edit] After a quick batch file or two and making sure that my source FLAC set was encoded at -5. Lossless FLAC / TAK / Wavpack all encoded without using the 512 sample blocksize parameters. So, for my 55 problem sample set:
CODE
             +------------+------------+------------+
             |    TAK     |    FLAC    |  Wavpack   |
+------------+------------+------------+------------+
| lossless   | 749 kbit/s | 787 kbit/s | 772 kbit/s |
+------------+------------+------------+------------+
| --insane   | 634 kbit/s | 654 kbit/s | 661 kbit/s |
| --extreme  | 564 kbit/s | 583 kbit/s | 599 kbit/s |
| --standard | 488 kbit/s | 508 kbit/s | 529 kbit/s |
| --portable | 405 kbit/s | 425 kbit/s | 444 kbit/s |
| -q 0       | 299 kbit/s | 321 kbit/s | 342 kbit/s |
+------------+------------+------------+------------+
sauvage78
thks Nick, now that noise shaping was done in 1.1.0, is there still room for improvement (lower bitrate) in 1.2.0, or are you reaching the end of what you wanted to do with lossyWAV ? I mean now the todo list for 1.2.0 seems rather small from my non-technical point of view ... shouldn't 1.2.0 be a 1.1.1 now ? will there be just cosmetic changes & bugfix from now on ???
Nick.C
QUOTE (sauvage78 @ Jul 14 2008, 21:21) *
thks Nick, now that noise shaping was done in 1.1.0, is there still room for improvement (lower bitrate) in 1.2.0, or are you reaching the end of what you wanted to do with lossyWAV ? I mean now the todo list for 1.2.0 seems rather small from my non-technical point of view ... shouldn't 1.2.0 be a 1.1.1 now ? will there be just cosmetic changes & bugfix from now on ???
My intention is to understand and implement SebastianG's new noise shaping method, but for that I will also have to introduce / find a PSY model of some kind.

I would hope that by using the new noise shaping method some additional bits can be removed for the same apparent quality level of output, thereby further reducing the bitrate. However, this could be some time in happening as my basic understanding of the topic has to improve somewhat before I start implementation.

Unlike last time, I will not be starting the development of 1.2.0 a day or two after the release of the new version.

In a lot of ways, I've achieved what I set out to do (about a year ago) in accordance with David's original method (albeit with a few tweaks here and there).

I have an idea for a cosmetic change for the --detail parameter and may make modifications so that its output can be written to the logfile, although as more than one instance of lossyWAV could be running I want to keep the time that the logfile is open for output to an absolute minimum.

Cosmetically, I am happier with 1.1.0 than I was with 1.0.0b, but that has only come about by changing the way that the screen / log output is handled.
Agent69
Out of curiousity, and with all due respect to its developer, is LossyWAV just for use with DAPs? Is there any reason to use it when disk space is not an issue?
carpman
Your question assumes that disk space is never an issue on HDDs.
I use lossyWAV but don't have a DAP.
Also it could be used to reduce bandwidth issues in terms of selling high quality transcodable files online (I have a big problem with buying audio that's trapped in a particular format, i.e MP3).
If you can save space without any loss in audible quality it seems sensible to do so. Space = money, so why not save it where you can, even if money is not an issue.

C.
Synthetic Soul
QUOTE (Axon @ Jul 14 2008, 20:50) *
Yeah, I'm mainly curious as to if comparing lossyTAK/lossyFLAC would yield an increase in, um, performance increase relative to a comparison of TAK/FLAC. Certainly some encoders seem to handle white noise better than others, so by reducing the bit depth in lossyWAV's fashion, that impact could be reduced.

Using Nick's figures above, it seems it is much of a muchness, although WavPack seems to benefit a little less (as can be seen by FLAC actually producing smaller bitrates with most settings:

CODE
             +------------------+------------------+------------------+
             |    TAK           |    FLAC          |  Wavpack         |
+------------+------------------+------------------+------------------+
| lossless   | 749 kbit/s       | 787 kbit/s        | 772 kbit/s      |
+------------+------------------+------------------+------------------+
| --insane   | 634 kbit/s (85%) | 654 kbit/s (83%) | 661 kbit/s (86%) |
| --extreme  | 564 kbit/s (75%) | 583 kbit/s (74%) | 599 kbit/s (78%) |
| --standard | 488 kbit/s (65%) | 508 kbit/s (65%) | 529 kbit/s (69%) |
| --portable | 405 kbit/s (54%) | 425 kbit/s (54%) | 444 kbit/s (58%) |
| -q 0       | 299 kbit/s (40%) | 321 kbit/s (41%) | 342 kbit/s (44%) |
+------------+------------------+------------------+------------------+


IIRC Thomas mentioned something about a specific LossyWAV setting coming for TAK.
Nick.C
QUOTE (Synthetic Soul @ Jul 16 2008, 14:42) *
IIRC Thomas mentioned something about a specific LossyWAV setting coming for TAK.
He did indeed smile.gif!
2Bdecided
QUOTE (Agent69 @ Jul 16 2008, 13:44) *
Out of curiousity, and with all due respect to its developer, is LossyWAV just for use with DAPs? Is there any reason to use it when disk space is not an issue?
Yes, most modern CDs don't deserve lossless (they're such bad quality) while requiring very high lossless bitrates. LossyWAV tames them with little to no side effects.

I can always find something better to do with the HDD space. A few TBs don't go far when you have video and photos, plus a lot of audio to back up.

Cheers,
David.
Axon
To say nothing about vinyl. Gawd! My needledrop FLACs are shrinking by 45% at --standard!

If there ever was a demonstration of the reduced dynamic range of vinyl, lossyWAV has to be it. But clearly, there's a silver lining to it.

Thanks for the new comparisons Synthetic. It's very interesting to see that TAK actually pulls ahead of FLAC by an additional 2% at -q0, although the gain is eliminated at --standard.
Nick.C
QUOTE (2Bdecided @ Jul 16 2008, 17:10) *
QUOTE (Agent69 @ Jul 16 2008, 13:44) *
Out of curiousity, and with all due respect to its developer, is LossyWAV just for use with DAPs? Is there any reason to use it when disk space is not an issue?
Yes, most modern CDs don't deserve lossless (they're such bad quality) while requiring very high lossless bitrates. LossyWAV tames them with little to no side effects.

I can always find something better to do with the HDD space. A few TBs don't go far when you have video and photos, plus a lot of audio to back up.

Cheers,
David.
As David said (and lossyWAV is an implementation of his lossyFLAC proposal) there are little or no side effects - even lossyWAV --insane|FLAC yields a 15% (or more) saving over lossless FLAC.

Also, I am also noticing that newer albums seem to compress less well in FLAC - but compress as well as older material after both have been processed with lossyWAV.

[edit] I use lossyFLAC solely with CorePlayer on my iPAQ h2210 with 32GB CF card and 4GB SD card. My FLAC archive is safe on my server. As an aside, I processed 262 CD images (252h32m38s) from FLAC on my server to lossyWAV -q 1.25|FLAC -5 in 3h05m last night - so transcoding for immediate use is really quite feasible (Intel C2D @ 3.0GHz, 4GB RAM). My iPAQ would play lossless FLAC of course, but my hearing is not good enough to determine the difference and circa 335 kbit/s for lossyWAV -q 1.25|FLAC -5 is palatable from a size viewpoint. I believe that lossyFLAC requires less processor effort when decoding than, say, MP3 - so I should get longer battery life (although I installed a 3600mAh battery just to be sure.... smile.gif). [/edit]
Brent
I searched the entire 1.1.0 development thread, but I couldn't get the answers I'm looking for: has anyone ever succesfully ABX'ed --portable? Or, if there just is no direct answer to that question, what's the lowest quality setting anyone has ever succesfully ABX'ed? I'm curious how paranoid I should be to use a higher setting (I myself am quite happy with --portable, but I obviously havn't had time to check my entire library for errors, let alone ABX'ing).
Nick.C
That's a good question - I believe that Mardel or Sauvauge78 managed to ABX at -q 2 with specific samples using an older beta of lossyWAV, which prompted a bit of tightening of internal preset parameters. However, as --portable == -q 2.5, I think it includes a suitable safety margin over -q 1 - but it's all down to your ears / equipment / listening environment really....

As my sig says, I am happy with -q 1.25 for my purposes (listening to it right now on my PC) - however my lossyFLAC collection is my lossy copy - I keep the lossless FLAC safely tucked away.

halb27 is, I believe, best placed to advise on archive quality lossyFLAC - his setting is --extreme for archiving.
halb27
QUOTE (Brent @ Jul 16 2008, 21:36) *
... has anyone ever succesfully ABX'ed --portable? ...

I just finished another listening test as we have a new final release and there were some changes since my last test. Moreover I have some CDs to encode which I didn't want to encode before this final release.

I decided to use the --portable quality as I don't remember having ever tested it since noise shaping is involved with it as a default. I didn't read your post before my test.

I used my usual set of potential problems Atem-lied, harp40_1, bruhns, badvilbel, trumpet, furious, keys_1644ds, triangle-2_1644ds, bibilolo, herding_calls, S37_OTHERS_MartenotWaves_A, dither_noise_test, Fiocco, Under The Boardwalk, utb, 00000_00595ms, Livin_In_The_Future, eig_essence, castanets.

I was not able to abx a problem. If there was a suspicion against transparency for me furious is a candidate for this, but it's beyond my ability to abx it (and I'm not sure if my weak suspicion is unconscious placebo cause using foobar I've seen furious' bitrate is relatively low compared to that of other problem samples).

I also tested a series of full length tracks and everything was fine to me.

To finish it up I encoded my regular music test set using --extreme and listened to the error files. In many - if not most situations - the error is inaudible even with no music to mask the noise. When noise is audible it's very low volume. I also looked at the added noise's spectrum and everything works as expected: noise is in the very high frequency region most of the time and seldom goes down to 10 kHz (judging from Nero wave editor display).

So everything's fine, and even --portable provides excellent results for my ears.

Thank you, Nick, for your great work.


As for your question, Brent: lossyWAV has had a long life as far as the various versions are concerned. With regard to the versions since the introduction of the quality naming scheme like --portable I'm pretty sure nobody abxed --portable. Moreover we can't but talk about nowadays quality of --portable since the default inclusion of noise shaping. That's not a long time ago.
Nick.C
QUOTE (halb27 @ Jul 16 2008, 21:05) *
Thank you, Nick, for your great work.
A pleasure, Sir! Said before but worth repeating - lossyWAV would not have happened without David's proposal publication, Matlab script and guidance (& restraint smile.gif), our early collaborative efforts and all of your ABX testing (if the output was good enough to require to be ABX'ed wink.gif).
carpman
QUOTE (Nick.C @ Jul 16 2008, 16:30) *
QUOTE (Synthetic Soul @ Jul 16 2008, 14:42) *
IIRC Thomas mentioned something about a specific LossyWAV setting coming for TAK.
He did indeed smile.gif!

Just out of interest, did he mention an approx. timescale for this?

C.
Axon
Now this is cute. Just for grins I tried lossyWAV --standard on a 24/44 needledrop I had. The bitrate dropped from 1504kbps compressed to 506kbps - a 65% reduction in file size? -q1.25 was 403kbps.

That makes a lot of sense though, most notably because my needledrops are not RIAA equalized, so the amplifier noise is largely flat.
Brent
QUOTE (halb27 @ Jul 16 2008, 22:05) *
QUOTE (Brent @ Jul 16 2008, 21:36) *

... has anyone ever succesfully ABX'ed --portable? ...

As for your question, Brent: lossyWAV has had a long life as far as the various versions are concerned. With regard to the versions since the introduction of the quality naming scheme like --portable I'm pretty sure nobody abxed --portable. Moreover we can't but talk about nowadays quality of --portable since the default inclusion of noise shaping. That's not a long time ago.

As a lowly enduser not dedicated enough to evaluate every -q setting, the reason I enquired about --portable was that I supposed there was some rationale behind it being -q2.5. --standard is q5 and the commandline utility informs me that it is considered to be transparent, but from the looks of it --portable is transparent too. Because I read about a lot of testing in the q0-q2 range, I wondered wat exactly the reason could be to go any higher than --portable, because most reported no succes in ABX-ing in the topend of that range.

So, thank you very much for your results. And, if I may, what exactly is your reason to go with --extreme for your noise test? Are you considering moving your lib to that setting, and you've picked it over lower q settings just to be on the safe side?
halb27
QUOTE (Brent @ Jul 17 2008, 11:17) *
.. what exactly is your reason to go with --extreme for your noise test?

For finding out whether --portable is transparent to me I did the listening test.
For testing the machinery not to have a bug I decided to listen to the error noise and look at it's spectrum.
I did this with --extreme on one hand because this is my usual encoding setting (I don't keep a losslesss archive any more, so I want extremely high quality with my lossyWAV archive). On the other hand --extreme is necessary to see most of the noise go to the extreme end of the spectrum. You can also expect only from a very high quality setting to hear only low volume noise or no noise at all when there's no masking by the music. This is totally irrelevant for judging about transparency (that's what ABX listening tests are for), I just wanted to make sure noise gets low volume in an absolute sense with an extremely high quality setting. Just a more formal, technical test whether everything's fine.
shadowking
i think -standard quality losswav is aiming for transparency for every sample and every listener . Lower setting like Q3 or 2.5 or lower might very well be transparent for the vast majority of cases. If keeping the bitrate down is a concern, one could use a lower quality like 2.5 and still get very high quality and enough for that transcode to layer 3 if needed.

The other advantage of quality 6 / 7 is that you could make your lossywav archive smaller if needed like transcode it to a lower quality setting with great safety. Example: Say I have a lossywav Q6 archive at home and I need high quality transcodable on-demand files for my internet radio station which is in another location. I could transcode Q6 archive to quality 3 and transcode that to a streaming format like layer 3 or he-aac. Its likely to work well.

Another scenario is sound processing and editing. I could edit Q6 archive dozens of times of my music projects. Another is a high quality backup - transcode Q6 to Q3 as my everyday archive keeping Q6 as the master on some other offline harddrive. the space used will be nearly the same as normal lossless in such a situation but there is the advantage of splitting archives across drives. These overkill setting are lossless replacements. In theory an online music store should be able to use Q6 / 7 to transcode to lossy with great confidence.
2Bdecided
QUOTE (shadowking @ Jul 17 2008, 10:36) *
Another scenario is sound processing and editing. I could edit Q6 archive dozens of times of my music projects. Another is a high quality backup - transcode Q6 to Q3 as my everyday archive keeping Q6 as the master on some other offline harddrive. the space used will be nearly the same as normal lossless in such a situation but there is the advantage of splitting archives across drives. These overkill setting are lossless replacements. In theory an online music store should be able to use Q6 / 7 to transcode to lossy with great confidence.
"Should" is the important word in all those statements. This needs some testing and ABXing before we can claim it comfortably. I did make a start, but got waylaid.

Cheers,
David.
botface
QUOTE (Axon @ Jul 16 2008, 19:02) *
To say nothing about vinyl. Gawd! My needledrop FLACs are shrinking by 45% at --standard!

If there ever was a demonstration of the reduced dynamic range of vinyl, lossyWAV has to be it.

I'm not sure that's a valid conclusion. 45% is about right for FLAC:LossyFLAC irrespective of the source. I did some tests a few months ago when I first came accross LossyWav. I didn't keep most of the results but I do still have one set of test data left. This was the first side of Dire Straits 1st LP (or first 5 tracks if you're looking at the CD).

For CD the figures were:
The uncompressed WAVs were 200 Mb
After FLAC processing (can't remember the level I used) it became 111Mb
Processing Via LossyWav --Standard/FLAC 5 it became 63.2Mb.
That's a reduction of around 43% compared with the FLAC'd data and 68.4%(ish) against the WAVs

For Vinyl the figures were:
The uncompressed WAVs (@16/44.1) were 194.4 Mb (presumably slightly less than CD due to speed inaccuracy of the turntable)
After FLAC processing (again I can't remember the level I used) it became 109Mb
Processing Via LossyWav --Standard/FLAC 5 it became 62.6Mb.
That's a reduction of around 43% compared with the FLAC'd data and 67.8%(ish) against the WAVs

I'd say they're pretty similar.

I also checked the same tracks recorded from vinyl at 24/48 to see what happened. The resultant LossyWav --Standard/FLAC 5 files came out at 64.7Mb. That's about 79% smaller than the original WAVs (which were 315.5Mb) but still bigger than the same files at 16/44.1 albeit by only a couple of Mb. However, I interpreted it to mean that recording from vinyl at 24/48 was worthwhile as LossWav was obviously finding some of the extra data to be worth keeping, although it may only be the potential extra data in the 22 - 24 kHz range, while the resulting files were only marginally bigger than the 16/44.1 ones.

Obviously one can't draw any hard conclusions from a few tracks but at the time I did get similar figures with several other albums in different genres.
2Bdecided
If you take a 44.1kHz file and simply resample it to 48kHz, I think you'll see the bitrate go up - losslessFLAC or lossyFLAC. There's no mechanism in either to directly recover the redundancy.

That has some relationship to your experiment, though not an exact one.


As you say though, it's interesting the difference was so small. It's not just the vinyl noise floor that lossyWAV is quantising during the music itself.

Cheers,
David.
botface
QUOTE (2Bdecided @ Jul 17 2008, 14:59) *
If you take a 44.1kHz file and simply resample it to 48kHz, I think you'll see the bitrate go up - losslessFLAC or lossyFLAC. There's no mechanism in either to directly recover the redundancy.

That has some relationship to your experiment, though not an exact one.


As you say though, it's interesting the difference was so small. It's not just the vinyl noise floor that lossyWAV is quantising during the music itself.

Cheers,
David.

Sorry, I don't understand your final paragraph - presumably my lack of understanding on the subject.

Using 24/48 will result in over 60% more data being produced than 16/44.1. I ended up with about 3% extra data after Lossywav and FLAC had processed it. What surprised me was that there was any extra data left at all and that LossyWav didn't "throw away" all of the extra as I was assuming that it would all be below the noise floor. As I say, I took it to mean that there was some useful data in that extra 60%
2Bdecided
Sorry - I didn't write that very well.

What I meant to say is that 24 or 16 probably makes little difference in this case, whereas 44.1 vs 48 will.

You could convert the 24/48 to 16/48 and then throw it at lossyWAV+FLAC and see if there's any difference at all. You might prove my guess to be completely wrong.

Cheers,
David.
raygrote
Lossy Wav really sounds very interesting. I've got a few questions about it.
1. How do I use Lossy Wav with Foober, or if there isn't a way, is their some kind of frontend for it?
2. If I compress a wav to flac, then run lossy flac, will the resulting file be much smaller than just running lossy wav on the wav file? I assume it wouldn't make too much difference, since to my understanding all lossy codecs use lossless compression in some way. Any info please?
Thanks.
botface
QUOTE (2Bdecided @ Jul 17 2008, 17:15) *
What I meant to say is that 24 or 16 probably makes little difference in this case, whereas 44.1 vs 48 will.

You could convert the 24/48 to 16/48 and then throw it at lossyWAV+FLAC and see if there's any difference at all. You might prove my guess to be completely wrong.

Cheers,
David.

Good idea. Interesting result.

I converted to 16/48 using Cooledit. I did it twice. 1st time I used triangular dither and no noise shaping (I didn't want to affect the high frequencies too much). The 2nd time I used no dither or noise shaping. In both cases I ran the resulting file through Lossywav --standard/FLAC -5. In both cases the file ended up at 64.36 Mb.

So, it would seem that most, but not all, of the increase in file size is due to hf content above 22050kHz - assuming my choice of dither/noise shaping was appropriate.

I guess that a much wider choice of material and larger numbers of files would be needed before any real conclusions could be drawn but as I said; interesting
carpman
QUOTE (raygrote @ Jul 17 2008, 19:50) *
1. How do I use Lossy Wav with Foober, or if there isn't a way, is their some kind of frontend for it?
2. If I compress a wav to flac, then run lossy flac, will the resulting file be much smaller than just running lossy wav on the wav file?

For 1 and 2 see: http://wiki.hydrogenaudio.org/index.php?title=LossyWAV
LossyWAV won't make the WAV file smaller, it's a pre-processor for lossless encoders, it processes the WAV file to make it easier for FLAC et al to compress it. The WAV pre-processing is lossy; the subsequent lossless encoding (i.e. to FLAC or TAK) is lossless.

C.

[EDIT: Typos ; EDIT2: Clarity ; EDIT3: Clarification -- OMG!!!]
Nick.C
^^ What he said.... smile.gif
carpman
Don't listen to me, just read the wiki; it's clearer and better written. biggrin.gif

C.
raygrote
Thanks for info.
It still doesn't work with Foober, though. I'm using Foober 0.9.5 and Lossy Wav 1.1.0. I just copied and pasted the commandlines posted on the wiki for flac files, and adjusted everything the way they said, all I did was change the paths to the lossywav and flac files. I set encoder to c:\windows\system32\cmd.exe, extention to lossy.flac, format is lossless )or hybrid), and max bitrate supported 24.
The commandline I enter is:
/d /c e:\"data files"\codecs\lossywav - --standard --silent --stdout| e:\"data files"\codecs\flac - -b 512 -5 -f -o%d
And the error I get is
"Source: "E:\data files\audio\abx1.wav"
An error occured while writing to file (The encoder has terminated prematurely with code 1; please re-check parameters) : "E:\data files\audio\abx1.lossy.flac"
Additional information:
Encoder stream format: 44100Hz / 2ch / 16bps
Command line: "C:\WINDOWS\SYSTEM32\cmd.exe" /d /c e:\"data files"\codecs\lossywav - --standard --silent --stdout| e:\"data files"\codecs\flac - -b 512 -5 -f -o"abx1.lossy.flac"
Working folder: E:\data files\audio\

Conversion failed: The encoder has terminated prematurely with code 1; please re-check parameters"


I just know I did something wrong.
carpman
QUOTE (raygrote @ Jul 17 2008, 21:38) *
I set encoder to c:\windows\system32\cmd.exe, extention to lossy.flac, format is lossless )or hybrid), and max bitrate supported 24.

Well that's all correct.

QUOTE (raygrote @ Jul 17 2008, 21:38) *
The commandline I enter is:
/d /c e:\"data files"\codecs\lossywav - --standard --silent --stdout| e:\"data files"\codecs\flac - -b 512 -5 -f -o%d

Above I've highlighted in red what may be a problem. Try it with a space between -o%d, so it's:
-o %d

Or, paste this into your parameters:
CODE
/d /c e:\"data files"\codecs\lossywav - --standard --silent --stdout| e:\"data files"\codecs\flac - -b 512 -5 -f -o %d

Try that first.

C.

[EDIT1] By the way Nick, if that is the problem the wiki needs changing as that's also in the wiki setting for FLAC.
[EDIT2] Just ran mine with -o%d (i.e. without the space) on foobar 9.4.3 and also 9.5.4 and it worked okay. I don't have any spaces (thus e:\"data files"\etc\etc ) in my encoders' addresses. So I'm out of ideas.
[EDIT3] Just ran it with identical parameters including no space with -o%d and with an address with spaces enclosed in " " and it worked fine.
halb27
QUOTE (raygrote @ Jul 17 2008, 22:38) *
... /d /c e:\"data files"\codecs\lossywav - --standard --silent --stdout| e:\"data files"\codecs\flac - -b 512 -5 -f -o%d ...

Guess it's the "data files" which causes the problems, or the space between the | and the flac path.
To find out I'd (temporarily?) use a lossywav/flac path like: e:\codecs\lossywav.
sauvage78
-o%d is ok, --stdout| e: maybe is not ... try --stdout|e: without space
Dynamic
QUOTE (botface @ Jul 17 2008, 20:12) *
So, it would seem that most, but not all, of the increase in file size is due to hf content above 22050kHz - assuming my choice of dither/noise shaping was appropriate.


Thanks to the way lossless encoders work, it's almost certainly down to the increased number of samples, because lossyWAV should ignore extra content above 16 kHz (and in any case, good sample rate conversion shouldn't add significant HF content not in the original). The fact there's no difference in bitrate between the two dither types when processed by lossyWAV is confirmation that your dither was below the original noise floor.

There are almost 9% more samples per second at 48kHz than at 44.1kHz, but with limited frequency response, it probably results in lower prediction error per sample (the error is what has to be stored by a lossless encoder), though this is not compensating fully for the 9% increase in number of samples per second.
raygrote
I tried all suggestions, nothing is working. I'm starting to think this may have something to do with cmd.exe? Later I'll try moving the codecs to the directory used in the wiki for the heck of it and see what happens.
Perhaps CMD.exe has problems with external hard drives?
Nick.C
QUOTE (raygrote @ Jul 18 2008, 00:49) *
I tried all suggestions, nothing is working. I'm starting to think this may have something to do with cmd.exe? Later I'll try moving the codecs to the directory used in the wiki for the heck of it and see what happens.
Perhaps CMD.exe has problems with external hard drives?
I have noticed that if I switch my server on and try to convert files with foobar2000 immediately I hear it finish its boot then the foobar2000 conversion will fail. However, if I use windows explorer to "look at" the root directory of the external drive where the audio files are first then the foobar2000 conversion will work perfectly.
botface
QUOTE (raygrote @ Jul 18 2008, 00:49) *
I tried all suggestions, nothing is working. I'm starting to think this may have something to do with cmd.exe? Later I'll try moving the codecs to the directory used in the wiki for the heck of it and see what happens.
Perhaps CMD.exe has problems with external hard drives?

I had a few problems when I had spaces in the directory, quotes or no quotes. So, I moved LossyWav and FLAC to a directory with no saces in the name (C:\auydio\lossywav) and it's worked perfectly ever since. In case it's of any interest here's my command line :
Parameters: /d /c c:\audio\lossywav\lossywav - -q 5 --silent --low --stdout|c:\audio\lossywav\flac - -b 512 -5 -o%d

(Yes, I'm still using the old "q" paramter instead of --standard).

I guess this shows how useful it would be if someone could knock up a GUI front end
2Bdecided
QUOTE (Dynamic @ Jul 18 2008, 00:32) *
There are almost 9% more samples per second at 48kHz than at 44.1kHz, but with limited frequency response, it probably results in lower prediction error per sample (the error is what has to be stored by a lossless encoder), though this is not compensating fully for the 9% increase in number of samples per second.
That's right. Even if you resample 44.1kHz to 48kHz (so there's no increase in the actual information - the extra frequency range is completely empty) the lossless bitrate will still go up. Most lossless codecs simply aren't designed to make use of this kind of redundancy (I think TAK might though).

Cheers,
David.
Synthetic Soul
QUOTE (Synthetic Soul @ Jul 16 2008, 14:42) *
Using Nick's figures above, it seems it is much of a muchness, although WavPack seems to benefit a little less (as can be seen by FLAC actually producing smaller bitrates with most settings):

CODE
             +------------------+------------------+------------------+
             |    TAK           |    FLAC          |  Wavpack         |
+------------+------------------+------------------+------------------+
| lossless   | 749 kbit/s       | 787 kbit/s        | 772 kbit/s      |
+------------+------------------+------------------+------------------+
| --insane   | 634 kbit/s (85%) | 654 kbit/s (83%) | 661 kbit/s (86%) |
| --extreme  | 564 kbit/s (75%) | 583 kbit/s (74%) | 599 kbit/s (78%) |
| --standard | 488 kbit/s (65%) | 508 kbit/s (65%) | 529 kbit/s (69%) |
| --portable | 405 kbit/s (54%) | 425 kbit/s (54%) | 444 kbit/s (58%) |
| -q 0       | 299 kbit/s (40%) | 321 kbit/s (41%) | 342 kbit/s (44%) |
+------------+------------------+------------------+------------------+

In addition, here's some figures for the corpus that I use for my lossless comparison:

CODE
              TAK               FLAC              Wavpack
================================================================
lossless      917 kbps          941 kbps          938 kbps
================================================================
--insane      629 kbps (69%)    639 kbps (68%)    661 kbps (70%)
--extreme     538 kbps (59%)    548 kbps (58%)    574 kbps (61%)
--standard    448 kbps (49%)    459 kbps (49%)    485 kbps (52%)
--portable    362 kbps (39%)    375 kbps (40%)    394 kbps (42%)

As per Nick's results, the difference between codecs is minimal; however my corpus benefits a lot more from LossyWAV preprocessing.

Once again, thank you all for your hard work.
halb27
QUOTE (Synthetic Soul @ Jul 18 2008, 12:45) *
... however my corpus benefits a lot more from LossyWAV preprocessing. ...

The results for your corpus are pretty similar to the results of my small 12 full length regular track test set, and these are very similar to those of Nick's many-albums test set.
If there's one thing I don't like with Nick's great work it's his preferred usage of his test set consisting of problem snippets for the biggest part of it which I'm afraid leads to a wrong impression to most people. huh.gif
Nick.C
QUOTE (halb27 @ Jul 18 2008, 12:33) *
If there's one thing I don't like with Nick's great work it's his preferred usage of his test set consisting of problem snippets for the biggest part of it which I'm afraid leads to a wrong impression to most people. huh.gif
As we're past the second full release, I'll produce a similar table to that above using my 10 album test set. The reason I stick to the 55 problem sample set is that it is (relatively, as I have added a few samples to it over the last six months) constant and relative changes in bitrates at quality presets can be compared. No need to stick to that now - I'll publish some bitrates tonight.
Nick.C
QUOTE (halb27 @ Jul 18 2008, 12:33) *
If there's one thing I don't like with Nick's great work it's his preferred usage of his test set consisting of problem snippets for the biggest part of it which I'm afraid leads to a wrong impression to most people. huh.gif
As requested - results for my 10 album test set:
CODE
             +------------------+------------------+------------------+
             |       TAK        |       FLAC       |     Wavpack      |
+------------+------------------+------------------+------------------+
| lossless   | 820 kbit/s       | 854 kbit/s       | 852 kbit/s       |
+------------+------------------+------------------+------------------+
| --insane   | 615 kbit/s (75%) | 632 kbit/s (74%) | 641 kbit/s (75%) |
| --extreme  | 532 kbit/s (65%) | 548 kbit/s (64%) | 563 kbit/s (66%) |
| --standard | 447 kbit/s (55%) | 463 kbit/s (54%) | 481 kbit/s (56%) |
| --portable | 359 kbit/s (44%) | 376 kbit/s (44%) | 390 kbit/s (46%) |
| -q 0       | 266 kbit/s (32%) | 285 kbit/s (33%) | 296 kbit/s (35%) |
+------------+------------------+------------------+------------------+
Wiki article indicative bitrate table amended to reflect these results.
halb27
QUOTE (Nick.C @ Jul 19 2008, 21:30) *
QUOTE (halb27 @ Jul 18 2008, 12:33) *
If there's one thing I don't like with Nick's great work it's his preferred usage of his test set consisting of problem snippets for the biggest part of it which I'm afraid leads to a wrong impression to most people. huh.gif
As requested - results for my 10 album test set:
CODE
             +------------------+------------------+------------------+
             |       TAK        |       FLAC       |     Wavpack      |
+------------+------------------+------------------+------------------+
| lossless   | 820 kbit/s       | 854 kbit/s       | 852 kbit/s       |
+------------+------------------+------------------+------------------+
| --insane   | 615 kbit/s (75%) | 632 kbit/s (74%) | 641 kbit/s (75%) |
| --extreme  | 532 kbit/s (65%) | 548 kbit/s (64%) | 563 kbit/s (66%) |
| --standard | 447 kbit/s (55%) | 463 kbit/s (54%) | 481 kbit/s (56%) |
| --portable | 359 kbit/s (44%) | 376 kbit/s (44%) | 390 kbit/s (46%) |
| -q 0       | 266 kbit/s (32%) | 285 kbit/s (33%) | 296 kbit/s (35%) |
+------------+------------------+------------------+------------------+
Wiki article indicative bitrate table amended to reflect these results.

Thank you, Nick. A near-perfect match with Synthetic Soul's results as well as mine (371/462/542 kbps for --portable/--standard/--extreme when using FLAC).
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2009 Invision Power Services, Inc.