Help - Search - Members - Calendar
Full Version: lossyWAV 1.1.0 Development Thread.
Hydrogenaudio Forums > Lossy Audio Compression > Other Lossy Codecs
Pages: 1, 2, 3, 4, 5, 6
shadowking
QUOTE(collector @ May 24 2008, 22:18) *

QUOTE(shadowking @ May 23 2008, 20:47) *

Your are kind off right. I still think lossywav can take on and beat Lame 320k at similar bitrates simply due to the mp3 native defect.

In bitrate/quality perhaps, but not in filesize. Average filesize of my testset Q1 lossyflacs is almost twice the size of my mp3 V1, let alone V4. DAP users won't like that.


No arguments there. I think the typical losswav user is:

- Experienced user wanting very high quality and predictable transcoding at half or less bitrate of lossless. Maybe this applies to a clueless user too.
- A more naive user want good enough Flac sound in his / her DAP saving themselves transcoding. They can use lower bitrates and still be very happy. They really should make correction files because normal codec @ 160..190k is similar quality to 260 k hybrid but they are much smaller.
no404error
QUOTE(collector @ May 24 2008, 14:18) *

In bitrate/quality perhaps, but not in filesize.

Really? smile.gif For bitrate above 256, lossywav better then anybody lossy-codecs.

CODE

lame --preset insane - 9.502.369
lame --abr 280 - 8.653.434
lame -b 256 - 7.636.113

oggenc -b 270 - 7.961.898
oggenc -b 280 - 8.350.866

lossywav -q 0 / flac -5 -b 512 --keep-foreign-metadata - 8.087.723

It is an example, but I will make a test for 88 albums if you need.
[JAZ]
QUOTE(collector @ May 24 2008, 13:18) *

In bitrate/quality perhaps, but not in filesize. Average filesize of my testset Q1 lossyflacs is almost twice the size of my mp3 V1, let alone V4. DAP users won't like that.


filesize = duration*bitrate.

You've got a point with your reply, but the point itself is a bit out of place:

-V1 or -V4 are not CBR 320kbps ( just like the number 3 is not the number 5 ).
When someone uses such a high bitrate with mp3, the main reason is to get as high quality as possible, with a compatible-enough format. (i.e. not just "transparency")

The point of lossyWav origin was to couple it with the most supported lossless codec (or so was the pretention). Nowadays, lossywav not only works with FLAC, but with other lossless codecs, so possibly increasing the range of situations where it can be used.

Said that, what an user may want from a CBR 320kbps file, may more probably be achieved with lossywav -q 4 or -q 5, which is around 400~450kbps.
But 400~450kbps is the half of what the lossless file requires.

Personally, i see -q 0 and -q 1 as a way to give a lossless-like file at a reduced bitrate, with the added advantage that if the produced file sounds wrong, the quality setting can go up until it dissapears. (not like cbr 320)
halb27
QUOTE(carpman @ May 24 2008, 06:07) *

I've lost touch a little. Am I right in thinking:
10 is pretty much the old -1
7.5 is pretty much the old -2
5 is pretty much the old -3

It's rather:
--extreme (-q 7.5) is old -1
--standard (-q 5.0) is old -2
--portable (-q 2.5) is old -3.
QUOTE(carpman @ May 24 2008, 06:07) *

... I wouldn't go any lower than 2.5, especially since transparency is pretty much guaranteed with MP3 at lower bitrates, why would you choose a "mp3 replacement" (or whatever term you like) when LossyWAV, at sub 2.5 bitrates, is not as safe as MP3? ...

I feel like you for the lowest quality settings though due to the mp3 restrictions shadowking mentioned on occasion lossyWAV may yield the better quality even below 300 kbps.
The real benefits of lossyWAV start pretty much exactly where mp3 has its highest bitrate. So lossyWAV --portable drops in fine where mp3 quality stops.
Moreover with nowaday's DAPs capacities a bitrate of more than 300 kbps can be used without pain by more and more people.

But as is yours this is just my opinion towards a very low bitrate setting.
No matter that I wouldn't like to use a very low bitrate setting I wouldn't care however to have such a quality preset. Maybe it's just asthetics, but with 2 quality presets above --standard it would be nice IMO to have 2 below it. Maybe the lowest preset quality shouldn't be exactly -q 0 (though this would perfectly correspond to the -q setting / quality preset mapping we have so far). v1.0.1k --portable yields a bitrate of 362 kbps with my full length regular music test set. Maybe -q 1 (307 kbps) or -q 1.5 (327 kbps) may be an attractive candidate for the lowest preset quality. Both are quite a bit more attractive than --portable bitratewise, and quality even for problem samples is expected not to be annoying (at least for -q 1.5).

So I'd like to see a lowest quality preset, and my proposal is -q 1.5 for the incarnation. My proposal for the name is just '--low' (pretty much of an understatement at least for -q 1.5, but IMO this matches the quality scale of the other quality presets).

ADDED later:
It just comes to my mind that in a way lossyWAV has a BIG advantage over other lossy codecs: it is LOSSLESSLY FUTURE-SAFE.
Assume we use FLAC as the final codec. Assume in 10 years from now you can't play FLAC on your DAP. You then simply transcode to the then preferred lossless codec which is a lossless process.
Taking this into account it's ok IMO to call the lowest qualty preset (in case it's a little bit higher than -q 0) an 'mp3 alternative' - an alternative of comparable quality as very high bitrate mp3 but with this extra advantage of future-safety.
The Sheep of DEATH
I'm sorry I didn't follow this thread as closely as I would have liked, but I've observed that LossyWAV 1.01k produces significantly higher bitrates at q0 than 1.01f, which had the lowest I've seen so far and which I was quite satisfied with. Is there any possibly of remapping the quality scale such that that the low bitrate of 1.01f is reestablished at q0? It seems a lot like I'm going up a whole quality scale (0 to 1 or the like) from f to k with the same settings...
lvqcl
I tried to encode "ginnungagap..." sample using lossyWAV 1.0.1k and various options.
For me, "-q 0 -m 5.333" sounds better than "-q 1" for that particular sample. Both results in 335 kbps bitrate.
collector
QUOTE
' date='May 24 2008, 05:47' post='566920']
You've got a point with your reply, but the point itself is a bit out of place:
-V1 or -V4 are not CBR 320kbps ( just like the number 3 is not the number 5 ).
When someone uses such a high bitrate with mp3, the main reason is to get as high quality as possible, with a compatible-enough format. (i.e. not just "transparency")

V4 is not that high with 160 kbps. And I've seen numerous discussions about smaller filesizes for limited diskspace on mp3-DAPs. But I only interfered here because the lower q settin was proposed to be 'mp3' somewhere. Whether transcoding from a flac image to lower flac or to mp3 doesn't matter for me..

The names are only important to the non-technical users. Tweakers will test the various q-settings.
halb27
QUOTE(lvqcl @ May 24 2008, 16:30) *

I tried to encode "ginnungagap..." sample using lossyWAV 1.0.1k and various options.
For me, "-q 0 -m 5.333" sounds better than "-q 1" for that particular sample. Both results in 335 kbps bitrate.

Sounds like there's hope for improving the very low bitrate settings.
collector
BTW, Nick, thanks for changing the scale factor and for adding the --writelog.

All of a sudden redirecting stopped working for "lossywav --longhelp>>longhelp.txt" a few versions ago. crying.gif
sauvage78
... I am really perplex about people telling lossywav -q0 or -q1 is a viable alternative to any modern lossy codec at VBR 256 or 320Kbps for any use ... first using 320Kbps instead of 256Kbps even for a paranoid user is always sub-optimal, no matter the modern lossy codec 256Kbps is always a better choice than 320Kbps, 320Kbps has always been the "my-bitrate-is-higher-than-yours-so-my-files-are-better-than-yours" setting of newbies, if there would be a 321Kbps setting these knowledge-less newbies would use it anyway.
So I wouldn't compare lossywav -q 0/-q 1 to lossy 320Kbps but to lossy 256Kbps.

Now comparing any modern lossy codec at 256Kbps with lossywav -q0, you compare, heavyly tested techniques (DCT), on files that are smaller & 99.9% transparent (& even when it's not transparent, it's listenable & only ABXable by people with golden ears on specific problems samples) ... with a technique that is still beta, have been proven faulty at -q0 & -q1 & produces bigger files.

I like lossywav, but I would moderate your enthousiasm, there is no way lossywav -q0 or -q1 actually beats any modern lossy codec at 256kbps transparency-wise or by quality vs. space. As far as I tested any modern lossy codec is more likely to be transparent at 256Kbps VBR than lossywav -q 0.

At -q2.5 it seems it achieves the same quality as pure lossy 256Kbps (transparency), but what you gain in transcodability you lose it in hard disk space, there is no magic. As far as I tested lossywav is not the magic codec that have all the advantage of lossless (quality) & lossy (space) ...

I see it as spreading audio bullshit ... so please people stop telling lossywav -q 0 is "almost" the same as lossless, it is definitly not. That being said I understand the enthousiasm as lossywav is the first really interesting hybrid codec IMHO. It seems a lot of people are using -q 0 thinking its quality is like -q 2.5 (transparent) ... it's not & that's exactly why it was left out of the presets. If you want an alternative to lossy 256Kbps, use -q2.5.
lvqcl
QUOTE(halb27 @ May 24 2008, 18:39) *

QUOTE(lvqcl @ May 24 2008, 16:30) *

I tried to encode "ginnungagap..." sample using lossyWAV 1.0.1k and various options.
For me, "-q 0 -m 5.333" sounds better than "-q 1" for that particular sample. Both results in 335 kbps bitrate.

Sounds like there's hope for improving the very low bitrate settings.

Well, if noise added by lossyWAV is big enough so one can hear it -- he also can hear how it changes its volume (by 6db) several times per second. And these jumps are more annoying for me than the noise itself. huh.gif
Nick.C
QUOTE(collector @ May 24 2008, 15:40) *
BTW, Nick, thanks for changing the scale factor and for adding the --writelog.

All of a sudden redirecting stopped working for "lossywav --longhelp>>longhelp.txt" a few versions ago. crying.gif
Sorry - all text output now goes to "con" and not STDOUT - so that the --stdout parameter works correctly.

I'll make sure that I include the --longhelp text with each beta release at post #1.


QUOTE(lvqcl @ May 24 2008, 16:30) *
QUOTE(halb27 @ May 24 2008, 18:39) *
QUOTE(lvqcl @ May 24 2008, 16:30) *
I tried to encode "ginnungagap..." sample using lossyWAV 1.0.1k and various options.
For me, "-q 0 -m 5.333" sounds better than "-q 1" for that particular sample. Both results in 335 kbps bitrate.
Sounds like there's hope for improving the very low bitrate settings.
Well, if noise added by lossyWAV is big enough so one can hear it -- he also can hear how it changes its volume (by 6db) several times per second. And these jumps are more annoying for me than the noise itself. huh.gif
Unfortunately, bits being what they are, if you round off an extra bit the noise will jump by 6dB.

My suggestion would be to use noise shaping and possibly increase --minbits from 2.333. You may also want to push --limit up to circa 17kHz.


QUOTE(sauvage78 @ May 24 2008, 16:08) *
... I am really perplex about people telling lossywav -q0 or -q1 is a viable alternative to any modern lossy codec at VBR 256 or 320Kbps for any use ... first using 320Kbps instead of 256Kbps even for a paranoid user is always sub-optimal, no matter the modern lossy codec 256Kbps is always a better choice than 320Kbps, 320Kbps has always been the "my-bitrate-is-higher-than-yours-so-my-files-are-better-than-yours" setting of newbies, if there would be a 321Kbps setting these knowledge-less newbies would use it anyway.
So I wouldn't compare lossywav -q 0/-q 1 to lossy 320Kbps but to lossy 256Kbps.

Now comparing any modern lossy codec at 256Kbps with lossywav -q0, you compare, heavyly tested techniques (DCT), on files that are smaller & 99.9% transparent (& even when it's not transparent, it's listenable & only ABXable by people with golden ears on specific problems samples) ... with a technique that is still beta, have been proven faulty at -q0 & -q1 & produces bigger files.

I like lossywav, but I would moderate your enthousiasm, there is no way lossywav -q0 or -q1 actually beats any modern lossy codec at 256kbps transparency-wise or by quality vs. space. As far as I tested any modern lossy codec is more likely to be transparent than lossywav -q 0.

At -q2.5 it seems it achieves the same quality as pure lossy 256Kbps (transparency), but what you gain in transcodability you lose it in hard disk space, there is no magic. As far as I tested lossywav is not the magic codec that have all the advantage of lossless (quality) & lossy (space) ...

I see it as spreading audio bullshit ... so please people stop telling lossywav -q 0 is "almost" the same as lossless, it is definitly not. That being said I understand the enthousiasm as lossywav is the first really interesting hybrid codec IMHO. It seems a lot of people are using -q 0 thinking its quality is like -q 2.5 (transparent) ... it's not & that's exactly why it was left out of the presets. If you want an alternative to lossy 256Kbps, use -q2.5.
Go and read the development thread. The lowest quality presets (also, coincidentally numerically the lowest....) are not pretending to be anything other than a lower bitrate version of the higher quality presets - but with a greater probability of artifacts. The lowest bitrate settings have always been pushed by myself and a few others who wish a lower bitrate version and are aware enough of the method not to fool themselves into thinking that the resulting files are comparable in any way to lossless.

If you don't like what lossyWAV is - it's a free world (thus far) - so don't use it.

As to "there is no way lossywav -q0 or -q1 actually beats any modern lossy codec at 256kbps transparency-wise" - have you posted your ABX logs on this one. While you're at it go searching for "killer samples" and see how other methods deal with them. Transparent at 320kbps - not always....


QUOTE(The Sheep of DEATH @ May 24 2008, 15:21) *
I'm sorry I didn't follow this thread as closely as I would have liked, but I've observed that LossyWAV 1.01k produces significantly higher bitrates at q0 than 1.01f, which had the lowest I've seen so far and which I was quite satisfied with. Is there any possibly of remapping the quality scale such that that the low bitrate of 1.01f is reestablished at q0? It seems a lot like I'm going up a whole quality scale (0 to 1 or the like) from f to k with the same settings...
You could try: -q 0 -m 1.5 --spf 22222-22223-22224-12234-12345-12356 as this should revert to the previous spreading function that you were happy with and also I remember that in 1.0.1f the dynamic minimum-bits-to-keep parameter was not being applied. Have fun!
shadowking
QUOTE(halb27 @ May 22 2008, 04:29) *

QUOTE(Nick.C @ May 21 2008, 19:40) *

QUOTE(Mardel @ May 21 2008, 18:27) *
There is a somthing up.
I'm hearing heavily distorted sound at -q0.
Thanks for the sample, at what position in the audio are you hearing the distortion? ...

I just abxed sec. 6.4-8.6 8/10 (and I'm sure those who know this track will do better). I wouldn't call this spot heavily distorted though, but that's a matter of taste.
We know -q 0 isn't a real quality mode (though usually it's quite okay). My personal favorite for low bitrate is -q 1.5. It's not significantly higher in bitrate (312 kbps on average for my regular music test set) but to me quality does a little jump there. The spot I mentioned with your track, Mardel, is ok to me using -q 1.5.

Mardel and sauvage78, do you mind trying -q 1.5?


Hmmm.. Its really obvious and i didn't even try to abx - blocky flactuating noise @ 300 k . Its what i hear with wavpack @ 196 k . The trouble is that the track isn't very dynamic either so there could be much bigger problems for classical music. Its too dangerous (Q0) IMO for lossywav. Q0 could be a quality collapse zone.
Nick.C
QUOTE(shadowking @ May 24 2008, 16:53) *
QUOTE(halb27 @ May 22 2008, 04:29) *
QUOTE(Nick.C @ May 21 2008, 19:40) *
QUOTE(Mardel @ May 21 2008, 18:27) *
There is a somthing up.
I'm hearing heavily distorted sound at -q0.
Thanks for the sample, at what position in the audio are you hearing the distortion? ...
I just abxed sec. 6.4-8.6 8/10 (and I'm sure those who know this track will do better). I wouldn't call this spot heavily distorted though, but that's a matter of taste.
We know -q 0 isn't a real quality mode (though usually it's quite okay). My personal favorite for low bitrate is -q 1.5. It's not significantly higher in bitrate (312 kbps on average for my regular music test set) but to me quality does a little jump there. The spot I mentioned with your track, Mardel, is ok to me using -q 1.5.

Mardel and sauvage78, do you mind trying -q 1.5?
Hmmm.. Its really obvious and i didn't even try to abx - blocky flactuating noise @ 300 k . Its what i hear with wavpack @ 196 k . The trouble is that the track isn't very dynamic either so there could be much bigger problems for classical music. Its too dangerous (Q0) IMO for lossywav. Q0 could be a quality collapse zone.
Maybe this would be a problem that could be dealt with by using an FFT length longer than 1024 samples, or a codec-block length greater than 512 samples.

We kept the codec-block length short to specifically allow the bits-to-remove to change rapidly rather than calculating over longer blocks. This may, at -q 0, be counter productive. As David said earlier - if no artifacts are noticable at -q 0 there will be questions as to why the bitrate is not pushed lower by more aggressive settings.

As an aside, I've just finished transcoding the portion of my collection in FLAC on my server (3838 tracks, 11 days 12 hours) : 102GB, 886kbps FLAC > lossyWAV --portable / FLAC -5 -b 512: 42.1GB, 363kbps.
shadowking
I've encoded that sample with wavpack 4.5b @ 235 kbit -x ABR and the quality is way better. Tried wavpack @ 196 k and I still find the quality better. Perhaps low bitrate + noiseshaping + ABR has a strength here.

Abx:

-lossywav sample - obvious - no need
- Wv 196 - obvious - no need
- Wv 235 - hiss on drum crunch 8/8 easy
- Wv 270 - can't abx

Maybe its not a fair comparison as lossywav is working way outside its quality trigger areas here while wavpack can often operate nicely even in lowish bitrates vs other hybrids (shorten, rkau, maybe losswav) as Bryant has stated once.
Nick.C
QUOTE(shadowking @ May 24 2008, 17:08) *
I've encoded that sample with wavpack 4.5b @ 235 kbit -x ABR and the quality is way better. Tried wavpack @ 196 k and I still find the quality better. Perhaps low bitrate + noiseshaping + ABR has a strength here.

Maybe its not a fair comparison as lossywav is working way outside its quality trigger areas here while wavpack can often operate nicely even in lowish bitrates vs other hybrids (shorten, rkau, maybe losswav) as Bryant has stated once.
I think that this sample is one of those where the particular properties which cause lossyWAV to produce suboptimal output need to be determined, understood and mitigated. It reminds me of when we were having problems with Atem_Lied and David Bryant pointed out that the lowest bins were in the circa 200Hz area - we fixed that with the skewing parameter....

Time to get the thinking cap out again and work out this particular problem.
shadowking
I've been abx fatigued for a while + a switch to linux. I've got foobar abx working with WINE and my strength is coming back so hopefully I can provide more abx input and samples. I still need to setup lossywav encoder settings etc.
halb27
@shadowking:
Yes, I think wavPack lossy does a better job below 300 kbps, as is expected from most of the usual codecs at a comparable bitrate.
I guess it's favorable in this bitrate area that wavPack lossy has a) the lossy variant integrated into the usual machinery with especially no restrictions to the blocksize and b) a dynamic noise shaping procedure which seems to work pretty well.
Moreover lossyWAV at such a low bitrate runs very much below its safe quality control mechanisms so that it's rather a miracle that it does work so relatively good.
The low bitrate settings have the advantage of being easy to test and to improve the machinery. Maybe enlarging the minimum_bits_to_keep is enough to increase the quality.

Anyway and apart from all improvement we will get hopefully I'd like to repeat my proposal for another '--low' preset which maps to 1.0.1k's -q 1.5.
It may bring down the discussion a bit on -q 0.
shadowking
Sure I understand. People naturaly explore the lowest settings in any codec. Wavpack newbies probably went straight for 196 k esp non HA people, then were shocked.
Nick.C
QUOTE(halb27 @ May 24 2008, 18:12) *
@shadowking:
Yes, I think wavPack lossy does a better job below 300 kbps, as is expected from most of the usual codecs at a comparable bitrate.
I guess it's favorable in this bitrate area that wavPack lossy has a) the lossy variant integrated into the usual machinery with especially no restrictions to the blocksize and b) a dynamic noise shaping procedure which seems to work pretty well.
Moreover lossyWAV at such a low bitrate runs very much below its safe quality control mechanisms so that it's rather a miracle that it does work so relatively good.
The low bitrate settings have the advantage of being easy to test and to improve the machinery. Maybe enlarging the minimum_bits_to_keep is enough to increase the quality.

Anyway and apart from all improvement we will get hopefully I'd like to repeat my proposal for another '--low' preset which maps to 1.0.1k's -q 1.5.
It may bring down the discussion a bit on -q 0.
Yes - a --lowest parameter might be a good idea. I'm going to look more closely at the detailled output for Mardel's sample (at present the --detail parameter only outputs one channel.... blink.gif ). I will also have a look at implementing an optionto limit the increase in number of bits to remove between blocks (something we tried before but discounted.....).

@shadowking - your ear-time will be greatly appreciated. smile.gif
halb27
Motivated by lvqcl's good experience with a higher -m value I played around a bit with my full length regular music track set:

a) bitrate increase for -m 5:

--insane:     611 kbps > 611 kbps
--extreme:  534 kbps > 534 kbps
--standard: 452 kbps > 460 kbps
--portable:  362 kbps > 385 kbps
-q 1.5:        327 kbps > 358 kbps
-q 0:           266 kbps > 332 kbps

b) bitrate increase for -m 4:

--standard: 452 kbps > 453 kbps
--portable:  362 kbps > 367 kbps
-q 1.5:        327 kbps > 334 kbps
-q 0:           266 kbps > 290 kbps

Looking at this it looks reasonable to use -m 5 for the -q settings which correspond to --insane, --extreme, and maybe --standard.
For the lower settings the bitrate penalty is a bit high though we may well reconsider the internal mappings of -q x to -nts y and -snr z in case a relatively high -m value proves to be essential.
-m 4 however IMO is appropriate also for the low bitrate settings (if we allow -q 0 to go up significantly in bitrate), and we should never use a lower -m value.

Thinking about the meaning of minimum-bits-to-keep I wonder now about the use of a very low value of ~2.5. I guess we should want to have a little bit more bits to be kept even for the lowest setting.

So I think lvqcl's listening test with a relatively high -m value (compared to what we had so far) is very valuable, and going up with -m was already taken into consideration by yourself, Nick.
shadowking
What about mapping unsafe settings like Q0 or similar to Q 1.5 ?? Advanced users can manually use switches to get the real Q0. The lame devs did similar mappings.
Nick.C
QUOTE(shadowking @ May 24 2008, 19:38) *
What about mapping unsafe settings like Q0 or similar to Q 1.5 ?? Advanced users can manually use switches to get the real Q0. The lame devs did similar mappings.
Do you mean make the "new" -q 0 aka --lowest = current -q 1.5? This would require all of the internal parameters that go to make up the preset to be available from the command line (no real problem).

or a "hidden" preset selection parameter for old -q 0? ph34r.gif
halb27
I guess -m 4 for -q <5, -m 5 for -q >= 5 (or a gliding -m like -m 4+0.15*q), and a --low preset which maps to -q 1.5 does it all.
lvqcl
I downloaded source code for 1.0.0b version.
Nick.C, can I ask you to add a new option? BTRlimit or something like this.
Then, in procedure process_this_codec_block insert one line
CODE
if min_fft_result.btr>BTRlimit then min_fft_result.btr:=BTRlimit;

just before calling remove_bits:

CODE
...
          if min_fft_result.sminbin>min_fft_result.savebin then
            fft_average:=fft_average+1
          else
            fft_minimum:=fft_minimum+1;
        end;
      end;

>>  if min_fft_result.btr>BTRlimit then min_fft_result.btr:=BTRlimit; << that line

    remove_bits;

    if (detailled_output=1) and (silent=0) then
      remove_bits_output
    else
      inc(current_codec_block);
...
Nick.C
QUOTE(lvqcl @ May 24 2008, 21:42) *
I downloaded source code for 1.0.0b version.
Nick.C, can I ask you to add a new option? BTRlimit or something like this.
Then, in procedure process_this_codec_block insert one line
CODE
if min_fft_result.btr>BTRlimit then min_fft_result.btr:=BTRlimit;

just before calling remove_bits:

CODE
...
          if min_fft_result.sminbin>min_fft_result.savebin then
            fft_average:=fft_average+1
          else
            fft_minimum:=fft_minimum+1;
        end;
      end;

>>  if min_fft_result.btr>BTRlimit then min_fft_result.btr:=BTRlimit; << that line

    remove_bits;

    if (detailled_output=1) and (silent=0) then
      remove_bits_output
    else
      inc(current_codec_block);
...
I'd already re-introduced that as the static_maximum_bits_to_remove method I reintroduced at 1.0.1k.

However.... Thanks to Mardel's sample and my desire to see what was happening with it, I "fixed" the detailled output to show per-channel bits-to-remove and I think I have found a problem whereby some channels are having more bits-to-remove than I think they should. This will not necessarily be a quick fix. I would hope to post beta 1.0.1m tomorrow night.
Nick.C
QUOTE(Nick.C @ May 24 2008, 22:11) *
However.... Thanks to Mardel's sample and my desire to see what was happening with it, I "fixed" the detailled output to show per-channel bits-to-remove and I think I have found a problem whereby some channels are having more bits-to-remove than I think they should. This will not necessarily be a quick fix. I would hope to post beta 1.0.1m tomorrow night.
I didn't find anything, although I had suspicions. However, I re-implemented the old max-inter-block-change in bits to remove (bits-to-remove can only increase by 1 bit per codec block). This wastes a few bits but should limit the change in noise level to 6dB every 11.6ms (for 44.1kHz audio). I would appreciate feedback to see if this has improved Mardel's sample at all at -q 0.

lossyWAV beta 1.0.1m attached to post #1 in this thread.
sauvage78
later today I will read the last pages of this thread, I am willing to test Mardel's sample at -q 0 V1.0.1l vs. -q 0 V1.0.1m, it shouldn't be too long/hard, unfortunatly I don't have V1.0.1l, I only have V1.0.1j & m, so if someone could repost V1.0.1l, I could do the job. Also if you have any sample with heavy problem at -q 0 to -q 2 like Mardel's sample, I am interested as long as it's not beyond my earing possibilities.

I also plan to do a quick transcodability test for my own use, I'll pass the Castanets sample thru lossywav -q 2.5 + nero aac VBR 96Kbps vs direct nero aac VBR 96Kbps & see if I can ear a difference, I test at 96Kbps because at 128Kbps it becomes hard to ABX (but still possible) & at 64Kbps with SBR I think it's too agressive if there would be a difference it would be destroyed anyway.
Nick.C
QUOTE(sauvage78 @ May 26 2008, 09:55) *
later today I will read the last pages of this thread, I am willing to test Mardel's sample at -q 0 V1.0.1l vs. -q 0 V1.0.1m, it shouldn't be too long/hard, unfortunatly I don't have V1.0.1l, I only have V1.0.1j & m, so if someone could repost V1.0.1l, I could do the job. Also if you have any sample with heavy problem at -q 0 to -q 2 like Mardel's sample, I am interested as long as it's not beyond my earing possibilities.

I also plan to do a quick transcodability test for my own use, I'll pass the Castanets sample thru lossywav -q 2.5 + nero aac VBR 96Kbps vs direct nero aac VBR 96Kbps & see if I can ear a difference, I test at 96Kbps because at 128Kbps it becomes hard to ABX (but still possible) & at 64Kbps with SBR I think it's too agressive if there would be a difference it would be destroyed anyway.
Thanks for volunteering your listening time, it is much appreciated. The beta 1.0.1l that you are looking for never existed as the character 'l' is too similar to '1' and '|' in some fonts. Things have moved on a bit and I have made enough changes to warrant the release of a new beta version.

From memory (correct me if I'm wrong, halb27) eig, triangle_2_1644ds, furious and badvilbel would be good (i.e. problematic) samples to try.

lossyWAV 1.0.1n attached to post #1 in this thread.
sauvage78
sorry I was meaning V1.01k,

I started by my little transcoding test ... so far it's an ABX fiasco crying.gif , I couldn't tell the difference between a Direct nero AAC Q0.35 (96Kbps) Vs. a Lossywav -q2.5+Nero Q0.35 (96Kbps) transcoding ... I tried at Nero 64Kbps & I failed too, everything sounded terrible but I couldn't spot any regression due to lossywav q2.5 ... so I am currently testing a nero Q0.65 (256Kbps)+nero Q0.35 (96Kbps) transcoding Vs. a lossywav q2.5+nero Q0.35 (96Kbps) transcoding, just to see how bad is nero aac 256Kbps as a source compared lossywav q2.5 which I can't ABX. All this on the castanets sample.
Nick.C
QUOTE(sauvage78 @ May 26 2008, 12:55) *
sorry I was meaning V1.01k,

I started by my little transcoding test ... so far it's an ABX fiasco crying.gif , I couldn't tell the difference between a Direct nero AAC Q0.35 (96Kbps) Vs. a Lossywav -q2.5+Nero Q0.35 (96Kbps) transcoding ... I tried at Nero 64Kbps & I failed too, everything sounded terrible but I couldn't spot any regression due to lossywav q2.5 ... so I am currently testing a nero Q0.65 (256Kbps)+nero Q0.35 (96Kbps) transcoding Vs. a lossywav q2.5+nero Q0.35 (96Kbps) transcoding, just to see how bad is nero aac 256Kbps as a source compared lossywav q2.5 which I can't ABX. All this on the castanets sample.
In terms of ABX'ing lossyWAV output, the most beneficial (to me) method would be to ABX the lossless original sample against the lossyWAV output rather than ABX processed samples from two versions of lossyWAV. I am very interested to find out if the quality of Mardel's sample when processed with 1.0.1n is as easy to ABX as it was with the lossyWAV version available when he introduced the sample.
sauvage78
Ok, I will test it soon.

If anyone is interested by the result of my personnal little transcoding test, lossywav -q 2.5 vs. Nero q0.65 as a source, it happened that the result was a success: lossywav -q 2.5 is a better source than Nero q0.65 for transcoding, the usual DCT artefact for the castanets sample (flat metallic attack) was slightly more pronounced with nero q0.65 as a source. In the end it is not surprising, but even if I can ABX it 9/12, I wouldn't tell the difference was incredible, you have to know exactly what to listen to, close your eyes & concentrates. So for me the interest of lossywav relies much more in its "splittability+no psychoacoustic" (CDImage.lossy.tak as archive) than in its "transcodability" alone. For me lossywav stands as a lossy codec by itself ... if I ever start using it when it will become stable, I think I will simply stop using Nero AAC & DCT codecs.
sauvage78
Ok, as you wanted me to ABX each lossywav version separatly vs. original, I first tested Lossywav V1.01n -q 1.0 Vs. Original:

foo_abx 1.3.3 report
foobar2000 v0.9.5.3
2008/05/26 15:36:25

File A: C:\Documents and Settings\Bureau\Test\Ginnungagap.lossyV1.01n-q1.wav
File B: C:\Documents and Settings\Bureau\Test\Ginnungagap.original.wav

15:36:25 : Test started.
15:36:44 : 01/01 50.0%
15:37:07 : 02/02 25.0%
15:37:28 : 03/03 12.5%
15:37:49 : 04/04 6.3%
15:38:14 : 05/05 3.1%
15:38:46 : 06/06 1.6%
15:40:01 : 07/07 0.8%
15:40:18 : 08/08 0.4%
15:41:24 : 09/09 0.2%
15:42:28 : 10/10 0.1%
15:42:57 : 11/11 0.0%
15:43:23 : 12/12 0.0%
15:43:28 : Test finished.

----------
Total: 12/12 (0.0%)

The artefact is still here, still very pronounced. Was easyly ABXable from the first try.
I didn't separatly re-ABX the original sample with a problem by Mardel vs. original as I think it is useless. (See the old log, it would be the same)

Instead I started ABXing the original sample with a problem by Mardel vs. Lossywav V1.01n -q 1.0, as I first wanted to do, in order to see if there was any progression/regression. I could ABX it too so easyly that I stopped after 4 try as I was 100% sure of myself.

foo_abx 1.3.3 report
foobar2000 v0.9.5.3
2008/05/26 15:46:19

File A: C:\Documents and Settings\Bureau\Test\Ginnungagap.lossyV1.01n-q1.wav
File B: C:\Documents and Settings\Bureau\Test\Ginnungagap.lossywithnoise.wav

15:46:19 : Test started.
15:46:41 : 01/01 50.0%
15:47:08 : 02/02 25.0%
15:47:40 : 03/03 12.5%
15:48:21 : 04/04 6.3%
15:48:26 : Test finished.

----------
Total: 4/4 (6.3%)

(The following conclusion is wrong see edit.)
The good news is that there is a progression, the bad news is that the artefact is still here & still severe.
I would describe the progression as less deep tearing, softer noise, shorter distortion duration.
The sound return to stability faster after the distortion which has less amplitude.
But this is still comparing rotten apples with rotten oranges, if you compare to the original, the artefact is still severe. I would say it's like a 0.5 quality increase in the old scale of V1.01f not much more than that.


Major Edit:
Bad news, I just realized that I tested -q 0 vs -q 1 in the 2nd ABX test (1st test is valid even if I tested a better setting than I should have) so the improvement I heard must be 100% normal. I must re-test mad.gif ... but even if the test is bad, it now almost means for certain that there is near to zero improvement because it can only be worst for V1.01n at -q 0 than at -q 1 crying.gif

Edit2: I re-did the test with the correct parameters, it was un-ABXable sad.gif , I cannot tell any progression/regression at all. I fear what you changed between V1.01f & V1.01n has no effect on this specific sample. Sorry ...
halb27
lvqcl got good results with Mardel's problem sample using a significantly higher bits-to-keep value of -m 5.333.
-m 5.333 is bit of a hard bitrate penalty for the very low bitrate settings, but maybe a value of -m 4 brings a significant progress (we don''t need full transparency with -q 0 or -q 1, non-annoyance should be enough).

sauvage78, do you mind trying lossyWAV with the additional option of -m 4?
sauvage78
ok I will try it to see if it helps.

Edit:

V1.01n -q0 -m 4 Vs. original

foo_abx 1.3.3 report
foobar2000 v0.9.5.3
2008/05/26 17:35:50

File A: C:\Documents and Settings\Bureau\Test\Ginnungagap.lossyV1.01n-q0-m4.wav
File B: C:\Documents and Settings\Bureau\Test\Ginnungagap.original.wav

17:35:50 : Test started.
17:36:13 : 01/01 50.0%
17:36:30 : 02/02 25.0%
17:36:42 : 03/03 12.5%
17:37:03 : 04/04 6.3%
17:37:20 : 05/05 3.1%
17:37:34 : 06/06 1.6%
17:38:05 : 07/07 0.8%
17:38:20 : 08/08 0.4%
17:38:25 : Test finished.

----------
Total: 8/8 (0.4%)

Conclusion 1:
Easy to ABX from original, always the same flaws more or less pronounced.

V1.01n -q0 -m 4 Vs. -q0

foo_abx 1.3.3 report
foobar2000 v0.9.5.3
2008/05/26 17:39:29

File A: C:\Documents and Settings\Bureau\Test\Ginnungagap.lossyV1.01n-q0.wav
File B: C:\Documents and Settings\Bureau\Test\Ginnungagap.lossyV1.01n-q0-m4.wav

17:39:29 : Test started.
17:41:02 : 01/01 50.0%
17:42:16 : 02/02 25.0%
17:43:38 : 03/03 12.5%
17:44:32 : 04/04 6.3%
17:44:43 : Test finished.

----------
Total: 4/4 (6.3%)

Conclusion 2:
There is a minor improvement but if I recall well the -q1 setting which I tested by mistake in my previous post sounded almost same or better (but not worst 100% sure), so if there is a big bitrate increase I am not sure it worth it.

for efficiency I dunno, so far I only tested lossywav as pure wav so I have no clue what ends in a bigger file, -q 0 -m 4 or -q 1 ... it be must tested further with & without -m 4 at comparable bitrate ... for non-annoyance as far as I tested use -q 2.5 on this killer sample.
lvqcl
I still think the main problem for that sample is that added noise changes between -36 and -42dBFS several times in a second. (at 1.1...1.7 s.)

Maybe program should look ahead and stores results of analysis of several frames after and before current? Then, some sort of filter (median?) can be introduced. So the problem of 'unstable' noise can be fixed.
GeSomeone
QUOTE(Nick.C @ May 25 2008, 14:25) *
I re-implemented the old max-inter-block-change in bits to remove (bits-to-remove can only increase by 1 bit per codec block). This wastes a few bits but should limit the change in noise level to 6dB every 11.6ms (for 44.1kHz audio). I would appreciate feedback to see if this has improved Mardel's sample at all at -q 0.

Of course it is OK to test this. I just remember last time this came up that 2Bdecided said something to the effect that gradualy changing the bits to removed wouldn't help, it even might have a chance of giving some kind of modulation effect, and inaudible noise should be inaudible anyway (and if it's not when it should, tune somewhere). O.t.h. that was only valid for the transparent settings, so who knows unsure.gif
halb27
QUOTE(sauvage78 @ May 26 2008, 17:21) *

...
V1.01n -q0 -m 4 Vs. original
...
Easy to ABX from original, always the same flaws more or less pronounced.

Thanks for testing.
I didn't expect it to bring transparency (the bitrate penalty for -m 4 isn't so large), but I hoped it would make things less annoying. No need for a detailed comparison -q 0 -m 4 vs. -q 1 IMO. There should have been some progress IMO with -m 4 to justify further investigations.

In case no new ideas come up it looks like we can't seriously improve in the bitrate range of current -q 0 or -q 1, and --portable is a good solution for the lowest preset.

QUOTE(Nick.C @ May 26 2008, 13:29) *

... From memory (correct me if I'm wrong, halb27) eig, triangle_2_1644ds, furious and badvilbel would be good (i.e. problematic) samples to try. ....

These are good samples to try, as is Bruhns, Living In The Future, and other tracks from my potential problem collection which can be downloaded from here for a while.
Nick.C
QUOTE(halb27 @ May 26 2008, 18:25) *
In case no new ideas come up it looks like we can't seriously improve in the bitrate range of current -q 0 or -q 1, and --portable is a good solution for the lowest preset.
I introduced the -H, --highskew <n> (0<=n<=36) parameter at beta 1.0.1n. This is analogous to the low-frequency skew which is built in to lossyWAV but acts above 3.45kHz rather than below. This in conjunction with -l, --limit may provide a solution for the Ginnungagap sample.
2Bdecided
QUOTE(GeSomeone @ May 26 2008, 17:31) *
Of course it is OK to test this. I just remember last time this came up that 2Bdecided said something to the effect that gradualy changing the bits to removed wouldn't help, it even might have a chance of giving some kind of modulation effect, and inaudible noise should be inaudible anyway (and if it's not when it should, tune somewhere). O.t.h. that was only valid for the transparent settings, so who knows unsure.gif
Yes.

The basic problem here is that lossyWAV's tuning is aiming for transparency. For argument's sake, let's imagine that in tuning, there will probably have been moments when things were well above transparency, and bits were being wasted, but no one really cared. The worst moments will be "just above" transparency.

If you force things into non-transparent territory, two things happen. Firstly, some moments will stay transparent because they have a high margin, while others will plunge into dramatically non transparent because they were on the edge already. Secondly, the various tweaks that are in there to gain transparency don't perform "linearly" in the range of non-transparent encoding; they are set up to keep the noise inaudible - they have no mechanism to make audible noise "least bad".


Please be aware that these are two separate mechanisms. If you can't hear something, you can't hear it. Simple as that. If you can hear something, figuring out how annoying it is is a whole different ball game.

So I guess (and it's only a guess) that the wildly fluctuating noise is exactly what lossyWAV should do to keep the noise inaudible. However, for settings where the noise does become audible, it'll be very annoying.

I emphasise this is all me guessing - I'm not trying to say "this is the way it is" because I don't know.

Cheers,
David.
Nick.C
QUOTE(2Bdecided @ May 27 2008, 13:32) *
Yes.

The basic problem here is that lossyWAV's tuning is aiming for transparency. For argument's sake, let's imagine that in tuning, there will probably have been moments when things were well above transparency, and bits were being wasted, but no one really cared. The worst moments will be "just above" transparency.

If you force things into non-transparent territory, two things happen. Firstly, some moments will stay transparent because they have a high margin, while others will plunge into dramatically non transparent because they were on the edge already. Secondly, the various tweaks that are in there to gain transparency don't perform "linearly" in the range of non-transparent encoding; they are set up to keep the noise inaudible - they have no mechanism to make audible noise "least bad".


Please be aware that these are two separate mechanisms. If you can't hear something, you can't hear it. Simple as that. If you can hear something, figuring out how annoying it is is a whole different ball game.

So I guess (and it's only a guess) that the wildly fluctuating noise is exactly what lossyWAV should do to keep the noise inaudible. However, for settings where the noise does become audible, it'll be very annoying.

I emphasise this is all me guessing - I'm not trying to say "this is the way it is" because I don't know.

Cheers,
David.
Thanks for the insight David. I realise that -q 0 is pushing the envelope when bits-to-remove is concerned - but at the same time there are several user out there who "complain" (wink.gif) when I increase the bitrate at -q 0 with some revised setting or other.

I suffered a bit of a setback as the USB stick I was using to transport lossyWAV about failed and I had to revert to the source from beta 1.0.1e. The changes were mostly cosmetic or parametric and so no real trouble. In doing so, I have not yet re-enabled the "bits-to-remove increase of one per codec-block" code.

In re-modifying the code to bring it up to what 1.0.1n was (from memory and the change log) I had another thought about the spreading-function.

The spreading function string was:
CODE
  Frequency_Limits                  : Array [0..spread_zones+1] of Double = (20,1378.125,3445.3125,8268.75,12403.125,16000);
  spreading_function_string         : string[precalc_analyses*(spread_zones+2)-1]='22222-22222-22223-12223-12234-12345';
I have increased the number of spread-zones to 8 from 5 by making the inter-boundary frequency change constant (didn't have to move any boundaries, just interleaved 3 more).
CODE
  Frequency_Limits                  : Array [0..spread_zones+1] of Double = (20,1378.125,3445.3125,5512.5,8268.75,10335.9375,12403.125,14470.3125,16000);
  spreading_function_string         : string[precalc_analyses*(spread_zones+2)-1]='22222222-22222222-22222223-12222223-12222224-11222234';
This allows finer control by halving the width of the 3 highest spread-zones to approximately 2kHz each (except the highest one).

Bitrates for my 55 problem sample set are as follows:
CODE
|-------------|------------|------------|------------|------------|------------|
| Version     | --insane   | --extreme  | --standard | --portable |    -q 0    |
|-------------|------------|------------|------------|------------|------------|
| 1.0.1n      | 645 kbit/s | 571 kbit/s | 491 kbit/s | 410 kbit/s | 312 kbit/s |
|-------------|------------|------------|------------|------------|------------|
| 1.0.1o      | 652 kbit/s | 580 kbit/s | 504 kbit/s | 421 kbit/s | 321 kbit/s |
|-------------|------------|------------|------------|------------|------------|
| 1.0.1o --LC | 670 kbit/s | 601 kbit/s | 527 kbit/s | 441 kbit/s | 336 kbit/s |
|-------------|------------|------------|------------|------------|------------|

lossyWAV beta 1.0.1o attached to post #1 in this thread.
halb27
QUOTE(Nick.C @ May 29 2008, 09:34) *

...
CODE

|-------------|------------|------------|------------|------------|------------|
| 1.0.1o --LC | 670 kbit/s | 601 kbit/s | 527 kbit/s | 441 kbit/s | 336 kbit/s |
|-------------|------------|------------|------------|------------|------------|

What's the --LC option, Nick?
Nick.C
QUOTE(halb27 @ May 29 2008, 08:48) *
QUOTE(Nick.C @ May 29 2008, 09:34) *
...
CODE
|-------------|------------|------------|------------|------------|------------|
| 1.0.1o --LC | 670 kbit/s | 601 kbit/s | 527 kbit/s | 441 kbit/s | 336 kbit/s |
|-------------|------------|------------|------------|------------|------------|
What's the --LC option, Nick?
--linkchannels (but it's a bit wide for the table).
GeSomeone
QUOTE(Nick.C @ May 29 2008, 09:34) *
I realise that -q 0 is pushing the envelope when bits-to-remove is concerned - but at the same time there are several user out there who "complain" when I increase the bitrate at -q 0 with some revised setting or other.

Yeah, we like to complain wink.gif. But seriously, I only would not like it if the bit rate at -q 5 would rise because something at -q 0 was fixed..

QUOTE
In re-modifying the code to bring it up to what 1.0.1n was (from memory and the change log) I had another thought about the spreading-function.
[..]
This allows finer control by halving the width of the 3 highest spread-zones to approximately 2kHz each (except the highest one).

Although I don't understand what you are trying to accomplish, in general it is normal to take wider bands for higher freqs. To the ear 1 octave is a doubling of the frequency ( 2kHz - 4 kHz is the same (tonal) difference as 10 kHz - 20 kHz).
But maybe this has no relevance to lossyWav whatsoever sweat.gif
halb27
I used 1.0.1o with my regular music test set.
For --portable bitrate is the same as with 1.0.1k (362 kbps), for --standard it's 449 kbps against 1.0.1k's 452 kbps, for --extreme bitrate comes down to 529 kbps (1.0.1.k: 534 kbps), and for --insane bitrate is 607 kbps (1.0.1k: 611 kbps).
So from --standard upwards, bitrate is a little bit lower than when using 1.0.1k.

Bitrate of -q 0 is 270 kbps (I didn't keep the 1.0.1k's bitrate).

I did a short listening test using -q 0 on Mardel's sample and to me it's fine. I'm not sensittive towards this problem however.
Nick.C
QUOTE(halb27 @ May 29 2008, 19:42) *
I used 1.0.1o with my regular music test set.
For --portable bitrate is the same as with 1.0.1k (362 kbps), for --standard it's 449 kbps against 1.0.1k's 452 kbps, for --extreme bitrate comes down to 529 kbps (1.0.1.k: 534 kbps), and for --insane bitrate is 607 kbps (1.0.1k: 611 kbps).
So from --standard upwards, bitrate is a little bit lower than when using 1.0.1k.

Bitrate of -q 0 is 270 kbps (I didn't keep the 1.0.1k's bitrate).

I did a short listening test using -q 0 on Mardel's sample and to me it's fine. I'm not sensittive towards this problem however.
Thanks for the listening time - it's always appreciated.

I've been working on simultaneous piped input and output within foobar2000. I have managed to convert a single file using this method! emot-toot.gif However if I try to create a single album image from a number of tracks it crashes.... I'll work on this further as there must be a simple reason why it's working for a single file and not working for multiple files.
Nick.C
QUOTE(GeSomeone @ May 29 2008, 16:28) *
QUOTE(Nick.C @ May 29 2008, 09:34) *
I realise that -q 0 is pushing the envelope when bits-to-remove is concerned - but at the same time there are several user out there who "complain" (wink.gif) when I increase the bitrate at -q 0 with some revised setting or other.
Yeah, we like to complain wink.gif. But seriously, I only would not like it if the bit rate at -q 5 would rise because something at -q 0 was fixed..
QUOTE
In re-modifying the code to bring it up to what 1.0.1n was (from memory and the change log) I had another thought about the spreading-function.
[..]
This allows finer control by halving the width of the 3 highest spread-zones to approximately 2kHz each (except the highest one).
Although I don't understand what you are trying to accomplish, in general it is normal to take wider bands for higher freqs. To the ear 1 octave is a doubling of the frequency ( 2kHz - 4 kHz is the same (tonal) difference as 10 kHz - 20 kHz).
But maybe this has no relevance to lossyWav whatsoever sweat.gif
The increase in bitrate at --standard is due (at least in part) to the increase from 3.25 minimum-bits-to-keep to 3.50, also slightly from the tightening of the spreading-function.

The increase in the number of spread-zones still allows consecutive zones to have the same number of bins averaged, but it also allows for the possibility of averaging different numbers of bins in each "half" of the old zone (for those that were split).
carpman
Nick, halb27
Small issue, but I thought you might be interested in this (re. foobar setup batch file in LossyWAV wiki).
I posted in the other thread, but this seems to be the place now for all things lossyWAV.

C.
Nick.C
QUOTE(carpman @ May 29 2008, 20:16) *
Nick, halb27
Small issue, but I thought you might be interested in this (re. foobar setup batch file in LossyWAV wiki).
I posted in the other thread, but this seems to be the place now for all things lossyWAV.

C.
Thanks for that - I saw it at the time, but my loss of code kind of focussed my attention elsewhere.... I'll amend the wiki.
halb27
I wondered why bitrate came down a bit with 1.0.1o as compared to 1.0.1k and played around with --shaping. The default shaping (--shaping q-value/10) has gone. With the corresponding --shaping bitrate goes up.
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2008 Invision Power Services, Inc.