Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: PinkNoisePlusSine_32kHz (Read 7320 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

PinkNoisePlusSine_32kHz

32 kHz sample, a mix of pink noise with amplitude 0.02 and 4000 Hz sine with amplitude 0.98. It is ABX-able even at "extreme" preset of LossyWAV. With growth of the frequency LossyWAV becomes more effecient, and lower quality preset can be used.



LossyWAV "extreme" preset:

foo_abx 1.3.4 report
foobar2000 v1.2.3
2013/03/04 02:14:38

File A: S:\320 GB HDD\Аудио-тесты\NEW\My Tests\PinkNoisePlusSine_32kHz\PinkNoisePlusSine_32kHz.flac
File B: S:\320 GB HDD\Аудио-тесты\NEW\My Tests\PinkNoisePlusSine_32kHz\PinkNoisePlusSine_32kHz_exm.flac

02:14:38 : Test started.
02:15:34 : 01/01  50.0%
02:16:00 : 02/02  25.0%
02:16:30 : 03/03  12.5%
02:18:22 : 04/04  6.3%
02:19:12 : 05/05  3.1%
02:19:43 : 06/06  1.6%
02:28:33 : 07/07  0.8%
02:29:40 : 08/08  0.4%
02:40:29 : 09/09  0.2%
02:43:57 : 10/10  0.1%
02:43:59 : Test finished.

----------
Total: 10/10 (0.1%)

More files

PinkNoisePlusSine_32kHz

Reply #1
32 kHz sample, a mix of pink noise with amplitude 0.02 and 4000 Hz sine with amplitude 0.98. It is ABX-able even at "extreme" preset of LossyWAV. With growth of the frequency LossyWAV becomes more effecient, and lower quality preset can be used.

Warning to anyone - it's really loud and piercing (I'm glad I was running foobar2000's EBU R128 normalizer on un-replaygained files when I first tried it) so turn down your volume first

Firstly, all the files in your ZIP were at 44.1kHz sampling rate - there's nothing 32kHz about them. The spectrum extends all the way to 22.05 kHz Nyquist limit.

I tried a simple lossyWAV 1.3.0 'xtraportable' setting

lossyWAV.exe PinkNoisePlusSine.wav -q X

and it added a lot of roughness to the otherwise unobtrusive pink noise, which might well be clipping-related.

It was trivial to ABX 10/10 (below codebox) - fb2k DSP disabled.

No time to try higher settings now. It would be good for me to confirm that 'standard' is problematic and diagnose what switches might remove the roughness and try various modes because standard should probably remain transparent to all samples no matter how extreme in volume or artificial, and (subject to listening confirmation) it might be that additions in v1.3.0 or v1.2.0 have broken that (e.g. by relaxing the allowed clipping).

It might then be possible to modify standard or produce a 'safe standard' that's resistant to artificial near-full-scale samples, if it's clipping that's the problem.

Code: [Select]
foo_abx 1.3.4 report
foobar2000 v1.2.2
2013/03/10 07:16:49

File A: C:\Users\Public\Music\Test\lossyWAV\PinkNoisePlusSine.flac
File B: C:\Users\Public\Music\Test\lossyWAV\PinkNoisePlusSine.lossy.wav

07:16:49 : Test started.
07:17:54 : 01/01  50.0%
07:18:05 : 02/02  25.0%
07:18:17 : 03/03  12.5%
07:18:24 : 04/04  6.3%
07:18:45 : 05/05  3.1%
07:18:55 : 06/06  1.6%
07:19:13 : 07/07  0.8%
07:19:30 : 08/08  0.4%
07:19:36 : 09/09  0.2%
07:19:54 : 10/10  0.1%
07:19:58 : Test finished.

 ----------
Total: 10/10 (0.1%)
Dynamic – the artist formerly known as DickD

PinkNoisePlusSine_32kHz

Reply #2
I take the point, however the RG128 values for the two files are about -21.1dB - it's hardly likely to be encountered when processing music.

When fixed noise shaping is used for these samples the roughness attenuates somewhat.

I appreciate you uploading the samples - I will work to determine *why* this happens and seek to improve it.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

PinkNoisePlusSine_32kHz

Reply #3
My earlier test, used lossyWAV v1.3.0, this was the commandline and stats:
Code: [Select]
C:\Users\Public\Music\Test\lossyWAV>lossyWAV.exe PinkNoisePlusSine.wav -q X
lossyWAV 1.3.0, Copyright (C) 2007-2011 Nick Currie. Copyleft.
This is free software under the GNU GPLv3+ license; There is NO WARRANTY, to
the extent permitted by law. <http://www.gnu.org/licenses/> for details.
Filename  : PinkNoisePlusSine.wav
File Info : 44.10kHz; 1 channel; 16 bit; 00:10.00, 0.841MiB
Results   : 7.7320 bits; 12.39x; 00:00.80; [F]
Clipping  : 0.0081 bits lost, 242 clips remaining.

Having suspected clipping, and knowing that 3 clips per block are allowed at Xtraportable, and either 1 or 0 clips per block at extreme (it's 1 clip at -q 7.0, 0 clips at -q 8.0, not sure about -q 7.5, but I suspect it allows 1 clip per block), I tried using the following (this time I happened to use 1.3.0k but without it's special new mode, so the same as 1.3.0 release):

Code: [Select]
C:\Users\Public\Music\Test\lossyWAV>lossyWAV.exe PinkNoisePlusSine.wav -q X --ma
xclips 1
lossyWAV 1.3.0k, Copyright (C) 2007-2013 Nick Currie. Copyleft.
This is free software under the GNU GPLv3+ license; There is NO WARRANTY, to
the extent permitted by law. <http://www.gnu.org/licenses/> for details.
Filename  : PinkNoisePlusSine.wav
File Info : 44.10kHz; 1 channel; 16 bit; 00:10.00, 861.3kiB
Results   : 7.6705 bits; 16.12x; 00:00.61; [F]
Clipping  : 136 clips remaining.

At first I couldn't tell a night and day difference like before, though oddly guessed exactly wrong 5 times in a row. Then I could just about pick up a slight roughness in the pink noise on the lossyWAV version so I reset the ABX and I was then at p=0.055 (5.5% probability I was guessing in my chosen 10 trials). ABX log in CODEBOX below (again conducted with no ReplayGain, no DSP and my soundcard's analogue volume control. I don't have time to ABX with maxclips=0 or a higher quality setting today - I only did this because it's a quick check of my suspicion, but would like to see where maxclips needs to be to ensure that -standard is transparent with 'whatever you can throw at it', even insanely loud full-scale sinusoid in noise test signals like this. I can well imagine that a simple gain adjustment like --scale 0.95 or 0.98 (number is a guess) could render it transparent by removing the signal-dependency of the clipping on positive peaks.

The allowed clipping works rather well for real music where strings of even 2 or 3 consecutive samples at positive full scale aren't normally audible. It seems a shame to waste bits making standard work for test tones, unless we provide a -testtones switch that limits clipping more than usual. (Edit - actually, it might not be an awful lot of bits if clipping remains sporadic. Personally, I tend to encode most of my music with Album Gain applied anyhow as if it were mastered at a sensible level, so I wouldn't often get anywhere near full scale.)

Code: [Select]
foo_abx 1.3.4 report
foobar2000 v1.2.2
2013/03/10 17:19:12

File A: C:\Users\Public\Music\Test\lossyWAV\PinkNoisePlusSine.flac
File B: C:\Users\Public\Music\Test\lossyWAV\PinkNoisePlusSine_lw130k_Xtraportable_maxclips_1.wav

17:19:12 : Test started.
17:20:09 : 00/01  100.0%
17:20:27 : 00/02  100.0%
17:21:10 : 00/03  100.0%
17:21:27 : 00/04  100.0%
17:21:43 : 00/05  100.0%
17:22:41 : 01/06  98.4%
17:22:58 : 01/07  99.2%
17:23:09 : 02/08  96.5%
17:23:24 : 02/09  98.0%
17:23:49 : 03/10  94.5%
17:24:26 : Trial reset.
17:25:32 : 01/01  50.0%
17:26:04 : 02/02  25.0%
17:26:35 : 02/03  50.0%
17:27:30 : 03/04  31.3%
17:27:58 : 03/05  50.0%
17:28:08 : 04/06  34.4%
17:28:18 : 05/07  22.7%
17:28:58 : 06/08  14.5%
17:29:21 : 07/09  9.0%
17:29:43 : 08/10  5.5%
17:29:57 : Test finished.

 ----------
Total: 11/20 (41.2%)
Dynamic – the artist formerly known as DickD

PinkNoisePlusSine_32kHz

Reply #4
Firstly, all the files in your ZIP were at 44.1kHz sampling rate - there's nothing 32kHz about them. The spectrum extends all the way to 22.05 kHz Nyquist limit.
I tried a simple lossyWAV 1.3.0 'xtraportable' setting

You tried wrong file. 44 kHz is a different sample. It is some bug of the forum: I attached only  PinkNoisePlusSine_32kHz.flac file, zip archive with 44 kHz version somehow was moved here from this topic automatically. Zip archive with 32 kHz sample and other correspondig files can be found at link More files.
Quote
It might then be possible to modify standard or produce a 'safe standard' that's resistant to artificial near-full-scale samples, if it's clipping that's the problem.

It is not clipping. I tried "--maxclips 0" - no changes, still ABX-able on "extreme".

I take the point, however the RG128 values for the two files are about -21.1dB - it's hardly likely to be encountered when processing music.

Volume does not mean much here. The main rule is that some quiet noise should present there (pink noise is the best, but white noise is possible) and some simple signal, louder about >5 times than the noise. I also tried pink noise 0.02 and sine 0.2 (track gain = -7.33 dB), and the same, ABX-able on "extreme". Have to say I listened it at such a high volume, at which I would never listen real music (it was not very painfull for the ears).
Quote
When fixed noise shaping is used for these samples the roughness attenuates somewhat.

Yes, because I don't like the idea of "--shaping 1"', I tried "--shaping 0" and couldn't ABX it on "standard".
Quote
I appreciate you uploading the samples - I will work to determine *why* this happens and seek to improve it.

Quick look on spectrogram clarifies it:

(On the left is the original, on the right is lossyWAV "extraportable")
ANS (adaptive noise shaping) tries to mask too much noise around the sine, which is very thin (only one 4000 Hz frequency), so there is no enough room for this noise, and it becomes easily audible. I think it wouldn't happen, if lossyWAV could calculate, how much noise can be inaudibly processed by ANS (i.e. some combination of how much frequencies are there in the original signal and how loud they are). For example, if there are many musical instruments there, much more noise can be masked effectively.
Nick.C, have you ever thought about using various shaping technics at the same time? For example, some portion of noise to process by ANS and other leave as it is or use "shaping 1"? Is it possible? I don't know exactly, how this things works, so it is just a hypothesis.

Dynamic, once again, 44 kHz sample is from another topic. And the whole problem is not about clipping or volume of the sine wave (and the whole sample). lossyWAV adds some noise, which is audible on loud enough volume of listening (but not painfull).

PinkNoisePlusSine_32kHz

Reply #5
Thank you for the spectrogram softrunner - very helpful.

EDIT: Image has gone!!!!

I assume that, as well as sounding fine, it looks fine with shaping disabled too?

Cheers,
David.

PinkNoisePlusSine_32kHz

Reply #6
It is not clipping. I tried "--maxclips 0" - no changes, still ABX-able on "extreme".
It is certainly clipping.

Volume does not mean much here. The main rule is that some quiet noise should present there (pink noise is the best, but white noise is possible) and some simple signal, louder about >5 times than the noise. I also tried pink noise 0.02 and sine 0.2 (track gain = -7.33 dB), and the same, ABX-able on "extreme". Have to say I listened it at such a high volume, at which I would never listen real music (it was not very painfull for the ears).
Volume is important as the higher the number of (at or near) maximal samples in the lossless version, the more clipping on rounding will occur. Also, the adaptive noise shaping method needs some amplitude to work with - if it is forced to always go near zero additive or subtractive for already large amplitude samples then this affects the effectiveness of the method.

Yes, because I don't like the idea of "--shaping 1"', I tried "--shaping 0" and couldn't ABX it on "standard".
Why do you not like the idea of full fixed noise shaping? --shaping 0 effectively switches all noise shaping off, as does --adaptive off.

Quick look on spectrogram clarifies it:
Yes, but just because you can see a difference does not mean that you can hear it, unless as you have already admitted you push the volume levels to marginally uncomfortable levels.

Try using "--scale 0.95 --maxclips 0" for processing.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

PinkNoisePlusSine_32kHz

Reply #7
I assume that, as well as sounding fine, it looks fine with shaping disabled too?

With "--shaping 0" it looks like for other samples: some noise is added at the whole bandwidth, without any concentration at some frequencies. Actually, it looks close to WavPack hybrid spectrograms.

It is certainly clipping.

How it can be a clipping?, if lossyWAV says itself:

s:\320 GB HDD\?????-?????\NEW\My Tests\PinkNoisePlusSine_32kHz>lossywav pinknoiseplussine_32khz.wav --quality 7.5
lossyWAV 1.3.0j, Copyright © 2007-2013 Nick Currie. Copyleft.
This is free software under the GNU GPLv3+ license; There is NO WARRANTY, to
the extent permitted by law. <http://www.gnu.org/licenses/> for details.
Filename  : pinknoiseplussine_32khz.wav
File Info : 32.00kHz; 1 channel; 16 bit; 00:10.00, 625.0kiB
Results  : 4.3544 bits; 22.52x; 00:00.44;
Clipping  : 0 clips remaining.

s:\320 GB HDD\?????-?????\NEW\My Tests\PinkNoisePlusSine_32kHz>lossywav pinknoiseplussine_32khz.wav --quality -2.5 --maxclips 0
lossyWAV 1.3.0j, Copyright © 2007-2013 Nick Currie. Copyleft.
This is free software under the GNU GPLv3+ license; There is NO WARRANTY, to
the extent permitted by law. <http://www.gnu.org/licenses/> for details.
Filename  : pinknoiseplussine_32khz.wav
File Info : 32.00kHz; 1 channel; 16 bit; 00:10.00, 625.0kiB
Results  : 6.8368 bits; 21.47x; 00:00.46;
Clipping  : 0 clips remaining.

Quote
Why do you not like the idea of full fixed noise shaping? --shaping 0 effectively switches all noise shaping off, as does --adaptive off.

Because "--shaping 1" is less stable when EQ or some DPS plugins are used (I checked it on Sound Solution 1.31 DPS plugin, "--shaping 0" sounds clearly better, than "--shaping 1").

Quote
Quick look on spectrogram clarifies it:
Yes, but just because you can see a difference does not mean that you can hear it, unless as you have already admitted you push the volume levels to marginally uncomfortable levels.

That is the whole problem with ANS: there is no way to know, where it will be audible, and where not. It may be inaudible on "extraportable", where "shaping 0" and "shaping 1" are clearly audible, and it may be audible on "standard", where "shaping 0" and "shaping 1" are not audible. "shaping 0" and "shaping 1" are less dependant on material, that is encoded.

Quote
Try using "--scale 0.95 --maxclips 0" for processing.

I tried it: still no changes (I used lossyWAV 1.3.0j, have not tried latest versions so far):

Code: [Select]
foo_abx 1.3.4 report
foobar2000 v1.2.3
2013/03/30 10:52:58

File A: D:\@Portable@\Audio\=Scripts=\lossyWAV Transcode\pnps32.wav
File B: D:\@Portable@\Audio\=Scripts=\lossyWAV Transcode\pnps32 --scale 0.95 --maxclips 0 -q 7.5 (1).flac

10:52:58 : Test started.
10:53:21 : 01/01  50.0%
10:53:39 : 02/02  25.0%
10:56:46 : 03/03  12.5%
10:59:36 : 04/04  6.3%
11:19:30 : 04/05  18.8%
11:20:42 : 05/06  10.9%
11:20:57 : 06/07  6.3%
11:21:20 : 07/08  3.5%
11:22:19 : 08/09  2.0%
11:23:10 : 09/10  1.1%
11:24:03 : Test finished.

----------
Total: 9/10 (1.1%)

PinkNoisePlusSine_32kHz

Reply #8
It is certainly clipping.

How it can be a clipping?, if lossyWAV says itself:

s:\320 GB HDD\?????-?????\NEW\My Tests\PinkNoisePlusSine_32kHz>lossywav pinknoiseplussine_32khz.wav --quality 7.5
lossyWAV 1.3.0j, Copyright © 2007-2013 Nick Currie. Copyleft.
This is free software under the GNU GPLv3+ license; There is NO WARRANTY, to
the extent permitted by law. <http://www.gnu.org/licenses/> for details.
Filename  : pinknoiseplussine_32khz.wav
File Info : 32.00kHz; 1 channel; 16 bit; 00:10.00, 625.0kiB
Results  : 4.3544 bits; 22.52x; 00:00.44;
Clipping  : 0 clips remaining.

s:\320 GB HDD\?????-?????\NEW\My Tests\PinkNoisePlusSine_32kHz>lossywav pinknoiseplussine_32khz.wav --quality -2.5 --maxclips 0
lossyWAV 1.3.0j, Copyright © 2007-2013 Nick Currie. Copyleft.
This is free software under the GNU GPLv3+ license; There is NO WARRANTY, to
the extent permitted by law. <http://www.gnu.org/licenses/> for details.
Filename  : pinknoiseplussine_32khz.wav
File Info : 32.00kHz; 1 channel; 16 bit; 00:10.00, 625.0kiB
Results  : 6.8368 bits; 21.47x; 00:00.46;
Clipping  : 0 clips remaining.

Yes, and I think that you may have uncovered a bug in the text output code - there should be a statement of the number of bits lost for each of those processes, 0.0160 and 0.3144 respectively. This means that the adaptive noise shaping method has very little additive space to work at some points in the sample.

Quote
Why do you not like the idea of full fixed noise shaping? --shaping 0 effectively switches all noise shaping off, as does --adaptive off.

Because "--shaping 1" is less stable when EQ or some DPS plugins are used (I checked it on Sound Solution 1.31 DPS plugin, "--shaping 0" sounds clearly better, than "--shaping 1").

Understood.

Quote
Quick look on spectrogram clarifies it:
Yes, but just because you can see a difference does not mean that you can hear it, unless as you have already admitted you push the volume levels to marginally uncomfortable levels.

That is the whole problem with ANS: there is no way to know, where it will be audible, and where not. It may be inaudible on "extraportable", where "shaping 0" and "shaping 1" are clearly audible, and it may be audible on "standard", where "shaping 0" and "shaping 1" are not audible. "shaping 0" and "shaping 1" are less dependant on material, that is encoded.

Yes, but the adaptive noise shaping method tries to put the noise in the same place as the signal. Maybe some further refinement of the target spectral shape is required.

Thanks for taking the time to carry out ABX testing - I would appreciate it if you could try again using "--feedback" in conjunction with "--scale 0.95 --maxclips 0". I would be more interested in results for the 44.1kHz sample than the 32kHz sample.

One final observation - lossyWAV has been heavily developed using 44.1kHz samples. 32kHz compatibility is included because it is possible rather than because using lossyWAV on 32kHz audio is considered to be a good idea. I think that the simplest solution would be to disable noise shaping for samples where the nyquist frequency is within the hearing range of a significant percentage of the population as the adaptive noise shaping method relies on putting at least some of the noise above 16kHz - obviously impossible with a 32kHz sample-rate.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

PinkNoisePlusSine_32kHz

Reply #9
Thanks for taking the time to carry out ABX testing - I would appreciate it if you could try again using "--feedback" in conjunction with "--scale 0.95 --maxclips 0". I would be more interested in results for the 44.1kHz sample than the 32kHz sample.

One final observation - lossyWAV has been heavily developed using 44.1kHz samples. 32kHz compatibility is included because it is possible rather than because using lossyWAV on 32kHz audio is considered to be a good idea. I think that the simplest solution would be to disable noise shaping for samples where the nyquist frequency is within the hearing range of a significant percentage of the population as the adaptive noise shaping method relies on putting at least some of the noise above 16kHz - obviously impossible with a 32kHz sample-rate.

Ok, I've made quiet version of this sample at 44 kHz here, so, as I realize, "--scale 0.95 --maxclips 0" is no more needed.