Note: Though evolved via another thread I started, I post this one, since my original question and purpose is moot. Hope this is OK.

Hello Everyone,

I need some advice. My purpose is to long term archive about 2000 original Waves. They have all been edited to some extent. At minimum, every file has had 1.25 seconds of silence added to the beginning & adjusted to ~3 seconds of silence at the end, and the gain has been normalized to peak @ minus (-)0.20 dB. Many noise reduction operations were performed, mostly on older recordings. Many more "softening" of first notes or audible sounds, improved fades, and other results of sloppy studio production have been corrected. But the integrity of the original recordings has not been compromised in any way. No enhancements were done, that is to say no equalizations, filters, etc. were used to alter the audio content in any way. It's just that I have better reproductions of the originals due to the hundreds of hours I have spent correcting flaws.

Now, I am faced with the question of whether or not to "balance" the gain on these files before archiving them. FWIW, this collection includes lots of standards and classical, oldies, etc. but almost no hard rock or metal,etc.. As you can see, it's not like I can use the original CD's for master backups, they're there for an emergency or catastrophy, I suppose, but the original master recordings will be the files in question and are superior to the original CD's for the reasons mentioned above. From these masters mp3's, mpc's, oggs, etc. will be cranked out from here on. BTW, I use Monkey's for lossless because that was recommended by other members when I started and I have been very happy with it.

If I had it to do again I may not have gain normalized the files, I may have waited and lastly balanced the gains. But though gain analyses are helpful and save time I have a hard time seeing how you can eliminate listening from the process, if you want to really get all the levels right, so that every track is at the level you want it to be. Especially with so many different CD's. So, maybe there's no getting around the listening. Anyway, it's too late now, because I have 2000 files and all are at (peak) -0.2 dB. But since I've been working in a totally lossless environment, I am OK to this point.

So, assuming it's the thing to do, what is the best way to 'balance' the gain on these files? Or is there a good argument for just leaving them as they are? Are there other factors that may come into play I haven't mentioned? Does anyone have any other suggestions that might help this project?

Right now I'm leaning to 'balancing' the gain on these files so it's done once and for all, and there'll be no need to adjust the gains in the future. The mpc's, mp3's, etc. created in the future will not need gain adjustments this way. But I want to be sure to do it right, and I prefer to eliminate the redundancy (and expense) of making copies as they are now. But only if the gain adjustment can be applied in a way that doesn't compromise the audio integrity of the file.

From what some of you have told me, I should use WaveGain for this operation. But (1)after seeing the lossless disclaimer in the readme, though I may not have read it thoroughly, and (2) noticing things like 'medium noise shaping' I was scared off. Why? Because I can not afford to compromise the integrity of the recording in any way, since I am creating original masters. Regarding this question, DickD told me "It's only lossy in rounding the very last bit and applying dither to avoid audible harmonic distortions caused by truncation. This is what you want." If I understand that correctly, maybe WaveGain is the solution. If so, I would appreciate any tips you WaveGain users might have.

Thanks in advance for your input.

Given that you indicate that these files are for all practical purposes irreplaceable, I would leave the originals, if you can really call them that, without further change and apply the appropriate replaygain version to the files after they have been encoded to mpc, mp3, etc.

I am not suggesting that the use of WaveGain will materially damage them, but it is not a truly reversible process and if these are so precious, I wouldn't alter them further.

- Using Wavegain will affect your files in a lossy manner, the thing is that you won't notice the diff in quality
- If you use FLAC for compressing losslessly yu can add replaygain to the FLAC in a completely lossless manner.
- The author of ReplayGain, 2bdecidec, has expressed that is better to use the *gain of each particular format.
- You haven't done anything losslessly because every editing in a Wave file is lossy, even more on fixed point waves.
- If you care so much about the 'originals' don't edit them anymore, apply each format's replay gain implementation.

I fully understand your concern about your Wave files. I have the same sentiment when it comes to original files.

Since you eventually encode those Waves to MP3's, MPC's and oggs, and maybe use each format's ReplayGain (RG), the first thing i tested was, how big of a difference really is there between the RG implementations? After all, it's known that Frank Klemm made some improvements to the original RG code for the MPC implementation, and that MP3Gain is limited to 1.5 dB steps. The encoding process itself should slightly alter the volume levels, too. As a test file, i used the song "Eskobar - Someone New", a 3:31 minute pop song. Since Ogg Vorbis focuses more on lower bitrates, i used -q4 and -q6 for encoding. It isn't tuned much beyond -q6 anyway, so that seems realistic. For MP3 and MPC, i chose the ever-so-popular standard and insane switches.



Klemm's modification of the MPC ReplayGain code is evident: About 0.5 dB less reduction. The fluctuation between --quality 5 and --quality 7 is minimal. MP3Gain, restricted by it's 1.5 dB steps, is both times 0.25 dB off, without any modification to the original RG code, AFAIK. VorbisGain seems to battle a bit with the encoding error: Slightly overshooting with a bigger fluctuation.

That's where i wondered, how big is the error introduced by encoding? With the lossy codecs, the error should get bigger, the lower the quality is chosen. I encoded the WAV, decoded without any additional switches and applied WaveGain.



FLAC is lossless. MPC introduces no encoding error. LAME is steady, with a minimal error. Ogg Vorbis understandably fails a bit at -q4, but -q6 also can't live up to it's expectations. Still, the dB change is absolutely nothing to worry about.

Back to your original question. If you want to save some time, i think you can safely use WaveGain on your files. It will do the same thing that the decoders do with your ReplayGain'ed lossy files, plus noise shaping. This pushes the scaling error + dither (noise) to the upper frequency regions, where you don't hear it. Of course, this will be hard-coded into your Waves, and there's no turning back. But it won't be audible at all. For the least "cosmetic" effect on your Waves, use light noise shaping. It goes without saying that you don't use any 'extra gain'.
What speaks against this method in theory, apart of the file alteration, is the volume reduction, which might influence lossy encoding. But since the codecs have dynamically adjusting ATH curves, so to speak, this issue should be minor.

However, i personally would use each formats' own ReplayGain. Your Wave files stay as they are, and as i've written above, the inter-format gain differences are small.

there is (almost - the only exception would be portable mp3 players for which you can also use mp3gain) no reason to apply replaygain (in a hard wavegain fashion) pripor to encoding into any format imho (especialy if you are on windows platform and foobar user), it is nowadays quite easy to replaygain ANY type of files you might have with that player. (i guess foobar uses only one calculation method of rg for any type of files, if you think that slightly different calculation methods that comes with different formats are the problem (actualy they aren't)).

(p.s. from what i remember Mr. Klemm was talking about the 'overimportant' low bass range when calculating Rg and 'not important enough' mid bass range (in the original implementation) which was obviously fixed in mpc version..., so one could say that the original method is only non-compatible with itself )

You mean, by increasing the gain to -0.2 dB I incurred a loss of audio? Because I can't see how adding or subtacting silence at the front or back of a file causes any audio loss. Are you referring to inaudible quantization errors maybe, or something else?

Of course there is loss, what on earth was I thinking? It's just that my software allows me to run multiple operations in one pass, incurring just one loss instead of many, (not applicable in this case, however, since gain normal was the only operation I performed) and thus I sometimes forget about loss/noise factors when using it, since I have been using this method for months.

But I have experienced what I think you mean. On a high quality Wave file I have performed say, 6 or 8 independent operations (the old way) - equalizations, noise reductions, gain changes, etc., etc., etc. and compared that result with running them in one pass using my multifilter. Comparing the results, with the former method I could hear some noise that was introduced (quantization or otherwise) because of the audio loss incurred by the multiple operations. So, thanks for pointing that out.

Well, the concensus says the best method is to replay gain the lossy files created from the Waves. Makes sense to me, and I'm glad I felt uneasy about it and came here for advice. What a great forum this is, when you can get a tough, technical question answered by, well, experts. Not to mention the enormous amount of time the correct answer saves me. Thanks to everybody.

Posted by John33 @ Mar 31 2003 - 05:35 PM
I call them that only because the music itself wasn't altered in any way, save for the inaudible loss caused by one edit operation. I know from experience with thousands of retail CD's how rare it is to get anything close to the studio master, anyway. Thanks for the input.

BTW, I do rip and compare identical recordings with DiamondCut Live5, often 3 or 4 cuts of the same song from different CD's. And I have never found identical files, there is always some difference, however slight, and that was really only once (Mamas and Papas collections). That was the only time I had trouble deciding which file was the higher quality. And almost always, there is an audible difference. And I'm not talking about just the Peter Pan Productions of the world, poor quality compilations (I finally threw most of these away). I have found this to be true with all the major brands as well, though not to the same degree as the renegade marketers.

Posted by CiTay on Mar 31 2003 - 06:50 PM

Thanks, and appreciate your test results.

Posted by /\/ephaestous on Mar 31 2003 - 05:45 PM

Good enough for me.

One more point on loss of volume: in mp3 format, at least using Lame alt-presets, more volume is lost the lower the bitrate of encoding. I've tested as CiTay has, and starting with wavegained tracks with a volume level of 89 dB, 128 kbps (alt-preset cbr 128) mp3 encodes come in at 88.4 or 88.5 on mp3gain; 160 kbps at 88.6 or 88.7, and alt-preset standard closer to 89.

IveyLeaguer, I think I understand what you want to do better than I did in the original thread.

You seem to be unsure about the amount of gain to apply and would like to listen to the recordings and might want to adjust the gain later after listening to the files after automatic ReplayGain is applied.

Having normalised, you've lost or altered the intentional differences between tracks that may be there for emotional effect, so Album Gain isn't useful in your case.

I'd agree that applying ReplayGain to the lossy files individually is the safest method, but you might want to modify the values to create intentional loudness differences for emotional effect or because a solo flute track ought to sound quieter than a full blown orchestra.

Applying dither once (when normalising) adds a little noise to the dither that was already applied during CD mastering. (Of course, if it was a noisy vintage recording, this dither is tiny compared to background noise, even after you've done noise reduction on it).

You'll add some more dither noise when scaling the loudness to 89 dB using ReplayGain. However, if you later decide to make the track a little quieter, or a little louder because you believe the track ought to be quieter or louder than most for emotional effect, you'll have applied a third lot of dither (fourth if you include the original CD mastering dither) and will typically have about 4 times the dither power (6 dB more noise) and twice the rms amplitude of dither (but 4 times the peak amplitude because the probability distribution function is more gaussian, and less triangular).

To avoid changing your archived files (.APE format, I believe) any more times than necessary, applying dither each time, I'd suggest you leave them intact to begin with at least. If you use foobar2000 player you can scan them all for ReplayGain and add the automatically calculated gain data to the APEv2 tag. (Right click in fb2k to find the menus)

When playing back with Preferences/Playback/TrackGain turned on, foobar2000 will apply the gain on the fly at 32-bit floating point resolution then dither back to 16-bits for output to your soundcard. It doesn't permanently change the file. You can then audition all your tracks and see if the suggested TrackGain sounds appropriate to you.

If the suggested gain is too much or too little, you can right-click the file on the playlist and use Edit ReplayGain info (advanced) to make modifications (usually 1.5 dB steps are around the smallest audible differences) and re-audition the volume of the track relative to other tracks. If you want to revert to the original suggestion, you can rescan the file for TrackGain.

Once you're completely happy with the changes you want, you can either:

• Leave them unchanged, with just the tag to tell fb2k what gain you want. If you want to encode to .MP3 .MPC etc, you can decode from APE while applying Track Gain using fb2k's 'Convert' function to write it to disk (e.g. WAV (PCM 16 bit dithered), use replaygain) before encoding.
The alternative is decoding in Monkey's Audio, but this won't carry across the gain change to the WAV, so it won't be carried into the .MP3 or .MPC. You could manually copy across the .MPC's replaygain values from the original .APE (where you want to override the automatic value) or manually apply the right Wavgain. mp3gain will only work in 1.5 dB steps, but can also apply manual gain directly to the file if desired.

• If you want to crystallise the intended volume you've chosen into the files by making a permanent volume change to the audio, you could use foobar2000's diskwriter/Convert (with ReplayGain and an appropriate dithered output format turned on, but with DSP turned off) or apply the gain listed in the tags using Wavgain before converting back to Monkey's Audio .APE format. This way you ensure that you only add one more dither operation to the files by avoiding any further permanent automated changes of gain that you might want to correct later.

One more thing - to be sure of avoiding audible impairment of your original files, you must avoid clipping.

As most of your music will need negative gain, it won't be clipped. Music with positive gain over 0.2 dB, will exceed full scale and will clip when ReplayGain is applied (I know this because you've already said that you normalised to -0.2 dB relative to full scale, which defines the peak amplitude in all your tracks).

For tracks where clipping would occur (e.g. required Track Gain is +2.3 dB but peak is -0.2 dB) you can:

• Apply no gain (or +0.2 dB at most, which will sound no different to no gain to the human ear) and accept that the track will be very slightly quieter than intended but with excellent sound quality. (This is the safest way, and in fb2k, it's what track-based clipping prevention does)

• Reduce the intended volume of all your music so the one with the highest gain doesn't clip (easiest is to apply a Pre-Amp in the DSP, e.g. -6.00 dB gives an 83 dB sound pressure level instead of the usual 89 dB).

• Listen to the music where it is clipped and try to determine if the clipping causes an audible difference (e.g. use a WAV editor to copy the short section of audio, apply the excessive gain, causing clipping, then apply the inverse gain, and compare it to the unclipped version using a blind testing tool such as WinABX). If you're sure the difference is inaudible, you could let it clip.

• Use a limiter to deliberately distort samples that would be clipped in a softer fashion, making the sound nicer to the ear. E.g. Foobar2000's Advanced Limiter which looks ahead (hence 'advanced') and applies dynamic compression only when the output samples have exceeded full scale, leaving the rest of the audio unaffected even if it almost reaches full scale. This does add harmonic distortion to the clipped peaks, but it's more musical and less harsh than simple clipping and doesn't distort non-clipping peaks, unlike a Hard Limiter or Soft Limiter.

By the way, search the boards for suggestions for archiving precious lossless compressed files you don't want to lose. Some people say .FLAC is more resilient than .APE to losing data (the whole file won't be lost by one corrupted bit - only a short section) and recommend using .MD5 files and similar to store error correction information and assist full recovery of corrupted files. Choosing the best CD-R media for long-term storage or making a copy on a second hard disk might also help.

Timcupery: I think the CBR presets in LAME use --scale to reduce the likelihood of clipping. The lower the bitrate, the greater distortion is likely, the more lowpass filtering occurs, and the more likely a peak is to differ greatly from the peak value on the CD. That's why scale is reduced slightly.

However, for transparent encodings (lame --alt-preset standard, musepack standard with --xlevel, vorbis -q6 etc.) the sound will be virtually identical to the original, so the ReplayGain calculated by foobar2000 should be nearly identical (which matches my experience). The table in a previous message used slightly different ReplayGain implementation (for different formats), hence the differences of much less than 1 dB. Such differences will be almost impossible to perceive with the human ear & brain.

Ivyleaguer: Exact Audio Copy (EAC) with correctly calibrated offsets and secure mode (or Test & Copy with matching CRCs) is widely recommended here for accurate ripping of CDs, which suits your objective for precious recordings and may avoid glitches or read damaged CDs, and will virtually ensure you get bit-accurate rips.

However, you're right that most different releases of the same track aren't bit identical even then, because each compilation is usually remastered or at least dithered down to 16-bit again from the originals. Some may even have been noise-reduced, remixed (e.g. stereo soundstage of multitrack recordings), had stereo ambience effects applied or been dynamically compressed (to make them LOUD at the expense of destroying their full dynamic range).

Boy, outstanding stuff, DickD.
A couple of questions.

Posted by DickD on Apr 1 2003 - 07:37 AM


Ideally, would a -0.3 dB normalisation be a better value, assuming you want to normalize?


Indeed. I plan to do just that.


I try hard to avoid it, but any degree of this noise is easily removed by my software (Waves & Lossless, of course). So, if it's really necessary ......


Good advice. I nearly overlooked this. Though it's time consuming, one capability I have to combat this is the ability to perform multiple operations simultaneously. For example, I can gain normalise and remove all dither noise in one pass of the filter, which gives me a file with (what amounts to) one application of dither, quantization noise, etc. Not an ideal way to go, granted. But I did remove noise from the worst of the recordings in question, mostly oldies and standards. This capability is the reason I nearly overlooked it (until pointed out by /\/ephaestous).


I will, what is your opinion?


Yes, I got off to a pretty good start some months ago, thanks to the people here. EAC, Lame, Monkeys, Razorlame, etc.

Thanks a lot, DickD. I really appreciate your input.

No, I think you missed my point. In fact I don't see any big reason to normalise at all (esp if you work at more than 16-bit when editing then convert back to 16-bit dithered as the final stage). If your original CD was at or very near full-scale you might actually prefer to normalise to a lower volume before editing to give a little headroom if you plan to do anything that might increase sample values and cause clipping (such as digital noise reduction (perhaps) or frequency filtering, crossfeeding or time-stretching)

All I was saying is that, since you happened to have normalised every track to -0.2 dB, I know for a fact that your highest peaks are 0.2 dB below full scale, so I know for a fact that any TrackGain value of more than 0.2 dB will exceed full scale (causing clipping) once the gain is applied. (I used to normalise radio recordings to 100%, 0.0 dB, many years ago before editing, when there was no ReplayGain)

Very few tracks are so dynamic that they'll clip at the 89 dB loudness target of ReplayGain (but I have one or two, mostly classical such as Rachmaninov that do clip - just - as Lame APS MP3s in Album Gain, so I set the mp3gain Album Gain nearer to 87.5 dB instead, just to be safe). In fact, most pop, rock and jazz recordings will probably be reduced in volume from your normalised settings or changed only slightly when you adjust them to 89 dB loudness using the ReplayGain method. You simply need to keep an eye out for any that have a positive gain applied (e.g. in foobar2000, you can examine the tags) and make a decision about whether to clip or not to make them louder.



You mean applying Waves Digital Noise Reduction? Hmm, this could be worse! Digital Noise Reduction should be use sparingly because it cannot know for sure what is noise and what it signal (it just has pretty good statistics about it from a sample of pure noise). A few applications of dither when editing probably still aren't audible, so there's little point risking loss of the desired signal by overusing Digital NR algorithms.

I made some recordings years ago from bad FM radio reception, which just sound really bad to me now. I guess the noise was excessive, but I ended up with artificial bleeps and smushed sybillants (ss, sh, th, ch) in the vocals. Partly this was probably CoolEdit 96's Noise Reduction not being as good as the likes of EAC's, but partly it was over agressive application. I'd rather have more noise (which I can mentally hear through) than un-natural artificially-processed sounding music. Some of my later efforts were much better and still sound great.

I understand that Waves NR is very good at preserving signal quality, however.



The same point applies about overuse of Digital NR. If doing heavy editing it may be safest to work in, say 24-bit, if possible, then dither back to 16-bit once you're finished. I don't have a 24-bit editing program, so I avoid too many steps to prevent excessive noise building up (my software works in 24-bit while processing but delivers the result in 16-bit dithered after each operation). Usually my analogue sources start with noise well above the last bit, so that's not much of a problem for me.



I don't backup onto CD-Rs, and don't have too much precious original audio to back up anyhow, but I'd be tempted by FLAC and MD5 files for error recovery on Taiyo-Yuden CD-R media. APE and MD5 might be good too, but my opinion isn't backed up by personal experience.

I actually have some stuff on backup tapes, CD-RW and a hard disk. One of the CD-RW's got corrupted, but I had spare sources of the same files. It might have served me well to periodically copy my CD-RWs to refresh the information in the dye layer every couple of years before uncorrectable errors occurred. For the MD5 stuff, I'd do some testing to see that I could recover corrupted files OK, and I'd put all the relevant software on each CD-R I made so I could decode and error-correct the files OK in future.

Anyhow, don't get too hung up about dither noise and so on, but avoid making too many unretrievable changes that you may later regret. If you keep a copy of the files as you have them now (even temporarily while you do the work), you can at least go back if you change your mind about which method produces acceptable sound quality.

For example, if you have a bunch of .APEs, you could create copies in a different folder or a different disk, if you have space, before you go about editing them.

Best of luck,

DickD

MD5 won't help with error correction, only error detection. The FLAC decoder can recover from errors but can't fix them (no audio codec I know of can do that); bad frames will be muted. Some formats cannot even recover from errors; one bad bit trashes the rest of the stream.

If you want to add error correction you need another layer like parity or reed-solomon on top; there have been topics here about that.

Josh

Thanks, Josh.