R128GAIN: An EBU R128 compliant loudness scanner Options

[pbelkner]

I've just uploaded on sourceforge a first version of r128gain, an EBU R128 (http://tech.ebu.ch/loudness) compliant loudness scanner:

http://sourceforge.net/projects/r128gain/files/

• process single files (tracks):
  
  • In order to just scan and analyze one file (WAV, FLAC or WV):
    
    CODE
    r128gain <file>
  • In order to scan, analyze and tag one file (currently only FLAC, result will be in <directory>):
    
    CODE
    r128gain <file> <directory>
  • In order to scan, analyze one file (WAV, WV) and encode and tag into FLAC (result will be in <directory>):
    
    CODE
    r128gain <file> <directory> flac
• recursively process directories (the content of a directory is considered an album):
  
  • In order to just recursively scan and analyze a directory:
    
    CODE
    r128gain <directory>
  • In order to recursively scan, analyze and tag a directory:
    
    CODE
    r128gain <directory> <directory>
    r128gain <directory>/* <directory>
  • In order to recursively scan, analyze a directory and encode and tag into FLAC:
    
    CODE
    r128gain <directory> <directory> flac
    r128gain <directory>/* <directory> flac

Tagging is currently supported only for FLAC:

• REPLAYGAIN_REFERENCE_LOUDNESS: always -23 LUFS (corresponding to EBU R128)
• REPLAYGAIN_TRACK_GAIN: loudness difference to -23 LUFS in LU measured for this track
• REPLAYGAIN_TRACK_PEAK: true peak measured at 192 kHz for this track
• REPLAYGAIN_ALBUM_GAIN: loudness difference to -23 LUFS in LU measured for the whole album
• REPLAYGAIN_ALBUM_PEAK: true peak measured at 192 kHz for the whole album

These tags should be honored by each Replay Gain compliant media player.

[C.R.Helmrich]

Wow, thank you so much for this tool! I was already mentally preparing myself to have to write my own R128 scanner, but here it is! And in version 0.2, it already does all I need.

Having said that...

When I pass a file path with spaces and without "enclosing quotes", or a file which doesn't exist, it prints an assert and crashes.

Other than that, it seems to analyze as expected, but I have some questions after looking at the source code.

• For the loudness analysis, all input files are resampled to 48 kHz, right?
• For the peak finder, the files are additionally resampled to 192 kHz, right?
• The way I understand R128 is:
  Pass 1: analyze entire file/album, compute loudness taking into account the absolute gate of -70 LUFS.
  Pass 2: analyze entire file/album again, this time also taking into account the relative loudness gate derived from the result of pass 1. Is that how you implemented it?

Thanks again for your work!

Chris

[pbelkner]

Wow, thank you so much for this tool! I was already mentally preparing myself to have to write my own R128 scanner, but here it is! And in version 0.2, it already does all I need.

Thanks

When I pass a file path with spaces and without "enclosing quotes", or a file which doesn't exist, it prints an assert and crashes.

The scanner is heavily based on SoX (http://sox.sourceforge.net/). This is SoX philosophy: use assert() in production code, i.e. if a pre-condition isn't fulfilled just die.

• For the loudness analysis, all input files are resampled to 48 kHz, right?
• For the peak finder, the files are additionally resampled to 192 kHz, right?

48 kHz is due to BS 1770 because they define filter coefficients for that sample rate only. R-REC-BS.1770-1-200709-I!!PDF-E.pdf states:

These filter coefficients are for a sampling rate of 48 kHz. Implementations at other sampling rates will require different coefficient values, which should be chosen to provide the same frequency response that the specified filter provides at 48 kHz. The values of these coefficients may need to be quantized due to the internal precision of the available hardware. Tests have shown that the performance of the algorithm is not sensitive to small variations in these coefficients.

It is not obvious for me how to quantize the given coefficients with respect to other sample frequencies, hence I decided to re-sample to 48 kHz

On the other hand R-REC-BS.1770-1-200709-I!!PDF-E.pdf states regarding true peak determination:

1. Attenuate: 12.04 dB attenuation
2. 4 × over-sampling
3. Emphasis: Pre-emphasis shelving filter, zero at 14.1 kHz, pole at 20 kHz (optional)
4. DC block (optional)
5. Absolute: Absolute value
6. Max: Highest value detection (optional, included if DC block is included).

The current version of R128GAIN combines all this in order to avoid multiple passes:

1. Attenuate by 0.5 (i.e. ca. 6 dB, future versions will correct this).
2. Up-sample to 192 kHz (i.e. approximately 4 x over-sampling)
3. Determine "true" peak.
4. Down-sample to 48 kHz in order to match defined filter coefficients.
5. Apply R128 algorithm.

The way I understand R128 is:
Pass 1: analyze entire file/album, compute loudness taking into account the absolute gate of -70 LUFS.
Pass 2: analyze entire file/album again, this time also taking into account the relative loudness gate derived from the result of pass 1. Is that how you implemented it?

tech3341.pdf states:

1. using an absolute 'silence' gating threshold at -70 LUFS for the computation of the absolute-gated loudness level, and
2. using a relative gating threshold, 8 LU below the absolute-gated loudness level, and
3. the measurement input to which the gating threshold is applied is the loudness of the400 ms gating blocks measured using an ITU-R BS.1770 method without gating, that is, summed across channels;
4. a constant overlap between consecutive gating blocks of at least 50% is required (for increased precision especially when measuring programs of short duration).

R128GAIN interprets this as follows:

• the -70 LUFS is absolute, hence no second path is needed.
• the 8 LU is relative to the 400ms gating block. R128GAIN uses a running 400ms gating block, hence 50% overlap is guarantied and no second pass is needed.

[C.R.Helmrich]

It is not obvious for me how to quantize the given coefficients with respect to other sample frequencies, hence I decided to re-sample to 48 kHz

For 44.1 kHz, I resampled the 48-kHz filter impulse responses and tried to find new coefficients which match the given transfer function. Here is what I got:

RLB: numerator = [1 -2 1], denominator = [1 -1.9891 0.98913], accurate to ~0.02 dB above 50 Hz
Pre: numerator = [1.535 -2.633 1.151], denominator = [1 -1.647 0.701], accurate to ~0.05 dB, will tune it a bit more soon

R128GAIN interprets this as follows:

• the -70 LUFS is absolute, hence no second path is needed.
• the 8 LU is relative to the 400ms gating block. R128GAIN uses a running 400ms gating block, hence 50% overlap is guarantied and no second pass is needed.

Sorry, but I don't think that's correct. Look at Tech doc 3341, Annex 1. I think the 8 LU is relative to the entire file/album (summation in equation 7, the Jg are the set of 400-ms gating blocks).

Chris

This post has been edited by C.R.Helmrich: Jan 6 2011, 15:35

[pbelkner]

R128GAIN interprets this as follows:

• the -70 LUFS is absolute, hence no second path is needed.
• the 8 LU is relative to the 400ms gating block. R128GAIN uses a running 400ms gating block, hence 50% overlap is guarantied and no second pass is needed.

Sorry, but I don't think that's correct. Look at Tech doc 3341, Annex 1. I think the 8 LU is relative to the entire file/album (summation in equation 7).

What R128GAIN does is the following (in principle):

• Create an empty gating block capable of holding samples up to 400ms using a ring buffer.
• For each input sample:
  
  • If the gating block is full remove the first sample from it.
  • Add the current sample to the end of the gating block.
  • If the gating block is full:
    
    • Pick the sample cached in the middle of the gating block.
    • Depending on the (un-gated) loudness measure of the gating block decide, whether to add the picked sample to the overall statistics.

That's my understanding of Tech doc 3341, Annex 1, at least in principle.

Hopefully

EBU Tech Doc 3343 ‘Practical Guidelines for Production and Implementation in accordance with EBU R 128’

will clarify this, if published some day.

[googlebot]

What R128GAIN does is the following (in principle):

• Create an empty gating block capable of holding samples up to 400ms using a ring buffer.
• For each input sample:
  
  • If the gating block is full remove the first sample from it.
  • Add the current sample to the end of the gating block.
  • If the gating block is full:
    
    • Pick the sample cached in the middle of the gating block.
    • Depending on the (un-gated) loudness measure of the gating block decide, whether to add the picked sample to the overall statistics.

That's my understanding of Tech doc 3341, Annex 1, at least in principle.

I agree with C.R.H. that this interpretation doesn't seem to follow Tech 3341, Annex 1.

1. A measurement interval is first divided into blocks of 400ms length (with at least 50% overlap). Basically doable with a ring buffer.
2. From this set of blocks, eliminate all blocks with loudness below -70 LUFS. (I don't understand why you speak of adding samples (and not blocks) to the "overall statistic").
3. Calculate the total loudness for the latter, decimated set of blocks.
4. This loudness minus 8 LU is your final threshold.
5. Decimate the already decimated set of blocks again: Remove all blocks below the final threshold (which can only be known after a complete pass).
6. The "ungated" total loudness of the resulting set of blocks is your result.

I think it would be simpler to work with block indices (a list, array, or bitmap) than a ring buffer.* Read the input stream two times block by block and skip the calculated indices. Usually, with the buffering left to the OS, you should be reading the second pass from memory automatically.

PS The wording in Annex 1 could be better. Especially the "gated loudness" LKG in (6) and (8) should use different symbols. But mathematically it is not ambiguous. It took me over an hour to crunch the whole thing, though.

* Of course, you can still use something as a ring buffer for I/O. But I would put a block-wise abstraction layer on top of it to make the overall design simpler.

This post has been edited by googlebot: Jan 7 2011, 22:14

[pbelkner]

I agree with C.R.H. that this interpretation doesn't seem to follow Tech 3341, Annex 1.

Probably you're both are right, I have to think about it.

What I have in mind (probably not correct) is the following:

• BS.1770 defines a loudness measure to which all samples contribute the same way.
• R128 adds a gate: some samples have to be removed from the pure BS.1770 measure.
• In R128GAIN the gate is local to each sample, i.e. for each sample it consist of -200ms to +200ms samples relative to the sample under consideration. Depending on the pure (un-gated) BS.1770 loudness measure of the relative -200ms to +200ms block the sample under consideration is added to or removed from the (gated) BS.1770 loudness measure of the track/album.

The block relative to each sample is what I call the running gate because it's easy to update it's sum of squares at each step.

• The "ungated" total loudness of the resulting set of blocks is your result.

What do you mean by "loudness of a set of blocks"? Doesn't it imply to count samples more than once?

It seems to me that what I've implemented is the limit of what you get if you let go the overlap to 100%. If this is true than it would be fully compliant because they require 50% at a minimum.

This post has been edited by pbelkner: Jan 7 2011, 22:21