R128GAIN: An EBU R128 compliant loudness scanner Options

[pbelkner]

I've just uploaded on sourceforge a first version of r128gain, an EBU R128 (http://tech.ebu.ch/loudness) compliant loudness scanner:

http://sourceforge.net/projects/r128gain/files/

• process single files (tracks):
  
  • In order to just scan and analyze one file (WAV, FLAC or WV):
    
    CODE
    r128gain <file>
  • In order to scan, analyze and tag one file (currently only FLAC, result will be in <directory>):
    
    CODE
    r128gain <file> <directory>
  • In order to scan, analyze one file (WAV, WV) and encode and tag into FLAC (result will be in <directory>):
    
    CODE
    r128gain <file> <directory> flac
• recursively process directories (the content of a directory is considered an album):
  
  • In order to just recursively scan and analyze a directory:
    
    CODE
    r128gain <directory>
  • In order to recursively scan, analyze and tag a directory:
    
    CODE
    r128gain <directory> <directory>
    r128gain <directory>/* <directory>
  • In order to recursively scan, analyze a directory and encode and tag into FLAC:
    
    CODE
    r128gain <directory> <directory> flac
    r128gain <directory>/* <directory> flac

Tagging is currently supported only for FLAC:

• REPLAYGAIN_REFERENCE_LOUDNESS: always -23 LUFS (corresponding to EBU R128)
• REPLAYGAIN_TRACK_GAIN: loudness difference to -23 LUFS in LU measured for this track
• REPLAYGAIN_TRACK_PEAK: true peak measured at 192 kHz for this track
• REPLAYGAIN_ALBUM_GAIN: loudness difference to -23 LUFS in LU measured for the whole album
• REPLAYGAIN_ALBUM_PEAK: true peak measured at 192 kHz for the whole album

These tags should be honored by each Replay Gain compliant media player.

[googlebot]

Didn't try the tool, yet, but had a look at the source. Great work!

It would be nice, if you could also include your test suite.

[pbelkner]

Didn't try the tool, yet,

You really should do that. After all that's the only thing that counts. Currently I'm in the process to convert my whole audio collection in order to get an impression. The first results are very promising.

but had a look at the source. Great work!

Thanks

It would be nice, if you could also include your test suite.

Here we go:

CODE

$ r128gain ../sounds/ebu-loudness-test-setv01
../sounds/ebu-loudness-test-setv01
formats: no handler for file extension `txt'
  analyzing ...
    1kHz Sine -20 LUFS-16bit.wav (1/16): -19.2 LUFS, -3.8 LU (peak: 0.100734: -10.0 dBFS)
    1kHz Sine -26 LUFS-16bit.wav (2/16): -25.2 LUFS, 2.2 LU (peak: 0.050508: -13.0 dBFS)
    1kHz Sine -40 LUFS-16bit.wav (3/16): -39.2 LUFS, 16.2 LU (peak: 0.010260: -19.9 dBFS)
formats: no handler for file extension `txt'
    seq-3341-1-16bit.wav (4/16): -22.2 LUFS, -0.8 LU (peak: 0.071316: -11.5 dBFS)
    seq-3341-2-16bit.wav (5/16): -32.2 LUFS, 9.2 LU (peak: 0.023049: -16.4 dBFS)
    seq-3341-3-16bit.wav (6/16): -26.7 LUFS, 3.7 LU (peak: 0.071468: -11.5 dBFS)
    seq-3341-4-16bit.wav (7/16): -26.8 LUFS, 3.8 LU (peak: 0.070850: -11.5 dBFS)
    seq-3341-5-16bit.wav (8/16): -22.2 LUFS, -0.8 LU (peak: 0.100845: -10.0 dBFS)
    seq-3341-6-5channels-16bit.wav (9/16): -22.3 LUFS, -0.7 LU (peak: 0.063133: -12.0 dBFS)
    seq-3341-6-6channels-WAVEEX-16bit.wav (10/16): -22.9 LUFS, -0.1 LU (peak: 0.063133: -12.0 dBFS)
    seq-3341-7_seq-3342-5-24bit.wav (11/16): -21.5 LUFS, -1.5 LU (peak: 0.358341: -4.5 dBFS)
    seq-3341-8_seq-3342-6-24bit.wav (12/16): -22.8 LUFS, -0.2 LU (peak: 0.718299: -1.4 dBFS)
    seq-3342-1-16bit.wav (13/16): -21.8 LUFS, -1.2 LU (peak: 0.100089: -10.0 dBFS)
    seq-3342-2-16bit.wav (14/16): -16.0 LUFS, -7.0 LU (peak: 0.177974: -7.5 dBFS)
    seq-3342-3-16bit.wav (15/16): -22.2 LUFS, -0.8 LU (peak: 0.100089: -10.0 dBFS)
    seq-3342-4-16bit.wav (16/16): -25.9 LUFS, 2.9 LU (peak: 0.100075: -10.0 dBFS)
    ALBUM: -22.7 LUFS, -0.3 LU (peak: 0.718299: -1.4 dBFS)

Please note: Unfortunately version 0.1 had a minor glitch in interpreting the wildcard. I've just uploaded 0.2.

This post has been edited by pbelkner: Jan 5 2011, 21:34

[C.R.Helmrich]

Wow, thank you so much for this tool! I was already mentally preparing myself to have to write my own R128 scanner, but here it is! And in version 0.2, it already does all I need.

Having said that...

When I pass a file path with spaces and without "enclosing quotes", or a file which doesn't exist, it prints an assert and crashes.

Other than that, it seems to analyze as expected, but I have some questions after looking at the source code.

• For the loudness analysis, all input files are resampled to 48 kHz, right?
• For the peak finder, the files are additionally resampled to 192 kHz, right?
• The way I understand R128 is:
  Pass 1: analyze entire file/album, compute loudness taking into account the absolute gate of -70 LUFS.
  Pass 2: analyze entire file/album again, this time also taking into account the relative loudness gate derived from the result of pass 1. Is that how you implemented it?

Thanks again for your work!

Chris

[pbelkner]

Wow, thank you so much for this tool! I was already mentally preparing myself to have to write my own R128 scanner, but here it is! And in version 0.2, it already does all I need.

Thanks

When I pass a file path with spaces and without "enclosing quotes", or a file which doesn't exist, it prints an assert and crashes.

The scanner is heavily based on SoX (http://sox.sourceforge.net/). This is SoX philosophy: use assert() in production code, i.e. if a pre-condition isn't fulfilled just die.

• For the loudness analysis, all input files are resampled to 48 kHz, right?
• For the peak finder, the files are additionally resampled to 192 kHz, right?

48 kHz is due to BS 1770 because they define filter coefficients for that sample rate only. R-REC-BS.1770-1-200709-I!!PDF-E.pdf states:

These filter coefficients are for a sampling rate of 48 kHz. Implementations at other sampling rates will require different coefficient values, which should be chosen to provide the same frequency response that the specified filter provides at 48 kHz. The values of these coefficients may need to be quantized due to the internal precision of the available hardware. Tests have shown that the performance of the algorithm is not sensitive to small variations in these coefficients.

It is not obvious for me how to quantize the given coefficients with respect to other sample frequencies, hence I decided to re-sample to 48 kHz

On the other hand R-REC-BS.1770-1-200709-I!!PDF-E.pdf states regarding true peak determination:

1. Attenuate: 12.04 dB attenuation
2. 4 × over-sampling
3. Emphasis: Pre-emphasis shelving filter, zero at 14.1 kHz, pole at 20 kHz (optional)
4. DC block (optional)
5. Absolute: Absolute value
6. Max: Highest value detection (optional, included if DC block is included).

The current version of R128GAIN combines all this in order to avoid multiple passes:

1. Attenuate by 0.5 (i.e. ca. 6 dB, future versions will correct this).
2. Up-sample to 192 kHz (i.e. approximately 4 x over-sampling)
3. Determine "true" peak.
4. Down-sample to 48 kHz in order to match defined filter coefficients.
5. Apply R128 algorithm.

The way I understand R128 is:
Pass 1: analyze entire file/album, compute loudness taking into account the absolute gate of -70 LUFS.
Pass 2: analyze entire file/album again, this time also taking into account the relative loudness gate derived from the result of pass 1. Is that how you implemented it?

tech3341.pdf states:

1. using an absolute 'silence' gating threshold at -70 LUFS for the computation of the absolute-gated loudness level, and
2. using a relative gating threshold, 8 LU below the absolute-gated loudness level, and
3. the measurement input to which the gating threshold is applied is the loudness of the400 ms gating blocks measured using an ITU-R BS.1770 method without gating, that is, summed across channels;
4. a constant overlap between consecutive gating blocks of at least 50% is required (for increased precision especially when measuring programs of short duration).

R128GAIN interprets this as follows:

• the -70 LUFS is absolute, hence no second path is needed.
• the 8 LU is relative to the 400ms gating block. R128GAIN uses a running 400ms gating block, hence 50% overlap is guarantied and no second pass is needed.

[Fandango]

On a completely other note... What about patents? Is it safe to use your tool in other projects?

This post has been edited by Fandango: Jan 6 2011, 01:53

[Notat]

Congratulations on this!

I know that BS.1770 may be used royalty free. I assume the same is true for R128 but I'll find out.

[mudlord]

What is the license for the tool?

[A_Man_Eating_Duc...]

i can't for the life of me get this app to just analyse and then tag, it always encodes the FLAC's again.

CODE

D:\New Downloads\r128gain-0.2.exe>r128gain.exe "d:\New Downloads\r128gain-0.2.exe\2009 - This Is War
" "d:\New Downloads\r128gain-0.2.exe\2009 - This Is War"
d:\New Downloads\r128gain-0.2.exe\2009 - This Is War
analyzing ...
01. Escape.flac (1/12): -13.4 LUFS, -9.6 LU (peak: 1.001484: 0.0 dBFS)
02. Night Of The Hunter.flac (2/12): biquad: biquad clipped 4 samples; decrease volume?
-5.7 LUFS, -17.3 LU (peak: 1.138934: 0.6 dBFS)
03. Kings And Queens.flac (3/12): biquad: biquad clipped 1 samples; decrease volume?
-6.4 LUFS, -16.6 LU (peak: 1.136147: 0.6 dBFS)
04. This Is War.flac (4/12): biquad: biquad clipped 4 samples; decrease volume?
-7.0 LUFS, -16.0 LU (peak: 1.135694: 0.6 dBFS)
05. 100 Suns.flac (5/12): -18.2 LUFS, -4.8 LU (peak: 0.602695: -2.2 dBFS)
06. Hurricane.flac (6/12): -7.4 LUFS, -15.6 LU (peak: 1.061867: 0.3 dBFS)
07. Closer To The Edge.flac (7/12): -5.3 LUFS, -17.7 LU (peak: 1.309490: 1.2 dBFS)
08. Vox Populi.flac (8/12): -6.0 LUFS, -17.0 LU (peak: 1.083542: 0.3 dBFS)
09. Search And Destroy.flac (9/12): -7.1 LUFS, -15.9 LU (peak: 1.085039: 0.4 dBFS)
10. Alibi.flac (10/12): -8.9 LUFS, -14.1 LU (peak: 1.017255: 0.1 dBFS)
11. Stranger In A Strange Land.flac (11/12): -7.2 LUFS, -15.8 LU (peak: 1.082148: 0.3 dBFS)
12. L490.flac (12/12): -9.3 LUFS, -13.7 LU (peak: 1.010430: 0.0 dBFS)
ALBUM: -7.2 LUFS, -15.8 LU (peak: 1.309490: 1.2 dBFS)
encoding ...
01. Escape.flac (1/12) ... done.
02. Night Of The Hunter.flac (2/12) ... done.
03. Kings And Queens.flac (3/12) ... done.
04. This Is War.flac (4/12) ... done.
05. 100 Suns.flac (5/12) ... done.
06. Hurricane.flac (6/12) ... done.
07. Closer To The Edge.flac (7/12) ... done.
08. Vox Populi.flac (8/12) ... done.
09. Search And Destroy.flac (9/12) ... done.
10. Alibi.flac (10/12) ... done.
11. Stranger In A Strange Land.flac (11/12) ... done.
12. L490.flac (12/12) ... done.

Is there any possibility of getting proper switches implemented?
e.g.

CODE

r128gain <switch> <directory> <directory>
switch:
--a --analyse
--at --analyse-tag
--ate --analyse-tag-encode

[pbelkner]

On a completely other note... What about patents?

I'm no lawyer, but the least I can say is the following: At the time of this writing the implementation of R128GAIN is exclusively based on information publicly available from the following documents:

1. ITU-R BS.1770 ‘Algorithms to measure audio programme loudness and true-peak audio level’
2. EBU Technical Recommendation R 128: Loudness normalisation and permitted maximum level of audio signals
3. EBU Tech Doc 3341: ‘Loudness Metering ‘EBU Mode’ metering to supplement loudness normalisation in accordance with EBU R 128’
4. EBU Tech Doc 3342 ‘Loudness Range: A descriptor to supplement loudness normalisation in accordance with EBU R 128’

None of them contain any reference to a patent, especially not in their list of references.

What is the license for the tool?

Is it safe to use your tool in other projects?

It's GPLv3 (http://www.gnu.org/licenses/gpl.html)

[pbelkner]

i can't for the life of me get this app to just analyse and then tag, it always encodes the FLAC's again.

That's a restriction of this (early) version. But even if you only tag you have to overwrite the existing file. Overwriting your files is the last thing I wanna do!

Currently I'm thinking about how to preserve the encoded streams. That's especially important for lossy codecs as e.g. MP3. Hopefully FFmpeg will offer a solution.

Is there any possibility of getting proper switches implemented?
e.g.

CODE

r128gain <switch> <directory> <directory>
switch:
--a --analyse
--at --analyse-tag
--ate --analyse-tag-encode

The command line syntax will most likely change in the future. But as already stated, overwriting your files is the last thing I consider.

[mudlord]

It's GPLv3 (http://www.gnu.org/licenses/gpl.html)

Any reason why? To suit "Free" software uses only? Doing so restricts other players like FB2K from using that code, even if a different implementation is made....>_>

[mudlord]

On a completely other note... What about patents? Is it safe to use your tool in other projects?

No. Its not safe for non "open source" projects.

The only way for major closed source players like FB2K to use it would involve complete, clean room reimplementations. Which means you run into the same issues as ReplayGain with differing implementations.

This post has been edited by mudlord: Jan 6 2011, 13:23

[Zao]

Note that "use" in mudlord's post is about reusing code from it, not launching it as a standalone tool.

If you ever split out the guts of the tool into a backend library, please consider relicensing it under a non-copyleft license like MIT or zlib, so it can be used in other software like a foobar2000 component.

[cpchan]

The only way for major closed source players like FB2K to use it would involve complete, clean room reimplementations. Which means you run into the same issues as ReplayGain with differing implementations.

Huh, it is a stand alone program and not a library that you link to. As long as the closed sourced players invokes it as an external helper program, there is no problem. The closed source players can even distribute it as long as the license is intact and the source code, or a link to the source, is provided.

[mudlord]

The only way for major closed source players like FB2K to use it would involve complete, clean room reimplementations. Which means you run into the same issues as ReplayGain with differing implementations.

Huh, it is a stand alone program and not a library that you link to. As long as the closed sourced players invokes it as an external helper program, there is no problem. The closed source players can even distribute it as long as the license is intact and the source code, or a link to the source, is provided.

I'd rather a implementation of it in the host audio player rather than a outside application.

[Zao]

Even if the end artifact is a program, this prevents anyone from lifting source code from it (with proper attribution), or preventing anyone from forking it into a library with a sane license. So the license matters, even for a standalone application.

[cpchan]

If you ever split out the guts of the tool into a backend library, please consider relicensing it under a non-copyleft license like MIT or zlib, so it can be used in other software like a foobar2000 component.

Yes, and let other people leech off your work without contributing back.

[cpchan]

Even if the end artifact is a program, this prevents anyone from lifting source code from it (with proper attribution), or preventing anyone from forking it into a library with a sane license.

What is sane to you is not sane for others..

So the license matters, even for a standalone application.

True. For me it is the GPL.

[Zao]

If you ever split out the guts of the tool into a backend library, please consider relicensing it under a non-copyleft license like MIT or zlib, so it can be used in other software like a foobar2000 component.

Yes, and let other people leech off your work without contributing back.

Eh? Using non-copyleft licenses and being a douche are orthogonal concepts. If you want to steal code, you can just ignore licenses like Miriam and steal anything you want.
What I'm saying is that the choice of license (uninformed or informed) prevents use of part of the software in other software.
If more kinds of applications can use the code, the quality of the code improves. More software exercising the code in different ways makes it more robust. An interface that only has one client tends to be very brittle and idiosyncratic.

[Zao]

What is sane to you is not sane for others..

Replace "sane" with "non-copyleft open source" then, as "sane" is an subjective choice of words.

[cpchan]

If more kinds of applications can use the code, the quality of the code improves. More software exercising the code in different ways makes it more robust. An interface that only has one client tends to be very brittle and idiosyncratic.

Huh, how can the quality of the code improve since the changes in the closed source program is by definition "closed"? No one outside can see them.

[Zao]

If a developer uses a library with one of the usual non-copyleft open source licenses, he's bound by the license to make any alterations to the library available.
As such, if the library needs alterations to work in the software (may it be open source or not), the changes the developer makes will be made available (and probably contributed back in the form of patches) to the original author.
If the library works perfectly fine, all is good, both for the original author and for the developer.

[lvqcl]

Please remember that

The scanner is heavily based on SoX

and SoX is covered by LGPL.

[cpchan]

If a developer uses a library with one of the usual non-copyleft open source licenses, he's bound by the license to make any alterations to the library available.

Really, how about BSD or MIT?

Back on topic. For a library, I will actually choose LGPL. This allows any program (open or closed) to be linked against it. However, if the library is modified (and distributed), the changed source code must be made available.