R128GAIN: An EBU R128 compliant loudness scanner Options

[[JAZ]]

The gain that is written in a replaygain tag is relative. And by definition, a relative "something" needs an absolute value to be compared to.

Replaygain strictly defines (or at least now that the documentation is being rewritten) that the absolute value to which the relative gains are calculated, be -89dB.


What is it that I am missing?

[benski]

Has anyone read the paper from Dolby at AES128 comparing ITU-R BS.1770 to Replay Gain. They suggest a -18 LU reference loudness for comparison with Replay Gain. I'm in contact with Martin Wolters (one of the authors of the paper) to try to see if we can pinpoint the reference level exactly.

Also, I strongly object to writing REPLAYGAIN_ALBUM_GAIN and REPLAYGAIN_TRACK_GAIN values that are referenced to something other than 89dB ReplayGain (which corresponds somewhere between -17 LU and -18 LU). ReplayGain is meant as a standard for playback and the exact means of how the volume adjustment was determined do not matter as long as the reference loudness is the same. It might have been done manually, RMS average, or using the filter recommended in the original proposal. The spec even allows for such differences in algorithms to exist!

If pbelkner is unwilling to do that, that's his decision as it's also his tool. But it's not the only tool - Raiden has released libebur128 which kode54 has already integrated into a Foobar2000 scanner, and which I'm also using as a basis for a Winamp scanner. I just want to sort out the reference level / calibration issue before releasing anything official.

This post has been edited by benski: Jan 29 2011, 23:57

[pbelkner]

If pbelkner is unwilling to do that

Did you ever had a closer look at R128GAIN?

Version 0.7 released:

http://sourceforge.net/projects/r128gain/files/

1. Of course, EBU 128 mode is the default:
   
   
   
   
   
   • It uses the gated BS.1770 algorithm.
   • It determines peaks at 4 x up-sampled rate (True Peak).
   • It writes tags with respect to -23 LUFS (compatible with EBU 128).
   
   
2. Instead you may choose ReplayGain compliance:
   
   
   
   
   
   • It uses the classic "gain_analysis.c" as it is found e.g. in WaveGain and as it re-distributed with the WA SDK.
   • Peaks are determined without up-sampling (no "True Peak") as required by Notat.
   • Tags are written according to the RG standard (i.e. just as calculated by "gain_analysis.c" without any alteration).
   
   
3. Finally you may choose your own profile. The following seems to be the most wanted:
   
   
   
   
   
   • You may choose the BS.1770 algorithm.
   • You may choose "True Peak" (i.e. up-sampling for peak determination).
   • You may choose to write the tags RG compatible (i.e. with respect to -18 LUFS).
   • You may fine tune the -18 LUFS.
   • In order to get an idea how to fine tune the -18 LUFS you may first run a linear regression between RG and BS.1770 (as proposed by yourself).

How do you come to say that I'm unwilling to write the tags compatible with RG? With this tool you are the one deciding how the tags are written.

Other news:

• Converting to FLAC is available from the GUI.
• You may choose to tag the files in place (please be very careful with this option, if something's going wrong your files may be lost forever).

EDIT: Command line options:

CODE

$ r128gain --help
An EBU R128 (http://tech.ebu.ch/loudness) compliant loudness scanner.
For details refer to "http://r128gain.sourceforge.net/".

Usage: r128gain [options] (file|directory)+ [-o <directory> [flac]]
Options:
  --r128             Run in EBU R128 compliant mode (default).
  --rg               Run in ReplayGain compliant mode.
  --r128-compatible  Calibrate output according to EBU R128.
  --rg-compatible    Calibrate output according to ReplayGain.
  --rg-calibration=<float>  Aequivalent to use for ReplayGain
                     loudness (default: -18.0).
  --true-peak=on,--true-peak  Up-sample for peak determination.
  --true-peak=off,--fast  Switch off up-sampling.
  --in-place         Allow overwriting of files in place.
  --regression       Calculate linear regression between EBU R128
                     and ReplayGain.
  --duration         Print out duration.
  --version          Display version information.
  --help             Display this information.

This post has been edited by pbelkner: Jan 30 2011, 01:46

[pbelkner]

LIB1770 v0.1 released

I've started to factor out some code from R128GAIN and to put it into a library called LIB1770:

http://sourceforge.net/projects/r128gain/files/lib1770/

The library is under LGPL 2.1 (or any later).

The public API resembles the one known from the wide spread RG implementation "gain_analysis.c":

CODE

ctx=bs1770_ctx_open()

for each album {
  for each track {
    while there are samples
      bs1770_ctx_add_sample(ctx, rate, channels, sample)

    bs1770_ctx_track_lufs(ctx, rate, channels)
  }

  bs1770_ctx_album_lufs(ctx)
}

bs1770_ctx_close(ctx)

Following is an example program which reproduces the EBU R128 test vector:

CODE

#include <stdio.h>
#include "bs1770_ctx.h"

// assume 2 channel interleaved/48 kHz/64 bit float raw pcm.
#define CHANNELS  2
#define RATE      48000

#define BUF_SIZE  (CHANNELS*4096)

int main(int argc, char **argv)
{
  static double buf[BUF_SIZE];

  bs1770_ctx_t *ctx;
  int i,j;
  FILE *f;
  size_t n;
  bs1770_sample_t sample;

  // print out library version.
  fprintf(stderr, "lib1770 v%s\n", bs1770_version);

  // open a 1770 context.
  if (NULL==(ctx=bs1770_ctx_open()))
    goto cleanup;

  // for each track ...
  for (i=1;i<argc;++i) {
    int channel=0;  // offset in sample vector.

    // open the raw pcm stream.
    if (NULL==(f=fopen(argv[i],"rb")))
      continue;

    // display the name of the current track.
    fprintf(stderr, "%s: ", argv[i]);
    fflush(stderr);

    // while there are samples ...
    while (0<(n=fread(buf, sizeof buf[0], BUF_SIZE, f))) {
      // consume the samples read.
      for (j=0;j<n;++j) {
        // build up a sample vector accross the channels.
        // NOTE: client code may apply a "channel map" right here.
        //   lib1770 assumes the order "left", "right", "centre",
        //   "left sorround", "right sorround".
        if (channel<BS1770_MAX_CHANNELS)
          sample[channel]=buf[j];

          // if the sample vector is done add it to the 1770 statistics.
          if (++channel==CHANNELS) {
            bs1770_ctx_add_sample(ctx, RATE, CHANNELS, sample);
            channel=0;
          }
      }
    }

    // print out track statistics (implicitely clears).
    fprintf(stderr, "%.1f LUFS.\n", bs1770_ctx_track_lufs(ctx,
        RATE, CHANNELS));

    // close the raw stream.
    fclose(f);
  }

  // print out album statistics (implicitely clears).
  fprintf(stderr, "ALBUM: %.1f LUFS.\n", bs1770_ctx_album_lufs(ctx));
cleanup:
  // close the 1770 context.
  if (NULL!=ctx)
    bs1770_ctx_close(ctx);

  return 0;
}

In order to run the test vector you first have to convert it into 2 channel interleaved/48 kHz/64 bit float raw pcm. This can be achieved using SoX in the following way:

CODE

sox a.wav -c 2 -b 64 -e float b.raw

The above example program produces the following output:

CODE

$ example ../sounds/ebu-loudness-test-setv01/*.raw
lib1770 v0.1
../sounds/ebu-loudness-test-setv01/1kHz Sine -20 LUFS-16bit.raw: -20.0 LUFS.
../sounds/ebu-loudness-test-setv01/1kHz Sine -26 LUFS-16bit.raw: -26.0 LUFS.
../sounds/ebu-loudness-test-setv01/1kHz Sine -40 LUFS-16bit.raw: -40.0 LUFS.
../sounds/ebu-loudness-test-setv01/seq-3341-1-16bit.raw: -23.0 LUFS.
../sounds/ebu-loudness-test-setv01/seq-3341-2-16bit.raw: -33.0 LUFS.
../sounds/ebu-loudness-test-setv01/seq-3341-3-16bit.raw: -23.0 LUFS.
../sounds/ebu-loudness-test-setv01/seq-3341-4-16bit.raw: -23.0 LUFS.
../sounds/ebu-loudness-test-setv01/seq-3341-5-16bit.raw: -22.9 LUFS.
../sounds/ebu-loudness-test-setv01/seq-3341-6-5channels-16bit.raw: -27.8 LUFS.
../sounds/ebu-loudness-test-setv01/seq-3341-6-6channels-WAVEEX-16bit.raw: -28.9 LUFS.
../sounds/ebu-loudness-test-setv01/seq-3341-7_seq-3342-5-24bit.raw: -23.0 LUFS.
../sounds/ebu-loudness-test-setv01/seq-3341-8_seq-3342-6-24bit.raw: -23.0 LUFS.
../sounds/ebu-loudness-test-setv01/seq-3342-1-16bit.raw: -22.6 LUFS.
../sounds/ebu-loudness-test-setv01/seq-3342-2-16bit.raw: -16.8 LUFS.
../sounds/ebu-loudness-test-setv01/seq-3342-3-16bit.raw: -20.0 LUFS.
../sounds/ebu-loudness-test-setv01/seq-3342-4-16bit.raw: -20.0 LUFS.
ALBUM: -21.9 LUFS.

Please note the deviation for multichannel. This is because these samples where down-mixed beforehand by SoX into two channels.

[Notat]

Correct me if I'm wrong but it is my understanding that in R128 mode, R128Gain writes "REPLAYGAIN_*" tags with R128 values. I'd find no fault if it instead wrote "EBUR128_*" tags or somesuch in this mode.

[jangk]

Replaygain strictly defines (or at least now that the documentation is being rewritten) that the absolute value to which the relative gains are calculated, be -89dB.

There is no absolute value such as -89dB.

Absolute values are dBu, dBm, dBV, dBW, dBµV, dB (A), dBFS etc.

R128gain (and R128) uses unambiguously Full Scale FS as a reference, so the target is -23 LUFS (which in fact corresponds to -23 dBFS).


Jean

[GHammer]

Correct me if I'm wrong but it is my understanding that in R128 mode, R128Gain writes "REPLAYGAIN_*" tags with R128 values. I'd find no fault if it instead wrote "EBUR128_*" tags or somesuch in this mode.

I'd think this is for compatibility with RG capable players.
Kind of useless to have a tool that works, appears to be better than an existing tool, but have no way to hear the results.

So, if I wish to use this tool, how do you propose that I listen with the calculated gain if not by using existing tags?

[googlebot]

So, if I wish to use this tool, how do you propose that I listen with the calculated gain if not by using existing tags?

By using R128 based loudness estimation AND writing REPLAY_GAIN_* based values calibrated to the same loudness as what ReplayGain compatible players expect (so that all tracks can be played back at about the same volume).

The mode, that pbelkner calls "R128 compliant" hijacks the REPLAY_GAIN tag format an writes incompatible values into it. Incompatible since those tracks will play back only about half as loud as traditional RG tracks (~5 dB). This is totally unnecessary and only due to pbelkner's somewhat odd interpretation of what R128 'compliance' should be inside a ReplayGain compliant tag.

Of course, one could have the benefit of both worlds, the better loudness estimation of R128 and playback compatibility to existing ReplayGain tagged collections - with REPLAY_GAIN_* tags but without the 5 dB difference! I do not really see the benefit of more than two options, the latter plus one plain R128 with R128_* tags.

Instead pbelkner now just drowns the user in options. I do not see the benefit of that either. Lets not forget, why one chooses to use RG, to get consistent loudness over a music collection. The current presets will introduce a lot of ~5dB inconsistencies. Up to know I associate ReplayGain with simplicity. Why change that without necessity? More choice is not always better.

This post has been edited by googlebot: Jan 30 2011, 11:27

[pbelkner]

Instead pbelkner now just drowns the user in options.

This tool is not and was never aimed for the mass market. It's aimed to the curious who want to learn something along the way.

BTW: That's the only reason why I'm bothering myself.

[lvqcl]

There is no absolute value such as -89dB.

Absolute values are dBu, dBm, dBV, dBW, dBµV, dB (A), dBFS etc.

And ReplayGain uses dB SPL.

( http://replaygain.hydrogenaudio.org/calibration.html , http://en.wikipedia.org/wiki/Sound_pressur..._pressure_level )

[pbelkner]

Version 0.7.1 released:

http://sourceforge.net/projects/r128gain/files/

It's a bug-fix. There where reportedly noticeable gaps introduced at the end of re-written MP3s. In order to fix that changed av_write_frame() into av_interleaved_write_frame(). Hope this helps.

[C.R.Helmrich]

Has anyone read the paper from Dolby at AES128 comparing ITU-R BS.1770 to Replay Gain. They suggest a -18 LU reference loudness for comparison with Replay Gain. I'm in contact with Martin Wolters (one of the authors of the paper) to try to see if we can pinpoint the reference level exactly.

I read the paper. They only compare BS.1770, not R128 (= gated BS.1770), to ReplayGain. In addtion, the major issue, which is mentioned there, is that the "exact" conversion constant when using the formula

RG(dB) = constant - R128(LUFS)

depends on the loudness of the material you use to measure that constant. As you confirmed yourself, the best-fit conversion involves a slope different from 1:

The optimum slope of -0.81 indicates a dependency on loudness. This can be explained by the loudness dependent level difference between the near maximum frame power (Replay Gain) and the long term power average (ITU-R BS.1770). Due to dynamic range processing and limiting this difference tends to get smaller for louder music. This observation is supported by an experiment where in the Replay Gain procedure the near maximum frame power analysis was replaced by a long term power average. Then the least squares fit between this modified Replay Gain and the ITU-based loudness has indeed a slope of -1.

That's probably still the case when converting from R128 to RG. So what you measured yourself was probably quite appropriate (it tended to emphasize contemporary loud music, right?).

Chris

This post has been edited by C.R.Helmrich: Jan 30 2011, 15:27

[M]

Version 0.7.1 released:

http://sourceforge.net/projects/r128gain/files/

It's a bug-fix. There where reportedly noticeable gaps introduced at the end of re-written MP3s. In order to fix that changed av_write_frame() into av_interleaved_write_frame(). Hope this helps.

Looks interesting, to say the least! There may be a bug in the bug-fix version (although I haven't tested any previous versions, yet; I just became aware of your project this morning).

Although I can add files via the "Choose" button, they are listed without any file extension... so when I add "C:\test\mytestaudio.flac" and then press the "OK" button, the dialogue gives me this message:

CODE

SetDlgItemURL successfully loaded.
SoX successfully loaded.
FFmpeg successfully loaded.
Couldn't stat "C:\test\mytestaudio"
Done. Hit any key to continue ...

(Also, for what it's worth, hitting any key won't close the dialogue. Hitting the "Enter" key will, but all the others type their respective characters.)

- M.

[pbelkner]

There may be a bug in the bug-fix version

Sounds very strange. Unfortunately I'm not able to reproduce it. Tried also spaces and special characters in directory and file names. On my XP 32 and Vista 64 systems everything looks fine.

What Windows version are you on?

[pbelkner]

they are listed without any file extension...

May be this is because you've possibly disabled file extensions in Windows Explorer. Hopefully the next version will fix this.

Probably you can work around this by just enabling file extensions in Windows Explorer.

[M]

There may be a bug in the bug-fix version

Sounds very strange. Unfortunately I'm not able to reproduce it. Tried also spaces and special characters in directory and file names. On my XP 32 and Vista 64 systems everything looks fine.

What Windows version are you on?

Vista Home Premium (32-bit), but that isn't the issue. Fortunately, this should be an easy thing to fix, for future versions!

In my system, I keep "Hide extensions for known file types" checked by default. Your selector dialogue sees the extension, but when it drops the filename into the processing queue, it strips the extension away... thus the "Couldn't stat" message. When I un-hid the extensions system-wide, the R128GAIN suddenly worked.

The "Hit any key to continue" option still ignores anything but "Enter," though.

- M.

Edit: I see I took too long to think about/type my response... you came to the same conclusion, as I was composing.

This post has been edited by M: Jan 30 2011, 19:33

[benski]

Has anyone read the paper from Dolby at AES128 comparing ITU-R BS.1770 to Replay Gain. They suggest a -18 LU reference loudness for comparison with Replay Gain. I'm in contact with Martin Wolters (one of the authors of the paper) to try to see if we can pinpoint the reference level exactly.

I read the paper. They only compare BS.1770, not R128 (= gated BS.1770), to ReplayGain. In addtion, the major issue, which is mentioned there, is that the "exact" conversion constant when using the formula

RG(dB) = constant - R128(LUFS)

depends on the loudness of the material you use to measure that constant. As you confirmed yourself, the best-fit conversion involves a slope different from 1:

The optimum slope of -0.81 indicates a dependency on loudness. This can be explained by the loudness dependent level difference between the near maximum frame power (Replay Gain) and the long term power average (ITU-R BS.1770). Due to dynamic range processing and limiting this difference tends to get smaller for louder music. This observation is supported by an experiment where in the Replay Gain procedure the near maximum frame power analysis was replaced by a long term power average. Then the least squares fit between this modified Replay Gain and the ITU-based loudness has indeed a slope of -1.

That's probably still the case when converting from R128 to RG. So what you measured yourself was probably quite appropriate (it tended to emphasize contemporary loud music, right?).

Chris

Yes you are correct. Re-reading the paper it is clear that they only used BS.1770 and used the loudness of the entire track, so it's not quite an appropriate comparison to ReplayGain (95th percentile) or R128 (gated loudness).

[pbelkner]

In my system, I keep "Hide extensions for known file types" checked by default. the same conclusion, as I was composing.

This is harder as it seems. Fortunately I've already found example code how to resolve it. There you find the following comment:

CODE

/* The normal code to retrieve the item's text is
   not very useful since user may select "hide common
   extensions" */

The "normal code" is exactly the one you currently find in R128GAIN. Changing this will take some time.

The "Hit any key to continue" option still ignores anything but "Enter," though.

Fortunately there is a very simple and obvious solution: the next version will write "Hit enter to continue"

This post has been edited by pbelkner: Jan 30 2011, 20:38

[[JAZ]]

Maybe you should try using common controls instead. Something in line with this:

CODE

OPENFILENAME ofn; // common dialog box structure
char szFile[_MAX_PATH]; // buffer for file name

szFile[0]='\0';
// Initialize OPENFILENAME
ZeroMemory(&ofn, sizeof(OPENFILENAME));
ofn.lStructSize = sizeof(OPENFILENAME);
ofn.hwndOwner = GetParent()->m_hWnd; //You have to supply an HWND here.
ofn.lpstrFile = szFile;
ofn.nMaxFile = sizeof(szFile);
ofn.lpstrFilter = "Example 1 (*.ex1)\0*.ex1\0Example 2 (*.ex2)\0*.ex2\0All (*.*)\0*.*\0";
ofn.nFilterIndex = 1; //Which of the filters above to select by default. Index starts at 1.
ofn.lpstrFileTitle = NULL;
ofn.nMaxFileTitle = 0;
//ofn.lpstrInitialDir = // Use this if you want to set a default directory to show.
ofn.Flags = OFN_PATHMUSTEXIST | OFN_FILEMUSTEXIST;

// Display the Open dialog box.
if(::GetOpenFileName(&ofn)==TRUE)
{
///szFile contains the filename (full path including extension) and ofn.nFilterIndex contains the filter selected in the filter combobox
}

[gjgriffith]

Congratulations for developing a great tool.

Is it considered correct for mono flac files to be tagged for 3dB more gain than their stereo (dual mono) equivalent? This is not intuitive to me, as practically all playback devices will send the same mono signal to the two speakers (effectively making it 3 dB louder already).

Am I missing something?

[kode54]

I have moved foo_r128scan related posts to the relevant topic.

[Notat]

Yes you are correct. Re-reading the paper it is clear that they only used BS.1770 and used the loudness of the entire track, so it's not quite an appropriate comparison to ReplayGain (95th percentile) or R128 (gated loudness).

This work compares 19 different loudness measurement algorithms. R128 did not yet exist when this study was done but several gated versions of BS.1770 were tested. No conversion factors are computed but the raw test data is included so perhaps it could be calculated.

[pbelkner]

Maybe you should try using common controls instead.

Thanks for the hint. But that's exactly what I'm doing (cf. "r128gui_args.c").

The problem with the pure GetOpenFileName() approach is that it doesn't allow users to pick directories. Fortunately I came across this and learned about hook procedures and sub-classing. I was happy having found a solution ... but it appears to work only in the most trival cases.

Letting users pick directories from a common controls dialog appears to me more advanced then implementing EBU R128/BS.1770.

This post has been edited by pbelkner: Jan 31 2011, 11:18

[pbelkner]

Am I missing something?

Propably not. Maybe the solution is having a "treat mono as stereo" switch in the next version.

[2Bdecided]

The mode, that pbelkner calls "R128 compliant" hijacks the REPLAY_GAIN tag format an writes incompatible values into it. Incompatible since those tracks will play back only about half as loud as traditional RG tracks (~5 dB). This is totally unnecessary and only due to pbelkner's somewhat odd interpretation of what R128 'compliance' should be inside a ReplayGain compliant tag.

Of course, one could have the benefit of both worlds, the better loudness estimation of R128 and playback compatibility to existing ReplayGain tagged collections - with REPLAY_GAIN_* tags but without the 5 dB difference! I do not really see the benefit of more than two options, the latter plus one plain R128 with R128_* tags.

I agree. If you're writing REPLAY_GAIN tags they should be 100% compatible with ReplayGain!

If you want to add an extra tag which stores the actual conversion factor used (so someone can convert back to the correct EBU R128 value if they want), by all means do so.

If users really want their audio to be 5dB quieter (or whatever reference level they choose), they can set the ReplayGain pre-amp to -5dB (or whatever).

Cheers,
David.