R128GAIN: An EBU R128 compliant loudness scanner Options

[pbelkner]

Removing the album art with e.g. mp3tag solved the problem. R128GAIN wrote RG information to the tags and did not crash.

Thanks for the hint. Possibly it's a bug, I've never tried it with embedded album art.

[jangk]

Thank you for the quick reply.

Do you think you can fix R128GAIN the way it will work also with embedded artwork?

I would prefer not to strip artwork from all my flac-files.

Thanks a lot
Jean

[pbelkner]

Do you think you can fix R128GAIN the way it will work also with embedded artwork?

Hopefully yes. But first I have to have a closer look into it.

[jangk]

Thank you and good luck

[unsword]

Just realized that when run with the --a85 option the Loudness result is still notated in LUFS - although the same thing, I think it should be reported as LKFS in accordance with ATSC A/85

Thanks

[pbelkner]

Version 1.0-alpha-7-1 released:

Home: http://r128gain.sourceforge.net/
Download: http://sourceforge.net/projects/r128gain/files/r128gain/1.0/

What's new?

• There's a new format "Lame" available. Other then the other formats the "Lame" format applies the gain (track or album) and creates a respective MP3. This is most likely what you want for your mobile MP3 player.
  The "Lame" format requires "libmp3lame.dll" to be in the sub-folder "r128gain-tools" (you may get it from RareWares).

• Do you think you can fix R128GAIN the way it will work also with embedded artwork?

  It works for MP3 but unfortunately it currently seems not to be possible for FLAC:
  • FFmpeg's MP3 muxer allows images (cf. video_codec):
    CODE
    AVOutputFormat ff_mp3_muxer = {
        .name              = "mp3",
        .long_name         = NULL_IF_CONFIG_SMALL("MP3 (MPEG audio layer 3)"),
        .mime_type         = "audio/x-mpeg",
        .extensions        = "mp3",
        .priv_data_size    = sizeof(MP3Context),
        .audio_codec       = AV_CODEC_ID_MP3,
        .video_codec       = AV_CODEC_ID_PNG,
        .write_header      = mp3_write_header,
        .write_packet      = mp3_write_packet,
        .write_trailer     = mp3_write_trailer,
        .query_codec       = query_codec,
        .flags             = AVFMT_NOTIMESTAMPS,
        .priv_class        = &mp3_muxer_class,
    };
  • FFmpeg's FLAC muxer doesn't allow images (at least currently, cf. video_codec):
    CODE
    AVOutputFormat ff_flac_muxer = {
        .name              = "flac",
        .long_name         = NULL_IF_CONFIG_SMALL("raw FLAC"),
        .mime_type         = "audio/x-flac",
        .extensions        = "flac",
        .audio_codec       = AV_CODEC_ID_FLAC,
        .video_codec       = AV_CODEC_ID_NONE,
        .write_header      = flac_write_header,
        .write_packet      = flac_write_packet,
        .write_trailer     = flac_write_trailer,
        .flags             = AVFMT_NOTIMESTAMPS,
    };
  Anyway, "r128gain" shouldn't crash any more if an image is present.

• Just realized that when run with the --a85 option the Loudness result is still notated in LUFS - although the same thing, I think it should be reported as LKFS in accordance with ATSC A/85

  Fixed.

CODE

$ r128gain --help
An EBU R128 (http://tech.ebu.ch/loudness) compliant loudness scanner.
For details refer to "http://r128gain.sourceforge.net/".

Usage: r128gain [options] (file|directory)+ [-o <directory> [<format>]]

Options:
  --r128              Run in EBU R128-2011 compliance mode (default).
  --r128-2011         Run in EBU R128-2011 compliance mode (default).
  --r128-2010         Run in EBU R128-2010 compliance mode.
  --a85               Run in ATSC A/85:2011 compliance mode.
  --a85-2011          Run in ATSC A/85:2011 compliance mode.
  --rg2               Run in ReplayGain2 compliance mode.
  --rg                Run in ReplayGain compliance mode.
  --reference=<float>  Set reference loudness in LUFS.
  --r128-compatible   Calibrate output according to EBU R128.
  --rg-compatible     Calibrate output according to ReplayGain.
  --db                Use dB as unit rather then LU/TP.
  --partition=<int>   BS.1770 overlap
                      (overlap in % = (1 - 1/partition) * 100%,
                      default: 4, i.e. 75% overlap).
  --gate=<float>      BS.1770 gate (-10.0 .. -8.0, default: -10.0).
  --rg-calibration=<float>  Aequivalent to use for ReplayGain
                      loudness (default: -18.0).
  --no-peak           Don't calculate the maximum peak.
  --sample-peak       Calculate the maxium peak without up-sampling.
  --true-peak         Calculate the maxium peak at 192 kHz (default).
  --range=on,--range  Calculate loudness range (default).
  --range=off,--no range  Don't calculate loudness range.
  --tags=[rg|bwf]     Write ReplayGain (default) or BWF tags.
  --fast              Switch off up-sampling and don't calculate
                      loudness range.
  --mono=off          Treat mono as stereo (default).
  --mono=on,--mono    Don't treat mono as stereo.
  --quiet             Supress output to stdout.
  --progress=on       Display progress (default).
  --progress=off      Don't display progress.
  --traditional       Format output traditionally.
  --cpown             Copy owner attributes (experimental).
  --cpmod             Copy access rights (experimental).
  --command=<string>  Run command on each track.
  --overwrite         Overwrite already existing output files.
  --in-place          Overwrite original files (not recommended).
  --loglevel=<integer>  Set FFmpeg loglevel.
  --regression        Calculate linear regression between EBU R128
                      and ReplayGain.
  --duration          Print out duration.
  --version           Display version information.
  --ffmpeg=<path>     Directory of the FFmpeg shared libraries.
  --sox=<path>        Directory of the SoX shared libraries.
  --lame=<path>       Directory of the Lame shared libraries.
  --magick=<path>     Directory of the ImageMagick shared libraries.
  --help              Display this information.

Format:
  mkv                 Wrap into MKV container (recommended).
  flac                Encode into FLAC (not recommend).
  lame [album|track [<float>]]  Convert into MP3 and apply album (default)
                      or track gain with quality (default: 2.0).
$ _

[jangk]

Hi,

with the new version, R128GAIN did not crash anymore while treating flac files with embedded artwork (on a sample of 20 files).

Thank you very much for your responsiveness.

Best regards
Jean

[pbelkner]

There was a report over there at the FFSoX Player thread regarding an issue with FFmpeg's OOG decoder. That issue also affects R128GAIN, hence you should upgrade to version 1.0-alpha-7-2:

http://sourceforge.net/projects/r128gain/files/r128gain/1.0/

[MrMod]

First, thank you for the excellent implementation of EBU-R128. After watching the youtube videos on the Loudness Wars from 2011, I'm convinced that R128 is best measure for loudness of general material that we have.

I recently purchased The Lord of the Rings extended blu-ray set, and ran into a problem. As you might know, the three films are split into two discs each. The soundtrack is DTS MA 6.1 24-bit, which results in very large files when combined from the two discs. Due to the size of soundtracks, I always use FLAC to help audio processing tools deal with the file sizes. Unfortunately though, it appears that r128gain has difficulty with FLAC files over 4GB in size. Once the progress reaches 100%, it remains there for a long time before failing due to a malloc error. However, SoX does not have this limitation for processing gain on FLAC files greater than 4GB in size (in this case a 6.92GB file).

I was able to circumvent this issue by having r128gain process the two parts as an album, and then use SoX to apply the gain to the whole FLAC soundtrack. I realize that having FLAC files greater than 4GB is pretty uncommon, but if r128gain could handle them natively, I believe others would benefit in the future.

Again, thank you for this project as I know how much effort it takes to do these sorts of projects.

[Nologic]

@ pbelkner -

Thanks for everything thus far and all the hard work.

I'm not sure but I thought that R128Gain worked on MP4 files...not sure if there was some change in FFMpeg or what...or if I'm just recalling correctly.

At present I don't see any tags set or anything in the atoms...which I think is the only place tags are placed on MP4's (.mp4, .m4a, .m4v, .m4b)

AtomicParsley should display all atom information in a MP4 container.

CODE

AtomicParsley.exe path\to.mp4 -t

Which should display something like:

Atom "----" [replaygain_track_gain] contains: -9.07 dB
Atom "----" [replaygain_album_gain] contains: -9.07 dB
Atom "----" [replaygain_album_peak] contains: 1.116816
Atom "----" [replaygain_track_peak] contains: 1.116816

Or

Atom "----" [replaygain_track_gain] contains: 0.53
Atom "----" [replaygain_track_peak] contains: 0.27
Atom "----" [replaygain_track_minmax] contains: 96,179

The last one is what is written by AACGain...the first one clue...just an example I seen on the web.

Maybe the following lib might be of use: TagLib

Again many thanks for all that you have done.

[skamp]

Am I doing it right?

CODE

./r128gain --rg-compatible --in-place --tags=rg ./album.flac

"album.flac" is a 74 minutes album. That command took 264 seconds to complete on 2.2 GHz mobile Core i7, vs. 19 seconds for the equivalent metaflac command, and 13 seconds for fb2k. That's 14-20 times slower

[pbelkner]

Am I doing it right?

CODE

./r128gain --rg-compatible --in-place --tags=rg ./album.flac

"album.flac" is a 74 minutes album. That command took 264 seconds to complete on 2.2 GHz mobile Core i7, vs. 19 seconds for the equivalent metaflac command, and 13 seconds for fb2k. That's 14-20 times slower

In order to avoid comparing apples with beans you should provide the "--fast" option. The "--fast" option switches off calculating the loudness range and calculating true peak which is on by default and not done by the other tools. Especially true peak calculation is really time consuming because of up-sampling.

[skamp]

I see. The equivalent would actually be the following command line (--fast suppresses peak calculation):

CODE

./r128gain --rg-compatible --sample-peak --range=off --tags=rg --in-place ./album.flac

That command completes in 30 seconds. Still a ways off from metaflac and fb2k, but much better. I have to say, the default settings are so slow, they're downright off-putting.

A few observations:

• Writing tags fails when the file is passed as argument without a path component, e.g. "album.flac" instead of "./album.flac". "./" should be implied. The error message is: "[1/1] "album.flac" ... Error opening "/album.flac"."
• Writing tags to a flac file seems to completely replace the existing vorbis comments block, without using existing padding, thus forcing a full rewrite of the entire file. That's highly inefficient, and padding blocks were meant to avoid exactly that. Use them!
• An unnecessary and deceptive "encoder" tag gets added in the vorbis comments block. The vendor field also gets changed to the name of the ffmpeg encoder.
• Range tags get added even though their values weren't calculated. Instead of writing them with zeroed values that could be confusing, they shouldn't be written at all.

[pbelkner]

I'm not sure but I thought that R128Gain worked on MP4 files...not sure if there was some change in FFMpeg or what...or if I'm just recalling correctly.

This has never worked and by looking into the respective FFmpeg muxer "libavformat/movenc.c" it seems to be impossible (at least for now).

That's the reason why "r128gain" offers the MKV format: The untouched audio and video streams taken from the MP4 container are just wrapped into a MKV container which can be tagged as intended.

[pbelkner]

Version 1.0-alpha-7-3 released:

Home: http://r128gain.sourceforge.net/
Download: http://sourceforge.net/projects/r128gain/files/r128gain/1.0/

What's new?

• The "Lame" format now offers intermediate values between "album gain" and "track gain" as discussed in the thread "Replay Gain: Album Gain vs. Track Gain":
  
  
  
  
  Other then the other formats the "Lame" format applies the gain (track, album, or percentage) and creates a respective MP3. You may consider this useful for creating MP3s for your mobile MP3 player.
  
  

• Writing tags fails when the file is passed as argument without a path component, e.g. "album.flac" instead of "./album.flac". "./" should be implied. The error message is: "[1/1] "album.flac" ... Error opening "/album.flac"."

  
  Fixed.
  
  

• Range tags get added even though their values weren't calculated. Instead of writing them with zeroed values that could be confusing, they shouldn't be written at all.

  
  Fixed.

Some remarks:

• Writing tags to a flac file seems to completely replace the existing vorbis comments block, without using existing padding, thus forcing a full rewrite of the entire file. That's highly inefficient, and padding blocks were meant to avoid exactly that. Use them!

  
  A core design decision for "r128gain" is to base it's I/O on FFmpeg's framework of libraries, i.e. libavformat" and "libavcodec".
  
  The obvious advantage of this approach is not to program against a particular format or codec but against an abstraction of all of them, hence "r128gain" potentially can deal with any of them.
  
  The obvious disadvantage is the same as with all "least common denominator" approaches: you loose fine grained control and performance. In particular this means that "r128gain" has to rewrite a file even if it is not re-encoding it.
  
  Thank you for the remark anyway. Maybe I consider to implement some direct ways in case of common use cases.
  
  

• An unnecessary and deceptive "encoder" tag gets added in the vorbis comments block. The vendor field also gets changed to the name of the ffmpeg encoder.

  
  This is due to the FFmpeg muxer, cf. above.

[xTobix]

Hi Peter,

thanks for the update indeed!

Version 1.0-alpha-7-3 released:

Home: http://r128gain.sourceforge.net/
Download: http://sourceforge.net/projects/r128gain/files/r128gain/1.0/

Your comments are much appreciated too.
In fact, I would much appreciate if you would consider to increase the scanner performance (decrease the CPU+HDD consumption time) since this is my main reason to not convert my entire music library from RG to EBU.

Happy coding,
xTobix

[MrMod]

Unfortunately though, it appears that r128gain has difficulty with FLAC files over 4GB in size. Once the progress reaches 100%, it remains there for a long time before failing due to a malloc error. However, SoX does not have this limitation for processing gain on FLAC files greater than 4GB in size (in this case a 6.92GB file).

I did some more digging into this issue, and I was mistaken. It turns out the problem isn't the file size, but the number of samples that the file contains. If there are more than 2^29 (536.87M) samples, then r128gain hits 100% and hangs. To reproduce the issue, use SoX to create a stereo 24-bit pinknoise file:

contains 533M samples and finishes

CODE

SoX -S -n -c 2 works.flac synth 3:10:00 pinknoise

contains 547M samples and hangs

CODE

SoX -S -n -c 2 fails.flac synth 3:15:00 pinknoise

I believe the issue is simply a 32-bit integer being used for counting the number 4x oversampling samples to process. My 2 cents.

[jaynyc]

hi. just tried this for the first time, very impressive. i am able to successfully scan:

• FLAC
• WAV
• WMA
• MP3
• M4A (ALAC)
• M4A (AAC)

and successfully write tags for all except M4A (AAC). The program crashes for M4A (AAC) when it tries to write.

CODE

Problem Event Name: APPCRASH
Application Name: r128gain.exe
Application Version: 0.0.0.0
Application Timestamp: 50b266f2
Fault Module Name: avformat-54.dll
Fault Module Version: 0.0.0.0
Fault Module Timestamp: 50b2545c
Exception Code: c0000094
Exception Offset: 00232a9b
OS Version: 6.1.7601.2.1.0.256.48
Locale ID: 1033
Additional Information 1: 0a9e
Additional Information 2: 0a9e372d3b4ad19135b953a78882e789
Additional Information 3: 0a9e
Additional Information 4: 0a9e372d3b4ad19135b953a78882e789

I followed the instructions on the project home page

In order to upgrade to full FFmpeg support (i.e. to all FFmpeg supported formats and codecs, unfortunately not including Wavpack) do the following:

• Google for "ffmpeg autobuild".
• Pick the latest W32 shared build and substitute the corresponding DLLs in R128GAIN's sub-directory "r128gain".

What am I doing wrong? thank you

--jay

[Nologic]

@ jaynyc -

I posted about this just a few posts above yours...however there is a work around...well haven't tried it but should work.

What needs to be done for the time being is to process the file with R128Gain, then take that information and pass it to AACGain.

CODE

r128gain.exe --r128-2011 --true-peak --db "C:\test.mp4" --command="aacgain.exe /r /f /g %TGDB% \"%NAME%\""

[pbelkner]

What am I doing wrong?

Nothing.

I've tried to reproduce the crash but was unable. According to your report the program crashes in "libavformat", hence future versions of FFmpeg may avoid this.

But anyway, even if the program doesn't crash it is currently unable to write tags to MP4. You may consider Nologic's advice:

CODE

r128gain.exe --r128-2011 --true-peak --db "C:\test.mp4" --command="aacgain.exe /r /f /g %TGDB% \"%NAME%\""

This post has been edited by pbelkner: Dec 2 2012, 19:05

[pbelkner]

Writing tags to a flac file seems to completely replace the existing vorbis comments block, without using existing padding, thus forcing a full rewrite of the entire file. That's highly inefficient, and padding blocks were meant to avoid exactly that. Use them!

In fact, I would much appreciate if you would consider to increase the scanner performance (decrease the CPU+HDD consumption time) since this is my main reason to not convert my entire music library from RG to EBU.

I believe the issue is simply a 32-bit integer being used for counting the number 4x oversampling samples to process. My 2 cents.

Most if not all of these issues are introduced by the frameworks on which "r128gain" is based. Fortunately we can expect them to disappear as time evolves.

In the meantime some specialized tools may help out, as e.g. "flac1770":

http://www.hydrogenaudio.org/forums/index....showtopic=98163

[flloyd]

How do I get R128Gain to work with MP3s on Linux/Ubuntu? All of the help I've ssen for MP3s references Win32 builds of FFMPEG which I'm assuming would not work on Linux. Is there a way to get R128Gain to use my current or new install of FFMPEG or Libav (since I believe that's what Ubunutu now uses)?

Thanks in advance for any input.

[pbelkner]

How do I get R128Gain to work with MP3s on Linux/Ubuntu? All of the help I've ssen for MP3s references Win32 builds of FFMPEG which I'm assuming would not work on Linux. Is there a way to get R128Gain to use my current or new install of FFMPEG or Libav (since I believe that's what Ubunutu now uses)?

Thanks in advance for any input.

You have two options:

1. Copy FFmpeg's shared objects (*.so) from the installation directory to R128GAIN's "r128gain-tools" folder.
2. Start R128GAIN with the option "--ffmpeg" pointig to the directory where FFmpeg's shared objects (*.so) are located.

Please note that R128GAIN is not guarantied to run with LIBAV. The binaries are build using FFmpeg.

[MrMod]

Most if not all of these issues are introduced by the frameworks on which "r128gain" is based. Fortunately we can expect them to disappear as time evolves.

I was able to reproduce the hanging issue outside r128gain by running:

CODE

sox -V -S -r 48000 -b 24 -n -n synth 3:10:00 pinknoise rate 96000 rate 192000

So I dove into the SoX source code and located the problem in the rate effect. They are improperly casting
size_t to an int which fails at the 2^31 boundary for integers. To fix the issue I created a patch file.

sox-14.4.0.patch:

CODE

diff -cr sox-14.4.0/src/rate.c sox-14.4.0/src-update/rate.c
*** sox-14.4.0/src/rate.c    Tue Dec 27 22:15:32 2011
--- sox-14.4.0/src-update/rate.c    Tue Dec  4 11:04:47 2012
***************
*** 397,403 ****
    size_t remaining = samples_out - p->samples_out;
    sample_t * buff = calloc(1024, sizeof(*buff));
  
!   if ((int)remaining > 0) {
      while ((size_t)fifo_occupancy(fifo) < remaining) {
        rate_input(p, buff, (size_t) 1024);
        rate_process(p);
--- 397,403 ----
    size_t remaining = samples_out - p->samples_out;
    sample_t * buff = calloc(1024, sizeof(*buff));
  
!   if (samples_out > p->samples_out) {
      while ((size_t)fifo_occupancy(fifo) < remaining) {
        rate_input(p, buff, (size_t) 1024);
        rate_process(p);

and added it to r128gain's Makefile:

CODE

.PRECIOUS: unpack/$(SOX)/configure
unpack/$(SOX)/configure: $(DOWNLOAD)/$(SOX).tar.bz2
    mkdir -p unpack
    tar xfvj $< -C unpack
    cd unpack; patch -p0 -i $(SRC)/patch/$(SOX).patch    <---- added
    touch $@

I also submitted a bug report to the SoX sourceforge team. In the meantime you are welcome to use this patch file
if you wish. Thanks again for r128gain, it is a great tool.

[flloyd]

How do I get R128Gain to work with MP3s on Linux/Ubuntu? All of the help I've ssen for MP3s references Win32 builds of FFMPEG which I'm assuming would not work on Linux. Is there a way to get R128Gain to use my current or new install of FFMPEG or Libav (since I believe that's what Ubunutu now uses)?

Thanks in advance for any input.

You have two options:

1. Copy FFmpeg's shared objects (*.so) from the installation directory to R128GAIN's "r128gain-tools" folder.
2. Start R128GAIN with the option "--ffmpeg" pointig to the directory where FFmpeg's shared objects (*.so) are located.

Please note that R128GAIN is not guarantied to run with LIBAV. The binaries are build using FFmpeg.

Since I am using Ubuntu I am unfortuantely stuck with Libav. Trying option number two I get the following error:

CODE

'/home/chris/Downloads/r128gain-1.0-alpha-7-3/r128gain' --rg2 --ffmpeg=/usr/lib/x86_64-linux-gnu/ --overwrite '/home/chris/Desktop/(2003) Glass Wars'
SoX sucessfully loaded.
Error loading FFmpeg.
Falling back to SoX.
/home/chris/Desktop/(2003) Glass Wars
formats: no handler for file extension `mp3'
formats: no handler for file extension `mp3'
formats: no handler for file extension `mp3'
formats: no handler for file extension `mp3'
formats: no handler for file extension `mp3'
formats: no handler for file extension `mp3'
formats: no handler for file extension `mp3'
Done.

When I tried downloading FFMPEG files from here and replacing the files in the r128gain-tools folder I get the following error:

CODE

'/home/chris/Downloads/r128gain-1.0-alpha-7-3/r128gain' --rg2 --overwrite '/home/chris/Desktop/(2003) Glass Wars'
SoX sucessfully loaded.
Error loading FFmpeg.
Falling back to SoX.
/home/chris/Desktop/(2003) Glass Wars
formats: no handler for file extension `mp3'
formats: no handler for file extension `mp3'
formats: no handler for file extension `mp3'
formats: no handler for file extension `mp3'
formats: no handler for file extension `mp3'
formats: no handler for file extension `mp3'
formats: no handler for file extension `mp3'
Done.

Any suggestions?