R128GAIN: An EBU R128 compliant loudness scanner Options

[pbelkner]

Version 1.0-alpha-2 released:

Home: http://r128gain.sourceforge.net/
Download: http://sourceforge.net/projects/r128gain/files/

under linux even as a command line utility

64 bit linux distros which most of us use these days

Calling the new alpha ('r128gain-1.0-alpha-1-win32-native') directly from the commandline gives the Windows error popup:

'r128gain.exe hat ein Problem festgestellt und muss beendet werden.' (encountered a problem and has to be closed.)
AppName: r128gain.exe AppVer: 0.0.0.0 ModName: msvcrt.dll
ModVer: 7.0.2600.5512 Offset: 0000ee96

Example commandline:
r128gain.exe --help
r128gain.exe "C:\tmp\02.wav"

Simply running 'r128gain.exe' will start the GUI, subsequent processing works fine then. I'm using WindowsXP SP3 32bit.

What's new?

• Added a CLI (command line) only version (i.e. without GUI.)
• Provided Linux 64 bit builds.
• Fixed crashing CLI on Windows XP.

[dr.schanker]

Thank you for the fast update.
As described, no more crashing - tested 'r128gain-1.0-alpha-2-win32-native' and 'r128gain-1.0-alpha-2-win32-cli'.

[Darkflames0]

Hi,
I don't know if it's the good place to post, but it's the only forum I have found about r128gain.
I have just updated my sox version to 14.4.0 but it seems that now the r128gain encoder is no longer working, printing me this error :

CODE

Error loading SoX

I'm on a 64bits Ubuntu using the cli version, what can I do to solve this problem.
I would like to precise that before I updated SoX it was working well.
Thx
Darkflames0

[pbelkner]

Hi,
I don't know if it's the good place to post, but it's the only forum I have found about r128gain.
I have just updated my sox version to 14.4.0 but it seems that now the r128gain encoder is no longer working, printing me this error :

CODE

Error loading SoX

I'm on a 64bits Ubuntu using the cli version, what can I do to solve this problem.
I would like to precise that before I updated SoX it was working well.
Thx
Darkflames0

At start-up R128GAIN tries to load "libsox.so" from the (sub-)folder "r128gain-tools". The error message "Error loading SoX" is generated in case loading "libsox.so" fails.

[chi]

At start-up R128GAIN tries to load "libsox.so" from the (sub-)folder "r128gain-tools". The error message "Error loading SoX" is generated in case loading "libsox.so" fails.

On Debian-based systems, several format handlers are separate modules that are loaded by libsox. I’d guess there might be a version mismatch between those modules (probably now updated to 14.4.0) and this local libsox.so that pbelkner mentions.

[pbelkner]

At start-up R128GAIN tries to load "libsox.so" from the (sub-)folder "r128gain-tools". The error message "Error loading SoX" is generated in case loading "libsox.so" fails.

On Debian-based systems, several format handlers are separate modules that are loaded by libsox. I’d guess there might be a version mismatch between those modules (probably now updated to 14.4.0) and this local libsox.so that pbelkner mentions.

The "libsox.so" provided with R128GAIN is compiled from the SoX 14.4.0 sources. Further, (most of) the external dependencies (as e.g. "libFLAC.a") are statically linked to the provided "libsox.so".

EDIT: But yes, this might be the reason. Given it is the reason, the solution could be to substitute the local "libsox.so" from the (sub-)folder "r128gain-tools" with the system's version.

This post has been edited by pbelkner: Jun 19 2012, 16:18

[kode54]

I've uploaded a modified lib1770 and posted some details about it here. It may be useful if you ever get around to implementing a multi-threaded scanner. It also makes it possible to compile the library with MSVC, by guarding some C99 code and providing C89 alternatives. I hope Microsoft decides to adopt C99 some day, though. I didn't include the full distribution or the Makefiles, but I did throw in a MSVC 2010 project file.

[pbelkner]

I've uploaded a modified lib1770 and posted some details about it here. It may be useful if you ever get around to implementing a multi-threaded scanner. It also makes it possible to compile the library with MSVC, by guarding some C99 code and providing C89 alternatives. I hope Microsoft decides to adopt C99 some day, though. I didn't include the full distribution or the Makefiles, but I did throw in a MSVC 2010 project file.

Thanks a lot, kode54. Do you have any information regarding a performance gain due to the multithreaded interface? I guess in practice it is bounded by I/O anyway. At least this is true for the current implementation of R128GAIN because it reads the input files sequentially. Do you think it would be a better idea to read and process the files constituting an album in parallel? At least a bounded number of them? How many?

[kode54]

It may be a good idea to ask Peter about what sort of reading and buffering he does with foo_rgscan, as it seems to gain a lot more over mere parallelization.

For starters, I would try processing up to four files at a time, depending on how many processors and/or cores are available, and also buffer (at least parts of) those files sequentially before processing. Probably a little complicated, and may not be worth the effort. Hmm...

[skamp]

Queue and cache files sequentially, and work on them in parallel as each file is done getting cached.

[pbelkner]

Queue and cache files sequentially, and work on them in parallel as each file is done getting cached.

Yes, of course. I've thought about it for a minute a while ago, but just for a minute.

The point is that it adds a lot of complexity to the program. But it's not obvious to me whether it really contributes to the program's over all performance.

After all the files have to be read from disk. That's a mechanical (i.e. slow) operation, at least these days in almost all cases, putting some hard constraints to the program's overall performance.

Another point is that parallel processing makes only sense to me on machines with more than one core. The number of effective parallel threads is limited by the number of available cores.

That's why I asked the question. Maybe ther're some measurements available comparing parallel vs. sequential implementations in order to justify whether adding much complexity to the program is really worthwhile.

This post has been edited by pbelkner: Jul 31 2012, 10:18

[skamp]

The point is that it adds a lot of complexity to the program. But it's not obvious to me whether it really contributes to the program's over all performance. […] Maybe ther're some measurements available comparing parallel vs. sequential implementations in order to justify whether adding much complexity to the program is really worthwhile.

See this, for instance. Both caudec and fb2k's RG scanner show a tremendous advantage in performance with that sort of implementation, in such cases where data can be computed much faster than it can be read off a HDD. Multiple concurrent reads on HDDs kill their performance and create a much larger bottleneck than necessary.

[pbelkner]

See this, for instance.

Ok, I understand that you are an expert for such kinds of programs.

On top of your Caudec thread you define the constraints under which this kind of optimization makes sense (higlighting by me):

It leverages multi-core CPUs with lots of RAM by copying input files to a tmpfs mount, and running multiple processes concurrently (one per file and per codec).

I'm not certain whether the majority of users will fit into this. Hence we end up with a similar question: Is it worthwhile to add the complexity to a program when it is not obvious how many people will really profit from it?

[skamp]

Ok, I understand that you are an expert for such kinds of programs.

Hardly!

Is it worthwhile to add the complexity to a program when it is not obvious how many people will really profit from it?

I guess it depends on how you look at it. The trend is to have multiple CPU cores, and more and more RAM. I'd say that taking that trend into account is forward thinking and future proofing, hence worthwhile. My motivation behind caudec was to leverage the added processing power of my gear at the time. Owners of recent hardware will greatly benefit from the added complexity, while there isn't really a downside for owners of older hardware. Depends on how motivated you are I guess, and how much fun you think you'll have doing it

[Nologic]

Any chance you would add command line options for setting Sox & FFmpeg to different folders...than the assumed default.

--path-sox
--path-ffmpeg

I'd just like to only have one copy of each on my system is all.

Thanks for your time and all your effort.

This post has been edited by Nologic: Aug 8 2012, 10:23

[pbelkner]

Version 1.0-alpha-3 released:

Home: http://r128gain.sourceforge.net/
Download: http://sourceforge.net/projects/r128gain/files/r128gain/1.0/

Any chance you would add command line options for setting Sox & FFmpeg to different folders...than the assumed default.

--path-sox
--path-ffmpeg

I'd just like to only have one copy of each on my system is all.

Thanks for your time and all your effort.

What's new?

• Added "--ffmpeg" and "--sox" command line options in order to allow for providing the path to the respective shared libraries.
• Fixed missing the "TP", "TPDB", "AP", and "APDB" environment variables.

This post has been edited by pbelkner: Aug 15 2012, 17:35

[Nologic]

Sweet many many thanks for this!

[pbelkner]

Version 1.0-alpha-4 released:

Home: http://r128gain.sourceforge.net/
Download: http://sourceforge.net/projects/r128gain/files/r128gain/1.0/

I've uploaded a modified lib1770 and posted some details about it here. It may be useful if you ever get around to implementing a multi-threaded scanner. It also makes it possible to compile the library with MSVC, by guarding some C99 code and providing C89 alternatives. I hope Microsoft decides to adopt C99 some day, though.

Queue and cache files sequentially, and work on them in parallel as each file is done getting cached.

What's new?

• The core library "lib1770" was restructured in order to avoid duplicated computations. Scanning the EBU R128 test vector demonstrates that there is performance boost of about 40% (without true peak computation).
• The new "lib1770" now supports parallel computations. As demonstrated by the multithreaded "example2" program compared to sequential "example1" program parallelism may give another performance boost of about 50% (not yet used by R128GAIN).
• "lib1770" should now compile with MSVC.

[kode54]

You made a typo on line 85 of bs1770_ctx.c:

CODE

-    bs1770_nd_t *rp=rp+ctx->size;
+    bs1770_nd_t *rp=mp+ctx->size;

[pbelkner]

You made a typo on line 85 of bs1770_ctx.c:

CODE

-    bs1770_nd_t *rp=rp+ctx->size;
+    bs1770_nd_t *rp=mp+ctx->size;

Thanks a lot, kode54. I've uploaded r128gain-1.0-alpha-5 as well as lib1770-0.8.1.

[Goran Tomas]

I have a labeling consistency question - why do you have options labeled as R128-1 and R128-2?

There is no R128-1 and R128-2. There is R128 and there are ITU BS.1770-1 and ITU BS.1770-2. R128 is based on ITU BS.1770-2.

This might confuse people and it's not technically correct to refer to something that doesn't exist (such as R128-1 or R128-2).


Regards,
Goran Tomas

[pbelkner]

There is no R128-1 and R128-2.

From a formal point of view you may be right. But if you have a look at the standard document's history (on page 5) you'll notice that there's

1. a first issue from 2010, and
2. a revised issue (the current) from 2011.

Among others the two versions differ as follows:

• First issue from 2010 (unfortunately not online any longer):
  
  • The document's header calls it "Tech 3341",
  • it refers to ITU-R BS.1770, defines
  • the relative gating threshold as 8 LU and
  • the overlap between consecutive gating blocks to at least 50%.
• Revised issue from 2011:
  
  • The document's header calls it "Tech 3341-2011",
  • it refers to ITU-R BS.1770-2, defines
  • the relative gating threshold as 10 LU and
  • the overlap between consecutive gating blocks to at least 75%.

For the sake of simplicity R128GAIN refers to "Tech 3341" (2010) as R128-1 and to "Tech 3341-2011" as R128-2.

This post has been edited by pbelkner: Sep 23 2012, 20:29

[Goran Tomas]

Exactly. R128 was revised and from that point refers to ITU BS.1770-2. If you measure/normalize something according to ITU BS.1770, it's no longer valid to call it R128 compliant. It has to be ITU BS.1770-2. There's only one R128 recommendation. That's why I'm saying this is confusing to the users. It refers to something that doesn't exist.

My suggestion would be to have an ITU BS.1770 measurement and have a R128 measurement. This would be technically correct and would be relateable exactly to the documents and recommendations as they exist. Moreover, this expands the potential use of this tool to much larger territories such as the USA, because the US standard (ATSC A/85) at the moment uses ITU BS.1770. So, a European user would use R128 and have gated measurement (as it is now, with 10LU relative gate and an absolute gate). A US user would use ITU BS.1770 and have no relative gate (only -70LUFS absolute gate) as this is what's valid for their territory.

This is exactly what some professional product do, to be able to be compliant to both A/85 and R128. I wouldn't introduce more confusion with two R128 revisions (there's already too much confusion). The 2010 R128 revision has been superseded and the current revision is valid and it's called only R128.


Regards,
Goran Tomas

[unsword]

Firstly - Thanks to pbelkner for the hard work and diligence put into producing an excellent piece of software

Using the command line version I have been able to update to gated techniques a tool to measure the loudness of audio associated with MPEG-2 video files which was hacked together over a year ago using FFMPEG, SOX and the BBC BS.1770 based loudness software.

I need to make one point regarding the previous posting:

... Moreover, this expands the potential use of this tool to much larger territories such as the USA, because the US standard (ATSC A/85) at the moment uses ITU BS.1770. So, a European user would use R128 and have gated measurement (as it is now, with 10LU relative gate and an absolute gate). A US user would use ITU BS.1770 and have no relative gate (only -70LUFS absolute gate) as this is what's valid for their territory.

ATSC A/85 does not mandate any specific version of BS-1770. It actually says "Users should utilize the current version of BS-1770 for measurements". This has resulted in some confusion as to whether gated measurement techniques should be used - but those in the broadcasting leading the efforts in this field are very clear that gated measurements as introduced in BS1770-2 should be used. I have heard that an update to ATSC A/85 is likely to be published shortly and expect this topic will be clarified.

Oh - and just to add to the fun (and confusion ?) - ITU-R BS.1770-3 was just published

http://www.itu.int/dms_pubrec/itu-r/rec/bs...;!PDF-E.pdf

[Goran Tomas]

ATSC A/85 does not mandate any specific version of BS-1770. It actually says "Users should utilize the current version of BS-1770 for measurements". This has resulted in some confusion as to whether gated measurement techniques should be used - but those in the broadcasting leading the efforts in this field are very clear that gated measurements as introduced in BS1770-2 should be used. I have heard that an update to ATSC A/85 is likely to be published shortly and expect this topic will be clarified.

Indeed, the A/85 is unclear which BS.1770 standard it refers to and this has generated some confusion in the broadcast industry. However, my experience is opposite to yours. From what I've seen, broadcasters and manufacturers have concluded that A/85 refers to the original BS.1770 because that's what existed when the A/85 was published, and not BS.1770-2. But you could easily argue both cases, which is why I hope that the ATSC will revise A/85 document to make this explicit.

My comments were made to suggest an improvement to the labeling in the tool provided (which I think is great and will be very useful) and make those labels more technically correct. If you Google R128-1 or R128-2, these don't exist. This adds to the confusion and in my opinion, can unnecessarily make this tool look less professional. And that would be a shame as, like I said, this could be very, very useful to a lot of people - consumers and professionals alike.


Regards,
Goran Tomas

This post has been edited by Goran Tomas: Sep 25 2012, 11:27