I don't want to break up the file as they are. I've ripped them into single files for each CD, not only for listening purposes, but also for backup purposes. And I don't want to keep redundant copies of the albums.
So personally I don't prefer single file for an album at all, the reason I use single files is because they represent the medium. For me this is how it makes sense.
Actually the really weird thing here is that there's almost no file-per-track paradigm with foobar2000 anymore, since the CUE_TRACKnn_* fields were introduced! BUT for some reason this wasn't expanded on the RG values!
The funny thing is that you can give some tracks of an image with embedded cue file a differing "album title" without a problem, because the differing album title is stored and then read from the CUE_TRACKnn_* tag fields. Actually I find the CUE_TRACKnn_ fields really nifty! One of the best new features of v0.9!
My proposal is to move the RG value to the CUE_TRACKnn_ tag fields and this issue will be resolved. Yes, it is an issue. Different records on the same CD, they really exist. And different recordings need different album RG.
As an alternative, in case RG value must stay in the cue sheets, it would be possible to move the REM REPLAYGAIN_ALBUM comments from the header to the corresponding TRACK sections. But as I said I don't like them there anyway, because I don't understand why they are still in the cue sheet.
Why are they still in the cue sheet? Because of Burrrrn? Because of some "official" specs that say "album stuff must be in the header! track stuff must come after TRACK!"? Come on, that's impracticable! Just like the whole cue sheets are impracticable for storing meta data! Maybe there is another valid reason, but if no-one can explain that reason to me, I can't blame myself for not liking it.

PS: I wouldn't have addressed this if there was another way. I appreciate your suggestion, especially because it is something I haven't thought about. But for me it is not a solution, but a subpar workaround that doesn't resolve the issue here.