Okay, so I've been encoding mp3's using Lame 3.96.1 --preset fast medium (a.k.a. -V4 --vbr-new) and recently, accidentally noticed a song which there was obvious artifacting. So I cut out the wav clip of the offending portion, encoded it using a variety of LAME settings and versions, and worked on ABX-ing. First, the sample is here. It's in wav format (not compressed), but it's very short, so only 270 kb, so it's a quick download anyway.

Here's the interesting thing: the sample is a big problem for 3.96.1 -V4 --vbr-new, but not for -V4 (which defaults to --vbr-old). 3.96.1 -V2, but old and new vbr, seem okay to my ear.
Moreover, Lame 3.97a7 has big problems with this sample on -V4, at both the old and new vbr settings. Again, it seems fine to my ear on -V2 (preset standard) at both old and new vbr settings.
Note that I was ABX-ing, and these encodes were extremely easy to ABX; I could run through very quickly and rack of 15/15, and was extremely confident in my selections.
Lame 3.90.3 --alt-preset fast medium
Lame 3.90.3 --alt-preset medium
Lame 3.96.1 -V4 --vbr-new
Lame 3.97a7 -V4
Lame 3.97a7 -V4 --vbr-new

I could not ABX these encodes with any regularity:
Any --(alt-)preset standard/V2 encodes, whether "fast" (vbr-new) or not
Lame 3.96.1 -V4 (so, default to --vbr-old, a.k.a. --preset medium)

I find both of these facts interesting, independently and together
a) 3.96.1 preset medium has trouble with this sample only on --vbr-new
b) 3.97a7 preset medium has trouble with this sample on --vbr-new and --vbr-old

[edit: fixed link misspelling]

Interesting Tim, I wonder if you could abx it with 3.97_a7 at -V3. The reason I ask is that I've noticed that -V3 on the new 3.97 alpha actually gives me about the same bit rate as -V4 does with 3.96.1

Yeah, I haven't tried -V3 at all, partly because I've done all my recent encoding wiht --preset standard or medium. The main thing is, this is the first time I've run into an obvious artifact with --preset medium, and it's only when using the fast/new vbr algorithm on 3.96.1. Just after I'd been convinced enough to use the fast/new vbr algorithm (b/c it is a lot faster, which is nice). So I've switched back to just -V4 (preset medium) with the default on the old vbr algorithm.

I can confirm that the problem is quite noticeable at 3.96.1 V4 fast (ABX 8/8). V3 fast is also ABXable for me (ABX 8/8) but it's barely there. I couldn't ABX it at V4 with vbr-old.

EDIT: LAME 3.97 V3 fast is also ABXable (10/11). I'd say it's more or less the same as 3.96 fast V3 in terms of quality.

Well, it's good to have someone else's ears getting the same results here as mine - specifically that 3.96.1 -V (vbr-old, by default) isn't ABX-able.

It makes sense to me why the sample is tough to encode - there's that ringing tone going on at the high end at the same time as those drums in the 2nd half of the sample.
One interesting note is that an encode of the song I have in "cheap" (unknown origin - not LAME) CBR 160 kbps mp3 doesn't have this smearing problem on the drums. But that may be partly due to that encoder's obvious sacrificing of high-end clarity, so it's still ABX-able easily enough, it just doesn't have the noticeable artifacting.

edit: add new results. The obvious smearing artifact shows up in Lame 3.90.3 --preset cbr 160, and in 3.96.1 -V5 (both --vbr-new and old). So it appears -V4 is on the cusp of where this artifact is obviously noticeable, and --vbr-new is noticeably worse than --vbr-old.

Is this problem common enough to avoid fast-medium and stick with medium?