I understand that some encoder implementations actually zeroed the mdct coefficients after a certain frequency as done in MP3 encoders

I was wondering if that is a good idea in AAC since the TNS tool is supposed to extract a band unlimited periodic waveform in the mdct domain?

What is the consequences of this action in AAC?

Aren't you allowed to mark the end band for TNS?

Even with mp3, a lowpass should increase smearing.

Yes that is true, but I don't think it is a good idea or is there an error with the specification?

The temporal envelope in the mdct domain is a periodic waveform extending from 0 coefficient to the end.

I think frequency cut-off of transients is a bad idea.

I don't understand your problem. Yes, it's a periodic waveform, but you can tell the TNS filter to stop at the point where you start zeroing coefficients.

Where's the problem?

It's not necessarily periodic -- but correlated. However, you're on to something there. TNS should go a bit beyond the last sample that hasn't been quantized to zero (assuming a low-to-high processing direction). After inverse TNS filtering in the decoder the MDCT coefficients beyond the cut-off arn't zero anymore but this is a good thing since it preserves the temporal shape even in this area.

The encoder should do its "lowpassing" (zeroing of upper coeffs) after the TNS analysis filter instead of before. This also preserves the temporal shape in the cut-off region.

Of course it is ALWAYS periodic - otherwise how it is possible that the temporal shape is predictable?
A complex transient is just a series of shifted time samples of impulses and since the MDCT transformed of a single impulse is a sine way, then for a complex transient, its temporal shape in the MDCT domain is just the sum of all these individual sine waves, which will result in another periodic wave.



The problem is that it might produce some distortions / artifacts in the reconstructed transient. You will have to verify this issue with a simple Matlab simulation. (I don't have my laptop with me!)

periodic => correlated (implication, assuming you have multiple cycles in your data!)
correlated <=> predictable via linear filters <=> non-flat energy distribution in the dual domain (equivalence)
Predictability only implies a correlated signal. It's not necessarily periodic.
Consider shaped noise. It can be predicted but is not periodic.

Example: generate a signal via
x[n] = 0.9*x[n-1] + random(); // random() returns iid samples
Given the past samples x[n] can be predicted by 0.9*x[n-1].
No periodicity.

Now, when you first said "periodic" it sounded like you were thinking about sections of MDCT coefficients which show multiple cycles of a periodic signal. The above statement of yours also suggests you believe this.

This is not the case in general. In general (even for 99,99% of all transients) the length of the "cycle" is 4N (N=count of MDCT samples you calcualted) which is the smallest common multiple of the sines' cycle lengths. So, the bunch of MDCT samples is just the quarter of "a periodic signal" => You can't predict anything unless the signal is also correlated because you won't encouter multiple cycles of a periodic signal within your MDCT samples.

Well, there seemed to be a "limit" to the order of the LP filter. Somehow larger period sine waves aren't filtered out by the LP filter, that is if they are there at all! Somehow I believed the window shape must have played a role in rejecting some of these large period sine waves.