I'm experimenting with wavelet filterbanks and audio compression.
Maybe someone can help to enlight me a bit, because i am no genius when it comes
to mathematics.

How long should be the filter-length? i experimented with short integer-wavelets
but got horrible results if i don't normalize the transform in the quantization step.
Also longer-filters seam to have a better time/frequency resolution.

Do i have to overlap the transform to avoid block-boundary-artifacts?
In my current version i only do a symmetric extension if neccessary.

is it possible to direct apply the calculated maskers in the wavelet-domain?
in my experiments that didn't seem to be the case, because i need about 300kbit
to get acceptable quality.

has someone experimented with adaptive filterbanks?

any help would be great

pest

I can tell you the result of your experiments already: it's going to suck. Now, didn't I just save you a lot of time?



It depends on what kind of filters you are using and what kind of thing you want out of them.



Yes if the filterbank is time-varying, no if it is not.



How are you calculating the maskers? What is the nature of the wavelets?



300kbits what? mono/stereo? What kind of quantization? Using lossless entropy coding?



Loads

Thanks for your answers,



i have enough of it



you're right. i just thought there is something like a critical length to get
acceptable results because i read often about lengths around 20. i could
make the length adaptive to match the signal characteristics.



time-varying means different framesizes?




i'm calculating the maskers with a simple mpeg-1 psychoaccoutic model.
the wavelets are biorthogonal and short (9 taps)




Stereo streams. i am using linear quantization with a deadzone and
a binary bitplane coder which kicks huffmans ass.
perhaps i should switch to something nonlinear
because the results are really ok for samples with low applitude.




is there a simple way to determine a good basis based on psychoaccoustics?
i'm using an energy criterion but that seems to favor the wrong bands.

thanks, you're great

You can safely assume that there's no agreed-upon way to apply wavelets to audio coding. I mean, if all results suck, it's rather hard to talk about things as "critical length" and "acceptable results".



I was thinking about adaptive filterbanks, which seems to be what everybody wants to do with wavelets.



Take a look at the frequency responses of the MPEG-1 model's FFT and that of your wavelets.



Binary bitplane coder? Kick Huffman's ass? You are probably seriously misunderstanding something in that area, but without any details it's impossible to see what.



Maybe. Maybe you're missing some kind of scalefactor or other normalization structure?



In my opinion this is the optimal one: you always pick a DCT basis, never the wavelet one

If you want more comments it's probably a good idea to give a complete overview of what you're doing. Otherwhise any reponse is just shooting in the dark, really.

really? do you know why there's so much attention for wavelets in audio coding?



it's different - back to school



the entropy-coder scans every bitplane. the probability of the current bit being '1' is measured
with an adaptive context-model based on the neighbourhood and
encoded with a range-coder. it's slow, but it works really good.

the coder is really simple and i don't think that i've completly missed something.
short description:
the input is splitted in frames of 2048 samples.
the psychoaccoustic model calculates the maskers.
i do the wavelet-transform with a basis (17 subbands) that resembles some of the critical bands.
then i quantizise the bands based on the maskers and encode the bands with the bitplane-coder.

Great that you've helped me. i'm looking deeper into the current lack of correct scaling.

[edit]
number of subbands added

People like buzzwords? Ignorance? No baseline to compare against so you can get away with publishing basically anything?

I don't know of any single good reason, though.



It doesn't have to be exactly the same, but you probably saw the problem already?



Ah, so it's really more like arithmetic coding/CABAC



Why would this ever perform better or even comparable to a DCT based coder?

That's an important point.

The thing is -- like JJ mentioned in one of his presentations (I lost the link, but Garf may have it somewhere) -- a transform should do 2 things:
- allow accurate quantization noise distribution according to psychoacoustics
- give an energy compact representation of the audio signal (decorrelation)

The bad news: These can't usually be met simultaneously. Example: Sinusoids are compactly represented via MDCT but the MDCT gives a pretty low temporal resolution (too low for upper frequencies) which might result in pre-/post echo.

While the wavelet approach may result in a filterbank which kind of resembles critical bands (this would help distribute quantization noise the way the psychoacoustic model suggests) it really sucks in decorrelating sinusodial signal parts and thus requires a high bandwidth in those situations.

Switching between MDCT and and Wavelet packet transform is possible but IMHO really not worth the effort compared to blocksize-switched MDCT+TNS.


Sebastian

I try to compensate the lack on sinusoids with a more sophisticated entropy-coding
and i like the possibilities you can do with wavelets. there's no final goal, but after
implementing a simple wavelet-video-codec i'm trying this one. you can always learn something




i'm currently experimenting with linear-predictive coding on the subband data. one thing which
bothers me, is that in most papers they suggest using subband-prediction prior quantization
but i fear my lms-filter could get unstable if the quantization-noise is too high. is there any
disadvantage in using it after quantization?

http://mue.music.miami.edu/AES/PerceptualCoding.ppt Might be what you were looking for. If it's not, you could check http://mue.music.miami.edu/AES/ where there is a bunch of stuff jj did at Miami in two days running.

I don't know to much about multi-rate signal processing, but are there any wavelets that have good decorrelation properties? Also, does the KLT have good decorrelation properties? without going off topic just out of curiousity I have seen it used elsewhere and I am curious about that?

Sure, but that doesn't have anything to do with the transform itself...




It's optimal by definition.

Well, you have to realize that good decorrelation properties are, in general, analogous to having sharp, narrow frequency response.

The KLT is defined as optimum for the distribution it was designed for, of course.

I see interesting. Thank you for clarifying that. I had seen the transform used in a research implementation of an MPEG-2 AAC encoder and it's the only other transform, I have ever seen used aside from DCT-IV for audio coding at least.

Really? A KLT was used where in the MPEG_2 AAC research? I am very curious, if you could expound a bit more I'd be gratefl.

I forget the authors name the paper comes from UCLA Electrical Engineering department. I think the authors first name is Chris Kayakis. I have it looming around my HD somewhere. Let me take a look and if I find it I will upload it for you. The authors were using KLT transform and then comparing the difference between using scalar quantization entropy scheme to VQ one as backend. This was a multichannel coder I should add.

Ahhh, you're talking about Chris Kyriakis (sp?), then, was it perhaps USC, and that the KLT was used to attempt to diagonalize the multichannel signal??

Perchance, could you tell me how the reflected the perceptual thresholds back to the original channel signals?

You had me there for a minute, I rather thought you meant using a KLT as a transform filterbank. Under such conditions, one wonders how one would transmit the basis vectors, eh?

Yes, it was the KLT and it was a diagonal matrix. I was confused I haven't read it in a long time and I thought they were using the KLT as a transform filterbank. I think that's were the confusion set in. Again, I don't understand multi-rate signal processing that well just the regular stuff. I was just questioning it's decorrelation properties, which according to this algorithmic implementation appear to be quite good and provide better results. A SMR increase of 2.2 dB is convincing, even though it's a rather small improvement for individual scale factor bands. It was a rather clever idea. Scrap the VQ thing that was a different paper.

http://viola.usc.edu/newextra/Publication/...E-TSAP_Yang.pdf

ahah! here it is. I am interested in seeing the results from this adaptive filterbank the author is experimenting with.

Has there been research into the audio modeling properties of curvelets, bandlets, ridgelets etc? Other than terrifically slow, which I've found quite enough in image coding.

The problem here in lies that multi-resolution anaylsis to an extent is really not suited for stationary signals. It can give quite remarkable results in terms of image coding. As was explained in this thread, there is on going research though in using wavelets in adaptive filterbanks for audio compression and there have been an number Research papers written based upon it. The results are half and half.

Yep, that was for doing interchannel diagonalization, not for the main MDCT filterbank. It is possible to design a KLT like thing for the MDCT. You do a great lot of work, wind up with an n^2 instead of n log n complexity, and get just about zip, at the cost of a great headache and the need to send the basis vectors every once in a while to rather a lot of resolution.



Well, if you were to go to Wm Yost's book on the phsysiology of the ear, Brian Moore's book on the psychology of hearing, and such, you could derive answers from the measurements into the actual filterbank structure of the ear, I'll bet.



I'd say zero for 'n' myself. Deepen Sinha wrote the first fairly complete one I've heard of at the U of Minnesota, working for Ahmed Tewfik.

It was great on castinettes. It was horrible on stationary signals.

Johnston and Brandenburg started in that direction, did this thing called something like the "hybrid coder", took a fast look at the results, and ran like the wind.

I know how important complexity is, so I can sympathize with you their.




I am not an Electrical Engineer, but I still think it would be possible to design a filterbank that would give optimal results. I could be wrong about this, but I mean is that not the whole point of doing research and experimenting with different strategies? that's much two cents.

Well, consider. In order to have the requisite frequency resolution to have good gain, you must have frequency response similar to each bin of an MDCT.

That means that your first filter in your filter tree has to be the same length as the whole MDCT.
The second one has to be half the length, at half the sample rate.
The third one has to be 1/4 that length, at 1/4 the sample rate.

And so on.

Now, what kind of total impulse response length are you going to wind up with? What sort of pre-echo problems?

But what are "optimal results?"

<tounge in cheek>
I believe you are mistaken. The paper clearly reports "near transparent" coding at 55-63kbps mono (or 110-128kbps stereo without any channel coupling). Clearly, that's an excellent result.
</tongue in cheeck>

I have some other paper from Deepen Shinha, which was written together with J. Johnston, also about a hybrid scheme. The idea looks plausible, and they reported improvement over PAC, which was not bad in itself. I wonder what became of that. If I'd have to guess, they discovered that the impulselike signals were almost never quite so isolated, and their wavelet filterbank was losing out for most mixed cases exactly because the energy compaction properties of the wavelet filterbank was worse than that of the DCT. Or maybe it was just too slow. I'm just wagering a wild guess here, anyway.

Well, perhaps it could be that while the wavelet decomposition is very good for transients, and the MDCT very good for stationary signals, that the switching between the two is hideously inefficient in terms of both perception and rate gain, and makes the switching so painful as to be nearly useless.

I believe John Princen published a paper on how to deal with mixed cases, but it suffered the same kind of problems in switching, and proved not to reduce the rate much if implimented in n^2 complexity anyhow.

Hmm, interesting stuff. Has there been research into finding and subtracting transients out of the waveform and handing them to a wavelet sidechannel to compress? Or just piping the residue (however that's convenient to define) of an mdct coding into a wavelet engine? Rather than trying to switch off between the two for each window. You'd have to dampen it so that only areas that had high enough residual energy were passed on, so that the coding would be more efficient and as importantly, encoding and decoding much more performant in unaffected windows.

The braindead way would just be a to code an ogg q0 or equiv aac and pass the raw residue passed through a kind of noise gate, but that still wastes some bits on the ogg side on what would be better modeled by the wavelet.

Maybe that just exposes my noobishness around audio coding.

The MDCT is a transform. So I really wonder what kind of "residue" definition one could give.




Raw residue? Noise gate?

Non comprende.

It's not as simple as it looks. THE MDCT is a mathmatical transform. It takes a series of coeffcients or samples, in the time domain and transform them into the frequency doman. The DCT-IV is mostly always used in audio coding, because of block boundary problems that would concur and the way the data is recontructed via a synthesis filterbank (decode process). It satifies certain mathmatical condition, which make it optimal. What you are referring to is the entropy coding scheme here, the backend stuff. The only reason it looks promising is, because you would never have to worry about any sort of overshoots via Gibbs Phenomenon. That's were pre-echo problems tend to arise, but there are other ways of dealing with it that are considered optimal.





The problem here is that, Vorbis is designed so that it can support a hybrid filterbank, but as was discussed in this thread. There have been several research papers written, which attempt to implement an adaptive filterbank, which is similiar to a hybrid filterbank that can model different parts of a signal more effciently. Anyway to cut to the chase... the results of using adaptive filterbanks with wavelets are half and half. SebastianG at best claims that it can be done, but it's not worth the headache. It would be very similiar to MDCT + TNS (AAC). The only difference would be that it would have to be a wavelet packet transform of some sort. That's similiar to what the original author of this thread is trying to do.




I think what he is trying to imply is deploying a entropy coding scheme to transmit the wavelet coeffcients, in much the same way Vorbis handles residue data via a VQ book. In essence having some sort of synthesis filterbank during decoding step, which is able to reconstruct the data.

Ah, I didn't realize that the adaptive and hybrid were so similar, thanks.

Garf, I meant the residue after the quantization step, somehow run through IDCT and piped into a wavelet codec. But I guess techniques of this sort have already been largely discarded for more efficient and performant techniques. TNS and other advances really seem to make a big difference for AAC.

The terminology is annoying and confusing, but basically an adaptive filterbank is a hybrid filterbank or quite similiar to one. That's just a technical way of saying it



Right. In this case there is no sense in going to the extra trouble. It's easier to use a tool like TNS, which is optimal and can give considerable performance gains. That's what the argument here is and some folks disagree.

"Huh?" I really don't understand a word of what you're trying to say.



I don't believe this at all. I don't think you'd be able to get a compaction as good as the DCT in the steady signal frequencies, and even getting close would require a massive performance penalty over the plain DCT.

It should be worse, not similar. That's why I keep saying the results of the wavelets are going to suck.

This is if we're talking about the transform part. As for operations on the DCT coeffcients themselves, I don't have such a strong opinion; I haven't read much work on that nor particularly done research in that area. So any references would be interesting.



TNS is optimal? Any proof/paper/reference for such a very bold claim?

I'm not disputing that it's very good, but optimal, boy, that is something different entirely.

ok... so maybe I don't understand what the word "optimal" means . Optimal as in the filterbank has good performance or optimal as in it satistfies certain mathmatical conditions? and to think I was considering Technical Writing as career.




I am sure that's the case, but what the hell I mean it's an experiment right?. I meant similiaries between what Sebastian stated and what the author of the thread is trying to do, something along those lines. If they are similar at all




I will have to look into that.

Something which is optimal cannot be improved.

The KLT is optimal in the sense stated earlier.

I do not see any reason to believe TNS is optimal.

Have you checked the amplitude responses of your subbands' synthesis filters? It might be interesting to see how quantization noise gets spread... For example: If you recursivly split the (lower) band via the CDF9/7 filters. One synthesis filter (for the band "689-1378 Hz" at fs=44.1kHz) has the following response:

So, if you add some quantization noise in that band you'll also have the noise's alias across the spectrum and around 4000 Hz a rejection of only 27 dB for example. Admittedly, the performance looks better than I expected. However, I think that the analysis filters' responses look worse (not the same 'cause CFD9/7 aren't orthogonal) which means you'll end up with a lot of aliasing in the subbands. (Yes, it get's cancelled in the synthesis stage but in terms of energy compaction the performance might really suck 'cause one sinusoid not only ends up in one subband but in many (This is always the case but you typically want to reduce that via analysis/synthesis filters with a higher rejection.))

edit: for the sake of completeness here's the amplitude response of the analysis filter for that band:

Obviously the rejection is poorer compared to the synthesis filter.


Yeah ... If you do it prior quantization you don't need the exact same implementation of the filter on the decoder side (in terms of accuracy/precision, rounding errors) if you keep the filters "stable enough" (which is not that hard -- you should use the quantized samples to "train" the filter and add some damping here and there). OTOH, if you do the prediction on the quantized samples as a noiseless encoding step you need to quantize the prediction as well and code the difference to the already quantized sample. Accuracy/Precision is important here. You need to do the exact same thing (preferably only integer arithmetic) in the decoder as you did in the encoder since a very small change of the prediction can cause the quantizer to round to a different integer sample near thresholds. In such a case you're f****d.

The AAC specification contains something similar to improve the coding performance for sinusoids even on the already very narrow MDCT-subbands (backward-adaptive 2nd order prediction) at the cost of higher complexity, of course. You might want to grab the spec to see how it is done.

Sebastian

i already noticed that, and thanks for your detailed explanation. i'm looking deeper into it,
it's just too warm here
i normalize the transform across the bands so that noise in deeper subbands
has the same impact on the whole transform.
do you think the cdf97 filter is suitable for audio?



the adaptive lms filter is lossless (yet it uses floating point) , but why do i have to quantize the
prediction? the filter predicts the quantized subband-data. it's all working as expected.



the lms-filter uses up to 32nd order and it's slow as hell, but the improvement with 2nd-order
was too low, because the entropy-coder catches most of the linearity already

back to the beach
have a nice day

No.


Feeding the adaptive filter with quantized samples is fine. I had these two possibilities in mind though:

in both cases "updatefilter" is called with the distorted (by quantization) samples and "predict" should only work on "quantized_{n-1}, quantized_{n-2}, ...". The difference is: method 1 quantizes before subtracting the prediction and method 2 quantizes after subtraction of the prediction. method 1 requires you to implement the exact same predictor (encoder & decoder) while in method2 small differences in rounding errors are fine.


Oh wow! 32nd order backward adaptive predictor?!
So, you try to patch the bad properties of your filterbank using very CPU intense prediction tools.
Hmmm...

Sebastian

I came across this thesis today.
("Wavelet Filterbanks in Perceptual Audio Coding")

I just skimmed through it and it looks interesting. It covers many types of subband transforms (QMF, generalized QMF aka biorthogonal wavelets, MDCT, ....) as well as psychoacoustic principles (20 pages).
The filterbank section is really huge ... see page 85 for example. it nicely illustrates the f***ed magnitude responses of a full DPWT (iterated DWT on all bands) for a IMHO-not-that-smart choice of filters...

have fun,
Sebastian

Thanks. I am reading through it right now. I just read through the overview on Perceptual Audio Coding.

That's interesting, that author thinks Zelinski and Noll was a perceptual coder. Ditto Crochiere's sub-band coders. Be careful, his history is just wrong there. Hall, Atal and Schroder was the first perceptual attempt, really, and that for speech only. Oddly, as well, there is no mention of AAC.

Thank you for clarifying. (So, I'm assuming Zelinsky, Noll, Crochiere all employed (subband) transform methods to solely exploit redundancies in contrast to actually shape noise spectrally).

There may be no mention of AAC in the "historic overview" part but AAC references are all over the place (for example page 1 as "state-of-the-art perceptual coding", as a scheme employing the MDCT in the section describing filterbank requirements, as a scheme that employs a scalar quantizer in the quantization chapter, as a scheme that employs coefficient grouping and huffman coding for noiseless coding, ...)

I still havn't read it completely but it doesn't look too bad and he came to conclusions that are "compatible" to mine -- and yours I think . I do miss informations, though: What coder did he actually use (w.r.t. side info coding, lossless coding) to "informally" test the four kinds of filters (including his remez-derived ones)? (not that I care much, but it seems to be missing)

Sebastian

It's a little harder than I'd like to reply substantively here.

In any case, Zelinsky and Noll was a rate-distortion codec, not a perceptual codec. They had a hint in that a full rate-distortion floor didn't work as well as they expected, but they didn't take it at the time. Crochiere (Subband coding) was also approaching it from a rate distortion viewpoint, although the 16kb/s codec that got used for the older voice messaging did in fact have a frequency partitioning that wasn't too far off of a start at a perceptual codec. The partitioning for the 9.6kb speech SBC using analog filters and integer band sampling undoubtedly benefited from perceptual effects, but I can confidently say that that wasn't the first intention of the particular partition. Quite some codec that, you've never seen that many second-order analog sections in one place before, I think, and you may wish to never see such again.

The whole problem with wavelets and with tree filterbanks in general is that they must have a longer impulse response for the same frequency resolution than the same multiband SBC (say an MDCT, which is of course a multiband SBC with exact reconstruction), assuming that the tree goes beyond one stage, of course. There is really no way out of that conundrum.

Also, if one wishes to switch between the two, the "conversion" filters are the worst of both worlds. The rate hit from that kills the ability to switch pretty much dead in my experience. You can think what you like of my unspecified experience, just go try it yourself.

I already did and came to the same conclusions
However, I think that the "conversion filters" used in EPAC (you are referring to?) are not the end of the story. Other designs are possible that may have better "edge" properties (concerning time/frequency localization). (yup, I've 3 designs up my sleeve but I'm not sure whether it's worth publishing since the performance compared to AAC-like structures will still be inferior I suppose)

Sebastian