New Standard: USAC, sussesor of (HE)-AAC Options

[IgorC]

It's a new standard (Unified Speech and Audio Coding – USAC, aka MPEG-D part 3).
It has reached a status of Final Draft International Standard.

Basic information http://en.wikipedia.org/wiki/Unified_Speech_and_Audio_Coding
There are some papers about new compression schemes in internet.

There was a verification test where USAC was compared to HE-AAC v1/v2 on range of bitrates 8-96 kbps. http://mpeg.chiariglione.org/meetings/tori...orino_press.htm
The report should be available in September – October.

Some anonymous source has indicated that USAC was clearly superior to HE-AAC and it did very good on speech though something like "USAC at 48 kpbs = HE-AAC at 64 kbps" shouldn't be expected. It's more like 20-25% of bitrate gain at 20-32 kbps and 10-12% at 64 kbps. That's still pretty good.

There is no information if USAC will be used at higher bitrates as ( LC-AAC's area). The core coder of USAC includes a new enhanced SBR (eSBR).
Both SBR and new eSBR have no advantage over LC-AAC at bitrates higher than 80 kbps. It’s unclear if eSBR can be disabled at higher bitrates.


It will be interesting to hear opinions and if somebody knows something more about it.

[IgorC]

USAC verification test report

[C.R.Helmrich]

Thanks for posting that link, Igor! I've also been waiting for the publication of the report, namely Figures 3 and 6.

To answer your question: (e)SBR - just like any other coding tool in USAC, I think - can be turned off. In fact, we did so when testing the new stereo tool intended for "bitrates where SBR is typically not used" (the report of that experiment is available here).

By the way, in case anyone is wondering: I think (hope) that the final version of the standard will also specify native support for gapless playback and ReplayGain/R128.

And one more note: the name USAC will probably change.

Chris

[FreaqyFrequency]

Cool stuff! Always exciting to see the prospects of where better coding will be able to take us, especially low-bitrate streaming. If we are able to achieve perceptual transparency in bitrates under 50kbps within the next five years, that would be pretty amazing.

[IgorC]

Thanks for posting that link, Igor! I've also been waiting for the publication of the report, namely Figures 3 and 6.

To answer your question: (e)SBR - just like any other coding tool in USAC, I think - can be turned off. In fact, we did so when testing the new stereo tool intended for "bitrates where SBR is typically not used" (the report of that experiment is available here).

And one more note: the name USAC will probably change.

Chris

Thank You too for paper, Chris. Though I've already read all related papers from that link and other as well.

[polemon]

For me, as a Linux user, developer of embedded systems and free software advocate, I look at it with a certain skepticism.

While the technology is exciting and interesting, it will leave out the users from becoming developers and/or contributors. USAC might very well linger around like AAC does for the most part. It is out of question, that USAC will be heavily patented. There are some encoders out there, that produce great results when encoding AAC, but most of them aren't free. As I understand it, due to patents, it is quite dificult to use AAC in embedded devices. That's one of the reasons, why most car navigation systems use Ogg/Speex for speech, etc.
I recon, apart from some niche applications, USAC will remain obscured, just like AAC for the average user. As for developers, they will turn to whatever is best, free and available. Once reason more to use Xiph things, as many game developers do as well.

[FreaqyFrequency]

@polemon, you're familiar with Opus, I assume? Without derailing the thread, I'd be curious to see USAC and Opus put to the test when the bitstreams freeze, and the encoders become available. The licensing aspect of USAC (and AAC, and MP3 before it) is of course always an issue for commercial usage, but dealing purely with quality/transparency for equivalent kbps seems to be a more fair way to judge their respective value, for consumers at least.

[polemon]

Well, I don't want to be an evangelist of any kind, but I believe, that sooner or later all kinds of multimedia material will be detached from the physical media. Once that is the case, it only makes sense to use the best encoders available. Right now, this is arguably using AVC/AAC. If I want to make a video, I would use the best encoder available to me, but the situation right now, leaves me with using sub-par AAC encoders on Linux. I could work-around those problems, with WINE or a virtualized Windows, but it's this extra work that let's me sigh whenever I need to do it. Which is kinda pitiful.

I'm just saying, that even though this should be all exciting and fun, it is quite a lot of extra work to be done, for an otherwise trivial task. I use NeroAAC for AAC, and x264 for video. and then muxing the two together. I can't just use a tool like ffmpeg to do all that in one sweep. That's a big boo, don't you think? When I want to use qtaacenc, I need to make it under Windows (using WINE for it sucks). Using a virtual machine with a shared folder helps a lot, but still, it's quite a complicated setup for just making a video.

[smok3]

polemon, interesting, but if you wanna do it "right", its at least 2pass anyway:

1. get some replaygain or r128 scanner listen to the audio track (this could be ffmpeg piping to analyzer), store rg data variable
2. encode video and audio (with rg correction)

unless i'am wrong

This post has been edited by smok3: Oct 13 2011, 19:39

[C.R.Helmrich]

I recon, apart from some niche applications, USAC will remain obscured, just like AAC for the average user.

Well, (HE-)AAC is used in DAB+, DRM, DMB, Youtube, iTunes Store, H.264 video from mobile devices, and some more. Whether these are niche applications is debatable, I'd say.

Think of it this way: years or R&D and over a million Euros/Dollars went into the standardization of AAC and now USAC. I wouldn't understand why one should give away the use of such highly specialized, sophisticated technology for free.

sooner or later all kinds of multimedia material will be detached from the physical media. Once that is the case, it only makes sense to use the best encoders available.

Both absolutely true. But the media creators (music distributors, digital camera manufacturers, etc.) already nowadays license the AAC encoders, so it seems they are willing to pay for the technology. Of course it's a different thing if you create your own media content, e.g. edit home videos or your band's music. Isn't there any commercial software like Adobe Premiere for Linux with in-the-box state-of-the art encoders?

Chris

This post has been edited by C.R.Helmrich: Oct 13 2011, 20:42

[hlloyge]

Problem with Linux isn't finding encoders for it, it's the expected price and the fact that, for that price, there is no such complete toolbox available as is Premiere for Windows.
Personally, I don't use speech codecs, but I welcome any improvement in that field. Because knowledge could be used in building new, more efficient general purpose encoders.

[C.R.Helmrich]

Well, USAC is just that: a general purpose codec. Quality improvements were made not only w.r.t. speech but also high-frequency reconstruction, stereo, overall entropy, and parametric transform coding.

Chris

This post has been edited by C.R.Helmrich: Oct 13 2011, 21:27

[IgorC]

Nice,

It will be great to see an implementation of it any time soon. A good implementation helps a lot for fast adoption of standard. As example, x264 plays important role in H.264 standard.

[polemon]

Well, (HE-)AAC is used in DAB+, DRM, DMB, Youtube, iTunes Store, H.264 video from mobile devices, and some more. Whether these are niche applications is debatable, I'd say.

As a product alone, those kind of media is available, sure. But once I want to manage, convert, etc my own media library, things get difficult. What I mean is: I can't use the codecs as a tool for my own purposes.

Think of it this way: years or R&D and over a million Euros/Dollars went into the standardization of AAC and now USAC. I wouldn't understand why one should give away the use of such highly specialized, sophisticated technology for free.

Debating on this issue will get us nowhere. This subject is quite vast, many books have been written about it. One could bring up Linux etc, with companies giving away their operating systems for free. But as I said, we should not immerse in that kind of discussion as it will lead to flame war.

sooner or later all kinds of multimedia material will be detached from the physical media. Once that is the case, it only makes sense to use the best encoders available.

Both absolutely true. But the media creators (music distributors, digital camera manufacturers, etc.) already nowadays license the AAC encoders, so it seems they are willing to pay for the technology. Of course it's a different thing if you create your own media content, e.g. edit home videos or your band's music. Isn't there any commercial software like Adobe Premiere for Linux with in-the-box state-of-the art encoders?

Like I said in my first paragraph, as a product, that codec is available, but not as a tool. There is quite sophisticated encoders available for Linux. Use of open standard codecs is encouraged, but things like x264 (AVC) are available as well. It is no problem to create those kind of media completely free and with state of the art codecs. The only thing that bugs one, is that not all codecs are freely available. Sometimes, it is even dangerous to use them, in case of license violations.

[IgorC]

I'd be curious to see USAC and Opus put to the test when the bitstreams freeze, and the encoders become available.

While it's possible to compare any codecs still it should be mention that USAC and Opus target for different applications. USAC will have high delay and Opus is low delay codec.
It's a possibility that later there will be low delay fork of USAC as AAC-LD (low delay LC-AAC), AAC-ELD (low delay HE-AAC v1), AAC-ELD v2 (low delay HE-AAC + a new parametric stereo from MPEG Surround). The cost of low delay comes with a loss of coding efficiency (1.3-1.5x higher bitrate for the same quality).
For example AAC-ELD 32 kbps (delay 33.3 ms) has an equivalent quality as HE-AAC 24 kbps. (1.33x higher bitrate). Furthermore if the delay of ~20-25 ms is required then unfortunately there is nothing can be done but to disable low delay SBR.

This post has been edited by IgorC: Oct 13 2011, 22:39

[Garf]

Think of it this way: years or R&D and over a million Euros/Dollars went into the standardization of AAC and now USAC. I wouldn't understand why one should give away the use of such highly specialized, sophisticated technology for free.

I like your precise choice of words here The standardization process tends to be involved with as many political as technical considerations, and due to its nature, a huge drain on resources for all involved. Sunk costs are never a correct consideration for what to do in the future, so I feel this point is moot.

The adoption of a technology like USAC depends mostly on what complementary services the companies with an interest in it can offer. AAC didn't get adapted much until iTunes came along for the simple reason that the technical merits it has are not enough of a pressing advantage by itself. I believe that's also why in some of its core target markets (broadcasting), adoption is still very low.

So, as unfortunate as that might be for technical people like you and me, USAC's popularity will have almost nothing to do with how much R&D and especially money you and your competitors-collegues poured in until now. It will depend entirely on whether someone can create a service in demand and feels that USAC is the best fit codec to achieve his goals. And USAC's merits might also be political ones for that decision.

PS. What happened to MPEG Spatial Audio?

[C.R.Helmrich]

The only thing that bugs one, is that not all codecs are freely available. Sometimes, it is even dangerous to use them, in case of license violations.

Don't worry, I'm sure nobody participating in this thread has any intention of starting a flame-war. I'm just trying to understand your point. So, let's assume your Linux distributor of choice licenses a HE-AAC (or USAC) decoder library (i.e. pays for it), and bundles it with Linux so that every software can access that decoder to play encoded files. Would that solve the problem?

I like your precise choice of words here ...

Well, yes, I could also live without all the MPEG meetings and politics. But I was trying to focus on the non-political aspects. Already at Fraunhofer a dozen colleagues or so experimented, implemented, and tested for USAC full-time since the project was initiated a few years back. And of course there is still work to be done, especially tuning the encoder, creating conformance data, and marketing

Chris

P.S.: If by MPEG Spatial Audio, you mean MPEG Surround and SAOC: They are slowly appearing, e.g. duringWimbledon.

[smok3]

your Linux distributor of choice licenses a HE-AAC (or USAC) decoder library (i.e. pays for it), and bundles it with Linux so that every software can access that decoder to play encoded files. Would that solve the problem?

there are serious issues with that:

- the library as binary or sources?
- to pay for a library it would not be in real OS spirit (although i imagine that certain mainstream distros would do it)

the clash is mostly due to the fact that there is real world outside, so basically linux users would want to use their other hardware (ipods?) to play the same files...., otherwise webm would most likely be already adopted (i think google still bundels chrome with h.264 decoder...).

p.s. so the idea is to make "use open media" sentence romantic.

This post has been edited by smok3: Oct 14 2011, 20:55

[Garf]

Don't worry, I'm sure nobody participating in this thread has any intention of starting a flame-war. I'm just trying to understand your point. So, let's assume your Linux distributor of choice licenses a HE-AAC (or USAC) decoder library (i.e. pays for it), and bundles it with Linux so that every software can access that decoder to play encoded files. Would that solve the problem?

The distributor would almost certainly be guilty of multiple GPL violations at that point. Your proposal is not possible in practice, and that is very much by design.

[Garf]

- to pay for a library it would not be in real OS spirit (although i imagine that certain mainstream distros would do it)

Paying for open source/free software is perfectly fine, often done, expressly allowed by the licenses and encouraged by the FSF, so it most certainly is not against the "real OS spirit", whatever you claim that to be.

The problem is that a patent license does not carry over its rights to a third party, and *that* will run afoul of the GPL.

the clash is mostly due to the fact that there is real world outside, so basically linux users would want to use their other hardware (ipods?) to play the same files...., otherwise webm would most likely be already adopted (i think google still bundels chrome with h.264 decoder...).

Yeah, Google is a bunch of hypocrites in that area. Note that the open-source version of the browser (that nobody actually uses) doesn't include H.264 for obvious reasons.

[smok3]

i meant paying (or just allowing closed binaries) to appear in distro, or binaries with suspicious licence/problematic patent issues.

(afaik chromium is actually used a lot in mainstream distros, certainly the first thing i do after new install is get chromium/remove others)

This post has been edited by smok3: Oct 15 2011, 20:01

[ksuman]

Nice,

It will be great to see an implementation of it any time soon. A good implementation helps a lot for fast adoption of standard. As example, x264 plays important role in H.264 standard.

Are there any floating / fixed - point reference or commercial implementations for USAC? If so, can anyone point me the links?

[IgorC]

Reference software
http://www.itscj.ipsj.or.jp/sc29/open/29view/29n12756b.txt
http://www.itscj.ipsj.or.jp/sc29/open/29view/29n12756att.zip

USAC verification test report

Now the encoded files are also available publicly.
Here are some pre-decoded files.
Original http://uploading.com/files/4d54acef/origin...eo_concat.flac/
HE-AAC 64 kbps http://uploading.com/files/3ad6m31c/HEAAC_...s_align_3.flac/
USAC 64 kbps http://uploading.com/files/d85ea535/stereo...421_64s_1.flac/

This post has been edited by IgorC: Jun 3 2012, 18:19

[Speckmade]

If we are able to achieve perceptual transparency in bitrates under 50kbps within the next five years, that would be pretty amazing.

Given that I think I remember figures of about 64 kbps for the amount of data that the brain receives from your ears that would truely be something.
On top of that you can use those new general purpose data compression tools, that can compress input regardless of content by ~30 % - and because their effect is given independent from the contents you can recursively feed their output back into them and compress every piece of audio down to 1 bit. - Amazeing!

[Canar]

On top of that you can use those new general purpose data compression tools, that can compress input regardless of content by ~30 % - and because their effect is given independent from the contents you can recursively feed their output back into them and compress every piece of audio down to 1 bit. - Amazeing!

You're funny.