IPB

Welcome Guest ( Log In | Register )

4 Pages V   1 2 3 > »   
Reply to this topicStart new topic
Lossless Extensions for Opus (Backwards Compatible), How to embed lossless deltas inside an Opus stream
wswartzendruber
post Feb 24 2013, 19:58
Post #1





Group: Members
Posts: 85
Joined: 11-December 06
Member No.: 38563



After reading a good chunk of the Opus RFC, I believe it may be possible to use Opus padding to embed lossless deltas for each packet. It states that this padding may be of any size, and that while an Opus encoder itself must set any padding to zero, the decoder must accept any value. It also states that when the decoder has finished reading bytes from a frame for decoding, it may not spill over into the padding for further input. Hrm... Sounds like an arbitrary extension field to me.

My only concern is about just how much padding decoders would be willing to accept before deciding that the packet is a DOS attack.
Go to the top of the page
+Quote Post
NullC
post Feb 24 2013, 20:58
Post #2





Group: Developer
Posts: 200
Joined: 8-July 03
Member No.: 7653



QUOTE (wswartzendruber @ Feb 24 2013, 10:58) *
to embed lossless deltas for each packet.
Deltas are not going to do you any good. The opus specification is quite intentionally not bit exact, and the decoder gives slightly different output on different systems and different versions of the decoder. The design of the format also does not reliably result in compact simple lossless deltas (e.g. the phase shift in hybrid mode, or how HF gets filled with 'folded' noise).

For completeness, the RFC also specifies that the padding MUST be set to all zerosó though certainly some future revision to the RFC could make use of the freedom that comes from also mandating that the decoder ignore the padding.

This post has been edited by NullC: Feb 24 2013, 21:00
Go to the top of the page
+Quote Post
Garf
post Feb 24 2013, 21:09
Post #3


Server Admin


Group: Admin
Posts: 4853
Joined: 24-September 01
Member No.: 13



QUOTE (NullC @ Feb 24 2013, 20:58) *
The opus specification is quite intentionally not bit exact, and the decoder gives slightly different output on different systems and different versions of the decoder.


This is easily solvable: mandate that decoders that want to use this lossless extension use a specified decoder that *is* bit exact with some reference (for example a fixed point implementation) before applying the correction data.

Old decoders will decode not-bit exact and sound fine, and the new ones will decode bit exact and know how to apply the correction info.
Go to the top of the page
+Quote Post
lvqcl
post Feb 24 2013, 21:23
Post #4





Group: Developer
Posts: 3219
Joined: 2-December 07
Member No.: 49183



And how to losslessly encode 44.1 kHz audio?
Go to the top of the page
+Quote Post
Garf
post Feb 24 2013, 21:25
Post #5


Server Admin


Group: Admin
Posts: 4853
Joined: 24-September 01
Member No.: 13



QUOTE (lvqcl @ Feb 24 2013, 21:23) *
And how to losslessly encode 44.1 kHz audio?


Ah, good point. NullC also pointed out to me on IRC this setup will fail while seeking.
Go to the top of the page
+Quote Post
wswartzendruber
post Feb 24 2013, 21:36
Post #6





Group: Members
Posts: 85
Joined: 11-December 06
Member No.: 38563



When I came up with this idea, my primary use case was movies. With that said, I can't say that I really care about encoding 44.1 kHz audio. Is this idea worthless without this feature?

Also, I'm wondering if DTS doesn't already hold a patent on extendable audio streams (DTS, DTS ES, DTS 96/24).

This post has been edited by wswartzendruber: Feb 24 2013, 22:08
Go to the top of the page
+Quote Post
saratoga
post Feb 24 2013, 22:08
Post #7





Group: Members
Posts: 4718
Joined: 2-September 02
Member No.: 3264



QUOTE (wswartzendruber @ Feb 24 2013, 15:36) *
When I came up with this idea, my primary use case was movies. With that said, I can't say that I really care about encoding 44.1 kHz audio. Is my idea worthless without this feature?


Why Opus? Something like Wavpack would probably work more efficiently unless you want ultra-low bitrate lossy files.
Go to the top of the page
+Quote Post
wswartzendruber
post Feb 24 2013, 22:10
Post #8





Group: Members
Posts: 85
Joined: 11-December 06
Member No.: 38563



Compatibility with existing Opus decoders.
Go to the top of the page
+Quote Post
db1989
post Feb 24 2013, 22:14
Post #9





Group: Super Moderator
Posts: 5174
Joined: 23-June 06
Member No.: 32180



Which, as has already been explained, is not possible.

Opus was not designed as a lossless format. If they had wanted it to have that capability in the future, they would not have implemented various features that rule out such usage. In other words, either it was never considered as an option, or it was considered and dismissed on technical or other grounds.

This post has been edited by db1989: Feb 24 2013, 22:15
Go to the top of the page
+Quote Post
wswartzendruber
post Feb 24 2013, 22:21
Post #10





Group: Members
Posts: 85
Joined: 11-December 06
Member No.: 38563



Nowhere in this thread has it been explained that this isn't possible. NullC has said that RFC6716 encoders are to set any padding to zero. But this isn't going to be an RFC6716 encoder. I aim to write a custom encoder that generates lossless streams that can still be played back by RFC6716 decoders.

Do DTS-ES encoders follow the specs for DTS encoders? Heck no. But they're still compatible.

EDIT: Now with that said, some good points have been raised and there are issues that need to be solved. Bit-exact inaccuracy is the main one.

This post has been edited by wswartzendruber: Feb 24 2013, 22:32
Go to the top of the page
+Quote Post
saratoga
post Feb 24 2013, 22:34
Post #11





Group: Members
Posts: 4718
Joined: 2-September 02
Member No.: 3264



QUOTE (wswartzendruber @ Feb 24 2013, 16:10) *
Compatibility with existing Opus decoders.


Yeah, but why?
Go to the top of the page
+Quote Post
wswartzendruber
post Feb 24 2013, 22:43
Post #12





Group: Members
Posts: 85
Joined: 11-December 06
Member No.: 38563



QUOTE (saratoga @ Feb 24 2013, 17:34) *
QUOTE (wswartzendruber @ Feb 24 2013, 16:10) *
Compatibility with existing Opus decoders.


Yeah, but why?

To have a single encode that you can use on the greatest number of devices. Looking at the parties backing Opus, I see it becoming a widely-supported codec. Consider Dolby Digital EX and DTS-ES. They would have never taken off had they not been compatible with existing implementations of their respective decoders.

In order to convince me this isn't a good idea, I would need to be pointed to a widely-supported lossless audio codec, or one that will arguably become this way in the near future.
Go to the top of the page
+Quote Post
saratoga
post Feb 24 2013, 22:54
Post #13





Group: Members
Posts: 4718
Joined: 2-September 02
Member No.: 3264



Mp3, aac.
Go to the top of the page
+Quote Post
wswartzendruber
post Feb 24 2013, 23:00
Post #14





Group: Members
Posts: 85
Joined: 11-December 06
Member No.: 38563



QUOTE (saratoga @ Feb 24 2013, 17:54) *
Mp3, aac.

Why yes, MP3 does have lossless extensions. It's also heavily patented.
Go to the top of the page
+Quote Post
saratoga
post Feb 25 2013, 00:10
Post #15





Group: Members
Posts: 4718
Joined: 2-September 02
Member No.: 3264



Do you really care? The MP3 patent license is almost always paid anyway on hardware devices (e.g. android, dvd players, etc) and no one is going to go after some niche hybrid lossless format for royalties since theres no money in it.

It sounded like you wanted compatibility. If so, just use MP3 or AAC.

This post has been edited by db1989: Feb 25 2013, 12:55
Reason for edit: deleting pointless full quote
Go to the top of the page
+Quote Post
wswartzendruber
post Feb 25 2013, 01:32
Post #16





Group: Members
Posts: 85
Joined: 11-December 06
Member No.: 38563



mp3HD can't go beyond stereo and 16-bit. I'm also unaware of any lossless extensions for AAC. And I do care about patents because this isn't just about me. I would also like for others to be able to use this, as well.

This post has been edited by db1989: Feb 25 2013, 12:55
Reason for edit: "
Go to the top of the page
+Quote Post
saratoga
post Feb 25 2013, 01:36
Post #17





Group: Members
Posts: 4718
Joined: 2-September 02
Member No.: 3264



QUOTE (wswartzendruber @ Feb 24 2013, 19:32) *
mp3HD can't go beyond stereo and 16-bit. I'm also unaware of any lossless extensions for AAC.


There are actually lossless extensions for both, but aren't you planning to implement your own system? Who cares what already exists?

QUOTE (wswartzendruber @ Feb 24 2013, 19:32) *
And I do care about patents because this isn't just about me. I would also like for others to be able to use this, as well.


Realistically the number of people who would ever use something like this is so small you will never attract enough attention for patent licensing to matter.
Go to the top of the page
+Quote Post
wswartzendruber
post Feb 25 2013, 01:43
Post #18





Group: Members
Posts: 85
Joined: 11-December 06
Member No.: 38563



QUOTE (saratoga @ Feb 24 2013, 20:36) *
There are actually lossless extensions for both, but aren't you planning to implement your own system? Who cares what already exists?

Realistically the number of people who would ever use something like this is so small you will never attract enough attention for patent licensing to matter.

Interestingly enough, you answered your own question. The new system must remain compatible with existing decoders precisely because nobody will have a new one. At least at first.

Think of it like this, "Hey, here's this lossless codec you can use, and it still plays on anything that can decode Opus."

This post has been edited by wswartzendruber: Feb 25 2013, 01:44
Go to the top of the page
+Quote Post
saratoga
post Feb 25 2013, 02:54
Post #19





Group: Members
Posts: 4718
Joined: 2-September 02
Member No.: 3264



There are no existing hybrid lossless decoders for Opus, so if your goal is to be compatible with an existing lossless implementation (which is not what you said above but what you seem to be implying here), you cannot use Opus. You would have to use the existing Mp3 or AAC variants.

If your goal is to be compatible only with an existing lossy format, then you can use any of the above mentioned formats, either extending it to your needs or simply using existing software and standards. In this case, I'd probably recommend just using MPEG-4 SLS, as it is backwards compatible with anything that supports AAC and is already standardized. You could also use MP3 and simply define your own correction format, but since you want 5.1 support, it would probably be easier to use AAC since there is already widespread support for 5.1 AAC.

This post has been edited by db1989: Feb 25 2013, 11:14
Reason for edit: deleting pointless full quote
Go to the top of the page
+Quote Post
wswartzendruber
post Feb 25 2013, 03:10
Post #20





Group: Members
Posts: 85
Joined: 11-December 06
Member No.: 38563



QUOTE (saratoga @ Feb 24 2013, 21:54) *
There are no existing hybrid lossless decoders for Opus, so if your goal is to be compatible with an existing lossless implementation...


Now I think I see the problem. There is a fundamental misunderstanding of what I am saying. Here is the principal of operation for the encoder:

1. Take the current chunk of input PCM and use an Opus encoder to generate a compressed, lossy, Opus packet.
2. Decode this packet (probably with the fixed-point decoder) and hang on to the PCM values it returns.
3. Subtract these PCM values from the input PCM values and compress those with a lossless codec.
4. Store this this new, losslessly-compressed chunk of data in the Opus padding area.

Here's the principal of operation for a standard Opus decoder:

1. Take the current input Opus packet and decode it to PCM.
2. Ignore the padding.

Here's the principal of operation for the decoder I would make.

1. Take the current input Opus packet and decode it to PCM with the fixed-point decoder.
2. Take the padding and feed it into the lossless decoder.
3. Add the decoded Opus value and the decoded lossless value together and store it as output PCM.

Does this make more sense now?

This post has been edited by wswartzendruber: Feb 25 2013, 03:11
Go to the top of the page
+Quote Post
wswartzendruber
post Feb 25 2013, 05:18
Post #21





Group: Members
Posts: 85
Joined: 11-December 06
Member No.: 38563



I also missed the part above where this proposition will mess up seeking. I'm hoping someone can explain why. That seems more like a container issue to me.
Go to the top of the page
+Quote Post
jmvalin
post Feb 25 2013, 05:43
Post #22


Xiph.org Speex developer


Group: Developer
Posts: 473
Joined: 21-August 02
Member No.: 3134



QUOTE (wswartzendruber @ Feb 24 2013, 23:18) *
I also missed the part above where this proposition will mess up seeking. I'm hoping someone can explain why. That seems more like a container issue to me.


It's not a container issue. if you seek within an Opus file, what you decode from that point will not be bit-exact with a decoder that started from the beginning of the file. The difference is so small that it's not audible, but it still means losing the lossless property as soon as you seek.

But really, I think most comments have so far missed the real issue here: A lossless extension to Opus would be mostly pointless. A 48 kHz 16-bit PCM stereo file is 1536 kb/s, and on average a codec like FLAC will compress about 50%, so that means 768 kb/s. If you consider that Opus gives you pretty good quality at 96 kb/s, then it means that Opus+FLAC would cost you 864 kb/s. OTOH, a lossless extension to Opus, would compress slightly worse then than a "native" lossless codec like FLAC (partly due to things like intensity stereo and spectral folding), probably costing us about at *least* 5%, meaning that we'd end up at 806 kb/s. So in the end, we have a messy new extension that saves us only 7% compared to just carrying two sets of files. There's just no point, really. Get over it.

This post has been edited by jmvalin: Feb 25 2013, 05:43
Go to the top of the page
+Quote Post
wswartzendruber
post Feb 25 2013, 06:02
Post #23





Group: Members
Posts: 85
Joined: 11-December 06
Member No.: 38563



Your first paragraph raises a very good and valid point. This is a technical issue that I will have to assess.

Your second paragraph, on the other hand, is mainly a matter of philosophy. You also appear to be a bit confused. "We" aren't going to be having any extension. "I" will be having an extension that others will be free to use. I do not require your blessing to continue, nor quite frankly, do I care if I have it. Especially not with that attitude.

I am looking for technical issues. So far, two very good ones have been raised.

This post has been edited by db1989: Feb 25 2013, 11:14
Reason for edit: deleting pointless full quote
Go to the top of the page
+Quote Post
2Bdecided
post Feb 25 2013, 12:23
Post #24


ReplayGain developer


Group: Developer
Posts: 4959
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



QUOTE (jmvalin @ Feb 25 2013, 04:43) *
we have a messy new extension that saves us only 7% compared to just carrying two sets of files.
..and sometimes it's worse, and it's always worse than just a pure lossless file that doesn't depend on a lossy core.

If you must go down that route and want to hack around with something "free", WavPack Hybrid and lossyWAV/lossyFLAC + correction files are built with this in mind.

Cheers,
David.
Go to the top of the page
+Quote Post
wswartzendruber
post Feb 25 2013, 17:00
Post #25





Group: Members
Posts: 85
Joined: 11-December 06
Member No.: 38563



Why does DTS-HD have a lossy core stream?
Go to the top of the page
+Quote Post

4 Pages V   1 2 3 > » 
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 24th April 2014 - 10:55