IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
A way to transcode 320 to VBR that is only slightly lossy?
hightype
post Jun 7 2013, 00:44
Post #1





Group: Members
Posts: 9
Joined: 7-June 13
Member No.: 108529



Hi,
I don't know if I should start a new topic for this - I have found this topic the most suitable to ask.

I have lots of 320 CBR files encoded with LAME Encoder. Is there a way to transcode them into VBR - with the initial file settings (stereo/joint stereo, filters and so on) left untouched - but in a slightly _lossy_ way, without decompressing the whole file?

E.g. I'd like to reach an average reduced bitrate of ~200-220 kbps, where some of the frames that contain high frequencies remain untouched in the transcoded file? (And similarly, mono frames such as bassdrum intros get reduced to whatever bitrate is appropriate for them.)

Is there an option for this in LAME? (Or, is there a 'lossy MP3Packer' available that I don't know of? smile.gif

Thanks!

This post has been edited by greynol: Jun 7 2013, 00:50
Reason for edit: Post was split from another discussion.
Go to the top of the page
+Quote Post
db1989
post Jun 7 2013, 00:59
Post #2





Group: Super Moderator
Posts: 5141
Joined: 23-June 06
Member No.: 32180



No. What you hypothesise is bitrate peeling, which is impossible by definition for MP3.

It has never been implemented well in a popular format: look at Vorbis, for instance. I believe MPEG-4 SLS is capable of scaling down to multiple levels, but I am unsure whether it is even released for general use, and at a minimum, it is not popular.

In theory, one could design a program that processes an input MP3 according to a set of criteria by which some frames would be left alone, some would be decoded and re-encoded to a smaller number of bits, and the whole lot would be spliced together into a new file, but I have to wonder whether it would be worth it considering the possibility of synergistic degradation owing to transcoding. And then you remember the bit reservoir, which might mess everything up, although I lack the technical experience to conclude either way. Trying to do a thought experiment but not having much success…

But anyway, you would want the decision whether to leave a frame at 320 kbps or not to be decided on the basis of psychoacoustics… but how can one be sure that psychoacoustic metrics are reliable when they are derived from a stream that has already been psychoacoustically compressed?

I leave these questions to other users who have the required technical knowledge of the format itself, but I suspect the idea is unfeasible, perhaps indicating why no one has ever implemented it yet.

QUOTE
some of the frames that contain high frequencies
MP3 does not work that way. A frame contains all frequencies within a specific region of time (number of samples). Frequencies are not divided into bands and distributed among different frames.

This post has been edited by db1989: Jun 7 2013, 01:36
Go to the top of the page
+Quote Post
hightype
post Jun 7 2013, 01:23
Post #3





Group: Members
Posts: 9
Joined: 7-June 13
Member No.: 108529



Well, I'm only an end-user (and not a professional in the field) so I probably didn't put what I wanted really accurately.

I looked up 'bitrate peeling' and from what I understand it is having multiple streams within one file to serve low bandwidth users.

However, my basic goal to reduce MP3 file size is to save HD/storage space. (320 CBR -> ~200-220 VBR is slightly lossy and reduces file size to circa 60-70%. Storage is an issue if I have eg. gigabytes of DJ sets or radio recordings, stored in 320...)

This post has been edited by hightype: Jun 7 2013, 01:27
Go to the top of the page
+Quote Post
db1989
post Jun 7 2013, 01:36
Post #4





Group: Super Moderator
Posts: 5141
Joined: 23-June 06
Member No.: 32180



I understand what you want. Users with more experience of the format on a technical level can comment on the feasibility of your proposal. However, I have doubts. Please see the third to fifth paragraphs I just added to my previous post above.
Go to the top of the page
+Quote Post
hightype
post Jun 7 2013, 02:08
Post #5





Group: Members
Posts: 9
Joined: 7-June 13
Member No.: 108529



Yes, I've read it and thank you for the reply!

First, as I understand, at the moment, there is no solution/software to do this, so decompressing and re-encoding the file is the only way (and to choose the new compression ratio by ear)...

What you wrote however is really interesting... that is, a program where the amount of degradation (lossyness) could be set.
Perhaps MP3Packer could introduce a lossy function... or something like a 'degrade' option could be introduced in LAME... again, where all the parameters of the original MP3 would remain - except the bitrate of course.

Unfortunately I'm not a developer so I couldn't help with that... I don't know how much demand there is for such a program or option, but I think I'm not alone with this smile.gif
Go to the top of the page
+Quote Post
Destroid
post Jun 7 2013, 03:37
Post #6





Group: Members
Posts: 544
Joined: 4-June 02
Member No.: 2220



Interesting problem, of which I personally don't have much personal vestment since I would (obviously) make new encodes at my target criteria. But finding solutions is a hobby of mine. smile.gif

Does the resulting files have to be MP3? I recall a previous discussion where a user claimed transcoding to another newer/better lossy scheme had less annoying artifacts. I think specifically of AAC.

That concept db1989 described also sounds interesting and quite technical. I'd be interested in follow the discussion about how MP3 files can be decimated and carefully re-integrated.


--------------------
"Something bothering you, Mister Spock?"
Go to the top of the page
+Quote Post
Makaki
post Jun 7 2013, 05:25
Post #7





Group: Members
Posts: 52
Joined: 20-May 13
Member No.: 108227



Look at the bright side, they may be taking a lot of space now, but in a few years, you will be happy you have them at a high bit rate.

Ask yourself this, how would you feel if because of a choice you made long ago, they would have been at 128 kbps (or less).
Go to the top of the page
+Quote Post
hightype
post Jun 7 2013, 05:31
Post #8





Group: Members
Posts: 9
Joined: 7-June 13
Member No.: 108529



Well, AAC is a relatively new format and not widely supported. MP3 is the most widespread format and properly encoded files can be played on even 10+ yr old hardware and software so as for compatibility it has quite a 'staying power'...

AAC and OGG (and others) have far better technical qualities and are quite sympathetic but I personally like to archive all music in MP3 for the previous reasons. I don't know if AAC or OGG will have the same 'staying power' as MP3.

However, could you please give a link to the AAC transcoding topic?
Go to the top of the page
+Quote Post
hightype
post Jun 7 2013, 05:43
Post #9





Group: Members
Posts: 9
Joined: 7-June 13
Member No.: 108529



QUOTE (Makaki @ Jun 7 2013, 05:25) *
Ask yourself this, how would you feel if because of a choice you made long ago, they would have been at 128 kbps (or less).


Haha, I know what you mean smile.gif (I have MP2 files at 128 kbps - it was the state-of-art technology way back then... with all its artifacts it sounded still better than noisy cassette tapes... didn't even dream about CD quality lossless files then...)

This post has been edited by hightype: Jun 7 2013, 05:47
Go to the top of the page
+Quote Post
mobyduck
post Jun 7 2013, 10:02
Post #10





Group: Members
Posts: 570
Joined: 24-April 03
From: Italy
Member No.: 6159



Most probably I'm not understandig what you're asking (otherwise it would have already been suggested) but I'll throw that in anyway: maybe MP3 repacker is what you're looking for?

HTH.

Alessandro
Go to the top of the page
+Quote Post
2Bdecided
post Jun 7 2013, 12:10
Post #11


ReplayGain developer


Group: Developer
Posts: 4945
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



mp3 repacker is lossless and might help. If it doens't...

If you care more about quality than space, leave them as they are.
If you care more about space than quality, transcode them to whatever quality you wish. AFAIK there is no clever way of doing this for mp3. No smart rendering. No partial decode transcoding.

Cheers,
David.
Go to the top of the page
+Quote Post
hightype
post Jun 7 2013, 17:18
Post #12





Group: Members
Posts: 9
Joined: 7-June 13
Member No.: 108529



Thanks for the new comments!

In the original post I was already referring to MP3 Repacker - I'm a statisfied user of it (it is a brilliant idea and software!), but, it is lossless optimization.
My point was to achieve MP3 optimization/size-reduction in a slightly (or controllably) lossy way, that results in even more reduced file sizes.
MP3Packer reduces a high bitrate MP3 to about 90% to its original size on average but I wonder if it's theoretically possible to achieve about 60-70% or less (of course it can only be done in a lossy way).

DB1989 started to think on the technical realization, that is, what could be 'chipped off' next from MP3 files to preserve as much original data as possible but to further reduce file size.

This post has been edited by hightype: Jun 7 2013, 17:24
Go to the top of the page
+Quote Post
greynol
post Jun 7 2013, 18:15
Post #13





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



You're basically talking about selective re-encoding on a frame-by-frame basis, I think. From what I understand the only frames where there would be no degradation in data would be the ones that are kept at 320 or could otherwise be packed losslessly. The remaining frames would be decoded to PCM and then subjected to an additional generation of mp3 encoding.

I don't know enough about about the way in which continuity is kept between frames in order for them to transition smoothly under regular circumstances to know if this will be problematic when re-encoding on a frame-by-frame basis.

This post has been edited by greynol: Jun 7 2013, 19:37


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
hightype
post Jun 7 2013, 20:11
Post #14





Group: Members
Posts: 9
Joined: 7-June 13
Member No.: 108529



Greynol: I think you put it in the best way - 'selective re-encoding' might be a good name for the process and also a good idea.

Some 'chipping' ideas:
- in 320 CBR even silent frames are encoded at 320 kbps, they could be re-encoded at a low bitrate (32 or 96).
- entirely mono frames could be re-encoded at half bitrate (in case the original file is in full stereo)
- frames that don't contain too much high freqs or are just bass could be encoded at lower bitrates

However, as far as I know - and DB1989 also mentioned it -, the basic problem is handling the bit reservoir problem, which, as I understand, is storing extra bits of a frame that is too complex to be stored in one frame at the given bitrate, in neighboring frames.
MP3Packer also rearranges these 'bits reservoir' when optimizing files so I think it wouldn't be too much of an issue (and for this reason, even fully 320 frames can't be left _structurally_ intact if there is a bit reservoir - but that doesn't affect quality, just the MP3 bit structure is re-arranged).

The bottom line is - that it would be cool have a solution to reduce MP3 bitrate (and thus the file size) while keeping audio quality at the maximum possible.
Go to the top of the page
+Quote Post
[JAZ]
post Jun 7 2013, 20:30
Post #15





Group: Members
Posts: 1706
Joined: 24-June 02
From: Catalunya(Spain)
Member No.: 2383



- silent frames: Mp3 repacker should do that.
- mono/stereo: This can only be done by reencoding. There's no way to reinterpret the data.
- "easy" frames: Mp3 repacker should do that. (except if the encoder just tried to fit as much data as possible, instead of adding padding. In this case, it might be possible to manipulate the data, but you would need to add a psy-model to determine what to throw out ).


Sincerely, for your target usage, the whole discussion is completely uninteresting. As you said, bitrate peeling (and the idea of being able to reduce the bitrate from an existing file) has always been connected with streaming/downloading, and with the clear definition that the peeled file is of lower quality.

On the other hand, your target is "something which reduces the bitrate more than mp3repacker, but doesnt' reduce the quality as much as transcoding".

This post has been edited by [JAZ]: Jun 7 2013, 20:37
Go to the top of the page
+Quote Post
Dynamic
post Jun 7 2013, 21:23
Post #16





Group: Members
Posts: 793
Joined: 17-September 06
Member No.: 35307



As suggested you can ignore bitrate peeling. In Vorbis, past attempts at peeling were not as good quality as transcoding, from what I remember reading on xiph.org

Assuming you actually decide to transcode, there was an old test done by a very good and patient tester called guruboolez some years ago. Those tests were, I believe, starting with material at around 200-250 kbps (e.g. Lame -V0 to -V2) and reducing to the region of 128 kbps.

You're starting at a much higher bitrate, with margin over transparency and coming down to -V1 or -V2 which would normally be transparent, so the amount of degradation will probably be much more subtle and hard to hear from the point of view of general psychoacoustic masking.

Those guruboolez tests indicated that encoding from MP3 to MP3 was worse than encoding from MP3 to another format (e.g. Vorbis or AAC), and it was suspected this might be down to matching frame and granule boundaries causing effects beyond simple masking.

I think a follow-up test indicated that introducing a few milliseconds delay (e.g. 10 ms) to the audio broke the window-boundary match and helped the quality of MP3 to MP3 encoding somewhat. This would then break gaplessness (which in any case is not supported on all MP3 decoders such as 10-year-old hardware devices) but it might be possible to adjust the LAME gapless header to compensate for the extra samples of silence added at the start if it was important enough to you to get in and edit the gapless info in this way. Switching to a different encoder (AAC is probably the most widely supported at present) might work out better.
Go to the top of the page
+Quote Post
greynol
post Jun 7 2013, 21:28
Post #17





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



If you introduce delay you will have to re-encode everything, rather than select frames.

I too think transcoding to AAC would be the best bet assuming it meets compatibility requirements and assuming you don't have access to the original lossless source, bypassing transcoding altogether.

This post has been edited by greynol: Jun 7 2013, 21:30


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
hightype
post Jun 8 2013, 01:09
Post #18





Group: Members
Posts: 9
Joined: 7-June 13
Member No.: 108529



Thanks for the follow-ups!

[JAZ]: "something which reduces the bitrate more than mp3repacker, but doesnt' reduce the quality as much as transcoding"
Yes, this sums up my initial position the best way!

However, as a conclusion, lacking any direct software solution for this at the moment, the best solution (in terms of quality) would be transcoding the files to AAC.

Adding some pre-delay to the de-encoded file to 'mess up' frame boundaries is also a good idea (when transcoding MP3 into MP3).

Well, thanks for the input guys, I'll try to experiment with these solutions.
Go to the top of the page
+Quote Post
Zarggg
post Jun 8 2013, 02:19
Post #19





Group: Members
Posts: 533
Joined: 18-January 04
From: bethlehem.pa.us
Member No.: 11318



I'd like to add another (late) vote to transcoding to AAC. I've done that in the past for iPod transfers where I have encodes in Vorbis, Musepack, or WMA but no longer have access to the original source. I haven't noticed any audible encoding errors or artifacts. As always, YMMV.

This post has been edited by Zarggg: Jun 8 2013, 02:25
Go to the top of the page
+Quote Post
Porcus
post Jun 10 2013, 17:24
Post #20





Group: Members
Posts: 1779
Joined: 30-November 06
Member No.: 38207



hightype:

I have previously asked why there is no way to decode and then use a lossless encoder to get down to the mp3's bitrate. (Obviously it is in principle possible, as the .mp3 file losslessly contains the decoded signal, though of course not the original.) The best explanation I got, is that this would be very close to .zip'ing an encrypted file down to the size of the .zip'd unencrypted version; it can be done in principle, but in practice that would amount to password cracking.

And what you ask for, is sorta even more. Not only do you want to fit the decoded signal in the original mp3 file without knowing that file, you want to fit something that sounds like it into something smaller.


--------------------
One day in the Year of the Fox came a time remembered well
Go to the top of the page
+Quote Post
hightype
post Jun 11 2013, 13:43
Post #21





Group: Members
Posts: 9
Joined: 7-June 13
Member No.: 108529



Porcus: Well, I can't really grasp the comparison you proposed (as I'm not a pro on data compression)...
I can only reiterate, my (theoretical) goal would be, as Greynol put it, 'selective re-encoding' of MP3 frames, kind of what MP3Packer does now, but in a (controllably) lossy way.
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 16th April 2014 - 08:46