lossless mp3 - mp3HD, backwards compatible with standard mp3 Options

[monoton]

How do I get an mp3 file that is of exact length as the wave? I quickly tried converting a wav file to mp3 (with Lame) and back to wav but there was about 0.05 seconds of something totally irrelevant added in the beginning... well, I could simply truncate that later... but for what... why can't the stream just match exact size in number of samples?

[Alex B]

Useless format:

• The lossless part is stored in ID3v2 tags.
• Size of ID3v2 tags is limited to 256MB by specifications; as a result, lossless part of an mp3hd file can't be larger than 256MB.

I wonder what those people will think of next. Maybe resurrect VQF or something.

I agree.

As a test, I tried to encode a 500 MB wav file:

CODE

C:\Test\mp3hd>mp3hdencoder -br 192000 -if t.wav -of t.mp3

******************************************************************************
* *
* mp3HD Commandline Encoder V1.4.0 *
* (Evaluation Version) *
* *
* Revision 1007 *
* Build Jan 30 2009 *
* *
* © 2007 - 2009 Thomson S.A. *
* © 1996 - 2009 Fraunhofer IIS *
* All rights reserved. *
* *
* This software and/or program is protected by copyright law and *
* international treaties. Any reproduction or distribution of *
* this software and/or program, or any portion of it, may result *
* in severe civil and criminal penalties, and will be prosecuted *
* to the maximum extent possible under law. *
* *
******************************************************************************
frame 114579Error: cd-chunk is greater than ID3 Tag Size
Error: Closing ID3 Tag Handle

In addition, I don't see -ofl mentioned in the options. I suppose they really don't care about gapless (lossy) MP3 playback. Though, in any case FhG's implementation is not widely supported.

I wonder if the lossless plugin can play gaplessly. (I have not tested it yet.)

[lvqcl]

How do I get an mp3 file that is of exact length as the wave? I quickly tried converting a wav file to mp3 (with Lame) and back to wav but there was about 0.05 seconds of something totally irrelevant added in the beginning... well, I could simply truncate that later... but for what... why can't the stream just match exact size in number of samples?

1) don't use old versions of lame. Latest version is 3.98.2.
2) don't use -t switch.

[Alex B]

I did some gapless playback tests.

The -ofl switch does not work with the CLI encoder, but apparently the "-ofl" info is added automatically because the native Winamp plugin can play the lossy part of mp3HD files gaplessly and the encoder delay and zero padding values are displayed in the "view file info" window. This works only when the mp3HD plugin is not installed.

However, surprisingly the mp3HD plugin does not play the same files gaplessly when it is set to play only the lossy part. My test files were played exactly like standard MP3 files without any gapless playback headers. When the plugin is set to play the lossless part playback is correctly gapless.

I tested also LAME encoded "gapless" MP3 files. Fortunately the installed mp3HD plugin was not activated and the native Winamp plugin played the files correctly.

EDIT

I removed the word "surprisingly". As far as I understand, decoding accurate file lengths correctly from lossy MP3 data is not simple and there is really no need for the additional gapless decoding code when the file contains also the lossless part that can be played with the decoder.

A converter that could quickly extract the lossy part for portable use would make the format more usable. It could be a simple file copying & tagging tool, but preferably it should not first duplicate the complete source file because the target location may be size restricted and the available bandwidth may be limited. Perhaps it could first copy the file to a temporary location, remove the losless tag and after that move the file to the destination location.

This post has been edited by Alex B: Mar 20 2009, 22:36

[rpp3po]

Lossy formats are hardly ever decoded deterministically, so your idea will never work (at least not losslessly). In SLS (so also mp3HD I assume) the original AAC/MP3 stream is not decoded by a normal lossy decoder.

Deterministic decoding is irrelevant in this case. You only have to guarantee that the same decoder is used at delta generation time and at reconstruction time. A non-deterministic decoder will still generate the same WAV file every time it is run and that is all what is needed, a CPU is still a deterministic machine executing the same code multiple times.

This post has been edited by rpp3po: Mar 20 2009, 21:54

[menno]

Lossy formats are hardly ever decoded deterministically, so your idea will never work (at least not losslessly). In SLS (so also mp3HD I assume) the original AAC/MP3 stream is not decoded by a normal lossy decoder.

Deterministic decoding is irrelevant in this case. You only have to guarantee that the same decoder is used at delta generation time and at reconstruction time. A non-deterministic decoder will still generate the same WAV file every time it is run and that is all what is needed, a CPU is still a deterministic machine executing the same code multiple times.

Not just the same decoder, but the exact same build even. That means it will only work on 1 platform, with just 1 specific decoder build, making it much more useless than mp3HD IMO.

[rpp3po]

Not just the same decoder, but the exact same build even. That means it will only work on 1 platform, with just 1 specific decoder build, making it much more useless than mp3HD IMO.

Hmm, the platform dependency would be quite a disqualifier. In the case of MP3 and AAC, is this caused by how decoders are usually implemented or already inherent within the reference encoder's ISO spec?

This post has been edited by rpp3po: Mar 20 2009, 22:26

[menno]

Not just the same decoder, but the exact same build even. That means it will only work on 1 platform, with just 1 specific decoder build, making it much more useless than mp3HD IMO.

Hmm, the platform dependency would be quite a disqualifier. In the case of MP3 and AAC, is this caused by how decoders are usually implemented or already inherent within the reference encoder's ISO spec?

Performing floating point operations in different order might produce different results (10*(a+b) or 10*a+10*b can give different results), so the exact output is dependent on the compiler since it can optimize things away or reorder things. The ISO spec allows these minimal differences between decoders. I guess you could get deterministic results between compilers if you would use an integer based decoder, but it would still have to be the exact same codebase everytime you use it.

[saratoga]

Not just the same decoder, but the exact same build even. That means it will only work on 1 platform, with just 1 specific decoder build, making it much more useless than mp3HD IMO.

Hmm, the platform dependency would be quite a disqualifier. In the case of MP3 and AAC, is this caused by how decoders are usually implemented or already inherent within the reference encoder's ISO spec?

Its because of how floating point math works. You don't have to use floating point math to decode MP3, but its most common on PCs since integer is slower.

[rpp3po]

Well, then a packaged integer only decoder would be needed for delta-generation and reconstruction. That makes it less "generic" than I had initially thought. It would only work for a certified set of (to be included) decoders.

This post has been edited by rpp3po: Mar 21 2009, 00:15

[m0rbidini]

Oh my... The only good thing that comes to mind when reflecting about this is that Thomson/FhG must have ordered a market research which concluded that the demand for lossless will grow substantially and then decided to milk the MP3 buzzword a little more... again. Everything else gives me a headache, especially the thought of this turning out to be a success.

[DOS386]

Useless format:

• The lossless part is stored in ID3v2 tags.
• Size of ID3v2 tags is limited to 256MB by specifications; as a result, lossless part of an mp3hd file can't be larger than 256MB.

I wonder what those people will think of next. Maybe resurrect VQF or something.

Agree, useless horrible hack.

What's the most attractive in your opinion:
"Buy HD MUSIC. Cristal clear sound. Compatible with every player"
or
"Buy lossless music. CD Quality. Download the MP3 codec, the GUI and follow our two page tutorial to make it work"

None of those. I prefer:
"Buy lossless music. CD Quality. Download audio file, codec is free, open source and well designed"

Also, since FLAC is the weakest of the "useful" lossless codecs and this hack is even weaker and at same time much slower


This post has been edited by DOS386: Mar 21 2009, 12:17

[guruboolez]

None of those. I prefer:
"Buy lossless music. CD Quality. Download audio file, codec is free, open source and well designed"

I also share your choice, but I repeat, installing a piece of software is an annoyance for many people (customers) and they could flee any store requires such operation. Moreover, the codec is sometimes useless. For example, there's no plug-in for people using iTunes and I don't see a vast public learning to transcode their flac to a third-party tool in order to play-it on iTunes and transfert it on iPod. The success of amazon.com and iTunes Music store comes from simplicity (download and play & transfer) – not from freedom or open-source. Unfortunately I would say…

This post has been edited by guruboolez: Mar 21 2009, 12:26

[N!Ce]

Interesting read. I basically agree with guruboolez, the only purpose of this format is marketing and getting more users (customers) into the lossless world. It appears quite inferior compared to other lossless formats however, it's usually not the superior things that prevail, but the easier to handle ones. Keep in mind, the vast majority of music listeners are 'newbies' on that part. That's why I agree that it's a rather useless format, but still likely to have a certain success.

Btw, the in my opinion more useful solution for the lossy-lossless dual archiving would be, finally establishing a useful lossless format. The argument of being able to store more files when they are lossy while it won't make a difference on a DAP anyway, will constantly decrease and finally vanish. Because by this rate it won't take too long until you will be able to store more lossless files on your DAP than you will be able to listen to. That's when the argument of using only one filetype for everything will weight much more. All we need is to get a useful lossless format as established and known as mp3 is at the moment, but that is unfortunately not likely to happen soon.


So far my 2 HD cents.

[shakey_snake]

Interesting read. I basically agree with guruboolez, the only purpose of this format is marketing and getting more users (customers) into the lossless world.

Yeah by tricking them.

You can't really force peoples hands like this.

Lusers get massive mp3s and those-in-the-know get a hackish crappy format. It's not good for anyone.

[seanyseansean]

None of those. I prefer:
"Buy lossless music. CD Quality. Download audio file, codec is free, open source and well designed"

I also share your choice, but I repeat, installing a piece of software is an annoyance for many people (customers) and they could flee any store requires such operation. Moreover, the codec is sometimes useless. For example, there's no plug-in for people using iTunes and I don't see a vast public learning to transcode their flac to a third-party tool in order to play-it on iTunes and transfert it on iPod. The success of amazon.com and iTunes Music store comes from simplicity (download and play & transfer) – not from freedom or open-source. Unfortunately I would say…

We will never get mainstream pop on lossless formats for download.

The next big thing will be surround or higher bitrate formats.

Once everyone has their entire libraries on itunes, like we did with CDs, the industry will simply resell us our music in 'HD!!!!'. It's not worth the bandwidth and marketing cost to sell a mathematically exact format, when they can sell us a lossy 'HD!!!!' format instead.

You have to hope and pray that red book and blu-ray formats remain hacked.

Props to whoever brought up VQF - I remember from my earlier ripping days with some crappy Yamaha encoder. Happy memories

This post has been edited by seanyseansean: Mar 21 2009, 21:19

[lvqcl]

About quality of MP3 part: I encoded the same file with mp3HD Encoder (V1.4.0) and mp3 surround Encoder (V1.5, Encoder-Library V04.01.01):

CODE

mp3hdEncoder -if test.wav -of mp3hd.mp3 -br 256000
mp3sEncoder -if test.wav -of mp3surr.mp3 -br 256000 -q 1 -ofl

mp3 part of mp3hd.mp3 is exactly the same as mp3surr.mp3.

This post has been edited by lvqcl: Mar 22 2009, 02:22

[kode54]

I tried replacing the lossy portion with LAME encoded MP3 data, and after successfully transplanting the mp3HD tag and Xing header onto the LAME data, it fails 6 frames in with a hash error:

CODE

Decode frame 6 in CD-Quality
(Mp3HdSsc) error: Decoder - wrong hash

I guess either the first 6 frames were perfect matches, or it hashes the decoded audio data and those were silent frames. So much for slipping in some other lossy data to see what the decoder produces.

[aconverse]

Not just the same decoder, but the exact same build even. That means it will only work on 1 platform, with just 1 specific decoder build, making it much more useless than mp3HD IMO.

Hmm, the platform dependency would be quite a disqualifier. In the case of MP3 and AAC, is this caused by how decoders are usually implemented or already inherent within the reference encoder's ISO spec?

Performing floating point operations in different order might produce different results (10*(a+b) or 10*a+10*b can give different results), so the exact output is dependent on the compiler since it can optimize things away or reorder things. The ISO spec allows these minimal differences between decoders. I guess you could get deterministic results between compilers if you would use an integer based decoder, but it would still have to be the exact same codebase everytime you use it.

If it's anything like SLS (which you initially claimed) then corrections are applied in the transform domain then an integer version of the transform is applied. You can't just mash the corrections onto the output of a regular decoder.

[lvqcl]

http://www.hydrogenaudio.org/forums/index....st&p=544529

That's why SLS also operates in the frequency domain which is incompatible to the MP3 filterbank.

This post has been edited by lvqcl: Mar 22 2009, 19:48

[cybertoon]

heres a sample for you all
HD VBR 1:
http://rapidshare.com/files/212268246/Asaf...r_Lies_-_HD.mp3

and MP3 VBR:
http://rapidshare.com/files/212274166/Asaf...es_-_256VBR.mp3

[saratoga]

I think you could do this with mp3 like follows:

1) Create your correction file prior to the IMDCT step.
2) Define a specific integer implementation of the IMDCT and inverse subband filterbanks
3) Have the encoder compute any roundoff error during the decode phase and store this in a second correction file

Hopefully you can design the filter banks so that 3 is very, very small (or even zero) and thus can be compressed to be quite efficiently.

[odyssey]

I don't really know what I should think of this. In general I think it's a nice proof-of-concept, but in reality I don't think I would use it.

1. Why would I carry gigantic files if I only get mp3-sound (on devices that don't support it, which would be close to most).
2. If the correction-parts are not separate, I can't even use this to avoid a secondary library for my mobile-encodes.

[ameyer17]

You would theoretically be able to strip the lossless data from the tags.
For example, I edited the tags in foobar2000 (running in WINE, in the unlikely case that it matters) and the file size went from over 18 MB to under 5 MB.

Though I still think this format is useless.

This post has been edited by greynol: Apr 2 2009, 19:38

[SebastianG]

http://www.hydrogenaudio.org/forums/index....st&p=544529

That's why SLS also operates in the frequency domain which is incompatible to the MP3 filterbank.

I actually didn't test it. So, this was me guessing. The argument is that MP3 uses two filterbanks that are concatenated (PQMF + MDCT + alias reduction) and SLS is based on the MDCT only. But maybe this concatenation is "similar enough" to what a plain MDCT would produce given a good window function. But maybe they really use an integer MP3 decoder and update the time domain data using the "HD" data. I don't know.

Cheers!
SG