Hello,

I work on my own lossless format. For the moment, my codec called "Audio Multichannel Vers 1.0" can only encode in 2:1 ratio with the huffman encoding. I search a new algorithm to increase this ratio (4:1 is a good goal) An idea ?

For information, my lossless codec can encode in 24 bits, 96 Khz, with ID3 2.x and multichannel support (7.1)...
This codec is for the future DVD Audio ripping but it's the beginning...

Many thanks for your answers,

Best regards,

Christophe Fantoni
Author of the french book :
- "How to make your own DVD"
- "Digital Recording on CD and DVD"
Published by Dixit (www.dixit.fr)

Please, don't use ID3 v2.x tagging system. Use APE 2.0 or your own tag format.

Depending on your license you could or could not use some ideas from Flac

I may be missing something, but isn't 2:1 already better than any other lossless codec out there? And if so, is it likely there'll be much to improve the algorithm at all, let alone to 4:1?

Cheers, Paul

Exactly what I was thinking -- is this 2:1 based on testing of a variety of real music or just one very easy-to-compress passage?

Since this codec is multichannel, shouldn't you be able to compress it way more if you use "Joint Multichannel"?

You decided on the video container format already ? If you would like to look at matroska for this purpose, we would gladly teach you through how to using it, and your tagging problems were also solved this way ....

I believe that the higher the sampling rate, and the more channels there are, the "easier" it is to losslessly compress the audio. Read up on Meridian Lossless Packing (MLP), since there is a relatively large amount of information regarding this algorithm (it is used for DVD-A).

A quick and dirty google search led me to this:
http://www.extremetech.com/print_article/0...8,a=1531,00.asp

What's wrong with ID3V2?

Any alpha\beta\stable versions of your new codec ?
PS: I think it's impossible to archieve 4:1 ratio

1. It is impossible to guarantee a 2:1 lossless compression ratio (or n:1, n>1) for all input.
2. If you just need something to encode 24/96 7.1, FLAC will already do it.
3. There are already many lossless codecs. That doesn't mean you shouldn't do something on your own but if you come with some improvements, consider contributing to an existing codec.
4. MLP *will* drop data if it can't achieve the target rate (see #1). They try very hard to avoid saying that in their docs.

Josh

Hold on.. you mean you're reaching 2:1 ratio on your test samples overall, using just huffman coding ?

A few questions:
- how many channels were in use ?
- were all channels actually different from each other ?
- are you sure you used only huffman coding ?

These questions because, reaching a 2:1 ratio on typical stereo material using just huffman coding sounds very, very unlikely to me.

BTW, yes, if you achieved such a breakthrough, you definitely should use a good tagging system too

Well, a lot of people put all kind of crap in ID3 v2.x tags, it's not so clean format. And I think that file tagged with ID3 isn't gapless, but I'm not sure?

Hello people,

Well, many answers... It's a very good thing. My own answers, with no order.
Before, just a precision : my codec can compress in lossy and lossless format.

- In the lossy option, my codec use an ADPCM modification (4:1), to keep audio quality, and my huffman variation (2:1) For the moment, I can compress quicky my audio file up to 8:1. It's good for real time but it's not enough for me. In fact, I work to integrate my first psychoacoustic model to increase the audio quality and reduce the filesize, but it's very harder... Also, I search an Delphi or Pascal/ASM example to make that more easily. For the moment, I read many, many, PDF file with many informations... And it's not easy...

- In the lossless option, my codec use huffman coding variations (with no patent) and can compress all the multichannel files up to 2:1 (1.0 to 7.1). Each channel are compressed with no audio interpolation. (no joint multichannel/stereo but real stereo) In each frame, each channel are encoded individually... It's good, but it's not the Monkey Audio ratio...

For the moment, I can compress WAV file from 11Khz to 96Khz, but 96Khz in not really a limit (for example, I can up to 192Khz with many modifications) And, I can use ID3v1.x, ID2v2.x, and APE tag, or make my own tagging system, if there are no license violation (for no problem) In fact, my goal is to make a real standalone player/recoder (hardware) which use my own technology... with no patent. I make my own format for that raison... (To create your own format is not really a problem ; the problem is to use algorithm with no patent)

After, I would like to create my own compagny to use commercially my own products and my own technology... For me, it's a big project... Many external (french) compagnies are very interessting to help me, to sell or distribute my hardware products in the future, but my audio codec is for the moment not really ready.

Since few weeks, I work on a first release witch include an sampling converter and an multichannel encoder in the program. But I would like to distribute this first release with an WinAmp plugin and a DirectShow Filter to decode more easily - but I have found no source to make this kind of program in Delphi... My encoder is for the moment a very simple command line, work perfectly in DOS Box 16/32 bit.

Many thanks for your answers,

Best regards,

Christophe Fantoni
Author of the french book :
- "How to make your own DVD"
- "Digital Recording on CD and DVD"
Published by Dixit (www.dixit.fr)

So what's the advantage over existing codecs? Encoding speed? Smooth degradation to lossy? Compression ratio? Do you have some examples of how it compares to say, FLAC? (the only non-proprietary codec I know that does 7.1)

Josh

* ADPCM is not really a state-of-the-art technology, you know And integrating psychoacoustic with it looks like a challange. Most current lossy codecs work with some frequency transforms.

* I still can't understand your point about 2:1 compression. Is it for only one test file? How other codecs perform on it then?

* And Josh has a point as well

-Eugene

MLP will warn you at the authoring stage that it can't do what you're asking it to do. It won't "silently" drop the audio quality.


It can't encode 6 channels of uncorrelated white noise at 24/96 in the target bitrate, but then, white noise only needs around 5 bits for transparency, so it's not a major problem!

I've not seen a report of anyone finding a "problem sample" of real music which could not be encoded onto DVD-A at 24/96/6-ch - have you?

Cheers,
David.

Hello,

Well, my lossless codec is for my own standalone project... It's not an open source codec, for many plateforms, like FLAC for example. A command line version, for DOS, will be released in few weeks for feedback and test only. For the moment, my codec is not really finished and I test many solutions to increase audio quality and reduce filesize (like my "psychoacoustic model integration" or another solution, like "frequency transform")

In fact, I made an ADPCM variation with bitrate choosing. It's not the fixing ADPCM (standard) like in MS ADPCM or IMA compression. I choose this format for two raisons : to encode/decode quickly, in real time, and because many compagnies like Sega (Dreamcast) , Sony (Playstation I/II), Nintendo (Gamecube), Microsoft (XBOX) and Apple use it... After, I use my own huffman variation to reduce by two the framesize. And now, my lossly codec can compress quickly up to 8:1. It's a good beginning for my project, but it's not the state of the art... It's for that I search better solution to increase my ratio (my goal is to 12:1)

In my lossless techchnology, each frame is compressed only with huffman coding, and huffman coding can compresse easilly yo to 2:1. For the moment, my huffman coding variation, with no patent for me, work perfectly, and make the job. Beware, it's not the standard huffman coding, it's my own variation. To understand, you can imagine each frame like a file compressed with ZIP/ARJ/LHA/RAR/ACE. Each "file" is normally reduce by two (2:1) with this kind of compression. If you're add all filesize, the average compression will be 2:1. For me, it's the same method but with my own huffman coding... It's not fantastic, but it's work...

To finish, if you wan to compare my AM1 to FLAC compression, my own codec in not only a (de)compression tool, it's also an audio tool. For example, my codec include in standard :

- Samplerate converter : Upsampling and downsampling your audio file up to 96Khz at 24 bits with great quality.
- Multichannel encoder : Matrix your audio file in Dolby Surround or Dolby Prologic or add many channel in one file to make a real multichannel audio file. (to encode it in AC3 or DTS, or maybe in WMA 9 Multichannel)
- Normalize : to normalize you audio file (after ripping) before encoding.
- etc

You can use this option with my own format or with WAV file (maybe AIFF) in the export option. This kind of options aren't include in classic lossy/lossless codec. You must use external tool to make it, but with my codec it's build-in.

Sorry for my (very simple) english and many thanks for your answers,

Best regards,

Christophe Fantoni
Author of the french books :
- "How to make your own DVD"
- "Digital Recording on CD and DVD"
Published by Dixit (www.dixit.fr)
Visit my website : www.christophefantoni.com
(french only, but with little english text)

Christophe,

This looks like an interesting project, but before you seriously start, I advise you to make sure you'll actually surpass existing codecs (both lossy and loseless). In both cases, even in public domain there are tough (and free !) competitors.

It's very difficult to compete everywhere at once, so you would probably have to forus more precisely.

About ADPCM, you can use it in the time-domain but it won't help you in frequency-domain. There are many similar ways to quantize samples, thanks to probabilities and hearing properties - and most codecs are using several of them ! All of the basic methods have already been used, and many have been surpassed (additions were SBR, arithmetic coding, transform coding, channel coupling etc).

But remember, no matter what you do: have fun !

Hello,

My two methods are free patent. You know, the SBR algorithm (better) are patented. MP3 Pro and PlusV codec use it, but it's not popular format like MP3 standard or Ogg Vorbis. (better, and not very popular -- it's strange) For my project, I search another encoding methods, with no patent (with no license violation/problem), to increase my ratio and to reduce the filesize. I work harder for that... Maybe arithmetic or transform coding...
In fact, it's very easy to integrate ADPCM and huffman coding in electronic developpement. And my goal is to make a chip (hardware) who use my own compression (with no patent) It's the first step of my project... I don't wan to pay, many, many money, to sell products whitch use my format because all the algorithms, or the format, are patented. It's not my goal to loose money...

For me, ADPCM/huffman coding is the beginning, the first version.
If I can make better codec, I make it...

Best regards,

If you aim for lossless compression, go for it.

If you want to make a chip to perform lossy compression, why not use Vorbis ? Years of work went into it and I assure you, it takes more than a signal processing expert to build this.

There are so many formats that beg for improvement, so why create a new one ?

Cheers

What does DVD-A MLP do when bitrate reaches the limit
- lowpass filtering
- reducing number of bits

???

I believe, not much.

AFAIK, when encoding to many channels and running out of room, it simply advises the operator to reduce either the sampling rate (ie: 96kHz instead of 192) or to reduce the number of channels.

Sounds like a little naive to me. If this was true, why do the best current lossless codecs only reach compression ratios around 75% for example with metal music?

The other thing which has been said (but I don't know if you noticed) is that it's mathematically impossible to create a lossless codec that always acheives 2:1 compression or better. So, if this is really important to you, you need to understand the limitations, and/or stop wasting your time trying to acheive the impossible!

If you want to avoid psychoacoustics (and the related patents and speed implications) there are "near lossless" solutions out there.

e.g. see http://www.hydrogenaudio.org/forums/index....=ST&f=32&t=8416

These will get you well beyond 2:1 before audible problems set in. Much better than ADPCM to my ears (Depends on content). It's also a much less well researched area, so there are probably new techniques and advances just waiting to be discovered!


If you want to go down the psychoacoustic codec route, as others keep saying, ogg is patent-free and available now.

Cheers,
David.

P.S. compared to the CD bitrate, you get roughly 2:1 compression at a fixed bitrate from NICAM digital stereo. This technology is about two decades old. Which means that any patents must be ready to expire.

I've heard Bob Stuart say that you have the option to reduce the bit depth, bandwidth, or sampling rate on some or all channels.

Whether this is what happens on all software implementations, I don't know. All these posibilities would certainly help the encoder, and all could be done manually on a trial-and-error basis, even with the most stupid imaginable MLP software implementation! I don't know what's actually out there, I only know the theory.


MLP includes a bit buffer, and (theoretically) would only be forced to use it for top resolutions (e.g. 24/96/6-ch). Lower resolutions will fit within the DVD bitrate without compression, though of course MLP allows a disc of any resolution to contain more audio than it could otherwise.


Surely it's a good thing that MLP doesn't automatically do anything to prevent the bitrate limit from being exceeded? Surely if it did do something automatically, this would make it a lossy codec? That's totally against the ethos behind DVD-A. (At least, as far as Meridian are concerned. Verance may have different ideas). It's much better, in the (currently never encountered) situation where the bitrate limit is exceeded, to let the mastering engineer decide what "compromises" to use.

Cheers,
David.

That's correct. As soon as you compress data without loss, regardless of how you do it, you can always find samples which will get inflated by your algorithm.

The art is in tuning the algorithm to be more efficient for "real-life" samples than for all others.

Since all compression can be expressed as "prediction", you can always generate a sample which defeats your prediction all the time.

Edit: this is reply to 2Bdecided's previous post, not the latest one.

Is your 2:1 ratio a best case or the average ratio?
If it is the average ratio, it is a very good performance for a huffman coding.

Hello,

Great news ! I think to use the Levinson-Durbin algorithm, for linear prediction, to increase my ratio compression up 15:1 in my lossy option (with no audio artefact) With an psychoacoustic model, I think I can up to 64 Kbps

For my lossless option, I have no idea to increase the 2:1 ratio, but now I can really stabilise this ratio...

Best regards,

That statement is bold at best!

no audio artifacts at 96kbps? (15:1)

15:1.. who told you that ? Do you even know what the Levinson Durbin recursion is ? (You can use the DSP blockset in Matlab).


Why about you implementing it first ? Then you can say 64kbps is easy.

You can't just start a new project in Visual Studio/Delphi/whatever, then add in an ADPCM encoder, the best psychoacoustic model, some DSP stuff, Levinson Durbin, subband decomposition, MDCT, a Blum-Blum-Shub random number generator, a wavelet-based brownian movement simulator and a Feistel network cipher using a cryptographic hash of the Matrix series, and expect it to blow everything out of the water, now can you ?

Some people just love to make such statements. I can never understand what this means
But interesting thing is that the author of Bonk audio codec (also based on LPC), also claims to reach transparency at 95 kbps.

-Eugene

Hello,

Well, to resume my audio codec :

- lossy and lossless compression
- can convert audio file up to 24 bits, 96 Khz
- multichannel support (up to 7.1)
- lossy compression use ADPCM and huffman coding up to 8:1 for slow computer (Pentium) and Levinson-Durbin algorithm with huffman coding up to 15:1 for fast computer (Pentium II 300 Mhz), for real time encoding or decoding... Huffman and Levinson-Durbin coding are modified by me to stablise and increase audio quality.
- lossless compression can compress up to 2:1 with only huffman coding.
- encoding profiles are includes, to help beginners.
- codec build-in with audio tools like samplerate converter, multichannel mixer, and normalize function...

For the moment, my pre-alpha release can compress and decompress with my lossy and lossless "fast method" and I work harder to implement the Levinson-Durbin algorithm for "slow method". (since few hours)

In the future, I would like to include :

- Psychoacoustic model to reduce more the filesize and compress, in lossy mode, up to 64-80 Kbps in stereo.
- more tools, like pop/click eliminator.
- etc...

To finish, I work on a white paper whitch explain how my codec work really for the moment. In fact, I must search if the algorithm I would like to include in my audio codec is patented or not patented. To have more information, please wait, because my codec is still under construction...

Many thanks,

He says 'almost', which is very subjective and a lot more defensible. Anyway, would be nice to give Bonk a spin.

Christophe, we've been lax on what you've been claiming since you sounded a lot like an enthousiast that has a little clue and wants to learn more, but things are getting out of hand.

I want to VERY STRONGLY recommend you read the Hydrogen Audio terms of service, notably point 8:



Almost none of the claims you have been making is remotely credible. You're arriving here, not showing much clue, but claim to have invented cold fusion in the meantime.

You can't do that here. Show evidence or stop making those claims.

*LOL* (w00t)

yup...

hi, Christophe ! i have an idea !
forget about levinson-durbin algorithm for linear prediction, and forget about huffman trees. why not using the famous jensen's diamond prediction sequence using Suslin w1-trees ? it is much stronger. it had been proven to to be able to predict ANY denumerable set.
i can give you a hand if you like, as that's one of my favourite subjects.


you can find info in here :
http://DSPguru.notrace.dk/diamond.ps

Hello,

Many thanks for your answer, DSPGuru... It's a real answer for me. I have posted my specs on this forum to have many opinions about the algorithm I would like to use for making my audio codec. For the moment, my codec is under construction with a first base. (ADPCM and huffman coding) Off course, it's not the definitive specs... All suggestions are welcomes...

I'm new on this forum and I don't know all your internals "rules". But my place here is only to exchange opinion about algorithm coding. Not to say "my codec is the best" or "your codec is the best". It's not the reality for me...
For example, I use - personaly - the very best MPC format to encode my own CD Audio, but this format is partially patented : it's use MPEG technology. I can't include it on my standalone projet. (in my chip) Why ? because I don't want to paid license to use it. I prefer make my own format, with no patent, but not so powerful.
FLAC and Ogg Vorbis are already two very good formats. But, for industrials, it's not two "serious" formats. (but powerful -- I think) Why ? Because there are no compagny behind to "stabilise" it. (security) It's very important !
For the DivX format is not a problem : Jerome Rota, DivX father, has created this compagny : DivX Network. (not only) Now, many industrials, like Philips or Cirrus Logic, work on a chip for DVD Player. It's the good way to impose your format in the internationnal market. I'm not FLAC or Ogg Vorbis father. I can't create a compagny to exploit this two formats : for me, it's a license violation... And I can support this two formats without the help of the creator... Maybe in the future...

My personal goal is to make my own format, to create my compagny and my chips (with my own technology) for the industrial... It's in a good way to "kill" the MP3 license. I have already many contacts, in France, very interessing by my project. Why ? To create standalone player/recorder, pocket player, etc... Maybe, in the furure, I can include many others formats in my chip (with/without license) but I have not enough money to paid or to support this kind of format. For me, it's only the beginning...

Well, DSP Guru Jensen's diamond prediction is a very powerful algorithm. I'm very interessting to include it in my project... A patent exist ? I's very important for me. I'm very interessting by your help. (I search help - I'm here for that) Have you another intorformations about this algorithm ? (your Postscript file is too short) We can use it with only audio encoding, and maybe with video encoding ?

Many thanks for your answers,

Best regards,

He failed to mention that his codec (when finished) can be implemented in hardware using only 3 transistors and a small amount of cheese-whiz.

I disagree. First, there is a company behind Vorbis and FLAC, called Xiph. Second, having a company behind it does not make it serious or stabilize it.

A big company with deep pockets is required to "defend" a proprietary/patent-encumbered codec but you already said you're avoiding patents. If you want to make a new proprietary/patent-encumbered format it will surely fail without DEEP pockets; you are going up against guys like Microsoft, Fraunhofer, Dolby, etc.

If you are talking about "security" a la DRM, well, no such format will succeed outside a police state. Technology has permanently altered the supply curve for information and time will prove this out. To "defend" your DRM you will again need DEEP pockets.


Not true. First, the format specifications are wide open. Second, the reference code is under Xiph's BSD license variant which does not require permission to use.


To do this you need to make significant advancements over existing formats and/or wield incredible political and economic power.

Josh

It's got nothing to do with best or not. This board requires that you state factual information that you can back up. You still seem to be completely unable to.



It's funny that you appear to be better informed than the author of the format itself. Mind pointing out to us exactly what MPEG patent is being infringed?

The rest of your post is also riddled with fallacies, but I see jcoalson already addressed those.

Beating mp3 will not be that simple if you want to be patent-free. You have to remember that mp3 is the result of about 10years of work by people at Thomson, AT&T and FhG.
But this could be done. As an example Vorbis done it. You just have to remember that this will be very hard.


Sidenote: DivX is not a format created at all. You probably know that DivX 3.x is a hack of an mpeg-4 codec written by Microsoft. Even if v4 and v5 are not warez anymore, they are really mpeg-4. That is why you have some decoder chips. Those chips are not targetted to be DivX decoders, but mpeg-4 decoders.

Hello people,

Well, you can close this thread if you want. Now, I don't think you can't help me to increase the ratio of my audio codec (the subject of this thread) When my codec will be finish, I will post here my first encoder to have your opinion... (I respect your opinion, but I don't know if you respect me and my work)

To finish, I work with compagny like Thomson, FhG and Coding Technology since few years (I know really the license problem) Why ? Because I'm a technical book writer. For the moment, I'm writed two books, about digital recording, in French (yes, really -- two books, and I work on the 3th) I work with the industrials since few years for my job. I know what this kind of compagny want to help you, in your project. In the beginning, I would like to share my experience with you, to help you in the Ogg Vorbis or FLAC hardware implantation...

I'm not only a codec developper. I'm a book writer and reporter.
For example, have you testing the new AAC Plus codec, the AAC format with SBR technology ? Me, yes. For the moment, no player or no recorder for people ; this tool is only reserved for book writer, or reporter... like me... to promote the format. And, it's very impressive !
Another exemple : In my last book, I promote the Ogg Vorbis format. I explain how this format is born, with the help of Christopher Montgomery - the Ogg Vorbis father -, and how to encode in Ogg Vorbis... Monty send me your logo in EPS format to illustrate my book. (and promote your format) For that, I'm your friend...

Now, you can think what do you want about me...

Best regards,

Would be very nice indeed.


http://www.hydrogenaudio.org/forums/index....=ST&f=2&t=10530

-Eugene

We are not trying to bash you.
We are just trying to warn you that some things might be more difficult than what you are expecting.

Claiming that your goal is to reach transparency at 96kbps means that either your are a genius or either that you have to learn a little more about the current state of the art in audio compression.

Some sentences on your website seems to indicate that you do not really deeply know the current audio compression field:

"Please note that the German Institute Fraunhofer (FhG) is a research institute belonging to Thomson Multimedia."
FhG is not own by Thomson MM.

"Technically, joint stereo records low frequencies like monophonic signals..."
In such joint stereo (intensity stereo) it is wise to join the UPPER part of the spectrum

"Above 160 Kbps, proven to be the best quality encoders are Blade, Lame, and Gogo"
Blade is very bad regarding quality, and many people consider that Lame is way better than Blade.

"To conclude, please note that a Japanese web surfer has recently inserted into ACM the source code of the encoder ISO Lame. (Code that I have equally translated in French)"
Steve Lhomme made the ACM version of Lame.


But if you are willing to exchange knowledge (meaning that you are willing to give us some of your knowledge while learning from us in return), you are welcome here.


PS: if you are planning to write another book, I can be technical editor for it. I prefer spending some time on correcting a book than to see this book spreading a few erroneous information once published.

archangel Gabriel

Hello Gabriel,

Many thanks for your answers. Yes, you can read a part of my work is on my website, to promote my books.... for free, off course. (I explain technology process) A part of this work was translated in english by a friend. I don't forget my english reader...

But, you don't known one or two things about me and my work...

- "How to make your own DVD", my first book, is published in 2000 by Dixit, my editor. In France, it's the first book about DVD making... I write it in 1999 and all my informations was right before this year.

- "Digital Recording On CD and DVD", my second book, is published in 2002, always by Dixit. I write it in 2001 and all my informations was right before this year. No error.

Now, your're right. My text has many errors and your correction is right... in 2003 !!! But you must replace my work before the writing (2001) and publishing date (2002)... And don't forget : an editor in general can't publish a new book version when his content is changed by the time events... For me, it's normal... On the web, it's not the same thing : real time update. Don't confuse this two powerful medias...

I suggest you to read the press release about my books... (in french, only)
I'm not a beginner but a real professionnal...

Best regards,

has anyone ever tried making a lossless codec that also uses lossy techniques, kind of like how (I believe, but could be wrong) how 'huffyuv' works with video? I believe huffyuv encodes the frame with jpg and then uses huffman coding on the difference between the original and the jpeg to obtain a lossless copy. I would imagine it would be quite possible to do something like an mp3 or vorbis encode, get the difference from the mp3/vorbis copy and the original copy, and then do a lossless compression on the difference data...but I wonder if it would actually be any good from a size standpoint. If it actually gave anything resembling decent results, sizewise, that would be great. Small lossless audio.

<edit> Small indeed, hahah. OK, I tried a little test:



So, at least with this one song (although it'll probably be the same for any song) the existing lossless codec works better than my idea anyways. Obviously the difference data couldn't really be compressed much better than the original, so the image theory didn't translate over to audio practice well at all. I don't know if using a lossy step other than mp3 would help much or not, it'd probably end up being roughly the same no matter which lossy codec was used I imagine. The difference data was interesting to listen to, from a curiosity standpoint anyways hehe.

If the goal is to make a lossless compression, I think you should use non-psychoacoustic lossy codec. Since psychoacoustic is not aiming at making waveform as close as possible to original but to make waveform that is heard as close as possible to the original.

Hey Christophe ! I don't care who are you and how much books you wrote. I just see the current situation:
you have post a lot of threads with no result at all ! I can ... I wrote ... I want ... I will make ... My codec can ... But it's look like that you promoting yourself and your books. Before post something you better make
at least very-wet-alpha and give it us to try and ask some help or opinions. And I'm realy think that you'll
fail to create something special ...