Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: OptimFROG DualStream (hybrid like) released! (Read 14733 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

OptimFROG DualStream (hybrid like) released!

Hi!

OptimFROG Dual Stream is aimed at filling the big gap between
perceptual coding and lossless coding. The goal is to offer real
transparent audio coding (not only perceptually transparent) at
half or less the bitrate generally used by lossless coding, and
also to permit progressive consistent increase of the quality
level, until lossless coding is reached.

To eliminate the eternal problem of choosing between lossless and
near lossless, it has an option to create a correction file, which
may be eventually stored separately and used at a later time to
restore the original. The advantage is that the two files, OFS and
OFC have together approximately the size of the lossless coded
original file (slightly greater with up to 1%).

OptimFROG Dual Stream is in alpha stage. Although it was intensively
tested on 39 audio CDs, I am still searching for hidden problems.

It is a branch from the main OptimFROG version 4.506 project, with
modifications to allow the extended functionality and different
file extension (OFS - OptimFROG Small) in order not to mix lossless
and lossy files. I am not sure if it will remain a separate branch
or it will merge into the main project.

Feedback is greatly appreciated, especially problem reports with
the program crashing or worse, producing audio "garbage" (very
bad quality audio files).

I would like to thank to David Bryant, the Wavpack author
for the idea of the correction file (a file which, when used
combined with the lossy file, permits lossless restoring of
the original file).
Although the two programs have common user functionality, they
are based upon very different principles. There is almost no
similarity between their implementation.
Also, thanks for his great program which made me consider
finishing and opening to the public OptimFROG Dual Stream, of
which first working version I did more than a year earlier,
in April 2002.

Download it (Win32 console+foobar2000+Winamp2) from
http://losslessaudiocompression.com/Downlo...m_4507a_zip.php
You may check all the the details directly at
http://losslessaudiocompression.com/DualStream.php
or by visiting http://losslessaudiocompression.com

Florin Ghido

OptimFROG DualStream (hybrid like) released!

Reply #1 –
Hmmmmmmmm. Looks interesting. I'm just downloading a copy now to check it out further. Might post an opinion or two once I've played with it for a while. 

Thanks,

Den.

OptimFROG DualStream (hybrid like) released!

Reply #2 –
Interesting, indeed. Congratulations, Ghido. 
It seems that Mr. Roberto Amorim was right and this is the way for the lossless codecs... 

OptimFROG DualStream (hybrid like) released!

Reply #3 –
Some early test results:

Took bryant's furious sample, which seems to be one of the few samples that Wavpack lossy actually screws up. Wavpack lossy has trouble with this up to and including 400 kbits, but it gets better (not transparent) with 400 kbits and joint stereo enabled.

ofs --quality 0 --mode extra furious.wav, yields a 181 kbit file, that I can not readily ABX...    (No obvious artifacts).

I now have a few other things I want to try. Only one complaint so far...it's very slow, and on my PC, seeking is worse than Wavpack.

 

Den.

OptimFROG DualStream (hybrid like) released!

Reply #4 –
What algorithms are you using in lossy mode? Quantization only, like WV? Some sort of psychoacoustic model? DCT?

@AngelGR: Thanks.

OptimFROG DualStream (hybrid like) released!

Reply #5 –
@Ghido

Looks like Roberto beat me to it, regarding asking you what techniques are used. I can't easily hear additional noise/hiss, like I often do with Wavpack lossy. I haven't extensively tested it yet though... 

Den.

OptimFROG DualStream (hybrid like) released!

Reply #6 –
This sounds awesome.  Great idea, expanding on the wavpack-hybrid theme!

I don't have time to test for a few days, hopefully i can check it out asap.

OptimFROG DualStream (hybrid like) released!

Reply #7 –
Wow this new lossless+lossy is like a revolution! Very exciting stuff indeed

OptimFROG DualStream (hybrid like) released!

Reply #8 –
Sounds very good even with --quality 0. And the udial sample (with the strong high-pitched tones) doesn't fool this new lossy mode at all!

OptimFROG DualStream (hybrid like) released!

Reply #10 –
Well, instead I found another problem sample for both optimfrog and wavpack lossy. It's not quite music but maybe interesting anyway: keys from the pcabx site. Both lossy encoders add a lot of audible noise to the key-jangling.

OptimFROG DualStream (hybrid like) released!

Reply #11 –
Den: I look forward to hearing your results.
superdumprob
____________________________________________

"If we knew what it was we were doing, it would not be called research, would it?" - Albert Einstein

OptimFROG DualStream (hybrid like) released!

Reply #12 –
Yeah ... sounds good ... Idea is good ... BUT after some tests I realize that --seek, --optimize and --mode
keys effect lossy compression time and ratio. These keys work good for lossless but I think it's too much for
lossy because it's hard to determine which combination will give best results. I think Ghido you must simplify
not only lossy but lossless options too. But anyway good job !

PS: does it mean that extension .ofr changed to .ofs ?
      Why in_ofs doesn't play .ofr ?

PSS: Oh Shit, looks like it's needed to use both in_ofs and in_ofr. I think Ghido you must restructure your
          project ...

OptimFROG DualStream (hybrid like) released!

Reply #13 –
Quote
Well, instead I found another problem sample for both optimfrog and wavpack lossy. It's not quite music but maybe interesting anyway: keys from the pcabx site. Both lossy encoders add a lot of audible noise to the key-jangling.

Good sample! It's actually a little like the "furious" sample, but more noise and less tone in the high frequencies.

Using noise shaping in WavPack (-s1) makes this almost perfect, but I don't approve of using custom command lines to generate good results.

OptimFROG DualStream (hybrid like) released!

Reply #14 –
Quote
Good sample! It's actually a little like the "furious" sample, but more noise and less tone in the high frequencies.

Using noise shaping in WavPack (-s1) makes this almost perfect, but I don't approve of using custom command lines to generate good results.


I agree with regards to custom command lines. I know it's a big ask when I have zero experience in programming a codec, but it's best when an encoder can detect special cases and adjust to suit automatically, unless this is deliberately overridden by the user. 

Quote
Den: I look forward to hearing your results.


OK, some initial findings. My initial interest in this was to see whether it transcodes into ATRAC (for Minidisc) as well as Wavpack lossy does. I ripped/encoded/transcoded Blue Monday - New Order and listened to it all the way through, plus conducted some blind ABX tests on the final transcoded ATRAC3 track against that encoded direct from the wavs, ie

1. wav -> DualStream --quality 3 (350 kbits) -> ATRAC3  and
2. wav -> DualStream --quality 2 (318 kbits) -> ATRAC3 and
3. wav -> DualStream --quality 1 (287 kbits) -> ATRAC3 and
3. wav -> DualStream --quality 0 (259 kbits) -> ATRAC3 and
4. wav -> ATRAC3.

I could not ABX with --quality 2 and 3 against the directly ATRAC encoded wav, which is the same for Wavpack lossy @320 handled the same way. At --quality 0 and 1, I think I was occasionally picking up some background noise around some notes, but I was not consistently getting it correct.    So DualStream compares very well against Wavpack lossy for transcoding purposes, and both of them leave Vorbis q10, mpc q10 and LAME insane behind in terms of introduced transcoding artifacts. 

Repeated above with Relax - Frankie goes to Hollywood using quality 2 and 0 (346 - 247 kbits). They all sounded fine on Minidisc and could not be readily discerned from the same direct encoded from wav. 

I then encoded and transcoded 3 CDs across to ATRAC3 via DualSteam quality 2, which generated ofs files between 261 - 324 kbits. (The 3 CDs were Stan Ridgeway - Songs That Made This Country Great, New Order - Substance CD1 and Moby - Play) These were then listened to all the way through, and the quality was excellent, with no noticeable artifacts through relaxed, but attentive listening.  B)

DualStream certainly transcodes very well (like Wavpack lossy) at similar, and sometimes lower bitrates. What doesn't help though is the slow encoding/decoding speed, but this will hopefully improve in newer versions. I can normally decode and replaygain 2 hours 40 minutes of music for MD in about 5-10 minutes with Wavpack lossy. This takes nearly an hour with DualStream on my aging Duron 750 MHz. 

Now for some ABX tests with DualStream directly against the original wavs. Watch this space.
>_<
Den.

OptimFROG DualStream (hybrid like) released!

Reply #15 –
Right. Just took the opening 20 seconds of Stan Ridgway - Lonely Town. This section features a bluesy intro with some sliding and picked electric guitar, and percussion instruments. It is well recorded and has plenty of space in between the notes that allows me to pick up noise or hiss.

ofs --quality 0, resulted in a 251 kbit file. ABX results - 11/15 first time, then 15/15 and 15/15. 
ofs --quality 1, resulted in a 284 kbit file, ABX results - 6/15 first time, then 13/15 and 14/15. 
ofs --quality 2, resulted in a 319 kbit file, ABX results - 8/15, 9/15, 8/15. 

Very nice. At quality 0, there was slight noise around the notes. It wasn't fuzzy, just background noise, and I had to listen very closely to pick it. It seemed to come in and out around each note, rather than as a blanket background hiss like I tend to detect in wavpack lossy. At quality 1, I thought the noise had gone at first, but after I tried harder, my ability to pick it improved.
At quality 2, forget it. It was completely transparent as far as I was concerned.

Just to confirm that my ears were not broken, I also repeated the above with Wavpack lossy.

wavpack @265 kbits (actual was 271 kbits), ABX results 15/15. Background hiss.
wavpack @320 kbits, ABX results 15/15. Background hiss.
wavpack (3.97 experimental) @271 kbits (actual was 271 kbits), ABX results 8/15, 10/15, 10/15.   

Nothing like a little healthy competition. Time for some more testing.  B)

Den.

OptimFROG DualStream (hybrid like) released!

Reply #16 –
Another sample which causes a hiss problem with both DualStream and Wavpack lossy is the opening sequence of Ring Of Fire - Stan Ridgway. I will post it somewhere when I get to my other PC.

ofs --quality 2, 293 kbit file. ABX 15/15, no practice required. Distinct change in the background hiss. Very similar to that from wavpack lossy 3.97 at similar bitrate.

wavpack lossy @265, actual file was 272 kbit. ABX 15/15, again distinct hiss, no practice required to pick it. Same again with noise shaping enabled in the experimental version.

Of the two, preferred the ofs version, as hiss was slightly more subtle, but still very easy to pick.

Repeated with ofs --quality 3, 4 and 5.

ofs --quality 3, 310 kbit file. ABX 15/15 first attempt. Hiss.
ofs --quality 4, 348 kbit file. ABX 14/15 first attempt. Hiss. Getting more subtle now, but still clearly there.
ofs --quality 5, 388 kbit file. ABX 9/15 first attempt. Repeated 11/15, and then 12/15.  Getting hard now. 

wavpack @320. ABX 13/15, then 15/15. Hiss.
wavpack @388. ABX 15/15 Hiss.
wavpack (experimental with noise shaping). ABX 11/15. then 14/15. Much better than the other Wavpack attempts, but still there if you listen carefully.  >_<

It would appear with this sample, (pulsing distorted guitars?) that the hiss is more prevalent with both, but if you throw more bits at it, DualStream comes out slightly better, sooner.

The noise shaping option in experimental Wavpack lossy certainly improves things over straight 3.97, but this is not probably an ideal sample to try the shaping option out on.

Den.

OptimFROG DualStream (hybrid like) released!

Reply #18 –
Blimey Den, you have been busy! Thanks a lot. Interesting stuff.
superdumprob
____________________________________________

"If we knew what it was we were doing, it would not be called research, would it?" - Albert Einstein

OptimFROG DualStream (hybrid like) released!

Reply #19 –
Between DualStream and Wavpack lossy, plus 4.00 on it's way, there is plenty of interesting development going on. By the way, the sample I ABX'ed is here in Wavpack lossless format.. 

Den.

OptimFROG DualStream (hybrid like) released!

Reply #20 –
Quote
Nice work Florin! I've made a front-end for this new encoder:
http://home.wanadoo.nl/~w.speek/download/F...ntendOFR-DS.zip


Nice work Speek! Makes playing with DualStream a whole lot easier... 

Quote
What algorithms are you using in lossy mode? Quantization only, like WV? Some sort of psychoacoustic model? DCT?


Well Roberto, curiosity killed this cat (or dog actually) again and I conducted a little experiment with Wavpack lossy and DualStream. (Love that name...)

Take x.wav and encode with wavpack lossy. Decode resulting file into a wav file (#1), then reencode #1  with the same Wavpack lossy settings. Decode resulting file into a wav file (#2). Compare wavs #1 and #2 with EAC and you will find they are bit for bit indentical.    (HA regulars already know this from atici's post in the MPC section.)

Repeat the above with Dualstream, and you get two identical sized wav files, but when you do a comparison with EAC, they are very different inside.

What does this mean? Buggered if I know, but I guess there is some modelling going on in there somewhere, compared with Wavpack's straight predictor based approach.

Just out of interest, I also repeated the Wavpack experiment with noise shaping enabled on an experimental version, and bingo, they still match.

Fascinating...  B)

Den.

OptimFROG DualStream (hybrid like) released!

Reply #21 –
But wait, there's more.

I can't leave this and the experimental Wavpack lossy encoder alone at the moment.

It occurred to me that the test conducted before is not necessarily fair (apples vs oranges  ), in that it was comparing CBR (Wavpack) against VBR (DualStream quality mode) so I repeated my tests.

Wavpack @270 kbits (actual 273 kbits) : ABX 15/15. Very distinct elevated background hiss. Annoying. 
Wavpack @320 kbits (actual 315 kbits) : ABX 15/15. Same as above, but hiss was not as annoying, still obvious.
Wavpack @400 kbits (actual 400 kbits) : ABX 15/15. Hiss quite soft now, but still readily detected. 

All the above was conducted in high quality mode, no shaping, no joint stereo.

Wavpack X @270 kbits (actual 273 kbits) : ABX 15/15. Very distinct elevated background hiss. Better than Wavpack 3.97 but still annoying.
Wavpack X @320 kbits (actual 315 kbits) : ABX 15/15. Same as above, but hiss was not as annoying, still obvious. Again, better than 3.97.
Wavpack X @400 kbits (actual 400 kbits) : ABX 12/15, then 13/15. Need to concentrate now, but can still be detected if I listen carefully at high volumes. 

Each of these were better than the corresponding 3.97 version by varying degrees. The "relocatable" noise shaping option seems to offer some benefits during my early tests. If you decide to play with it, most music benefits with a -s0 to -s-1 type setting. -s1 has made every sample from my music collection much worse so far. :x

ofs --bitrate 270 kbits (actual 270 kbits) : ABX 15/15. Distinct background hiss. Quite annoying. Comparable to Wavpack X with noise shaping enabled.
ofs --bitrate 320 kbits (actual 319 kbits) : ABX 15/15. Hiss is reduced, but clearly there. Not annoying, but clearly there.
ofs --bitrate 400 kbits (actual 399 kbits) : ABX 8/15, then 9/15. Pretty damn close! 

On a whim, I also conducted "ABX" tests within foobar with Wavpack X vs DualSteam equivalent.

@ 270, could ABX 15/15, Dualstream was slightly less annoying, hiss slightly softer.
@ 320, could ABX 15/15, Dualstream was slightly less annoying, hiss slightly softer.
@ 400, could ABX 15/15, Dualstream by a nose, because I could still barely detect hiss in Wavpack X, but not Dualstream. 

All of these tests were with Ring Of Fire intro. Same as before. Rather than post further test results here, I will probably start a new thread when I have something else to report.

For now, I am staying with Wavpack because of the speed benefits, but will split my encodes 1/3 Wavpack 3.97, 1/3 Wavpack X and 1/3 Optifrog DualStream in order to get a range of comparisons between the three.

Hope this is useful to someone out there.

Den.

OptimFROG DualStream (hybrid like) released!

Reply #22 –
Quote
What algorithms are you using in lossy mode? Quantization only, like WV? Some sort of psychoacoustic model? DCT?

If I got this explanation right, no psychoacoustic or even noise shaping is performed (difference is pure white noise).

@den: could you perform this test with reencoding (for being bit-identical) with CBR OFS files? I think some chances are...

-Eugene
The  greatest  programming  project of all took six days;  on the seventh  day  the  programmer  rested.  We've been trying to debug the !@#$%&* thing ever since. Moral: design before you implement.

OptimFROG DualStream (hybrid like) released!

Reply #23 –
not tech stuff but, .ape tagging support would be good

OptimFROG DualStream (hybrid like) released!

Reply #24 –
Quote
If I got this explanation right, no psychoacoustic or even noise shaping is performed (difference is pure white noise).


It may be just a difference as white noise, but it isn't using a straight predictor method like Wavpack lossy otherwise you would get identical files like you do from reencoding a decoded Wavpack encoded file, if you catch my drift.   

Quote
@den: could you perform this test with reencoding (for being bit-identical) with CBR OFS files? I think some chances are...


... that they are completely different. I did the test with CBR using the --bitrate 320 switch. I just repeated it then for my own peace of mind. Using the quality switch to do this, being VBR, did not make sense.

Just to clarify, I took my test wav, and encoded it using ofs --bitrate 320 test.wav.
Took resulting ofs file, and decoded it back into another wav. (Call this wav2).
Took wav2 and encoded again using ofs --bitrate 320 wav2.
Took resulting ofs file and decoded into another wav. (Call this wav3).

Compare wav2 and wav3 in EAC. Each is completely unique. 

As I said, fascinating.  B)

I'd love to read some comments from ghido on this!

Den.