Another attempt at merging GT3b1 into 1.0.1

Topic: Another attempt at merging GT3b1 into 1.0.1 (Read 7936 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.

Another attempt at merging GT3b1 into 1.0.1

2004-01-21 06:39:12

A lot of us Vorbis users want the best of both worlds in one exe or dll: GT3b1 for high q values and 1.0.1 for low q values. A while ago, john33 tried to merge the sources into one package but there were issues with it. In the meantime, john33 provided us with a dynamic oggenc but for us users of the Vorbis dlls in CDex, its a hassle to swap dlls all the time.

I decided to try my hand at merging Garf's tunings into the Xiph.org 1.0.1 source code. I started with the 1.0.1 code and steadily merged GT3b1 into it, line by line, value by value. My plan of attack was:

1. Do a diff between 1.0 and 1.0.1 to note Monty's quality tweaking in low quality modes
2. Do a diff between 1.0 and GT3b1 to note Garf's quality tweaking in high quality modes
3. Do a diff between 1.0.1 and GT3b1
4. For each different: value or line:
- If it is not part of the 1.0 => 1.0.1 change, use GT3b1
- If it is a 1.0 => 1.0.1 change, but there is no change between 1.0 and GT3b1, keep the 1.0.1
- If there is a conflict (ie it is a 1.0 => 1.0.1 change and a change between 1.0 => GT3b1), merge the two as best I can

The files that were changed include (I didn't include the changes in setup_8.h since I assume that is the 'floggy' code):

modes/psych_44.h
modes/setup_44.h

After a few brief tests, the combined encoder seems to work ok. The nominal bitrates of q > 5 are exactly the same as GT3b1. The only discrepancy is that the resulting files of this encoder are only slighter larger than either GT3b1 and 1.0.1.

Anyway, I'm going to upload the source code and ask john33 to help me compile this into an oggenc for Windows (since I'm using linux for development and I dont know how to build in Windows).

The vendor tag of files produced by this encoder is: Xiph.Org/Sjeng.Org libVorbis I 20030909 (GTune 3 beta 2) EXPERIMENTAL

Once the binaries for Windows are up, can someone test it thoroughly to see if it behaves ok and there is no drastic break in quality? Thanks.

Another attempt at merging GT3b1 into 1.0.1

Reply #1 – 2004-01-21 22:33:26

First of all I would like to thank you and John33 for your effort.
2. Why did you decide to name it GT3b2? There were already two GT3b2.
a. Garf's GT3b2 - unlucky one
b. John33's GT3b2 - has nothing to do with GT3b2 (correct me if I wrong) (as you mentioned there were issues with it)
c. and finaly yours.
I think many people can be entagled with this situation. Why not to give it completely different name. Of course containig mentioning of Garf and you.
For example GT3b1+1.0.1QK
3. What is the matter for the file size difference? Is it constant at different WAVs? If it is constant, can it depend on 'ventor tag' you have changed?

Another attempt at merging GT3b1 into 1.0.1

Reply #2 – 2004-01-22 01:12:23

Quote

First of all I would like to thank you and John33 for your effort.
2. Why did you decide to name it GT3b2? There were already two GT3b2.
a. Garf's GT3b2 - unlucky one
b. John33's GT3b2 - has nothing to do with GT3b2 (correct me if I wrong) (as you mentioned there were issues with it)
c. and finaly yours.
I think many people can be entagled with this situation. Why not to give it completely different name. Of course containig mentioning of Garf and you.
For example GT3b1+1.0.1QK
3. What is the matter for the file size difference? Is it constant at different WAVs? If it is constant, can it depend on 'ventor tag' you have changed?

I named it GT3b2 because that is what Garf wanted it to be named when John33 merged it in. Hence I stuck to the same name, since I'm not doing anything different (ie. not adding any new tunings). If I twiddled with some numbers (which I might consider doing), then I might call it a new name.

As it turns out, the changes I made were exactly the same as what John33 made, even though we both did this independently and without each other's knowledge. It makes me wonder whether two people can make the same mistake independently.

At this point, I don't know why there are these filesize changes. The only variable that conflicted between the two versions is _psy_noiseguards. Since Garf made no changes to this since 1.0, I decided to stick with the 1.0.1 values. John33 also did the same thing. It is not a quality dependent setting so it will ultimately have a global effect.

Anyway, I think the best judge of whether this encoder works is through listening rather than bit-matching. If nothing is broken and it still sounds fine, then perhaps it is safe to use.

Another attempt at merging GT3b1 into 1.0.1

Reply #3 – 2004-01-22 21:10:23

I can upload a win32 binary of this somewhere if anyone is interested. I haven't tested it throughfully (preparing for exams) but quality seems OK. I see an increase of +20bytes between this and a 1.0.1 encode (-q3), although bitrate is exactly same (and switching to original vendor tag doesn't help here). I don't have gt3b1 so I can't say for higher bitrates.

Regards.

Another attempt at merging GT3b1 into 1.0.1

Reply #4 – 2004-01-22 21:44:12

I wonder why there is a need to test? Can't you just do something automated like:

Encode with merged, encode with original/Garf's one (depending on setting), decode both and than compare binary wavs. They should be identical. If you do that for some CDs and some settings (esp the settings wich are switch points in merged library between garf/original) you can be pretty sure, you did it correctly. Of course you need to compile all encoders using the same compiler/settings and use the same decoder. That's it, or do i miss anything?

Another attempt at merging GT3b1 into 1.0.1

Reply #5 – 2004-01-23 03:08:00

Quote

I wonder why there is a need to test? Can't you just do something automated like:

Encode with merged, encode with original/Garf's one (depending on setting), decode both and than compare binary wavs. They should be identical. If you do that for some CDs and some settings (esp the settings wich are switch points in merged library between garf/original) you can be pretty sure, you did it correctly. Of course you need to compile all encoders using the same compiler/settings and use the same decoder. That's it, or do i miss anything?

The problem is there is a conflict in one particular tuning value that is quality independent. John33 and myself chose to stick with the 1.0.1 values and this ultimately will affect the way GT3b1 works. From observations, the merged encoder, when operating in the GT3b1 quality range, will tend to give slighter higher bitrates than GT3b1.

I can confirm that if we change this tuning value (_psy_noiseguards) back to the GT3b1 value, this combined encoder gives exactly the same bitrate and filesize as GT3b1. I just tested it at q 5 and q 6. I will test whether this is the same for the 1.0.1 mode.

Now the question is, if we use the 1.0.1 version of _psy_noiseguards, do we break quality for GT3b1 mode? That is why I want some quality testing. Given that Monty made this tweak when going from 1.0 to 1.0.1, I would prefer to leave this setting in.

Another attempt at merging GT3b1 into 1.0.1

Reply #6 – 2004-01-23 03:42:28

Oops, I just tested 1.0.1 mode at q 0 and it gave different bitrates than the official 1.0.1 and I found the problem. The _psy_tone_0db had the old GT3b1 value at q 0.

After fixing this, this merged encoder gives the same bitrate at q 0 and q 3 as (according to ogginfo) as the official 1.0.1 encoder. q 4 is slightly different.

Just to summarise the preliminary findings:

- This merged encoder gives the same bitrates to Xiph.org's 1.0.1 encoder at q 0 and q 3
- This merged encoder will give slightly different (usually higher) bitrates than GT3b1 at q 5 and 6

Another attempt at merging GT3b1 into 1.0.1

Reply #7 – 2004-01-23 09:36:34

Ultimately the best thing to do is to try it on some problematic clips and note whether it works well. This is probably more interesting that staring at the bitrates. It'll also tell you whether the conflicing value is best set to GT3b1 or 1.0.1 setting.

Also try low sampling rates & bitrates, because that is where john33's previous try went astray.

Another attempt at merging GT3b1 into 1.0.1

Reply #8 – 2004-01-23 13:11:24

A Windows oggenc2.3 compile and a Linux oggenc compile, both ICL 7.1, are now available at the Ogg Vorbis page at Rarewares.

Let me stress, these are intended my testing purposes only. The encoding quality is, as yet, not determined. Tests are needed across the quality range to determine whether these compiles produce output equivalent to v1.0.1 at the lower bitrates, and equivalent to GT3b1 at the higher bitrates.

Testing by those with 'well trained ears' would be very much appreciated.

Another attempt at merging GT3b1 into 1.0.1

Reply #9 – 2004-01-24 01:03:56

Thanks to John33 for putting up the win32 compile on Rarewares.

Let the testing begin.

Another attempt at merging GT3b1 into 1.0.1

Reply #10 – 2004-01-24 06:10:21

Just did a quick rip using the new codec at q8.

On a 11 track album here is what the total file size came in at:

New ogg codec 94,499K

Old ogg codec 94,094K

Diff of 404K

seems like a lot of bites to me....sound quality to my untrained ears seems the same.

rickshaw

Another attempt at merging GT3b1 into 1.0.1

Reply #11 – 2004-01-26 01:24:54

Quote

New ogg codec 94,499K

Old ogg codec 94,094K

Diff of 404K

seems like a lot of bites to me....sound quality to my untrained ears seems the same.

It's only a 0.43% increase in filesize, hardly significant.

Another attempt at merging GT3b1 into 1.0.1

Reply #12 – 2004-01-26 04:16:01

But if it is merged correctly, wouldn't it be exactly the same filesize as GT3B1 at Q8, or does that also depend on how it is compiled?

Another attempt at merging GT3b1 into 1.0.1

Reply #13 – 2004-01-26 04:20:30

Quote

But if it is merged correctly, wouldn't it be exactly the same filesize as GT3B1 at Q8, or does that also depend on how it is compiled?

No, it is not expected to be the same filesize since:

1. As you pointed, it depends on how it was compiled.
2. There is one setting which conflicts between 1.0.1 and GT3b1. I've chosen to stick to 1.0.1 but if the testing proves that its not good, I'll switch it back to the GT3b1 values, which should produce identical files.

Another attempt at merging GT3b1 into 1.0.1

Reply #14 – 2004-01-26 05:06:44

Quote

2. There is one setting which conflicts between 1.0.1 and GT3b1. I've chosen to stick to 1.0.1 but if the testing proves that its not good, I'll switch it back to the GT3b1 values, which should produce identical files.

Is it not possible to use both values and have q5 as the decision level when to use which?

Another attempt at merging GT3b1 into 1.0.1

Reply #15 – 2004-01-26 05:14:47

Quote

Quote
2. There is one setting which conflicts between 1.0.1 and GT3b1. I've chosen to stick to 1.0.1 but if the testing proves that its not good, I'll switch it back to the GT3b1 values, which should produce identical files.

Is it not possible to use both values and have q5 as the decision level when to use which?

I'm sure that can be done, but if it doesn't break quality, then it's probably not worth the effort. The files aren't too much bigger. So whether the quality is the same is the big question.

Another attempt at merging GT3b1 into 1.0.1

Reply #16 – 2004-02-15 22:10:20

QuantumKnot and I have decided that in the absence of any adverse criticism of the quality of the 'test' compiles of GT3b2, it should be released. So, you will find a range of compiles at Rarewares, including an oggenc for Linux.

The versions of oggdropXPd that are offered are v1.7.8. This differs from v1.7.7 in that the Playlist creation definiton has been simplified/improved. Anyone using this facilty will need to amend their current definition otherwise they will get unexpected results.

v1.7.8 of the other versions of oggdropXPd will be available over the next 24 hours.

Another attempt at merging GT3b1 into 1.0.1

Reply #17 – 2004-02-15 23:14:11

Great news

Oh, when you are free, can you make GT3b2 dlls for CDex too?

Another attempt at merging GT3b1 into 1.0.1

Reply #18 – 2004-02-15 23:17:07

Quote

Great news

Oh, when you are free, can you make GT3b2 dlls for CDex too?

Yep, I'll do a full set of everything.

Another attempt at merging GT3b1 into 1.0.1

Reply #19 – 2004-02-16 13:44:41

Availablity of new compiles will be compromised for a while as my ftp access to Rarewares is screwed ATM. If this should become protracted, I'll make them available elsewhere.

Another attempt at merging GT3b1 into 1.0.1

Reply #20 – 2004-02-16 22:53:15

Just for reference, for my entire collection encoded in GT3b2 q5 compared to GT3b1 q5, it only gained 17.6 MB with both sets being around 10 GB in total size, so the "bloat" from that one setting is VERY small. That is a gain of only 0.17% (.0017).

Another attempt at merging GT3b1 into 1.0.1

Reply #21 – 2004-02-16 23:05:11

Quote

Just for reference, for my entire collection encoded in GT3b2 q5 compared to GT3b1 q5, it only gained 17.6 MB with both sets being around 10 GB in total size, so the "bloat" from that one setting is VERY small. That is a gain of only 0.17% (.0017).

As long as it doesn't break quality. That's all that matters

Another attempt at merging GT3b1 into 1.0.1

Reply #22 – 2004-02-24 17:21:17

Quote

Oops, I just tested 1.0.1 mode at q 0 and it gave different bitrates than the official 1.0.1 and I found the problem. The _psy_tone_0db had the old GT3b1 value at q 0.

After fixing this, this merged encoder gives the same bitrate at q 0 and q 3 as (according to ogginfo) as the official 1.0.1 encoder. q 4 is slightly different.

Just to summarise the preliminary findings:

- This merged encoder gives the same bitrates to Xiph.org's 1.0.1 encoder at q 0 and q 3
- This merged encoder will give slightly different (usually higher) bitrates than GT3b1 at q 5 and 6

Obviously then, GT3b2 needs to be tested most critically at q4, correct? Then, of course, q5 needs to be tested to make sure that that one setting doesn't mess with anything.

If successful, the result will be that we will no longer need oggdropXPd V.1.7.7 using GT3b1 based on LibVorbis v1.0 or oggdropXPd V.1.7.7S using GT3b1-based-on-LibVorbis-v1.0 and v1.0.1 (and for some users like me, we won't need v1.0.1 or POST v1.0.1 CVS versions anymore either.) So right now, the different versions available because of transition is hellishly confusing, but if tests are successful with GT3b2, the confusion will be over.

Another attempt at merging GT3b1 into 1.0.1

Reply #23 – 2004-02-26 05:34:57

Quote

If successful, the result will be that we will no longer need oggdropXPd V.1.7.7 using GT3b1 based on LibVorbis v1.0 or oggdropXPd V.1.7.7S using GT3b1-based-on-LibVorbis-v1.0 and v1.0.1 (and for some users like me, we won't need v1.0.1 or POST v1.0.1 CVS versions anymore either.) So right now, the different versions available because of transition is hellishly confusing, but if tests are successful with GT3b2, the confusion will be over.

We released a test version of GT3b2 a month or so ago and so far, we haven't heard of any major problems with quality. So basically it's been released and is the recommended encoder now.

Another attempt at merging GT3b1 into 1.0.1

Reply #24 – 2004-02-26 18:07:47

Yeah, I mean, I am using it on all my music, encoded at q5, but that isn't where the real testing needs to be made. Yeah, sure, we can tell ourselves that it is "recommended" all we want, but just because no one has posted any problems with it could be because no one has really tested it. The Vorbis boards have been REALLY slow lately, and I have seen NO ONE release any tests at q4. q4 is where there is an averaging of Post-1.0.1 and gt3b1. Are we sure nothing screwy is going on between q4 and q5? Has anyone done any ABX testing to compare?

I guess the issue here is that it is going to be difficult to directly compare gt3b2 q4 with Post-1.0.1 or gt3b1 because of the differences in bitrate...

Edit: I notice that the gt3b1 encoders have been removed and the Ogg Vorbis page has been streamlined at Rarewares' new domain, especially focusing on gt3b2. I am glad that everyone "trusts" this new release, as I do. But I still think we should do some ABX testing to be sure here.

Notice