Feltzkrone
Jul 29 2003, 23:25
I'm working on a program similar to EncSpot. As EncSpot probably is dead (or isn't it, where's the author?) I've made my self into MP3 tech and already figured out some curiousities for
- XiNG MP3 Encoder v1.0
- BladeEnc v0.94.2
- Canna MP3 Maker v1.1 (rare, I think)
Regarding to the values I collected yet, I can already identify the three mentioned encoders EXACTLY. As I don't see any point in doing such a proggy as "closed-source" and shareware or something else which people have to pay for, I hope we can discuss identification schemes in public and some of you may be willing to contribute.
For me it's not that important to integrate things like drag'n'drop and shell support in general. The main aim is to identify most of all possible MP3 Encoders and to do that task as accurate as possible.
And here are my first questions:
1) Is it enough to decode the header and side info from MP3 frames in order to identify the codec? At least for the three mentioned encoders it's enough, but I still don't get into the detectable differences between all those FhG encoders...
2) May I use information about the frame padding manner which is used? Some encoders start with a non-padded frame followed by a couple of padded frame and others do it vice-versa. That's something which gives a good hint at the possible MP3 encoders.
3) May I use the header flags (copyright, private, original) to identify an encoder? Many (old) encoders didn't allow you to set or clear these bits, so that's quite a good "mark", too.
4) Where can I find information about LAME tags, XiNG VBR Info and FhG VBRI info?
The problem is for questions 2 and 3 is that there probably are utilities which can change this information. For example if you cut MP3s with an appropriate utility I think the padding chain is broken, too. At least this is detectable, so the tool could tell if an MP3 was cut or not, which would be a feature which EncSpot didn't have (as probably any encoder starts with a non-padded frame first or with a complete chain of padded frames a cut would be identified easily).
criZZb
Jul 30 2003, 00:16
Have you tried browsing through the source of Naoki Shibata's mp3guessenc?
Portions of that code were used in EncSpot, AFAIK.
Vietwoojagig
Jul 30 2003, 00:48
There was a commandline-tool of encspot once on the Guerillasoft-Homepage with all the sources, but I cannot find it anymore, since this page has gone. Does someone knew, where to find both (tool+sources)?
Gabriel
Jul 30 2003, 00:52
You should definitively used padding info, and original/copyright flags. It is the way FhG encoders are differenciated.
Feltzkrone
Jul 30 2003, 05:57
Thanks for your info! And thanks for the link to the EncSpot sources (I've never known that they are public)...
Ofcourse I'll have to use information wether padding is used or not, but what I wanted to ask is, if the padding -manner- can be used too.
Here something which I found out for 128kbit MP3s:
(number = amount of consecutive padded frames)
(x = single non-padded frame)
(> ... < = looped till the end)
1) BladeEnc
> 23 x 24 x <
2) Xing (old)
> x 23 x 24 <
3) Cool Edit Mp3 Me! (Fast) + MusicMatch Jukebox 5 (Fast) (FhG)
12 > x 23 x 24 <
4) mp3enc v3.1 (FhG) + l3enc v2.61 (FhG)
> x 24 x 23 <
Well, something special for (3): Both MP3s made from the same wav look very similar. In fact the block types, bit reservoir usage and some other values were identically over the whole MP3, so I guess it's almost the same algorithm which is used here. Note that MMJB5 is quite old compared to Cool Edit's MP3(Pro) Export feature. Seems that FhG didn't change their -fast- algorithm, just slightly. EncSpot identifies them as "FhG (fastenc or mp3enc)". But using the padding manner information we can seperate fastenc from mp3enc. Of course I'll have to try the same for all other bitrates.
Gabriel
Jul 30 2003, 06:01
I was not thinking about the 1 slot padding, but about ancillary data. When you have few information to code, encoders are padding ancillary data with bytes. This padding is different between encoders.
as an example, Lame (recent ones) is using its own name and version info as padding. Some encoders are using 0x00, some are using 0xFF.
Feltzkrone
Jul 30 2003, 09:39
This is what I figured out so far (it still must be proven by testing it on enough examples):
Explanations:
scfs => scfsi
scal => scalefactor_scale usage ($0000=never, $ffff=always)
mode => channel mode (modes 0-3)
mext => channel mode extension (extension 0-3)
pman => padding manner (#non-padded, #padded, #non-padded ......)
flag => P=private, C=copyright, O=original, -=never used, +=always, !=optional
mbeg° => main_data_begin (i.e. number of slots of bit reservoir used)
bigv° => big_values
glgn° => global_gain
r0ct° => region0_count
r1ct° => region1_count
°: 0=average, 1=min, 2=max, 3=usage ($0000=never, $ffff=always)
BladeEnc v0.94.2
----------------
=> pman_128=$00,$17,$01,$18,$01,$17,$01,$18
=> mode[0]=$FFFF (stereo only)
=> emph[0]=$FFFF (no emphasis possible)
=> flag=P-,C-,O+,CRC-
=> mbeg[0]<$040
=> bigv[2]=$120
=> glgn[2]=$0D2
=> glgn[3]=$FFFF
=> r0ct[2]<$F
=> scal=$0000
=> no scfs
Canna MP3 Maker v1.1
--------------------
=> pman_128=$00,$17,$01,$18,$01,$17,$01,$18 (forced)
=> mode[0]=$FFFF (stereo only)
=> flag=P-,C!,O!,CRC-
=> mbeg[0]<$040
=> bigv[2]=$120
=> glgn[3]=$FFFF
=> r0ct[2]<$F
=> scal=$FFFF
=> no scfs
=> 128 bytes missing in last frame
FhG MusicMatch Jukebox v5.0 Level High (mp3enc?)
------------------------------------------------
=> pman_128=$01,$18,$01,$17,$01,$18,$01,$17 (forced)
=> mext[1]=$0000, mext[3]=$0000 (on joint stereo no intensity stereo frames)
=> emph[0]=$FFFF
=> flag=P-,C-,O-,CRC-
=> bigv[2]=$0CB
=> glgn[3]=$FFFF
=> r0ct[2]=$F
=> r1ct[2]=$7
=> scal used properly
=> no scfs
FhG MusicMatch Jukebox v5.0 Level Low (fastenc?)
------------------------------------------------
=> pman_128=$00,$0c,$01,$17,$01,$18,$01,$17 (forced)
=> mext[1]=$0000, mext[3]=$0000 (on joint stereo no intensity stereo frames)
=> emph[0]=$FFFF
=> flag=P-,C-,O-,CRC-
=> glgn[3]=$FFFF
=> r0ct[2]<$F
=> scal used properly
=> no scfs
Comments
------------
1) it seems that encoders relying on ISO never use big values > $120
2) ISO-encoders (canna + bladeenc) don't use bit reservoir properly
3) ISO-encoders don't use joint-stereo
4) it seems that BladeEnc uses a maximum global_gain value of $d2
5) region0_count: seems that some encs always use highest possible value ($0f) and others never use it
6) region1_count: mp3enc seems to always use the maximum of $07
7) global_gain: xing (not included here) is the only encoder which doesn't set values>0 on empty (=silent) frames
8) MusicMatch 5.0 with processing level "low" seems to be the same as fastenc
9) MM50 with proc. level "high" seems to be the same as mp3enc
10) FastEnc uses a funny 1-slot padding method (like no other encoder) and thus could be easily identified (except on 160kbps, you'll see later)
So this is what I basically figured out, and I'm requesting for your comments. I first want to make the pure frame analyizer working (as exact as possible), then, if I'm satisfied with it, I'll spot on VBR tags, ancillary data and things like that.
EDIT: If anybody wonders... I'm not doing this in order to "copy" encspot! The aim is to get more precise and detect more "marks" that encoders put.
FastEnc incorrectly reports joint-stereo in the header for bitrates > 160 kbit/s, while it actually uses all SS frames.
Is there some way to identify Soundjam/iTunes mp3's?
ff123
Feltzkrone
Jul 30 2003, 11:01
Are these encoders available for M$ Windows or DOS?

Either you can send me some example MP3s (that's illegal, isn't it, but I swear I won't hear them

) or links to the encoders (if they are DOS/win32).
EncSpot never used these information, but I'll use, and possibly these can be used to detect the encoders you mentioned:
1) often the big_values maximum values are in certain ranges for certain encoders
2) region0_count and region1_count are sometimes even fixed to certain values
EDIT: My program is now able to detect (and seperate from each other) these enc's and variants...
Canna MP3 Maker v1.1
BladeEnc v0.94.2
Fraunhofer MP3Enc v3.1
Fraunhofer MP3 Producer Pro v2.1
Fraunhofer FastEnc (MMJB50)
Fraunhofer FastEnc v1.02 (the difference is the original flag)
XING MP3 Encoder v1.0
BTW: If you put something in like Lame, Gogo or FhG L3Enc it simply says that the encoder wasn't identified, so the actually used detection routines are somewhat exact (i.e. it hopefully won't tell you encoder XYZ was used if it wasn't).
treech
Jul 30 2003, 11:02
This would make for a really cool foobar plgin, plz consider doing that when you get it going
Feltzkrone
Jul 30 2003, 11:07
Hey, I'll release all info in plain text form if I'm ready. As I'm using Delphi the thing won't be portable. But isn't Foobar a win32 tool? If so, ofcourse I'll do!

EDIT: Another Xing issue...
I have used test files (5 wave files of completly different music styles) and put them into all encoders which I could get, and then I let EncSpot run over these to see what it is capable of.
Two encoders were used:
(1) XING MP3 Encoder v1.0 (1998, windows GUI program)
(2) XING's TOMPG.EXE v3.0 (1997, console program)
Results on (1):
- EncSpot identifies it as Xing (old)
- it never uses other channel mode than plain stereo
Results on (2):
- EncSpot identifies it as Xing (old)
- you have the option to use stereo, joint-stero or dual channel, so...
- EncSpot identifies it as Xing (very old) on joint-stereo mode
- EncSpot identifies it as unknown on dual channel mode
In fact, these results aren't satisfying. After all I had a further look at the values my tool printed out, and it seems like (1) is just a restricted (regarding to channel mode settings) and slightly changed (if more exact in detecting silent sound blocks) alternate of (2). The results will basically be the same, and except channel modes the very same marks are present (padding, big_value maximum etc.).
EDIT2: Okay, here the truth! In fact there are two different XING engines, one completed in 1997 which is used in TOMPG.EXE v3.0 (anybody got an earlier version?) and XING MP3 Encoder v1.0, and the other made in 1999 which is used in Products like XING MP3 Encoder v1.5, AudioCatalyst and Audiograbber AND AFAIK MusicMatch v4.xx
EDIT3: EncSpot issue! Canna MP3 Maker is wrongly identified as FhG (fastenc or mp3enc)! It seems this is due to the fact that scalefactor_scal is used (one of two differences to BladeEnc), but ALWAYS, i.e. for every granule, for every channel. This is the best "mark" one encoder can set, as I've never seen this on other encoders. My tool reports SCAL=$FFFF (=> scalefactor_scale used in 100% of all occurances).
EDIT4: Look at
http://www.mp3pro-soft.narod.ru/download/E.../EncInfo0_1.rar and grab this !attempt! of mp3 encoder identification util. It uses ATL library for Delphi which is available as open-source.
1. The identification is lamely done through frame header flags only
2. The number of frames is calculated by filesize/framesize on CBR, or taken from VBR info tag
3. It's lame (for the above reasons)

4. Excerpt: "From the similar programs differs by an exactitude, simple interface and fast reading of an information from the file."
treech
Jul 30 2003, 12:51
foorbar = win32 app and written in c++
rjamorim
Jul 30 2003, 12:55
I can flood you with old & rare MP3 encoders. I have a pretty big collection here.
If you are interested, just PM me.
Feltzkrone
Jul 30 2003, 14:05
Yet detected encoders list (31.07.03)
Canna MP3 Maker v1.1 [ISO Engine]
BladeEnc v0.94.2 [ISO Engine]
SoloH mpeg Encoder v0.07a [ISO Engine] *added
Fraunhofer MP3Enc v3.1
Fraunhofer MP3 Producer Pro v2.1
Fraunhofer FastEnc (MMJB50)
Fraunhofer FastEnc v1.02
XING Engine v1 (1997)
EmP3-N-Coder (1999) [XINGv1 hack] *added
XING Engine v2 (1999)
BTW: Please note that tests have just been done with 128kbit 44.1KHz Stereo MP3s, so it's not sure that detection will work for other mp3s atm.
Better detection as EncSpot
- seperation of three ISO engines (two detected as Blade, one as fastenc/mp3enc)
- seperation of FhG MP3Enc and FhG FastEnc
- seperation of EmP3-N-Coder (was detected as XING)
- xing (old) and xing (very old) actually is the same engine
Feltzkrone
Jul 30 2003, 17:14
Arsed! 
It's driving me mad. It looks like you really can't decide between FhG FastEnc and FhG Producer Pro (ACM). Since the MP3 Me! Export Plugin(?) for Cool Edit and Nero offer options to disable padding and all other values which may be used to detect encoder marks are very similar between FastEnc and ACM the only possible decision fact is blown away. Wanna see the results of this shit?
1. As I knew that this plugin(?) offers options to disable padding settings each header flag (copyright, original, private) independently I tried to make detection not relying on this.
2. EncSpot relies on 1-slot padding on decision between FastEnc and ACM. And here we go...
=> EncSpot reports non-padded FastEnc MP3s as FhG (ACM or Producer Pro)
=> MyTool does NOT, but it detects ACM files as FastEnc, too
A little bit of tweaking is possible, but it won't work for all MP3 files made with these two encoders. Maybe it's time to figure out new relationships between some values...
EDIT: Simple work-around => if something was detected as ACM first it won't be detected as FastEnc. That's pretty the logic which I didn't want to use, but however. Note that the thing is not working clearly. There may be a few MP3s made by FastEnc which aren't padded (due to that f**** MP3 Me! Plugin(?)) and have a maximum region1_count value of $7 (value $6 is the maximum value for most FastEnc MP3s, but there are few exceptions I guess). So there's a chance of approx 1% that some FastEnc files will be wrongly identified as ACM MP3s, I think only few people will override the plugin's default and switch off ISO padding and I think there are just few MP3s which use region1_count values of $7.
Gabriel
Jul 31 2003, 01:16
When you will test other bitrates, I think that you will also need to know combination that can not happen for encoders.
Example: joint stereo is not possible with ISO encoders.
I think that perhaps you should set up a webpage/textfile somewhere with a big table indicating parameters for each encoder. It would be easier for us to see what you already took into consideration and what you did not.
Misc things: ISO ref code included a wrong CRC method (crc was wrong for each frame if used). It was corrected in a specific Blade version.
Bit reservoir variance for ISO-based should be quite low.
I think that you could gather some interesting info by extracting which huffman tables are used.
Gabriel
Jul 31 2003, 01:31
Here is a list of potential points to be used as identification hints: (to be edited if needed)
*mpeg1/2/2.5
*mono/stereo/dual/ms/is/mixed/forced is/forced ms
*crc on/off/wrong
*long/short/mixed blocks
*use of bit reservoir
*variance of bit reservoir
*sfscale
*scfsi
*subblock gain
*region0
*region1
*use of different block size for both channels
*private bits
*use of huffman tables
*padding method
*ancillary data padding
*original/copyright
*free format by using free format frame size
*free format by regulary alternating normal frame sizes
*max big_values value (8191 or 8206)
*global gain
I would like to test your program...is there anyway you can upload it or something? Thanks
Gabriel
Jul 31 2003, 02:07
Here is a list of known mp3 encoders (to be edited if needed):
Dist10
Blade
Soloh
Canna
Plugger
SCMPX
L3enc 2.0
L3enc post 2.0
Mp3enc
SWA export plug-in
Mp3Producer/Audioactive/ACM
Fastenc
FastEnc+mp3Pro
TomPG
XingV1
XingV2
QDesign
Uzura3
Shine
Lame
Gogo
ARM
Intel IPP
Mpegger
guruboolez
Jul 31 2003, 02:33
QUOTE(Gabriel @ Jul 31 2003, 09:07 AM)
Here is a list of known mp3 encoders (to be edited if needed):
(...)
Where is my favorite one : Plugger™ ?
EDIT : added (message can be deleted)
Feltzkrone
Jul 31 2003, 02:41
Thanks, Gabriel!

1. The huffman table selection (scalefac_compress, table_select) could not give any hint as far as I can see. All encoders almost use the same / don't use the same tables. That's what I've seen so far, and the same is for subblock gain.
2. I primarily want to make a tool for MPEG1-LayerIII files. Once the detection for MP3 files is complete, I may think about adding support for MPEG2 or MPEG2.5 files aswell as other layers (need info about the frame format).
3. Use of region0_count, region1_count, main_data_begin, scalefac_scale, scfsi and big_values and global_gain is already analyzed and used for identification. Aswell as the padding method (1-slot padding) and use of SS/IS/MS/Mixed frames. Private bits were never set (those from the side info) as far as I could see.
4. I have just added SCMPX encoder (thanks rjamorim) to the detection list and it can easily be identified as it is based on ISO BUT used bit reservoir in a normal way(!). Seems that this one is tweaked a lot.
5. What I will have to check (and is not shown by the tool yet, so thanks for these suggestions):
- variance of bit reservoir
- statistics of "empty" frames (especially the "padding" in those)
- freeformat MP3s (can somebody explain the format?)
6. There are some encoders listed which aren't available for Win32 or don't work with Win2k. Any chance of some of you willing to help (i.e. getting five example waves (loslessly compressed) and encoding them at 128k and giving info about the encoder capabilities)?
7. I try not to use frame header flags (private, copyright, original) as these can be altered too easily. As I said before, the MP3pro export thing in plain MP3 mode offers options to set/clear these flags individually and 1-slot padding is optional too. So I don't want the detection algorithms to rely on it. The same is for VBR headers. They will be used if e.g. LAME or FhG fastenc/mp3enc is detected (in order to obtain version numbers and other info), but I won't give them priority as these can be altered, exchanged or generated too easily, too. I even think about adding a header repair and VBRI=>XING conversion function as most players doesn't support VBRI headers, so I don't want to get the tool arsed by itself.
8. What's "different block size for both channels"? Will I have to decode main data? If so, it's probably a difficult thing as I'm using delphi and will have to "convert" C sources to Delphi ones...
But big thanks for all the info! I guess I will either upload a pre-alpha version (user interface will be poor) at the weekend or next week. Please note that I've tested mostly 128kbit MP3s and -some- 192kbit and VBR MP3s. It worked fine for them. I'll release a table what I've tested so far. Still I did not try it on mono and <44.1KHz files, the results for these will be a surprise I guess!

EDIT: Plugger is already detected!
Gabriel
Jul 31 2003, 02:50
QUOTE
1. The huffman table selection (scalefac_compress, table_select) could not give any hint as far as I can see. All encoders almost use the same / don't use the same tables.
It could. I've read several papers about speed-optimized mp3 encoder (mainly for arm cores). A frequent optimization is to not try every huffman table, only only a few ones which should, overall, be the most efficient ones.
I think that perhaps you could discover that some tables are never used by some encoders.
QUOTE
freeformat MP3s (can somebody explain the format?)
Described in the mp3 standard (I am assuming you have a copy).
Explanation: 2 ways of achieving freeformat
*using the free format in the bitrate index
*alternating between standard bitrates. Example: try encoding 20kbps with mp3enc.
Feltzkrone
Jul 31 2003, 03:05
If bitrate index indicates free format, I've read that in fact any frame size may appear and the application has to calculate that frame size N (or N+1 on frames with padding) on its own, right? Shouldn't be that problem.
For the second way of free format: Achieved by alternating between standard bitrates, so each frame should show a valid bitrate index, right? If so, is the "average" bitrate fixed in any case or not? Is just alternated between two bitrates (i.e. if encoded in 144kbps alternating between 128 and 160 only)?
EDIT: Nice info on the huffman table selection! But as far as I have seen, no single encoder I've tried yet uses all huffman tables (perhaps on low bitrates only?). Anyway, I'll take a further look at it.
EDIT2: Feel free to download this pre-pre-pre-alpha at
http://fkr.nm.ru/encscanp1.zip and don't except a user interface, hehe. You can just open one file at a time and scan it. If you encounter any problems (yes, it may crash on non-mp3 files) or wrongly or non-identified (especially these) MP3s feel free to post or PM the log and tell me which encoder was used. VBR headers are detected but not used, just skipped (in order to get real frame information only). Sourcecode included.
Gabriel
Jul 31 2003, 03:15
QUOTE
If bitrate index indicates free format, I've read that in fact any frame size may appear and the application has to calculate that frame size N (or N+1 on frames with padding) on its own, right? Shouldn't be that problem.
In this case frame size is constant
QUOTE
Achieved by alternating between standard bitrates, so each frame should show a valid bitrate index, right? If so, is the "average" bitrate fixed in any case or not? Is just alternated between two bitrates (i.e. if encoded in 144kbps alternating between 128 and 160 only)?
The case I know is mp3enc alternating between 16 and 24kbps frames, in order to provide a "constant" bitrate of 20kbps
Gabriel
Jul 31 2003, 03:17
QUOTE
What's "different block size for both channels"?
Short block for left and long block for right, as an example. Many hardware decoders seems to choke on this, although it is in the standard.
Feltzkrone
Jul 31 2003, 03:24
Thanks! That's all I need to know. For those who didn't notice my previous post, it contains a link to a pre-pre-pre-alpha version of my tool.
takehiro
Jul 31 2003, 05:28
QUOTE(Gabriel @ Jul 31 2003, 05:50 PM)
QUOTE
1. The huffman table selection (scalefac_compress, table_select) could not give any hint as far as I can see. All encoders almost use the same / don't use the same tables.
It could. I've read several papers about speed-optimized mp3 encoder (mainly for arm cores). A frequent optimization is to not try every huffman table, only only a few ones which should, overall, be the most efficient ones.
I think that perhaps you could discover that some tables are never used by some encoders.
Old LAME(before arround 3.80) and many ISO based encoders alyways use "limitted" huffman tables on short blocks. It could be usable to detect them.
rjamorim
Jul 31 2003, 06:49
QUOTE(Feltzkrone @ Jul 31 2003, 05:41 AM)
If so, it's probably a difficult thing as I'm using delphi and will have to "convert" C sources to Delphi ones...
There's a Delphi MP3 decoding component available here:
http://www.sams.co.jp/kei/mp3plyr/manual/manual_e.htmIt's under the GPL.
According to MAD's accuracy table, this decoder is very accurate:
http://www.underbit.com/resources/mpeg/aud...dio/compliance/
Feltzkrone
Jul 31 2003, 06:56
Nice! You really know where to get rarities!

It's huge stuff and the timeconsuming things are precompiled, but it doesn't matter. I'll take a further look. Especially this will be intersting if I want to provide functions like EncSpot's "Play around Sync Error" thing. Thx!
EDIT: I've gone some more into detail. One beautiful characteristic is if a silent granule channel (par2_3_len=0) different encoders may put different global_gain values here. As the global_gain value is redundant here, I suposse it's done the same way everytime (am I right?)...
EDIT2: Found a reliable way to choose between the FhG flavours. Info like padding and header flag bits set won't be needed anymore, just some silent granules (which won't occur on every mp3, but on many!
All right, I tested your program. It works well so far (I tested with cbr 192 and also some vbr and abr files), but I still found some bugs.
I have only one mp3 file where EncSpot says it is encoded with "l3enc“; your program however doesn't find any match for any encoder at all.
It is cbr 192 and has "Joint Stereo".
Another file isn't recognized neither; CBR 192, Stereo, FhG (fastenc, medium or high quality mode).
However, the following file, CBR 192, Joint-Stereo, FhG (fastenc, low quality mode) has been recognized as "FhG FastEnc (2000)".
Keep on your great work!
[Edit]
I also tested some tough ones like --aps (lame 3.92) , "Joint Stereo" , 32khz ->well, the result was "Lame"
Gabriel
Aug 1 2003, 01:55
QUOTE
One beautiful characteristic is if a silent granule channel (par2_3_len=0) different encoders may put different global_gain values here.
Good idea. However, be carefull with it, as it could be altered by mp3gain.
(could this be used to guess if mp3gain was used?)
Feltzkrone
Aug 1 2003, 13:04
Yes Gabriel, it could be used, I guess. As the values set on silent blocks will just be necessary to decide between FhG FastEnc and FhG ACM (not even in every case), once an Encoder is identified by the "normal" characteristic, the tool would know which values should be present on silent blocks for the specific encoder.
Then if they're changed the tool could "guess" that the gain was changed by MP3Gain or similar. BTW: Xing's global gain values are set to zero, so if MP3Gain was used to decrease the value, silent granules are not to be used, but as XING uses a fixed maximum global gain value (I still must analyze this on frame-by-frame) which is used in every MP3 I have scanned yet, I guess THAT will be changed, too. I would just have to improve the identification scheme a way that it doesn't need global gain values. Difficult, but I think this can be done.
@Jojo: For the two MP3s which weren't identified... Is it possible to mail or PM me the log for these? That way I can check what values were expected and which statistical values were calculated (somewhere must have been a mismatch which may have to be corrected, or the MP3s really weren't FastEnc as EncSpot isn't perfect neither and it has it's own bugs, too, as I already found out)...
I have several albums encoded with lame 3.90.2 with -api ( 320kbps joint stereo ) that are not identified as LAME encoder, here there is one log :
CODE
TSIZ=$00000272
FOFS=$00000272
ID3T=$00850780
pman=[02.09][01.09][01.09][01.09][01.08][01.09][01.09][01.09][01.09][01.08][01.09][01.09][01.09][01.09][01.08][01.09]
mode=[0000/FFFF/0000/0000] mext=[F6BA/0000/0944/0000] emph=[FFFF/0000/0000/0000] flag=[P--O-]
mbeg=[14D/032/18C/FFEF] priv=[00/FF/00/0000] scfs=[16DF/4387/4FE7/55AF]
parl=[7DF/30D/CC3/FFEB] bigv=[0DA/03C/117/FFEB] glgn=[9F/50/FD/FFFF]
wsfs=[155B] btyp=[0000/4875/6EE5/48A4] bmix=[0000] sbgn=[X-X]
scfc=[XXXXXXXXXXXXXXXX] tabs=[XXXX-XXXXXXXXX-XXXXXXX--XXXXXXXX]
r0ct=[8/1/F/FAEA] r1ct=[5/1/7/069F]
pref=[0001] scal=[194E] cnt1=[6335]
GUESS => ISO [Canna MP3 Maker v1.1-v1.2] ................ no
GUESS => ISO [BladeEnc v0.94.2] ......................... no
GUESS => ISO [SoloH mpeg Encoder v0.07a] ................ no
GUESS => FhG ACM Codec (1997) ........................... no
GUESS => FhG FastEnc (2000) ............................. no
GUESS => FhG L3Enc (1997) ............................... no
GUESS => FhG MP3Enc (1998) .............................. no
GUESS => SCMPX v1.51 .................................... no
GUESS => XING Engine v1 (1997) .......................... no
GUESS => XING Engine v1 hack (1998) EmP3-N-Coder ........ no
GUESS => XING Engine v1 hack (1998) PLUGGER+ Pro v0.4 ... no
GUESS => XING Engine v2 (1999) .......................... no
GUESS => XING Engine v2 (1999) (no padding) ............. no
GUESS => LAME Engine .................................... no
Feltzkrone
Aug 2 2003, 01:09
Would you mind to tell me what kind of music this is? The reason for not identifying it as LAME is the very high global gain maximum value. I hear loud and (by default) clipped music, too, namely rough black metal (not of CoF kind) but I just couldn't encounter values as high as this one (glgn's third value = $FD). Of course this will be fixed, i.e. I'll kill restrictions for LAME regarding to the maximum value.
Perhaps we can use some values to even give a hint at the musical style. Would be quite a funny thing to play with...

BTW: The MP3Gain detection thing WILL work, I've tested it.
QUOTE(Feltzkrone @ Aug 1 2003, 11:04 AM)
@Jojo: For the two MP3s which weren't identified... Is it possible to mail or PM me the log for these?
all right, you got them! Please keep me posted about the results. thanks
Feltzkrone
Aug 2 2003, 03:19
First file (l3enc, 192cbr, js) suspicious things:
The file mets all characteristics of a typical l3enc file except the fact, that an average bit reservoir is used which is dramatically lower than the minimum value the tool expects. I think l3enc identification of my tool is unreliable as I didn't have that many l3enc-MP3s to scan. Anyway: Sometimes it might help to know which type of music is present in the MP3.
Second file (fastenc, medium or high quality mode):
Two things obviously don't fit to fastenc:
- the padding manner (but this all-frame-padding could have been done by Fraunhofer's latest export plugin used in CoolEdit Pro for example, so this needs to be fixed anyway - but medium or high quality mode isn't used here, so I can't say that this behavior can be shown by some fastenc encoder really)
- region0_count maximum value is 15 ($F), the identification algo expects a value lower than $F (but I didn't test enough files yet, so this may not work for real)
All in all of course the tool needs to be improved. Thanks to your logs I got so far I know more and more things which can't be used for identification. As silent granule channels can be used as a good hint and maybe other characteristics, which I haven't encountered yet, too, I try to do detection in another way:
I won't check the characteristics to fit completely to a given encoder scheme, instead I will use an identification accuracy scale so that the result will be shown as encoder name (the one which fits best to given mp3) plus something like an accuracy value, where 100% means that the util is sure that it's right and any value lower than this will give you a hint, that it's probably right, but it can't swear.
Expect an updated version tomorrow...
Thanks for your fast report. I also got only one file that has been encoded with l3enc. Is this actually a good or a bad encoder? It is one of my favorite songs and I want to have it in a good quality. EncSpot indicates it as green, so I thought it were fine...
Feltzkrone
Aug 2 2003, 08:05
In chronological order I -guess- that this list is right:
(1997) L3Enc
(1997) ACM
(1998) MP3Enc
(2000) FastEnc
Seeing the characteristics of the resulting MP3 files I -guess- that:
ACM (Producer Pro) is based on L3Enc and its faster than L3Enc. Many of the resulting MP3 characteristics values when encoding the same file with both encoders are the very same, so these stay definetly in relation! As ACM is faster than L3Enc and from listening tests of my own I -think- L3Enc even produces better quality than the ACM codec (I think there's more artefacts in 128kbps ACM MP3s than in 128kbps L3Enc files). But in any case, I think that L3Enc is NOT worse than ACM. However... I can't say anything about FastEnc or MP3Enc (which are used in FhG's newest licensed products).
For further info would refer to actual blind test results.
I'm so glad! Because I ain't couldn't find the song nowhere!
However, I got:
23% of my mp3's encoded in FhG (ACM or producer Pro)
7% FhG (fastenc or mp3enc)
70% Lame 3.x
one Gogo (after 3.0) and one l3enc
I found another bug ->see your PM Folder! EncSpot says it is Xing(new), but I hope so desperately that ENcSpot is wrong...
QUOTE(Feltzkrone @ Aug 2 2003, 09:09 AM)
Would you mind to tell me what kind of music this is?
The reason for not identifying it as LAME is the very high global gain maximum value.
That one was "iio - At the end" , it's dance / trance music. It's strange... it is not hard at all, and now that you say it, the files are MP3gained.
It is probably just the consequence of bad mastering we're having here, I assume... Recopilatories of music...
I also habe pretty much "mp3gained" all of my mp3's. And it works finde on most samples...however, I still found some new bugs which I'll send right now to the author since it would take up too much space here in the forum...
Feltzkrone
Aug 3 2003, 04:36
Jojo, here's what I'd say...
[Another thingy thing

]
This file is XINGv2 (or "Xing (new)" in EncSpot language), just the maximum big_values value is too big. Maybe it was encoded by an encoder that's based on XINGv2 but is slightly changed. All other value expectations for XINGv2 were satisfied.
[third attack]
Everything points at Lame but the global gain values are lower than expected. For the difference between Gogo and Lame I still haven't found any way to decide between them, perhaps because Gogo is based on Lame. But I'll take a look at the EncSpot commandline sourcecode to find out how it decides between them.
[one more...]
And again, a global gain issue. I don't know if they were MP3gained...
but as that tool gets more and more popularity, I guess the best thing to do is to kill all expectations for global gain, padding and frame header flags in my identification code. That makes it harder ofcourse, but in the result I want to have a prog which can't be arsed by manipulating 1-slot padding or the header flags. Padding and header flags are number one identification factors for EncSpot, and that's not good. As I said I want to make a tool that's more reliable...
So I'd say the best is to stop the pre-pre-pre-alpha testing (as it's too buggy because it relies on global gain, padding and header flags) and wait for pre-pre-alpha. This will be completed sometime next week, maybe Monday or Thursday, as there's many things to code completely different.
Jojo, you helped me alot by sending me your logs, thanks for that!
Feltzkrone
Aug 4 2003, 15:29
I don't have much time between Monday and Friday, but I have alredy rewritten identification code. It's now based on frame-by-frame analysis, i.e. every single frame is checked for every single possible encoder and is rated from 100% fitting to impossibly fitting.
So in the results table instead of "yes" or "no" you'll find percentage values giving a hint about the reliability of the identification. Good values are those above 90% (in this case it's pretty sure that the analyzed file was made by the encoder suggested). Values smaller than that show that it wasn't possible to make an exact decision. Anyway, the Encoder shown with the highest value _should_ be the one which was used, but if the value for that Encoder is below let's say 70% you really can't trust the result or an encoder was used which isn't supported yet.
I already got weird results for one album where EncSpot said that FastEnc or MP3Enc has been used. Well, for each single MP3 of it I get values smaller than 10%, which means that it can't be FastEnc or MP3Enc. Weird, but the same "algorithm" detected each of every other FastEnc MP3 I scanned so far with values higher than 99%.
Well, the problem is still to decide between the FhG flavours. The best hint now give silent granule channels. And header flags (copyright, private, original) aren't used anymore, so EncScan cannot be arsed!
Another new feature is the option to stop analyzation if one encoder was identified with a value higher than 95% and all others have values that are by at least 33% smaller than the highest one - and the first 512 frames must have been scanned. That speeds up analyzations quite a lot and is still of average reliability (for those who want to scan thousands of MP3s in one step).
Pre-pre alpha will appear at the weekend. And if I've got enough time I'll also include a usable interface (especially for batch scanning with subdirectories).
gazzyk1ns
Aug 4 2003, 18:19
I think I speak for a lot of us quieter posters here when I say that we appreciate all the hard work you're putting into this project - a tool like the one you are developing (i.e. EncSpot but better) will be extremely useful.
Indeed! It will be eXtremely useful.
Unfortunatelly I'm not well-versed enough in MP3 algorithms to help you out, but I will participate in testing.
You´re program is really great. But I noticed that several lame-files are not detected.
I don´t know if you´re already checking it, but I´ve read somewhere that Blade computes all CRCs wrong. Most likely they aren´t randomly wrong, so you could test if a CRC is wrong if it´s "Blade-correct", if yes, its Blade. Also you check for futures like CRC, which encoder allows CRC?
I´m not sure if you doing that, but instead of passing all frames through the same analysis and then output a percent value what could have been used, it would be better (and faster) to change the anylyse every time.
If in frame 1 short blocks are used, it cannot be Xing, so Xing -> 0% and you don´t need to check for xing-specific things anymore. Encspot does that, but very primitive, it scans the entire file and then check. Also it doesn´t work, if you tell the encoder not to use short blocks it will detect Xing/Plugger etc., also it was Uzura3 for example.
Also, for mass-scaning dramtically speed-up: Read lame-tags, so you don´t need to analyse lame-files anymore.
Edit: Removed some chars that very inserted after every "´" in the post.
Feltzkrone
Aug 4 2003, 23:04
Yes, S_O, that's what everybody would do, but I won't for reliability reasons...

1. LAME Tags can be faked (on CBR) and VBR Tags can be calculated for non-tagged VBR MP3 (a function that I'll provide as I hate it playing VBR MP3 files without VBR header for seeking and time calculation reasons). So I will use VBR Tags only if the correct encoder was found and a Tag exists in order to find out the version number for Lame for example. I know that in the actual pre-pre-pre-alpha there are problems with Lame, but these mostly do appear because of unexpected global gain values (which MP3Gain may be a reason for). This will be fixed in the pre-pre-alpha.
2. Blade's wrong CRCs won't be needed as this encoder's MP3s have some special chracteristics which don't appear on other ISO-based encoders. Blade already is well seperated from other ISO encoders.
3. To seperate XING from other encoders it will check if the bit reservoir is used extremely (very high values for XING) and the behavior on silent blocks (if they appear they give the best hint at the encoder). I know LAME may contain no short blocks and thus if they don't appear this won't give "minus points" for Lame rating. Well, you should wait until the weekend. Maybe for the already implemented encoders things will work fine then.
4. FastEnc and MP3Enc are used in FhG's newest export routines (Cool Edit, Music Match etc.) and things are very customizable, i.e. you can set all header flags as you want, enable ISO-padding, no padding or full padding and write CRCs as you want. Same is for Lame, so for FastEnc, MP3Enc and Lame such "marks" can not be used.
In general I try not to use things which can be manipulated too easily, so I won't use global gain values, header flags, padding (except on encoders where padding is forced) and some other things like VBR headers.
Letting fall out some encoders from analysis wouldn't speed up things dramatically. It's a good idea and maybe I'll use it, but I can't say as I think this really doesn't speed up things. (As I said it already stops if it's sure to have found the correct encoder). Anyway, thanks for your suggestions!
Feltzkrone
Aug 9 2003, 09:26
Some news for you: I didn't have much time last days but now I'm working on the program interface. It will be similar to EncSpot: On the left side you'll have a browsable directory tree and on the right side a list of all examined files.
On top of the directory tree there's a checkbox for scanning mp3s not only in the selected directory but in its subdirectories, too (for mass scanning).
The directory tree doesn't have shell support, so you can't do drag'n'drop here or perform manipulation (moving, renaming etc.), but on the other side browsing is already lightning fast and file/directory changes are detected and worked with properly, i.e. if another program (or you) creates/removes/renames directories the changes will appear immediately, so pressing F5 is obsolete and collapsing+expanding subtrees is not needed like in other programs (Nero for example).
What comes next is the MP3 scanning and listing. Together with encoder identification implementation this will take until next weekend, I think. Stay tuned!
EDIT: The interface is ready and does support recursive MP3 scanning (i.e. select a directory and it will scan all MP3s in this dir and its subdirs). I've got two hours to copy'n'pase the mp3 routines into the new gui, perhaps you can download the pre-pre-alpha later...
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please
click here.