Help - Search - Members - Calendar
Full Version: ID3v2 Tag Saving Is Destructive
Hydrogenaudio Forums > Hosted Forums > foobar2000 > Support - (fb2k)
keihin
I'm finding that saving ID3v2 tags using F2k 0.9 is destructive in some cases.

My ripping and tagging toolchain is as follows:

EAC->FLAC->MP3(convert in F2k)->Tag&Rename->ReplayGain(in F2k)

I don't worry too much about tags until after the MP3 encoding step. After that I tag to a combination of ID3v2 and ID3v1 (for backwards compatibility) in Tag&Rename. ID3 embedded album art jpgs are added in most cases. Next I use F2k to scan for replaygain and add replaygain tags. When F2k saves tags back to the files, some of the ID3v2 data is lost. Specifically, the ID3v2 artist, tracknumber and album art data is missing.

Looking at pre/post copies in a binary compare confirms that the data has been changed.

So, my question is, is there a way to have F2k retain the existing ID3v2 tag data when saving replaygain data? Is there any reason it is saving back the entire tagset and not just the new replaygain tags?

Thanks.
lav-chan
I've never run into that problem, actually. I don't use ID3v1, but i have applied ReplayGain to MP3s that had v1+v2 on them (because i downloaded them from somewhere), and it didn't affect the tags at all.
keihin
Good to know it's not happening to you, but it has been 100% reproducible in my tests. Happy to provide sample before and after files if that is helpful.
lav-chan
Well i don't know if i'll be able to help you, since i'm not really an expert, but that would be cool if you could, 'cause i want to try. :x


edit:
Maybe this is a problem with the versions of ID3v2 that foobar and T&R use? I know foobar uses 2.4, but i stopped using T&R when i found foobar a year or two ago, so i don't know what they use (or if their implementation is compliant).
keihin
T&Rs docs claim ID3v2.4 compliance. I don't see a setting to have it preferentially write 2.4 instead of 2.3 tags. I've had no problems with iPod or Slimserver software consuming these tags.

When reopening one of the files written to by F2k, T&R shows the artist and track number correctly in the ID3v1 view (album art missing), but not in the ID3v2 view. Looks like F2k is only saving back the one set of tags instead of both.

Unless there is interest on the part of others in changing the overall tag saving behavior, I'm not too sure that would be the best resolution. To my mind, I wonder why F2k saves back all of the tags when adding RG tags, instead of just adding the delta RG data.
lav-chan
Just to be clear: The tags are missing in both foobar and T&R, right? Or is it just T&R?
keihin
F2k's property window shows a value for artist and tracknumber. The "other info" pane shows "tagtype = id3v2|id3v1". But, it seems to be getting the artist and tracknumber data from the v1 tags. Looking directly at the file data in a binary comparison tool, it shows that the second copy of this data in the tag header is missing after the F2k update. So the data looks OK in the F2k "view", but is truncated for some other uses, such as the case when only the v2 tags are used.
Florian
foobar2000 0.9 writes ID3v2.4 tags where text is encoded as UTF-8 and unsynchronization is applied to all binary frames (like APIC for cover art).

Current Tag&Rename treats UTF-8 encoded ID3v2.4 tags as ISO-8859-1 and ignores unsynced APIC frames. Seems like Tag&Rename only supports a part of the ID3v2.4 standard and you should ask their support staff on how to proceed.
keihin
I'm not sure what you mean by "unsynchronization". Does that mean that this data is discarded? If so, why?

Tag&Rename claims full Unicode support since the v3.2 release. Have you seen evidence that this is not correct in this version?

Would you be at all interested in seeing sample files showing the before and after states?
NogginJ
i dont even use 0.9 yet because of all these little inconsitencies, but i have heard that it only saves id3v2 tags when necessary. so if all of your data fits into the confines of an id3v1 tag, foobar just uses that.

im curious to know, if fb2k decides that id3v2.4 is needed, does it keep the id3 tag too? i have some hardware that only reads id3v1 tags and it would be sad if they all disapeared.

also, maybe you can change how this all works in some preferences?
Mike Giacomelli
QUOTE(NogginJ @ Apr 1 2006, 03:13 PM)
i dont even use 0.9 yet because of all these little inconsitencies, but i have heard that it only saves id3v2 tags when necessary. so if all of your data fits into the confines of an id3v1 tag, foobar just uses that.
*



IIRC, foobar uses whatever tag type is already there. If you have ID3v2, you get ID3v2.4. If you had APEv2, you get that.
HMage
I've ran into another problem - I have a file that's tagged both id3v1 and id3v2, but id3v1 is truncated for one or another reason. foobar reads id3v1 even though more complete set of data is available on id3v2 and is ignored by foobar.

And, some data written by other mp3 managing software is lost during foobar's mp3 tag rewriting, since foobar either doesn't know about them or doesn't take care of them while deciding what version of id3 tag to use. Plus it somehow mangles my tags the oddest way that my traktor evaluation copy loses quite a bunch of info written by it to mp3 files (FBPM, cue points, etc are either rounded to integer or discarded)

And, for some reason foobar adds a square symbol that's visible to latest version of iTunes (v6.0.4.2 for the time of writing) but it's not visible by foobar. Maybe line ending characters?

Winamp v2.95 isn't detecting any id3 tag at all after retagging with foobar2k v0.9
lav-chan
QUOTE(keihin @ Apr 1 2006, 02:07 PM)
I'm not sure what you mean by "unsynchronization".  Does that mean that this data is discarded?  If so, why?
*

I'm not an expert but i think he's talking about the dealie that's applied to ID3v2 frames to make old players ignore the tags.
TrNSZ
QUOTE(HMage @ Apr 1 2006, 06:55 PM)
I've ran into another problem - I have a file that's tagged both id3v1 and id3v2, but id3v1 is truncated for one or another reason. foobar reads id3v1 even though more complete set of data is available on id3v2 and is ignored by foobar.

And, some data written by other mp3 managing software is lost during foobar's mp3 tag rewriting, since foobar either doesn't know about them or doesn't take care of them while deciding what version of id3 tag to use. Plus it somehow mangles my tags the oddest way that my traktor evaluation copy loses quite a bunch of info written by it to mp3 files (FBPM, cue points, etc are either rounded to integer or discarded)

And, for some reason foobar adds a square symbol that's visible to latest version of iTunes (v6.0.4.2 for the time of writing) but it's not visible by foobar. Maybe line ending characters?

Winamp v2.95 isn't detecting any id3 tag at all after retagging with foobar2k v0.9
*


Winamp v2.95 is outdated when it comes to ID3v2 and does not correctly support ID3, and fails to read ID3v2.4 tags all together. This is updated and fixed in the latest versions of Winamp.

I'm going to look into thi to see if I can find any technical reasons why you are having problems with Native Instruments software. What version of Traktor Studio are you having these problems with?

The square symbol seems to be a known issue, but I've yet to see this on iTunes here (but only tested with v6.0.4.3/PPC, at least with files with both ID3v1 and ID3v2.4 tags)! iTunes correctly identifies these files as ID3v2.4. There are no squares in the playlist, at least. I do not use iTunes very much, so maybe I am not looking in the right place.

I also do not think that data should be lost, but when the tag is written in ID3v2.4, the software may not be reading the tag correctly. This is actually very common ID3 issues, and part of the reason why fb2k was not supporting ID3 out of the box in older releases.

The fb2k implementation seems to be one of the most proper taggers (and follows the specs closely). Other software that uses ID3v2.4 properly (such as software using the KDE library) seems to share all the same problems with compatability.
TrNSZ
QUOTE(NogginJ @ Apr 1 2006, 06:13 PM)
i dont even use 0.9 yet because of all these little inconsitencies, but i have heard that it only saves id3v2 tags when necessary. so if all of your data fits into the confines of an id3v1 tag, foobar just uses that. [... snip ...] also, maybe you can change how this all works in some preferences?
*


Sadly, I just read some of the specs, and I think there are real problems with writing out ID3v2.3 tags for compatability as this would introduce even more problems.

To make a long story short, ID3v2.4 allows proper multiple vales for tags. There is no way to avoid data loss when writing older tags unless you somehow combine the values using some (arbritrary/incompatible) method or you don't write certain data at all. The ID3 standard says that you cannot process frames that you don't understand. This would mean a giant implementation supporting tons of tagging types, many of which are incompatible between v2.3 and v2.4. Many frames changed names. Some frames types are supported only in ID3v2.4 and not 2.3, and some only in v2.3 and not v2.4!

Downgrading tags (for example, offering individual ID3v2.4 and ID3v2.3 tag type preferences) is very difficult task since the standard says that you must ignore frames you don't recognize or use! So any specific tags used by other applications or that fb2k does not use could get lost forever if you have to rewrite the whole tag using a different format.

This is the inherent flaw of a bad standard such as ID3v2, and not the fault of foobar2000. If you want to convert ID3v2.4 tags to ID3v2.3 tags, data loss seems practically unavoidable unless you want to end up writing broken tags.

(Actually knowing the data the tag types contain is a must, you can't just replicate what you don't understand, either! There are different text types (UTF-16 vs. UTF-8) and different ways of storing numerical data between these types.)

I hope everything above is correct, and I hope that Florian or someone else with more knowledge can correct me where I am wrong, since this is just from a quick glance at the specs. It really sucks ID3 is such as mess.
TrNSZ
QUOTE(lav-chan @ Apr 1 2006, 07:47 PM)
QUOTE(keihin @ Apr 1 2006, 02:07 PM)
I'm not sure what you mean by "unsynchronization".  Does that mean that this data is discarded?  If so, why?
*

I'm not an expert but i think he's talking about the dealie that's applied to ID3v2 frames to make old players ignore the tags.
*


You can find details at http://support.microsoft.com/KB/814129. This is a patch by Microsoft to add support for unsynchronization to WMP9!

Unsynchronization is required for proper gapless playback, because unless an MP3 implementation knows what an ID3 tag is, without unsynchronization, it will play back as audio data. Unsynchronization is very important to produce files that are not broken but play back properly. However, broken implementations such as Windows Media Player and iTunes have problems working with unsync'd tags all together.

This is what Microsoft says on that page and describes the problem well:

CODE
... Old players that do not understand ID3 headers may interpret an ID3 header that does not have this flag set as an actual audio frame. This can sound like a "pop" to the listener.

STATUS
Microsoft has confirmed that this is a bug in the Microsoft products that are listed at the beginning of this article.


So, I doubt information is deleted, but it appears to be deleted to any software that does not support unsync tagging, which is the proper way to tag, otherwise you risk breaking playback on other players. There is NOTHING in the MP3 specifications that says you have to recognize ID3 tags, BTW. Many of these "old players" are implemented in hardware, so not using unsync is more the wrong way to go.

Software can always be updated or replaced so it works with unsync. Not using it would mean breaking lots of existing hardware.
HMage
QUOTE(TrNSZ @ Apr 1 2006, 06:05 PM)
Winamp v2.95 is outdated when it comes to ID3v2 and does not correctly support ID3, and fails to read ID3v2.4 tags all together.  This is updated and fixed in the latest versions of Winamp.


Allright. Just wanted to do a quick check for old software compatability. As a sidenote, I know quite a bunch of people who stay away from winamp 5 because it's quite cumbersome compared to v2.


QUOTE(TrNSZ @ Apr 1 2006, 06:05 PM)
I'm going to look into thi to see if I can find any technical reasons why you are having problems with Native Instruments software.  What version of Traktor Studio are you having these problems with?


Traktor 2.

As I've checked, it seems that v0.9 is writing id3 tags differently than v0.8. I don't lose exact BPM now. But I still can lose the artist and track name if they're longer than id3v1 permits for some reason.

Exact actions to replicate the behaviour:
1. we have a file that doesn't have any id3 tags at all.
2. write with traktor DJ studio tags to the file, use long title and artist ("a very long string test for foobar's id3 tag handling", for example)
3. reload tags from file in foobar.
4. change first letter in both title and artist to yield possible checks for modification.
5. write tags to file in foobar.
6. read tags from file in traktor.
7. the artist and title strings will be cropped down to "b very long long string test for fo".


QUOTE(TrNSZ @ Apr 1 2006, 06:05 PM)
The square symbol seems to be a known issue, but I've yet to see this on iTunes here (but only tested with v6.0.4.3/PPC, at least with files with both ID3v1 and ID3v2.4 tags)!  iTunes correctly identifies these files as ID3v2.4.  There are no squares in the playlist, at least.  I do not use iTunes very much, so maybe I am not looking in the right place.


I tried to reproduce the issue again:
1. a file with no id3 tags.
2. tag the file with itunes, add artist: "Test artist", album: "Test album", track: "test track". Probably iTunes will add sound check data right after adding the test song to the library and write it as an id3 tag.
3. Read the tags with foobar,
4. add date: "2006".
5. write tags with foobar back to file.
6. start playing back the file. The squares will be on title, album and track.

The issue causes losing the tracks sometimes from the library, and adding duplicates of the same song because, for iTunes, their titles don't match.


QUOTE(TrNSZ @ Apr 1 2006, 06:05 PM)
I also do not think that data should be lost, but when the tag is written in ID3v2.4, the software may not be reading the tag correctly.  This is actually very common ID3 issues, and part of the reason why fb2k was not supporting ID3 out of the box in older releases.


I understand that. But I think it's still can be quite a hassle that unknowingly, by using foobar, an user will suddenly 'loose' the tags on half of their mp3 library after a year, for example.

*
keihin
Here's a followup to the original thread topic.

My issue was in trying to use a tagging toolchain that included both Tag&Rename v3.2 (for basic tags + album art) and Foobar2000 v0.9 for replaygain tagging.

Each of these programs appears to me to the the among the best at performing it's individual task. They just don't work together very well since upgrading to F2k v0.9. As of this release, when Foobar writes the replaygain tags, it creates problems with the album art, artist and tracknumber information.

The software packages that I need to be able to properly consume the resulting tag sets are SlimServer and the iPod firmware.

The reason that this occurs appears to be complex. There seem to be compatibility issues between ID3 tag version (v1, v2.3, v2.4) specs, as well as the individual software implementations of the specs.

The result is simply that the tags generated by one program aren't guaranteed to be interoperable with other programs. The developer of each program seems to genuinely think they've done the right thing, but the result is still the same.

Just in case anyone else has run into this issue, I am sharing a workaround method that appears to solve my problem.

====================================================

Method to use Tag&Rename v3.2 for tagging in concert with Foobar2000 v0.9 for ReplayGain without data loss.

0. Foobar - strip all tags (right click -> Tagging -> Remove tags from file(s))
1. Tag&Rename - tag everthing _except_ album art (use ID3v2 only mode)
2. Foobar - add ReplayGain tags (don't apply to audio data)
3. Tag&Rename - add _only_ album art (use ID3v2 only mode)

Problem:

When Foobar2000 v0.9 writes back tag data after a ReplayGain scanning run, it does so in a way that interferes with the tagging pattern used by Tag&Rename v3.2. This can result in the loss of Artist, Tracknumber and AlbumArt data. I don't fully understand the incompatibility, but have devised the a workaround. Using the above method, I've found it possible to successfully save MP3 files that contain full ID3v2.4 tags, embedded album art and album/track replaygain adjustment tags.
====================================================
NogginJ
that sounds like a real pain in the ass.

i love foobar and was really looking forward to upgrading, but if it is actually destroying tags created by other programs (dj software - djdecks to be exact), then i just cant do it.

are flac tags a problem too?
ryn
I found another way of causing tag destruction when I try to apply ID3v2.4 tags in both iTunes and 0.9. It seems to be dependent on the album art in this case.

1. Stripped all the tags via Foobar. Set ID3v2 only.
When I checked the files in iTunes, all the tags remained except for album art.
2. Tagged files in Foobar.
3. Checked iTunes: Squares were at the end of Artist, Album and track fields and the "total tracks" field was gone.
4. Selected 'Get Info' in iTunes, went to 'Info' tab. Square dissapeared.
5. Readded 'total tracks'. Checked foobar, tag appears. Everything fine.
6. Reapplyed Album Art via iTunes.
7. 'Reload info from file(s)' in Foobar. All tags are gone in Foobar, all tags remain in iTunes.

I was able to get Foobar to read (or copy) all the iTunes tags once but I haven't been able to recreate it yet.

Edit: This even happens when I try to specify APEv2 only. It will wipe the album art when I add tags and when I re-add the art in iTunes it will clear all the Foobar tags.

Edit: I started using Florian's MP3Tag and it works perfectly. I guess I won't be using the masstagger anymore. Thanks.
foosion
QUOTE(NogginJ @ Apr 2 2006, 11:25 PM)
are flac tags a problem too?
*
No, FLAC never suffered from the same problems as MP3 did with ID3, because FLAC actually has a tag format defined in its specification, and it doesn't have the problem of multiple incompatible versions of a pseudo-standard tag format (see ID3v2.x).
keihin
QUOTE(NogginJ @ Apr 2 2006, 02:25 PM)
that sounds like a real pain in the ass.

are flac tags a problem too?


It is kinda painful. It means jumping back and forth between the different programs, refreshing tag data each time. But it gets me the results I want. I make good backups, so I should only have to do it once. And just about a thousand more CDs to go...

FLAC uses it's own tag format. That seems to work w/o issue.
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2008 Invision Power Services, Inc.