I've just started working on a project similar in some respects to foobar, but different in a lot of others. I don't want to say to much in case it doesn't go anywhere, but lets just say it has different goals from foobar.

Anyway, I was wondering how foobar stored information in it's database. More specifically, what data structures does it use?

For example, say you wanted to store tag information. You need to store the name of the tag (e.g. "Artist"), the data the tag contains (e.g. "Jeff Buckley"), and In my case I want to store another value (e.g. "blah"). Now this could be done with a multi-dimensional array. In C# this would look like:


In this case, you could store 30 different tags, with 3 collumns in the array.

Here's my (very) crude ASCII table (feature request for next IPB ):

-------------------------------
Artist | Jeff Buckley | Blah
Title | Halleluiah | Blah
-------------------------------

And on it goes...

Problems:
1. Multi-dimensional arrays are slow in C#. Very slow, at least compared to C++.
2. No Type Checking as such. All values have to be strings. This isn't to bad, but would still require some casting.
3. Probably about a million other reasons why this is a bad idea.

So my questions:
1. What sort of data structure does foobar use in it's database?
2. Is it still suitable if another value needs to be stored?

Tags are stored in a file_info class that implements a map from strings to lists of strings. In versions prior to 0.9 this is obfuscated a bit: the implementation maps strings to strings, but allows duplicate keys.

Storing everything as string is not that bad as that is the usual way to store tags in files anyway, except for binary data which is currently ignored. Even if it is supported in the future, it would not be cached.

So you mean something like (where items listed as -> are the mapped list of strings):

Artist
-> Jeff Buckley

Title
-> Halleluiah

Genre
-> Rock
-> Blues

I don't understand how this can work. What about custom field names? I am obviously missing something. And what if you wanted to store more information related to each field. For the sake of an example:

Title; Halleluiah; String
Cover; Blah; Binary Data

You are missing the fact that a custom field name is just another string, and as I said foobar2000 does not store binary data in its metadata cache. It might be possible to load such data on demand in the future. You'll have to figure out for yourself, if that way of handling things is suitable for your own application.

The binary data was just an example. I know foobar doesn't store it in it's cache.

I still don't understand how this works though. You say there are strings mapped to a list of strings.

What data does the first string point to?
What's data does each string in the mapped list point to?

For example. Say there's a string called a1, which is mapped to a list of strings called b1 in some kind of hash table. If a1 stores the field name, then you would need a lot of variables to store each field name and it's appropriate data. Does that mean there is a maximum number of field names foobar can store in it's cache?

Edit: I'm not trying to discover the secrets of foobar so I can recreate my own version or anything. It will be quite a long time before I can develop an appliction anywhere near as sophisticated as foobar. I'm just interested to know how foobar does it.

A key-value pair corresponds to a tag: the key is the tag name, the value is the list of tag values. I thought that would have been clear, but maybe I should have explicitly said that before.

The number of tag names and values foobar2000 can store is only limited by available memory.

Not sure how much help this can be, but Ogg Vorbis comments have ASCII field names, and UTF-8 contents.

Maybe you can gain some insights by reading its spec.

http://xiph.org/ogg/vorbis/doc/v-comment.html

I think I'm starting to understand this.

So, to put it visually, there is a two-column table. Column A contains the key (field name) and column b contains the name of a list (that stores strings, i.e. the field data). If this is correct (which seems doubtful cause my head seems to be falling apart), is it correct to assume that this table and the list (which must be some sort of array of lists) grow automatically, thus limitless field names are possible?

@kjoonlee
That will be extremely helpful later, thanks very much.

Does this mean that prior to 0.9 it was like this:

ARTIST=foo
ARTIST=bar
ARTIST=2000
(edit: somewhat like Vorbis comments)

but in 0.9 it's like this?

ARTIST=foo|bar|2000
(edit: somewhat like APEv2)

I'm making blind guesses, please bear with me

Somewhat like that. The concept can be visualized with pseudocode like this:

Old:

New:

The actual implementation is of course a bit different, but organizes data in the same way.

Cool, thanks foosion. I think I understand it now.

I think the solution for my program will be to write a seperate class that can store three different values, and then create an array of objects of that class for each tag in the file.