Hi all,

Does anyone know of any studies that have isolated and measured how much the duration of a track is a factor in a listener choosing to skip a song?

ALL other things being equal, it seems to me that track duration ought to have a major effect on listening behaviour:

Let's say a "skip" is only counted if the listener hits NEXT when < 50% of the song has been played.
And let's say a song one likes (cancelling out current mood, listening circumstances etc..) is somehow 2 lengths at the same time (impossible I know), and let's say that hitting NEXT will skip to ANY OTHER song (i.e. random play, so we cancel out "I prefer the one that follows") etc...

This theoretical song (SONG X) is identical in all others ways except duration (somehow ):

SONG X1 = 3 mins
SONG X2 = 30 mins

Clearly it is more of a committment to listen to the 30 minute song. The question is how long does a song have to be before the probability that one will skip to the next/another song increases sharply (all other thing being equal).

My guess (and it's only a guess) is that for me, on average, in leisure, non-rushed listening circumstances, the probability that I will hit next jumps sharply where a track is between 8 and 12 mins long, then increases gradually and probably has another jump at about 20 - 25 mins and is then pretty flat.

Does anyone know of any studies on this? I'm sure there must have been measurements made by marketing types otherwise how else would they know what to ask for from their artists : "Could you do one like just like your last hit, but a bit different and if you could make it 3 minutes and 6 seconds, and include the name Versace that would be really cool" , kind of thing.

The reason I ask is that I'm interested in ways to measure/rate music and counting avoidance of a song (i.e. how often it is skipped) tells you almost as much as counting how often it is played. So it's a useful variable in rating music. The problem is how much weight to give to that variable and how much is it related to duration.

I'd really appreciate any info or leads on this, I've googled, but have found little of use. I keep getting submerged in papers on behavioral shifts due to music therapy and gripes about the annoying behaviour of various audio software.

C.

Your premise presupposes that the listener knows beforehand the length of the track, and therefore the extent of the commitment to see it through. I would expect that in most cases that would not be true; instead the listener would typically allow an intuitive amount of time to go by in order to gauge a) the nature of the song, and whether they like that kind as a generic type and then b) whether they're engaged by that particular performance. By extension, I would expect that listeners who reject a song generically (e.g. I don't like metal) are apt to hit the next button sooner than listeners who reject a song particularly (e.g. I don't like THIS metal song).

Other factors probably include variables such as the emotional temperament of the listener at the time (e.g. cranky, irritable, relaxed, open); the nature of the equipment reproducing the song (e.g. crappy boom box vs. sublime audiophile system); hearsay on the performance (e.g. my girlfriend hates this group); extreme familiarity (e.g. I don't care if I ever hear 'Light My Fire' again), and so on.

In my experience as a producer/director of TV and radio spots, my rule-of-thumb goal is to engage the listener or viewer in the first 5 seconds. Since people are predisposed to dislike advertising, I have a short window to get them interested. I would suppose that for music that window is somewhat longer, but in my personal experience, I know whether I like a song within 20 seconds. If I wait until a minute or two has gone by before I hit the NEXT button it's only because I feel the song has told me everything it can by that point and is simply repeating itself. Or it fooled me and got boring after a great start.

And I, like everyone else in the wired world, just hate to be bored...

Yes it does.

Perhaps I wasn't as clear about the premise, as I thought I had been.

As I said my interest is in relation to rating one's music; so this assumes a personal digital audio library/collection and a software audio player to play it. I'm not quite sure what you had in mind, but for example I know what music I have in my music collection, so if I see for example: "Das Lied Von Der Erde - Abschied" or "When The Music's Over (Live)" I know I'm in for the long haul, because I know the former is 30 mins and the latter is 15, and I know this a) because I've listened to them before and b) because foobar2000 (in my case) tells me how long they are.



"ALL other things being equal, it seems to me that track duration ought to have a major effect on listening behaviour"

Is an important sentence in regard to my initial question.

C.

The problem is that your asumptions are taking "mono-directional" effects into account, but lack analysis of "inter-actions".

Here's one example: your asumption is "all other things being" equal, but this is only possible, if tracklength has only ONE totally isolated and mono-directional effect - namely, the question which you are asking. You are not taking into account, that a change in tracklength may automatically also change those "other things" which are supposed to be equal.

For example, certain kinds of music are "longform".... that is, one can only find out if one likes them, by listening to them for at least 3-4 mins (with the entire track being 10+ mins long). Now, if you'd reduce the tracklength, then the track could no longer have this "longform-style", because this minimum length of track is necessary in the first place, to create this style of music. So if you'd change the tracklength, then you'd basically listen to some other kind of music, and therefore change those other things, which are supposed to be equal.

Therefore, i think that what you could achieve, is reliable stats per "usage-case" - and this usage case may INCLUDE the tracklength, so that you may not be able to reliably draw relations between tracklengths/usage cases, because often, it isnt possible to seperate both from each other.

- Lyx

P.S.: More background-info...

At the root of this misunderstanding, is a flawed model of causality. Most people are using a "mono-directional" model of causality:

A -----influence/message/effect------> B

That is, A and B practically exist in isolated parallel worlds.... A then sends a general message to B, with "reprograms" B. This "arrow/message" is, what those people actually mean, when they say "effect". They asume that a certain "cause" creates a standardized "message/effect"... with its content always being the same, regardless of the target (the "aspect" is ignored). In short, its an isolated mono-directional message-flow.

In a more complex environment, this flawed model then lets people imagine, that there are various isolated "ingredients" which all send a message/influence..... which then create a holistic "result". Imagine various nodes being arranged like a circle, with each having an arrow pointing towards the center where there is another big node called "result". They therefore asume, that because of this the individual "ingredients" and their influences can be isolated. This is, what you are trying to do.

However, truth works quite differently:
1. Other nodes may influence the effect of the node in question.
2. The whole is more than the sum of its parts. That is: the result/final effect... is a synthesis of all influences... if one influence is changed or ignored, then the result may take on a very different character. Or the other way around explained: taking a result and disassembling it into its ingredients, will make you lose something. This doesnt mean that reductionism is useless, but it means that it cannot give you a "complete" answer.
3. The effect also depends on the target, not just on the source - the "aspect" is relevant. A projectile fired at something will create a different result, depending on the target. So, the "result" actually depends on BOTH, the "message-sender" AND the "message-receiver".
4. There is no such thing as "things which are causes" and "things which are effects" - or more specifically, each thing is both. The terms "cause" and "effect" just define an "order"... so what happened first and what happened afterwards. Sounds obvious and therefore boring? Well, it is regulary forgotten: People often look at an isolated POINT in time, where A causes B, and stop right there... thus, not taking into account what happens "afterwards" and what happened "before". Why is that important? Well, only by this tunnel-like view, people come to the errorneus conclusion, that some-thing can cause something, without also being affected by it itself - and the other way around (this indirectly is strongly related to the mechanics of "placebo").

Hi Lyx

An interesting and thoughtful reply, as always.
I'm currently conducting this experiment with myself (using the "forbidden" foo_custom_info ) as this has a %skip% count. What I'll end up with (after about 3 years! ) is a load of data which will include track length and how often played, skipped etc ... I'd expect that when I analyse that data I will be able to correlate track length to played/skipped. i.e. all the other factors will average themselves out.

Now I understand your point, that tracklength is a profound factor influencing the actual piece of music, so "This theoretical song (SONG X) is identical in all others ways except duration" cannot be. One cannot use that as the basis for the "research" because it doesn't exist.

You seem to be saying that because no song is 100% alike (and thus is not 100% equivalent in regard to taste), we cannot isolate track length as a factor and thus you can never know. But, if I find that on average as track length goes up so does the propensity to skip, and if I also find that this pattern occurs regardless of genre, and furthermore when I look at single artists, I see the same pattern. Am I not in this way cancelling out many of the other possible variables, just as the overall average will cancel for mood, and other subjective factors?

Is this what you were referring to here?:

If so doesn't what I said counter that, at least if what I said makes sense?

The reason for my question is that a) I didn't want to wait so many years for the data, and b) I'd rather factor myself out (i.e. it's more interesting if this averaged across a large number of people).

C.

<edit> Ah, my post is prior to your edit - which I'm now reading </edit>

Lyx

I think I understand what you say regarding cause and effect.

Afterall, I used to read the British Medical Journal and ... "80% of smokers also regularly consumed pizza leading us to conclude that pizza was a strong factor regarding nicotine addiction" ...

I did find what you said a little hard to understand due to so many nuanced categories/terms:
"arrow/message", message-flow, message/influence, "message/effect", "effect", "aspect" etc ...

I like to keep thing simple (where possible); in a complex world I understand it's not always possible; but I don't really want to get all philosophical, as that can lead to semantics and playing around with language and defining terms and all that.

Are you saying that trends, like skipping songs, cannot be correlated / identified in any meaningful way?

I'm basing what I'm looking at on my experience: I know that when I'm listening to music the length of the track is a strong factor. For example, if I want to fall asleep to some music, I will use my playlist generator to come up with a random selection of songs BUT I will also make sure none are greater than say 8 mins, because I don't want to be lumped with something I don't feel like listening to for too long, because when I'm lying in bed I don't want to have to get up and press NEXT. Doesn't that very act of setting a "threshold" duration suggest a strong link (at least for me) between track duration and skipping? And if it does, can this trend not be studied?

C.

I think some of this has to do with the individual as well. A lot of people I know have told me that they very rarely listen to longer songs, and prefer the shorter form. I, on the other hand, adore longer songs, and would easily listen to a 30 or 40 minute song. I think this has something to do with me preferring to listen to albums, whereas the current trend is towards listening to individual songs.

I would have to agree. I know long songs and listen to them quite often, I am a Tool fan and many of their songs are in the 8-15 minute range. They even had an album, Lateralus, where the whole thing sounded as if it were one song with small interludes here and there. I don't mind listening to long songs from bands that I like. I recently heard a 15 minute Mars Volta song and I definitely would have hit skip on that one. I think it mainly comes down to the person and their listening preferences. People who listen to the radio a lot would probably skip any song over 6 minutes long as they are used to being fed one track every 3-4 minutes. I myself won't skip a long track. There are even times where I often have to pause the track to turn off my car or turn off my iPod. I start the song back from the very beginning once I turn my car back on or turn my iPod back on. I am all for episodic songs as long as they are done right. Tool and DragonForce are two bands that come to mind but I know that there are others. Korn's cover of Another Brick In The Wall goes for nearly 8 minutes and their Issues album is meant to be listened through all in one sitting from front to back.

I cannot answer this in a "binary way" (so either it is totally possible to isolate tracklength as factor, or it totally isn't possible). What i was trying to say, is that changing the tracklength, may also influence the other factors.... and the degree and probability of this happening, is the bigger the change in tracklength is. For example, inside a certain genre of music, i may have enough after about 5 mins - and adding or removing up to 1 min of tracklength will not change the style of music. With other genres in turn, this "range" may be different. So what i'm warning against isn't your approach in general, but that its dangerous to "escalate" it.

Anyways, it would help if you could clarify the following:
- Which "meaning" are you trying to identify? You are looking at what the listener does, and try to deduct a "reason/meaning" from that action of the listener. You are using it like a symbol: A certain action of the listener symbolizes something. What is unclear to me is: WHAT are you trying to recognize? What kind of information are you trying to deduct?

- For which purpose are you trying to deduct that information? What do you want to use it for?


There are multiple reasons for me being difficult to understand in that regard:

- the existing terms are already used very vaguely and imprecisely. When people for example use the term "effect" then they dont always mean the same thing. If i noticed one thing over time, then its very easy to unintentionally hide errors in language, by common conventions in how sentences are built (we learn certain "slangs" and "sentence-parts" of whom we take their meaning for granted - in short, when we read sentences, we often dont actually analyze their meaning anymore, but just associate certain sentence modules with an idea - sentence-parts become like pictures which are associated with a meaning (so, they become symbols). I however am not using language that way - i do not try to hide those "inaccuracies" but on the contrary, put them right in the front... for example by using different terms interchangeable when what we typically mean with them is identical, even though we asume them to mean something different.

- I often hit the limit of existing languages and find myself describing something for which there are no popular terms. At that point, i start to use a more "metaphorical" and "imaginative" communication style to circumvent those limitations. One of those techniques is pick multiple terms which mean something similiar to what i mean - what i in that case implicitely expect from the reader, is that he someway combines the meanings of those terms, to construct a new meaning for which there is no term yet.

Anyways, the clear up some of the terms:
Whenever something happens - or even whenever any "relation" is established.... you always have at minimum the following scheme:
A sender "A" transmits information to a receiver B. However, A and B are in multiple additional ways affected by this process. For example, if you fire a projectile, then the act of firing it also modifies yourself - even the mere act of THINKING about firing it, changes you (notice the relation with placebo?). Therefore, this whole process can better be thought of like an interaction between A and B - a relationship. Thus, practice looks like A <------> B, instead of A -------> B.

With this in minds, the terms used by me should make more sense.



The link is "the longer you have to endure something which you dislike, the more you are annoyed" - and there being a tolerance timespan "how much annoyance you are willing to accept". In that case, you're basically testing your patience and tolerance towards "bad" things.

Thanks. Yes a degree of caution seems very sensible. I see your point.

In many ways I'm not trying to identify a meaning. I think I've already identified a meaning in relation to skipping a track. I think we both use foobar2000. If I use the foo_playback_custom component (don't want to get into deprecated components! - done that one) the %skip% count function is very well designed IMO. It only counts a skip if the user hits the next button; it doesn't count a skip if the user selects a particular track to play instead while another track is playing. So what does this mean, with regard to automatically RATING music?:

1) Shuffle/Random Play
Clicking NEXT means: I'd rather listen to anything else other than this track at this precise moment in time.

This suggests a clearly negative response to the track that is currently playing.

2) Normal Linear Play
Clicking NEXT means: I'd rather listen to the NEXT track rather than this track at this precise moment in time.

This suggests a negative response to the track that is currently playing in relation to the next track (which is known by the user), unless they've shuffled their playlist, in which case it's like (1).

3) Selecting Another Track (i.e. not hitting NEXT)
Can mean although this track is playing I know exactly what I want to listen to at this precise moment in time.

This doesn't necessarily imply a negative response to the current track; it only really means that it is not the absolutely optimal track that fits one's mood entirely at that moment.

-----

With this in mind, what I'm interested in is some way to approximate the impact of tracklength on the above "meanings".

This all came about through my attempt to automatically rate music. So I came up with foo_DAR for foobar2000 (I am not concerned here about the pragmatic aspects of whether foobar2000 is the right player for such a rating system). The rating manages to handle many variables and I've got very good results. But the one variable that I am really struggling to "interpret" or factor in is SKIPPING. The following graphs show the effect of skipping probability estimations on 2 versions of the ratings formula (based purely on guessing the effect of duration on skipping).

The scenario below is 8 tracks of varying duration all played 5 times. The longer the track the more the probability of it being skipped (guesswork alert! due to lack of data).

NOTE: The reds and oranges are on the secondary Y axis, blues (ratings) are on the primary Y axis.
Song Duration is in minutes.



So here, the linear idea (I had to start from somewhere ) is that "skipping probability" increases by 0.05% per second (and I have to work with averages/assumptions and approximations - afterall Auto Rating, AFAIK is not an exact science - but it is much more accurate than subjective 3 star ratings based on whim - afterall this is based on actual listening behaviour).

Anyway, Version 4 had a decent response to the skipping increase. So here, a 20 minute song played 5 times will also have been skipped 3 times.

But, if I make the formula respond to the dotted line above (i.e. my estimation of my own behaviour) then this is what happens to the ratings (notice I have a problem at 25 mins, but not at 10 mins):



If I had some accurate data I could test the rating's response, but at present I can't; I'm just guessing - not estimating.

I've tested this formula with all the other variables and it's very robust (however, I think one of the reasons I've struggled with skipping is that I'm not quite sure how to treat it, because I'm not quite sure how one acts (I don't even have enough personal data yet - and it's not something that one can synthesise, i.e. it has to be natural real-time behaviour and so cannot be simulated).

This was the purpose of the original question. Because the formula gives weighted value to track duration;
this if from a foo_DAR post in the foobar2000 forum:

"The reason I didn't like the idea was because it defeats the purpose of foo_DAR which is to allow your actual behaviour to inform you about how much you really like a song; i.e. it attempts to by-pass unconscious judgements about a song. For example I was unpleasantly (in relation to how cool I might think I am) shocked about how much I liked Ready For The Times To Get Better by Crystal Gayle. Silly judgement, sure .. but that's why I wrote foo_DAR. I believe it's a better judge than "I" am.

[...] The reason for the duration is that foo_DAR_full has %skip% and the longer the track the more likely it is to be skipped, but I'm now thinking at present this may be a little clumsy and I might need to smooth this somewhat (that's what Version 4 above is doing).

[...] I'll see if I can find a way to normalise this and yet still provide advantage for long tracks like Free Jazz by Ornette Coleman (37 mins). Sitting through that might make me cool but it's quicker to listen to Ready For The Times To Get Better by Crystal Gayle 10 times.

I hope this gives you some idea what I'm trying to understand and why.


Thanks, that's very succinct and clear.



Yes, absolutely. As with (1), (2) and (3) above, all of these are related to testing one's patience and tolerance towards "bad" feelings in the present.

Part of what interested me about Auto Rating music is the futility (and the effort, manually rating 10,000 tracks is not a happy task) of static ratings schemes. Because what I can't get enough of today, I might feel pretty ambivalent toward tomorrow, such is the whimsical nature of mood in relation to taste. However, over time, one's actual listening behaviour tells the user what, in reality, they ACTUALLY like, and this in turn, at least for me, informs ME what I am actually like (versus what I like to think I'm like).

Music is interesting, as it's all about feelings, mood, and many intangibles, but it is also about social acceptance and values. With music as with many things in life, we have the opportunity to learn something about what we are truly like (to understand ourselves a little more) or to deceive ourselves.

If I find that I like Crystal Gayle more than Captain Beefheart, then perhaps I've been deceiving myself for the sake of wanting to be cool (in the eyes of some projected insecurity).

So, to measure this I would do the following:
(AVG foo_DAR rating) x (10000 + the total tracks by artist) and see which was the highest number.

I hope this helps flesh out what I'm trying to do. Ultimately, if I manage to do a really decent job on this auto rating formula I'd like someone to write a plugin that is somehow compatible with foobar or musikCube or whoever can/wants to impliment it. Musikcube, with its SQL db is very suited to such a scheme.

Thanks for your input Lyx, it's really helpful and much appreciated.

C.

I'm sure the broadcast industry has done studies on this (maybe the NAB ???). In the "old days" (AM radio in the 60s & 70s) radio stations wouldn't play anything over about 3 minutes. I don't know what the actual limit was, but they would almost never play a 4-minute song. Sometimes there would be a full-length "album version" and a shorter "single version". But most of the time, the songs on the album were short also. (And yes, there is a scene in the Jimi Hendrix movie where a record producer is forcing him to cut-down his song.)

When FM radio started playing "popular" music, they would play the longer "album cuts" and "album versions" of the hits.

Now days, most songs are slightly longer... I'd guess that the "typical" song is about 4 minutes, and I doubt that radio stations avoid, or refuse to play, songs unless they are over 5 minutes... But, I'll bet most radio stations do have song-length limits.

Nope, if the skip happens quickly, then the user may simply be navigating the playlist with prev/next hotkeys instead of via the mouse. I strongly suggest ignoring any "skips" which happen in the first 10 secs.

Yes, i know - that would take away exactly the range, which can also tell you the most if you dislike a song. However, with so many variables and aspects in place, which can easily result in misinterpretation of user-behaviour, i'd rather follow a "conservative" approach.... that is: sacrifice flexibility for stability.

I'd also think about if such a complex system as you are developing is actually necessary. Perhaps not weighting speed of skipping at all, just using a static modifier, regardless of skip-position..... and then only counting skips in the range between 00:10 to 04:30, would already be enough. (if you want extra robustness against misinterpreting "navigation" as "dislike", use something like 00:15 - 04:30).

Here's more food for thought: One weakness of all auto-rating systems which i've seen, is that they do not put a softlimit on the score which a track can gain in a given period of time. In other words: they weight shortterm "likes" too high. How much score a track gains per play, should be dependend on how long ago the previous play was.

Yes, I agree that makes a lot of sense. I've for some time wanted Peter to introduce both a more advanced %skip% and %play_count% whereby users could set such variables. But I hadn't thought about a range within a track. I like your suggestion.

Like I say I don't want this to become a foobar post, which I am in danger of making it, but do you think a plugin is possible in a player such as foobar which does all the external calculations outside the core and simply supplies the player with the result?

And if so would that get round the titleformatting and system date issues discussed over at the foobar forums?

That's as OT as I'm going to drift.

Again, thanks Lyx for your input.

C.

Yes. You aren't even restricted to tags and configfiles for storing relevant data. A plugin can for example use an SQLite DB for storage. Its all just a question of how much effort one is willing to invest. Foobar components can do nearly everything imaginable, if one is willing to invest the required effort. The inefficiency of existing 3rd party plugins comes from an overly pragmatic "patchwork-mentality"... that is: just think about how to get what one wants as quickly as possible with as little shortterm effort as possible, by someway patching together existing elements. Outside of this patchwork-mentality, nearly everything is possible.


Yes, as hinted at in my response above.

- Lyx

Recording behavior is simple, one pushed a button, play, next, fast forward, etc.
How to map this to a rating system is dark, you know the behavior but you don’t know the reasons.
Makes me wonder what would happen if you simply start with a data logger.
Each time a song is played, a log entry is made say:
Titel (ID), Date/time, duration (percentage of track duration?)

So you build a database you can analyze and derive a rating for each song. As indeed, if one played a song 50 times but the last time was 2 years ago, appreciation obvious is waning. Probably using the time dimension is more fruitful than the skip function in establishing a rating.

I cannot speak generally, but from my own experience, the track length has next to nonexistent impact on whether I press next or not. Perhaps even the weather has greater impact on that. But I'm definitely not an average Joe when it comes to genres...
I can imagine that a person brainwashed by commercial pop junk (usually heard on most-listened radio stations) might actually lose patience when exposed to anything over 4 minutes, though.
I think this much more depends on what lentgth is typical for the genre of the song. People listening to techno mixes sometimes don't even have the option of skipping a track, how would you take this into account?

Exactly.

Yes, I agree. As Lyx said, as long as this occurs outside the core of a player you can pretty much run some entirely complimentary package. Unfortunately I am not a programmer. I just wanted to rate my music and when I found I'd done a decent job, I thought I'd share it.

They're both useful. The foo_DAR rating "algorithm" uses both (as well as other variables). Have a look here if you're interested (also bottom of 2nd post has more info). It also penalises tracks you add to your library but take a long time to get round to playing, the penalty is then worked off the more frequently the track is subsequently played.

Like I say I'm pretty happy with the variables on offer. You can do (and I think I have done) a lot with:

Date Added to Library
Date First Played
Date Last Played
Total Plays (user defines % of track played before a play is counted)
Total Skips (user defines time range for a skip to occur before a skip is counted) *
Time Now
Track Length

And if you look at them they are in the main profoundly unambiguous. Except SKIP!

Really many players could probably quite easily act as a data logger (foobar has many components that really do just that) that is why I chose foobar.

What I'd like to see is pretty much a combination of a) what Lyx said was possible, and b) Roseval's data logger. A stand alone app, that collected stats (would be cool if it could plug in to foobar, musikcube, winamp, and a few others) and allowed users to write almost MS Excel style calculations. Then it outputs the result to the player, the player picks it up as just another piece of meta-data. The point here is that the rating is dynamic and time related, so the player is always picking up changes to the rating; i.e. if you are away for a few days your ratings will all have dropped because there's a bigger gap between Date Last Played and Now.

I see Martel's point, user's are different. I can sit through very long tracks too, but also if I'm not in the mood to listen to something, duration certainly has an effect. e.g. I may not be in the mood for Bartok's Allegretto from 15 Hungarian Peasant Songs, but it's 25 seconds long! It will pass very soon. As said above Ornette Coleman's Free Jazz will take 37 minutes to pass.

Clearly, also a ratings system has to take into account track duration, which foo_DAR does; I could listen to that Bartok piece 89 times, before Ornette has finished blowing his horn.


You could write a routine that treated pre-specified genre's differently. For example if the genre tag read "techno mix" an alternative rating formulation could be applied. But clearly it's not possible to please all the people all the time.

The problem for me remains.
The relationship between SKIPPED (counting (1) and (2) from above post) PLAYED and DURATION.

For example, say the full extent of the rating formula was:
PLAY COUNT - (SKIP COUNT / 2)

Is that overvaluing a SKIP or should I divide by 3?

That's the crux of the first problem.
The second problem is: Does SKIP increase with Track Duration? In my experience yes, others say no. The only way to tell is with some quantitative data.
Thanks again to everyone, for your very helpful feedback and ideas.

C.

What about something like PLAY COUNT - (SKIP COUNT * (TIME NOW/DURATION))
So a skip counts 'stronger' the later it occurs

As we do have Duration and PlayCount we have
TotalTime=Duration*PlayCount

If we would record at tag level something like
ListenTime=ListenTime+TimeNow
We have the actual listening time and as this could be easily done at tag level, are not in need of a database.
Maybe the difference between the 2 could be used in some way as a weight factor taken both skips and duration into account

Totally agree. However this kind of problem is most apparent when a new track is added - you can have a brief infatuation with the track and this has to be "softened". foo_DAR does this, I think quite well. As you can see below, to gain the equivalent rating of a track 10 months old played 40 times, a new track (of the same duration etc...) would have to be played 16 times in the first month. The Play Per Day burden on tracks in the 1st month is approx 4x (and in the 2nd month is 3x) that of 10 month old tracks.

However, what I was after in a rating system was something which allowed your present top rated songs to compete with your old favourites; thus IMO you don't want to surpress too much the effect of a current infatuation with an old song. Because, getting back to what I said earlier, I want to know what I really like (and what I'm really like), right now, in the present.



I don't see what TIME NOW means. Because clock time is useless unless it is acting as a range, i.e. the difference in seconds between TIME THEN and TIME NOW or something like that. Also you have to remember that SKIP COUNT is a recorded history of skips, so applying a PRESENT event to all PREVIOUS occurances is problematic.


Again, I don't understand the function of TimeNow. Surely ListenTime (however that is recorded) = ListenTime.
But the idea of recording actual time spent listening to a piece (perhaps in conjuntion with Play Count) might be handy and would remove the problem of SKIPPING (by ignoring the problem), also it gets rid of the problem of using track duration as a variable (again by ignoring it). I guess you could also ignore Play Count, as ultimately you are recording what % of the listener's life has been given to listening to this piece of music - and that's not a bad measure of how much you like a song. Part of the problem of Play Counts and Skip Counts is their binary nature. Whereas simply noting total time spent listening to X piece of music might be quite a nice approach.

NOTE: The rating system I designed was done within the practical confines of what integer data foobar2000 could capture and allow to be manipulated in a formula.


I've always been against holding such info in Tags. The reason being that you are constantly writing to Tags, this job IMO is better left to databases, which are designed for constant Read/Write access, I don't like messing with the actual file itself. Afterall audio files were never designed to be perpetually altered; whereas databases are. Also when you want to do a backup you are backing up all the info in one file. If you want to backup your new tag info, you have to backup Gigabytes of data.

Thanks Roseval. I like the idea of using this as the basis of a ratings system:
<edit>Thanks Roseval. I like the idea of using this as the "Play Count substitution" element for a ratings system: </edit>

TOTAL LOGGED PLAYING TIME (secs) / TRACK DURATION (secs)

This gives you a precise Play Count and ignores Skipping altogether and also gets round the Play Count issue of "how much is a song played before it is counted as played".
The only negative is this:
You have a 10 minute song, you listen to the last 5 minutes 10 times and never listen to the first 5 minutes:
It's counted as played 5 times.

But I don't have a problem with that and also it solves Martel's issue with Techno Mixes.
You could move throughout the Audio file listening to various bits and it's all counting toward how much you like the Techno Mix.

This would make foo_DAR better and would solve the Skip issue.
The only problem is finding a player / ratings / data logger (or whatever) application that records total listening time by track.

I'd be interested in Lyx's opinion as to whether a foobar2000 ratings plugin could manage this.

Once again thanks to everyone for their input, I'm starting to feel that recording total listening time is the way round the SKIPPING ISSUE and also PLAY COUNTING too. Killing 2 birds with 1 stone would be very nice.

C.

Being unfamiliar with Foobar I thought TimeNow in post 17 was the time played but you already understand what I mean, using the cumulative actual playing time.

A DB is the best solution but probably harder to implement. All players can do some simple counting like PlayCount=PlayCount+1 and write it in a tag.
For simplicities shake I suggested to use the same mechanism for PlayingTime.

In case of a backup, this is a pain in the ass. My player (WMP11) writes the PlayCount in a tag so when doing a backup, all songs played since last backup are in the new backup.
(Funny, if you delete WMP’s library and do a rescan, it won’t read the PlayCount)

Oh, I see ...
Sorry I wasn't clear. With "Time Now" I meant literally the current time (as in "system date/time").
This was something that was registered by a recently discarded component of foobar. It's required to calculate the difference in days between, for example, LAST PLAYED and NOW.

But yes, I got what you meant from the other stuff. It's a good idea.

In isolation that's fine, but the "algorithm" is doing lots of calculations and for efficiency's sake you really want the source data to be in one place.

Here I agree with Lyx, you want to do all this stuff outside the player's core, and also I much prefer a database over tags, as I said, predominantly due to the data backup issue, but also read/write issues.

C.

I don’t mind the read/write nor the backup issue much. Tagging is tagging so it is build to change and a 1000 Mbs network is faster than my hard disk.
But if you can log every user action you record detailed behaviour over time. This is a far better data source than a simplistic score like a cumulative one as Playcount. That’s the disadvantage of using tags, you lose the history.

I agree with this that a "balance" between the two extremes is needed. However, the optimal point in-between may also depend on the listener. So in an optimal case, a slider which allows the user to set this balance himself (Score weighting: prefer longterm favorites <-----> prefer current favorites) , and which can update the scores in realtime, without invalidating previous data, would be part of the system.



You mean not doing PLAYS * TRACKLENGTH but actually logging how much a track has been listened to? I'm not sure if this can be done with ONE component. The problem is how to hook into the actual playback of the track and get notified of starts, pauses, skips, etc. - and at the same time provide other services for rating. I'd estimate that it is possible, but i do not know how difficult to do it is. One thing which is certain however, is that IF you get that info, you're gonna try to save the collected data as fast as possible - which basically means, that whenever a track gets paused, stopped, skipped you will update the stats. Thats indeed a frequency of data-writes, for which a DB is better suited than tags.

As for the general question as to where to save such data..... opinions on this differ alot. My personal take on this is, that tag-metadata has NOT been designed with "user-specific" scenarios in mind. It has no standardized method to associate metadata with a user, so that for example two users can store different values to the same field. Plus, with the "social-networking" culture rocketing (yes, i'm talking about sharing files.... not just on P2P, but also i.e. at home), combined with using the same files on multiple hardware (thus making the "transport" of data an issue).... i do not really find storing user-specific data in the musicfiles themselves attractive.... i'd rather have some kind of standardized external "user-DB files" whichs metadata is not linked to audiofiles via filepaths, but instead via other means (so that filepaths may change, with the DB-metadata continueing to work).

Yes, exactly. As you can see it bypasses the whole SKIP issue.
I think the way round the constant stats update issue, is to have some kind of buffer. So that the component or whatever it is, gathers the information and processes it in the background, but only outputs the ratings to the player every x mins. Otherwise it's putting too much of a load on the player. In many ways I'd like it to be a manual update thing, like foobar's "Rescan Media Library", ReScan Ratings with an option to Rescan Ratings on startup or Rescan Ratings every hour. Either way this should be a low priority background process, otherwise it's taking CPU away from the player and other services.

Agree 100%.

Thanks again for the input. I've just integrated this idea into the my test sheet (Excel) and it works really well after initial testing.

It uses the following variables. As you can see no Play Count or Skip Count.

DAD = Days Since Added (to Library)
DLP = Days Since Last Played
DFP = Days Since First Played
TPT = Total Playing Time (total recorded playing time in seconds)
TL = Track Length (in seconds)

C.

Hi carpman,
this looks quite interesting! Specially because I'm also developing a AutoRating-Script for MediaMonkey since two years. Looks very similar by the way...

Are you maybe interested in a little 'joint venture'?

Hi Big Berny

What do you have in mind?

I don't know a great deal about MediaMonkey (aren't there 2 versions?) - I tried it not too long ago but found it too rigid - interface wise (that said it was a pretty brief test).

I'd be impressed if it was capable of recording total played time per track? (i.e. listen to a 2 minute segment one day and a four minute segment another and it records 6 mins played).

What variables does your script use?

Personally I'd really like to get this into some kind of outline (ultimately I'd really like it to work with foobar, as well as any other players that are capable). Results are looking pretty good, the problem is implimentation.

How's things with your script?

I'm working on some user defined variable at present as per what Lyx said:

Worked on this last night; there are 4 sliders that would allow the user a substantial degree of control.
I'd also like a save settings and an easy way to switch back to default settings. This is important for trouble shooting or sharing ratings etc.

Will post results when processed.

C.

Notice that if you are saving too much data (i.e. one DB-entry per play-stop period), you can reduce the amount of data for the price of less accuracy and a minor performance penalty. For example, you can have one DB-entry per track per day.... and then apply the scaling in realtime when reading the data. That would mean a massive reduction in data, for the price of not being able to differentiate the weighting inside of a day (but who needs that?), slightly higher CPU-use and not being able to merge stats from multiple DBs at finer granularity than a day (this affects syncing DBs).

Interesting thread.

While I agree that skip is a factor, I also submt that it can not be considered in a vacuum by itself, and that 'all other things being equal' simply does not exist.

Big Berny already has been working long on his auto-rate script, and there is also a different auto-rate script which has since seemingly fallen out of development and does not work with version 3 of MM (Diddeleedoo's auto-rate script).

Both scripts have/had some type of skip detection, which while not a native MM functionality, can be approximated via scripting.



Unfortunately, this is not tracked. However MM does keep a played table which tracks all completed play's history which can be used to build various historical views. Bex has an excellent play history script which can provide an enormous amount of info on your play history. It's like having last.fm charts but at a much greater detail.

The threads of the 2 mentioned auto-rate scripts, possibly already have some discussion/opinions on this particular topic (skip factor) and general auto-rating factors you (or others) might be interested in.

Big_Berny's Auto-Rate Accurate
DiddeLeeDoo's Auto Rate and Announcer

Well, no. Only version 3.0 has been developed since the last year. I like it because it's very easy to create scripts which can be quite complex. But it only works with MediaMonkey. That's why I'd also like to create a plugin which is supported by different players. Unfortunately I don't have enough programming knowledge (except for Delphi maybe) to create such a plugin. At least I'd need some help.

At the moment played time is not recorded but I can implement it quite quickly. Because the script is already prepared for that feature due to the SkipCounter I already created and works very similar. I think I'll add that also in a future versions, could indeed be very useful. At the moment I use Played, Skip, DateAdded, DateLastPlayed, DateFirstPlayed.

EDIT: The PlayedTime-counter is implemented.

But the special thing behind the script is IMHO the autocalibration it has. This means that you define in the options how much of songs (in percent) you want to have with which rating. IF you set for example 5 stars to 5%, then always the 5% of songs with the highest point-value (calculated by the formula) will get 5 stars. This way you don't have to modify the ranges by hand after a while when the point-value of each songs increase.

The thing I want to do for the next version is also to create a linear formula which easily can be modified with sliders. Maybe we also could try to find a good formula. One idea I also have and probably would be possible with an external plugin is that formula is based on the ratinglogic of the user. So you rate some songs from your library and the plugin then tries to find out which variables are how important to you. This could easily be done with a multivariate regression analysis.

Big_Berny

Big_Berny

It looks pretty sophisticated. I'm pretty sure that such a scipting language imposes far less constraints than foobar's Titleformatting language which obviously was not meant for such calculations (as the name implies it was mean to format titles); this has led to a departure in foobar2000's recent development, though I have no doubt a far more suitable alternative will replace the dreaded Title Formatting "code".

A couple of interesting things have been raised in this thread:

1) SKIP COUNT
This, as has been pointed out, is pretty ambiguous. So my feeling is that if it is to used it has to be used with caution and should only have a subtle negative influence on the ratings.
2) TOTAL PLAYED TIME
It seems we both like this as an important ratings variable (perhaps the most important)?
3) RESULT DISPLAY
This is an interesting area raised by your "Auto-Rate Script".
I personally like an accurate numeric value, clearly others like a symbol-based (stars/hearts) display.
An option to show either one or the other or both would be nice.
eg. foo_DAR gives a rating between 00001 and ... well, anything over 10000 is good something around 11000 is the highest I've got so far.

One thing I quite like about this is that ratings are universal, so, in theory if everyone selected a default setting they would be able to understand each other's ratings. Just a thought.

Good, I can leave the complicated stuff to you .

I'm more than happy to post an outline of how the ratings are calculated in foo_DAR (or rather how they would be if it could calculate total play time). Also you're welcome to have a look at the test spreadsheet and test it out, see if there's any ideas in there you like. Like I say it's not rocket science, but it does a lot with a little and most importantly I've been very pleased with the early "field test" (which mirror "modelled" testing) results.

DAR should really become: DADAR (date and duration adjusted ratings), or DADA Rating.

Personally, although I like the slider idea of your's and Lyx's to give a degree of tuning over to the user, I also like the idea that it should just work, and regardless of listening habits, should reflect the user's REAL likes and dislikes. That, IMO is the purpose of an Auto Rating algorithm - to accurately reflect what the user actually likes (rather than what they think they might like - which is what manual star ratings do).


@ Lyx ---- Didn't understand this bit: "and then apply the scaling in realtime when reading the data".
But overall I'm pretty sure I agree. Certainly there is no need to have date related calculations going any finer than units of days.

C.

You know, there is one thing that will throw off probably any type of auto-rate add in..

That alot of people simply play on shuffle/random with no weight being made of ratings or other play history.

Of course there are various implementations of 'smart' shuffle for different apps, but really I would expect that the manual selection of tracks is less common than whatever shuffle mode is available to a user of a particular app.

I myself mostly use a shuffle (script) based on last.fm relationships (both directly related tracks and/or related artists's tracks).

Perhaps my use scenario is not common, but what I would expect of any auto-rate based upon my listening is going to be spread between what I specifically choose to play, and popular/related tracks/artist. Depe

Which is fine with me, but this is because I am aware of the skewing this has on auto-rating.

Well IMHO the PlayCounter is more important than Total Played Time. Because I for example always listen to albums, so the PlayCounter of the songs is often the same. But that doesn't mean that I like longer songs more! But since I also want to pay attention to when a song only has been half I replaced the variable PlayCounter with (Total Played Time/Songlength). So I get values like 4.3 when a song has been played a bit more than 4 times.
About SkipCount you're right. That's also why the SkipDetection can easily be disabled in my script. And it only counts a Skip when at minimum 10 seconds and at maximum 80% of the song. Since the formula can be modified very easily you also can disable the influence of Skips.

And my script also is able to store a point-value in a customfield although the values are a bit high and don't look so nice like yours.
The idea behind the ratings is that they are also visible on portable players. For example on my ipod. So I can create a playlist on my ipod where I always find all my moste preffered songs with 5 stars. And the problem with point-values is IMHO that you have very low values at the beginning with a fresh database. But in reality that oesn't mean that you like non of your songs. You know what I mean? That's why my script compares all songs and rates the songs with the highest points with the highest ratings. If the points are only 100 or 100000 doesn't matter.

And compared to you I don't think the ratings are universal. Each person has its own listening habbits. So the formula to get the ratings (or points) isn't the same for all people and has to be adjusted. For example some people skip a lot of songs because they change the song quite often. Other people skip versy seldom and if they really skip it means that the son't like the song at all. So IMHO the formula has to be adjusted to the user to show what he REALLY likes. I also not interested in what the user might like but what he actually likes. Different views here?

@Teknojnky: I also use Auto-DJs but IMHO it doesn't affect the autoratings too much because 1. I also skip a song if I don't like it and 2. I don't use Auto-DJ all the time, mostly I use my ipod. So at least with my listening habbits it has not a big effect. But I understand that this could be a problem for some users...

Big_Berny

Which is using Total Played Time as a variable.
Precisely. Total Played Time/Songlength gives you a more precise Play Count.
So no disagreement there.

Not necessarily - that all depends on the algorithm.
The foo_DAR ratings for example will probably vary (depending on size of collection) betweem 7000 and 11000.
A new 3 minute track played only once after the first 30 days since added (and only played the day it was added to the library) will have a rating of: 10005.

I'm not sure. It looks to me that the two ratings systems are taking a different approach; probably equally valid. What would be fun is to run them side by side. Equally you might find interesting things happen when you use one to weight the other or simply multiply one by the other.

I quite like the idea of an application / component that allows the user to choose from a variety of ratings schemes. Cleary there's no absolutely right answer to the ratings problem. topdownjimmy wrote a hotness algorithm for foobar2k, again this is doing something completey different but equally valid.

C.

Ok, so your ratings aren't really universal either? If the ratings depend of the size of the library...

Well my gain just is to create a "ratingbot" which is able to rate like the user would. Like AI. I know that this is not completely possible but I think it could work quite good. And you can calculate the validity (grade of quality of the prediction) quite good.

I also think that your formula easily could be added to my script. You'd like to try? That's the Formula I use at the moment:

Could you translate your formula in similar terms or is it not possible?

Big_Berny

No they are. The reason I said it depends on the library is obviously if someone has 100,000 tracks and someone else has 1000 track and they both listen to their music for 2 hours a day. The person with 1000 tracks will generally have higher ratings, purely because he is recycling the same tracks more often. However, if they both added, the same track, at the same time and listened to it the same amount etc ... they would get the same rating.

C.

I've translated the foo_DAR formula, and have included a number of proposals regarding what I'd like it to do as a component, here.

C.