Hotness TF code translated for foo_dynfil, Has anyone done this already, if not what is wrong with my effort? Options

[carpman]

I thought I'd test the Hotness code in conjuntion with DAR. The idea being to output the DAR rating as a number between 1 and 5 and send that to Hotness (instead of Hotness using %rating%). I thought this would be trivial, but for some reason I can't get any sensible results from the Hotness version I've done for foo_dynfil. It seems to output large numbers until "rawhotness", after which it's just zero.

So, if anyone's already done this, I'd really appreciate it if you could post your code here [big thanks in advance].
Ultimately if there is a version, or if we get one done here, I'll post it on the hotness thread.

Alternatively, if someone fluent in titleformatting and familiar with Hotness (and foo_dynfil) could look over these two versions (see below ORIGINAL VERSION and DYNFIL VERSION) I'd appreciate it.

Here's topdownjimmy's Hotness code from here:

CODE

///////////////////////////////////////////
// HOTNESS - an algorithm for meta-rating
// v1.7.c (foo_cwb_hooks version)
// July 8, 2007 - by topdownjimmy@gmail.com
///////////////////////////////////////////
//
// configure baselines: define baseline playfrequency and decay period (in days)
//
$puts(baselinefrequency,90) // decrease if songs stay hot too long, or: high to accentuate success, low to accentuate recentness
$puts(baselinedecay,28) // decrease if too many songs are hot, or: high to accentuate success, low to accentuate recentness
//
// configure playback statistics
//
$puts(lp,[%last_played%])
$puts(fp,[%first_played%])
$puts(pc,[%play_count%])
$puts(rating,[%rating%])
$puts(avgrating,3)
//
// DO NOT EDIT BELOW THIS LINE //
$puts(baselinefrequency,$mul($get(baselinefrequency),24))
$puts(baselinedecay,$mul($get(baselinedecay),24))
$puts(lp_age,$add($substr($get(lp),12,13),$mul(24,$cwb_datediff($get(lp),2000-01-01))))
$puts(fp_age,$add($substr($get(fp),12,13),$mul(24,$cwb_datediff($get(fp),2000-01-01))))
$puts(age,$sub($get(lp_age),$get(fp_age)))
$puts(now_age,$add($substr(%cwb_systemdatetime%,12,13),$mul(24,$cwb_datediff(%cwb_systemdate%,2000-01-01))))
$puts(recentness,$sub($get(now_age),$get(lp_age)))
$puts(decay,$div($div($mul($get(pc),$get(baselinefrequency),$get(baselinedecay),$if2($get(rating),$get(avgrating)),100),$mul($max($get(age),$get(baselinefrequency)),$get(avgrating))),100))
$puts(rawhotness,$div($mul($max($sub($get(decay),$get(recentness)),0),100),$get(decay)))
$puts(forecast,$div($mul($max($sub($get(decay),$add($div($max(0,$sub($get(baselinedecay),$get(recentness))),2),$get(recentness))),0),100),$get(decay)))
$puts(hotness,$div($add($get(rawhotness),$get(forecast)),2))
$set_global(hotness,$get(hotness))
// END HOTNESS //

ORIGINAL VERSION
Here's the same code without the annotations (for easy comparison to the dynfil version):

CODE

$puts(baselinefrequency,90)
$puts(baselinedecay,28)

$puts(lp,[%last_played%])
$puts(fp,[%first_played%])
$puts(pc,[%play_count%])
$puts(rating,[%rating%])
$puts(avgrating,3)

$puts(baselinefrequency,$mul($get(baselinefrequency),24))
$puts(baselinedecay,$mul($get(baselinedecay),24))
$puts(lp_age,$add($substr($get(lp),12,13),$mul(24,$cwb_datediff($get(lp),2000-01-01))))
$puts(fp_age,$add($substr($get(fp),12,13),$mul(24,$cwb_datediff($get(fp),2000-01-01))))
$puts(age,$sub($get(lp_age),$get(fp_age)))
$puts(now_age,$add($substr(%cwb_systemdatetime%,12,13),$mul(24,$cwb_datediff(%cwb_systemdate%,2000-01-01))))
$puts(recentness,$sub($get(now_age),$get(lp_age)))
$puts(decay,$div($div($mul($get(pc),$get(baselinefrequency),$get(baselinedecay),$if2($get(rating),$get(avgrating)),100),$mul($max($get(age),$get(baselinefrequency)),$get(avgrating))),100))
$puts(rawhotness,$div($mul($max($sub($get(decay),$get(recentness)),0),100),$get(decay)))
$puts(forecast,$div($mul($max($sub($get(decay),$add($div($max(0,$sub($get(baselinedecay),$get(recentness))),2),$get(recentness))),0),100),$get(decay)))
$puts(hotness,$div($add($get(rawhotness),$get(forecast)),2))
$set_global(hotness,$get(hotness))

DYNFIL VERSION
Here's my attempt to write it for foo_dynfil:

CODE

$puts(baselinefrequency,90)
$puts(baselinedecay,28)

$puts(lp,%last_played%)
$puts(fp,%first_played%)
$puts(pc,%play_count%)
$puts(rating,%rating%)
$puts(avgrating,3)

$puts(baselinefrequency,$mul($get(baselinefrequency),24))
$puts(baselinedecay,$mul($get(baselinedecay),24))
$puts(lp_age,$add($substr($get(lp),12,13),$mul(24,$date_diff($get(lp),2000))))
$puts(fp_age,$add($substr($get(fp),12,13),$mul(24,$date_diff($get(fp),2000))))
$puts(age,$sub($get(lp_age),$get(fp_age)))
$puts(now_age,$add($substr(%now%,12,13),$mul(24,$date_diff(2000))))
$puts(recentness,$sub($get(now_age),$get(lp_age)))
$puts(decay,$div($div($mul($get(pc),$get(baselinefrequency),$get(baselinedecay),$if2($get(rating),$get(avgrating)),100),$mul($max($get(age),$get(baselinefrequency)),$get(avgrating))),100))
$puts(rawhotness,$div($mul($max($sub($get(decay),$get(recentness)),0),100),$get(decay)))
$puts(forecast,$div($mul($max($sub($get(decay),$add($div($max(0,$sub($get(baselinedecay),$get(recentness))),2),$get(recentness))),0),100),$get(decay)))
$puts(hotness,$div($add($get(rawhotness),$get(forecast)),2))
$get(hotness)

What am I not getting here? Any help is greatly appreciated.

C.

EDIT: grammar.

This post has been edited by carpman: Mar 1 2011, 16:57

[q-stankovic]

I tried that also in the early stage of foo_dynfil but without any luck. I just replaced the date functions and variables from cwb_hooks with foo_dynfils expressions. No idea!

[Yirkha]

$puts(lp_age,$add($substr($get(lp),12,13),$mul(24,$date_diff($get(lp),2000))))
$puts(fp_age,$add($substr($get(fp),12,13),$mul(24,$date_diff($get(fp),2000))))
should be
$puts(lp_age,$add($substr($get(lp),12,13),$mul(24,$date_diff(2000,$get(lp)))))
$puts(fp_age,$add($substr($get(fp),12,13),$mul(24,$date_diff(2000,$get(fp)))))

Also I think it might be faster to do just
$puts(lp_age,$div($time_diff(2000,$get(lp)),3600))

Moreover, as long as you don't have everything rated, it's necessary to change
$puts(rating,%rating%)
to
$puts(rating,[%rating%])

But whatever, it still seems to suck :B

[carpman]

Yeah, the problem seems to be that "recentness" is always > "decay".
This is the kind of number I get for RECENTNESS ... 195744
This is the kind of number I get for DECAY ... 2016

That's got to be wrong, because the "rawhotness" line does this:

CODE

$puts(rawhotness,$div($mul($max($sub($get(decay),$get(recentness)),0),100),$get(decay)))

DECAY minus RECENTNESS = <0
So MAX of 0 or <0 = 0
Then multiplies this by 100, so 0 x 100 = 0
Then you get 0 divided by "decay", which I guess fb2k will call 0.
So rawhotness = 0, then everything that follows is 0.

This suggests to me that the output of recentness is wrong. But I don't know what it's supposed to be unfortunately.

The only thing I can think of is that:
$substr(%cwb_systemdatetime%,12,13) <> $substr(%now%,12,13)

It would be helpful if a hotness expert could chime in.
Anyway, thanks Yirkha and q-stankovic, it's good to know it's not me.

C.

[Yirkha]

Yeah, the problem seems to be that "recentness" is always > "decay".

What? It's not. For only about ~11 % of all tracks here, but not never. I have a lot of tracks with hotness 0, then it goes gradually up to 92. Is the output value meant to be in percents?

This is the kind of number I get for RECENTNESS ... 195744
This is the kind of number I get for DECAY ... 2016

195744 is way too big. Have you switched the order of parameters in $date_diff() as I have written above?

This suggests to me that the output of recentness is wrong. But I don't know what it's supposed to be unfortunately.

What?
$substr(2000-01-02 12:34:56,12,13) == '12'
now_age = $substr(%now%,12,13) + 24 * $date_diff(2000,%now%)
lp_age = $substr(%last_played%,12,13) + 24 * $date_diff(2000,%last_played%)
recentness = now_age - lp_age
Hmm, what could that be? Maybe number of hours between now and the time it was last played?

The only thing I can think of is that:
$substr(%cwb_systemdatetime%,12,13) <> $substr(%now%,12,13)

What? As long as the "common fb2k timestamp format" hasn't changed, it must be exactly the same.

I'm just saying that a playlist ordered by this doesn't seem like a hotlist to me or anything.

CODE

$puts(baselinefrequency,90)
$puts(baselinedecay,28)

$puts(lp,%last_played%)
$puts(fp,%first_played%)
$puts(pc,%play_count%)
$puts(rating,[%rating%])
$puts(avgrating,3)

$puts(baselinefrequency,$mul($get(baselinefrequency),24))
$puts(baselinedecay,$mul($get(baselinedecay),24))
$puts(lp_age,$div($time_diff(2000,$get(lp)),3600))
$puts(fp_age,$div($time_diff(2000,$get(fp)),3600))
$puts(age,$sub($get(lp_age),$get(fp_age)))
$puts(now_age,$div($time_diff(2000),3600))
$puts(recentness,$sub($get(now_age),$get(lp_age)))
$puts(decay,$div($div($mul($get(pc),$get(baselinefrequency),$get(baselinedecay),$if2($get(rating),$get(avgrating)),100),$mul($max($get(age),$get(baselinefrequency)),$get(avgrating))),100))
$puts(rawhotness,$div($mul($max($sub($get(decay),$get(recentness)),0),100),$get(decay)))
$puts(forecast,$div($mul($max($sub($get(decay),$add($div($max(0,$sub($get(baselinedecay),$get(recentness))),2),$get(recentness))),0),100),$get(decay)))
$puts(hotness,$div($add($get(rawhotness),$get(forecast)),2))
$get(hotness)

Maybe I should play with the baseline* parameters?

[carpman]

This is the kind of number I get for RECENTNESS ... 195744
This is the kind of number I get for DECAY ... 2016

195744 is way too big. Have you switched the order of parameters in $date_diff() as I have written above?

No, I hadn't. I'd ignored that (see EDIT below) and just read the shortened version:

$puts(lp_age,$div($time_diff(2000,$get(lp)),3600))

which I carelessly took to be a more concise version of the same thing. Doh! Thus all the rest of my stuff is rubbish, as you pointed out. By "it still seems to suck" I'd thought you'd meant that it still doesn't work. So I hadn't even tried the code, thinking there must be something else wrong. Wrong end of the stick, completely.

Like q-stankovic, I thought a simple find and replace of the various terms would suffice. I think I'll look at this more closely to see what it's actually doing, and whether it makes sense. From what you've said, it doesn't seem to ...

I'm just saying that a playlist ordered by this doesn't seem like a hotlist to me or anything.

I'll try it with your amendments later. Thanks for the explanation.

C.

EDIT:

I realise that a large part of my misunderstanding has come from an assumption I'd made about all of these difference functions. I'd assumed that:

"The difference between 10 and 20" = "The difference between 20 and 10", i.e. the difference is the absolute difference, so:
Difference <> Subtraction.

As for DAR, I'm always only interested in the "gap" between values, and not whether the output is positive or negative. Clearly there's more information to be had when the order of the values differentiated between give a positive or negative outcome. And that's obviously how you've made the $xxxx_diff functions.

This post has been edited by carpman: Mar 2 2011, 19:40

[carpman]

This post assumes readers are already familiar with:

• foo_dynfil
• DADA Auto-Rating (DAR)

Hotness + DAR

Hotness is a %last_played% formula, which takes into account %rating% and %play_count%.

The main problem with Hotness (as a standalone measure) is that it cannot know the recent frequency of plays.
For example:
If you have a track that you played 10 times a day for 3 days, then never played again for 3 years, then played it once today, it will have a Hotness of 90, and that Hotness will take 77 days to fade to 0. This is Hotter than a track that you added 1 year ago, which you played 11 times yesterday !!!

Now, this is not a criticism of Hotness, because that data is simply not available to it. However, this problem can be remedied to an extent when cross-referenced with DAR. For example (in foo_random_pools):

(%added% BEFORE 2009) AND (%_dynamic_rating% GREATER 9200) AND (%_hotness% LESS 1)

Because Hotness takes into account frequency and recentness of plays, in conjunction with DAR it can indicate that the frequency of plays (to get the DAR above 9200) occured early on in the track's life, meaning that this was a hot track in the past. (This is the case in the screenshot below, see the track with the DAR of 10347).

Used conversley, if the DAR is low, but the Hotness is high, then this is a genuinely Hot track now.
If the DAR is high and Hotness is high, then this is a popular track you've played recently, that's all you know.
Likewise if DAR is low and Hotness is low, this is an unpopular track, not recently played.

But to restate, the real benefit comes from what you learn from a DAR / Hotness mismatch.

Now, manual ratings (i.e. subjective user input distorts this) so the best thing to do is to use DAR to feed Hotness its rating. The code below does just that: The DAR rating is converted to a number between 1 and 5 and fed to Hotness (thanks Yirkha for the Hotness code):

You need to alter the maxdar / mindar values as per here ["Section II: DAR INDEXED RATING (WITH DOTS)"].

SETUP & CODE

Add a field in foo_dynfil called "hotness", then add this:

CODE

$puts(maxdar,11000)
$puts(mindar,8000)
$puts(maxsub,$sub($get(maxdar),0))
$puts(r3,$ifgreater(%_dynamic_rating%,$get(maxsub),$get(maxsub),%_dynamic_rating%))
$puts(r4,$ifgreater($get(r3),0,$get(r3),1))
$puts(minmax,$sub($get(maxdar),$get(mindar)))
$puts(darind1,$sub($get(r4),$get(mindar)))
$puts(darind2,$div($mul($get(darind1),5),$get(minmax)))
$puts(darind3,$ifgreater($get(darind2),1,$get(darind2),1))

$puts(baselinefrequency,90)
$puts(baselinedecay,28)

$puts(lp,%last_played%)
$puts(fp,%first_played%)
$puts(pc,%play_count%)
$puts(rating,$get(darind3))
$puts(avgrating,3)

$puts(baselinefrequency,$mul($get(baselinefrequency),24))
$puts(baselinedecay,$mul($get(baselinedecay),24))
$puts(lp_age,$div($time_diff(2000,$get(lp)),3600))
$puts(fp_age,$div($time_diff(2000,$get(fp)),3600))
$puts(age,$sub($get(lp_age),$get(fp_age)))
$puts(now_age,$div($time_diff(2000),3600))
$puts(recentness,$sub($get(now_age),$get(lp_age)))
$puts(decay,$div($div($mul($get(pc),$get(baselinefrequency),$get(baselinedecay),$if2($get(rating),$get(avgrating)),100),$mul($max($get(age),$get(baselinefrequency)),$get(avgrating))),100))
$puts(rawhotness,$div($mul($max($sub($get(decay),$get(recentness)),0),100),$get(decay)))
$puts(forecast,$div($mul($max($sub($get(decay),$add($div($max(0,$sub($get(baselinedecay),$get(recentness))),2),$get(recentness))),0),100),$get(decay)))
$puts(hotness,$div($add($get(rawhotness),$get(forecast)),2))
$get(hotness)

To get a last played column like this:



You need to add "lp_display" as a field in foo_dynfil, then add this code:

CODE

$date_diff(%last_played%)

In a column (CUI, not sure about DUI) add:

CODE

$if(%last_played%,$puts(diff,%_lp_display%)
$ifgreater(1,$get(diff),$rgb(150,0,0)Today , $ifgreater(2,$get(diff),1
$rgb(170,170,170), $get(diff) $rgb(170,170,170))),-- no info --)
$rgb(180,180,180)|$blend($rgb(240,240,240),$rgb(255,50,10),%_hotness%,100)
$char(9679)$rgb(180,180,180)|

Here's Hotness as a standalone dot (as per above but without %last_played% stuff):

CODE

$blend($rgb(240,240,240),$rgb(255,50,10),%_hotness%,100)$char(9679)$rgb(180,180,180)

HOTNESS: Last Played, Playcount and Rating

Here's some analysis regarding what Hotness does and what affects what:

As the %play_count% increases so the number of days since last_played (before Hotness goes to zero) also increases.




The effect of rating in conjunction with playcount on Hotness:




Doubling the rating has the same effect as doubling the playcount. So "a rating of 2 and a PC of 20" = "a rating of 4 and a PC of 10", both tracks in this scenario (when all else being equal) take being not played for 49.4 days to drop to a Hotness of 0.




HOTNESS: The Formula

Finally, here's a break down of the formula:


LP AGE = LP - 1/1/2000 (Hrs)
FP AGE = FP - 1/1/2000 (Hrs)
NOW AGE = NOW - 1/1/2000 (Hrs)

bf = baseline frequency (90 x 24)
bc = baseline decay (28 x 24)
avg rating = 3



1. RECENTNESS: Hrs Since Last Played



2. DECAY:

(pc x bf x bd x rating x 100) / ((MAX age OR bf) x avg-rating)
______________________________________________________

100



3. RAWHOTNESS:

100 x (MAX decay - recentness OR 0)
______________________________________________________

decay



4. FORECAST:

(MAX "decay - (((MAX bd - recentness OR 0) / 2 ) + recentness)" OR 0) x 100
______________________________________________________

decay



5. HOTNESS:

(forecast + rawhotness) / 2


********************************************************

Hope some people will find that helpful.
If anyone wants to analyse this further, let me know and I'll make my Hotness spreadsheet available.

C.

EDIT1: clarity
EDIT2: amended hotness code: $puts(baselinedecay,28) was 33, which was my setting, not the hotness default.

This post has been edited by carpman: Mar 7 2011, 16:47

[スラッシュ]

Thanks dude, I tried to convert it before and couldn't figure it out. Will definitely be using this!

[carpman]

Hotness Anomaly

In the scenario below, there are 2 tracks (red and blue), both are 500 days old, both have the same rating and both end up with the same playcount. The difference is that one (red) was played 50 times early in its life and the other (blue) took longer to get to 50 plays. One would expect that the track that was last played most recently (blue) would have the higher hotness. But after a certain duration of not being played Blue's Hotness drops below Red's Hotness:

Settings: baseline frequency = 90, baseline decay = 28, rating = 3 (however this happens with any rating):



Similar scenario via foo_dynfil:



Summary:

I think Hotness has its uses as a kind of "alternative %last_played% filter" for things like foo_random_pools (auto-playlists etc.) so, for example, popular songs played recently have longer to wait before being re-played than less popular ones.

The following setting prevents the anomaly described above, as Hotness drops to zero before the anomaly can occur (so not for everyone, but I thought I'd offer it up)

baseline frequency = 120, baseline decay = 6

I think that's me done with my Hotness curiosity. Thanks again Yirkha for sorting out the formula for foo_dynfil.

C.

[bernhold]

So awesome! Thanks for all this carpman. Interesting read and useful tools