Help - Search - Members - Calendar
Full Version: Python Grabber scripts
Hydrogenaudio Forums > Hosted Forums > foobar2000 > Uploads - (fb2k)
2E7AH
Some users reported errors using the scripts I posted here
The errors are result of incorrectly saving the copied scripts, probably indentation errors, so I'm posting those scripts as ready made files here:

Click to view attachment
auldyin
QUOTE (2E7AH @ May 28 2009, 15:27) *
Some users reported errors using the scripts I posted here
The errors are result of incorrectly saving the copied scripts, probably indentation errors, so I'm posting those scripts as ready made files here:

Click to view attachment


Many thanks!!

Works a treat


2E7AH
Click to view attachment Search format: artist-title. LRC (time-stamped) lyrics. 26 252 records Link

Click to view attachment Search format: artist-album-title. 184 912 records Link

Click to view attachment Search format: artist-title. We have added around 1.000.000 new song lyrics (2007) Link

Click to view attachment Search format: artist-title. This is the same server as lyricsreg.com. More then 700 000 records Link

Click to view attachment Search format: artist-title. PLyrics is lyrics source dedicated to punk music Link

Click to view attachment Search format: artist-title. Artists: 39,266; Songs: 446,280; Albums: 14,226 Link

For questions about python grabber go here
2E7AH
Mistype error corrected and some specific parsing, and also descripton should refer to lyricslib.com (not to lyricsreg.com):

Click to view attachmentSearch format: artist-title. This is the same server as for lyricslib.com. More then 700 000 records Link
2E7AH
Mistype error corrected and some specific parsing, and also descripton should refer to lyricslib.com (not to lyricsreg.com):

Click to view attachmentSearch format: artist-title. This is the same server as for lyricslib.com. More then 700 000 records

Click to view attachmentSearch format: artist-title. More than 1.000.000 records
2E7AH
Click to view attachment
elenhil
Same error as before:

CODE
foo_grabber_python: Traceback (most recent call last):
  File "C:\Program Files\foobar2000\pygrabber\scripts\LyricsTXT.py", line 41, in Query
    url = 'http://www.lyricstxt.com/lyrics/'+artist[0]+'/'+artist+'/'+title+'_lyrics.html'
IndexError: string index out of range

2E7AH
That can only happen if you don't have artist tag, right?
2E7AH
Updated AMG review:
  • faster
  • if fetching multiple releases (release with no review got assigned to previous review value) corrected smile.gif

Click to view attachment
elenhil
QUOTE (2E7AH @ Jun 26 2009, 18:17) *
That can only happen if you don't have artist tag, right?

Quite possible!
tere
maybe there is a way to make a plugin also www.lololyrics.com unsure.gif
2E7AH
QUOTE (tere @ Jun 29 2009, 07:24) *
maybe there is a way to make a plugin also www.lololyrics.com unsure.gif

I'll look through it tomorrow. As I see quickly right now they don't have some straight search engine, but we'll see smile.gif
2E7AH
Click to view attachmentSearch format: artist-title. We are the leading lyrics site specialized only on electronic music with over 8000 lyrics spread over genres such as Hardstyle, Hardcore/Gabber, Drum & Bass and Psychedelic Trance link

Possible problems with accented characters. I tested the script very little lacking the music for which lyrics are offered
tberman333
QUOTE (2E7AH @ Jun 26 2009, 10:57) *
Updated AMG review:
  • faster
  • if fetching multiple releases (release with no review got assigned to previous review value) corrected smile.gif

Click to view attachment


Where did this one go??
q-stankovic
Hello 2E7AH! smile.gif

I tried to download some of your scripts , but i have absolutely no idea what to do with such an information: "Click to view attachment". Hope you can help.

By the way:

Would really love to see a script for that hip-hop lyrics archive
http://www.ohhla.com/
2E7AH
Some strange interest in scripts today
I'll upload them again these days, I was waiting for new python grabber release but still waiting

QUOTE (q-stankovic @ Sep 22 2009, 17:31) *
Would really love to see a script for that hip-hop lyrics archive
http://www.ohhla.com/

Sorry q-stankovic, but I can't test hip-hop lyrics because I don't have that kind of music (same for reggae, funk/soul, latin etc)
If you have some other lyrics vendor I would try, and if you have some LRC provider than certainly
q-stankovic
QUOTE (2E7AH @ Sep 23 2009, 21:05) *
Sorry q-stankovic, but I can't test hip-hop lyrics because I don't have that kind of music (same for reggae, funk/soul, latin etc)


Do you need real files to be tagged with LYRICS or would it be enough to have a database.fpl file? In the latter case i could create one and send it to you.
2E7AH
hm... I don't know what you mean with database playlist, but for simplicity just select couple of your hip-hop albums and save playlist as M3U (if you have skipyrich's EXTM3U then better)

I'll do the script, but it will be somewhere this week and not in a day or two, because of some other things wink.gif
2E7AH
Also maybe isn't so good idea making lyrics grabber for this site:
1. they don't have search engine as I see
2. if the author isn't some f***ing "50 cent" or something like that, they redirect to FTP and there is not simple way to compare album/title name with their FTP folders - i.e. some random artist: "504 Boyz" has album "Lil' Romeo, Magic" and the folder name is "rm_bside" and then title "Tight Whips" with filename "t_whips.504.txt"

So sorry again, if there isn't search engine I won't bother
q-stankovic
QUOTE (2E7AH @ Sep 25 2009, 18:37) *
I'll do the script, but it will be somewhere this week and not in a day or two, because of some other things wink.gif

I never expected that you write the script so fast!


QUOTE (2E7AH @ Sep 25 2009, 19:56) *
Also maybe isn't so good idea making lyrics grabber for this site ...


That's a pity! Thanks for considering.
2E7AH
Here are updated AMG scripts:

Click to view attachment
- AMG_Genre.py
- AMG_Mood.py
- AMG_Review.py
- AMG_Style.py
- AMG_Theme.py

All AMG releases should return %genre% at least, then %style%, %amg_review%, %mood%, %theme% - in this line of importance
%artist% tag should be as they are on AMG - i.e. MusicBrainz tags artist as "Gabrielle Roth and the Mirrors", but maybe AMG tags tham as "Gabrielle Roth & the Mirrors" so there will be no results. But this are special cases perhaps

I'll soon update other missing scripts too
2E7AH
Updated AMG review script:

Click to view attachment
- changed %artist% to %album artist%
- now stops on first (most probable) release match, and not on last as it was (I forgot to add break in a loop smile.gif)
- releases with large review now get full review (was "Read more...")

I guess this is final update. Report for problems or suggestions are welcomed
Benji99
Hey 2E7AH,

Any chance you could make a script for http://www.uta-net.com/ ? It's a fantastic lyrics site for Japanese songs.
The best way to run searches is probably by song name, and the word in japanese word for that is: 曲名
Also this word means artist: 歌手名
An example of a search you can do using a artist name and song names in english letters would be:
Artist: Mr.Children
Song: Gift

I've noticed however that if you run a search for a song that's written with Japanese characters, the keyword in the search gets converted into hex (or something similar). An example would be if you search for this: 大阪で生まれた女 (Song name)

Huge thanks in advance!
2E7AH
biggrin.gif Aren't you from Canada?

It looks easy to find the lyrics response, but the lyrics are presented in Flash, so impossible with python grabber. Sorry
Benji99
QUOTE (2E7AH @ Oct 28 2009, 21:58) *
biggrin.gif Aren't you from Canada?

It looks easy to find the lyrics response, but the lyrics are presented in Flash, so impossible with python grabber. Sorry


I'm from Canada, but I can read/understand Japanese smile.gif, also I lived in japan for 1 year and accumulated a pretty big collection of Japanese music!

It's too bad about the above site. Actually, I found an alternative site that uses the same back-end for Lyrics but doesn't seem to use flash.
Here it is: http://music.goo.ne.jp/lyric/index.html

1/2 way down the page you'll see a search box labelled: 検索 - 歌詞情報
Same concept as the other site, if you select 曲名 you can search by song. (アーティスト名 is for searching by artist name)
The button you need to click to search is labelled: 歌詞検索

Hope you can do something with this one!
Thanks again smile.gif
2E7AH
I'll try this one, it should be OK (although don't know if characters will mess something but will see)

Do you know how many lyrics they have?
Benji99
QUOTE (2E7AH @ Oct 28 2009, 22:19) *
I'll try this one, it should be OK (although don't know if characters will mess something but will see)

Do you know how many lyrics they have?



Thanks!
Well according to uta-net, they have 84,000 songs in the database.
I got tons of obscure Japanese songs, for example stuff from the 1960s, etc, and every single one I was able to find in the database.

Thanks again!
2E7AH
Benji99, here is the script:

Click to view attachment
I've tested it by tagging some files with title/artist present on site, and it worked. If you find any problems post

What I've learned?
- Japanese glyphs have many encodings
- some sites don't like python
- even more about Unicode
smile.gif
2E7AH
Once again AMG scripts smile.gif

Now GENRE, STYLE, MOOD and THEME can be assigned at once with:
Click to view attachment
and new AMG review with custom user-agent report, loosen artist match and option to print some info in console:
Click to view attachment

Here is example for AMG_Release:

1. Select custom tag in python grabber settings:



2. run the script and update files

3. select Properties > Tools > Automatically fill values
source: Other and your custom tag
pattern: Genres: %genre% \\ Styles: %style% \\ Moods: %mood% \\ Themes: %theme%



4. then remove AMG tag and with Ctrl - click select newly added tags (GENRE, STYLE, MOOD and THEME) and select "Split values" then OK

If we have GENRE and STYLE tags and don't want to update them, than we enter this pattern i.e.: %tmp% \\ Moods: %mood% \\ Themes: %theme% so that GENRE and STYLE remains untouched

As a reminder all AMG scripts rely mostly on correct release (%album%) name
And do comment about problems, I'm rewriting this scripts as I run to some inconsistencies
2E7AH
Download this AMG release script:
Click to view attachment
Problem with previous here: http://www.hydrogenaudio.org/forums/index....st&p=666954

Now use this pattern:
Genres:%genre% \\ Styles:%style% \\ Moods:%mood% \\ Themes:%theme%
tberman333
Thank you! This is great!

QUOTE (2E7AH @ Nov 10 2009, 05:08) *
Download this AMG release script:
Click to view attachment
Problem with previous here: http://www.hydrogenaudio.org/forums/index....st&p=666954

Now use this pattern:
Genres:%genre% \\ Styles:%style% \\ Moods:%mood% \\ Themes:%theme%

2E7AH
Enjoy smile.gif
I didn't forgot about composer/performer conversation, I'll post that soon wink.gif

Here is masstagger script for cleaning the %amg% tag (Canar's version): just run it after the script (if %genre% and %style% should be preserved delete first two action from masstagger script):
Click to view attachment
tberman333
QUOTE (2E7AH @ Nov 10 2009, 10:22) *
Enjoy smile.gif
I didn't forgot about composer/performer conversation, I'll post that soon wink.gif

Thank you very much... I will wait on tagging any Various Artist Albums until that one comes.

QUOTE (2E7AH @ Nov 10 2009, 10:22) *
Here is masstagger script for cleaning the %amg% tag (Canar's version): just run it after the script (if %genre% and %style% should be preserved delete first two action from masstagger script):
Click to view attachment


I was just in the process of trying to figure out how to use Masstagger to do this.. they timing on this script is perfect!

By the way, I have been using the Python scripts on a few albums this morning and they are working great!
Benji99
QUOTE (2E7AH @ Oct 31 2009, 00:12) *
Benji99, here is the script:

Click to view attachment
I've tested it by tagging some files with title/artist present on site, and it worked. If you find any problems post

What I've learned?
- Japanese glyphs have many encodings
- some sites don't like python
- even more about Unicode
smile.gif



Huge thanks for this script!!
It works really well, except for a couple small bugs, if you have some free time, ... wink.gif

1st bug:
Certain track titles make the script crash.
CODE
foo_grabber_python: Traceback (most recent call last):
  File "I:\Program Files\foobar2000\pygrabber\scripts\goo.py", line 63, in Query
    raw_title = handle.Format('[%title%]').decode("utf8").encode("euc_jp")
UnicodeEncodeError: 'euc_jp' codec can't encode character u'\uff5e' in position 13: illegal multibyte sequence


This seemingly happens when a track has the '~' character in the title,

A couple examples:
Track title: HIGH G.K LOW ~ハジケロ~
Artist: GreeeeN

Track title: 手紙 ~君たちへ~
Artist: GreeeeN


Although, this one works:
Track title: 島唄~ウチナーグチ・ヴァージョン~
Artist: THE BOOM


2nd bug, The script seems to have trouble finding tracks where there's a large amount of tracks with the same name

For example:
Track title: YOU
Artist: サザンオールスターズ

Track title: 海
Artist: サザンオールスターズ

I know how this 2nd bug can be fixed I think, I found out that the site has a more advanced search function:
http://music.goo.ne.jp/lyric/db.php
There you can enter both the artist (アーティスト名) and track title (曲名)
If you can modify the script to use that page instead, it would make it really accurate!

Huge thanks again!
Sebastien






2E7AH
QUOTE (Benji99 @ Nov 11 2009, 08:29) *
1st bug:
Certain track titles make the script crash.
CODE
foo_grabber_python: Traceback (most recent call last):
  File "I:\Program Files\foobar2000\pygrabber\scripts\goo.py", line 63, in Query
    raw_title = handle.Format('[%title%]').decode("utf8").encode("euc_jp")
UnicodeEncodeError: 'euc_jp' codec can't encode character u'\uff5e' in position 13: illegal multibyte sequence


This seemingly happens when a track has the '~' character in the title

Is that happening only with that character? It can be easily fixed if so.
That character is fullwidth tilde "~" not ordinar tilde "~".

QUOTE (Benji99 @ Nov 11 2009, 08:29) *
2nd bug, The script seems to have trouble finding tracks where there's a large amount of tracks with the same name

Yeah, I would expect that, because script only tries to find match in first result page, and there can be more pages for some common title names.
I'll check your suggestion, and try to make the script better smile.gif
T.P Wang
@2E7AH:
I think replace the u'\uff5e' is a workaround:
CODE
s = handle.Format('[%title%]').decode("utf8")
raw_title = string.replace(s, u'\uff5e', u'\u301c').encode("euc_jp")
Benji99
QUOTE (2E7AH @ Nov 11 2009, 02:05) *
Is that happening only with that character? It can be easily fixed if so.
That character is fullwidth tilde "~" not ordinar tilde "~".


Oops, forgot to respond to this, whenever it crashes, that character is always in the the track title.
Thanks smile.gif

Btw, as far as making a more complete AMG script. Since I wrote The Godfather scripts for this already years ago. There's a few inconsistencies with the site. For example, the way to displays the performer and composer. It changes sometimes, in particular, it handles Various Artists albums and albums where a few tracks are collaborated by 2nd performer differently. If you can read Delphi and interested in my logic for how I coded around it, drop me a PM with your email, I'll send them to you smile.gif

I've been wanting to update it in Python but I found Python really hard to read/understand... sad.gif
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2009 Invision Power Services, Inc.