Python Grabber scripts |
Python Grabber scripts |
May 28 2009, 15:27
Post
#1
|
|
|
Group: Validating Posts: 2424 Joined: 21-May 08 Member No.: 53675 |
Some users reported errors using the scripts I posted here
The errors are result of incorrectly saving the copied scripts, probably indentation errors, so I'm posting those scripts as ready made files here: [attachment=5122:Discogs.zip] |
|
|
|
![]() |
Mar 20 2010, 02:03
Post
#2
|
|
![]() Group: Members Posts: 41 Joined: 6-February 10 Member No.: 77932 |
Here's a modification of what I described above.
This python script which is a modification of 2E7AH's scripts fetches Genre and Style from discogs. When it meets a track from an album it has not processed before, it writes those tags to a text file. If it meets a track from an album it has processed a track from before, it does not check discogs - rather, it just fetches the tags from those text files. In essence, it fetches discogs genre and style while minimizing the amount of requests. This will allow you to tag thousands more files per 24 hours. For temporary storage of information, it uses text files. My text files are in the folder C:\Python26 .. Edit the script and change them for whatever values you want to use! CODE import urllib, urllib2, gzip, cStringIO import xml.etree.ElementTree from xml.dom import minidom from encodings import utf_8 from grabber import LyricProviderBase class Discogs_GetGenre(LyricProviderBase): def GetName(self): return 'Discogs GenStyles' def GetVersion(self): return '0.1' def GetURL(self): return 'http://www.discogs.com' def Query(self, handles, status, abort): result = [] api_key = '783001745d' for handle in handles: status.Advance() if abort.Aborting(): return result artist = handle.Format("[%artist%]") album = handle.Format("[%album%]") album_file = open("C:\Python26\!album.txt", "r") test = album_file.read() album_file.close() try: if album == test: text_file = open("C:\Python26\!tag3.txt", "r") writeout = text_file.read() text_file.close() result.append(writeout) else: URL_s = 'http://www.discogs.com/search?type=all&q=' + artist.lower().replace(' ','+') + '+' + album.lower().replace(' ','+') + '&f=xml&api_key=' + api_key request = urllib2.Request(URL_s) request.add_header('Accept-Encoding', 'gzip') response = urllib2.urlopen(request) data = response.read() unzipped_data = gzip.GzipFile(fileobj = cStringIO.StringIO(data)).read() res = minidom.parseString(unzipped_data) uri_1 = res.getElementsByTagName("uri")[0] rel_id = uri_1.childNodes[0].data.encode('utf-8').rpartition('/')[2] URL_r = 'http://www.discogs.com/release/' + rel_id + '?f=xml&api_key=' + api_key request = urllib2.Request(URL_r) request.add_header('Accept-Encoding', 'gzip') response = urllib2.urlopen(request) data = response.read() unzipped_data = gzip.GzipFile(fileobj = cStringIO.StringIO(data)).read() xml.etree.ElementTree.fromstring(unzipped_data) def getGenres(tree): genres = [] release = tree.find('release') genreList = release.find('genres') if genreList: for i in genreList: genres.append(i.text) return genres def getStyles(tree): styles = [] release = tree.find('release') styleList = release.find('styles') if styleList: for i in styleList: styles.append(i.text) return styles lyric=getGenres(xml.etree.ElementTree.fromstring(unzipped_data)) lyric.append('//') lyric.append(getStyles(xml.etree.ElementTree.fromstring(unzipped_data))) lyric=str(lyric).strip('[').strip(']').replace(',', ';').replace('\'','').replace('[','') result.append(lyric) album_file = open("C:\Python26\!album.txt", "w") album_file.write(album) album_file.close() text_file = open("C:\Python26\!tag3.txt", "w") text_file.write(lyric) text_file.close() except Exception, e: traceback.print_exc(file=sys.stdout) result.append('') album_file = open("C:\Python26\!album.txt", "w") album_file.write(album) album_file.close() text_file = open("C:\Python26\!tag3.txt", "w") text_file.write('') text_file.close() continue return result if __name__ == "__main__": LyricProviderInstance = Discogs_GetGenre() The output to the tag will be "%genre%; //; %style%". I don't know how to get rid of the semicolons just yet, but you can clean those up as you split the tag into genre and style tags. And yes, it's written quick and dirty. I haven't yet beautified it and the script probably won't look the same tomorrow. edit : I've tested it on some 5000 files and it works perfectly This post has been edited by tore: Mar 20 2010, 02:14 |
|
|
|
2E7AH Python Grabber scripts May 28 2009, 15:27
auldyin QUOTE (2E7AH @ May 28 2009, 15:27) Some u... May 28 2009, 18:19
2E7AH Search format: artist-title. LRC (time-stamped) l... Jun 22 2009, 23:24
2E7AH Mistype error corrected and some specific parsing,... Jun 24 2009, 12:46
2E7AH Mistype error corrected and some specific parsing,... Jun 24 2009, 15:15
2E7AH corrected code page for accented chars etc.
Jun 24 2009, 19:56
2E7AH RE: Python Grabber scripts Jun 24 2009, 21:33
elenhil Same error as before:
CODEfoo_grabber_python... Jun 26 2009, 12:40
2E7AH That can only happen if you don't have artist ... Jun 26 2009, 15:17
elenhil QUOTE (2E7AH @ Jun 26 2009, 18:17) That c... Jun 26 2009, 21:45
2E7AH Updated AMG review:
fasterif fetching multiple rel... Jun 26 2009, 16:57
tberman333 QUOTE (2E7AH @ Jun 26 2009, 10:57) Update... Sep 22 2009, 02:46
tere maybe there is a way to make a plugin also www.lol... Jun 29 2009, 06:24
2E7AH QUOTE (tere @ Jun 29 2009, 07:24) maybe t... Jun 29 2009, 23:53
2E7AH Search format: artist-title. We are the leading ly... Jun 30 2009, 16:36
q-stankovic Hello 2E7AH!
I tried to download some of y... Sep 22 2009, 17:31
2E7AH Some strange interest in scripts today
I'll up... Sep 23 2009, 20:05
q-stankovic QUOTE (2E7AH @ Sep 23 2009, 21:05) Sorry ... Sep 25 2009, 17:18
2E7AH hm... I don't know what you mean with database... Sep 25 2009, 17:37
2E7AH Also maybe isn't so good idea making lyrics gr... Sep 25 2009, 18:56
q-stankovic QUOTE (2E7AH @ Sep 25 2009, 18:37) I... Sep 26 2009, 15:07
2E7AH Here are updated AMG scripts:
- AMG_Genre.py
-... Oct 11 2009, 07:04
2E7AH Updated AMG review script:
- changed %artist% to... Oct 16 2009, 14:40
Benji99 Hey 2E7AH,
Any chance you could make a script for... Oct 29 2009, 02:32
2E7AH Aren't you from Canada?
It looks easy to find... Oct 29 2009, 02:58
Benji99 QUOTE (2E7AH @ Oct 28 2009, 21:58) Aren... Oct 29 2009, 03:13
2E7AH I'll try this one, it should be OK (although d... Oct 29 2009, 03:19
Benji99 QUOTE (2E7AH @ Oct 28 2009, 22:19) I... Oct 29 2009, 03:36
2E7AH Benji99, here is the script:
I've tested it ... Oct 31 2009, 06:12
Benji99 QUOTE (2E7AH @ Oct 31 2009, 00:12) Benji9... Nov 11 2009, 07:29
2E7AH QUOTE (Benji99 @ Nov 11 2009, 08:29) 1st ... Nov 11 2009, 08:05
Benji99 QUOTE (2E7AH @ Nov 11 2009, 02:05) Is tha... Nov 21 2009, 04:31
2E7AH Once again AMG scripts
Now GENRE, STYLE, MOOD an... Nov 10 2009, 09:31
2E7AH Download this AMG release script:
Problem with pr... Nov 10 2009, 11:08
tberman333 Thank you! This is great!
QUOTE (2E7AH ... Nov 10 2009, 15:17
2E7AH Enjoy
I didn't forgot about composer/performe... Nov 10 2009, 16:22
tberman333 QUOTE (2E7AH @ Nov 10 2009, 10:22) Enjoy ... Nov 10 2009, 18:10
Benji99 QUOTE (2E7AH @ Nov 10 2009, 11:22) Enjoy ... Mar 27 2010, 23:15
T.P Wang @2E7AH:
I think replace the u'\uff5e... Nov 21 2009, 03:09
garbanzo.bean 2E7AH, i'm trying to use your AMG script, but ... Jan 8 2010, 15:54
tberman333 QUOTE (garbanzo.bean @ Jan 8 2010, 09:54)... Jan 8 2010, 16:46
tore I have a quick question regarding the python disco... Mar 15 2010, 12:29
2E7AH QUOTE (tore @ Mar 15 2010, 12:29) Would i... Mar 15 2010, 13:15
tore By the way, I did find this little piece of inform... Mar 15 2010, 15:05
tore I've now tried playing around with your last.f... Mar 15 2010, 20:11
2E7AH look at the other last.fm script, i.e.
CODEtoptag... Mar 15 2010, 20:28
tore QUOTE (2E7AH @ Mar 15 2010, 20:28) look a... Mar 15 2010, 20:38
2E7AH I just pasted that part from that other last.fm sc... Mar 15 2010, 20:48
tore QUOTE (2E7AH @ Mar 15 2010, 20:48) I just... Mar 15 2010, 21:38
2E7AH hey, why ride motorcycle when I have my bike
le... Mar 15 2010, 21:51
tore QUOTE (2E7AH @ Mar 15 2010, 21:51) hey, w... Mar 15 2010, 22:34
2E7AH QUOTE (tore @ Mar 15 2010, 22:34) I made ... Mar 15 2010, 23:00
tore QUOTE (2E7AH @ Mar 15 2010, 23:00) QUOTE ... Mar 15 2010, 23:14
tore For an example of what information could be pulled... Mar 16 2010, 00:26
tore To anyone having a problem with the discogs 5000 r... Mar 19 2010, 15:05
2E7AH I didn't bothered with 5000 limit on Discogs, ... Mar 19 2010, 18:52
xtr Is it possible to use these scripts with a foo_run... Mar 21 2010, 23:05
2E7AH If you have python, I guess you know something abo... Mar 21 2010, 23:21
2E7AH Are you really counting on that
Script is there,... Mar 29 2010, 00:10
Benji99 I was thinking of writing a small program (evaluat... Mar 31 2010, 21:13
2E7AH I'm just working on the script, and it's a... Apr 1 2010, 03:15
Benji99 QUOTE (2E7AH @ Mar 31 2010, 22:15) I... Apr 4 2010, 22:15
Hitchhiker427 Does anyone still have 2E7AH's scripts for Lyr... Aug 6 2010, 00:55
grimes Found a Lyrics-site with API: ChartLyrics
Here is... Aug 7 2010, 21:03
Hitchhiker427 QUOTE (grimes @ Aug 7 2010, 15:03) Found ... Aug 9 2010, 20:14
Beta4Me For all that are interested...I am coding in PHP a... Oct 6 2010, 13:06
tberman333 QUOTE (Beta4Me @ Oct 6 2010, 08:06) For a... Oct 23 2010, 23:05
Beta4Me QUOTE (tberman333 @ Oct 24 2010, 08:05) Q... Oct 24 2010, 06:38
lo-fi yep. amg scripts not working here either Oct 24 2010, 00:41
Decalicatan_Decalicatan QUOTE (lo-fi @ Oct 24 2010, 01:41) y... Oct 24 2010, 09:41
Wooops Since most of the script links seem to be broken -... Oct 25 2010, 05:54
Beta4Me How did the original python grabber scripts work? ... Oct 26 2010, 02:47
mrinferno AMG redesigned their site and search methods mid-O... Oct 26 2010, 02:53
Beta4Me QUOTE (mrinferno @ Oct 26 2010, 12:53) AM... Oct 26 2010, 03:59
tberman333 QUOTE (Beta4Me @ Oct 25 2010, 22:59) QUOT... Oct 26 2010, 21:16
Beta4Me QUOTE (tberman333 @ Oct 27 2010, 07:16) Q... Oct 26 2010, 23:25
tberman333 QUOTE (Beta4Me @ Oct 26 2010, 18:25) Than... Oct 27 2010, 01:37
Decalicatan_Decalicatan I have uploaded one of the 2E7AH amg scripts (from... Oct 26 2010, 11:10
Beta4Me QUOTE (Decalicatan_Decalicatan @ Oct 26 2010,... Oct 26 2010, 11:15
Schuft QUOTE (Decalicatan_Decalicatan @ Oct 26 2010,... Feb 13 2011, 09:41
klonuo QUOTE (Schuft @ Feb 13 2011, 10:41) Is th... Feb 13 2011, 16:53
Schuft Ok - you're right - phyton installation was mi... Feb 16 2011, 18:37
MDMA Would it be possible to use this component to fetc... Feb 8 2012, 16:21
Fractal_Mortality Does anyone know where i could find updated script... Feb 3 2013, 20:30![]() ![]() |
|
Lo-Fi Version | Time is now: 26th May 2013 - 01:51 |