Thanks for Marc Landis' excellent work, I can now much more efficiently acquire and manage album art images.

What is the Album Art Downloader :
http://www.hydrogenaudio.org/forums/index....showtopic=51896

I wrote three scripts for the Album Art Downloader using mutithread programming techniques, expecting to search and download faster...
just extract and release these .boo files to your AlbumArt\Scripts folder

Notice that they may NOT work simultaneously, run one script at a time:
Script searching Amazon.com:(unlike the original amazon.boo, this script searches the html site(not the xml one) and returns much more results)
Click to view attachment

Script searching Amazon.co.jp(efficient in searching japanese album arts)
Click to view attachment

Script searching Last.FM(efficient in searching artists' images)
Click to view attachment

Attention the mutithread techniques used in these scripts is experimental, and bugs might be encountered.
Expecting feedbacks

Glad to see someone else writing scripts for this now!

Just one point, your amazon US one searches partially amazon.com, and partially amazon.co.jp - I assume not intentional?

I'm interested in why you wrote these threaded - are you using the newer XUI version? There may be problems with cancelling, as to cancel a search (including if it reaches the specified maximum number of results) the current thread is aborted from within the AddThumb method. I did not consider that the thread adding thumbnails might not be the search thread itself.

I'm also trying to figure out what work it is you are trying to parellelise. Is it that you want to be downloading multiple thumbnails at once? XUI is already fairly well multi-threaded, but downloading thumbnails is done one at a time, true. If downloading multiple thumbnails at once offers a benefit, it might be worth having this in XUI itself, rather than requiring scripts to multi-thread, which is a bit more complex than should be required!

If you are interested in writing complex scripts (and using XUI), you might want to take a look at the AlbumArtDownloader.Scripts.dll, which has a public interface on it: AlbumArtDownloader.Scripts.IScript. Any class implementing this interface in a dll placed in the scripts folder will be loaded and used as a script, which frees you up to use any .net language to write scripts. You could even use a Boo script to create a class from it, and it should be automatically compiled up and used, but for boo scripts its probably easier just to stick to the existing magic-named static methods system.

Anyway, if you have any questions, I'll be around!

Alex

Not intentional indeed...
It's a shame that i mistakenly submitted a half-finished one... and i have corrected it already.
Thanks for your check of my scripts... Actually i wrote my scripts based upon yours..

i am quite a green hand of Boo language...(i started using it just two weeks ago)
The reason why i wrote my scripts using multi-thread methods is that it takes too much time
to connect to Amazon.com, Amazon.co.jp or Last.FM in my position(somewhere in China)...
Occasionally i must spend 20 minutes or more waiting for the results being completely transported.

i've encountered lots of difficulties in managing multi-thread downloading..
first: how to record the number of results and keep the progressbar going...
second: i found Album Art Downloader will crash if i started too many threads, and now i use a
"thread pool" to restrict number of active threads.
these two problems afforementioned are what i solved last week, (using clumsy ways maybe...)

and, as you said, i found many problems which cannot be solved by myself.
the biggest one is what you mentioned:"There may be problems with cancelling" ,yes
when you push the "cancel" button, threads already started won't stop...
i found no way to handle this...
another problem is that the scripts may cause crash when results are too many.

i noticed that you said:
"it might be worth having this in XUI itself, rather than requiring scripts to multi-thread, which is a bit more complex than should be required!"
that is exactly what i want... because it does offer a much faster downloading speed.
you can compare my multi-thread scripts with scripts below:
Click to view attachmentClick to view attachmentClick to view attachment

i will read more about web crawler and try to write more efficient scripts, simply because it's a fun for me.

BTW, i found trouble when installing Microsoft .NET Framework 3.0.
The Communication Fundation seems not compatible with my computer, but i installed
Workflow Fundation and Presentation Fundation correctly.
Therefore i could only use AlbumArt Downloader ver. 0.4.1.0, any higher version reports a
fatal error:

"...System.IO.FileNotFoundException: Could not load file or assembly 'System.ServiceModel, Version=3.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' or one of its dependencies. 系统找不到指定的文件(System cannot find designated file)。
File name: 'System.ServiceModel, Version=3.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'..."

maybe i should try a Windows Vista system...

Now that's weird. I've never heard of installing just bits of the .net framework. As far as I know, you just download the installer from here and it installs all of it. It sounds like yours is a bit broken at the moment, you might want to try removing and reinstalling it. I haven't tried it on Windows Vista, but it should work fine too.

If you can't use the latest XUI version, then there probably isn't much I can do about improving performance, though, as you won't be able to try anything out. In general terms though, I think the main factor is going to be number of simultaneous image downloads. In AAD as it stands, this is one. Each thumbnail must finish downloading before it starts on the next. The other extreme of unlimited simultaneous downloads (just spinning off another download thread for each image as the URL is discovered) is also not desirable, though, as it tends to lead to instability and poor performance. I'm not sure whether an optimal number of simultaneous downloads can be found, or whether one at a time is still the best way to go.

You might like to try downloading the search results html pages on a single separate thread, then calling AddThumb on the main search thread in response to html pages being downloaded. That way the image downloading occurs on the main search thread, and will respond to cancelling properly... just an idea, anyway.

Alex

The way you mentioned couldn't eliminate speed limitations when searching websites whose structure illustrated below:
Click to view attachment

Assume that it takes 10 seconds to finish each "go deeper" thread, such as A1 and A2, and it takes 5
seconds to get a new result page (thread D and E)
In the method you suggested, because thread E and A are parallel, total time should be 10*N seconds(N is the number of results).
But in circumstance that we start A1, B1 and C1 together, total time should be around 10*N/M (M is the number of simultaneously working threads) seconds if bandwidth is enough.
I think M should be about 5 to 10 (just a experiential value)

However, I agree that stability should be considered first...

OK, I see where you're going with this. Yes, those intermediary pages will be a problem. They would also continue to be a problem even if AAD handled simultaneous downloading of thumbnails internally, as even though the AddThumb method would return immediately, you'd still have the download of the detail page running synchronously.

It would be interesting to see what sort of performance just having the detail pages downloaded simultaneously and keeping the image downloading on the main thread handled by AAD. This would indicate where the slow step was.

I'm thinking, the main search thread spins off one worker thread to download results pages (D&E), and a number (say 5) of detail download threads. It should then go into a loop watching an image download queue (probably waiting on a AutoResetEvent signaller to notify it the queue has changed).

As each results page is downloaded and parsed, every link to a detail page is added to a queue. The detail download threads should watch that queue, and when they find something in it, remove it from the queue and start downloading it (A1-C1). Care will have to be taken here to avoid them trampling on each other trying to process the same item. Once a detail download thread has downloaded a page, it parses it for the image link, and adds that link to the image download queue, and pokes the signaller to let the main search thread know that there are images in the queue.

The main search thread should then remove image urls from the image download queue and AddThumb them as results, looping round and returning to wait once the queue is empty.

Two cases need special consideration, ending and cancelling. For cancelling, you should trap the ThreadAbortedException around the queue watching loop in the main search thread, and if caught, abort all other threads and clear all queues too. For ending, I guess the only thing that knows when there are no more results is the results page download thread, so that would have to signal the main thread to stop waiting for the image download queue and return.

Personally, I would find something of this complexity a nightmare to write in Boo, without IDE support, but if you want to have a go, good luck!

Multithreaded programming is always interesting :-)

is the amazon.co.jp add-on working properly?
i've been trying to download a lot of album art for my albums, but when i try to use this i get none.
if i look on amazon.co.jp manually all the album art is there. am i doing something wrong?