I keep my music organized alphabetically with sub-folders for segments of the alphabet. Foobar doesn't seem to want to display artwork for songs in my "#-C" folder (ie X:\Music\#-C\Artist\Album\folder.jpg). Everything else works fine, so I can only assume that the number symbol is causing the problem here. Is there any way around this problem without renaming the folder?

# is indeed the problem, and Terrestrial has basically said it couldn't be fixed.
You'll have to rename the folder.