IPB

Welcome Guest ( Log In | Register )

3 Pages V  < 1 2 3 >  
Reply to this topicStart new topic
Help ripping ~30,000 CDs, Was “Help digitizing […]” ;)
rick.hughes
post Apr 2 2012, 15:40
Post #26





Group: Members
Posts: 110
Joined: 16-February 07
Member No.: 40679



QUOTE (Porcus @ Apr 2 2012, 09:57) *
- RAID is not backup. RAID is a way to reduce the number of times you need to resort to your backup.

- There is a proprietary solution called UnRAID...

I use unRAID but if your budget forces you to choose between a complete backup and some sort of RAID then you should go for a complete backup, possibly kept offsite.
Go to the top of the page
+Quote Post
LosMintos
post Apr 2 2012, 15:48
Post #27





Group: Members
Posts: 37
Joined: 30-January 08
Member No.: 50912



QUOTE (spoon @ Mar 30 2012, 17:41) *
The last R14.2 release should have eliminated this possibility, how ever for true multi drive ripping Batch Ripper was designed for that operation (and has been in use 24/7 for the last 4 years by the largest commercial ripping companies out there).
I never experienced problems with multiple instances (neither dBpoweramp nor EAC). But, just to point it out, would you recommend Batch Ripper even for ordinary PCs with two or three drives? Or is it exclusively bundled with extra hardware?

QUOTE (trail @ Mar 30 2012, 18:44) *
Then it's just the issue of "how do we back up and make 10TB of data network accessible on the budget of a college radio station?"
Yeah, it's a matter of cost. But, if you do not insist in quite and most energy-saving technology, you'll get 10 TB rather cheap (except hdd's itself, maybe). And, keep in mind, that you don't have to start with 10 TB. You can add HDDs later along with the ripping progress.

QUOTE (trail @ Mar 30 2012, 18:44) *
but the folders for different types of music is an excellent idea. We get lots of promo material and compilation albums so it's probably a good idea to separate the music into broad categories.

Here's the example folder hierarchy I was thinking of:
\Library\Category\Artist\Album (ID number we add when we get it)\Track number. Artist - Title

or for a real-life example:
Library\Full Albums\Modeselektor\Monkeytown (30458)\04. Modeselektor - Evil Twin.flac
My approach is similar. When it comes to artist's names or titles with exotic characters, you might want to rename/sanitize your folder names. Especially, when you work multiplatform (Linux file server, Windows Client). However, if you don't care so much, you could also use your ID number, probably structured: $left(%ID Number%, 2) / $left(%ID Number%, 3) / %ID Number% / %track% = %artist% = %title%.flac.

QUOTE (trail @ Mar 30 2012, 18:44) *
It just seems like the biggest hurdle now is to store and back up all these terabytes of data we're going to create by going lossless and still make them accessible to the other computers on our local network.
Again, you don't have to provide the full disk space _now_.
Go to the top of the page
+Quote Post
phofman
post Apr 2 2012, 15:57
Post #28





Group: Members
Posts: 261
Joined: 14-February 12
Member No.: 97162



We have been using linux raid in our company for a few years, from 2 to 8 drives configurations. This is just my experience:

Raid is definitely not a backup, I absolutely agree. In fact I was going to ask about the planned backup procedure. However, backing up 10TB of data on low budget is not a trivial task.

I assume the server should run 24/7 since it is the source of music for the radio station. That is why I suggested redundant raid and server hardware (yet inexpensive one).

Linux raid has no problem being used and running synchronization at the same time. Synchronizing 12TB software raid will take many hours, easily a day or two on a mildly loaded server. I agree that raid5 with only one drive of redundancy is not very safe. Six 3TB drives would allow the much safer raid6 of 12TB.

As for the backup, either the rather expensive tape, or IMO a much more flexible solution is a simple desktop PC with large case and motherboard with 6 sata ports, 5 3TB drives in RAID5, one small drive for system with linux (to make life easier) and run rsync every night/week over gigabit ethernet. I do not assume many changes on the main server data array so the synchronization would take just a few seconds. Preferrably the machine should be located in a different building. This solution would have the advantage of being able to take over the file-serving role of the main server quickly in case of hardware failure. It can be booted and shut-down automatically, by bios/halt command, to minimize electricity costs and hard drives wear.

This post has been edited by phofman: Apr 2 2012, 16:00
Go to the top of the page
+Quote Post
krabapple
post Apr 2 2012, 16:30
Post #29





Group: Members
Posts: 2159
Joined: 18-December 03
Member No.: 10538



Nothing to add except again: go lossless if you can. You can always make mp3 versions of your FLAC files for everyday use, and keep the lossless versions as an archive.
Go to the top of the page
+Quote Post
pdq
post Apr 2 2012, 16:39
Post #30





Group: Members
Posts: 3305
Joined: 1-September 05
From: SE Pennsylvania
Member No.: 24233



For backup I would suggest AudioSAFE. If you never need to use it then it is completely free. You only get charged if you need to access your backed-up data.

In fact, you might consider backing up the lossless files to AudioSAFE, before you convert them to mp3 and delete the original. If you never want the lossless version then it has cost you nothing to save them, but if your budget at a later time permits then you don't have to rerip.
Go to the top of the page
+Quote Post
rick.hughes
post Apr 2 2012, 19:12
Post #31





Group: Members
Posts: 110
Joined: 16-February 07
Member No.: 40679



The original CDs could also be considered the backup. If they are not CD-R then they should have a good lifetime. Depending on how difficult some might be to replace this might be all that is really needed for a backup. Store them at another location in case of disaster.
Go to the top of the page
+Quote Post
phofman
post Apr 2 2012, 19:25
Post #32





Group: Members
Posts: 261
Joined: 14-February 12
Member No.: 97162



QUOTE (rick.hughes @ Apr 2 2012, 19:12) *
The original CDs could also be considered the backup. If they are not CD-R then they should have a good lifetime. Depending on how difficult some might be to replace this might be all that is really needed for a backup. Store them at another location in case of disaster.


CDs are definitely a very good backup. It all goes down to the time it took to rip them. If that time spent has lower value than the backup solution, then yes. Considering the time it takes to rip a few thousand CDs which fit onto one harddrive with presumed lifetime of 2 - 3 years (loaded 24/7, regular consumer type HDD), I would tend to prefer building a simple low-cost backup PC. Quality of drives is not going stelar, in fact manufacturers are cutting the warranty period - http://www.tomshardware.com/news/seagate-w...land,14322.html .
Go to the top of the page
+Quote Post
garym
post Apr 2 2012, 22:25
Post #33





Group: Members
Posts: 505
Joined: 16-September 06
From: United States
Member No.: 35261



QUOTE (pdq @ Apr 2 2012, 10:39) *
For backup I would suggest AudioSAFE. If you never need to use it then it is completely free. You only get charged if you need to access your backed-up data.

In fact, you might consider backing up the lossless files to AudioSAFE, before you convert them to mp3 and delete the original. If you never want the lossless version then it has cost you nothing to save them, but if your budget at a later time permits then you don't have to rerip.


excellent ideas!
Go to the top of the page
+Quote Post
Jan S.
post Apr 3 2012, 10:42
Post #34





Group: Admin
Posts: 2548
Joined: 26-September 01
From: Denmark
Member No.: 21



QUOTE (garym @ Apr 2 2012, 22:25) *
QUOTE (pdq @ Apr 2 2012, 10:39) *
For backup I would suggest AudioSAFE. If you never need to use it then it is completely free. You only get charged if you need to access your backed-up data.

In fact, you might consider backing up the lossless files to AudioSAFE, before you convert them to mp3 and delete the original. If you never want the lossless version then it has cost you nothing to save them, but if your budget at a later time permits then you don't have to rerip.


excellent ideas!

If the files are deleted locally they will be dropped from AudioSAFE after some time. So you have to keep the files locally you want to backup on audioSAFE.
Go to the top of the page
+Quote Post
Porcus
post Apr 3 2012, 16:29
Post #35





Group: Members
Posts: 1779
Joined: 30-November 06
Member No.: 38207



QUOTE (phofman @ Apr 2 2012, 16:57) *
the rather expensive tape


The tape drives are expensive, but tapes are only slightly more than hard drives, so ... are there rental services? Or backup services where you would show up physically, have your drives copied, and if you are so unlucky you need restoration, show up with tapes and new drives? (I suspect that if such exist, they are at industry-grade pricing.)


--------------------
One day in the Year of the Fox came a time remembered well
Go to the top of the page
+Quote Post
Nessuno
post Apr 3 2012, 17:52
Post #36





Group: Members
Posts: 422
Joined: 16-December 10
From: Palermo
Member No.: 86562



QUOTE (Jan S. @ Apr 3 2012, 11:42) *
If the files are deleted locally they will be dropped from AudioSAFE after some time. So you have to keep the files locally you want to backup on audioSAFE.


Actually? That's quite a strange behavior for a backup system. Sounds more like an... asyncronous and delocalized RAID 1! wink.gif

Well, all in all it depends on how long is "some time"...

This post has been edited by Nessuno: Apr 3 2012, 17:54


--------------------
... I live by long distance.
Go to the top of the page
+Quote Post
spoon
post Apr 3 2012, 19:33
Post #37


dBpowerAMP developer


Group: Developer (Donating)
Posts: 2725
Joined: 24-March 02
Member No.: 1615



Your deletions are kept on audiosafe (the last change) indefinitely.

All we ask is that your account is active, that is a computer logs into audiosafe once every so often.


--------------------
Spoon http://www.dbpoweramp.com
Go to the top of the page
+Quote Post
Destroid
post Apr 4 2012, 11:54
Post #38





Group: Members
Posts: 544
Joined: 4-June 02
Member No.: 2220



QUOTE (trail @ Apr 2 2012, 00:42) *
Bad news, our budget means we probably won't be able to afford the equipment to go full FLAC. sad.gif We'll probably stick with V0.

I have to ask again, why insist on this setting? I don't want to derail this thread into the dozens (hundreds?) of other listening threads, but it seems storage costs are relevant. So, I refer to my previous post of -V3...-V5, or, AAC 64/96kbps.

The hardware considerations will add up, plus (as already mentioned) there are redundancy options abound.


--------------------
"Something bothering you, Mister Spock?"
Go to the top of the page
+Quote Post
trail
post Apr 5 2012, 20:41
Post #39





Group: Members
Posts: 9
Joined: 29-March 12
Member No.: 98189



QUOTE (Destroid @ Apr 4 2012, 06:54) *
QUOTE (trail @ Apr 2 2012, 00:42) *
Bad news, our budget means we probably won't be able to afford the equipment to go full FLAC. sad.gif We'll probably stick with V0.

I have to ask again, why insist on this setting? I don't want to derail this thread into the dozens (hundreds?) of other listening threads, but it seems storage costs are relevant. So, I refer to my previous post of -V3...-V5, or, AAC 64/96kbps.

The hardware considerations will add up, plus (as already mentioned) there are redundancy options abound.


It's just that I've always found V0 to be the optimum balance between quality and file size for myself. I don't have an aversion to other formats, (except AAC is probably out because we don't own any Macs or use iTunes frequently) but I'm not terribly well-versed in the trade off between file size and audio quality when using different VBR settings. But I did just do an A-B test between a song that I have both FLAC and V5 versions of and couldn't discern any noticeable difference between them. Other formats are definitely a consideration, I just assumed V0 was the happy medium.
Go to the top of the page
+Quote Post
mixminus1
post Apr 5 2012, 21:06
Post #40





Group: Members
Posts: 684
Joined: 23-February 05
Member No.: 20097



@Destroid: Yes, the OP's budget is limited, and no lossy codec can truly be an "archival" format.

However, since this is for a music archive at a radio station, we should assume that some form of additional processing and/or lossy transcoding may take place in the broadcasting/webcasting chain.

As such, to try and hit the "sweet spot" of minimizing storage space while providing maximum resistance to downstream artifacts, I think V0 makes the most sense, if AAC is really not an option.

While this test is almost seven years old now, I think it's still useful for the discussion at hand:

http://www.hydrogenaudio.org/forums/index....st&p=282909

That was done with LAME 3.97 a8, so not current, but not exactly "ancient", either, and it fared the worst as a source codec when transcoding to MP3 ABR 128 - and that was with -V0! Going to a lower bitrate certainly wouldn't improve things.

This post has been edited by mixminus1: Apr 5 2012, 21:08


--------------------
"Not sure what the question is, but the answer is probably no."
Go to the top of the page
+Quote Post
JJZolx
post Apr 6 2012, 01:43
Post #41





Group: Members
Posts: 378
Joined: 26-November 04
Member No.: 18345



If the budget is so limited that the station can't even afford the hard drive space on which to store the music in a lossless format, then I hate to say it, but 'm afraid the project is doomed from the get go. You're likely to patch together a system where the next guy to come in says "Just look at what they left me" and will want to start all over again, perhaps with a more realistic budget.

There is much better software available than foobar or JRiver that is actually designed for radio station programming. You have to take into account things like queueing advertisements, news briefs, pre-recorded shows, not to mention downloading those shows 24x7 as they become available. Then there's the whole side of things where you may want to digitize the outgoing on-air feed and make it available at one or more bitrates in an internet radio feed. Also, you may want to generate a web page displaying your playlist of songs as they're played. Managing all of that at the file level with something like foobar would be a nightmare, if it's even doable.
Go to the top of the page
+Quote Post
shadowking
post Apr 6 2012, 15:50
Post #42





Group: Members
Posts: 1523
Joined: 31-January 04
Member No.: 11664



QUOTE (Destroid @ Apr 4 2012, 21:54) *
QUOTE (trail @ Apr 2 2012, 00:42) *
Bad news, our budget means we probably won't be able to afford the equipment to go full FLAC. sad.gif We'll probably stick with V0.

I have to ask again, why insist on this setting? I don't want to derail this thread into the dozens (hundreds?) of other listening threads, but it seems storage costs are relevant. So, I refer to my previous post of -V3...-V5, or, AAC 64/96kbps.

The hardware considerations will add up, plus (as already mentioned) there are redundancy options abound.



I agree. I did 300+ albums with V4 years ago and don't regret it as they mostly sound great . That is 150k vs 250..300k(v0 / itunes) bloat of today. No need for RAID or anything fancy.

This post has been edited by shadowking: Apr 6 2012, 15:50


--------------------
Wavpack -b450x1
Go to the top of the page
+Quote Post
Porcus
post Apr 7 2012, 02:58
Post #43





Group: Members
Posts: 1779
Joined: 30-November 06
Member No.: 38207



OK, so OP cannot afford drives enough for lossless.

Let me assume that the CDs average at 40 minutes. (If they are singles -- much less.) 20 000 hours. 72 million seconds. Want to fit on a single hard drive of 3 TB = 24e+12 bits? Divide out to get one third megabit per second. That's the 320 rate. Might as well use V0.


--------------------
One day in the Year of the Fox came a time remembered well
Go to the top of the page
+Quote Post
trail
post Apr 9 2012, 04:20
Post #44





Group: Members
Posts: 9
Joined: 29-March 12
Member No.: 98189



QUOTE (mixminus1 @ Apr 5 2012, 16:06) *
@Destroid: Yes, the OP's budget is limited, and no lossy codec can truly be an "archival" format.

However, since this is for a music archive at a radio station, we should assume that some form of additional processing and/or lossy transcoding may take place in the broadcasting/webcasting chain.

As such, to try and hit the "sweet spot" of minimizing storage space while providing maximum resistance to downstream artifacts, I think V0 makes the most sense, if AAC is really not an option.

While this test is almost seven years old now, I think it's still useful for the discussion at hand:

http://www.hydrogenaudio.org/forums/index....st&p=282909

That was done with LAME 3.97 a8, so not current, but not exactly "ancient", either, and it fared the worst as a source codec when transcoding to MP3 ABR 128 - and that was with -V0! Going to a lower bitrate certainly wouldn't improve things.



Exactly what I was thinking. There is some processing and the like, but it's not any that really takes much away from the music. To put it in perspective, we're a college FM station, and it's safe to assume very few of our listeners are tuning in looking for pristine audio fidelity, you know? I obviously want it to sound as best as I can, but V0 sounds like the optimal trade-off. As a college kid I don't quite have taste, budget, or equipment of an audiophile laugh.gif but V0 hasn't ever disappointed my ears.
Go to the top of the page
+Quote Post
gorob
post Mar 11 2014, 03:16
Post #45





Group: Members
Posts: 1
Joined: 13-September 12
Member No.: 103122



I second dbpoweramp as well, it works great for ripping CDs to FLAC.
Some at their forum also suggested using dbpoweramp with the nimbie CD loader, the robot can feed 100 CDs automatically.
It surely beats feeding the disc one by one by hand…
Go to the top of the page
+Quote Post
JJZolx
post Mar 11 2014, 04:01
Post #46





Group: Members
Posts: 378
Joined: 26-November 04
Member No.: 18345



I wonder how this project turned out. Or if it even got off the ground.
Go to the top of the page
+Quote Post
eahm
post Mar 11 2014, 05:43
Post #47





Group: Members
Posts: 884
Joined: 11-February 12
Member No.: 97076



30,000 discs...I can't even imagine that many, amazing and insane if it's a common person. It must be a radio station or something right?
Go to the top of the page
+Quote Post
Porcus
post Mar 11 2014, 08:44
Post #48





Group: Members
Posts: 1779
Joined: 30-November 06
Member No.: 38207



QUOTE (eahm @ Mar 11 2014, 05:43) *
It must be a radio station or something right?


Yep - read the original posting ;-)


--------------------
One day in the Year of the Fox came a time remembered well
Go to the top of the page
+Quote Post
kennedyb4
post Mar 11 2014, 13:10
Post #49





Group: Members
Posts: 772
Joined: 3-October 01
Member No.: 180



Hi. I would like to add my advice to those suggesting you go lossless, particularly flac. I have ripped and re-ripped my cd's more times than I care to count. I used to use mp3 abr 192 then 320. Then I switched to aac at 128 and then opus for my rockbox.
Lots of replicated work for no reason except saving storage which is now crazy cheap.

You will save space in the endrun. $.02
Go to the top of the page
+Quote Post
yourlord
post Mar 11 2014, 18:23
Post #50





Group: Members
Posts: 172
Joined: 1-March 11
Member No.: 88621



This thread is 2 years old and the original poster appears to be long gone. Adding suggestions now is pointless.
Go to the top of the page
+Quote Post

3 Pages V  < 1 2 3 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 21st April 2014 - 04:45