Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Encoding Directly to a Flash based Player (Read 3095 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Encoding Directly to a Flash based Player

I've got a cheap 128MB (usb flash drive type) mp3 player, it's a pretty slow device and it takes about 5 to 6 minutes just to copy approx 120MB of files to it.  The other day it occured to me that if I had a fast enough encoder that could just about transcode on the fly at that speed (from lossless stored on the HDD).

Anyway I thought this might be quicker than encoding the files and then copying them, I thought at worst that's how it would be but was hoping that encoding and writing would to some extent be concurrent and I'd save time.

Well the result was surprising, it actually took very much longer than even the sum of the two individual processes (encoding + copying). I was careful to not use intermediate wave file and double checked that the encoder was not trying to place temporary files onto the flash player.

Anyway it took about 11 minutes to encode the selected files and about 5.5 minites to copy them when doing it seperately, but it took about 45 minutes to encode them directly to the flash player. What's the bottleneck here?

PS. At first I was using the "convert" function in foobar but I also tried encoding directly from lame (and fastencc) from wav files using the command line and there was a similar unexpected slow down (as in more than I'd expect due just to the device write speed limitaion) when encoding directly to a flash player.

?any clues?

Encoding Directly to a Flash based Player

Reply #1 –
Hi,

Windows is not efficient in combining write operations. That prevents small repeated writes to flash memory from being efficient.

Example: We have a player which uses a flash memory which is divided in 64kB segments. So, each of the following characters (X, 0, 1) represent:

X = 64kB of undefined data
1 = 64kB of '1' bits
0 = 64kB of '0' bits

Example (ideal):
Code: [Select]
data = XXXXXXXX
ERASE (VERY SLOW)
data = 11111111
WRITE
data = 10101010


Example (Windows):
Code: [Select]
data = XXXXXXXX
ERASE (VERY SLOW)
data = 11111111
WRITE
data = 10000000
ERASE (VERY SLOW)
data = 11111111
WRITE
data = 10100000
ERASE (VERY SLOW)
data = 11111111
WRITE
data = 10101000
ERASE (VERY SLOW)
data = 11111111
WRITE
data = 10101010


For every write to flash, a slow ERASE cycle are necessary. That's probably why it's so slow.

Can you try disabling the Windows option "optimize for quick removal" ? It may help.

Encoding Directly to a Flash based Player

Reply #2 –
Thanks for the info NumLOCK. So what you're saying is that unless a full segment is written at once then there may be numerous redundant erase cycles on the flash memory.

Is that a correct summary or am I misunderstanding you?

Encoding Directly to a Flash based Player

Reply #3 –
Absolutely well summarized 

It's still a supposition, but since audio encoders tend to generate one frame at a time, they make many small "file write" calls per megabyte. The hard disks don't care, but on flash memory, it generates many accesses on the same segments.

There are various kinds of flash memory controllers. The most clever ones do "wear leveling", which mean that each time a write is done, they move the target segment around, thus preventing individual segments from dying (as is the case with flash memory).

Most probably, "clever" controllers would combine small writes as well. It all depends on what's in your player.

Other OS's (such as linux) do very agressive write combining in the kernel, so you *might* get the same performance as plain mp3 encoding to hard-disk (or the performance of a plain file copy to flash, whichever is lower).

Regards 

Encoding Directly to a Flash based Player

Reply #4 –
Quote
PS. At first I was using the "convert" function in foobar but I also tried encoding directly from lame (and fastencc) from wav files using the command line and there was a similar unexpected slow down (as in more than I'd expect due just to the device write speed limitaion) when encoding directly to a flash player.

?any clues?

I just gave this a try from the win2000 command prompt , usb 1.1 flash player -there was only a 1 second difference between encoding to the player and the hard disk.
The player shows as a standard removable storage device using disk.sys and write caching is not available in properties (probably managed as a neccessity)

There must be some luck involved then with the players capability and maybe the OS version. This player takes a few minutes to fill up 256 megs.
no conscience > no custom

Encoding Directly to a Flash based Player

Reply #5 –
Thanks for the info guys. Based on this I changed the settings (policy) for the USB drive from "optimize for quick removal" to "optimize for performance" and now I can encode directly to the flash drive without any performance penalty.

In addition to the "optimize for performance" checkbox I also have an option of "Enable Write Cache".  But though I can tick this option every time I return to this properties-policy tab the tick is gone. I suppose it doesn't matter as the "optimize for performance" setting alone seems to be doing the trick.

I'm now very glad I asked this question and got this sorted out, it couldn't be good for the longevity of the flash drive to have all those redundant erase cycles going on.
 

Encoding Directly to a Flash based Player

Reply #6 –
Great !! 

Now, just ensure that you inform windows (using the icon in the notification area) before you unplug your player from the USB port, otherwise the buffers might not get flushed in time.