Using Linux to fix flash drive problems

I frequently have to deal with borked usb drives, where the drive is presenting itself as having some capacity below the real capacity, usually as a result of applying ISO images. This is fixable in Linux at the command line, but I can never remember all the steps, so below is the result of me finally getting my notes together in one place.

Steps below are based on the information at a digital forensics site, augmented by further research I had to do to handle problems that came up.


1) find the device (/dev/sdc, /dev/sde, etc) you want to work with
 
    sudo fdisk -l
    or
    sudo blkid
 
    are both good ways to see lists you can pick that information out of.
 
    blkid will require you to understand that you must drop the last digit, as it will list partitions like /dev/sdc1, /dev/sdc2, etc. "fdisk -l" will list both the partitions and the base device.

2) overwrite the device with with all zeros

    sudo dd if=/dev/zero of=/dev/sde bs=4M

    The block size can affect speed by quite a lot, but you have to have already performed trials on the device to really know *which* bs=xx numbers to use. When in doubt, just chunk in 4M and accept the wait.

    See this page for discussion of why /dev/zero is generally acceptable even in secure situations.
    See comments under the digital forensics article  for discussion of theoretical reasons some might still prefer to use 'shred' or 'random' overwriting methods.

3) create a new partition table of msdos type

    (You may have to eject/unmount first to prevent 'in use' parted errors)

    sudo parted /dev/sde mklabel msdos

    NOTE that parted calls tables by the name 'label', thus 'mklabel' instead of 'mkparttable' or some such. Dunno why, presumably some historical thing I'm too apathetic to look up.

4) create a new primary partition intended for fat32 filesystem, starting on the 8th sector after the partition table and ending at the end of available space:

    sudo parted /dev/sde mkpart primary fat32 8s 100%

    Starting sector alignment can affect performance a lot. Parted will complain if you specify a starting sector that is not optimal; you can tell it to ignore the problem and push on, but the device will very likely not perform at its best it you do that. Here's how to gather the info needed to tell parted the right starting point for best performance for a particular device:

get topology info from the system:
    cat /sys/block/sde/queue/optimal_io_size
    cat /sys/block/sde/queue/minimum_io_size
    cat /sys/block/sde/alignment_offset
    cat /sys/block/sde/queue/physical_block_size

Each of those commands will print something to the screen.

If optimal_io_size is not zero, then calculate the starting sector like this:

    start_sector = (optimal_io_size + alignment_offset)/physical_block_size

Assume the formula gave a start_sector value of 2048. Then your parted command would be:

    sudo parted /dev/sde mkpart primary fat32 2048s 100%

If optimal_io_size is zero, then if alignment_ offset is also zero and minimum_io_size is a power of 2, then use a start_sector of 1M:

    sudo parted /dev/sde mkpart primary fat32 1M 100%


If the above conditions are not met, use the reported minimum_io_size. Assuming minimum_io_size = 512:

   sudo parted /dev/sde mkpart primary fat32 512B 100%

      NOTE the 'B', specifying start point in bytes, not sectors.


If even minimum_io_size is not defined, use the physical_block_size. Assuming physical_block_size was reported as 512, the command would look the same as above. If yours is different, just replace the 512B with (physical_block_size)B, without the parenthesis.

I got straightened out on clearing that annoying parted alignment error by reading these pages:
     http://rainbow.chard.org/2013/01/30/how-to-align-partitions-for-best-performance-using-parted/
     http://h10025.www1.hp.com/ewfrf/wc/document?cc=uk&lc=en&dlc=en&docname=c03479326



5) format the partion as fat32 (note the '1' appended to the device path, for 1st partition)

    sudo mkfs -t vfat -F 32 /dev/sde1

6) view what you've got so far:

    sudo parted /dev/sde print

7) give the device a volume label:

    mlabel -i /dev/sde1 ::mydiskname

    Here again, specified sde1 rather than sde. The volume label part is 'mydiskname'. That is what will show up next to the drive letter in Windows Explorer, or as the path to the device in the media folder in linux ( /media/mydiskname in Crunchbang distribution, it varies a little in others).

    NOTE that mlabel is msdos-specific. If you formatted the drive as ext2,  you would do it differently (e2label, I think).

    ALSO NOTE:

• you need the mtools package to get mlabel.
• The mlabel syntax above is undocumented, at least in my system's man pages. I got the info from here.
• you may need to add  'mtools_skip_check=1' line to your ~/.mtoolsrc file (create file if isn't there) to prevent errors related to drive sizes not being what mlabel expects.