On Fri, 21 Sep 2018 14:34:06 +1000 Cameron Simpson <cs(a)cskk.id.au> wrote:
On 20Sep2018 21:31, Ranjan Maitra <maitra(a)email.com> wrote:
>On Fri, 21 Sep 2018 12:12:26 +1000 Cameron Simpson <cs(a)cskk.id.au> wrote:
>> On 20Sep2018 19:06, Ranjan Maitra <maitra(a)email.com> wrote:
>> >I am sorry to return to this, but I am still struggling to find out the
answer
>> >to how to find files that had bad blocks in my old drive.
[...]
>> >ddrescue on the new disk found no problems, but I know there are affected
>> >files (the source disk was over 95% full), and I would like to know what
>> >those files are called.
>>
>> See "man xfs_db". It looks like the "blockget -v -b
block-number" command may
>> provide useful info.
>
>Thanks very much for this information: it is very helpful.
>
>I do know the bytes and can figure out the sectors that are damaged. But I
>can't figure out how that translates to the block number of an xfs filesystem.
>Is there any reference for how to do this?
I would presume that it counts from the start of the partition containing the
file system. The xfs_db command "sb" consults the filesystem superblock and
should tell you the block size (4KiB is typical I think).
So I would start by:
- find the start sector of the partition containing the filesystem (counting
from zero); that is your base
- subtract that from the bad sector number; that should be the sector offset
into the XFS partition
- divide by the number of sectors per block (eg 512 byte sectors and 4KiB
blocks would mean 8 sectors per block); hopefully ddrescue tells you what
sector size it is using
Look up that block with "blockget" and then see if the corresponding file seems
to have corresponding damage in its data.
Disclaimer: I haven't tried this. But I can if you run into trouble, and I also
know an XFS expert I might consult if we can't figure this out ourselves.
I wanted to provide a bit more information on progress into my and my wife's efforts
to rescue files from one of her failing hard drives to another of the same type, but in
better health.
Here's what we did to discover the files that were damaged on the broken drive. I had
used ddrescue (wonderful utility, very fast) to get the files off of the failing disk onto
/dev/sdc.
I'm still not sure if what we did has worked because none of the files that were in
damaged sectors appear to be damaged, but they are big text files and it is hard to know
for sure.
(1) We located the start of the xfs partition in question:
$ sudo fdisk -l /dev/sdc
Disk /dev/sdc: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: ED0AA573-BFBA-4223-9036-D1B7EB0E49B9
Device Start End Sectors Size Type
/dev/sdc1 2048 7814023167 7814021120 3.7T Linux filesystem
It starts on byte partition_start_byte = 1048576. I notice my block size is block_size =
4096 (and confirm it with xfs_db -c "sb" -c "print").
(2) My wife wrote a script to extract the affected bytes from the ddrescue mapfile and
converted them into file system blocks (fsb), as used by xfs_db. Each ddrescue block has
a start_byte and a byte_length. The conversion is:
start_block = floor( (start_byte - partition_start_byte) / block_size )
block_length = ceil( byte_length / block_size)
We then pasted the results from all such sets of blocks together in a massive xfs_db
command as follows:
xfs_db -c "blockget -n" -c "fsb start_block" -c "blockuse
-n" -c "fsb start_block + 1" -c "blockuse -n" ... -c "fsb
start_block + block_length" -c "blockuse -n" ...
We waited several minutes, then parsed the affected files from the stdout of the above
command. The python script to do the job is below:
Here's the resource that finally helped me grok xfs well enough to get this far:
https://righteousit.wordpress.com/2018/05/21/xfs-part-1-superblock/
---- python script ----
#!/usr/bin/python3
import math
import subprocess as sp
device = "/dev/sdc1" # my device
mapfile = "mapfile" # ddrescue mapfile
partition_byte_offset = 1048576 # the first byte of my partition
block_size = 4096 # my block size in bytes
with open(mapfile, "r") as fh:
cmd = ["xfs_db", "-c", "blockget -n"]
for line in fh:
if not line.startswith("#"):
words = line.split()
if words[2] == '-': # bad bytes
start = int(words[0], 16)
length = int(words[1], 16)
start_block = math.floor((start - partition_byte_offset) / block_size)
length_block = math.ceil(length / block_size)
print(str(start_block) + ", " + str(length_block))
for blk in range(start_block, start_block + length_block):
cmd.extend(["-c", "fsb " + str(blk), "-c",
"blockuse -n"])
cmd.append(device)
#print(" ".join(cmd))
out = sp.run(cmd, stdout = sp.PIPE)
print(out.stdout)
---- end script ------
Is this the right thing to do?
Many thanks,
Ranjan