Re: Pointers to mirroring partitions (w/ encryption?) help?

[Justin Brown]

Here's some thoughts:

> Assume a CD sized (680MB) /boot

Some distros carry patches for grub that allow booting from Btrfs, so
no separate /boot file system is required. (Fedora does not; Ubuntu --
and therefore probably all Debians -- does.)

> perhaps a 200MB (?) sized EFI partition

Way bigger than necessary. It should only be 1-2MiB, and IIRC 2MiB
might be the max UEFI allows.

>  then creates another partition for mirroring, later. IIUC, btrfs add device 
> /dev/sda4 / is appropriate, then. Then running a balance seems recommended.

Don't do this. It's not going to provide any additional protection
that you can't do in a smarter way. If you only have one device and
want data duplication, just use the `dup` data profile (settable via
`balance`). In fact, by default Btrfs uses the `dup` profile for
metadata (and `single` for data). You'll get all the data integrity
benefits with `dup`.

One of the best features and initally confusing things about Btrfs is
how much is done "within" a file system. (There is a certain "the
Btrfs way" to it.)

> Confusing, however, is having those (both) partitions encrypted. Seems some 
> work is needed beforehand. But I've never done encryption.

(This is moot if you go with `dup`.) It's actually quite easy with
every major distro. If we're talking about a fresh install, the distro
installer probably has full support for passphrase-based dm-crypt LUKS
encryption, including multiple volumes sharing a passphrase. An
existing install should be convertable without much trouble. It's
ususally just a matter of setting up the container with `cryptsetup`,
populating `/etc/crypttab`, possibly adding crypto modules to your
initrd and/or updating settings, and rebuilding the initrd. (I have
first-hand experience doing this on a Fedora install recently, and it
took about half an hour and I knew nothing about Fedora's `dracut`
initrd generator tool.)

If you do need multiple encrypted file systems, simply use the same
passphrase for all volumes (but never do this by cloning the LUKS
headers). You'll only need to enter it once at boot.

> The additional problem is most articles reference FDE (Full Disk Encryption) 
> - but that doesn't seem to be prudent. e.g. Unencrypted /boot. So having 
> problems finding concise links on the topics, -FDE -"Full Disk Encryption".

Yeah, when it comes to FDE, you either have to make your peace with
trusting the manufacturer, or you can't. If you are going to boot your
system with a traditional boot loader, an unencrypted partition is
mandatory. That being said, we live in a world with UEFI Secure Boot.
While your EFI parition must be unencrypted vfat, you can sign the
kernels (or shims), and the UEFI can be configured to only boot signed
executables, including only those signed by your own key. Some distros
already provide this feature, including using keys probably already
trusted by the default keystore.

> mirror subvolumes (or it inherently comes along for the ride?)

Yes, that is correct. Just to give some more background: the data and
metadata profiles control "mirroring," and they are set at the file
system level. Subvolumes live entirely within one file system, so
whatever profile is set in the FS applies to subvolumes.

> So, I could take an HD, create partitions as above (how? e.g. Set up 
> encryption / btrfs mirror volumes), then clonezilla (?) partitions from a 
> current machine in.

Are you currently using Btrfs? If so, use Btrfs' `send` and `receive`
commands. That should be lot friendlier to your SSD. (I'll take this
opportunity to say that you need to consider the `discard` mount *and*
`/etc/crypttab` options. Discard -- or scheduling `fstrim` -- is
extremely important to maintain optimal performance of a SSD, but
there are some privacy trade-offs on encrypted systems.) If not, then
`cp -a` or similar will work. Obviously, you'll have to get your boot
mechanism and file system identifiers updated in addition to
`/etc/crypttab` described above.

Lastly, strongly consider `autodefrag` and possibly setting some
highly violatile -- but *unimportant* -- directories to `nodatacow`
via purging and `chattr +C`. (I do this for ~/.cache and /var/cache.)

> Yet not looking to put in a 2nd HD

If you change your mind and decide on a backup device, or even if you
just want local backup snapshots, one of the best snapshot managers is
btrfs-sxbackup (no association with the FS project).

On Fri, Jun 3, 2016 at 3:30 PM, B. S.  wrote:
> Hallo. I'm continuing on sinking in to btrfs, so pointers to concise help
> articles appreciated. I've got a couple new home systems, so perhaps it's
> time to investigate encryption, and given the bit rot I've seen here,
> perhaps time to mirror volumes so the wonderful btrfs self-healing
> facilities can be taken advantage of.
>
> Problem with today's hard drives, a quick look at Canada Computer shows the
> smallest drives 500GB, 120GB SSDs, far more than the 20GB or so an OS needs.
> Yet not 

Re: RAID1 vs RAID10 and best way to set up 6 disks

[Justin Brown]

> Mitchell wrote:
> With RAID10, there's still only 1 other copy, but the entire "original"
disk is mirrored to another one, right?

No, full disks are never mirrored in any configuration.

Here's how I understand Btrfs' non-parity redundancy profiles:

single: only a single instance of a file exists across the file system
dup: two instances of a file exist across the file system, and they
may reside on the same physical disk (4.5.1+ required to use dup
profile on multi-disk file system)
raid1: same as dup but the instances are guaranteed to be on different disks
raid0: single but  can be striped between multiple disks
raid10: data is guaranteed to exist on two separate devices but if n>2
the data is load balanced between disks*

Even though my explanation is imperfect, I hopes that illustrates that
Btrfs RAID is different than traditional RAID. Btrfs provides the same
physical redundancy as RAID, but the implementation mechanisms are
quite a bit different. This has wonderful consequences for
flexibility, and it's what allowed me to run a 5x2TB RAID10 array for
nearly two years and essentially allow complete allocation. The
downside is that since allocations aren't enforced from start (eg. MD
requiring certain number of disks and identical sizes), it's possible
to get weird allocations over time, but the resolution is simple: run
a balance from time to time.

> Christoph wrote:
> Especially, when you have an odd number devices (or devices with
different sizes), its not clear to me, personally, at all how far that
redundancy actually goes respectively what btrfs actually does... could
be that you have your 2 copies, but maybe on the same device then?

RAID1 (and transitively RAID10) guarantees two copies on different
disks, always. Only dup allows the copies to reside on the same disk.
This is guaranteed is preserved, even when n=2k+1 and mixed-capacity
disks. If disks run out of available chunks to satisfy the redundancy
profile, the result is ENOSPC and requires the administrator to
balance the file system before new allocations can succeed. The
question essentially is asking if Btrfs will spontaneously degrade
into "dup" if chunks cannot be allocated on some devices. That will
never happen.


On Fri, Jun 3, 2016 at 1:42 PM, Mitchell Fossen  wrote:
> Thanks for pointing that out, so if I'm thinking correctly, with RAID1
> it's just that there is a copy of the data somewhere on some other
> drive.
>
> With RAID10, there's still only 1 other copy, but the entire "original"
> disk is mirrored to another one, right?
>
> On Fri, 2016-06-03 at 20:13 +0200, Christoph Anton Mitterer wrote:
>> On Fri, 2016-06-03 at 13:10 -0500, Mitchell Fossen wrote:
>> >
>> > Is there any caveats between RAID1 on all 6 vs RAID10?
>> Just to be safe: RAID1 in btrfs means not what RAID1 means in any
>> other
>> terminology about RAID.
>>
>> The former has only two duplicates, the later means full mirroring of
>> all devices.
>>
>>
>> Cheers,
>> Chris.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: moving btrfs subvolumes to new disk

[Justin Brown]

I'm not an expert by any means, but I did a migration like this a few weeks ago.

The most consistent recommendation on this mailing list is to use the
newest kernels and btrfs-progs feasible. I did my migration using
Fedora 24 live media, which at the time was kernel ~4.3. I see your
btrfs-progs is a little old. Maybe the kernel is as well and has a
bug.

I know you tried a bunch of variations, but it would be helpful if you
posted the commands you've tried. Perhaps the most obvious problem
that could be occurring is trying to send a RW subvol, which is
illegal. I can't tell if you made a RO snapshot first, but your
success with ROOT indicates you're aware of this limitation. I'm
guessing that there's some problem with the destination path that
you're providing. The path should be the containing subvol where you
want it to go, not the destination name (i.e. btrfs receive
/var/media/backups/ not btrfs receive /var/media/backups/HOME).

Here's some send/receive commands that I've successfully used
recently. Maybe they'll point you in the right direction:

pull a RO snapshot from a remote system:

cd ~/ksp/backups
ssh 192.168.0.122 "sudo btrfs send
/home/justin/ksp/backups/2016-03-15-17:24" | sudo btrfs receive .
mv 2016-03-15-17:24 ../current

Note how I received into the backups directory and have no control
over the initial subvol name. I mv it to the proper location and name
afterwards.

move a RO snapshot between local file system:

cd /tmp/old-btrfs/
sudo btrfs subvol snap -r home home-snap
cd /tmp/new-btrfs/
sudo btrfs send /tmp/old-btrfs/home-snap | sudo btrfs receive .
sudo btrfs subvol snapshot home-snap home
sudo btrfs subvol delete home-snap

Fedora names it's /home subvol "home."

You *shouldn't* need to mess around with advanced mounting options to
get this to work.

On Sun, Mar 20, 2016 at 11:52 AM, Ryan Erato  wrote:
> I do plan on physically replacing the current drive with the new one
> and my fstab/boot comands use device. I never could get UUID or labels
> to work, but that's another project.
>
> However, this still leaves me unable to take advantage of btrfs
> features for implementing an incremental backup solution to another
> disk.
> Ryan Erato
> C. 414.630.5295
> rer...@gmail.com
> http://www.linkedin.com/in/ryanerato
>
>
> On Sun, Mar 20, 2016 at 1:55 AM, Phantom Guy  wrote:
>> You tried to hard.
>>
>> Build the new media If you need encryption or meets devices.
>>
>>
>> Then add the new media to the whole filesystem.
>>
>>
>> Then remove the old media from the filesystem.
>>
>>
>> # btrfs device add /dev/sdb1 /mountpoint
>>
>> # btrfs device del /dev/sda1 /mountpoint
>>
>> Then sync the filesystem and wait.
>>
>> Once the old device ID no longer part of the filesystem it's done.
>>
>> When the sync finishes, the /dev/sda1 device is nill.
>>
>> If you've designed your fstsb and boot commands correctly the label or UUID
>> has migrated so as long as you've prepped the disk with grub of whatever
>> then you can move the disks with impunity.
>>
>> You don't even need to be in any sort of maintenance mode.
>>
>>
>>>
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Fwd: Deleting a Subvol from a Cancelled Btrfs-Send

[Justin Brown]

Hugo,

Many thanks for the help. That was far easier than I was anticipating. :)

Now that I have that cleaned up, I'm having trouble with sending the
delta. Perhaps this is a limitation of send/receive because it seems
odd that btrfs-receive doesn't have a parent option.

I'm sending a snapshot from venus to spaceman using ssh for transport.

I prepared spaceman with

~$ mkdir -p /var/media/backups/venus/
~$ sudo btrfs subvol create /var/media/backups/venus/home

Then on venus, I created a read-only snapshot and sent it spaceman:

~$ sudo btrfs sub snap -r /home home-2014-09-26
~$ sudo btrfs send -vv home-2014-09-26 | ssh root@spaceman
/usr/sbin/btrfs receive -vv /var/media/backups/venus/home

All good so far. Now, I'm trying to send a delta:

~$ sudo btrfs send -vv -p home-2014-09-26 home-2014-10-01 | ssh
root@spaceman /usr/sbin/btrfs receive -vv
/var/media/backups/venus/home

receiving snapshot home-2014-10-01
uuid=60dd57bd-4cd0-394c-ba85-065f646829d7, ctransid=446614
parent_uuid=a994d4b5-6c89-9442-a5cc-ffa1d74b9847,
parent_ctransid=432944
At snapshot home-2014-10-01
It seems that you have changed your default subvolume or you specify
other subvolume to
mount btrfs, try to remount this btrfs filesystem with fs tree, and
run btrfs receive again!

I'm not sure what fs tree is supposed to mean, and Google didn't
have anything useful. On spaceman, the target btrfs file system is
multi-device RAID10 with the media subvol mounted at /var/media and
ovirt subvol at /var/ovirt. There are also the subvols
backups/venus/home and backups/venus/home/home-2014-09-26, which I
suppose is only mounted because I haven't rebooted or unmounted since
creating and sending them, respectively.

This seems like it's falling into the third caveat from `man btrfs-receive`:

3. default subvolume is changed or you don’t mount btrfs filesystem
with fs tree.

I don't have any idea what that means, and the Btrfs mount options
page (https://btrfs.wiki.kernel.org/index.php/Mount_options) doesn't
provide any help.

In a vain effort, I tried setting setting the default subvolume to both

~$ sudo btrfs subvolume set-default 4169
/var/media/backups/venus/home/home-2014-09-26

and then

~$ sudo btrfs subvolume set-default 4170
/var/media/backups/venus/home/home-2014-09-26

Neither had any effect on the error output.

It really seems like the command should be something like:

~$ sudo btrfs send -vv -p home-2014-09-26 home-2014-10-01 | ssh
r...@spaceman.lab.fandingo.org /usr/sbin/btrfs receive -vv -p
/var/media/backups/venus/home/home-2014-09-26
/var/media/backups/venus/home/

Any idea what's happening? I can't find a single example online of
sending a delta over ssh.

Thanks,
Justin

On Thu, Oct 2, 2014 at 1:53 AM, Hugo Mills h...@carfax.org.uk wrote:
 On Thu, Oct 02, 2014 at 12:05:39AM -0500, Justin Brown wrote:
 I'm experimenting with btrfs-send. Previously (2014-09-26), I did my
 first btrfs-send on a subvol, and that worked fine. Today, I tried to
 send a new snapshot. Unfortunately, I realized part way through that I
 forgot to specify the parent to only send a delta, and killed the send
 with ^C.

 On the destination, I'm left with:

 ~$ sudo btrfs subvol list /var/media/backups/venus/home/
 ID 2820 gen 57717 top level 5 path media
 ID 2821 gen 57402 top level 5 path ovirt
 ID 4169 gen 57703 top level 2820 path media/backups/venus/home
 ID 4170 gen 57575 top level 4169 path home-2014-09-26
 ID 4243 gen 57707 top level 4169 path home-2014-10-01

 Home-2014-10-01 was the partial send that was cancelled. I figured
 that I could delete this partial subvol and try again.

 ~$ sudo btrfs subvol del home-2014-10-01
 Transaction commit: none (default)
 ERROR: error accessing 'home-2014-10-01'

If you're not doing this from /var/media/backups/venus/home/ it
 won't succeed. You need to specify (either via a relative path or an
 absolute one) where the subvol is, not just what its name is.

(Consider what happens if you have two filesystems, each with a
 home-2014-09-26 subvol.)

Hugo.

 Obviously, trying to delete the subvol directory fails too:

 ~$ sudo rm -rf /var/media/backups/venus/home/home-2014-10-01/
 rm: cannot remove ‘/var/media/backups/venus/home/home-2014-10-01/’:
 Operation not permitted

 Is there anyway to delete this partial subvol?

 Thanks,
 Justin

 --
 === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
   PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
--- All hope abandon,  Ye who press Enter here. ---
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Fwd: Deleting a Subvol from a Cancelled Btrfs-Send

[Justin Brown]

I'm experimenting with btrfs-send. Previously (2014-09-26), I did my
first btrfs-send on a subvol, and that worked fine. Today, I tried to
send a new snapshot. Unfortunately, I realized part way through that I
forgot to specify the parent to only send a delta, and killed the send
with ^C.

On the destination, I'm left with:

~$ sudo btrfs subvol list /var/media/backups/venus/home/
ID 2820 gen 57717 top level 5 path media
ID 2821 gen 57402 top level 5 path ovirt
ID 4169 gen 57703 top level 2820 path media/backups/venus/home
ID 4170 gen 57575 top level 4169 path home-2014-09-26
ID 4243 gen 57707 top level 4169 path home-2014-10-01

Home-2014-10-01 was the partial send that was cancelled. I figured
that I could delete this partial subvol and try again.

~$ sudo btrfs subvol del home-2014-10-01
Transaction commit: none (default)
ERROR: error accessing 'home-2014-10-01'

Obviously, trying to delete the subvol directory fails too:

~$ sudo rm -rf /var/media/backups/venus/home/home-2014-10-01/
rm: cannot remove ‘/var/media/backups/venus/home/home-2014-10-01/’:
Operation not permitted

Is there anyway to delete this partial subvol?

Thanks,
Justin
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Failed Disk RAID10 Problems

[Justin Brown]

Chris,

Thanks for the continued help. I had to put the recovery on hiatus
while I waited for new hard drives to be delivered. I never was able
to figure out how to replace the failed drive, but I did learn a lot
about how Btrfs works. The approach to doing practically all
operations with file system mounted specially was quite a surprise.

In the end, I created a Btrfs RAID5 file system with the newly
delivered drives on another system and used rsync to copy from the
degraded array. There was a little file system damage that showed up
as csum failed errors in the logs from the IO that was in progress
when the original failure occurred. Fortunately, it was all data that
could be recovered from other systems, and there wasn't any need to
troubleshoot the errors.

Thanks,
Justin


On Wed, May 28, 2014 at 3:40 PM, Chris Murphy li...@colorremedies.com wrote:

 On May 28, 2014, at 12:39 PM, Justin Brown justin.br...@fandingo.org wrote:

 Chris,

 Thanks for the tip. I was able to mount the drive as degraded and
 recovery. Then, I deleted the faulty drive, leaving me with the
 following array:


 Label: media  uuid: 7b7afc82-f77c-44c0-b315-669ebd82f0c5

 Total devices 6 FS bytes used 2.40TiB

 devid1 size 931.51GiB used 919.88GiB path
 /dev/mapper/SAMSUNG_HD103SI_499431FS734755p1

 devid2 size 931.51GiB used 919.38GiB path /dev/dm-8

 devid3 size 1.82TiB used 1.19TiB path /dev/dm-6

 devid4 size 931.51GiB used 919.88GiB path /dev/dm-5

 devid5 size 0.00 used 918.38GiB path /dev/dm-11

 devid6 size 1.82TiB used 3.88GiB path /dev/dm-9


 /dev/dm-11 is the failed drive. I take it that size 0 is a good sign.
 I'm not really sure where to go from here. I tried rebooting the
 system with the failed drive attached, and Btrfs re-adds it to the
 array. Should I physically remove the drive now? Is a balance
 recommended?

 I'm going to guess at what I think has happened. You had a 5 device raid10. 
 devid 5 is the failed device, but at the time you added new device devid 6, 
 it was not considered failed by btrfs. Your first btrfs fi show does not show 
 size 0 for devid 5. So I think btrfs made you a 6 device raid10 volume.

 But now devid 5 has failed, shows up as size 0. The reason you have to mount 
 degraded still is because you have a 6 device raid10 now, and 1 device has 
 failed. And you can't remove the failed device because you've mounted 
 degraded. So actually it was a mistake to add a new device first, but it's an 
 easy mistake to make because right now btrfs really tolerates a lot of error 
 conditions that it probably should give up on and outright fail the device.

 So I think you might have to get a 7th device to fix this with btrfs replace 
 start. You can later delete devices once you're not mounted degraded. Or you 
 can just do a backup now while you can mount degraded, and then blow away the 
 btrfs volume and start over.

 If you have a current backups and are willing to lose data on this volume, 
 you can try the following

 1. Poweroff, remove the failed drive, boot, and do a normal mount. That 
 probably won't work but it's worth a shot. If it doesn't work try mount -o 
 degraded. [That might not work either, in which case stop here, I think 
 you'll need to go with a 7th device and use 'btrfs replace start 5 
 /dev/newdevice7 /mp' That will explicitly replace failed device 5 with new 
 device.]

 2. Assuming mount -o degraded works, take a btrfs fi show. There should be a 
 missing device listed. Now try btrfs device delete missing /mp and see what 
 happens. If it at least doesn't complain, it means it's working and might 
 take hours to replicate data that was on the missing device onto the new one. 
 So I'd leave it alone until iotop or something like that tells you it's not 
 busy anymore.

 3. Unmount the file system. Try to mount normally (not degraded).



 Chris Murphy
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Failed Disk RAID10 Problems

[Justin Brown]

Hi,

I have a Btrfs RAID 10 (data and metadata) file system that I believe
suffered a disk failure. In my attempt to replace the disk, I think
that I've made the problem worse and need some help recovering it.

I happened to notice a lot of errors in the journal:

end_request: I/O error, dev dm-11, sector 1549378344
BTRFS: bdev /dev/mapper/Hitachi_HDS721010KLA330_GTA040PBG71HXF1 errs:
wr 759675, rd 539730, flush 23, corrupt 0, gen 0

The file system continued to work for some time, but eventually a NFS
client encountered IO errors. I figured that device was failing (It
was very old.). I attached a new drive to the hot-swappable SATA slot
on my computer, partitioned it with GPT, and ran partprobe to detect
it. Next I attempted to add a new device, which was successful.
However, something peculiar happened:

~: btrfs fi df /var/media/
Data, RAID10: total=2.33TiB, used=2.33TiB
Data, RAID6: total=72.00GiB, used=71.96GiB
System, RAID10: total=96.00MiB, used=272.00KiB
Metadata, RAID10: total=4.12GiB, used=2.60GiB

I don't know where that RAID6 file system came from, but it did not
exist over the weekend when I last checked. I attempted to run a
balance operation, but this is when the IO errors became severe, and I
cancelled it. Next, I tried to remove the failed device, thinking that
Btrfs could rebalance after that. Removing the failed device failed:

~: btrfs device delete /dev/dm-11 /var/media
ERROR: error removing the device '/dev/dm-11' - Device or resource busy

I shutdown the system and detached the failed disk. Upon reboot, I
cannot mount the filesystem:

~: mount /dev/mapper/SAMSUNG_HD103SI_499431FS734755p1 /var/media
mount: wrong fs type, bad option, bad superblock on
/dev/mapper/SAMSUNG_HD103SI_499431FS734755p1,
   missing codepage or helper program, or other error

   In some cases useful info is found in syslog - try
   dmesg | tail or so.

BTRFS: device label media devid 2 transid 44804
/dev/mapper/WDC_WD10EACS-00D6B0_WD-WCAU40229179p1
BTRFS info (device dm-10): disk space caching is enabled
BTRFS: failed to read the system array on dm-10
BTRFS: open_ctree failed

I reattached the failed disk, and I'm still getting the same mount
error as above.

Here's where the array currently stands:

Label: 'media'  uuid: 7b7afc82-f77c-44c0-b315-669ebd82f0c5
Total devices 5 FS bytes used 2.39TiB
devid1 size 931.51GiB used 919.41GiB path
/dev/mapper/SAMSUNG_HD103SI_499431FS734755p1
devid2 size 931.51GiB used 919.41GiB path
/dev/mapper/WDC_WD10EACS-00D6B0_WD-WCAU40229179p1
devid3 size 1.82TiB used 1.19TiB path
/dev/mapper/WDC_WD20EFRX-68AX9N0_WD-WMC1T1268493p1
devid4 size 931.51GiB used 920.41GiB path
/dev/mapper/WDC_WD10EARS-00Y5B1_WD-WMAV50654875p1
devid5 size 931.51GiB used 918.50GiB path
/dev/mapper/Hitachi_HDS721010KLA330_GTA040PBG71HXF1
devid6 size 1.82TiB used 3.41GiB path
/dev/mapper/WDC_WD20EFRX-68AX9N0_WD-WMC300239240p1

Btrfs v3.12

Devid 6 is the drive that I added earlier.

What can I do to recover this file system? I have another spare drive
that I can use if it's any help.

Thanks,
Justin
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Fwd: Failed Disk RAID10 Problems

[Justin Brown]

Chris,

Thanks for the tip. I was able to mount the drive as degraded and
recovery. Then, I deleted the faulty drive, leaving me with the
following array:


Label: media  uuid: 7b7afc82-f77c-44c0-b315-669ebd82f0c5

Total devices 6 FS bytes used 2.40TiB

devid1 size 931.51GiB used 919.88GiB path
/dev/mapper/SAMSUNG_HD103SI_499431FS734755p1

devid2 size 931.51GiB used 919.38GiB path /dev/dm-8

devid3 size 1.82TiB used 1.19TiB path /dev/dm-6

devid4 size 931.51GiB used 919.88GiB path /dev/dm-5

devid5 size 0.00 used 918.38GiB path /dev/dm-11

devid6 size 1.82TiB used 3.88GiB path /dev/dm-9


/dev/dm-11 is the failed drive. I take it that size 0 is a good sign.
I'm not really sure where to go from here. I tried rebooting the
system with the failed drive attached, and Btrfs re-adds it to the
array. Should I physically remove the drive now? Is a balance
recommended?


Thanks,

Justin
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Help with space

[Justin Brown]

Apologies for the late reply, I'd assumed the issue was closed even
given the unusual behavior.  My mount options are:

/dev/sdb1 on /var/lib/nobody/fs/ubfterra type btrfs
(rw,noatime,nodatasum,nodatacow,noacl,space_cache,skip_balance)

I only recently added nodatacow and skip_balance in an attempt to
figure out where the missing space had gone.  I don't know what impact
it might have if any on things.  I've got a full balance running at
the moment which, after about a day or so, has managed to process 5%
of the chunks it's considering (988 out of about 18396 chunks balanced
(989 considered),  95% left).  The amount of free space has vacillated
slightly, growing by about a gig to shrink back.  As far as objects in
the file system missing, I've not seen any such.  I've a lot of files
of various data types, the majority is encoded japanese animation.
Since I actually play these files via samba from a htpc, particularly
the more recent additions, I'd hazard to guess that if something were
breaking I'd have tripped across it by now, the unusual used to free
space delta being the exception.  My brother also uses this raid for
data storage, he's something of a closet meteorologist and is
fascinated by tornadoes.  He hasn't noticed any unusual behavior
either.  I'm in the process of sourcing a 64 bit capable system in the
hopes that will resolve the issue.  Neither of us are currently
writing anything to the file system for fear of things breaking, but
both have been reading from it without issue other than the noticeable
impact in performance balance seems to be having.  Thanks for the
help.

-Justin


On Fri, Feb 28, 2014 at 12:26 AM, Chris Murphy li...@colorremedies.com wrote:

 On Feb 27, 2014, at 11:13 PM, Chris Murphy li...@colorremedies.com wrote:


 On Feb 27, 2014, at 11:19 AM, Justin Brown otakujunct...@gmail.com wrote:

 terra:/var/lib/nobody/fs/ubfterra # btrfs fi df .
 Data, single: total=17.58TiB, used=17.57TiB
 System, DUP: total=8.00MiB, used=1.93MiB
 System, single: total=4.00MiB, used=0.00
 Metadata, DUP: total=392.00GiB, used=33.50GiB
 Metadata, single: total=8.00MiB, used=0.00

 After glancing at this again, what I thought might be going on might not be 
 going on. The fact it has 17+TB already used, not merely allocated, doesn't 
 seem possible if there's a hard 16TB limit for Btrfs on 32-bit kernels.

 But then I don't know why du -h is reporting only 13T total used. And I'm 
 unconvinced this is a balance issue either. Is anything obviously missing 
 from the file system?

 What are your mount options? Maybe compression?

 Clearly du is calculating things differently. I'm getting:

 du -sch = 4.2G
 df -h= 5.4G
 btrfs df  = 4.7G data and 620MB metadata(total).

 I am using compress=lzo.

 Chris Murphy

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Help with space

[Justin Brown]

Absolutely.  I'd like to know the answer to this, as 13 tera will take
a considerable amount of time to back up anywhere, assuming I find a
place.  I'm considering rebuilding a smaller raid with newer drives
(it was originally built using 16 250 gig western digital drives, it's
about eleven years old now, having been in use the entire time without
failure, I'm considering replacing each 250 gig with a 3 tera
alternative).  Unfortunately, between upgrading the host and building
a new raid the expense isn't something I'm anticipating with
pleasure...

On Fri, Feb 28, 2014 at 1:27 AM, Duncan 1i5t5.dun...@cox.net wrote:
 Roman Mamedov posted on Fri, 28 Feb 2014 10:34:36 +0600 as excerpted:

 But then as others mentioned it may be risky to use this FS on 32-bit at
 all, so I'd suggest trying anything else only after you reboot into a
 64-bit kernel.

 Based on what I've read on-list, btrfs is not arch-agnostic, with certain
 on-disk sizes set to native kernel page size, etc, so a filesystem
 created on one arch may well not work on another.

 Question: Does this apply to x86/amd64?  Will a filesystem created/used
 on 32-bit x86 even mount/work on 64-bit amd64/x86_64, or does upgrading
 to 64-bit imply backing up (in this case) double-digit TiB of data to
 something other than btrfs and testing it, doing a mkfs on the original
 filesystem once in 64-bit mode, and restoring all that data from backup?

 If the existing 32-bit x86 btrfs can't be used on 64-bit amd64,
 transferring all that data (assuming there's something big enough
 available to transfer it to!) to backup and then restoring it is going to
 hurt!

 --
 Duncan - List replies preferred.   No HTML msgs.
 Every nonfree program has a lord, a master --
 and if you use the program, he is your master.  Richard Stallman

 --
 To unsubscribe from this list: send the line unsubscribe linux-btrfs in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Detailed RAID Status and Errors

[Justin Brown]

Hello,

I'm finishing up my data migration to Btrfs, and I've run into an
error that I'm trying to explore in more detail. I'm using Fedora 20
with Btrfs v0.20-rc1.

My array is a 5 disk (4x 1TB and 1x 2TB) RAID 6 (-d raid6 -m raid6). I
completed my rsync to this array, and I figured that it would be
prudent to run a scrub before I consider this array the canonical
version of my data. The scrub is still running, but I current have the
following status:

~$ btrfs scrub status t
scrub status for 7b7afc82-f77c-44c0-b315-669ebd82f0c5
scrub started at Mon Feb 24 20:10:54 2014, running for 86080 seconds
total bytes scrubbed: 2.71TiB with 1 errors
error details: read=1
corrected errors: 0, uncorrectable errors: 1, unverified errors: 0

It is accompied by the following messages in the journal:

Feb 25 15:16:24 localhost kernel: ata4.00: exception Emask 0x0 SAct
0x3f SErr 0x0 action 0x0
Feb 25 15:16:24 localhost kernel: ata4.00: irq_stat 0x4008
Feb 25 15:16:24 localhost kernel: ata4.00: failed command: READ FPDMA QUEUED
Feb 25 15:16:24 localhost kernel: ata4.00: cmd
60/08:08:b8:24:af/00:00:58:00:00/40 tag 1 ncq 4096 in
   res
41/40:00:be:24:af/00:00:58:00:00/40 Emask 0x409 (media error) F
Feb 25 15:16:24 localhost kernel: ata4.00: status: { DRDY ERR }
Feb 25 15:16:24 localhost kernel: ata4.00: error: { UNC }
Feb 25 15:16:24 localhost kernel: ata4.00: configured for UDMA/133
Feb 25 15:16:24 localhost kernel: sd 3:0:0:0: [sdd] Unhandled sense code
Feb 25 15:16:24 localhost kernel: sd 3:0:0:0: [sdd]
Feb 25 15:16:24 localhost kernel: Result: hostbyte=DID_OK
driverbyte=DRIVER_SENSE
Feb 25 15:16:24 localhost kernel: sd 3:0:0:0: [sdd]
Feb 25 15:16:24 localhost kernel: Sense Key : Medium Error [current]
[descriptor]
Feb 25 15:16:24 localhost kernel: Descriptor sense data with sense
descriptors (in hex):
Feb 25 15:16:24 localhost kernel: 72 03 11 04 00 00 00 0c 00
0a 80 00 00 00 00 00
Feb 25 15:16:24 localhost kernel: 58 af 24 be
Feb 25 15:16:24 localhost kernel: sd 3:0:0:0: [sdd]
Feb 25 15:16:24 localhost kernel: Add. Sense: Unrecovered read error -
auto reallocate failed
Feb 25 15:16:24 localhost kernel: sd 3:0:0:0: [sdd] CDB:
Feb 25 15:16:24 localhost kernel: Read(10): 28 00 58 af 24 b8 00 00 08 00
Feb 25 15:16:24 localhost kernel: end_request: I/O error, dev sdd,
sector 1487873214
Feb 25 15:16:24 localhost kernel: ata4: EH complete
Feb 25 15:16:24 localhost kernel: btrfs: i/o error at logical
2285387870208 on dev /dev/sdf1, sector 1488392888, root 5, inode
357715, offset 48787456, length 4096, links 1 (path:
PATH/TO/REDACTED_FILE)
Feb 25 15:16:24 localhost kernel: btrfs: bdev /dev/sdf1 errs: wr 0, rd
1, flush 0, corrupt 0, gen 0
Feb 25 15:16:24 localhost kernel: btrfs: unable to fixup (regular)
error at logical 2285387870208 on dev /dev/sdf1

I have a few questions:

* How is total bytes scrubbed determined? This array only has 2.2TB
of space used, so I'm confused about how many total bytes need to be
scrubbed before it is finished.

* What is the best way to recover from this error? If I delete
PATH/TO/REDACTED_FILE and recopy it, will everything be okay? (I found
a thread on the Arch Linux forums,
https://bbs.archlinux.org/viewtopic.php?id=170795, that mentions this
as a solution, but I can't tell if it's the proper method.

* Should I run another scrub? (I'd like to avoid another scrub if
possible because the scrub has been running for 24 hours already.)

* When a scrub is not running, is there any `btrfs` command that will
show me corrected and uncorrectable errors that occur during normal
operation? I guess something similar to `mdadm -D`.

* It seems like this type of error shouldn't happen on RAID6 as there
should be enough information to recover between the data, p parity,
and q parity. Is this just an implementation limitation of the current
RAID 5/6 code?

Thanks,
Justin
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Detailed RAID Status and Errors

[Justin Brown]

Chris,

Thanks for the reply.

 Total includes metadata.

It still doesn't seem to add up:

~$ btrfs fi df t
Data, single: total=8.00MiB, used=0.00
Data, RAID6: total=2.17TiB, used=2.17TiB
System, single: total=4.00MiB, used=0.00
System, RAID6: total=9.56MiB, used=192.00KiB
Metadata, single: total=8.00MiB, used=0.00
Metadata, RAID6: total=4.03GiB, used=3.07GiB

Nonetheless, the scrub finished shortly after I started typing this
response. Total was ~2.7TB if I remember correctly.

 All of this looks like a conventional bad sector read error. It's concerning 
 why there'd be a bad sector after having just written to it when putting all 
 your data on this volume. What do you get for:

 smartctl -x /dev/sdd

...
SATA Phy Event Counters (GP Log 0x11)
ID  Size Value  Description
0x0001  20  Command failed due to ICRC error
0x0002  20  R_ERR response for data FIS
0x0003  20  R_ERR response for device-to-host data FIS
0x0004  20  R_ERR response for host-to-device data FIS
0x0005  20  R_ERR response for non-data FIS
0x0006  20  R_ERR response for device-to-host non-data FIS
0x0007  20  R_ERR response for host-to-device non-data FIS
0x000a  21  Device-to-host register FISes sent due to a COMRESET
0x8000  4   185377  Vendor specific




 smartctl -l scterc /dev/sdd

SCT Error Recovery Control:
   Read: Disabled
  Write: Disabled

 btrfs device stats /dev/X

All drives except /dev/sdf1 have zeroes for all values. /dev/sdf1
reports that same read error from the logs:

[/dev/sdf1].write_io_errs   0
[/dev/sdf1].read_io_errs1
[/dev/sdf1].flush_io_errs   0
[/dev/sdf1].corruption_errs 0
[/dev/sdf1].generation_errs 0

 You should have btrfs-progs-3.12-1.fc20.x86_64, it's available since November.

I was performing this copy from live media and forgot to run updates
before starting. I've updated btrfs-progs to the newest version, which
matches what you provided. I'm running another scrub with the updated
btrfs-progs, and while it hasn't uncovered any errors yet, the scrub
won't be finished until tomorrow. (I know that won't do anything
besides indicate some kind of transient error or a problem with
0.20-rc1.) After that, I'll give balance a try to attempt to fix the
issue.

Thanks,
Justin
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html