btrfs_rename2 and hangs with 5.2.1

2019-09-09 Thread Russell Coker
The file log.txt has one of many kernel message logs (logged via syslog) about btrfs_rename2 that happened while the laptop in question was apparently working OK. The file dm.txt has part of the output of dmesg after the system got into a state where all disk access was going to D state, that w

Re: BTRFS losing SE Linux labels on power failure or "reboot -nffd"

2018-06-06 Thread Russell Coker
time that my message got to the SE Linux list. The kernel from Debian/Stable still has the issue. So using a testing kernel might be a good option to deal with this problem at the moment. On Monday, 4 June 2018 11:14:52 PM AEST Russell Coker wrote: > The command "reboot -nffd" (

BTRFS losing SE Linux labels on power failure or "reboot -nffd"

2018-06-04 Thread Russell Coker
The command "reboot -nffd" (kernel reboot without flushing kernel buffers or writing status) when run on a BTRFS system with SE Linux will often result in /var/log/audit/audit.log being unlabeled. It also results in some systemd- journald files like /var/log/journal/c195779d29154ed8bcb4e8444c4a1

Re: speed up big btrfs volumes with ssds

2017-09-04 Thread Russell Coker
On Monday, 4 September 2017 2:57:18 PM AEST Stefan Priebe - Profihost AG wrote: > > Then roughly make sure the complete set of metadata blocks fits in the > > cache. For an fs of this size let's say/estimate 150G. Then maybe same > > of double for data, so an SSD of 500G would be a first try. > >

read-only for no good reason on 4.9.30

2017-09-03 Thread Russell Coker
I have a system with less than 50% disk space used. It just started rejecting writes due to lack of disk space. I ran "btrfs balance" and then it started working correctly again. It seems that a btrfs filesystem if left alone will eventually get fragmented enough that it rejects writes (I've

Re: Strange behavior after "rm -rf //"

2016-08-11 Thread Russell Coker
http://selinux.coker.com.au/play.html There are a variety of ways of giving the same result that rm doesn't reject. "/*" Wasn't caught last time I checked. See the above URL if you want to test out various rm operations as root. ;) On 10 August 2016 9:24:23 AM AEST, Christian Kujau wrote: >On

BTRFS admin access control

2016-08-03 Thread Russell Coker
I've just written a script for "mon" to monitor BTRFS filesystems. I had to use sudo because "btrfs device stats" needs to be run as root. Would it be possible to do some of these things as non-root? I think it would be ideal if there was a "btrfs tunefs" operation somewhat comparable to "tun

Re: dear developers, can we have notdatacow + checksumming, plz?

2015-12-13 Thread Russell Coker
On Mon, 14 Dec 2015 03:59:18 PM Christoph Anton Mitterer wrote: > I've had some discussions on the list these days about not having > checksumming with nodatacow (mostly with Hugo and Duncan). > > They both basically told me it wouldn't be straight possible with CoW, > and Duncan thinks it may not

Re: 3.16.0 Debian kernel hang

2015-12-04 Thread Russell Coker
On Sat, 5 Dec 2015 12:53:07 AM Austin S Hemmelgarn wrote: > > The only reason I'm not running Unstable kernels on my Debian systems is > > because I run some Xen servers and upgrading Xen is problemmatic. Linode > > is moving from Xen to KVM so I guess I should consider doing the > > same. If I m

Re: 3.16.0 Debian kernel hang

2015-12-04 Thread Russell Coker
On Sat, 5 Dec 2015 12:08:58 AM Austin S Hemmelgarn wrote: > > I know that there are no plans to backport things to 3.16 and I don't > > think the Debian people are going to be very interested in this. So > > this message is a FYI for users, maybe consider not using the > > Debian/Jessie kernel for

3.16.0 Debian kernel hang

2015-12-04 Thread Russell Coker
One of my test laptops started hanging on mounting the root filesystem. I think that it had experience an unexpected power outage prior to that which may have caused corruption. When I tried to mount the root filesystem the mount process would stick in D state, there would be no disk IO, and t

Re: LWN mention

2015-12-01 Thread Russell Coker
On Mon, 9 Nov 2015 08:10:13 AM Duncan wrote: > Russell Coker posted on Sun, 08 Nov 2015 17:38:32 +1100 as excerpted: > > https://lwn.net/Articles/663474/ > > http://thread.gmane.org/gmane.comp.file-systems.btrfs/49500 > > > > Above is a BTRFS issue that is mentioned

Re: Bug/regression: Read-only mount not read-only

2015-12-01 Thread Russell Coker
On Wed, 2 Dec 2015 06:05:09 AM Eric Sandeen wrote: > yes, xfs does; we have "-o norecovery" if you don't want that, or need > to mount a filesystem with a dirty log on a readonly device. That option also works with Ext3/4 so it seems to be a standard way of dealing with this. I think that BTRFS

Re: Using Btrfs on single drives

2015-11-24 Thread Russell Coker
On Sun, 15 Nov 2015 03:01:57 PM Duncan wrote: > That looks to me like native drive limitations. > > Due to the fact that a modern hard drive spins at the same speed no > matter where the read/write head is located, when it's reading/writing to > the first part of the drive -- the outside -- much

LWN mention

2015-11-07 Thread Russell Coker
https://lwn.net/Articles/663474/ http://thread.gmane.org/gmane.comp.file-systems.btrfs/49500 Above is a BTRFS issue that is mentioned in a LWN comment. Has this one been fixed yet? -- My Main Blog http://etbe.coker.com.au/ My Documents Bloghttp://doc.coker.com.au/ -- To unsubscribe

Re: How to replicate a Xen VM using BTRFS as the root filesystem.

2015-10-28 Thread Russell Coker
On Wed, 28 Oct 2015 11:07:20 PM Austin S Hemmelgarn wrote: > Using this methodology, I can have a new Gentoo PV domain running in > about half an hour, whereas it takes me at least two and a half hours > (and often much longer than that) when using the regular install process > for Gentoo. On m

Re: Expected behavior of bad sectors on one drive in a RAID1

2015-10-20 Thread Russell Coker
On Wed, 21 Oct 2015 12:00:59 AM Austin S Hemmelgarn wrote: > > https://www.gnu.org/software/ddrescue/ > > > > At this stage I would use ddrescue or something similar to copy data from > > the failing disk to a fresh disk, then do a BTRFS scrub to regenerate > > the missing data. > > > > I wouldn'

Re: Expected behavior of bad sectors on one drive in a RAID1

2015-10-19 Thread Russell Coker
On Tue, 20 Oct 2015 03:16:15 PM james harvey wrote: > sda appears to be going bad, with my low threshold of "going bad", and > will be replaced ASAP. It just developed 16 reallocated sectors, and > has 40 current pending sectors. > > I'm currently running a "btrfs scrub start -B -d -r /terra", wh

Re: BTRFS as image store for KVM?

2015-10-03 Thread Russell Coker
On Fri, 2 Oct 2015 10:07:24 PM Austin S Hemmelgarn wrote: > > ARC presumably worked better than the other Solaris caching options. It > > was ported to Linux with zfsonlinux because that was the easy way of > > doing it. > > Actually, I think part of that was also the fact that ZFS is a COW > fi

qgroup problem

2015-10-01 Thread Russell Coker
(sysadm_t:SystemLow-SystemHigh)root@unstable:~/pol# btrfs qgroup show -r -e /tmp qgroupid rfer excl max_rfer max_excl 0/5 1647689728 1647689728 0 0 0/25816384 16384 524288000 0 0/2591638

Re: BTRFS as image store for KVM?

2015-10-01 Thread Russell Coker
On Sat, 26 Sep 2015 12:20:41 AM Austin S Hemmelgarn wrote: > > FYI: > > Linux pagecache use LRU cache algo, and in general case it's working good > > enough > > I'd argue that 'general usage' should be better defined in this > statement. Obviously, ZFS's ARC implementation provides better > per

Re: strange i/o errors with btrfs on raid/lvm

2015-10-01 Thread Russell Coker
On Sat, 26 Sep 2015 06:47:26 AM Chris Murphy wrote: > And then > > Aug 28 17:06:49 host mdadm[2751]: RebuildFinished event detected on md > device /dev/md/0, component device mismatches found: 2048 (on raid > level 10) > Aug 28 17:06:49 host mdadm[2751]: SpareActive event detected on md > device

Re: BTRFS as image store for KVM?

2015-09-23 Thread Russell Coker
On Fri, 18 Sep 2015 12:00:15 PM Duncan wrote: > The caveat here is that if the VM/DB is active during the backups (btrfs > send/receive or other), it'll still COW1 any writes during the existence > of the btrfs snapshot. If the backup can be scheduled during VM/DB > downtime or at least when ac

Re: BTRFS as image store for KVM?

2015-09-23 Thread Russell Coker
On Sat, 19 Sep 2015 12:13:29 AM Austin S Hemmelgarn wrote: > The other option (which for some reason I almost never see anyone > suggest), is to expose 2 disks to the guest (ideally stored on different > filesystems), and do BTRFS raid1 on top of that. In general, this is > what I do (except I use

Re: Understanding BTRFS storage

2015-09-01 Thread Russell Coker
On Fri, 28 Aug 2015 07:35:02 PM Hugo Mills wrote: > On Fri, Aug 28, 2015 at 10:50:12AM +0200, George Duffield wrote: > > Running a traditional raid5 array of that size is statistically > > guaranteed to fail in the event of a rebuild. > >Except that if it were, you wouldn't see anyone runnin

Re: Questions on use of NOCOW impact to subvolumes and snapshots

2015-08-20 Thread Russell Coker
On Thu, 20 Aug 2015 10:09:26 PM Austin S Hemmelgarn wrote: > > 2: Out of curiosity, why is data checksumming tied to COW? > > There's no safe way to sanely handle checksumming without COW, because > there is no way (at least on current hardware) to ensure that the data > block and the checksums b

Re: Questions on use of NOCOW impact to subvolumes and snapshots

2015-08-19 Thread Russell Coker
On Thu, 20 Aug 2015 11:55:43 AM Chris Murphy wrote: > > Question 1: If I apply the NOCOW attribute to a file or directory, how > > does that affect my ability to run btrfs scrub? > > nodatacow includes nodatasum and no compression. So it means these > files are presently immune from scrub check a

lockup

2015-08-14 Thread Russell Coker
I have a Xen server with 14 DomUs that are being used for BTRFS and ZFS training. About 5 people are corrupting virtual disks and scrubbing them, lots of IO. All the virtual machine disk images are snapshots of a master image with copy on write. I just had the following error which ended with

can we make balance delete missing devices?

2015-08-14 Thread Russell Coker
[ 2918.502237] BTRFS info (device loop1): disk space caching is enabled [ 2918.503213] BTRFS: failed to read chunk tree on loop1 [ 2918.540082] BTRFS: open_ctree failed I just had a test RAID-1 filesystem with a missing device. I mounted it with the degraded option and added a new device. I bal

lack of scrub error data

2015-08-13 Thread Russell Coker
Below is the result of testing a corrupted filesystem. What's going on here? The kernel message log and the btrfs output don't tell me how many errors there were. Also the data is RAID-0 (the default for a filesystem created with 2 devices) so if this was in a data area it should have lost da

Re: Damaged filesystem, can read, can't repair, error says to contact devs

2015-08-11 Thread Russell Coker
On Wed, 12 Aug 2015 07:24:00 AM Chris Murphy wrote: > > There is still data redundancy. Will a scrub at least notice that the > > copies differ? > > No, that's what I mean by "nodatasum means no raid1 self-healing is > possible". You have data redundancy, but without checksums btrfs has > no way

Re: btrfs raid1 metadata, single data

2015-08-07 Thread Russell Coker
On Fri, 7 Aug 2015 06:49:58 PM Robert Krig wrote: > What exactly is contained in btrfs metadata? Much the same as in metadata for every other filesystem. > I've read about some users setting up their btrfs volumes as > data=single, but metadata=raid1 > > Is there any actual benefit to that? I me

Re: mount btrfs takes 30 minutes, btrfs check runs out of memory

2015-08-01 Thread Russell Coker
On Sat, 1 Aug 2015 02:35:39 PM John Ettedgui wrote: > >> It seems that you're using Chromium while doing the dump. :) > >> If no CD drive, I'll recommend to use Archlinux installation iso to make > >> a bootable USB stick and do the dump. > >> (just download and dd would do the trick) > >> As its k

Re: INFO: task btrfs-transacti:204 blocked for more than 120 seconds. (more like 8+min)

2015-07-30 Thread Russell Coker
On Fri, 24 Jul 2015 11:11:22 AM Duncan wrote: > The option is mem=nn[KMG]. You may also need memmap=, presumably > memmap=nn[KMG]$ss[KMG], to reserve the unused memory area, preventing its > use for PCI address space, since that would collide with the physical > memory that's there but unused due

Re: INFO: task btrfs-transacti:204 blocked for more than 120 seconds. (more like 8+min)

2015-07-30 Thread Russell Coker
On Fri, 24 Jul 2015 05:12:38 AM james harvey wrote: > I started trying to run with a "-s 4G" option, to use 4GB files for > performance measuring. It refused to run, and said "file size should > be double RAM for good results". I sighed, removed the option, and > let it run, defaulting to **64GB

Re: RAID1: system stability

2015-07-22 Thread Russell Coker
On Tue, 23 Jun 2015 02:52:43 AM Chris Murphy wrote: > OK I actually don't know what the intended block layer behavior is > when unplugging a device, if it is supposed to vanish, or change state > somehow so that thing that depend on it can know it's "missing" or > what. So the question here is, is

why does df spin up disks?

2015-06-28 Thread Russell Coker
When I have a mounted filesystem why doesn't the kernel store the amount of free space? Why does it need to spin up a disk that had been spun down? -- My Main Blog http://etbe.coker.com.au/ My Documents Bloghttp://doc.coker.com.au/ -- To unsubscribe from this list: send the line "un

Re: BTRFS RAID5 filesystem corruption during balance

2015-06-19 Thread Russell Coker
On Sun, 24 May 2015 01:02:21 AM Jan Voet wrote: > Doing a 'btrfs balance cancel' immediately after the array was mounted > seems to have done the trick. A subsequent 'btrfs check' didn't show any > errors at all and all the data seems to be there. :-) I add "rootflags=skip_balance" to the kernel

Re: Carefully crafted BTRFS-image causes kernel to crash

2015-04-21 Thread Russell Coker
On Tue, 21 Apr 2015, Qu Wenruo wrote: > Although we may add extra check for such problem to improve robustness, > but IMHO it's not a real world problem. Some of the ReiserFS developers gave a similar reaction to some of my bug reports. ReiserFS wasn't the most robust filesystem. I think that

Re: The FAQ on fsync/O_SYNC

2015-04-19 Thread Russell Coker
On Mon, 20 Apr 2015, Craig Ringer wrote: > PostgreSQL is its self copy-on-write (because of multi-version > concurrency control), so it doesn't make much sense to have the FS > doing another layer of COW. That's a matter of opinion. I think it's great if PostgreSQL can do internal checkums and e

Re: how to clone a btrfs filesystem

2015-04-18 Thread Russell Coker
On Sat, 18 Apr 2015, Christoph Anton Mitterer wrote: > On Sat, 2015-04-18 at 04:24 +0000, Russell Coker wrote: > > dd works. ;) > > > > There are patches to rsync that make it work on block devices. Of course > > that will copy space occupied by deleted files too

Re: how to clone a btrfs filesystem

2015-04-17 Thread Russell Coker
On Fri, 17 Apr 2015 11:08:44 PM Christoph Anton Mitterer wrote: > How can I best copy one btrfs filesystem (with snapshots and subvolumes) > into another, especially with keeping the CoW/reflink status of all > files? dd works. ;) > And ideally incrementally upgrade it later (again with all > sn

directory defrag

2015-04-14 Thread Russell Coker
The current defragmentation options seem to only support defragmenting named files/directories or a recursive defragmentation of files and directories. I'd like to recursively defragment directories. One of my systems has a large number of large files, the files are write-once and read performa

Re: du accuracy

2015-04-12 Thread Russell Coker
On Fri, 10 Apr 2015, Liu Bo wrote: > > Above are some consecutive du runs. Why does the space used go from 1.2G > > to 1.1G before going up again? The file was created by "cat /dev/sde > > > 2gsd" so it definitely wasn't getting smaller. > > > > > > > > What's going on here? > > What's your m

snapshot space use

2015-04-08 Thread Russell Coker
# zfs list -t snapshot NAMEUSED AVAIL REFER MOUNTPOINT hetz0/be0-mail@2015-03-10 2.88G - 387G - hetz0/be0-mail@2015-03-11 1.12G - 388G - hetz0/be0-mail@2015-03-12 1.11G - 388G - hetz0/be0-mail@2015-03-13 1.19G - 388G - hetz0/be0-mail@20

Re: btrfs subvolume & default subvolume

2015-04-07 Thread Russell Coker
On Tue, 7 Apr 2015 02:03:04 PM arnaud gaboury wrote: > Would you mind give the return of # btrfs subvolume list and $ cat > /etc/fstab ? It would help me. TY ID 262 gen 1579103 top level 5 path mysql

Re: btrfs subvolume & default subvolume

2015-04-07 Thread Russell Coker
On Tue, 7 Apr 2015 10:58:28 AM arnaud gaboury wrote: > After more reading, it seems to me creating a top root subvolume is > the right thing to do: > # btrfs subvolume create root > # btrfs subvolume create root/var > # btrfs subvolume create root/home > > Am I right? A filesystem is designed to

Re: Btrfs hangs 3.19-10

2015-04-06 Thread Russell Coker
On Mon, 6 Apr 2015 03:21:18 AM Duncan wrote: > So... for 3.2 compatibility, extref must not be enabled (tho it's now the > default and AFAIK there's no way to actually disable it, only enable, so > an old btrfs-tools would have to be used that doesn't enable it by > default), AND the nodesize mu

Re: Btrfs hangs 3.19-10

2015-04-06 Thread Russell Coker
On Mon, 6 Apr 2015 07:40:03 AM Pavel Volkov wrote: > On Sunday, April 5, 2015 1:04:17 PM MSK, Hugo Mills wrote: > >That's these, I think: > > #define BTRFS_FEATURE_INCOMPAT_BIG_METADATA (1ULL << 5) > > #define BTRFS_FEATURE_INCOMPAT_EXTENDED_IREF(1ULL << 6) > > > > so it's definitely -

Re: Btrfs hangs 3.19-10

2015-04-04 Thread Russell Coker
On Sun, 5 Apr 2015 03:16:21 AM Duncan wrote: > Hugo Mills posted on Sat, 04 Apr 2015 13:00:47 + as excerpted: > > On Sat, Apr 04, 2015 at 12:55:08PM +, Russell Coker wrote: > >> As an aside are there options to mkfs.btrfs that would make a > >> filesystem mountab

Re: Btrfs hangs 3.19-10

2015-04-04 Thread Russell Coker
On Fri, 3 Apr 2015 05:14:12 AM Duncan wrote: > Well, btrfs itself isn't really stable yet... Stable series should be > stable at least to the extent that whatever you're using in them is, but > with btrfs itself not yet entirely stable... Also for stable operation you want both forward and bac

Re: btrfs fstrim status (RAID5)

2015-03-24 Thread Russell Coker
Debian/Wheezy userspace can't be expected to work as well as desired with a 3.19 kernel. Wheezy with BTRFS single or RAID1-1 works reasonably well as long as you have lots of free space, balance it regularly, and configure it not to resume a balance on reboot. Debian/Jessie works well with BTR

Re: btrfs balance fails with no space errors (despite having plenty)

2015-03-24 Thread Russell Coker
I've been having ongoing issues with balance failing with no space errors in spite of having plenty. Strangely it seems to most often happen from cron jobs, when a cron job fails I can count on a manual balance succeeding. I'm running the latest Debian/Jessie kernel. -- Sent from my Samsung Gal

Re: syntax for deleting subvolumes?

2015-03-20 Thread Russell Coker
On Thu, 19 Mar 2015 07:18:29 AM Erkki Seppala wrote: > > But as a user level facility, I want to be able to snapshot before > > making a change to a tree full of source code and (re)building it all > > over again. I may want to keep my new build, but I may want to flush > > it and return to known

Re: syntax for deleting subvolumes?

2015-03-20 Thread Russell Coker
On Fri, 20 Mar 2015 04:18:38 AM Duncan wrote: > If cp --reflink=auto was the default, it'd "just work", making a reflink > where possible, falling back to a normal copy where not possible to > reflink. > > However, I'd be wary of such a change, because admins are used to cp > creating a separat

Re: Is it safe or useful to use NOCOW flag and autodefrag mount option at same time?

2015-03-16 Thread Russell Coker
On Sun, 15 Mar 2015, peer@gmx.net wrote: > Following common recommendations [1], I use these mount options on my > main developing machine: noatime,autodefrag. This is desktop machine and > it works well so far. Now, I'm also going to install several KVM virtual > machines on this system. I wan

single GlobalReserve

2015-02-03 Thread Russell Coker
# btrfs fi df /big Data, RAID1: total=2.56TiB, used=2.56TiB System, RAID1: total=32.00MiB, used=388.00KiB Metadata, RAID1: total=19.25GiB, used=14.06GiB GlobalReserve, single: total=512.00MiB, used=0.00B Why is GlobalReserve single? That filesystem has been RAID-1 for ages (since long before the

Re: ignoring bad blocks

2015-01-11 Thread Russell Coker
On Sun, 4 Jan 2015 13:46:30 Chris Murphy wrote: > So do use dd you need to also use bs= setting it to a multiple of > 4096. Of course most people using dd for zeroing set bs= to a decently > high value because it makes the process go much faster than the > default block size of 512 bytes. You coul

Debian/Jessie 3.16.7-ckt2-1 kernel error

2014-12-23 Thread Russell Coker
I've attached the kernel message log that I get after booting kernel 3.16.7 from Debian/Unstable. This is the kernel branch that will go into Debian/Jessie so it's important to get it fixed. Below has the start of the errors, the attached file has everything from boot. I've got similar issues

Re: Balance & scrub & defrag

2014-12-11 Thread Russell Coker
On Wed, 10 Dec 2014 17:17:28 Robert White wrote: > A _monthly_ scrub is maybe worth scheduling if you have a lot of churn > in your disk contents. I do weekly scrubs. I recently had 2 disks in a RAID-1 array develop read errors within a month of each other. The first scrub after replacing sdb

Re: PROBLEM: #89121 BTRFS mixes up mounted devices with their snapshots

2014-12-02 Thread Russell Coker
Maybe we should have a warning in mkfs.btrfs for the problematic RAID layouts. On December 3, 2014 6:19:22 AM GMT+11:00, Phillip Susi wrote: >-BEGIN PGP SIGNED MESSAGE- >Hash: SHA1 > >On 12/1/2014 4:45 PM, Konstantin wrote: >> The bug appears also when using mdadm RAID1 - when one of the

Re: pro/cons of raid1 with mdadm/lvm2

2014-12-01 Thread Russell Coker
On Mon, 1 Dec 2014, Chris Murphy wrote: > On Sun, Nov 30, 2014 at 3:06 PM, Russell Coker wrote: > > When the 2 disks have different data mdadm has no way of knowing which > > one is correct and has a 50% chance of overwriting good data. But BTRFS > > does checksums on all

Re: pro/cons of raid1 with mdadm/lvm2

2014-11-30 Thread Russell Coker
When the 2 disks have different data mdadm has no way of knowing which one is correct and has a 50% chance of overwriting good data. But BTRFS does checksums on all reads and solves the problem of corrupt data - as long as you don't have 2 corrupt sectors in matching blocks. -- Sent from my Sam

Re: Balance and RAID-1

2014-11-27 Thread Russell Coker
On Fri, 28 Nov 2014, Zygo Blaxell wrote: > On Fri, Nov 28, 2014 at 01:37:50AM +1100, Russell Coker wrote: > > I had a RAID-1 filesystem with 2*3TB disks and 330G of disk space free > > according to df -h. I replaced a 3TB disk with a 4TB disk and df > > reported no change

Balance and RAID-1

2014-11-27 Thread Russell Coker
I had a RAID-1 filesystem with 2*3TB disks and 330G of disk space free according to df -h. I replaced a 3TB disk with a 4TB disk and df reported no change in the free space (as expected). I added a 1TB disk to the filesystem and there was still no change! I expected that adding a 1TB disk wou

resetting device stats

2014-11-26 Thread Russell Coker
When running Debian kernel version 3.16.0-4-amd64 and btrfs-tools version 3.17-1.1 I ran a btrfs replace operation to replace a 3TB disk that was giving read errors with a new 4TB disk. After the replace the "btrfs device stats" command reported that the 4TB disk had 16 read errors. It appears

strange device stats message

2014-11-25 Thread Russell Coker
I am in the middle of replacing /dev/sdb (which is 3TB SATA disk that gives a few read errors on every scrub) with /dev/sdc2 (a partition on a new 4TB SATA disk). I am running btrfs-tools version 3.17-1.1 from Debian/Unstable and Debian kernel 3.16.0-4-amd64. I get the following, the last sect

Re: open_ctree problem

2014-11-22 Thread Russell Coker
Strangely I repeated the same process on the same system (btrfs-zero-log and mount read-only) and it worked. While it's a concern that repeating the same process gives different results it's nice that I'm getting all my data back. On Sun, 23 Nov 2014, Russell Coker wr

open_ctree problem

2014-11-22 Thread Russell Coker
I have a workstation running Linux 3.14.something on a 120G SSD. It recently had a problem and now the root filesystem can't be mounted, here is the message I get when trying to mount it read-only on Debian kernel 3.16.2-3: [4703937.784447] BTRFS info (device loop0): disk space caching is enabl

Re: device balance times

2014-10-23 Thread Russell Coker
Also a device replace operation requires that the replacement be the same size (or maybe larger). While a remove and replace allows the replacement to be merely large enough to contain all the data. Given the size variation in what might be called the same size disk by manufcturers this isn't un

Re: NOCOW and Swap Files?

2014-10-23 Thread Russell Coker
Also it would be nice to have checksums on the swap data. It's a bit of a waste to pay for ECC RAM and then lose the ECC benefits as soon as data is paged out. -- Sent from my Samsung Galaxy Note 3 with K-9 Mail. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the bod

Re: strange 3.16.3 problem

2014-10-21 Thread Russell Coker
I've just upgraded the Dom0 (NFS server) from 3.16.3 to 3.16.5 and it all works. Prior to upgrading the Dom0 I had the same problem occur with different file names. All the names in question were truncated names of files that exist. It seems that 3.16.3 has a bug with NFS serving files with l

Re: strange 3.16.3 problem

2014-10-21 Thread Russell Coker
On Tue, 21 Oct 2014, Zygo Blaxell wrote: > On Mon, Oct 20, 2014 at 04:38:28AM +, Duncan wrote: > > Russell Coker posted on Sat, 18 Oct 2014 14:54:19 +1100 as excerpted: > > > # find . -name "*546" > > > ./1412233213.M638209P10546 # ls -l ./1412233213.

Re: strange 3.16.3 problem

2014-10-18 Thread Russell Coker
On Sun, 19 Oct 2014, Robert White wrote: > On 10/17/2014 08:54 PM, Russell Coker wrote: > > # find . -name "*546" > > ./1412233213.M638209P10546 > > # ls -l ./1412233213.M638209P10546 > > ls: cannot access ./1412233213.M638209P10546: No such file or director

Re: strange 3.16.3 problem

2014-10-18 Thread Russell Coker
On Sat, 18 Oct 2014, "Michael Johnson - MJ" wrote: > The NFS client is part of the kernel iirc, so it should be 64 bit. This > would allow the creation of files larger than 4gb and create possible > issues with a 32 bit user space utility. A correctly written 32bit application will handle files

strange 3.16.3 problem

2014-10-17 Thread Russell Coker
I have a system running the Debian 3.16.3-2 AMD64 kernel for the Xen Dom0 and the DomUs. The Dom0 has a pair of 500G SATA disks in a BTRFS RAID-1 array. The RAID-1 array has some subvols exported by NFS as well as a subvol for the disk images for the DomUs - I am not using NoCOW as performance

Re: deleting a dead device

2014-09-24 Thread Russell Coker
On Sun, 21 Sep 2014 11:05:46 Chris Murphy wrote: > On Sep 20, 2014, at 7:39 PM, Russell Coker wrote: > > Anyway the new drive turned out to have some errors, writes failed and > > I've > > got a heap of errors such as the above. > > I'm curious if smartc

Re: deleting a dead device

2014-09-21 Thread Russell Coker
On Sun, 21 Sep 2014, Duncan <1i5t5.dun...@cox.net> wrote: > Russell Coker posted on Sun, 21 Sep 2014 11:39:17 +1000 as excerpted: > > On a system running the Debian 3.14.15-2 kernel I added a new drive to a > > RAID-1 array. My aim was to add a device and remove one o

device delete progress

2014-09-20 Thread Russell Coker
We need to have a way to determine the progress of a device delete operation. Also for a balance of a RAID-1 that has more than 2 devices it would be good to know how much space is used on each device. Could btrfs fi df be extended to show information separately for each device? -- My Main Bl

deleting a dead device

2014-09-20 Thread Russell Coker
On a system running the Debian 3.14.15-2 kernel I added a new drive to a RAID-1 array. My aim was to add a device and remove one of the old devices. Sep 21 11:26:51 server kernel: [2070145.375221] BTRFS: lost page write due to I/O error on /dev/sdc3 Sep 21 11:26:51 server kernel: [2070145.37522

Re: how long should "btrfs device delete missing ..." take?

2014-09-11 Thread Russell Coker
It would be nice if a file system mounted ro counted as ro snapshots for btrfs send. When a file system is so messed up it can't be mounted rw it should be regarded as ro for all operations. -- Sent from my Samsung Galaxy Note 2 with K-9 Mail. -- To unsubscribe from this list: send the line "un

Re: No space on empty, degraded raid10

2014-09-10 Thread Russell Coker
On Mon, 8 Sep 2014, Austin S Hemmelgarn wrote: > Also, I've found out the hard way that system chunks really should be > RAID1, NOT RAID10, otherwise it's very likely that the filesystem > won't mount at all if you lose 2 disks. Why would that be different? In a RAID-1 you expect system problems

Debian 3.14.13-2 lockup

2014-08-18 Thread Russell Coker
I've attached the dmesg output from a system running Debian kernel 3.14.13 which locked up. Everything which needed to write to disk was blocked. The dmesg output didn't catch the first messages which had scrolled out of the buffer. As the disk wasn't writable there was nothing useful in /va

Re: Putting very big and small files in one subvolume?

2014-08-17 Thread Russell Coker
On Sun, 17 Aug 2014 12:31:42 Duncan wrote: > OTOH, I tend to be rather more of an independent partition booster than > many. The biggest reason for that is the too many eggs in one basket > problem. Fully separate filesystems on separate partitions separate > those data "eggs" into separate ba

Re: 40TB volume taking over 16 hours to mount, any ideas?

2014-08-08 Thread Russell Coker
On Fri, 8 Aug 2014 16:35:29 Jose Ildefonso Camargo Tolosa wrote: > uname -a > Linux server1 3.15.8-031508-generic #201407311933 SMP Thu Jul 31 > 23:34:33 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux > > The complete story: > > The filesystem was created on Ubuntu 12.04, running kernel 3.11. > mount op

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Russell Coker
On Tue, 5 Aug 2014 08:06:12 Duncan wrote: > Which is why I'm not particularly happy with seeing all the "btrfs is > still not stable, use at your own risk" warnings disappearing. With them > there, people who chose to run btrfs /could/ be expected to have done > their research and have btrfs sp

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Russell Coker
On Tue, 5 Aug 2014 16:51:44 Qu Wenruo wrote: > In fact such "defeat"(or whatever) is not really btrfs only problem. > In ext*, there is still similiar behavior: ext* has a up limit on the > number of inode after mkfs. > (When you mkfs.ext*, you are prompt the up limit of inodes) > However other me

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Russell Coker
On Mon, 4 Aug 2014 14:17:02 Peter Waller wrote: > For anyone else having this problem, this article is fairly useful for > understanding disk full problems and rebalance: > > http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-> > Full-Problems.html > > It actually covers

Re: Scan not being performed properly on boot

2014-08-03 Thread Russell Coker
On Sun, 3 Aug 2014 21:34:29 George Mitchell wrote: > I see what you are saying. Its a hack. But I suspect that most of the > distros are not yet accommodating btrfs with their standard mkinitrd > process. At this point modifying grub2 config does solve the problem. > If you know a reasonably

Re: Scan not being performed properly on boot

2014-08-03 Thread Russell Coker
On Sun, 3 Aug 2014 21:00:19 George Mitchell wrote: > > But just changing your boot configuration to use /dev/sdx is probably the > > best option. > > Assuming you are booting with grub2, you will need to use /dev/sdx in > the grub2 configuration file. This is known issue with grub2. Example > f

Re: Threads being NUMA aware

2014-08-03 Thread Russell Coker
On Sun, 3 Aug 2014 22:44:26 Nick Krause wrote: > On Sun, Aug 3, 2014 at 7:48 PM, Russell Coker wrote: > > Please get yourself a NUMA system and test this out. > > Unfortunately I don't have money for an extra machine as of now as I > am a student If you can't g

Re: Scan not being performed properly on boot

2014-08-03 Thread Russell Coker
On Mon, 4 Aug 2014 04:02:53 Peter Roberts wrote: > I've just recently started testing btrfs on my server but after just 24 > hours problems have started. I get booted to a busybox prompt user > ubuntu 14.04. I have a multi device FS setup and I can't say for sure if > it managed to boot initiall

Re: Threads being NUMA aware

2014-08-03 Thread Russell Coker
Please get yourself a NUMA system and test this out. -- Sent from my Samsung Galaxy Note 2 with K-9 Mail. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.ht

Re: ENOSPC with mkdir and rename

2014-08-02 Thread Russell Coker
On Sun, 3 Aug 2014 00:35:28 Peter Waller wrote: > I'm running Ubuntu 14.04. I wonder if this problem is related to the > thread titled "Machine lockup due to btrfs-transaction on AWS EC2 > > Ubuntu 14.04" which I started on the 29th of July: > > http://thread.gmane.org/gmane.comp.file-systems.btrf

Re: Questions on incremental backups

2014-07-18 Thread Russell Coker
On Fri, 18 Jul 2014 13:56:58 Sam Bull wrote: > On ven, 2014-07-18 at 14:35 +1000, Russell Coker wrote: > > Ignoring directories in send/recv is done by subvol. Even if you use > > rsync it's a good idea to have different subvols for directory trees > > with different ba

Re: Questions on incremental backups

2014-07-17 Thread Russell Coker
Daily snapshots work welk with kernel 3.14 and above (I had problems with 3.13 and previous). I have snapshots every 15 mins on some subvols. Very large numbers of snapshots can cause performance problems. I suggest keeping below 1000 snapshots at this time. You can use send/recv functionality

Re: Btrfs and LBA errors

2014-07-15 Thread Russell Coker
On Tue, 15 Jul 2014 11:42:05 constantine wrote: > Thank you very much for your advice. It worked! Great! > I verified that the superblocks 1 and 2 had similar information with > btrfs-show-super -i 1 /dev/sdc1 (and -i 2) and then with crossed > fingers: > btrfs-select-super -s 2 /dev/sdc1 > whic

Re: Btrfs and LBA errors

2014-07-11 Thread Russell Coker
On Fri, 11 Jul 2014 21:29:07 constantine wrote: > Thank you very much for your response: > > # file -s /dev/sda1 /dev/sdc1 > /dev/sda1: BTRFS Filesystem label "partition", sectorsize 4096, > nodesize 4096, leafsize 4096, > UUID=c1eb1aaf-665a-4337-9d04-3c3921aa67e0, 1683870334976/3010310701056 > by

Re: Btrfs and LBA errors

2014-07-11 Thread Russell Coker
On Fri, 11 Jul 2014 13:42:40 constantine wrote: > Btrfs filesystem could not be mounted because /dev/sdc1 had unreadable > sectors. It is/was a single filesystem (not raid1 or raid0) over /dev/sda1 > and /dev/sdc1. What does "file -s /dev/sda1 /dev/sdc1" report? -- My Main Blog http://e

Re: Btrfs transaction checksum corruption & losing root of the tree & bizarre UUID change.

2014-07-11 Thread Russell Coker
On Fri, 11 Jul 2014 10:38:22 Duncan wrote: > > I've moved all drives and move those to my main rig which got a nice > > 16GB of ecc ram, so errors of ram, cpu, controller should be kept > > theoretically eliminated. > > It's worth noting that ECC RAM doesn't necessarily help when it's an in- > tra

  1   2   3   >