Re: Unexpected raid1 behaviour

2017-12-19 Thread George Mitchell
On 12/19/2017 06:46 AM, Tomasz Pala wrote: On Tue, Dec 19, 2017 at 07:25:49 -0500, Austin S. Hemmelgarn wrote: Well, the RAID1+ is all about the failing hardware. About catastrophically failing hardware, not intermittent failure. It shouldn't matter - as long as disk failing once is kicked ou

btrfs mount fail after adding new drive to raid1 array

2019-04-15 Thread George Mitchell
After adding a new drive to a btrfs raid1 array I cannot remount it.  One thing that went wrong for me after having successfully done this many times before over the last five years is that in this case I failed to format the partition as btrfs and just left it unformatted assuming the btrfs co

Scrub CPU usage ...

2013-05-04 Thread George Mitchell
I just subscribed to this list so in case this subject has already been discussed at length, my apologies. I have been waiting for btrfs forever. I have been waiting for it to become reasonably stable. In the wake of escalating problems with my old hardware RAID setup, I decided now was the

Re: Scrub CPU usage ...

2013-05-04 Thread George Mitchell
On 05/04/2013 06:21 PM, Kai Krakow wrote: George Mitchell schrieb: 1) The system fails to boot intermittently due to dracut/initrd issues (btrfs: open_ctree failed). This is being worked on upstream and I am seeing a continual flow of patches addressing it, but so far no fix. This will take

Re: Scrub CPU usage ...

2013-05-05 Thread George Mitchell
On 05/05/2013 02:22 AM, Martin Steigerwald wrote: Hi George! Am Samstag, 4. Mai 2013, 11:39:59 schrieb George Mitchell: the next update. I am using btrfs raid 1 across five 500GB Seagate nearline drives and btrfs single on a Seagate 4TB backup drive. I am absolutely delighted with how this

Re: Best Practice - Partition, or not?

2013-05-08 Thread George Mitchell
On 05/08/2013 12:31 AM, Kai Krakow wrote: Helmut Hullen schrieb: If I want to manage a complete disk with btrfs, what's the "Best Practice"? Would it be best to create the btrfs filesystem on "/dev/sdb", or would it be better to create just one partition from start to end and then do "mkfs.btr

Virtual Device Support

2013-05-10 Thread George Mitchell
One the things that is frustrating me the most at this point from a user perspective regarding btrfs is the current lack of virtual devices to describe volumes and subvolumes. The current method of simply using a random member device or a LABEL or a UUID is just not working well for me. Havin

Re: Virtual Device Support

2013-05-19 Thread George Mitchell
just write it off. - George On 05/19/2013 04:04 AM, Martin wrote: On 10/05/13 15:03, George Mitchell wrote: One the things that is frustrating me the most at this point from a user perspective ... The current method of simply using a random member device or a LABEL or a UUID is just not

Re: Virtual Device Support

2013-05-20 Thread George Mitchell
Duncan, The problem affects btrfs volumes that span multiple drive. If you are using btrfs on a single drive that works just fine. But in a multidrive situation, sometimes it works (when umount guesses the right device name) and sometimes it fails (when umount guesses the wrong device name).

Re: Virtual Device Support

2013-05-20 Thread George Mitchell
On 05/20/2013 08:59 PM, Duncan wrote: Then I ran into hardware issues that turned out to be bad caps on my 8- year-old mobo (tho it was dual-socket first-gen opteron, which I had upgraded to top-of-its-line dual-core Opteron 290s, thus four cores @ 2.8 GHz, with 8 gigs RAM, so it wasn't as perf

Re: Virtual Device Support

2013-05-21 Thread George Mitchell
he way zfs works as this would allow some fun magic to happen outside of util-linux. In any case, I hope my thoughts are at least a little useful. Cheers! On Sun, May 19, 2013 at 7:49 AM, George Mitchell <mailto:geo...@chinilu.com>> wrote: In reply to both of these comments in

Re: Virtual Device Support ("N-way mirror code")

2013-05-23 Thread George Mitchell
On 05/23/2013 09:08 AM, Martin Steigerwald wrote: 3) As to my knowledge mount times of large partitions can be quite long with ReiserFS 3. That may well be, but I certainly wouldn't consider btrfs mount times "fast" in such cases. [root@localhost ghmitch]# time mount LABEL=BACKUP /backup re

Re: systemd "journalctl" is 1.89sec on ext4, 1.49 min on btrfs

2013-05-27 Thread George Mitchell
I have gotten what appear to be large increases in speed out of btrfs by defragmentation of meta data. The manual defragmentation process takes forever as you have to defragment incrementally directory by directory. I was at the point where KDE startup times were getting abysmal (along with jo

Is there a way to flag specific directories "nodatacow"?

2013-06-02 Thread George Mitchell
I am seeing massive journal corruptions that seem to be unique to btrfs and I am suspecting that cow might be causing them. My bandaid fix for this will be to mark the /var filesystem "nodatacow" at boot. But I am wondering if their is any way to flag a particular directory as "nodatacow" out

Possible solution to the "open_ctree" boot bug ...

2013-06-02 Thread George Mitchell
I am seeing a huge improvement in boot performance since doing a system wide file by file defragementation of metadata. In fact in the four sequential boots since completing this process, I have not seen one open_ctree failure so far. This leads me to suspect that the open_ctree boot failures

Re: Is there a way to flag specific directories "nodatacow"?

2013-06-02 Thread George Mitchell
On 06/02/2013 06:28 PM, Liu Bo wrote: On Sun, Jun 02, 2013 at 07:40:52AM -0700, George Mitchell wrote: I am seeing massive journal corruptions that seem to be unique to btrfs and I am suspecting that cow might be causing them. My bandaid fix for this will be to mark the /var filesystem

Re: Is there a way to flag specific directories "nodatacow"?

2013-06-02 Thread George Mitchell
On 06/02/2013 06:28 PM, Liu Bo wrote: On Sun, Jun 02, 2013 at 07:40:52AM -0700, George Mitchell wrote: I am seeing massive journal corruptions that seem to be unique to btrfs and I am suspecting that cow might be causing them. My bandaid fix for this will be to mark the /var filesystem

Re: Is there a way to flag specific directories "nodatacow"?

2013-06-03 Thread George Mitchell
On 06/02/2013 07:58 PM, Liu Bo wrote: On Sun, Jun 02, 2013 at 07:11:10PM -0700, George Mitchell wrote: On 06/02/2013 06:28 PM, Liu Bo wrote: On Sun, Jun 02, 2013 at 07:40:52AM -0700, George Mitchell wrote: I am seeing massive journal corruptions that seem to be unique to btrfs and I am

Re: Possible solution to the "open_ctree" boot bug ...

2013-06-06 Thread George Mitchell
On 06/06/2013 01:58 PM, Kai Krakow wrote: George Mitchell schrieb: I am seeing a huge improvement in boot performance since doing a system wide file by file defragementation of metadata. In fact in the four sequential boots since completing this process, I have not seen one open_ctree

How do I safely terminate COW on pre-existing files?

2013-06-06 Thread George Mitchell
I want to eliminate the COW feature on all of my OS files. It is a nice feature for user files, but I don't see a clear benefit for the actual OS files. And I suspect that COW induced fragmentation is causing or aggravating problems with my system including the boot open_ctree problem. I had

Re: How do I safely terminate COW on pre-existing files?

2013-06-07 Thread George Mitchell
pstream. Thanks so much for taking the time to respond to my question. It is MUCH appreciated. - George On 06/07/2013 04:52 AM, David Sterba wrote: On Thu, Jun 06, 2013 at 07:51:28PM -0700, George Mitchell wrote: I want to eliminate the COW feature on all of my OS files. It is a nice feat

Re: Possible solution to the "open_ctree" boot bug ...

2013-06-09 Thread George Mitchell
eb "George Mitchell" <mailto:geo...@chinilu.com>>: On 06/06/2013 01:58 PM, Kai Krakow wrote: George Mitchell mailto:geo...@chinilu.com>> schrieb: I am seeing a huge improvement in boot performance since doing a system wide file b

Re: Possible solution to the "open_ctree" boot bug ...

2013-06-10 Thread George Mitchell
akow wrote: Actually it should be called "rootdelay"... My fault... Am 07.06.2013 01:48 schrieb "George Mitchell" <mailto:geo...@chinilu.com>>: On 06/06/2013 01:58 PM, Kai Krakow wrote: George Mitchell mailto:geo...@chinilu.com>> schrieb:

Re: raid1 inefficient unbalanced filesystem reads

2013-06-28 Thread George Mitchell
On 06/28/2013 09:25 AM, Martin wrote: On 28/06/13 16:39, Hugo Mills wrote: On Fri, Jun 28, 2013 at 11:34:18AM -0400, Josef Bacik wrote: On Fri, Jun 28, 2013 at 02:59:45PM +0100, Martin wrote: On kernel 3.8.13: Using two equal performance SATAII HDDs, formatted for btrfs raid1 for both data an

Re: What is the current status of defragmentation?

2013-07-17 Thread George Mitchell
Hello Adam, I routinely defrag my filesystems and here is how I do it: find /home -type d -mtime -3 -o -type f -mtime -3 | egrep -v "Cache|cache" | while read file; do /usr/sbin/btrfs filesystem defrag -f -v "${file}"; done The above is what I use to defrag my data (non-OS) files. I use a w

Re: What is the current status of defragmentation?

2013-07-18 Thread George Mitchell
e. Thanks for pointing this out. On 07/18/2013 12:12 AM, Adam Ryczkowski wrote: On 07/18/2013 02:17 AM, George Mitchell wrote: find /home -type d -mtime -3 -o -type f -mtime -3 | egrep -v "Cache|cache" | while read file; do /usr/sbin/btrfs filesystem defrag -f -v "${file}"; d

Re: Lots of harddrive chatter on after booting with btrfs on root (slow boot)

2013-07-20 Thread George Mitchell
Sounds to me like a fragmentation issue. On 07/20/2013 08:15 AM, Jason Russell wrote: Hi, I've been using btrfs for my root partition for about a month on archlinux and recently Ive started using the i3 window manager and starting X manually and I now boot to run level 3 (multi-user.target for

Re: autodefrag by default, was: Lots of harddrive chatter

2013-07-21 Thread George Mitchell
On 07/21/2013 03:01 PM, Duncan wrote: Chris Murphy posted on Sun, 21 Jul 2013 10:20:48 -0600 as excerpted: On Jul 21, 2013, at 4:38 AM, Duncan <1i5t5.dun...@cox.net> wrote: What I'd suggest is to turn on the btrfs autodefrag mount option, and to do it *BEFORE* you start installing stuff on the

Re: autodefrag by default, was: Lots of harddrive chatter

2013-07-21 Thread George Mitchell
On 07/21/2013 08:37 PM, Shridhar Daithankar wrote: On Sunday, July 21, 2013 04:44:09 PM George Mitchell wrote: Unless auto-defrag can work around the in-use file issue, that could be a problem since some heavily used system files are open virtually all the time the system is up and running

You guys do an amazing job - I am blown away!

2015-08-17 Thread George Mitchell
Two years ago I installed btrfs across 8 hard drives on my desktop system with the entire system ending up on btrfs RAID 1. I did all of this with btrfs-progs-0.20. Since that time I have been dreading updating my system because of fear that the old btrfs volumes would become unstable in the

Re: uncorrectable errors after btrfs replace

2013-08-18 Thread George Mitchell
This is just a comment from someone following all of this from the sidelines. And that is that I see so much going on here with this procedure that is scares me. Once a single operation reaches a certain degree of complexity I get really scared because all it takes is a single misstep and my

Re: Does btrfs "raid1" actually provide any resilience?

2013-11-14 Thread George Mitchell
The read only mount issue is by design. It is intended to make sure you know exactly what is going on before you proceed. For example, a drive may actually be fine, but may have been caused by a cable failure. In that case you would want to fix the cable problem before you break the mirror b

Re: Does btrfs "raid1" actually provide any resilience?

2013-11-14 Thread George Mitchell
On 11/14/2013 09:35 AM, Lutz Vieweg wrote: On 11/14/2013 06:18 PM, George Mitchell wrote: The read only mount issue is by design. It is intended to make sure you know exactly what is going on before you proceed. Hmmm... but will a server be able to continue its operation (including writes

Re: Blocket for more than 120 seconds

2013-12-14 Thread George Mitchell
On 12/14/2013 04:28 PM, Hans-Kristian Bakke wrote: I would normally expect that there is no difference in 1TB free space on a FS that is 2TB in total, and 1TB free space on a filesystem that is 30TB in total, other than my sense of urge and that you would probably expect data growth to be more r

Re: btrfs on whole disk (no partitions)

2014-06-18 Thread George Mitchell
A lot of good comments on this topic already. I would just add that on large (TB) drives, not partitioning can result in some pretty slow mount and umount times (even applies to mounting subvolumes). That is one of the frustrating side effects I have noticed with a non-partitioned 4TB drive o

Re: open_ctree failed

2014-06-19 Thread George Mitchell
On 06/19/2014 02:50 PM, Tamas Papp wrote: On 06/16/2014 04:02 PM, Tamas Papp wrote: On 06/16/2014 03:26 PM, Tamas Papp wrote: hi All, There is a Dell XPS 13 laptop with and SSD. System: Ubuntu 14.04 amd64 Kernel is from the daily ppa, like 3.15rcX. Now, it's running live system: Linux ub

Re: btrfs on whole disk (no partitions)

2014-06-22 Thread George Mitchell
On 06/22/2014 12:49 AM, Imran Geriskovan wrote: The 64KB Btrfs bootloader pad is 8 sector aligned, so for 512e AF disks there's no problem formatting the whole drive. The alignment problem actually happens when partitioning it, using old partition tools that don't align on 8 sector boundaries. Th

Re: btrfs on whole disk (no partitions)

2014-06-22 Thread George Mitchell
On 06/22/2014 07:11 AM, Roman Mamedov wrote: On Sun, 22 Jun 2014 06:44:13 -0700 George Mitchell wrote: On 06/22/2014 12:49 AM, Imran Geriskovan wrote: The 64KB Btrfs bootloader pad is 8 sector aligned, so for 512e AF disks there's no problem formatting the whole drive. The alignment pr

Re: btrfs on whole disk (no partitions)

2014-06-22 Thread George Mitchell
On 06/22/2014 07:11 AM, Roman Mamedov wrote: On Sun, 22 Jun 2014 06:44:13 -0700 George Mitchell wrote: On 06/22/2014 12:49 AM, Imran Geriskovan wrote: The 64KB Btrfs bootloader pad is 8 sector aligned, so for 512e AF disks there's no problem formatting the whole drive. The alignment pr

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-20 Thread George Mitchell
On 07/20/2014 02:28 PM, Bob Marley wrote: On 20/07/2014 21:36, Roman Mamedov wrote: On Sun, 20 Jul 2014 21:15:31 +0200 Bob Marley wrote: Hi TM, are you doing other significant filesystem activity during this rebuild, especially random accesses? This can reduce performances a lot on HDDs. E.g.

Re: Scan not being performed properly on boot

2014-08-03 Thread George Mitchell
On 08/03/2014 08:31 PM, Russell Coker wrote: On Mon, 4 Aug 2014 04:02:53 Peter Roberts wrote: I've just recently started testing btrfs on my server but after just 24 hours problems have started. I get booted to a busybox prompt user ubuntu 14.04. I have a multi device FS setup and I can't say fo

Re: Scan not being performed properly on boot

2014-08-03 Thread George Mitchell
On 08/03/2014 09:14 PM, Russell Coker wrote: On Sun, 3 Aug 2014 21:00:19 George Mitchell wrote: But just changing your boot configuration to use /dev/sdx is probably the best option. Assuming you are booting with grub2, you will need to use /dev/sdx in the grub2 configuration file. This is

Re: Scan not being performed properly on boot

2014-08-03 Thread George Mitchell
On 08/03/2014 09:14 PM, Russell Coker wrote: On Sun, 3 Aug 2014 21:00:19 George Mitchell wrote: But just changing your boot configuration to use /dev/sdx is probably the best option. Assuming you are booting with grub2, you will need to use /dev/sdx in the grub2 configuration file. This is

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread George Mitchell
I really suspect a lot of bad block issues can be avoided by monitoring SMART data. SMART is working very well for me with btrfs formatted drives. SMART will detect when sectors silently fail and as those failures accumulate, SMART will warn in an obvious way that the drive in question is at

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread George Mitchell
Hello Clemens, On 01/09/2014 04:08 PM, Clemens Eisserer wrote: Hi George, I really suspect a lot of bad block issues can be avoided by monitoring SMART data. SMART is working very well for me with btrfs formatted drives. SMART will detect when sectors silently fail and as those failures accum

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread George Mitchell
On 01/09/2014 05:06 PM, Jim Salter wrote: On Jan 9, 2014 7:46 PM, George Mitchell wrote: I would prefer that the drive, even flash media type, would catch and resolve write failures. If it doesn't happen at the hardware layer, according to how I understand Hugo's answer, btrfs, at

Re: How does btrfs handle bad blocks in raid1?

2014-01-10 Thread George Mitchell
On 01/10/2014 07:27 AM, Duncan wrote: George Eleftheriou posted on Thu, 09 Jan 2014 17:49:48 +0100 as excerpted: I'm really looking forward to the day that typing: mkfs.btrfs -d raid10 -m raid10 /dev/sd[abcd] will do exactly what is expected to do. A true RAID10 resilient in 2 disks' failure

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread George Mitchell
On 01/14/2014 11:13 AM, Chris Murphy wrote: On Jan 9, 2014, at 6:31 PM, George Mitchell wrote: Jim, my point was that IF the drive does not successfully resolve the bad block issue and btrfs takes a write failure every time it attempts to overwrite the bad data, it is not going to remap that

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread George Mitchell
On 01/14/2014 01:00 PM, Roman Mamedov wrote: On Tue, 14 Jan 2014 12:29:28 -0800 George Mitchell wrote: what we are lacking at this point is a SMART capability to provide visual notifications to the user when any hard drive starts to seriously degrade or suddenly fails. You can configure

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread George Mitchell
On 01/14/2014 01:00 PM, Roman Mamedov wrote: On Tue, 14 Jan 2014 12:29:28 -0800 George Mitchell wrote: what we are lacking at this point is a SMART capability to provide visual notifications to the user when any hard drive starts to seriously degrade or suddenly fails. You can configure

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread George Mitchell
On 01/14/2014 01:14 PM, Chris Murphy wrote: On Jan 14, 2014, at 1:29 PM, George Mitchell wrote: And the key to monitoring hard drive health, in my opinion, is SMART and what we are lacking at this point is a SMART capability to provide visual notifications to the user when any hard drive

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread George Mitchell
On 01/14/2014 01:14 PM, Chris Murphy wrote: On Jan 14, 2014, at 1:29 PM, George Mitchell wrote: And the key to monitoring hard drive health, in my opinion, is SMART and what we are lacking at this point is a SMART capability to provide visual notifications to the user when any hard drive

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread George Mitchell
On 01/14/2014 01:14 PM, Chris Murphy wrote: And the key to monitoring hard drive health, in my opinion, is SMART and what we are lacking at this point is a SMART capability to provide visual notifications to the user when any hard drive starts to seriously degrade or suddenly fails. Gnome d

Re: btrfs and ECC RAM

2014-01-17 Thread George Mitchell
On 01/17/2014 04:23 PM, Ian Hinder wrote: Hi, I have been reading a lot of articles online about the dangers of using ZFS with non-ECC RAM. Specifically, the fact that when good data is read from disk and compared with its checksum, a RAM error can cause the read data to be incorrect, causing

Re: btrfs and ECC RAM

2014-01-19 Thread George Mitchell
Just my opinion, of course, but I simply cannot imagine how "an incorrect checksum could appear correct due to a memory error". Sorry, but I just cannot get my brain around that one. The odds against it happening would be beyond comprehension. I can easily imagine btrfs taking a system down d

Re: btrfs and ECC RAM

2014-01-20 Thread George Mitchell
After reading the recent posts on this topic I am beginning to think there is some real confusion between "check sums" and "parity". These are two different things which serve two different purposes. In each case, bad RAM would have different repercussions. But I still fail to see how, in the

Re: Incremental backup for a raid1

2014-03-13 Thread George Mitchell
On 03/13/2014 04:03 PM, Michael Schuerig wrote: On Thursday 13 March 2014 16:04:33 Chris Murphy wrote: On Mar 13, 2014, at 3:14 PM, Michael Schuerig wrote: On Thursday 13 March 2014 14:48:55 Andrew Skretvedt wrote: On 2014-Mar-13 14:28, Hugo Mills wrote: On Thu, Mar 13, 2014 at 08:12:44PM +

Re: Incremental backup for a raid1

2014-03-14 Thread George Mitchell
Actually, an interesting concept would be to have the initial two drive RAID 1 mirrored by 2 additional drives in 4-way configuration on a second machine at a remote location on a private high speed network with both machines up 24/7. In that case, if such a configuration would work, either ma

Re: Incremental backup for a raid1

2014-03-15 Thread George Mitchell
is n-way mirroring to get me out of the simplex trap completely. At that point, I can have more confidence in btrfs snapshop capability. On 03/15/2014 04:35 AM, Michael Schuerig wrote: On Thursday 13 March 2014 17:29:11 George Mitchell wrote: I currently use rsync to a separate drive to mai

Possible application issue ...

2014-04-06 Thread George Mitchell
I am seeming to have an issue with a specific application. I just installed "Recoll", a really nice desktop search tool. And the following day whenever my backup program would attempt to run, my computer simply stopped dead in its tracks and I was forced to do a hard reboot to get it back. S

Re: Possible application issue ...

2014-04-07 Thread George Mitchell
On 04/07/2014 05:42 AM, Duncan wrote: George Mitchell posted on Sun, 06 Apr 2014 22:25:03 -0700 as excerpted: I am seeming to have an issue with a specific application. I just installed "Recoll", a really nice desktop search tool. And the following day whenever my backup pro

Re: How to make BTRFS crawl

2014-04-11 Thread George Mitchell
Well, Akonadi brought my system to its knees long before I converted to btrfs, so somehow I am not surprised. I have kept akonadi disabled ever since.for everything except a portion of Thunderbird and that ONLY with sql-lite. Mysql will kill it in no time. So I am not sure that btrfs is the r