Re: [CentOS] ZFS on Linux in production?

2013-11-04 Thread Markus Falb
On 24.Okt.2013, at 22:59, John R Pierce wrote: On 10/24/2013 1:41 PM, Lists wrote: Was wondering if anybody here could weigh in with real-life experience? Performance/scalability? I've only used ZFS on Solaris and FreeBSD.some general observations... ... 3) NEVER let a zpool fill up

Re: [CentOS] ZFS on Linux in production?

2013-11-04 Thread Les Mikesell
On Mon, Nov 4, 2013 at 12:15 PM, Markus Falb wne...@gmail.com wrote: 3) NEVER let a zpool fill up above about 70% full, or the performance really goes downhill. Why is it? It sounds cost intensive, if not ridiculous. Disk space not to used, forbidden land... Is the remaining 30% used by

Re: [CentOS] ZFS on Linux in production?

2013-11-04 Thread John R Pierce
On 11/4/2013 10:43 AM, Les Mikesell wrote: On Mon, Nov 4, 2013 at 12:15 PM, Markus Falb wne...@gmail.com wrote: 3) NEVER let a zpool fill up above about 70% full, or the performance really goes downhill. Why is it? It sounds cost intensive, if not ridiculous. Disk space not to used,

Re: [CentOS] ZFS on Linux in production?

2013-11-04 Thread Nicolas Thierry-Mieg
On 11/04/2013 08:01 PM, John R Pierce wrote: On 11/4/2013 10:43 AM, Les Mikesell wrote: On Mon, Nov 4, 2013 at 12:15 PM, Markus Falb wne...@gmail.com wrote: 3) NEVER let a zpool fill up above about 70% full, or the performance really goes downhill. Why is it? It sounds cost intensive, if

Re: [CentOS] ZFS on Linux in production?

2013-11-04 Thread John R Pierce
On 11/4/2013 3:21 PM, Nicolas Thierry-Mieg wrote: but why would this be much worse with ZFS than eg ext4? because ZFS works considerably differently than extfs... its a copy-on-write system to start with. -- john r pierce 37N 122W somewhere on the middle

Re: [CentOS] ZFS on Linux in production?

2013-11-03 Thread Rajagopal Swaminathan
Greetings, On Fri, Oct 25, 2013 at 3:57 AM, Keith Keller kkel...@wombat.san-francisco.ca.us wrote: I don't have my own, but I have heard of other shops which have had lots of success with ZFS on OpenSolaris and their variants. And I know of a shop which could not recover a huge ZFS on freebsd

Re: [CentOS] ZFS on Linux in production?

2013-10-30 Thread Lists
On 10/25/2013 11:14 AM, Chuck Munro wrote: To keep the two servers in sync I use 'lsyncd' which is essentially a front-end for rsync that cuts down thrashing and overhead dramatically by excluding the full filesystem scan and using inotify to figure out what to sync. This allows

Re: [CentOS] ZFS on Linux in production?

2013-10-30 Thread Mailing List
To be honest is not easier to install on server FreeBSD or Solaris where ZFS is natively supported? I moved my own server to FreeBSD and I didn't noticed huge difference between Linux distros and freebsd, I have no idea what about Solaris but it might be still similar environment. Sent from

Re: [CentOS] ZFS on Linux in production?

2013-10-26 Thread Ray Van Dolson
On Thu, Oct 24, 2013 at 01:41:17PM -0700, Lists wrote: We are a CentOS shop, and have the lucky, fortunate problem of having ever-increasing amounts of data to manage. EXT3/4 becomes tough to manage when you start climbing, especially when you have to upgrade, so we're contemplating

Re: [CentOS] ZFS on Linux in production?

2013-10-26 Thread Ray Van Dolson
On Thu, Oct 24, 2013 at 01:59:15PM -0700, John R Pierce wrote: On 10/24/2013 1:41 PM, Lists wrote: Was wondering if anybody here could weigh in with real-life experience? Performance/scalability? I've only used ZFS on Solaris and FreeBSD.some general observations... 1) you need a

Re: [CentOS] ZFS on Linux in production?

2013-10-26 Thread George Kontostanos
On Sat, Oct 26, 2013 at 4:36 PM, Ray Van Dolson ra...@bludgeon.org wrote: On Thu, Oct 24, 2013 at 01:59:15PM -0700, John R Pierce wrote: On 10/24/2013 1:41 PM, Lists wrote: Was wondering if anybody here could weigh in with real-life experience? Performance/scalability? I've only used

Re: [CentOS] ZFS on Linux in production?

2013-10-25 Thread Warren Young
On Oct 24, 2013, at 8:01 PM, Lists li...@benjamindsmith.com wrote: Not sure enough of the vernacular Yes, ZFS is complicated enough to have a specialized vocabulary. I used two of these terms in my previous post: - vdev, which is a virtual device, something like a software RAID. It is one

Re: [CentOS] ZFS on Linux in production?

2013-10-25 Thread John R Pierce
On 10/24/2013 11:18 PM, Warren Young wrote: All of the ZFSes out there are crippled relative to what's shipping in Solaris now, because Oracle has stopped releasing code. There are nontrivial features in zpool v29+, which simply aren't in the free forks of older versions o the Sun code.

Re: [CentOS] ZFS on Linux in production?

2013-10-25 Thread Lists
On 10/24/2013 11:18 PM, Warren Young wrote: - vdev, which is a virtual device, something like a software RAID. It is one or more disks, configured together, typically with some form of redundancy. - pool, which is one or more vdevs, which has a capacity equal to all of its vdevs added

Re: [CentOS] ZFS on Linux in production?

2013-10-25 Thread John R Pierce
On 10/25/2013 10:33 AM, Lists wrote: LVM2 complicates administration terribly. huh? it hugely simplifies it for me, when I have lots of drives. I just wish mdraid and lvm were better integrated. to see how it should have been done, see IBM AIX's version of lvm.you grow a jfs file system,

Re: [CentOS] ZFS on Linux in production?

2013-10-25 Thread Chuck Munro
On 10/25/2013, 05:00 , centos-requ...@centos.org wrote: We are a CentOS shop, and have the lucky, fortunate problem of having ever-increasing amounts of data to manage. EXT3/4 becomes tough to manage when you start climbing, especially when you have to upgrade, so we're contemplating

Re: [CentOS] ZFS on Linux in production?

2013-10-25 Thread Mauricio Tavares
On Fri, Oct 25, 2013 at 1:40 PM, John R Pierce pie...@hogranch.com wrote: On 10/25/2013 10:33 AM, Lists wrote: LVM2 complicates administration terribly. huh? it hugely simplifies it for me, when I have lots of drives. I just wish mdraid and lvm were better integrated. to see how it should

Re: [CentOS] ZFS on Linux in production?

2013-10-25 Thread Warren Young
On 10/25/2013 00:44, John R Pierce wrote: current version of OpenZFS no longer relies on 'version numbers', instead it has 'feature flags' for all post v28 features. This must be the zpool v5000 thing I saw while researching my previous post. Apparently ZFSonLinux is doing the same thing, or

Re: [CentOS] ZFS on Linux in production?

2013-10-25 Thread Warren Young
On 10/25/2013 11:33, Lists wrote: I'm just trying to find the best tool for the job. Try everything. Seriously. You won't know what you like, and what works *for you* until you have some experience. Buy a Drobo for the home, replace one of your old file servers with a FreeBSD ZFS box,

Re: [CentOS] ZFS on Linux in production?

2013-10-25 Thread John R Pierce
On 10/25/2013 1:26 PM, Warren Young wrote: On 10/25/2013 00:44, John R Pierce wrote: current version of OpenZFS no longer relies on 'version numbers', instead it has 'feature flags' for all post v28 features. This must be the zpool v5000 thing I saw while researching my previous post.

Re: [CentOS] ZFS on Linux in production?

2013-10-25 Thread Warren Young
On re-reading, I realized I didn't complete some of my thoughts: On 10/25/2013 00:18, Warren Young wrote: ZFS is nicer in this regard, in that it lets you schedule the scrub operation. You can obviously schedule one for btrfs, ...with cron... but that doesn't take into account scrub time.

Re: [CentOS] ZFS on Linux in production?

2013-10-25 Thread Peter
On 10/26/2013 06:40 AM, John R Pierce wrote: to see how it should have been done, see IBM AIX's version of lvm.you grow a jfs file system, it automatically grows the underlying LV (logical volume), online, live. lvm can do this with the --resizefs flag for lvextend, one command to

[CentOS] ZFS on Linux in production?

2013-10-24 Thread Lists
We are a CentOS shop, and have the lucky, fortunate problem of having ever-increasing amounts of data to manage. EXT3/4 becomes tough to manage when you start climbing, especially when you have to upgrade, so we're contemplating switching to ZFS. As of last spring, it appears that ZFS On Linux

Re: [CentOS] ZFS on Linux in production?

2013-10-24 Thread John R Pierce
On 10/24/2013 1:41 PM, Lists wrote: Was wondering if anybody here could weigh in with real-life experience? Performance/scalability? I've only used ZFS on Solaris and FreeBSD.some general observations... 1) you need a LOT of ram for decent performance on large zpools. 1GB ram above your

Re: [CentOS] ZFS on Linux in production?

2013-10-24 Thread SilverTip257
On Thu, Oct 24, 2013 at 4:41 PM, Lists li...@benjamindsmith.com wrote: We are a CentOS shop, and have the lucky, fortunate problem of having ever-increasing amounts of data to manage. EXT3/4 becomes tough to manage when you start climbing, especially when you have to upgrade, so we're

Re: [CentOS] ZFS on Linux in production?

2013-10-24 Thread Lists
On 10/24/2013 01:59 PM, John R Pierce wrote: 1) you need a LOT of ram for decent performance on large zpools. 1GB ram above your basic system/application requirements per terabyte of zpool is not unreasonable. That seems quite reasonable to me. Our existing equipment has far more than

Re: [CentOS] ZFS on Linux in production?

2013-10-24 Thread Keith Keller
On 2013-10-24, SilverTip257 silvertip...@gmail.com wrote: On Thu, Oct 24, 2013 at 4:41 PM, Lists li...@benjamindsmith.com wrote: We are a CentOS shop, and have the lucky, fortunate problem of having ever-increasing amounts of data to manage. EXT3/4 becomes tough to manage when you start

Re: [CentOS] ZFS on Linux in production?

2013-10-24 Thread John R Pierce
On 10/24/2013 2:59 PM, Lists wrote: (*) ran into a guy who had 100s of zfs 'file systems' (mount points), per user home directories, and was doing nightly snapshots going back several years, and his zfs commands were taking a long long time to do anything, and he couldn't figure out why. I

Re: [CentOS] ZFS on Linux in production?

2013-10-24 Thread Lists
On 10/24/2013 02:47 PM, SilverTip257 wrote: You didn't mention XFS. Just curious if you considered it or not. Most definitely. There are a few features that I'm looking for: 1) MOST IMPORTANT: STABLE! 2) The ability to make the partition bigger by adding drives with very minimal/no

Re: [CentOS] ZFS on Linux in production?

2013-10-24 Thread Rainer Duffner
Am 25.10.2013 um 00:47 schrieb John R Pierce pie...@hogranch.com: On 10/24/2013 2:59 PM, Lists wrote: (*) ran into a guy who had 100s of zfs 'file systems' (mount points), per user home directories, and was doing nightly snapshots going back several years, and his zfs commands were taking a

Re: [CentOS] ZFS on Linux in production?

2013-10-24 Thread George Kontostanos
We tested ZFS on CentOS 6.4 a few months ago using a descend Supermicro server with 16GB RAM and 11 drives on RaidZ3. Same specs as a middle range storage server that we build mainly using FreeBSD. Performance was not bad but eventually we run into a situation were we could not import a pool

Re: [CentOS] ZFS on Linux in production?

2013-10-24 Thread John R Pierce
On 10/24/2013 4:12 PM, Lists wrote: On 10/24/2013 02:47 PM, SilverTip257 wrote: You didn't mention XFS. Just curious if you considered it or not. Most definitely. There are a few features that I'm looking for: 1) MOST IMPORTANT: STABLE! XFS is quite stable in CentOS 6.4 64bit. there was a

Re: [CentOS] ZFS on Linux in production?

2013-10-24 Thread Warren Young
On 10/24/2013 17:12, Lists wrote: 2) The ability to make the partition bigger by adding drives with very minimal/no downtime. Be careful: you may have been reading some ZFS hype that turns out not as rosy in reality. Ideally, ZFS would work like a Drobo with an infinite number of drive

Re: [CentOS] ZFS on Linux in production?

2013-10-24 Thread Warren Young
On 10/24/2013 14:59, John R Pierce wrote: On 10/24/2013 1:41 PM, Lists wrote: 1) you need a LOT of ram for decent performance on large zpools. 1GB ram above your basic system/application requirements per terabyte of zpool is not unreasonable. To be fair, you want to treat XFS the same way.

Re: [CentOS] ZFS on Linux in production?

2013-10-24 Thread John R Pierce
On 10/24/2013 5:31 PM, Warren Young wrote: To be fair, you want to treat XFS the same way. And it, too is unstable on 32-bit systems with anything but smallish filesystems, due to lack of RAM. I thought it had stack requirements that 32 bit couldn't meet, and it would simply crash, so it is

Re: [CentOS] ZFS on Linux in production?

2013-10-24 Thread John R Pierce
On 10/24/2013 5:29 PM, Warren Young wrote: The least complicated*safe* way to add 1 TB to a pool is add*two* 1 TB disks to the system, create a ZFS mirror out of them, and add*that* vdev to the pool. That gets you 1 TB of redundant space, which is what you actually wanted. Just realize,

Re: [CentOS] ZFS on Linux in production?

2013-10-24 Thread Lists
On 10/24/2013 05:29 PM, Warren Young wrote: On 10/24/2013 17:12, Lists wrote: 2) The ability to make the partition bigger by adding drives with very minimal/no downtime. Be careful: you may have been reading some ZFS hype that turns out not as rosy in realiIdeally, ZFS would work like a