Re: [OpenAFS] zfs File system

[Chas Williams (CONTRACTOR)]

In message <[EMAIL PROTECTED]>,Brian Sebby writes:
>sure to turn off atime on the ZFS pool as well as changing the block size

wouldnt turning off atime on namei filesystems be a good idea in general?
i dont think the fileserver uses this for anything.  i suppose this should
be in the adminguide somewhere.
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] zfs File system

[Brian Sebby]

This seems to have been discussed quite a bit already, but we also moved
our fileservers from inode to namei on ZFS in Solaris.  Fortunately, this
was part of a hardware upgrade, so all I had to do was set up the new
file servers using ZFS, and use vos move to move over the volumes.  Once
it was done, I just shut down the inode file servers.  As others have
mentioned, running on inode these days is asking for trouble - Sun has
made a number of changes to the UFS file system in the past couple of years,
and several of them have broken AFS.

It's worked really well for us.  I haven't noticed any significant 
performance hits, but I don't know how much of the gains we've seen are
due to ZFS/namei and which are due to simply being on faster and more
modern hardware.

We currently have two types of ZFS /vice partitions: one server has locally
attached disks that are combined in a RAID-Z2 (RAID 6) configuration, while
the others are concatenation devices, with the "single" device the servers
see actually being configured in a RAID-6 configuration on our SAN.  All
are using ZFS, and work well.

You may need to do a little tuning of the ZFS Adaptive Replacement Cache
to make sure it doesn't gobble up all of your memory, and you should make
sure to turn off atime on the ZFS pool as well as changing the block size
to 64k.

I highly recommend reading through Dale Ghent's presentation about using
AFS with ZFS and Zones from last year's OpenAFS Workshop.  That helped
me a lot when I migrated our AFS infrastructure to Solaris 10.  The only
caveat I wanted to mention is that the method of tuning the ZFS ARC has
changed since last year; at that time, you had to manually tweak the
kernel, but it's now an /etc/system parameter.


Brian

On Mon, Apr 21, 2008 at 01:43:12PM -0400, Prasun Gupta wrote:
> On solaris the recommended filesystem of use for building afs filesystem
> is ufs without logging turned on. This is really a very primitive file
> system, and it loses a lot of the new features in the filesystems.
> 
>  
> 
> Has anybody used zfs successfully and in what configuration ?
> 
> a)   striped zfs
> 
> b)   Raid5 zraid1 zfs
> 
> c)   Raid6 zraid2 zfs
> 
>  
> 
> Any recommendations will be greatly appreciated ?
> 
>  
> 
> Thanks
> 
> Prasun
> 
>  
> 

-- 
Brian Sebby  ([EMAIL PROTECTED])  |  Unix and Operation Services
Phone: +1 630.252.9935|  Computing and Information Systems
Fax:   +1 630.252.4601|  Argonne National Laboratory
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] zfs File system

[Jeffrey Altman]

Russ Allbery wrote:

Okay, that makes me feel better about changing defaults, although we
probably need to keep providing inode packages as well, so it would
probably mean two builds for Solaris.


Or we should be providing a single package that includes both binaries 
and permits the correct one to be installed.





smime.p7s
Description: S/MIME Cryptographic Signature

Re: [OpenAFS] zfs File system

[Russ Allbery]

Jason Edgecombe <[EMAIL PROTECTED]> writes:

> It does error out. A namei fileserver will refuse to start and log an
> error message if a vice partition used to be inode. This happens even if
> you run "rm -fr *". I had to run mkfs/newfs on my vice partitions in
> order to switch formats -- after moving the data off, of course.

Okay, that makes me feel better about changing defaults, although we
probably need to keep providing inode packages as well, so it would
probably mean two builds for Solaris.

-- 
Russ Allbery ([EMAIL PROTECTED]) 
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] zfs File system

[Jason Edgecombe]

Jason Edgecombe wrote:

Russ Allbery wrote:

Robert Banz <[EMAIL PROTECTED]> writes:
 

On Apr 21, 2008, at 1:10 PM, Russ Allbery wrote:



 

I think it's generally a good idea to stick with one server
implementation on all platforms since that way everyone runs the same
(tested) code, but I seem to recall the migration from inode to namei
is pretty heinous (as in you probably can't do it in place and need to
bring up another server and move everything).
  

Nothing wrong with leaving the code in there, just don't make it the
default.



The problem with changing the default is that I'm worried really bad
things then happen if you blindly update your Solaris file server from
before the default change to after the default change.

Maybe we could fix that so that it would safely error out.  (Maybe it
already does, actually; I've not checked.)

  
It does error out. A namei fileserver will refuse to start and log an 
error message if a vice partition used to be inode. This happens even 
if you run "rm -fr *". I had to run mkfs/newfs on my vice partitions 
in order to switch formats -- after moving the data off, of course.


on a good note, namei seems to be faster than inode node for vos 
operations. The speed-up with ufs logging vs. nologging is worth 
switching to namei.
BTW, in Solaris 9 9/05HW an later, ufs logging is on by default and must 
be disabled in the vfstab. This is different from previous behavior 
which would only turn on logging if the filesystem exceed a certain size.


We've decided to use namei for all new fileservers because of the 
performance boost. It is more noticeable on stuff like fibre channel arrays.


I'll stop replying to myself now... ;)

Jason
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] zfs File system

[Jason Edgecombe]

Russ Allbery wrote:

Robert Banz <[EMAIL PROTECTED]> writes:
  

On Apr 21, 2008, at 1:10 PM, Russ Allbery wrote:



  

I think it's generally a good idea to stick with one server
implementation on all platforms since that way everyone runs the same
(tested) code, but I seem to recall the migration from inode to namei
is pretty heinous (as in you probably can't do it in place and need to
bring up another server and move everything).
  

Nothing wrong with leaving the code in there, just don't make it the
default.



The problem with changing the default is that I'm worried really bad
things then happen if you blindly update your Solaris file server from
before the default change to after the default change.

Maybe we could fix that so that it would safely error out.  (Maybe it
already does, actually; I've not checked.)

  
It does error out. A namei fileserver will refuse to start and log an 
error message if a vice partition used to be inode. This happens even if 
you run "rm -fr *". I had to run mkfs/newfs on my vice partitions in 
order to switch formats -- after moving the data off, of course.


on a good note, namei seems to be faster than inode node for vos 
operations. The speed-up with ufs logging vs. nologging is worth 
switching to namei.


Jason
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] zfs File system

[Russ Allbery]

Robert Banz <[EMAIL PROTECTED]> writes:
> On Apr 21, 2008, at 1:10 PM, Russ Allbery wrote:

>> I think it's generally a good idea to stick with one server
>> implementation on all platforms since that way everyone runs the same
>> (tested) code, but I seem to recall the migration from inode to namei
>> is pretty heinous (as in you probably can't do it in place and need to
>> bring up another server and move everything).
>
> Nothing wrong with leaving the code in there, just don't make it the
> default.

The problem with changing the default is that I'm worried really bad
things then happen if you blindly update your Solaris file server from
before the default change to after the default change.

Maybe we could fix that so that it would safely error out.  (Maybe it
already does, actually; I've not checked.)

-- 
Russ Allbery ([EMAIL PROTECTED]) 
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] zfs File system

[Robert Banz]

On Apr 21, 2008, at 1:10 PM, Russ Allbery wrote:


Robert Banz <[EMAIL PROTECTED]> writes:

Well, the issue was if you're using it on a server, and using what  
a lot
of people still consider the default (the inode fileserver),  
apocalyptic

dataloss may occur.


Oh, right, I completely forgot about that.

I would say that in addition to recommending people use logging  
with ufs
(or better, zfs!), that we should also push for deprecation of the  
inode

fileserver ;)


I think it's generally a good idea to stick with one server  
implementation
on all platforms since that way everyone runs the same (tested)  
code, but
I seem to recall the migration from inode to namei is pretty heinous  
(as
in you probably can't do it in place and need to bring up another  
server

and move everything).


Nothing wrong with leaving the code in there, just don't make it the  
default.


I get the feeling that someone with a large amount of data in inode  
form might have the impetus to write up a migration tool ;)


-rob
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] zfs File system

[Jeffrey Altman]

Russ Allbery wrote:

I would say that in addition to recommending people use logging with ufs
(or better, zfs!), that we should also push for deprecation of the inode
fileserver ;)


I think it's generally a good idea to stick with one server implementation
on all platforms since that way everyone runs the same (tested) code, but
I seem to recall the migration from inode to namei is pretty heinous (as
in you probably can't do it in place and need to bring up another server
and move everything).


This is correct.  We do not currently have the ability to run inode on
one partition and namei on another partition which is what would be
necessary to perform a single server transition.

Jeffrey Altman




smime.p7s
Description: S/MIME Cryptographic Signature

Re: [OpenAFS] zfs File system

[Russ Allbery]

Robert Banz <[EMAIL PROTECTED]> writes:

> Well, the issue was if you're using it on a server, and using what a lot
> of people still consider the default (the inode fileserver), apocalyptic
> dataloss may occur.

Oh, right, I completely forgot about that.

> I would say that in addition to recommending people use logging with ufs
> (or better, zfs!), that we should also push for deprecation of the inode
> fileserver ;)

I think it's generally a good idea to stick with one server implementation
on all platforms since that way everyone runs the same (tested) code, but
I seem to recall the migration from inode to namei is pretty heinous (as
in you probably can't do it in place and need to bring up another server
and move everything).

-- 
Russ Allbery ([EMAIL PROTECTED]) 
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] zfs File system

[Robert Banz]

On Apr 21, 2008, at 11:41 AM, Russ Allbery wrote:


"Prasun Gupta" <[EMAIL PROTECTED]> writes:

On solaris the recommended filesystem of use for building afs  
filesystem

is ufs without logging turned on.


Where is this?  We should update it.  That's the recommendation for a
*cache* file system, but not for the server.


Well, the issue was if you're using it on a server, and using what a  
lot of people still consider the default (the inode fileserver),  
apocalyptic dataloss may occur.


I would say that in addition to recommending people use logging with  
ufs (or better, zfs!), that we should also push for deprecation of the  
inode fileserver ;)


-rob
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] zfs File system

[Russ Allbery]

"Prasun Gupta" <[EMAIL PROTECTED]> writes:

> On solaris the recommended filesystem of use for building afs filesystem
> is ufs without logging turned on.

Where is this?  We should update it.  That's the recommendation for a
*cache* file system, but not for the server.

-- 
Russ Allbery ([EMAIL PROTECTED]) 
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] zfs File system

[Robert Banz]

At my last job, we had switched to using ZFS exclusively for our AFS  
servers, and had great luck with it. Look back in the archives of this  
list for discussion of it, and check out one of my ex-coworker's  
presentations from the 2007 AFS workshop on just that subject:


http://elektronkind.org/osol/OpenAFS-ZFS.pdf


On Apr 21, 2008, at 10:43 AM, Prasun Gupta wrote:

On solaris the recommended filesystem of use for building afs  
filesystem is ufs without logging turned on. This is really a very  
primitive file system, and it loses a lot of the new features in the  
filesystems.


Has anybody used zfs successfully and in what configuration ?
a)   striped zfs
b)   Raid5 zraid1 zfs
c)   Raid6 zraid2 zfs

Any recommendations will be greatly appreciated ?

Thanks
Prasun