Re: [OpenAFS] looking for (old) paper discussing the global name space

[Steven Jenkins]

You might take a look at https://www.hpl.hp.com/hpjournal/95dec/dec95a3.pdf.
That's for DCE/DFS, not AFS, but it discusses how global namespace
management was done in the DCE/DFS context.

DNS also has AFSDB records, and you could look to find the RFC(s) that
covered those (e.g., RFC 5395)  However, I do not know that they were/are
widely-used.

Steven


On Wed, Oct 9, 2019 at 2:53 AM Benjamin Kaduk  wrote:

> Hi all,
>
> A bit off topic, but as NFSv4 is implementing cross-server referrals, the
> prospect of having a central server that just makes referrals to other
> (data-hosting) servers as a sort of global file system namespace has come
> up, and I offered to find a reference to use for discussion of the prior
> work.  For context, that reference would be used from
>
> https://tools.ietf.org/html/draft-ietf-nfsv4-rfc5661sesqui-msns-02#section-11.5.6
> .  I did find
>
> https://www.cs.cmu.edu/~satya/docdir/spasojevic-tocs-afs-measurement-1996.pdf
> which has some discussion of this topic in the scope of a broader
> retrospective on the wide-area filesystem, but I was wondering if anyone
> knew of a more-focused reference for the global nature of the namespace.
>
> Thanks,
>
> Ben
> ___
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-info
>

Re: [OpenAFS] the future

[Steven Jenkins]

On Mon, Oct 1, 2012 at 10:09 PM, Jeffrey Altman <
jalt...@your-file-system.com> wrote:
...

> > Can anyone who has experience migrating to/from OpenAFS from/to anything
> > else in the last 2-3 years please comment? If there's really something
> > free, functional, and already included then I'd like to know what the
> > heck it is.
>
> I will remind the community of OpenEFS  which
> was developed specifically to permit a large financial institution to
> use NFSv3 for global software distribution via a firm-wide name space.
> While it is true that AFS3 provides a large amount of administrator
> functionality in the box that is not present in competing products, that
> doesn't prevent organizations from spending money to replicate that
> functionality at a higher layer.
>
>
Some further notes on OpenEFS: the original version was written to sit on
top of AFS.  A rewrite at a different financial institution was done on top
of NFS, and that product became OpenEFS.  Some preliminary work by the
original author (ie, not me) has been done to make OpenEFS run on top of
OpenAFS, but that is incomplete at this time (cf the openafs branch of
efs-core).  If people are interested in seeing OpenEFS support OpenAFS,
they should contact the OpenEFS developers (disclaimer: I'm one of the
OpenEFS developers, but right now, the NFS-based version is the priority,
at least for my employer).

Thanks,
Steven

Re: [OpenAFS] Linux client connection timed out after server failure

[Steven Jenkins]

Two things come to mind:

1- Sometimes fileservers are also db servers, and the 'connection timed
out' was because I forgot to update CellServDB and restart the client.  If
you watch network traffic, you would see this as an attempt to connect to
the old IP address but on the vlserver port, not the fileserver port.
2- I don't recall the interaction of open files and fs checkv, but there
might be a bug there.  That would be straightforward to test.

Steven

On Thu, Aug 16, 2012 at 12:20 PM, chas williams - CONTRACTOR <
c...@cmf.nrl.navy.mil> wrote:

> On Thu, 16 Aug 2012 11:19:09 -0400
> Bob Hoffman  wrote:
>
> > On 8/16/12 10:53 AM, Derrick Brashear wrote:
> > > it should recover in 2 hours, which is the vldb refresh time.
> >
> > Apparently that doesn't work for me.  After six days, I was still
> > getting "Connection timed out."
>
> i had the same thing happen to me as well during a server change.
> after 2 hours, the volume information still hadnt updated.  fs checkv
> did fix things though.  seems like there might be a bug.
> ___
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-info
>

[OpenAFS] Link to afs3-standardization mailing list archives is incorrect/not available

[Steven Jenkins]

I tried to look up the afs3-standardization mailing list archives
today, and the link in the left-hand  navigation frame on the OpenAFS
website points to:

http://michigan-openafs-lists.central.org/mailman/listinfo/afs3-standardization

However, the server for that link returns a 403 (Forbidden) when I try
to access it.

On further digging,, the page for all of the lists
(https://lists.openafs.org/mailman/listinfo/) has a URL that appears
to be valid and correct:

https://lists.openafs.org/mailman/listinfo/afs3-standardization

Could someone please look into that and let me know the correct URL?

Thank you,
Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] patch : AFS-Monitor (Perl)

[Steven Jenkins]

On Tue, May 31, 2011 at 2:02 PM, Jeff Blaine  wrote:
> In case Alf never gets to integrating this patch and
> releasing 0.3.3, here is what is needed to get
> AFS-Monitor to *build* with modern OpenAFS.  I have
> not tested anything other than building yet, and I
> am not a Perl extension author of any sort.
> ...

I talked with Alf, and I'll be taking over ownership of the module.
If there are other patches, feel free to let me know.

I've created a repository on github
(http://github.com/stevenjenkins/AFS-Monitor) and have incorporated
Jeff's patch; feel free to follow, fork, submit pull requests, etc.
I'll be uploading to CPAN as well, so it should show up there soon.

Thanks,
Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Guide on howto built from source for debian?

[Steven Jenkins]

On Wed, Mar 2, 2011 at 5:37 AM, Lars Schimmer  wrote:
> Hi!
>
> For some better testing of gerrit patches I want to get into building
> OpenAFS from source.
> Any one has a guide/link on howto do it "easy" from source on debian
> (stable/unstable) amd64?
>
> Someone mentioned a build script from Russ Albery, any mor info on that?
>

Russ' notes in src/packaging/Debian/README.source aren't sufficient?
Note especially the 'Importing a New Upstream Release' section.
If additional information is needed, I'm sure a patch would be appreciated.

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Problems from update of openafs 1.4.12 to 1.5.77

[Steven Jenkins]

On Thu, Oct 7, 2010 at 11:37 AM, Mike Coyne  wrote:
> I updated my openafs server to 1.5.77 yesterday,  this morning several of my
> volumes will not attach. This may have been due to my forgetting to
> re-compile Amanda-afs against the 1.5.77 or something completely different.
> I tried to run the salvager on the volumes but they will not come back on
> line, when I try to run vos online  I get
>
>

Did you change your bos configuration to support Demand Attach?  The
syntax between non-DAFS and DAFS changed.  You might find this article
helpful:

http://blog.endpoint.com/2009/06/getting-started-with-demand-attach.html

(a copy used to be up on the OpenAFS lore wiki, but I don't see it there now)

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Improving AFS performance for Joomla and other web sites

[Steven Jenkins]

On Wed, Sep 8, 2010 at 5:32 PM, Edgecombe, Jason  wrote:
> Hi everyone,
>
>
>
> We currently server our entire web tree out of AFS. This includes Joomla and
> other PHP-based applications. While I was benchmarking our new server, I
> noticed that the following rough performance numbers:
>
>
>
> Static files served from AFS:        ~200
> requests per second
>
> Joomla home page served out of AFS:         ~30 requests per
> second
>
> Joomla home page served from local ext3:   1000 requests per second.
>
>
>
> I’m running a 8 cores with hyperthreading (16 threads). I’m running the
> Apache threaded worker MPM on RHEL5.5 with mod_fcgid and OpenAFS 1.4.12.1. I
> have Apache configured for ~1000 threads.
>
>
>
> My data (250GB) and stat cache for the AFS client should be big enough to
> not have to hit the server.
>
>
>
> My afsd has the following options: “ -fakestat -dynroot -chunksize 18 -stat
> 50 -daemons 32 -volumes 2000 -blocks 212713128”
>
>
>
> What is recommended to improve my AFS performance for Joomla?
>

Skipping past a lot of the usual caveats around performance in a
complex system and going (somewhat) straight to the AFS-specific bits:

1- It would be very useful to also compare static files served from
ext3 -- that can give you somewhat of a baseline (i.e., compare the
ratio of static files in ext3 to Joomla in ext3 versus static files in
AFS to Joomla in AFS).  Offhand, the slowdown you're seeing between
static files in AFS vs Joomla in AFS does not seem egregious (based on
my experience tuning other web application servers, but not Joomla
specifically).

2- To see if cache configuration is an issue, look at your hit/miss
rates.  You can use afsmonitor to look at that dynamically.

3- Depending on your ability to do further instrumentation, it would
be very helpful to look at traffic between your web server and AFS
servers (e.g., turn on audit logs) to see exactly what your web server
is requesting from servers.  Alternatively,  you can capture packet
traces with tcpdump/snoop/wireshark.  Don't forget your pt and
vlservers, by the way.  I personally prefer packet traces, as I'm able
to see oddities like bad addresses (and retries) that would not show
up in server logs.

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] 1.4 (fs) -> 1.5 (dafs) migration

[Steven Jenkins]

On Fri, Aug 20, 2010 at 10:17 AM, Simon Wilkinson  wrote:
>
> On 20 Aug 2010, at 15:05, Gémes Géza wrote:
>
>> We currently have a small cell (2 db and file-servers
>> (1.4.7.dfsg1-6+lenny1), ~50 volumes). But plan to move the user home
>> dirs to afs resulting in >1000 volumes. Being afraid of the fs based
>> startup times we decided to move to a 1.5 based dafs prior to complete
>> the data migration to afs.
>
> If you have sufficient hardware, I would urge you to perform your migration 
> by using AFS's tools, rather than playing fast and loose with where volumes 
> are mounted. So, bring up some dafs fileservers, and use vos move to migrate 
> volumes from the 1.4 servers to the 1.5 ones. This has the advantage that 
> you've got a very simple backout plan if things go wrong for any reason, as 
> well as allowing you to perform a staged migration without any downtime.
>
>> The data is on a Coraid SAN, so it will be
>> simply unmounted from the old fileservers, and mounted to the new ones,
>> but what operations would we need to perform on the ubik data?
>
> If you do want to go down this route, you will need to update the VLDB 
> location for every volume that has changed fileservers. vos syncvldb and vos 
> syncserv should do this for you - but there will be a period of outage for 
> all of your clients.
>

I agree with Simon: using vos move is much better, as it gives you a
staged migration (and backout) plan, increases availability of data,
all without adding the risks involved with re-attaching disks to a
different system.

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS-devel] Re: [OpenAFS] Re: 1.6 and post-1.6 OpenAFS branch management and schedule

[Steven Jenkins]

On Thu, Jun 17, 2010 at 7:09 PM, Andrew Deason  wrote:
...
>> My current feeling is that it would be great if we could ship both
>> fileservers, side by side, with different executable names - but I
>> haven't looked at any of the code to see how complex this would be to
>> achieve.
>
> I haven't actually tried this... but at least from the perspective of
> the end result binaries, this seems simple. (the build process will be
> annoyingly longer, though, at least).
>
> 1.5 bosserver always understands the 'fs' and 'dafs' bnodes, I'm pretty
> sure, regardless of whether DAFS is enabled or not. So you can have an
> 'fs' bnode pointing at the non-DAFS binaries, and a 'dafs' bnode
> pointing at the DAFS ones. You should be able to switch between DAFS and
> non-DAFS just by stopping and starting the fs and dafs bnodes.
>

A caveat here is the set of volserver, salvager, and salvageserver
binaries: you need different ones for DAFS than for non-DAFS (iirc --
it's been a few years since I looked at that code).  That's not a
problem, of course; I'm just pointing that out as some may not be
aware of that.  Andrew, Tom,etc, if that's  not the case now, feel
free to clarify.

Thanks,
Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Re: [OpenAFS-devel] 1.6 and post-1.6 OpenAFS branch management and schedule

[Steven Jenkins]

On Thu, Jun 17, 2010 at 4:13 PM, Russ Allbery  wrote:
> Stephan Wiesand  writes:
>> On Jun 17, 2010, at 21:44 , Russ Allbery wrote:
>
>>> Yes, absolutely.  It's one of the reasons why 1.6 has taken so long.
>>> There probably isn't any other sane way to drop in a major, disruptive
>>> change, but certainly the long-term goal is to ensure DAFS works in
>>> production, then make it the default (I expect for 1.10 or 2.0), then
>>> remove the non-DAFS code so that we get back down to one implementation
>>> (almost certainly not before 2.2).
>
>> sorry, I disagree. If you (the developers and the gatekeepers) are sure
>> that DAFS is the way forward, and reasonably close to being ready, 1.6
>> and on should not support anything else. Why defer this to 1.10?
>
> I'm pretty sure you can find lots of other people on these lists who can
> explain why they don't want to have to run DAFS to upgrade to 1.6.  :)
>

I thought that enabling DAFS to be on by default was the major feature of 1.6.

I do not quite see the point of moving the release numbering from 1.5
to 1.6 if DAFS is not enabled by default.

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

[OpenAFS] Re: [OpenAFS-devel] 1.6 and post-1.6 OpenAFS branch management and schedule

[Steven Jenkins]

On Wed, Jun 16, 2010 at 6:07 AM, Hartmut Reuter  wrote:
...
>>
>
> Without --enable-fast-restart after a fileserver crash the salvager used to
> salvage all volumes in all partitions before the start of the fileserver.
> On large fileservers this could take hours and sometimes the salvager went out
> of memory and crashed himself leaving still volumes not attachable.
>
> With the Demand Attach Fileserver (DAFS) this initial salvage is not necessary
> any more, however, each volume which was not cleanly detached before gets
> salvaged in the background. This is a nice feature which allows the most
> demanded volumes to come up soonly, I hope, but still salvaging will take 
> hours
> because it's the same amount of work that has to be done.
>

Keep in mind that DAFS never brings volumes online unless requested by
a client, so some volumes may never be attached; it also takes volumes
offline after a period of unuse, so a DAFS server will only need to
salvage the 'active' volumes after a crash.  The combination of those
two features greatly reduces the number of volumes to salvage, so it's
not actually doing the same amount of salvaging (in the general case)
as a traditional fileserver.

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Performance issue with "many" volumes in a single /vicep?

[Steven Jenkins]

On Wed, Mar 17, 2010 at 12:50 PM, Steve Simmons  wrote:
> We've been seeing issues for a while that seem to relate to the number of 
> volumes in a single vice partition. The numbers and data are inexact because 
> there are so many damned possible parameters that affect performance, but it 
> appears that somewhere between 10,000 and 14,000 volumes performance falls 
> off significantly. That 40% difference in volume count results in 2x to 3x 
> falloffs for performance in issues that affect the /vicep as a whole - 
> backupsys, nightly dumps, vos listvol, etc.
>
> My initial inclination is to say it's a linux issue with directory searches, 
> but before pursuing this much further I'd be interested in hearing from 
> anyone who's running 14,000 or move volumes in a single vicep. No, I'm not 
> counting .backup volumes in there, so 14,000 volumes means 28,000 entries in 
> the directory.
>

Could you provide filesystem information? (e.g., what filesystem, what
parameters given/used by mkfs, etc)  That information is often quite
significant.

In particular, since you mention Linux, I'm guessing you're using
ext3.  With journaling (ie, ext3 and not ext2)?  With directory
hashing turned on?

Thanks,
Steven Jenkins
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Re: advice on troubleshooting blocked cache manager on MacOS?

[Steven Jenkins]

On Wed, Jan 27, 2010 at 5:22 PM, Adam Megacz  wrote:
>
> Derrick Brashear  writes:
>> You don't. You can ask the vlserver, which is how the CM found out anyhow:
>> vos listaddrs -printuuid -noresolve
>
> Yikes, that list is full of incorrect addresses.  How on earth is the
> list compiled?
>

Fileservers registering themselves when they start.  I suspect you
want to look at the man pages for NetInfo and NetRestrict so that you
control how they register with the vlserver.

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] All limitations of OpenAFS

[Steven Jenkins]

On Tue, Jan 26, 2010 at 10:59 AM, Lars Schimmer
 wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Harald Barth wrote:
>>> - -max number of db servers in one cell?
>>
>> I have seen cells with up to 7 db servers. Most cells use 3. A few use five.
>
> I know that 3 db servers are fine to work with.
> But I do not know the exact size and setup of the cell - maybe it could
> be handy to setup a db server on every fs, to.

I would not suggest setting up a dbserver on every fileserver.  There
is only a limited gain to be had, and you are greatly increasing
complexity and timing in a failure situation.

3-5 dbservers should be adequate for almost all cells.

> And as the cell is planned in the 20-200 fileservers area, I just asked.

There is a hard limit of 254 fileservers per cell.  I have not
personally seen a cell with more than about 40-50 fileservers, but
there is no technical reason you could not have more.

> I would love to have 3-5 db servers distributed and fileservers
> seperate, but money and other limitations could force me...
>
>> I don't understand why you would want more than say 3. If you have
>> unreliable network connections (say between Asia and South America)
>> you should rather consider making more cells than one spanning the
>> whole area with the bad network bottleneck in the middle. Here "good"
>> is not bandwidth but good uptime, low packet drop and reasonable
>> roundtrip times (multiple satelite hops are bad).
>>
>>> For a new EU project
>>

I agree with Hartmut with respect to creating multiple cells rather
than creating a single, very large cell.  There are several schools of
thought with respect to architecting a very large namespace as you are
describing.  Could you explain the expectations your users would have
with respect to a WAN problem impacting your cell?

>...
> Our setup would look like:
> 1 cell EU wide, with 5-200+ local sites, each has at least one fileserver.
> Each site has groups AKA departments.
> Each group has special volumes on "its local" fileserver and on other
> fileservers RO/RW rights to special directories/volumes.
>
> (setup is guided by the rule "data HAS to be kept local and only local
> groups and external persons with special rights should be able to read
> it. Local groups should be able to make only very small subpart of data
> available to one or more other (external) groups).
>

It would be very helpful to understand why you desire a single cell.
I suspect your main concern is to keep a single username/userid,
group/groupid namespace.  If that's your desire, one strategy you
could use is:

- have one cell per site
- use a single Kerberos domain across all cells
- replicate your pts database across all cells
- set up a single namespace

Managing replication across 200 cells would be very interesting, however.

Another strategy would be to maintain multiple Kerberos domains and
PTS groups.  If you choose to use that strategy, you will need to
create and maintain careful naming conventions to manage the mappings
between users and locations across each cell.  This would look more
like the 'public' AFS space.

Also, as you mention, managing volume names will be a challenge.  Many
large sites have tools in place to manage namespace management (i.e.,
mappings between paths and volumes) along with administrative
delegation (e.g., allowing certain users to do certain operations like
vos release's but  not other operations).

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Setting the BOS Server's Restart Times

[Steven Jenkins]

On Mon, Jan 11, 2010 at 1:48 PM, Gary Gatling  wrote:
>
> Greetings,
>
> We run several openafs servers at NCSU on Solaris 10 and Red Hat Enterprise
> Linux.
>
> By default, the servers restart every Sunday morning at 4:00am. This was not
> a problem until we asked another group to take over our backups. They are
> saying that the server restarts are causing problems with the backups
> because of the schedule.
>
> My boss mentioned that the restart feature was added in transarc 3.2. We are
> running openAFS 1.4.11.
>
> Is the restart still needed? If we don't have a restart will we see memory
> leaks? Do other organizations have experience with turning the restart off?
> Did it hurt anything having it turned off? Anyone doing longer restarts by
> using crons to kick off the restarts like once per month instead of once per
> week in the BosConfig file? Would doing that be a bad idea?
>


Automatic restarts are not normally needed -- I know of many sites
that turned off weekly restarts years (e.g., 10+) ago.

I don't know of anyone using bos cron jobs to kick off restarts once
per month, but that's certainly doable and not a bad idea.  Also, even
longer gaps between restarts is common (e.g., on the order of once or
twice per year).

Steven Jenkins
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS-devel] Re: [OpenAFS] Thinking about 1.6

[Steven Jenkins]

On Thu, Dec 17, 2009 at 8:44 PM, Matt W. Benjamin  wrote:
> Hi,
>
> I do think that we need to avoid assuming lack of clamor for some feature 
> means folks don't think it's important.

Very good point.  Thanks for the reminder.

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS-devel] Re: [OpenAFS] Thinking about 1.6

[Steven Jenkins]

On Thu, Dec 17, 2009 at 2:29 AM, Alistair Ferguson
 wrote:
>
> Hi,
>
  Since, as you note, there are sites which actually do rely on this, are
 you in fact meaning to say that the option would exist, but not be 
 mentioned
 in the usage?  I guess I'd also like to hear from Steven on pthreaded-ubik.
  It appears that it actually isn't 100% stable, but is close, and 
 apparently
 it's issue(s) are difficult to provoke.

>>>
>>> What we have in the code base definitely has bugs; Are there fixes
>>> which have not made it back upstream?
>>>
>>
>> Not significant fixes, no, but there are some minor ones.
>>
>> At the moment, I know of no one interested in running pthreaded ubik,
>>
>
> Really ?  Then you haven't been paying attention ;)
>

Let's see..this mailbox has no mention of anyone wanting it since I
mentioned it to Jason as something I was working on in the
summer...neither does that mailbox...nor does that RT queue...nor this
or that mailing list..nor

As far as I can tell, Jason is the only one expressing any interest in
the past 8 months or so..and that was only for the monthly newsletter.

So I guess I haven't been paying attention.

:)

>
>> so there's a bit of chicken and egg -- i.e., I know what fixes need to
>> be done, but need some people willing to run it and give a green light
>> on it (or to kick me and tell me it's still broken).  And I'd need to
>> carve out some time to work on it.
>>
>
> AFAIK the major bug (the VL DB server handing clients incorrect file-servers
> addresses causing them to lose access to volumes) hasn't been resolved ?
>
>

Correct.  (Long, boring description with lots of details deleted -- if
someone wants those details, let me know).

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

[OpenAFS] Re: [OpenAFS-devel] Thinking about 1.6

[Steven Jenkins]

On Wed, Dec 16, 2009 at 1:04 PM, Simon Wilkinson  wrote:
...
> *) Make demand attach the default, but provide --disable-demand-attach-fs to
> allow old-style fileservers to still be built
>

First, I think this is a *really* good idea.

However, the bos configuration for the fileserver changes from
non-DAFS to DAFS, so we need to make sure that people know that as
part of the upgrade.  If you just replace the binaries and restart,
you won't be happy.

Tom or Andrew, do you have any suggestions there what we might do to
alleviate that (e.g, in code)?

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS-devel] Re: [OpenAFS] Thinking about 1.6

[Steven Jenkins]

On Wed, Dec 16, 2009 at 1:21 PM, Derrick Brashear  wrote:
> On Wed, Dec 16, 2009 at 1:16 PM, Matt W. Benjamin  wrote:
>> Hi Simon,
>>
>> I would generally agree with almost all of this, except, probably removing 
>> --enable-fast-restart?
>
> In a demand attach universe, this isn't really useful anymore

Agreed.

>
>> Since, as you note, there are sites which actually do rely on this, are you 
>>in fact meaning to say that the option would exist, but not be mentioned in 
>>the usage?  I guess I'd also like to hear from Steven on pthreaded-ubik.  It 
>>appears that it actually isn't 100% stable, but is close, and apparently it's 
>>issue(s) are difficult to provoke.
>
>
> What we have in the code base definitely has bugs; Are there fixes
> which have not made it back upstream?

Not significant fixes, no, but there are some minor ones.

At the moment, I know of no one interested in running pthreaded ubik,
so there's a bit of chicken and egg -- i.e., I know what fixes need to
be done, but need some people willing to run it and give a green light
on it (or to kick me and tell me it's still broken).  And I'd need to
carve out some time to work on it.

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] vos move failures?

[Steven Jenkins]

On Wed, Oct 21, 2009 at 11:06 AM, Eric Chris Garrison
 wrote:
> Derrick Brashear wrote:
>> On Wed, Oct 21, 2009 at 10:39 AM, Eric Chris Garrison
>>  wrote:
>>> I've been having trouble with moving some of our larger volumes (>200GB in
>>> size):
>>>
>>> [r...@rfs3 qla2xxx]# vos move public.sudoc_01 rfs3 h rfs3 e -localauth
>>>
>>> Failed to move data for the volume 536878704
>>>   Possible communication failure
>>> vos move: operation interrupted, cleanup in progress...
>>> clear transaction contexts
>>> move incomplete - attempt cleanup of target partition - no guarantee
>>> cleanup complete - user verify desired result
>>>
>>> The above took an hour or two, and I've seen it take longer to fail.
>>> Large volume moves don't always fail, just quite often.
>>>
>>> Another question is, if it's local to the same machine from one partition
>>> to another, why does 200GB take hours to "vos move"?  Any ideas?
>>>
>>> I'm using openafs-server-1.4.11-1.1.1 on RHEL 4.
>>
>> What's in the VolserLog?
>
> Wed Oct 21 10:05:19 2009 1 Volser: Clone: Cloning volume 536878704 to new
> volume 536890205
> Wed Oct 21 10:05:19 2009 VAttachVolume: Failed to open
> /vicepe/V0536878704.vol (errno 2)
>
> Also... on the retry that's running right now, I see a lot of these:
>
> Wed Oct 21 11:03:17 2009 trans 8 on volume 536890205 is older than 3450
> seconds
>

If  a volserver crashes and gets restarted, you'll want to look at the
previous log.

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] error on vos release

[Steven Jenkins]

On Thu, Jun 11, 2009 at 2:01 PM, Derrick Brashear wrote:
> On Thu, Jun 11, 2009 at 1:53 PM, Markus Köberl 
> wrote:
>> vos release prj -verbose
>>
>> prj
>>    RWrite: 536873899     ROnly: 536873900     Backup: 536873901
>>    number of sites -> 3
>>       server julius.spsc.tugraz.at partition /vicepb RW Site
>>       server cleopatra.spsc.tugraz.at partition /vicepa RO Site
>>       server julius.spsc.tugraz.at partition /vicepb RO Site  -- Not released
>
> so which of the servers has an orphaned copy of the readonly volume on
> it, not one of those referenced above?
>

My guess is that julius.spsc.tugraz has a copy of the RO on /vicepa.

But that's just a guess.

> remove it and your life will get much better.

Yep.

---
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Problem with klog

[Steven Jenkins]

On Fri, May 29, 2009 at 10:18 AM, David Robson  wrote:

>>
>> My suggestion is to not worry about klog at all and instead use kadmin
>> to create principals, kinit to get Kerberos tickets, and aklog to
>> convert those tickets to AFS tokens.
>>
>
> This works fine on the AFS server, but how do I get it to work on an AFS
> client?
>

Can you confirm that you're successfully getting Kerberos tickets on
the client?  e.g., can you kinit to a principal you know exists and
show the output with 'klist'?

Once you have a ticket, you should be able to run 'aklog' and convert
tickets to tokens. If that doesn't work, it would be helpful to see
the output of klist, as well as the output of the -d option to aklog.

> I thought that all that was required was for the sysadmin (on the client) to
> define my cell and server in /usr/vice/etc/CellServDB.local on the client,
> and
> restart openafs-client.  Users would be able to authenticate with
> kinit/aklog
>
> Obviously I was wrong.  What needs doing on the client?
>


You're correct -- my suspicion is that your Kerberos configuration
(/etc/krb5.conf) on your client does  not match what you have on your
server, but that's just a guess.

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Problem with klog

[Steven Jenkins]

On Fri, May 29, 2009 at 7:06 AM, David Robson  wrote:
...
> I have set up an AFS cell, a partition a volume and a user and an acl.
>
> On the server machine, I can authenticate as the user with kadmin and aklog,
> and then I have read/write access to the user's /afs home directory.  All
> good so far.
>
> However, I can't authenticate with klog, on the AFS server, or on client
> machines.
>
> If I run "klog ", I get the error message
>
> "Unable to authenticate to AFS because Authentication Server was
> unavailable."
>

Note that using klog + kaserver is one option, and that using kadmin
and aklog is a different option -- you can't mix the two.

As you discovered via googling, it's recommended that you use an
external Kerberos infrastructure rather than klog + kaserver.

> After a bit of googling, I find that I should be running the kaserver, I
> do so by running /usr/afs/bin/kaserver as root in the xterm.  How should
> it be run, and with which arguments??
>
> With kaserver running, I now get the error ...
>
> "Unable to authenticate to AFS because user doesn't exist."
>
> But the user exists!  I created it with kadmin -q "addprinc "
>

This is because your principal is in your third party KDC, not in the
kaserver.  To create principals in the kaserver, you use the 'kas'
command.  But again, since you already have a working 3rd party KDC,
just don't use the kaserver and klog at all.

> Further googling suggests I shouldn't be running kaserver, but kdc.
> However I AM running krb5kdc, but it doesn't seem to be listening on
> the same port as kaserver (7004)
>
> I am confused and stuck.  Can anyone put me in the right direction?
>

My suggestion is to not worry about klog at all and instead use kadmin
to create principals, kinit to get Kerberos tickets, and aklog to
convert those tickets to AFS tokens.

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] peak vnodes (fwd)

[Steven Jenkins]

On Mon, Mar 30, 2009 at 10:08 AM, Dimitris Zilaskos
 wrote:
...
>>
>> I would be very interested in seeing performance numbers of the two
>> configurations (the configuration  you have now vs a configuration
>> when you have set -stat to be 11000 or so).
>>
>
> Ok, so it does not mean anything "bad":)
>
> Do you have a particular test in mine I could run to produce some useful
> results for you? This is a rather old system which is being heavily
> hammered(web server over afs serving a lots of files to the public).
>
>

I do not have a particular test in mind, but since you are
encountering that message, if you are in a position where you could do
some testing, it would be very interesting to replay your weblogs, for
example, with your current setting and with -stat set to 11000 and see
if there are any significant performance differences.

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] peak vnodes (fwd)

[Steven Jenkins]

On Mon, Mar 30, 2009 at 8:29 AM, Dimitris Zilaskos
 wrote:
...
> After moving to 2.6.29 the following started showing up in kernel logs. What
> does it mean?
>
>
> uname -an
> Linux tassadar 2.6.29 #1 Sat Mar 28 22:52:51 EET 2009 i686 unknown unknown
> GNU/Linux
>
>
> dmesg|grep peak|head
> peak vnodes: 4164

It means that your system had to dynamically allocate some vnodes
because you needed more than whatever you had provided as an argument
to afsd (i.e., the '-stat' argument was too low).  Based on what you
included in your mail, you should probably set your -stat value to at
least 11000.

Note that you do not have to change your -stat value -- the AFS kernel
module will use the -stat value as the default starting value and
increase dynamically as needed.  However, if you provide a higher
-stat value, you will save some overhead as AFS will not need to do
the dynamic allocation.

I would be very interested in seeing performance numbers of the two
configurations (the configuration  you have now vs a configuration
when you have set -stat to be 11000 or so).

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] fileserver goes down overnight

[Steven Jenkins]

On Tue, Mar 24, 2009 at 2:13 PM, david l goodrich  wrote:
...
>> > sprawl# ps auxw | grep /openafs/
>> > root   376  0.0  0.0 2316     4 ?       DW    5:33PM 0:00.83 
>> > /usr/pkg/libexec/openafs/volserver
>> > root   727  0.0  0.0 8664  2384 ?       IW> > /usr/pkg/libexec/openafs/fileserver
>> >
...

Can you get a pstack and lsof of the volserver process?  (You may not
be able to even get that much info..).

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] [Q] Moving one volume from one cell to other cell

[Steven Jenkins]

On Sat, Mar 14, 2009 at 9:24 AM, Lars Schimmer  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> S.J.Chun wrote:
>> Hi,
>>
>> Can I move volume from one cell(original one) to the other cell? We have
>> a fileserver at a.com but we want to move it to b.com cell. Can we just move
>> it by changing ThisCell and sync{serv,vldb} thing?
>
> IMHO best way is to vos dump the volume and vos restore it on the new cell.
>

If you are only moving one volume, the suggestion Lars gave is best.
However, if you need to move an entire server, and you have a large
number of volumes, you could change CellServDB and ThisCell as you
have mentioned, then restart the fileserver and run vos syncvldb for
the fileserver in the new cell; you should also run a script to do the
appropriate vos remove/delentry's from the old cell. A post-move audit
should also be done to verify everything is correct.

Note that this assumes your Kerberos configuration lets you move a
fileserver from one cell to another without reconfiguration (i.e.,
adding new keys, removing old ones).  Also, this does not discuss what
you expect the client behavior to be in with the move -- you'll need
to take some care to ensure you have the right level of availability.
Specifically, you don't mention when and how you want to change the
mountpoints from one cell to another (i.e., before or after doing a
move) -- changing the mountpoints is very important.

The tradeoff between the two processes (vos dump/restore one volume at
a time versus moving an entire fileserver) is a tradeoff between time
and scope of potential outage.  Note that in the vos dump/restore
case, your clients can continue to access the existing server, even
after the dump/restores have taken place.  This may cause consistency
problems with your data.  On the other hand, doing vos dump/restore
limits the scope of a single outage -- problems are likely to be at
the individual volume level, not widespread across the server.

The fileserver reconfiguration, on the other hand, is an
all-or-nothing approach.  An accident or problem could have
significant impact in your environment.

You should consider your needs, and regardless of the option you
choose, you should test the process in a test environment and make
sure you are comfortable with the procedure before trying it in
production.

If I were doing this type of operation for the first time, I would
strongly prefer the single volume at a time approach.  On the other
hand, if I were trying to do large-scale Internet-level storage, I
would spend the time engineering the second approach so that the
operations staff could move a server from one cell to another in an
automated fashion.

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

[OpenAFS] Re: dynamic allocation of vcaches

[Steven Jenkins]

On Tue, Feb 10, 2009 at 8:48 PM, Steven Jenkins
 wrote:
> We have a patch (in RT 124344)

Derrick points out that the ticket number is actually 124334.  My
apologies on that typo.

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

[OpenAFS] dynamic allocation of vcaches

[Steven Jenkins]

We have a patch (in RT 124344) for adding support to Linux for dynamic
allocation of vcaches rather than the current hard limit provided by
the -stat flag to afsd.  It currently does not have a command-line
argument to turn it on (i.e., if the patch is applied, you get dynamic
support), and I want to make sure the interface we provide is a
reasonable one.  I propose:

afsd -stat  -dynamic-vcaches

where N would be the initial value of the number of vcache entries but
that the cache manager could dynamically allocate as many as needed).

Any concerns with that feature and interface?


Thanks,
Steven Jenkins
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] AFS FileServer and Heartbeat

[Steven Jenkins]

On Mon, Feb 2, 2009 at 9:13 AM, Cristina Bulfon
 wrote:
> Hello,
>
> We'd like to configure AFS Fileserver using  heartbeat in active/passive
> mode.
...
> On the heartbeat side we have the following haresource file
>
> afsitfs3.roma1.infn.it  IPaddr2::Y.Y.Y.Y/24/eth0:0
> afsitfs3.roma1.infn.it  drbddisk::r0  Filesystem::/dev/drbd1::/vicepa::xfs
> afsitfs3.roma1.infn.it  drbddisk::r1  Filesystem::/dev/drbd2::/usr/afs::ext3
> afsitfs3.roma1.infn.it Y.Y.Y.Y   afs
>
> On the afs side we've also configured the NetInfo and the NetRestrict
> file, in the NetInfo we put the IP address stored on VLDB server ( in this
> case Y.Y.Y.Y) while in the NetRestrict put the IP addresses of the two
> servers.
>
> The problem is that everything is working fine inside the LAN  but If I try
> to execute a simple command like "ls"  from outiside the LAN I got
>
> Connection Timed Out
>
> If we look at the FileLog file we saw that the Fileserver takes the IP and
> hostname as the real server instead of the virtual one.
> 
> Mon Feb  2 13:18:37 2009 FileServer host name is 'afsitfs3.roma1.infn.it
> Mon Feb  2 13:18:37 2009 FileServer afsitfs3.roma1.infn.it has address
> 141.108.26.29 (0x1d1a6c8d or 0x8d6c1a1d in host byte order)

This implies an error in the NetRestrict file -- it should contain the
address of the IP of the actual server.  Could you provide the actual
contents of the NetInfo and NetRestrict files?  That would aid in
debugging this problem.

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] OpenAFS and Xen wierdnesses: regular loss of afs server connectivity

[Steven Jenkins]

On Fri, Jan 30, 2009 at 12:35 PM, Chris Kurtz  wrote:
> Specs:
>
...
>
> Jan 30 10:28:48 www4 kernel: afs: Lost contact with volume location
> server 149.169.146.57 in cell mars.asu.edu
> Jan 30 10:30:03 www4 kernel: afs: volume location server 149.169.146.57
> in cell mars.asu.edu is back up
>

This isn't the fileserver at all (so the documentation I pointed you
to is interesting, but not necessarily relevant), but is the vlserver.
 Some of the same suggestions apply, though:

1- are the processes crashing? (bos status, logs, etc)
2- are you having quorum issues? (udebug, logs, etc)
3- can you give more resources to the vlservers?  For example, are
they running on their own VMs?
4- how is your cell configured?  (e.g., do you have 3 vlservers)

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] AFS design question: implementing AFS over a highly-distributed, low-bandwidth network

[Steven Jenkins]

On Mon, Jan 19, 2009 at 9:22 PM, Chaz Chandler  wrote:
> So, in your considered opinion, would it be wiser to go with one cell, put up 
> with potential quorum snafus, ensure the clients set their preferred server 
> to a local one, and move R/W volumes when users move locations?  Or to go 
> with multiple cells, perhaps one as master, and resolving ambiguities on a 
> per-volume basis depending on how that volume is intended to be used?
>

I see it as follows:

Option 1: single cell; leverage vos move, with some tools to handle
volume replication, but you'll probably be able to leverage existing
OpenAFS commands without needing a lot of further automation.

Pros:
- simple in the read-only case
- will work, even in the case where remote accesses are done
- will require some development, but  not as much as the other case
- disconnected operations may make this a nice option down the road
- there may be something you could do with vos shadow to help get
copies of data into each site

Cons:
- quorum will probably be a problem; long enough outage can mean that
none of your clients can function.  That's a pretty bad worst-case
scenario (i.e., this configuration does not degrade gracefully)

Option 2: each site is a separate cell; use incremental vos dump + vos
restore to move volumes from cell to cell (i.e., site to site), with
regular dumping+restoring taking place to ensure no site is more than
a day out of date, plus some changes to higher level (e.g.,
'container' volumes) and some recovery tools to help in an outage.

Pros:
- no ubik quorum problems
- loss of remote site won't impact local accesses (but will impact
remote accesses; however, the failure case will be to access something
at most 1 day old).
- each site will have a copy of the data, even if it is slightly out
of date, so a local operation could be done manually to help 'get back
in business'

Cons:
- more complicated
- will require more development than the other option
- messing up a container volume and pushing it out will trash everyone
(implication: don't mess up a container volume, do checking on
container volumes before pushing  them out.)

I would first test to see how having a single cell works with your
network (i.e., measure network utilitization) -- a simpler environment
would be great if it will work.  But I would plan to go with multiple
cells and use incremental dump and restore to move volumes around.

But I also have experience with that type of operation, and I've
wanted an excuse to write something open source that handles some of
the common cases Morgan Stanley's system does, so I'm very biased in
that decision.  As someone new to OpenAFS, that may be more complexity
than you want to tackle.

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] AFS design question: implementing AFS over a highly-distributed, low-bandwidth network

[Steven Jenkins]

urrent architectures
and implementations.

More study of this case should be done: it's the real hard one.

> - A user dir: large amounts of data updated from a single location, but user 
> may move to any other site at any time, potentially with up to a day of 
> transit time in which a volume could be moved to the destination site.

I would consider building a system that would let me have an offline
copy of the user volumes in each location, and synchronize  them on
some regular basis, depending on usage patterns.  You could then also
provide a utility like 'move to site X' that the users could run which
would find the current location of that home directory, take it
offline, do an incremental dump & restore, then bring the new volume
online.

An alternative to that would be disconnected operations: since I'm
guessing that your users will need their own data frequently, but
seldom will they need each others, it might work out that your users
can put their home volumes into the cache on their local system (this
would work best if the users had laptops that they carry from site to
site, but would  not work so well if there are fixed systems at each
site that they use), and then you could engineer something so that
when they re-connect to the network, automatically sync the volume
from their local system to the local site, updating the various
databases behind the scenes.

That assumes development work, however.  And I don't know if that
would meet your requirements.

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

[OpenAFS] exposing RPC code<->name mappings via rxgen extension, a library, and a new utility

[Steven Jenkins]

I would like to expose RPC code<->name mappings so that other programs
within OpenAFS can avoid hard-coding the mappings, as well as be able
to export them to the users (who might find them useful in debugging
network traces, for example, where their particular tool does not know
what a particular opcode corresponds to). From a user-level, it would
work as follows:

$ translate_rpc -name PR_INewEntry
500

It would accomplish this by extending rxgen to pul the procedure
identifier and opcode from the specification file: e.g., given the
following hunks of code:

"package" 
...
 :

["proc"] [] []
 ["split" | "multi"]
["=" ] ";"

would produce new tables which would automatically go into the .h file
for that specification file: e.g.,

_name[] = 
and
_opcode[ = 


Then, in src/util, we would #include those .h files in
translate_rpc.h. translate_rpc_lib.c would export two functions:

rpc_name_to_opcode
rpc_opcode_to_name

which other programs could use.

translate_rpc.c would implement the user-level command.

Any concerns about that approach and/or the utility itself?

Thanks,
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Current Limits of OpenAfs ?

[Steven Jenkins]

On Sat, Jan 10, 2009 at 1:08 PM, Matt Benjamin  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Adding to that, I'd suggest reading the slides from Alistair Ferguson's
> keynote at the 2008 AFS workshop.  The current client/server ratio
> (2000/1, going to 5000/1?) and configured callback's per file server (4
> million?) in one Morgan Stanley enviornment are mentioned, IIRC.
>

It should be emphasized that their highest client/server ratios occur
in what they call 'read-only' cells, which only contain read-only, not
read-write data.  Callbacks for those two different types of data are
handled very differently (e.g., file level for read-write data vs
volume level for read-only data); thus I suggest carefully
benchmarking with actual access patterns to determine how far you want
to scale.

I would be further cautious taking numbers from that presentation and
applying them elsewhere.  Slide 20 points out they currently have 1.5K
hosts per cell (750 per fileserver) now, and their goal is for OpenAFS
to scale to handle 10K hosts per cell (5K per server) down the road.
Those numbers are further muddled because they do not specify how many
read-only copies are available total for data (or even if those ratios
are for read-only or read-write cells), how (un)balanced access
patterns are, how much headroom for growth those ratios include,
hardware configurations, etc.

Fileservers aside, and without knowing more about what Gary is trying
to accomplish, the ~10 million user number is quite interesting.  I do
not know of any OpenAFS installation (or IBM AFS) installation that
has tried to scale beyond 100K or so users.  I know that we have done
tests in a lab with very large numbers of users, but having that scale
in production would likely present some new challenges.

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] enumerating a users memberships

[Steven Jenkins]

On Mon, Jan 12, 2009 at 11:28 AM, David Bear  wrote:
> Thanks to all the quick answers.
>
> When I attempt this for another user in our cell, I get an access denial. I
> assume than that I could do this if I were a member of system:ptsviewers?
>

Yes, that should work.  Also, members of system:administrators can
view membership.

I notice that the pts_membership.pod pages need to be updated to
reflect system:ptsviewers.  I'll submit that.

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Non existent RO volumes for root.afs and root.cell

[Steven Jenkins]

On Fri, Dec 12, 2008 at 6:45 PM, Shigeki Misawa  wrote:
> Hello,
>
> I have a problem with an AFS cell where the root.afs and root.cell volumes
> appear
> to reference non-existent read only volumes.
>
> vos examine root.afs yields :
>>  root.afs  536870912 RW 21 K  On-line
>>  testafs01.bnl.gov /vicepa
>>  RWrite  536870912 ROnly  536870921 Backup  536871046
>>  MaxQuota   5000 K
>>  CreationTue Oct 23 11:49:12 2001
>>  CopyWed May 23 12:31:55 2007
>>  Backup  Fri Dec 12 02:06:01 2008
>>  Last Update Fri Oct 29 16:37:42 2004
>> 151 accesses in the past day (i.e., vnode references)
>> > RWrite: 536870912 Backup: 536871046
>> number of sites -> 1
>>server testafs01.bnl.gov partition /vicepa RW Site
>
>
> Note that the 3rd line says "ROnly 536870921" but the 11th line does not
> mention a read only volume.
>
> Running "vos examine 536870921" yields:
>
>>  Volume 536870921 does not exist in VLDB
>> >  Dump only information from VLDB
>> >  root.afs
>>  RWrite: 536870912 Backup: 536871046
>>number of sites -> 1
>>server testafs01.bnl.gov partition /vicepa RW Site
>
> vos examine of root.cell yields similar problems.
>
> Is there a way to recover from this state ?
>

You should do (while having credentials as the AFS admin):

vos addsite testafs01.bnl.gov /vicepa root.afs
vos release root.afs

to recover the root.afs replica, and

vos addsite testafs01.bnl.gov /vicepa root.cell
vos release root.cell

to replicate root.cell.

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

[OpenAFS] Re: [OpenAFS-devel] interface for vos split

[Steven Jenkins]

On Thu, Jan 8, 2009 at 2:49 PM, Russ Allbery  wrote:
> "Steven Jenkins"  writes:
>
>> I can see why some would like that better, but, for me, the most obvious
>> is:
>>
>> /usr/afs/bin/vos split -id osd.1 -newname osd.1.a -dirname /afs/mycell./a/b/c
>>
>> i.e., an absolute path.
>
> I expect more of our users to use relative paths (to the current working
> directory) than absolute paths, actually.
>
>> The problem with that is that it more or less requires the user of vos
>> split to be running in a cache manager so that the volumes and paths
>> between / and the directory to split at can be resolved.
>
>> Putting that into a vos command is very difficult.  And traditionally
>> vos commands have had no dependencies on a cache manager (getting
>> credentials and determining the local cell information are the primary
>> exceptions, but there are no filesystem dependencies).
>
> I can see this problem, and from a layering perspective inside AFS, it's
> an annoying one.
>
> However, I think I can also say with fairly high confidence that the end
> user really does not care about this at all and would even be mystified by
> it.  I've never run vos from a system that didn't have a cache manager,
> and I'm a fairly sophisticated user.  The distinction made here is one
> that I doubt anyone in my team would even know.
>
> It's certainly reasonable to sacrifice usability sometimes to reduce code
> complexity.  That may be a good idea to do here -- I don't really have a
> strong opinion.  I just wanted to mention that if the command doesn't just
> take arbitrary paths, either absolute or relative to the current working
> directory, to find the directory on which to split, we are sacrificing
> usability and the typical user reaction is going to be "why can't it do
> that?"
>
>> fs getfid is in the cache manager.  Since vos is user-space, it doesn't
>> have access to the same routines.
>
> I don't think user-space is the word that you want here, since fs is also
> a user-space command.  But I know what you mean.
>
>> An RPC to the volserver was suggested as the best way to handle that.
>
> I think that's icky.  I'd use the cache manager.
>

Let me clarify a bit more here: your analysis gets quite close to why
I don't care for the -dirname syntax.

fs getfid (like virtually all of the fs subcommands) is implemented by
marshalling arguments and then making a PIOCTL call into the kernel.
Without a cache manager, you can't get a response to that PIOCTL.
Even with a cache manager running, you would need to marshal up the
arguments and make a PIOCTL, which means linking vos.c in with new
libraries.  vos is already huge; I think making it understand how to
do PIOCTL calls would be significant enough to where we would look at
getting rid of fs entirely (i.e., if vos can do one PIOCTL, adding the
rest is relatively straightforward).

I think that would be a bad idea -- hopefully that explanation helps
explain my perspective, but if not, feel free to ask for more
clarification.

On the other hand, if we try to implement relative name mangling in
vos (e.g., via a new volserver RPC), users will constantly question
"Why can't I use an absolute path here?".

Thus it seems to me most straightforward from a user-experience
viewpoint to require the vnode.  and not provide a -dirname option,
but make sure the man page shows how to use fs gettfid to determine
the vnode.  The user experience would not necessarily be ideal, but it
would be consistent, thus less frustrating to the user in the long
run.

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

[OpenAFS] Re: [OpenAFS-devel] interface for vos split

[Steven Jenkins]

First, my apologies for not getting the mailing lists right the first
time.  I'm cross-posting my reply to Russ in hopes that this thread
will be the main one.

On Thu, Jan 8, 2009 at 1:44 PM, Russ Allbery  wrote:
...
>>
>> # /usr/afs/bin/vos split -id osd.1 -newname osd.1.a -dirname
>> bin.1.4.8-osd.hartmut
>
> I'd like this a lot better.

I can see why some would like that better, but, for me, the most obvious is:

/usr/afs/bin/vos split -id osd.1 -newname osd.1.a -dirname /afs/mycell./a/b/c

i.e., an absolute path.

The problem with that is that it more or less requires the user of vos
split to be running in a cache manager so that the volumes and paths
between / and the directory to split at can be resolved.

Putting that into a vos command is very difficult.  And traditionally
vos commands have had no dependencies on a cache manager (getting
credentials and determining the local cell information are the primary
exceptions, but there are no filesystem dependencies).

But I'll add the relative pathname support if that's what others want.

>
>> So some questions on that:
>>
>> 1- Is the current user interface undesirable?
>> 2- Is the suggested  interface better?
>> 3- If the suggested interface is better, how should the interface be
>> implemented?  The suggestion I received was to add an RPC to vos;
>> would we want that interface exposed (e.g., as a vos command), or
>> would it be better to stay as an internal RPC (much as AFSVolForward
>> works today)
>
> Why wouldn't vos do the same thing that fs getfid currently does to get
> the vnode?

fs getfid is in the cache manager.  Since vos is user-space, it
doesn't have access to the same routines.  An RPC to the volserver was
suggested as the best way to handle that.  My follow-up question on #3
is primarily to ask if that's how we want it handled, and if we want
to expose that interface via a vos command.  In other words, would a
command like:

vos getvnode -volume $vol -relative_path path

be useful.

>
> It should, of course, double-check that the directory name given is within
> the volume that one is splitting.

Definitely.

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

[OpenAFS] vos split interface question

[Steven Jenkins]

I've been recently looking at Rx OSD again, and the vos split
functionality in particular.  I have some questions about the user
interface that I  think the community as a whole might want to give
input on:

First, a refresher on vos split:

# /usr/afs/bin/vos help split
/usr/afs/bin/vos splitvolume: split a volume at a certain directory.
Usage: /usr/afs/bin/vos splitvolume -id  -newname
 -dirvnode  [-cell ] [-noauth] [-localauth]
[-verbose] [-encrypt] [-noresolve] [-help]

so you can use vos split to take a large, unwieldy volume and break it
into more manageable pieces.

With respect to the interface, the user of vos split has to provide
the vnode of the directory where the volume should be split.  Now,
this is not extremely difficult, as fs getfid /path/to/directory will
yield the fid, so the user can extract the vnode from that.  For
example:

# fs getfid /afs/steven.endpoint.com/osd.1/bin.1.4.8-osd.hartmut
File /afs/steven.endpoint.com/osd.1/bin.1.4.8-osd.hartmut
(536871265.19.333) contained in volume 536871265

then

# /usr/afs/bin/vos split -id osd.1 -newname osd.1.a -dirvnode 19

However, there was a discussion some time back that requiring the user
to know the vnode is cumbersome, and that a better interface would let
the use specify the relative path from the root of the volume: e.g.,

# /usr/afs/bin/vos split -id osd.1 -newname osd.1.a -dirname
bin.1.4.8-osd.hartmut

So some questions on that:

1- Is the current user interface undesirable?
2- Is the suggested  interface better?
3- If the suggested interface is better, how should the interface be
implemented?  The suggestion I received was to add an RPC to vos;
would we want that interface exposed (e.g., as a vos command), or
would it be better to stay as an internal RPC (much as AFSVolForward
works today)
4- If the suggested interface is not better, what suggestions are
there for a more clear interface?

My personal thought is that while the current interface is a little
awkward, the new interface I sketched above is a little more
user-friendly but not necessarily a lot better.

I suggest that we simply keep the current interface, but if the new
interface(s) is(are) needed, I can add that.

Others' thoughts?

Thanks,
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

[OpenAFS] Re: [OpenAFS-devel] Deadline- Jan 9! CFP: 2009 OpenAFS & Kerberos Best Practices Workshop

[Steven Jenkins]

On Mon, Jan 5, 2009 at 5:16 PM, Esther Filderman  wrote:
> This is a reminder that the Call for Participation for the 2009
> OpenAFS & Kerberos Best Practices Workshop will end Friday January 9,
> 2009.


The following is a set of topics I'm interested in presenting, and I'd
like to get some feedback from potential attendees -- which of these
would be most/more useful?

1- Cache tuning: examples of things people do with filesystems and how
to tune your cache and architect your namespace to make those perform
better.  This will use standard performance monitoring utilities,
server audit logs, client performance statistics, and network traces.
Suggestions for particular performance cases welcomed.

2- Examples of some scalability issues in OpenAFS and how to see what
those issues/limits are in your environment before they happen to you
in production.  The set of questions is tentatively:
- do I need faster/bigger hardware?
- can I consolidate fileservers?
- should I replicate more ROs?
- can I safely double the amount of disk on my fileservers?
- can I safely double the number of clients I have in my cell?

4- VLDB internals and tricks: the structure of the vldb and things you
can do with the the various vldb utilities, including dumping it to
XML, editing that, and then rebuilding your vldb

5- A description of the life of an SA in a well-run OpenAFS cell,
complete with examples of the tools and utilities that are running for
monitoring the health of the system so the SA can rest peacefully, the
delegation mechanisms in place so that users can handle their own
admin, the architectural layout so that the network people are happy,
etc)

6- how to determine if you should split existing cells and how to
accomplish that migration with a minimum of pain

Thoughts, feedback, suggestions (and requests) welcomed.

Thanks,
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] client tuning using the -daemons option

[Steven Jenkins]

On Tue, Nov 11, 2008 at 12:00 PM, Jason Edgecombe
<[EMAIL PROTECTED]> wrote:

> What metrics can I use besides my applications run time to see that the
> client is optimized properly? xstat_fs_cache gives me so much data that
> it's confusing. How do I tell when increasing stat, volume, and daemons
> options would improve performance?
>

Brief answer:

- xstat_fs_test is for the fileservers -- it's probably not what
you're looking for to see how to tune your client's cache (but could
be useful when wanting to see an aggregate of N clients so that you
can tune your fileservers to scale better)
- xstat_cm_test is for clients -- that's probably what you want
- afsmonitor is the other tool you could use -- it's debatably more
user-friendly, especially if you set up a config file for it to only
show the info you want.  SLACs documentation for their Perl module is
pretty accessible:
http://search.cpan.org/~alfw/AFS-Monitor-0.3.2/pod/afsmon_stats.pod

You basically want to look at cache hits vs misses (both dcache &
vcache).  Those are the most helpful statistics.

There is a lot of information out there about cache tuning -- feel
free to ask more specific questions.  I know somewhere there are some
sample config files for afsmonitor, and I believe the old IBM docs
have each field in afsmonitor documented (although there has almost
certainly been some drift between those docs and what is present and
supported in the code now).

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] OpenAFS TAC

[Steven Jenkins]

On Tue, Nov 4, 2008 at 3:16 PM, Derrick Brashear <[EMAIL PROTECTED]> wrote:
...
>>
>> I'd say for myself that if this is the way the community plans to move
>> forward, that it might not be a bad idea for the community to simply get
>> started organizing an informal proto-TAC and see what issues it's going
>> to present, and what opportunities it may offer.
>
> Speaking again for myself only, I think such a thing where
> organizations likely to join the foundation as sponsor and non-sponsor
> members get seated (as do community members) until the foundation
> exists and we formally compose the TAC based on actual eligibility is
> somewhat desirable.
>

Speaking for myself & my organization, we would be happy to
participate in a pre-foundation, proto-TAC meeting.

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Mounting a backup

[Steven Jenkins]

...
>>
> This feature is available as an option to the afsd command. If you add
> the "-backuptree" option to the afs client startup script, then it will
> prefer backup volumes once in a backup volume. This "-backuptree" option
> is not enabled by default.
>

Wow.  Great.

I'm too out-of-date; I  need a 'What's changed in OpenAFS' view sometime.

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Mounting a backup

[Steven Jenkins]

On Tue, Oct 28, 2008 at 8:44 PM, Loren M. Lang <[EMAIL PROTECTED]> wrote:
> I am looking for a way to mount a snapshot of last nights backup.  All
> volumes have a backup volume currently and I then mounted root.cell
> as /afs/./archive.  archive was read-only and did not change when
> I changed the RW volume for root.cell and released it, but all the mount
> points under the backup volume traversed to the RO or RW volumes.  Is
> there a way to make the mount points traverse similarly to how mount
> points for volumes with RO clones work?


There is not a way to create a mountpoint that will 'automatically'
traverse backup volumes without specifying the backup volume itself,
for example, you'll need to do something like

fs mkmount -dir /afs/mycell/backups/somepath -vol myvol.backup

Changing traversal semantics while in .backup volumes (e.g., to be
'once in a  .backup volume, prefer to traverse to backup volumes)
could be done, but, to my knowledge, that's not on anyone's Road Map.

How helpful would that change be?

Thanks,
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

[OpenAFS] Unix Systems Programmer position at End Point

[Steven Jenkins]

The team I work in at End Point Corporation is looking for a strong
self-starter developer with a good Unix systems programming background
to work on the development and maintenance of software for a variety
of projects.

Required skills:
* Strong communication skills and professionalism
* Ability to dive into new technologies and quickly come up to speed
* Strong C & Unix programming experience
* Solid understanding and experience of full product lifecycle
* Solid understanding of TCP/IP
* Familiarity with representative protocols, architectural, and
operational issues in large, distributed systems (e.g., Kerberos, RDBMS,
DNS, LDAP, configuration management, backups, monitoring, etc.)
* Experience with at least two of: Linux, Solaris, HP-UX, AIX, BSD
* Knowledge of at least one scripting language (awk, Perl, Python, Ruby,
etc.)

Desired skills:
* Familiarity with distributed and/or clustered filesystems (NFS, AFS,
CIFS, GFS, NAS, etc.)
* Kernel level knowledge and experience with Unix-based systems
* Familiarity with RDBMS internals (e.g., DB2, Oracle, Sybase, MySQL,
PostgreSQL, etc.)
* Knowledge of Microsoft Windows internals

Things that will set you apart:
* strong Solaris administration and/or kernel development (dtrace, mdb, ZFS)
* strong skills on non-Linux operating systems
* a passion for analyzing core dumps (especially kernel dumps)
* AFS operational knowledge
* AFS development experience (e.g., prototypes of projects on the OpenAFS
roadmap would be particularly interesting)
* strong experience with configuration management systems (cfengine,
bcfg, lcfg, quattor, Puppet, etc)
* experience in automating virtual environments (e.g., libvirt, VMware
Perl API, etc)

More about the team & our work:
* not a web development position (see End Point's website for those)
* most of our work is currently in C (75%)
* some in Perl and shell (20% -- mostly test scaffolding and
automation, but some CPAN work, too)
* some work in Ruby (5% currently, but varies depending on projects --
note that this is
completely separate from the Ruby on Rails web development that other teams
at End Point do).
* other languages done occasionally (e.g., we've done a bit of Java &
Python over the past year, but
less than 1% of the other coding we've done)
* projects in AFS, PostgreSQL internals, Ruby internals, Varnish,
Quattor, Puppet, OpenSolaris
and various Perl modules on CPAN (and in various Linux distributions)
over the past year.

This is a full-time salaried position with benefits. Employees work
from home offices
in the U.S., or at our office in Manhattan.

Except in exceptional circumstances, applicants must be U.S. citizens
or permanent residents.
Principals only, please.

Please email Steven Jenkins  with information about
your interest, experience, and qualifications.

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Re: Win2K AFS server, setup SL4.5 test-cell server then migrate...

[Steven Jenkins]

On Tue, Oct 14, 2008 at 12:44 PM, avison48 <[EMAIL PROTECTED]> wrote:
>
> Respectful greetings AFS gurus,
>
>
> In trying to migrate the account-data & user-data off this antique Win2K
> IBM/TransArc3.5 AFS server onto RHEL4.5 OpenAfs server (& retire the Win2K
> server);
> would this idea work:
>
> I believe the RHEL4.5 server can become a secondary fileserver & database
> server to this antique Win2K AFS server. That would enable it to mirror all
> the user-account info & user-data - correct?
>

That is probably correct.

The primary question is where your current Kerberos authentication
comes from.  If you're running the kaserver on your IBM/Transarc AFS
server (and given your other emails, that appears to be the case),
then your migration should  not be too painful (except for updating
the CellServDB information on all the clients).

A secondary question is how your IP address space will look: given
that you'll have exactly 2 AFS servers, you could get into some
difficulty with respect to making changes.  If you can, bring up the
Red Hat server with an IP address lower than your current W2K AFS
server.

Finally, keep in mind that AFS will only automatically synchronize the
user-account info and the volume location information -- you will
actually have to move the user data yourself via vos commands (e.g.,
vos move).

> No clients at all would point to it - just want it to mirror all the data.
>
> Would it then be possible to shutdown AFS services on this new server,
> and configure it as an AFS server _not_ secondary to the antique Win2K AFS
> server, but as "the" AFS server of the afs cell.
> (I believe if it does not know about / try to contact the other AFS
> server, it could exist)
> With its own kerberos server running on it, its own Keyfile.
> Still with no AFS clients except itself.
>
> If it could "wake up" and own all the user-accounts & user-data as its own,
> in its status as an AFS server (not secondary) - that would be great.
>
> I'm concerned about where in its ... mirroring of user-account & user-data,
> would be critical ties or references to its old "master" AFS server.
>

In general, doing a bos removehost $server $server-to-forget + bos
restart $server $process of a server process removes references to
$server-to-forget.

However, for user data, you will actually  need to migrate data from
your existing AFS server to the new one: vos listvol -server
$old-server will tell you what is on your old server.  You should
cross-check that with vos listvldb -server $old-server to make sure no
references remain to your old server.

> Would that work?
>
> If not, how can the data with all the ACLs & etc be transferred to a new
> server?
>

Assuming you know the current afs password, you should be able to
follow the instructions about adding an additional server machine at
http://www.openafs.org/pages/doc/QuickStartUnix/auqbg006.htm#HDRWQ99,
paying special attention to the section on adding a database server.

At a high level, you need to:

- tell your existing AFS server about the new one
- configure the new AFS server
- tell all your clients about the new AFS server
- verify everyone is talking to the new one properly
- move all user data (i.e., vos move's, and any vos backup's, vos
addsite's, and vos release's needed).
- stop the old AFS server
- tell the new server to stop talking to the old server
- tell all the clients to stop talking to the old server

The documentation has details on those steps.  Feel free to ask,
though, if you have further questions.

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Mount point weirdness: fs lsm X, fs lq X return different volumes for same mount point.

[Steven Jenkins]

If you have tcpdump data for cache manager <-> vlserver and cache
manager <-> fileserver traffic during one of these corruptions, that
could be very helpful.  I've found tcpdump (or wireshark/tshark) to be
useful in tracking down issues like this because you can very quickly
see if the problem is

1- cache manager asking for the wrong thing to start with (possibly
cache corruption -- not conclusive because you have to determine if
the cache manager got the bad data and cached it, or if the cache
manager 'broke' the data; picking one client and clearing it's cache,
then re-trying can help answer that question).  Note that  in your
case, this is pretty unlikely, given that you saw it across multiple
clients on mutiple OSes.
2- vlserver giving a wrong answer
3- neither of the above, which means the fileserver is giving a wrong answer.

The usual suspects (e.g., cmdebug) are also helpful here.  It might
also be useful to get the callback state from the fileservers to see
what they think the cache managers have for data (if in case 3 above).
 Given that 'failed volume moves' seem to have been a trigger for
this, logfiles might have something interesting, especially if you can
provide volume names & volume id's for the X-volumes'

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

[OpenAFS] Re: [OpenAFS-announce] OpenAFS gatekeeper plan draft available

[Steven Jenkins]

On Thu, Sep 4, 2008 at 4:42 PM, Derrick Brashear <[EMAIL PROTECTED]> wrote:
> OpenAFS has started down a path to formally organizing. One piece of
> the plan involves formalizing roles, and so provided here is a draft
> plan for the role of the gatekeepers as well as expectations of
> contributors in their interactions. Comments welcome, either privately
> or ideally to openafs-info@openafs.org .
>
> http://www.openafs.org/foundation/gatekeepers.html
>

BTW, since you posted to openafs-announce, I'm responding here.  If
this should go elsewhere, let me know.

Under the contributors section:

4. should provide a unit test for any contribution.(*)

I suggest that there are problem some things that the developers could
agree as 'Must's and not just 'Should's.  For example,

Must provide Qt-style comments for all new or modified functions,
macros, and non-local variables.

If that simply finds its way into the details for item 6:

must provide documentation for submissions which add commands or
functionality or change user-visible behavior, meeting current
documentation standards.(*)

(i.e., the 'current documentation standards') I'd be happy with that as well.

And I'd encourage others to think about what would make sense from a
development standpoint and make other suggestions as well.

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

[OpenAFS] Re: [OpenAFS-announce] OpenAFS gatekeeper plan draft available

[Steven Jenkins]

On Thu, Sep 4, 2008 at 4:42 PM, Derrick Brashear <[EMAIL PROTECTED]> wrote:
> OpenAFS has started down a path to formally organizing. One piece of
> the plan involves formalizing roles, and so provided here is a draft
> plan for the role of the gatekeepers as well as expectations of
> contributors in their interactions. Comments welcome, either privately
> or ideally to openafs-info@openafs.org .
>
> http://www.openafs.org/foundation/gatekeepers.html
>

Looks good.

A note:

# must for major architectural changes engage the community for
guidance and the Advisory Board for a final decision on acceptance. A
review of the proposed changes must be provided to the community and
Advisory Board by the Gatekeepers. (Standards-based changes must go
through the protocol standards process rather than an internal
process)

Better is:

For major architectural changes, must engage.

I would also add functional tests in addition to unit tests.  Many
functional tests already exist (in src/tests), so improving that
should be encouraged.

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

[OpenAFS] Wanted: broken vldb's

[Steven Jenkins]

One of our developers, Mike Robinson, has been building some tools to
do vldb repair, and we're looking for some additional vldb's for
testing purposes.  In particular, we'd like some that are broken.  (If
you have broken prdb's or backup db's, that's ok, too, but his tools
aren't quite ready for testing on those yet.)

Please contact him directly at [EMAIL PROTECTED]

Thanks,
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Re-using Drive on New Server

[Steven Jenkins]

On Fri, Aug 8, 2008 at 1:43 PM, Jason C. Wells <[EMAIL PROTECTED]> wrote:
> I have created a new Openafs server using Debian.  I have moved the physical
> drive containing the AFS partitions and volumes into the new server.  I have
> mounted the partitions /vicepa and /vicepb.  vos listvol shows 3 offLine
> volumes on /vicepa and 9 offLine volumes on /vicepb.
>

Those will need to be salvaged via bos salvage.

> I didn't grab a list of volumes/mountpoints before I retired the old server.
>

Fortunately, the vldb  has that for you.  If you run

vos listvldb -server $old-server

you can find out that information.  To get the information properly
updated in the volume location server, you can run:

vos syncvldb -server $new-server

which will update your VLDB with updated location information for the
volumes that are now residing on your new server.

By the way, the mappings between filenames and volumes won't change --
all that needs to change is the location information in the VLDB
(i.e., the mappings between volumes and servers serving those
volumes), so once you do the vos syncvldb, you should be able to
access your data as before.  However, there are some circumstances
where you would need to run an 'fs checkvolumes' on each client --
from your description, that won't be needed in your case, but you
might take a look at that man page to understand where it might be
needed.

-- 
Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Can't create volume on Solaris 10

[Steven Jenkins]

On Wed, Aug 6, 2008 at 11:03 AM, Daniel Debertin <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I'm following the quickstart guide from openafs.org. Everything seems
> to work fine until ... I'm using version 1.5.51.
>
> Any ideas as to what I'm doing wrong? ...

1.5.51 is bleeding edge for Unix.  If you're wanting to help test
1.5.51, then using it is fine.  However, if you're new to AFS and are
just wanting to get up and running, you should use 1.4.7,  which is
the current production release.

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Can't create volume on Solaris 10

[Steven Jenkins]

On Wed, Aug 6, 2008 at 11:03 AM, Daniel Debertin <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I'm following the quickstart guide from openafs.org. Everything seems
> to work fine until the following command:
>
> afs0# vos create afs0 /vicepa root.afs -cell msi_afs -noauth
>
> Could not change quota, continuing...
>   : No such file or directory
> Failed to end the transaction on the volume root.afs 536870912
> : No such file or directory
> Error in vos create command.
> : No such file or directory
>
> I have /vicepa (just regular UFS) mounted with "-o nologging,quota",
> and have quotas enabled on that filesystem, though I have also tried
> it without quotas with the same result. I'm using version 1.5.51.
>
> Any ideas as to what I'm doing wrong? Let me know if you need more
> information.
>

That's odd.  I'd do a 'vos examine root.afs' to verify the volume is
there and then a manual 'vos setfields' to set the quota if it's
present.  It looks like the volume may not have gotten created,
though.

If you have FileLog, VolserLog, and VLLog, those would be useful, as
they may contain some hints as to why the create failed.

Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Serious trouble, mounting /afs, ptserver, database rebuilding

[Steven Jenkins]

On Wed, Jul 23, 2008 at 1:30 PM, kanou <[EMAIL PROTECTED]> wrote:
> And this:
>
> ==> /var/log/openafs/PtLog <==
> ptserver: Unknown code pt 11 (267275) Can't rebuild database because it is
> not empty
>

I suggest dealing with first things first: until your ptserver is up
and running correctly, your fileservers won't be up and running
correctly, and you won't be able to access files.

Could you clarify whether or not providing the -entries option to
prdb_check resulted in output for your database?  If that produces
good output, you can

1- rebuild the protection database
2- restart your ptservers
3- restart your fileservers
4- finally, you can see what problems you encounter accessing files

Steven Jenkins
End Point Corporation
http;//www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Serious trouble, mounting /afs, ptserver, database rebuilding

[Steven Jenkins]

On Wed, Jul 23, 2008 at 11:46 AM, kanou <[EMAIL PROTECTED]> wrote:
> Thanks for your answer.
> Well I found the file prdb_check. It doesnt print any errors. Only thing I
> can find is with
> ./prdb_check -database /var/lib/openafs/db/prdb.DB0 -uheader -verbose
> this line:
> Ubik header size is 0 (should be 64)
>

That error message isn't necessarily a problem.  However, you should
get output like:

Ubik Header
   Magic   = 0x354545
   Size= 0
   Version.epoch   = 1194566044
   Version.counter = 13041
Ubik header size is 0 (should be 64)
Database has 2384 entries

Additionally, you should see a messages about

Checking name hash table

Checking id hash table

Checking entry chains

Checking free list

Checking orphans list

Checking for unreferenced entries

Additionally, if you run with the -entries option specified, it will
dump out your database.  You can then use that to re-create your
contents.  Some scripting of that output would be required.

Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Serious trouble, mounting /afs, ptserver, database rebuilding

[Steven Jenkins]

On Wed, Jul 23, 2008 at 10:51 AM, kanou <[EMAIL PROTECTED]> wrote:
> Hello,
> well, there is a file called db_verify.c in the folder
> /usr/src/modules/openafs/ptserver but I don' know how to build it.

If I recall correctly, db_verify gets renamed to 'prdb_check' during
the install, so you should check for the existence of that file.

If you can't find it, you'll need to build it from source code: the
directions on the AFSLore wiki are a good place to start:

http://www.dementia.org/twiki/bin/view/AFSLore/HowToBuildOpenAFSFromSource

If you have problems building openafs-stable-1_4_x, you could get
openafs-stable-1_4_7 instead, as that is the latest official release.

Once you have built the tree, src/ptserver/db_verify should get built,
so you can simply copy it out of the source tree for your use.  If it
doesn't get built automatically for you, you can cd into src/ptserver
and do a 'make db_verify' manuall.

Also, feel free to ask for help here  or on the irc channel.

Steven Jenkins
End Point Corporation
http://www.endpoint.com/
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Replication

[Steven Jenkins]

On Wed, Jun 25, 2008 at 7:07 AM, Lara Lloret Iglesias
<[EMAIL PROTECTED]> wrote:
> Hello,
>
> I've just installed two afs servers on the same cell, but I've problems with
> them. I've done a mkmount and everything works fine, but when I try to make
> another mount point, it just duplicates the content of the first one. I know
> afs has a replication system but I don't want to enable it, how can I turn
> it off??
>


Could you clarify how you've done the mountpoint creation?  AFS allows
you to have multiple mountpoints of the same volumes, but doing that
is not typical.

For example, if I have the volumes user.steven and user.lari, I could
mount them at

/afs/mycell/user/steven (for user.steven) and
/afs/mycell/user/lari (for user.lari)

Those mountpoints would be created like:

fs mkmount -directory /afs/mycell/user/steven -vol user.steven
fs mkmount -directory /afs/mycell/user/lari -vol user.lari

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] AFS kerberos

[Steven Jenkins]

On Tue, Apr 15, 2008 at 9:55 AM, Ron Croonenberg <[EMAIL PROTECTED]> wrote:
> Hello all,
>
>  Is there a gui tool (for Fedora 6) to manage AFS accounts, kerberos
> accounts ?  Is there something to easily create accounts with in bulk ?
>  (I used USS for AFS with KAS)
>

For AFS accounts (i.e., pts), pts has a -source argument that lets you
provide a file of commands that can be fed in.  I do not know of an
equivalent on the Kerberos side (it might help if you specified the
KDC you are using: MIT, Heimdal, Microsoft, etc).

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Client hangs

[Steven Jenkins]

On Wed, Apr 2, 2008 at 4:48 PM, David Sonenberg <[EMAIL PROTECTED]> wrote:
> We are currently in the process of migrating from a Samba domain to
>  native Active Directory.  Yesterday we switched the DNS and DHCP over to
>  the AD server.  A handful of users' clients hang for 30-90 seconds when
>  they browse the folder structure.  This is only happening to a handful
>  of users some of which are on logged into the new AD, and some are
>  logged into the old domain.  When they log on as a local user the
>  problem goes away.  It appears to be a problem with the profiles, but I
>  can't figure out what.  Any ideas?
>

Do you have a network traffic dump from one of those clients?  A
misconfiguration causing a timeout would be the first thing I checked
(e.g., bad addresses in the CellServDB), but seeing the actual network
traffic would show you if the client's were trying to talk to
non-existent systems or perhaps doing something you didn't expect.

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] vos syncserv and vos syncvldb

[Steven Jenkins]

On Sun, Mar 23, 2008 at 1:58 PM, Christopher D. Clausen
<[EMAIL PROTECTED]> wrote:
> I was trying to help someone on #openafs yesterday with a hosed vldb due
>  to an AFS server being initial setup on "localhost."  Once we determined
>  that was indeed the problem (which was not easy in and of itself) and
>  corrected, attempts were made to fix the vldb by using vos syncvldb and
>  vos syncserv.  However, these did not seem to help and I had the user in
>  question simply shutdown their AFS servers and manually delete the
>  vldb.DB0 and vldb.DBSYS1.  This worked, but I suspect this is not the
>  correct way to solve the problem.
>
>  Can someone provide the correct steps to make an incorrect (say
>  "localhost") entry disappear from the vldb and vos listaddrs or at least
>  say in what situations vos syncserv and vos syncvldb should be used?
>

Would:

- vos listaddr -noresolve (to make sure 'bad-addr' is indeed 127.0.0.1)
- vos changeaddr bad-addr new-addr'

not work?

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Server crash

[Steven Jenkins]

On Dec 7, 2007 6:46 PM, Steve Devine <[EMAIL PROTECTED]> wrote:
...
> Just a datapoint ..  running vos listvol with the fast option gives you
> very little info about the vols. It is great if all you are after is
> volume ids but if you want to know whats offline vs whats online its of
> no use.

Completely true.  However, if you're after raw speed, that's an
option.  If you manage your AFS  infrastructure with something like
Russ's volume database, you can bypass the vlserver and query that
database.  Alternatively, after doing the diff, you can then query the
vldb for the volumes you have offline.

There are definitely trade-offs involved.  Thanks for making sure
people are aware of that one.

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Server crash

[Steven Jenkins]

On Dec 7, 2007 1:21 PM, Steve Devine <[EMAIL PROTECTED]> wrote:
>
...
> As it turns out if you run vos listvol on the server itself its a little
> quicker and now it seems we are back up for the most part.
> I have often wondered however if there is a good way to periodically
> check the status of the vols without running listvol an all the servers.
> I think trolling through the backup server logs might do it as they will
> report offline vols.

vos listvol has a -fast option.  If you're on a namei server,  you
could do an ls /vicep* and diff the output of two as a quick check.

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Server crash

[Steven Jenkins]

What do your bosserver, fileserver, volserver, and salvager logs say?

Steven

On Dec 7, 2007 11:53 AM, Steve Devine <[EMAIL PROTECTED]> wrote:
> We had a server crash this morning and I after bringing it back up I am
> unable to get a vos listvol back from it. Or it is taking a very long
> time. These partitions are greater than 150 G . Its been running for 30
> minutes now and nothing back yet.
> The server (Version 1.4.2 ) is compiled with fast restart and I am
> trying to see what vols are Off-line. Is there any other way to find out
> what vols are off line?
> /sd
>
> --
> Steve Devine
> Network Storage & Printing
> Academic Computing & Network Services
> Michigan State University
>
> 506 Computer Center
> East Lansing, MI 48824-1042
> 1-517-432-7327
>
> Baseball is ninety percent mental; the other half is physical.
> - Yogi Berra
>
> ___
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-info
>
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Automatic move of volumes

[Steven Jenkins]

On 10/24/07, Derrick Brashear <[EMAIL PROTECTED]> wrote:
> 
> > dump/restore is just a mechanism in lieu of a volume copy operation.
> > Versionized clones could be interesting in this context, but I would
> > prefer to stay away from that approach as it makes it harder to
> > recover & see changes in the base volume.  I think having one RW per
> > 'generation' of ROs is reasonable.
>
> this is all implementation decisions, and the ones you make that make me
> wretch, well, i'm not running it so i don't matter.
>

It's a community/gatekeeper/elder/afs3-standardization issue: if the
Road Ahead has generational clones, that's one thing.  If it doesn't,
then it would be helpful to know.

As for wretched decisions, I'm always open to hearing about better
ways to do things.

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Automatic move of volumes

[Steven Jenkins]

On 10/24/07, Christopher D. Clausen <[EMAIL PROTECTED]> wrote:
...
> That sounds like a mis-use of the remsite command, although that is an
> interesting way to "hide" RO volumes.
>

Keep in mind the context of my comment -- I was looking at someone's
'move volumes en masse' code.  It's not how I would recommend handling
things.

> I assume that a client that gets rebooted / crashes is going to start
> reading the RW when it comes back up though, right?
>

Right.  Clients don't even have to reboot, though: vldb information
times out after 2 hrs, iirc.

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Automatic move of volumes

[Steven Jenkins]

On 10/24/07, Derrick Brashear <[EMAIL PROTECTED]> wrote:
> ...
> where's Phil so I can beat him up some more?
>

Heh.  He's about 100 feet from where you got your most recent Mac.
But you'll have to get through Security unless you can lure him out by
offering him another tattoo or something..

And, for the record, my money is on Phil unless you bring a ranged
weapon or have the element of surprise.

> >
> > It has _everything_ to do with namespace management.  In absence of
> > better tools, people are using vos release to do just that.  Note that
> > vos release isn't a bad tool; it's just being stretched beyond its
> > design because people need a way to do versioning of their namespaces.
>
> you want to dump and restore volumes. that's ugly. it's not a namespace
> issue; you want versioned volume clones.
>

dump/restore is just a mechanism in lieu of a volume copy operation.
Versionized clones could be interesting in this context, but I would
prefer to stay away from that approach as it makes it harder to
recover & see changes in the base volume.  I think having one RW per
'generation' of ROs is reasonable.

With versionized clones, you would need to create a mechanism to have
potentially infinite numbers of clones, with arbitrary generation
identifiers (eg, some would be ok with '1', '2', ..., but some would
want 'alpha', 'beta', ..., or 'dev', 'prod', etc).  IMO, that's better
done outside of the volume itself.

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Automatic move of volumes

[Steven Jenkins]

On 10/24/07, Derrick Brashear <[EMAIL PROTECTED]> wrote:
...
> not everyone has VMS.

*nod*, although it has been documented in public (at least some of the
pieces) for quite some time.  It seems CMU takes a similar approach,
and I suspect others out there have 'rolled their own'.  From the 'me
too's using vos release to handle versioning, it appears having a
general system more widely available would scratch a real itch.

> it has nothing to do with namespace management.
> nothing is as easy as "vos release" to copy data around.
>

It has _everything_ to do with namespace management.  In absence of
better tools, people are using vos release to do just that.  Note that
vos release isn't a bad tool; it's just being stretched beyond its
design because people need a way to do versioning of their namespaces.

I think it would be very useful if someone had the time/energy to
build a 'vms-lite' that people could adopt at their sites.  That seems
a more strategic direction than trying to extend RO capabilities.
That may be more of an openafs-devel or AFS3-standardization
discussion rather than openafs-info, though -- I don't know.

I also realize that building something like that is non-trivial.  If
people are interested in doing that, though, it seems a widely-needed
tool.

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Automatic move of volumes

[Steven Jenkins]

On 10/24/07, Brandon S. Allbery KF8NH <[EMAIL PROTECTED]> wrote:
>
> On Oct 24, 2007, at 10:15 , Steven Jenkins wrote:
>
> > - the RO handling is not good -- what happens if the _only_ RO is on
> > the old server and the remsite happens?  Clients with existing
>
> remsite is irrelevant:  it just informs the vlserver of where an R/O
> replica will be stored in the future, it has no impact whatsoever on
> what R/Os (if any) exist *now*.
>
> --

remsite is _very_ relevant for clients that don't already know about
the RO that has been remsite'd -- when they ask the vlserver for the
volume, the vlserver will tell them that only the RW exists.

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Automatic move of volumes

[Steven Jenkins]

On 10/24/07, anne salemme <[EMAIL PROTECTED]> wrote:
...
>- having a well-known, scheduled time for releasing certain volumes
> to ensure that files become available at an approximately-scheduled time
> (useful for website management, or for dealing with vos releases of very
> popular volumes typically near the top of an afs tree)
...

What's wrong with simply doing vos dump with the timestamp then?  And
having an name encoding rule that lets you determine the base RW
volume + timestamp.  For example,

'base' volume = mydata
'timestamp' volume is md.2007102402 (for the snapshot created via vos
dump mydata | vos restore at 2am)
you can then create RO clones of md.2007102401; RO's of 'mydata' aren't needed

cutting over from the md20071001 to md.2007102402 means an
intermediate volume  needs to exist that contains that change, but
it's contents can be programmatically determined based on the time.
You would  have 'dev' and 'prod' links in that volume that always
point to the hour-appropriate versions.

In general, I now understand some of what people are doing with the
RW/RO differences -- thanks for the explanations.  But it seems to me
that there are ways to deal with this -without- needing to keep RWs
and ROs intentionally out of sync.

It would be interesting, though, if vos addsite/release had a built-in
generational mechanism to mark ROs.  That could lead to other ways to
solve these problems.

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Automatic move of volumes

[Steven Jenkins]

On 10/24/07, Kevin Hildebrand <[EMAIL PROTECTED]> wrote:
>
> We've actually had this need a number of times...  Say for instance,
> you've installed a new version of software in volume X for testing
> purposes, or as Derrick suggests, using the volume as a web backend.
>
> Then you run out of disk space, have a failing server, etc, and you need
> to move the RO replications - there's no easy way to do so without
> releasing the volume.
>

I sort of understand this need, but I suggest that it's caused by poor
namespace management, and that the solution should be to improve that
rather than try to keep your RWs and ROs out of sync with each other.

There are several ways to have versionized trees: two are
- at the top (ie, different root volumes)
- in the middle of the filesystem space via symlinks (eg, have a
volume 'in the middle' of your hierarchy that lets you have 'dev' and
'prod' links) -- take a look at slides 15 & 16 of
http://www-conf.slac.stanford.edu/AFSBestPractices/Slides/MorganStanley.pdf
for one way to do that

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Automatic move of volumes

[Steven Jenkins]

It's great that you sent actual code.

However,

- there is no error checking in here, so there are potential issues (I
realize your script is generating the commands, not actually doing
them, but it still needs to address error checking -- if nothing else
stick a '|| exit 1' at the end of each command)
- handling RWs first will break RW/RO clones (if any exist) -- you
probably don't want to do that.
- the RO handling is not good -- what happens if the _only_ RO is on
the old server and the remsite happens?  Clients with existing
connections & callbacks are ok, but any new clients needing that data
now have no place to get it (even though the RO exists, the vldb
doesn't know about it, so clients will go against the RW instead).
Also, the actual data is left on the old fileserver ,which may (or may
not) cause a problem down the road.  Much better is to do a vos remove
(which updates both the vldb & file server) after the new clone has
been brought on line.
- in general, handling RW/RO pairs is best handled separately

I realize my 'general idea' email isn't complete enough to really
build a robust script.

Steven

On 10/24/07, chas williams - CONTRACTOR <[EMAIL PROTECTED]> wrote:
...
> $rw eq "RO" && $do_ro && do
> {
> $volume =~ s/\.readonly$//;
>
> printf "vos remsite %s %s %s\n", $from_server, 
> $from_partition, $volume;
> printf "vos addsite %s %s %s\n", $to_server, 
> $to_partition, $volume;
> printf "vos release %s -verbose -f\n", $volume;
> };
> }
>
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Automatic move of volumes

[Steven Jenkins]

On 10/24/07, Derrick Brashear <[EMAIL PROTECTED]> wrote:
...
> perl scripts exist to do it and I think have been posted here in the past;
> they may even deal with the "RO already exists" case.
>

It would be nice if there were a repository of publically available
contrib stuff like that.

> the interesting case is where the RW has unreleased changes and you want to
> recreate the ROs as they are now. i don't know of distributed tools to do
> this.
>

I hadn't really thought about people intentionally keeping their RWs &
ROs out of sync w/each other.  I'm not clear why someone would want to
do that -- could you elaborate?

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Automatic move of volumes

[Steven Jenkins]

On 10/24/07, Steve Devine <[EMAIL PROTECTED]> wrote:
> Jacob Volstrup wrote:
...run vos listvol 'servername vicepX'
>  >somefile and then grep out the RW vols and use a perl script to move
> em. If you have RO vols then you need to make sure you deal with them.
> But its pretty straight forward.


Assuming your new location does not already have the ROs on it:

1 vos listvol to get the list of volumes to move; if the old location
is not empty, then, in this order:

2 For RO volumes: vos addsite to the new location, vos release, vos
remove old ROs
3 For RW volumes: vos move
4 For .backup volumes: vos backup (or backupsys, depending on how you
made them in the first place) once the RW volumes are moved
5 Goto 1

All this can be done while users are accessing their data, but if you
do this at a relatively quiet time you can avoid some potential
problems (eg, when this is done, snapshots are taken -- if the
volserver can't get a 'good' snapshot, the vos move will fail; clients
may see messages about a volume being busy; etc).

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] Commercial support for AFS.

[Steven Jenkins]

On 10/3/07, Jesse W. Asher <[EMAIL PROTECTED]> wrote:
>
> I was evaluating AFS for an implementation a few months back and I ran
> across a couple of companies that provided support for OpenAFS.
> Unfortunately, I can't remember who they were.  I realize that OpenAFS
> is open source and, personally, I'm sure net-based resources would be
> sufficient, but this company is voicing a concern about not having a
> company to fall back on for support.
>
> Can anyone recommend some companies that provide full support for
> OpenAFS?  The requirements should be:
>
> 1)  Can be called 7/24 for support.
> 2)  Able to help diagnose and resolve administrative and configuration
> issues.
> 3)  Able to diagnose and resolve code type issues (by creating and
> issuing patches).
> 4)  Be a viable company rather than just a fly by night entity.
>
> Any ideas out there?  Thanks!!
>

The place you saw some companies listed may be the AFS Lore wiki:
http://www.dementia.org/twiki/bin/view/AFSLore/WhereToGetHelp

Steven
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

[OpenAFS] Unix Systems Programmer position at End Point

[Steven Jenkins]

End Point Corporation is looking for a developer with a strong Unix 
systems programming background to work with other team members in the 
development and maintenance of software for a variety of projects. The 
successful applicant will be able to perform analysis of complex
systems, recommend, design and implement changes to existing systems, or
re-architect systems to meet emerging goals.

Required skills:
* Strong communication skills and professionalism
* Strong C & Unix programming experience
* Solid understanding and experience of full product lifecycle
* Solid understanding of TCP/IP
* Familiarity with representative protocols, architectural, and
operational issues in large, distributed systems (e.g., Kerberos, SQL,
DNS, LDAP, backups, monitoring, etc.)
* Experience with at least two of: Linux, Solaris, HP-UX, AIX, BSD
* Knowledge of at least one scripting language (awk, Perl, Python, Ruby,
etc.)

Desired skills:
* Familiarity with distributed and/or clustered filesystems (NFS, AFS, 
CIFS, GFS, NAS, etc.)
* Kernel level knowledge and experience with a Unix-based system
* Familiarity with RDBMS internals (e.g., DB2, Oracle, Sybase, MySQL, 
PostgreSQL, etc.)
* Knowledge of Microsoft Windows internals

This is a full-time salaried position. Employees work from home offices
in the U.S., or at our office in Manhattan. Applicants must be U.S.
citizens or permanent residents. Principals only, please.

Please email Steven Jenkins <[EMAIL PROTECTED]> with information about 
your experience and qualifications.


___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] vos listvol and vos exam

[Steven Jenkins]

--enable-fast-restart?

Sorry, I thought you said something else (but related).  My mistake.

FileLog should tell you know which volumes weren't attached (as others have
pointed out).

Steven

Re: [OpenAFS] vos listvol and vos exam

[Steven Jenkins]

On 6/28/07, Steve Devine <[EMAIL PROTECTED]> wrote:


Steven Jenkins wrote:
>
>
> On 6/28/07, *Steve Devine* <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>
> wrote:
>
> We have switched some of our servers over to binaries compiled
> with fast
> restart.
>
>
>
> Others may disagree, but my opinion is that Fast Restart is not yet
> ready for production use.  Testing is always helpful, though.
Man I'm sorry to hear that .. we used this because we wanted to shorten
the down time for bringing a server back on line when it has 1.2
possible disk space. That a long salvage window but maybe it would have
been worth it.



I was talking about something else not being ready for production, not Fast
Restart (which is fine for production).  My apologies.

Fast Restart can shorten your startup times, but it leaves the System
Administrator to do the salvaging.  Were I to use Fast Restart, I would

- parse the output of FileLog after startup (or vos listvol vs vos listvldb
output) to get a list of volumes to salvage manually,
- possibly prioritize those (eg, bring RW's online first, not RO's that have
good copies elsewhere, etc)
- manually salvage them

I'd have that in a script to be run after a restart.

The benefit of Fast Restart is that even if you have a lot of volumes that
need to be salvaged, the fileserver can be serving other volumes while you
manually salvage the rest.  Keep in mind that you'll have to salvage those
one at a time or bos will shut the fileserver down.  If you have to salvage
a lot of volumes, Fast Restart will actually slow down your recovery time,
so you'll want to consider that trade-off.  (ie, you can't salvage in
parallel while the fileserver is running)

Steven

Re: [OpenAFS] vos listvol and vos exam

[Steven Jenkins]

On 6/28/07, Steve Devine <[EMAIL PROTECTED]> wrote:


We have switched some of our servers over to binaries compiled with fast
restart.




Others may disagree, but my opinion is that Fast Restart is not yet ready
for production use.  Testing is always helpful, though.

This morning we had to bring a server down for maint and when we brought

it back up several of the vols where in need of salvage.
I ran vos listvol thinking I would get a list of vols that were offline.
Instead it showed all vols as On-line




That is news to me (ie, showing all volumes as on-line).  I would love to
have a copy of

- bos stat -long
- vos listvol output,
- FileLog,
- any/all salvage logs,
- VolserLog


yet I had to salvage over a dozen

that I have found so far.
I get waiting for busy volume errors.
Any advice how I can locate vols on a fileserver that need  salvaging
short of just biting the bullet and salvaging the whole partition.?
/sd




Jeff's advice of looking at FileLog to see what volumes require salvaging is
where I would start as well.  However, there is different behavior w/Fast
Restart, so that might not be of help (I've not seen the case you describe
in the testing I've done, so I'm not sure).

Thanks,
Steven

Re: [OpenAFS] Removing a backup volume

[Steven Jenkins]

On 6/25/07, Derrick Brashear <[EMAIL PROTECTED]> wrote:




On 6/25/07, Steven Jenkins <[EMAIL PROTECTED]> wrote:...


The root problem here is the underlying filesystem presumably offers poor
performance for deleting files, and the way to fix it is to use a filesystem
that doesn't. Deleting a volume is really deleting a tree of files and
directories, and it won't run any faster for OpenAFS than it does for
anything else.



Frank, let me ask some additional questions:

* What OS are you on?  (including distribution, release, etc)
* What is the underlying filesystem?  what features do you have enabled?  (
e.g., the output of dumpe2fs -h or equivalent on your system)

With that information, we might be able to help explain things more clearly
and completely.

Thanks,
Steven

Re: [OpenAFS] Removing a backup volume

[Steven Jenkins]

On 6/18/07, Frank Burkhardt <[EMAIL PROTECTED]> wrote:


Hi,

I'm currently removing a backup clone which belongs to a volume containing
~
55 GB in ~ 102000 files. 'vos status' shows a "DeleteVolume" transaction
which is running since 63 min now.

Is it supposed to take that long? I've seen this on all of our file
servers
- especially when performing clone operations (e.g. vos backup, vos
release). However: cpus are ~99.5% idle (cpu load is always ~1.0 ). The
fileserver removes the clone exclusively - noone else accesses content
from
the volume's volumegroup and it's the only volume on the server.

Is there a way to speed things up?



My understanding (as this problem was described to me last week) is that
this is 'normal' and 'expected'.  It's certainly not optimal.  There are no
simple suggestions I know of that will speed up these operations; however,
there have been some discussions around how to speed this up.

I may be wrong, however.

Steven

Re: [OpenAFS] Rebuild of crashed server

[Steven Jenkins]

On 6/14/07, David Sonenberg <[EMAIL PROTECTED]> wrote:


I have a openafs environment with 4 AFS servers, where the database is
replicated to all but one of the servers.  The server that houses the
volumes for the root of the directory has crashed.  The file system is
currently running fsck --rebuild-tree but I expect that it will not come
back.  I have recent dumps of all the volumes housed on that server, as
well as readonly copies on all the other servers.  My question is if I
need to recreate the dead server, what is the best way to go about it?



A lot depends on how you have set things up and what shape your backups and
RO's are in.  There are many, many ways to do a rescue of a dead server.  A
decent place to start is:

vos listvldb -server $deadserver to show you what the VLDB thinks is on the
server.  You can then use that and the backups you have to do vos restore's
for the RW volumes.  You can always vos dump from an RO and then vos restore
the RW from that.  If you don't have a clone, you'll have to restore from
backups.  If you don't have backups, your data is lost.

Once you have rebuilt the RW's, go ahead and do vos release's on the RW's
you've restored that have ROs.

At that point, compare the output of vos listvol vs vos listvldb for your
server -- they should be the same.

Note: _don't_ do a vos syncserv or vos syncvldb unless you know what you're
doing.

Good luck.

Steven

Re: [OpenAFS] cache manager sources

[Steven Jenkins]

On 6/5/07, Emanuele Di Pascale <[EMAIL PROTECTED]> wrote:


Hi, I'm new to AFS and to the mailing list. I need to design with a
certain degree of detail (pseudo-code and data structures included)
coherency policies for a cache manager quite similar to the AFS one, and I
was browsing the documentation and code of OpenAfs in search for
inspiration, but I eventually got lost due to the complexity of the project;
I was wondering if anyone could point me at the right direction either in
the source code (which files and modules implement the cache manager and
callbacks in OpenAfs?) or to a developer-oriented guide.
Thanks in advance to anyone willing to help :)





Have you looked at the specification in
http://www.openafs.org/cgi-bin/cvsweb.cgi/openafs/doc/pdf/fscm-ispec.pdf

It's fairly dated, but if this is for an academic project, it will be an
authoritative starting point.

You might want to also check out:

http://www.openafs.org/cgi-bin/cvsweb.cgi/openafs/doc/pdf/archov-doc.pdf

Steven

[Fwd: Re: SPAM-LOW: Re: SPAM-LOW: RE: [OpenAFS] Documetation for asetkey and aklog]

[Steven Jenkins]

Oops, sent privately to Ken by mistake.

Steve
--- Begin Message ---

Ken Hornstein wrote:
1. My message was information and an offer of help. It wasn't even a 
recommendation. If you don't want it, don't take it. You don't need to 
unload a bunch of attitude on me.


Sigh.  I apologize for that; the whole documentation mess has been something
I wished I could ignore, and the rant really wasn't directed at you; it
was just a general rant.


Thanks for saying so.

2. The theorem example was just that, an example. Would you be less 
annoyed if the roles were FAQ-specific stuff like 'question' and 
'answer'? With a little work, the subset of DocBook you'd have to know 
would be *tiny*, and the subset of LaTeX you'd have to know would be *zero*.


The problem I have is "tiny" is greater than "zero".  Even a tiny
amount of time is time I don't have.  This is completely aside from the
problem that I don't have DocBook on any of the systems I have here and
I'd have to install it (actually, okay, I was curious and I checked it
out; it's not clear to me from a few minutes of Google exactly _what_
DocBook consists of; is just XML schema?  If so, where do I get the
tools to process it?  _What_ tools do I need?)


In principle, you don't need anything. The file *you* maintain, could be 
as simple as:


Q: Why is AFS better than NFS?
A: Because we say so.

Q: Why should I believe you?
A: Because we run AFS.

I'm neglecting sectioning, etc. That's simple to deail with.

Something like that could be trivially converted to a DocBook instance 
in a few lines of scripting. (Maybe someone other than you could write 
that.)


It turns out that DocBook already has support for FAQ-like things. The 
output would look something like this:


[a few lines of boilerplate header stuff]
...

  

  Why is AFS better than NFS?


  Because we say so.

  
  

  Why should I believe you?


  Because we run AFS.

  
  ...

...

(I'm just typing this in off the top of my head.) There's also no reason 
you couldn't maintain the file in this format directly. It's not *that* 
much uglier than HTML.


To turn that into HTML, you need OpenJade and the DocBook stylesheets. 
It's a minor nuisance to get that installed, and that may tilt it for 
you. But it's not too bad. In return you get a nice indexed FAQ like 
this: http://www.dpawson.co.uk/docbook/reference.html/


Maybe you don't need hardcopy; for a FAQ, you probably don't. But 
someone who does could fairly easily translate that using XSLT or a 
simple script into LaTeX or whatever.



And while you say that the amount of DocBook that I need to know is
tiny, the real problem is that I have almost no experience with SGML or
XML, and I'd have to learn a bunch of stuff just to be able to GET
anywhere with it; we don't have any of the tools or experience in-house
here, and without that infrastructure I'd have to spend a ton of time
working on it to be able to make any progress with it.  Years ago
I had the time to spend a week messing around with a TeX installation
to get it working, but I don't anymore (okay, it may only take a day or
two, but again, don't have the time).


Understood. I'm just pointing out options. On the other hand, if you 
take an approach that others see as promising, they may be willing to help.


Steve

--- End Message ---

Re: SPAM-LOW: Re: SPAM-LOW: RE: [OpenAFS] Documetation for asetkey and aklog

[Steven Jenkins]

Ken Hornstein wrote:

This illustrates _exactly_ my feelings about documentation formats.
Basically, I don't give a shit about 90% of the worthless crap that
these systems do (yeah, I'm going to be writing a WHOLE LOT of theorems
in the Kerberos FAQ); what I want is a simple layout that looks
[remainder of rant deleted]


Three points:

1. My message was information and an offer of help. It wasn't even a 
recommendation. If you don't want it, don't take it. You don't need to 
unload a bunch of attitude on me.


2. The theorem example was just that, an example. Would you be less 
annoyed if the roles were FAQ-specific stuff like 'question' and 
'answer'? With a little work, the subset of DocBook you'd have to know 
would be *tiny*, and the subset of LaTeX you'd have to know would be *zero*.


3. The lesson that logical markup is better than visual has only been 
learned ten gazillion times. If you're happy with HTML markup, great. 
But it's not like this is an open question.


Steve

___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: SPAM-LOW: RE: [OpenAFS] Documetation for asetkey and aklog

[Steven Jenkins]

ted creedon wrote:

It would be advisable to investigate converting the IBM.htm directly into
docbook xml. The trouble with the Latex conversion is the reserved
characters (# . \ etc) in the htm don't map cleanly into Latex.

I suspect that docbook will never produce typeset pdf documents as
professional as Latex but the html should be acceptable. Knuth's Tex
algorithms are still the best.


I've done some work on using DocBook to make both web documents and 
high-quality PDFs. The approach I used was to put LaTeX hints into the 
DocBook elements using the 'role' attribute. Role is ignored by most 
downstream processing, but you can easily write a DocBook-to-LaTeX 
converter that uses these hints to invoke the right high-level LaTeX 
constructs.


For example, the closest thing in DocBook to a LaTeX theorem is a 
. If you mark up your document with role='theorem'>, the HTML converter does all it can, but the LaTeX 
converter makes it a theorem.


I have Ruby code if anyone wants it.

Steve

___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] OpenAFS on Gentoo

[Steven Jenkins]

Matthew Miller wrote:
On Mon, Mar 14, 2005 at 03:25:26PM -0800, Steven Jenkins wrote:
The good news is that the ebuild file for [the] version in Portage will 
probably work unmodified if you just copy it and change its name.
It does. I just did it.
Out of curiousity, what kernel version is this?
I'm running with OpenAFS 1.2.13 on 2.4.26-gentoo-r14, and 1.3.78 with 
2.6.10-gentoo-r6. I got the 1.3.77 ebuild from 
http://bugs.gentoo.org/show_bug.cgi?id=82075 and renamed it to get 
1.3.78. You have to insmod the module and run afsd manually (on 2.6.x), 
but it works.

In every case, this is client only. Can't say if the servers work.
Steve
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] OpenAFS on Gentoo

[Steven Jenkins]

Steven Jenkins wrote:
The good news is that the ebuild file for [the] version in Portage will 
probably work unmodified if you just copy it and change its name.
 
[instructions elided]

Let me know if it works.
It does. I just did it.
Steve
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] OpenAFS on Gentoo

[Steven Jenkins]

Danny wrote:
As I already explained in earlier posts, I'm having trouble setting up 
an AFS-cell. The cause of this trouble would be an outdated version of 
AFS. I'm working with version 1.2.10 while the latest version would be 
1.2.13. But as I mentioned in the title, I am working with the Gentoo 
distribution, and Gentoo works with portage. This means that I download 
and install all my programs with one command.
Now I've searched for the openafs packages, and come to the conclusion 
that the latest package of openafs is the 1.2.11 version.
Can someone tell me what I should do to get the latest version with 
Gentoo, or is this impossible?
Danny,
As someone mentioned earlier, support for OpenAFS in Gentoo is 
essentially useless.

The good news is that the ebuild file for version in Portage will 
probably work unmodified if you just copy it and change its name. But 
don't do that in /usr/portage.  Create /usr/local/portage (or wherever 
your make.conf says your PORTDIR_OVERLAY is. Create net-fs/openafs and 
copy everything from /usr/portage/net-fs/openafs there. Then you should 
be able to copy openafs-1.2.10-r1.ebuild to openafs-1.2.13.ebuild and 
type 'ebuild openafs-1.2.13.ebuild digest' and 'emerge openafs'. Note 
that this overlay will supersede Gentoo's release of 1.2.13, should it 
ever occur. If it does, you can just delete (or rename) your 1.2.13 ebuild.

Let me know if it works.
Steve
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info

[OpenAFS] Gentoo Linux looking for OpenAFS maintainers

[Steven Jenkins]

http://www.gentoo.org/news/en/gwn/20030721-newsletter.xml

There is a working ebuild for 1.2.9 (I'm running it), but it hasn't been 
committed to Gentoo CVS and marked stable.

I don't think I know enough about OpenAFS code to volunteer as a 
maintainer, but I can help out. As I say, it's running already, so major 
surgery isn't required.

Steve

___
OpenAFS-info mailing list
[EMAIL PROTECTED]
https://lists.openafs.org/mailman/listinfo/openafs-info

Re: [OpenAFS] OpenAFS on Linux 2.5.x

[Steven Jenkins]

Derrick J Brashear wrote:
Here's the thing: I can spend time fixing code, or I can spend time
arguing with people about patches to make it work now. I only have so much
time. This comes back to the original comment you thought was arrogant.
If I have fewer hours than I need to do everything I'd rather fix things
than argue with people about the changes we need to work at all. Your
point (I think it was your point) that no one else was doing it is true.
No one's volunteering, and I don't have money to hire an arguer;-)
I've been thinking about this.

There are some large organizations out there with serious and 
longstanding institutional commitments to AFS. A few that come to mind 
are MIT, Stanford, Michigan, CMU, Jet Propulsion Laboratory (where I 
work), CERN, Morgan Stanley. I know there are others.

I'm concerned about AFS not running on Linux 2.6. Other organizations 
may have the concern. Solving this problem will require writing code, 
but it will also require convincing the Linux kernel developers to 
incorporate the code. From previous discussion on this list, it seems 
that may not be easy.

It's too much to ask the OpenAFS developers to also be our user 
advocates. First of all, they don't have the time. But they also don't 
represent the user base in any official capacity, and maybe that's what 
we need to fix.

What do people think about forming an advocacy group? Maybe we'll make 
more headway with the Linux team if we ask them, as major organizational 
 users of AFS, to help ensure that Linux in particular and open source 
software in general continues to be responsive to our needs.

I'm sure I could arrange for some support for such an effort at JPL. I 
don't think it'll require much.

Comments? Suggestions?

Steve

___
OpenAFS-info mailing list
[EMAIL PROTECTED]
https://lists.openafs.org/mailman/listinfo/openafs-info