Re: [OpenAFS] jafs et al
On Tue, 13 Mar 2007, Marcus Watts wrote: I think for openafs, it would make sense to have a enable-pic configure flag - that could turn on pic mode globally. That could be used to The CCOBJ rule is designed for just that; you just need the configure glue. ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] jafs et al
Yes. What I started with is the java support in 1.5.15, which already includes all 3 of your patches. Okay, I made jafs working on openafs-1.4.0 + java-1.4.2, and unfortunately haven't got timeframe to maintain it, I just let you know my exp. I have already found older google messages - they were helpful in terms of understanding my more stupid mistakes, like -DSystem.loadLibrary=. I haven't seen anything that suggests anybody is actually using jafs for anything real. We were using it for admin purposes. Following is the patch I worked up to build on amd64_linux + sun jdk 1.5: /afs/umich.edu/group/itd/build/mdw/tmp/afs-m40-java.patch the start explains what changed. This is probably almost clean enough to submit as a patch to RT -- except I'm not convinced it actually works or would be of use. Note this doesn't address kaserver See also testAFS.java, it should run without error with a basic afs setup. (covers mainly those funcs I needed that time) vs. k5 at all. It does further address 64-bit problems, and also I've never tried k5 stuff. one linking/threading issue. Your patch turned on pic code for (nearly) all of openafs. This isn't necessarily ideal, nor is it (mostly) necessary. There's a performance loss in statically linked binaries that might bother people running servers built that way. The patch I have above turns pic code for libuafs libadmin, and builds pic static libraries for libafsrpc libafsauthent. This is probably not ideal either, but at least it's different. IIRC I had problems on platforms like ppc[64] and s390x with pic stuff (one crashes runtime if you don't turn on pic, other didn't allow mixing pic and non-pic or something like that). So please modify _only_ that platform which you are testing. Peter ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] jafs et al
On Tue, 13 Mar 2007, Peter Somogyi wrote: IIRC I had problems on platforms like ppc[64] and s390x with pic stuff (one crashes runtime if you don't turn on pic, other didn't allow mixing pic and non-pic or something like that). So please modify _only_ that platform which you are testing. So, why not just not mix pic and non pic and still only build what you need pic? ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] jafs et al
So, why not just not mix pic and non pic and still only build what you need pic? Okay, I agree. (Nowadays I won't have access for those platforms to test it - but you can proceed.) Peter ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] jafs et al
Derrick J Brashear [EMAIL PROTECTED] writes: On Tue, 13 Mar 2007, Peter Somogyi wrote: IIRC I had problems on platforms like ppc[64] and s390x with pic stuff (one crashes runtime if you don't turn on pic, other didn't allow mixing pic and non-pic or something like that). So please modify _only_ that platform which you are testing. So, why not just not mix pic and non pic and still only build what you need pic? It's worth noting that the AFS Perl module has the same problem, and this is the reason why it's not yet packaged for Debian. All the libraries it links against would need to be built PIC, and unfortunately that includes various libraries that are not part of libafsauthent or libafsrpc. I've looked a couple of times at what would be involved in building the right things PIC, and I think it's doable, but it's annoying. And I'm not sure if it's enough stuff for worries about performance problems to arise. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] jafs et al
Russ Allbery [EMAIL PROTECTED] had replied: Derrick J Brashear [EMAIL PROTECTED] wrote: On Tue, 13 Mar 2007, Peter Somogyi wrote: IIRC I had problems on platforms like ppc[64] and s390x with pic stuff (one crashes runtime if you don't turn on pic, other didn't allow mixing pic and non-pic or something like that). So please modify _only_ that platform which you are testing. So, why not just not mix pic and non pic and still only build what you need pic? It's worth noting that the AFS Perl module has the same problem, and this is the reason why it's not yet packaged for Debian. All the libraries it links against would need to be built PIC, and unfortunately that includes various libraries that are not part of libafsauthent or libafsrpc. I've looked a couple of times at what would be involved in building the right things PIC, and I think it's doable, but it's annoying. And I'm not sure if it's enough stuff for worries about performance problems to arise. I don't think there's any one best answer. There's actually several dimensions here: pic vs. nonpic lwp vs. pthread user vs kernel mode vs. user kernel mode although these aren't strictly independent. Perl has its own special brand of weirdness, especially in regards to threading, tossing mit kerberos 5 and glibc into the mix can produce fascinating but bad behavior. In some environments (many?) - linking against pthread would be very attractive, but not on linux. In some environments (many?) - mixing pic and non-pic code is kosher, but apparently not ibm big-endian iron. I think for openafs, it would make sense to have a enable-pic configure flag - that could turn on pic mode globally. That could be used to build static libraries for pam, perl whatever as one relatively clean approach, without compromising server performance. On some architectures (pretty much anything that has sun inspired shared libraries) it's easy and good to build libXXX_pic.a files along-side the .so.X shared library. It's all the same .o files in the library build directory, just bound a different way. For perl pam, I think it would be nice to have options to turn pthreads on or off, in those packages. That would make it easier to select the appropriate choice for the local environment. -Marcus Watts ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] jafs et al
On Tuesday, March 13, 2007 08:07:42 PM -0500 Marcus Watts [EMAIL PROTECTED] wrote: user vs kernel mode vs. user kernel mode Actually, we don't really have this dimension. No libraries are built for kernel-mode code; any code the kernel module requires from the rest of the tree is built separately and linked directly into the kernel module. The situation is similar for libuafs, though the other dimensions certainly exist for that library as a whole. Personally, I'd like to see a consistent set of libraries available in all six forms (lwp/pthread x shared/pic/nonpic). However, note that for many of our libraries, building pthread versions is more complex than just changing a few compiler switches -- there is still a lot of code in AFS which assumes that context switches can only happen at very specific points. -- Jeff ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] jafs et al
Jeffrey Hutzelman [EMAIL PROTECTED] writes: To: Marcus Watts [EMAIL PROTECTED], openafs-info@openafs.org cc: Jeffrey Hutzelman [EMAIL PROTECTED] Subject: Re: [OpenAFS] jafs et al On Tuesday, March 13, 2007 08:07:42 PM -0500 Marcus Watts [EMAIL PROTECTED] wrote: user vs kernel mode vs. user kernel mode Actually, we don't really have this dimension. No libraries are built for kernel-mode code; any code the kernel module requires from the rest of the tree is built separately and linked directly into the kernel module. The situation is similar for libuafs, though the other dimensions certainly exist for that library as a whole. Personally, I'd like to see a consistent set of libraries available in all six forms (lwp/pthread x shared/pic/nonpic). However, note that for many of our libraries, building pthread versions is more complex than just changing a few compiler switches -- there is still a lot of code in AFS which assumes that context switches can only happen at very specific points. -- Jeff Kernel mode code isn't part of a general purpose programming environment, but rather is part of just another application. But, this distinction could get less meaningful with time - with kerberos 5 kernel support we're moving into a world where library style api support makes more more sense. Both Solaris Linux appear to be wrestling with the implications of identifying which kernel mode environment features to support, in decidedly different ways. While in Unix kernel modules usually their own idiosyncratic toolset, in MicroSoft, they're just dll's. Even though kernel mode should be part of a different programming environment, I've found it useful to link test applications against selected bits of code built for the kernel to verify proper operation. I suppose though, it would be more fair to consider kernel mode as another sort of threading option rather than its own separate thing. That leaves libuafs as a much more anomalous case. Code-wise that's a fair call. But libuafs still has special include file and library organization, so I'm not sure we want to argue this isn't a special case. Yup, non-preemptive code won't work well with pthreads. Or for that matter with fine-grained kernel locks on modern SMP. More nits along this line include the server side of ubik (globals), and libvolser (trusted code runs on the client.) Oh, and kdump... -Marcus ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] jafs et al
Marcus Watts [EMAIL PROTECTED] writes: I don't think there's any one best answer. There's actually several dimensions here: pic vs. nonpic lwp vs. pthread user vs kernel mode vs. user kernel mode although these aren't strictly independent. Perl has its own special brand of weirdness, especially in regards to threading, tossing mit kerberos 5 and glibc into the mix can produce fascinating but bad behavior. In some environments (many?) - linking against pthread would be very attractive, but not on linux. Er, linking against pthread is very attractive to me on Linux. What am I missing? In some environments (many?) - mixing pic and non-pic code is kosher, but apparently not ibm big-endian iron. Mixing PIC and non-PIC code is not kosher in general. There just happens to be a few interesting exceptions where you can get away with it, including x86 Linux. You can't mix PIC and non-PIC on amd64 Linux, let alone weirder stuff. I think for openafs, it would make sense to have a enable-pic configure flag - that could turn on pic mode globally. That could be used to build static libraries for pam, perl whatever as one relatively clean approach, without compromising server performance. Well, that I could do now. But building AFS twice and then extracting libraries from the second build is really obnoxious from a packaging perspective, because now you also have to chase every package that depends on AFS libraries and get them to update what libraries they're linking with and tack _pic on to the end, or you have to have two directories of AFS libraries, or you have to install server binaries built with different libraries than you ship in the dev package. It's all doable and all ugly. The clean approach from a Linux perspective is to have a shared library with a versioned API that you can link against. It saves overhead, reduces memory usage, and makes packaging far cleaner. It's just a lot more work getting it right and keeping it right. But I'm going to keep banging the drum until hopefully we can find time to do it. :) -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] jafs et al
From: Peter Somogyi [EMAIL PROTECTED] Reply-To: [EMAIL PROTECTED] Organization: Gamax Kft To: openafs-info@openafs.org Subject: Re: [OpenAFS] jafs et al Date: Mon, 12 Mar 2007 14:43:06 +0100 User-Agent: KMail/1.9.1 Cc: Marcus Watts [EMAIL PROTECTED] References: [EMAIL PROTECTED] In-Reply-To: [EMAIL PROTECTED] MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: [EMAIL PROTECTED] how many people use jafs, jafsadm? (=lib/libjafsadm.so, jafs.jar, org.openafs.*, etc.) what do you use it for? what version of java do you use? what version of openafs? which hardware architecture(s)? do you use kaserver, kerberos 5, ? if you use kerberos 5 with java elsewhere, which java k5 library? do you have any sample code or applications that use this in interesting ways? See also http://rt.central.org/rt/Ticket/Display.html?id=21930 + search for jafs by content with query builder in rt to see its history + google on this list Peter Yes. What I started with is the java support in 1.5.15, which already includes all 3 of your patches. It is good to know the history of this -- the file src/libuafs/README that appears to go with this is only willing to talk about openafs 1.2.[69] and redhat linux 7.3, far older than your work. I have already found older google messages - they were helpful in terms of understanding my more stupid mistakes, like -DSystem.loadLibrary=. I haven't seen anything that suggests anybody is actually using jafs for anything real. Following is the patch I worked up to build on amd64_linux + sun jdk 1.5: /afs/umich.edu/group/itd/build/mdw/tmp/afs-m40-java.patch the start explains what changed. This is probably almost clean enough to submit as a patch to RT -- except I'm not convinced it actually works or would be of use. Note this doesn't address kaserver vs. k5 at all. It does further address 64-bit problems, and also one linking/threading issue. Your patch turned on pic code for (nearly) all of openafs. This isn't necessarily ideal, nor is it (mostly) necessary. There's a performance loss in statically linked binaries that might bother people running servers built that way. The patch I have above turns pic code for libuafs libadmin, and builds pic static libraries for libafsrpc libafsauthent. This is probably not ideal either, but at least it's different. -Marcus Watts ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
[OpenAFS] jafs et al
In looking at libadmin jafs, I'm wondering just how much attention this code has received. libadmin appears to have possible kaserver dependencies, but at least it builds. Libjafs appears to have not received much recent attention at all. I expect to end up with a variety of improvements to at least the documentation, and probably the build procedure for the java part especially. I already have some changes based on getting this to build for amd64_linux + jdk 1.5. I'm also considering teaching these pieces more about rxk5. But before I dig much further into this, I have some questions: how many people use libadmin? what do you use it for? with kaserver? with kerberos 5? with kadm5?? do you have any sample code or applications that use this in interesting ways? how many people use jafs, jafsadm? (=lib/libjafsadm.so, jafs.jar, org.openafs.*, etc.) what do you use it for? what version of java do you use? what version of openafs? which hardware architecture(s)? do you use kaserver, kerberos 5, ? if you use kerberos 5 with java elsewhere, which java k5 library? do you have any sample code or applications that use this in interesting ways? If you tried libadmin or jafs, but gave up -- why? Do you use these related libraries: libuafs or libjuafs or nsafs.so (webafs) in any other interesting way? If there anything else in this general area of stuff that you expect or wish to have, or that I ought to know? [ Mind you, I'm not going to promise anything, yet. ] -Marcus Watts ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
