Re: r358252 causes intermittent hangs where processes are stuck sleeping on btalloc

2020-05-26 Thread Rick Macklem
Konstantin Belousov wrote: >On Fri, May 22, 2020 at 11:46:26PM +0000, Rick Macklem wrote: >> Konstantin Belousov wrote: >> >On Wed, May 20, 2020 at 11:58:50PM -0700, Ryan Libby wrote: >> >> On Wed, May 20, 2020 at 6:04 PM Rick Macklem wrote: >> >> &g

Re: r358252 causes intermittent hangs where processes are stuck sleeping on btalloc

2020-05-22 Thread Rick Macklem
Konstantin Belousov wrote: >On Wed, May 20, 2020 at 11:58:50PM -0700, Ryan Libby wrote: >> On Wed, May 20, 2020 at 6:04 PM Rick Macklem wrote: >> > >> > Hi, >> > >> > Since I hadn't upgraded a kernel through the winter, it took me a while >&g

Re: RFC: merging nfs-over-tls changes into head/sys

2020-05-22 Thread Rick Macklem
John Baldwin wrote: >On 5/21/20 2:01 PM, Rick Macklem wrote: >> Hi, >> >> I have now completed changes to the code in projects/nfs-over-tls, which >> implements TLS encryption of NFS RPC messages. (This roughly conforms >> to the internet draft "Towards Remot

RFC: merging nfs-over-tls changes into head/sys

2020-05-21 Thread Rick Macklem
Hi, I have now completed changes to the code in projects/nfs-over-tls, which implements TLS encryption of NFS RPC messages. (This roughly conforms to the internet draft "Towards Remote Procedure Call Encryption By Default", which should soon become an RFC. For now, TLS1.2 is used instead of TLS1.3

r358252 causes intermittent hangs where processes are stuck sleeping on btalloc

2020-05-20 Thread Rick Macklem
Hi, Since I hadn't upgraded a kernel through the winter, it took me a while to bisect this, but r358252 seems to be the culprit. If I do a kernel build over NFS using my not so big Pentium 4 (single core, 1.25Gbytes RAM, i386), about every second attempt will hang. When I do a "ps" in the debugge

Re: TLS certificates for NFS-over-TLS floating client

2020-03-28 Thread Rick Macklem
John-Mark Gurney wrote: >Rick Macklem wrote this message on Thu, Mar 26, 2020 at 14:33 +: >> John-Mark Gurney wrote: >> [lots of stuff snipped] >> >Rick Macklem wrote: >> >> I had originally planned on some "secret" in the certificate (like a CN

Re: TLS certificates for NFS-over-TLS floating client

2020-03-26 Thread Rick Macklem
org on behalf of Rick Macklem Sent: Thursday, March 26, 2020 10:33 AM To: John-Mark Gurney Cc: Alexander Leidinger; freebsd-current@FreeBSD.org Subject: Re: TLS certificates for NFS-over-TLS floating client John-Mark Gurney wrote: [lots of stuff snipped] >Rick Macklem wrote: >> I had or

Re: TLS certificates for NFS-over-TLS floating client

2020-03-26 Thread Rick Macklem
John-Mark Gurney wrote: [lots of stuff snipped] >Rick Macklem wrote: >> I had originally planned on some "secret" in the certificate (like a CN name >> that satisfies some regular expression or ???) but others convinced me that >> that wouldn't provide anything

Re: TLS certificates for NFS-over-TLS floating client

2020-03-25 Thread Rick Macklem
John-Mark Gurney wrote: >Rick Macklem wrote this message on Mon, Mar 23, 2020 at 23:53 +: >> Alexander Leidinger wrote: >> John-Mark Gurney wrote: >> >>Rick Macklem wrote: >> >>> to be the best solution. The server can verify that the certificat

Re: TLS certificates for NFS-over-TLS floating client

2020-03-23 Thread Rick Macklem
Alexander Leidinger wrote: John-Mark Gurney wrote: >>Rick Macklem wrote: >>> to be the best solution. The server can verify that the certificate  >>> was issued by >>> the local CA. Unfortunately, if the client is compromised and the  >>> certificate is

Re: TLS certificates for NFS-over-TLS floating client

2020-03-20 Thread Rick Macklem
Miroslav Lachman wrote: >Rick Macklem wrote on 2020/03/19 03:09: >> Miroslav Lachman wrote: >>> >> [...] > >>> NFS (or any other server) should check list of revoked certificates too. >>> Otherwise you will not be able to deny access to user which you

Re: TLS certificates for NFS-over-TLS floating client

2020-03-20 Thread Rick Macklem
Jan Bramkamp wrote: >On 20.03.20 02:44, Russell L. Carter wrote: >> Here I commit heresy, by A) top posting, and B) by just saying, why >> not make it easy, first, to tunnel NFSv4 sessions through >> e.g. net/wireguard or sysutils/spiped? NFS is point to point. >> Security infrastructure that actu

Re: TLS certificates for NFS-over-TLS floating client

2020-03-19 Thread Rick Macklem
John-Mark Gurney wrote: >Rick Macklem wrote this message on Wed, Mar 04, 2020 at 03:15 +: >> I am slowly trying to understand TLS certificates and am trying to figure >> out how to do the following: >> -> For an /etc/exports file with... >> /home -tls -network 1

Re: TLS certificates for NFS-over-TLS floating client

2020-03-18 Thread Rick Macklem
Miroslav Lachman wrote: >Hiroki Sato wrote on 2020/03/04 05:35: > [...] > >> I do not think it is a good idea to use a certificate with an >> embedded secret for authentication and/or authorization. >> >> In the case that the client offers a certificate upon establishing a >> TLS connection

Re: when does a server need to use SSL_CTX_set_client_CA_list()?

2020-03-16 Thread Rick Macklem
Alexander Leidinger wrote: >Quoting Rick Macklem (from Sun, 15 Mar 2020  >23:27:58 +): > >> As such, it stills seems to be a bit of a mystery to me, but it  >> seems that putting >> all the certificates in a CAfile and not using a CApath directory is  >> th

Re: when does a server need to use SSL_CTX_set_client_CA_list()?

2020-03-15 Thread Rick Macklem
Ronald Klop wrote: >On Sat, 14 Mar 2020 02:28:22 +0100, Rick Macklem >wrote: > >> Hi, >> >> Since it is done in sample code, I have an option in the RPC-over-TLS >> server daemon that does the SSL_CTX_set_client_CA_list() call. >> When I test, I have not

Re: when does a server need to use SSL_CTX_set_client_CA_list()?

2020-03-14 Thread Rick Macklem
Garrett Wollman wrote: >Rick Macklem writes: >>Since it is done in sample code, I have an option in the RPC-over-TLS >>server daemon that does the SSL_CTX_set_client_CA_list() call. >>When I test, I have not used this option and the code seems to work. >>Maybe this is be

when does a server need to use SSL_CTX_set_client_CA_list()?

2020-03-13 Thread Rick Macklem
Hi, Since it is done in sample code, I have an option in the RPC-over-TLS server daemon that does the SSL_CTX_set_client_CA_list() call. When I test, I have not used this option and the code seems to work. Maybe this is because the client only has a single certificate? Here's the lame description

Re: TLS certificates for NFS-over-TLS floating client

2020-03-05 Thread Rick Macklem
Rick Macklem wrote: >Benjamin Kaduk wrote: >>Rick Macklem wrote: [stuff snipped] >>> A typical client mounting from outside of the subnet might be my laptop, >>> which is using wifi and has no fixed IP/DNS name. >>> --> How do you create a certificate

Re: TLS certificates for NFS-over-TLS floating client

2020-03-05 Thread Rick Macklem
Benjamin Kaduk wrote: >On Wed, Mar 04, 2020 at 03:15:48AM +0000, Rick Macklem wrote: >> Hi, >> >> I am slowly trying to understand TLS certificates and am trying to figure >> out how to do the following: >> -> For an /etc/exports file with... >> /home -tl

TLS certificates for NFS-over-TLS floating client

2020-03-03 Thread Rick Macklem
Hi, I am slowly trying to understand TLS certificates and am trying to figure out how to do the following: -> For an /etc/exports file with... /home -tls -network 192.168.1.0 -mask 255.255.255.0 /home -tlscert This syntax isn't implemented yet, but the thinking is that clients on the 192.168.1 su

Re: how to use the ktls

2020-02-03 Thread Rick Macklem
Benjamin Kaduk wrote: >On Tue, Jan 28, 2020 at 11:01:31PM +0000, Rick Macklem wrote: >> John Baldwin wrote: >> [stuff snipped] >> >I don't know yet. :-/ With the TOE-based TLS I had been testing with, this >> >doesn't >> >happen because the NIC

Re: easy way to work around a lack of a direct map on i386

2020-01-31 Thread Rick Macklem
-freebsd-curr...@freebsd.org on behalf of Konstantin Belousov Sent: Friday, January 31, 2020 7:31 AM To: Hans Petter Selasky Cc: Rick Macklem; freebsd-current@FreeBSD.org Subject: Re: easy way to work around a lack of a direct map on i386 On Fri, Jan 31, 2020 at 10:13:58AM +0100, Hans Petter Selasky

easy way to work around a lack of a direct map on i386

2020-01-30 Thread Rick Macklem
Hi, The current code for KERN_TLS uses PHYS_TO_DMAP() to access unmapped external pages on m_ext.ext_pgs mbufs. I also need to do this to implement RPC-over-TLS. The problem is that some arches, like i386, don't support PHYS_TO_DMAP(). Since it appears that there will be at most 4 pages on one o

Re: how to use the ktls

2020-01-28 Thread Rick Macklem
John Baldwin wrote: [stuff snipped] >I don't know yet. :-/ With the TOE-based TLS I had been testing with, this >doesn't >happen because the NIC blocks the data until it gets the key and then it's >always >available via KTLS. With software-based KTLS for RX (which I'm going to start >working on

Re: how to use the ktls

2020-01-27 Thread Rick Macklem
John Baldwin wrote: >On 1/26/20 8:08 PM, Rick Macklem wrote: >> John Baldwin wrote: >> [stuff snipped] >>> Hmmm, this might be a fair bit of work indeed. >>> >>> Right now KTLS only works for transmit (though I have some WIP for receive). >>> &

Re: how to use the ktls

2020-01-26 Thread Rick Macklem
John Baldwin wrote: [stuff snipped] >Hmmm, this might be a fair bit of work indeed. > >Right now KTLS only works for transmit (though I have some WIP for receive). > >KTLS does assumes that the initial handshake and key negotiation is handled by >OpenSSL. OpenSSL uses custom setockopt() calls to t

Re: how to use the ktls

2020-01-13 Thread Rick Macklem
John Baldwin wrote: >On 1/12/20 8:23 PM, Benjamin Kaduk wrote: >> On Thu, Jan 09, 2020 at 10:53:38PM +0000, Rick Macklem wrote: >>> John Baldwin wrote: >>>> On 1/7/20 3:02 PM, Rick Macklem wrote: >>>>> Hi, >>>>> >>>&g

Re: how to use the ktls

2020-01-09 Thread Rick Macklem
John Baldwin wrote: >On 1/7/20 3:02 PM, Rick Macklem wrote: >> Hi, >> >> Now that I've completed NFSv4.2 I'm on to the next project, which is making >> NFS >> work over TLS. >> Of course, I know absolutely nothing about TLS, which will make thi

how to use the ktls

2020-01-07 Thread Rick Macklem
Hi, Now that I've completed NFSv4.2 I'm on to the next project, which is making NFS work over TLS. Of course, I know absolutely nothing about TLS, which will make this an interesting exercise for me. I did find simple server code in the OpenSSL doc. which at least gives me a starting point for t

Re: getting rid of sys/nfs/nfs_lock.c

2019-12-29 Thread Rick Macklem
Dennis Clarke wrote: >On 12/28/19 7:30 PM, Rick Macklem wrote: >> Hi, >> >> sys/nfs/nfs_lock.c uses Giant. Since it has not been used by default since >> March 2008, I suspect it can be removed from head without any impact. >> Post March 2008, the only way this

Re: getting rid of sys/nfs/nfs_lock.c

2019-12-28 Thread Rick Macklem
Oh, I forgot to mention that, post March 2008, this code was replaced by the in kernel nlm found in sys/nlm, which is why it has been in use. From: owner-freebsd-curr...@freebsd.org on behalf of Rick Macklem Sent: Saturday, December 28, 2019 7:30 PM To

getting rid of sys/nfs/nfs_lock.c

2019-12-28 Thread Rick Macklem
Hi, sys/nfs/nfs_lock.c uses Giant. Since it has not been used by default since March 2008, I suspect it can be removed from head without any impact. Post March 2008, the only way this code could be executed is by both building a kernel without "options NFSLOCKD" and deleting nfslockd.ko from the k

Heads up: Large patch that adds NFSv4.2 has been committed to head/current

2019-12-12 Thread Rick Macklem
Hi, r355677 is a large patch that adds NFSv4.2 support to the NFS client/server. It has survived a "make universe" for all arches that would build (some mips and sparc64 failed for reasons unrelated to this patch). However, I have not been able to do a build with a recent GCC. If there are build p

merge of NFSv4.2 support into head/current

2019-11-25 Thread Rick Macklem
Hi, I have completed development and a testing cycle for the NFSv4.2 (RFC-7862) code in base/projects/nfsv42 on subversion. NFSv4.2 is a minor revision to NFSv4.1 and adds support for the following optional features: - lseek(SEEK_DATA/SEEK_HOLE) - posix_fallocate() - posix_fadvise(POSIX_FADV_WIL

re: Reproducable deadlock in NFS client

2019-10-03 Thread Rick Macklem
Hi Peter, You could try a couple of things: 1 - kib@ just put a patch up on phabricator that reorganizes the handling of vnode_pager_setsize(). D21883 (If you could test this patch, that might be the best approach.) or 2 - The only differences between the post r352392 code and th

Re: RFC: should lseek(SEEK_DATA/SEEK_HOLE) return ENOTTY?

2019-08-16 Thread Rick Macklem
Ian Lepore wrote: >On Sun, 2019-08-11 at 09:12 -0600, Alan Somers wrote: >> On Sun, Aug 11, 2019 at 8:57 AM Ian Lepore wrote: >> > >> > On Sun, 2019-08-11 at 09:04 +0200, Gary Jennejohn wrote: >> > > On Sun, 11 Aug 2019 02:03:10 + >>

Re: RFC: should lseek(SEEK_DATA/SEEK_HOLE) return ENOTTY?

2019-08-11 Thread Rick Macklem
Ian Lepore wrote: >On Sun, 2019-08-11 at 09:12 -0600, Alan Somers wrote: >> On Sun, Aug 11, 2019 at 8:57 AM Ian Lepore wrote: >> > >> > On Sun, 2019-08-11 at 09:04 +0200, Gary Jennejohn wrote: >> > > On Sun, 11 Aug 2019 02:03:10 + >>

RFC: should lseek(SEEK_DATA/SEEK_HOLE) return ENOTTY?

2019-08-10 Thread Rick Macklem
Hi, I've noticed that, if you do a lseek(SEEK_DATA/SEEK_HOLE) on a file that resides in a file system that does not support holes, ENOTTY is returned. This error isn't listed for lseek() and seems a liitle weird. I can see a couple of alternatives to this: 1 - Return a different error. Maybe ENX

Re: should a copy_file_range(2) syscall be interrupted via a signal

2019-07-07 Thread Rick Macklem
Konstantin Belousov wrote: >On Fri, Jul 05, 2019 at 08:59:23PM +0000, Rick Macklem wrote: >> Konstantin Belousov wrote: >> >On Fri, Jul 05, 2019 at 07:30:54PM +0200, Jilles Tjoelker wrote: >> >> On Fri, Jul 05, 2019 at 12:28:51AM +, Rick Macklem wrote: >>

Re: test program for copy_file_range(2)

2019-07-05 Thread Rick Macklem
Alan Somers wrote: >On Fri, Jul 5, 2019 at 9:11 AM Rick Macklem wrote: >> >> Alan Somers wrote: >> >On Thu, Jul 4, 2019 at 6:38 PM Rick Macklem wrote: >> >> >> >> I have a little program for testing the copy_file_range(2) syscall I've

Re: should a copy_file_range(2) syscall be interrupted via a signal

2019-07-05 Thread Rick Macklem
Konstantin Belousov wrote: >On Fri, Jul 05, 2019 at 07:30:54PM +0200, Jilles Tjoelker wrote: >> On Fri, Jul 05, 2019 at 12:28:51AM +0000, Rick Macklem wrote: >> > I have been working on a Linux compatible copy_file_range(2) syscall >> > (the current code can be found at

Re: should a copy_file_range(2) syscall be interrupted via a signal

2019-07-05 Thread Rick Macklem
Hans Petter Selasky wrote: >On 2019-07-05 02:28, Rick Macklem wrote: >> I am thinking that copy_file_range(2) should do this also. >> However, if it returns an error, it is impossible for the caller to know how >> much >> of the data range got copied. > >Ho

Re: should a copy_file_range(2) syscall be interrupted via a signal

2019-07-05 Thread Rick Macklem
Mark Johnston wrote: >On Fri, Jul 05, 2019 at 12:28:51AM +0000, Rick Macklem wrote: >> Hi, >> >> I have been working on a Linux compatible copy_file_range(2) syscall >> (the current code can be found at https://reviews.freebsd.org/D20584). >> >> One outst

Re: [Differential] D20584: add a linux compatible copy_file_range(2) syscall

2019-07-05 Thread Rick Macklem
jilles wrote in copy_file_range.2:99 > The Linux man page (from > http://man7.org/linux/man->pages/man2/copy_file_range.2.html ) says that a > non-zero flags argument will cause >the call to return an [EINVAL] error. I > think that is better than ignoring the argument >completely since it allows

Re: test program for copy_file_range(2)

2019-07-05 Thread Rick Macklem
Alan Somers wrote: >On Thu, Jul 4, 2019 at 6:38 PM Rick Macklem wrote: >> >> I have a little program for testing the copy_file_range(2) syscall I've been >> working on. (The current version is attached, in case anyone is interested.) >> >> It take a few mi

test program for copy_file_range(2)

2019-07-04 Thread Rick Macklem
I have a little program for testing the copy_file_range(2) syscall I've been working on. (The current version is attached, in case anyone is interested.) It take a few minutes to run on a slow system and uses about 6Gbytes of disk space for the file system the output file is on. (It creates 2 file

should a copy_file_range(2) syscall be interrupted via a signal

2019-07-04 Thread Rick Macklem
Hi, I have been working on a Linux compatible copy_file_range(2) syscall (the current code can be found at https://reviews.freebsd.org/D20584). One outstanding issue is how it should deal with signals. Right now, I have vn_start_write() without PCATCH, so that it won't be interrupted by a signal,

patch to add a Linux compatible copy_file_range(2) syscall

2019-06-09 Thread Rick Macklem
Hi, I just put a patch in phabricator that is intended to add a Linux compatible copy_file_range(2) syscall. My main interest in having this is that NFSv4.2 will know how to do file copying locally on the NFS server, saving all the reads/writes across the wire. It copies the file byte range in t

Re: adding a syscall to libc?

2019-06-08 Thread Rick Macklem
Konstantin Belousov wrote: >On Sat, Jun 08, 2019 at 02:57:27AM +0000, Rick Macklem wrote: >> Hi, >> First off, thanks Kostik for the fine explanation. I agree with Oliver that it should be captured somewhere like the wiki. I'm no wiki guy, so hopefully someone else will do th

adding a syscall to libc?

2019-06-07 Thread Rick Macklem
Hi, I've started working of a copy_file_range() syscall for FreeBSD. I think I have the kernel patched and ready for some testing. However, I'm confused about what I need to do in src/lib/libc/sys? - Some syscalls have little .c files, but other ones do not. When is one of these little .c files

Re: RFC w.r.t. toggling debugging on/off for mountd via a signal

2019-05-19 Thread Rick Macklem
Cy Schubert wrote: [lots of stuff snipped] >Instead of syslog() calls, DTrace probes are designed for this type of >instrumentation. DTrace us way too obscure for me. Never used it, probably never will. (Remember I'm the guy who still uses "ed" to edit a

Re: RFC w.r.t. toggling debugging on/off for mountd via a signal

2019-05-18 Thread Rick Macklem
Alan Somers wrote: >On Sat, May 18, 2019 at 7:59 PM Rick Macklem wrote: >> >> Hi, >> >> I've been working with Peter Errikson on a patch for mountd that adds a new >> option >> for incremental updating of exports. This seems to be helping a lot w.r

RFC w.r.t. toggling debugging on/off for mountd via a signal

2019-05-18 Thread Rick Macklem
Hi, I've been working with Peter Errikson on a patch for mountd that adds a new option for incremental updating of exports. This seems to be helping a lot w.r.t. performance on an NFS server with lots (1+) of exported file systems. I have debug syslog() calls in the code, which I/Peter thin

patch that replaces a single linked list with a hash table of lists (mountd.c) for review

2019-05-15 Thread Rick Macklem
Hi, I just put a patch for mountd.c in phabricator as D20270, which replaces the single linked list of structures for exported file systems with a hash table of lists. This is part of what I hope will fix the performance of mountd when reloading the exports file(s) for a server with a lot of expo

Re: Do the pidhashtbl_locks added by r340742 need to be sx locks?

2019-04-10 Thread Rick Macklem
Mateusz Guzik wrote: >On 4/11/19, Rick Macklem wrote: >> Hi, >> >> I finally got around to looking at what effect replacing pfind_locked() >> with >> pfind() has for the NFSv4 client and it is broken. >> >> The problem is that the NFS code nee

Do the pidhashtbl_locks added by r340742 need to be sx locks?

2019-04-10 Thread Rick Macklem
Hi, I finally got around to looking at what effect replacing pfind_locked() with pfind() has for the NFSv4 client and it is broken. The problem is that the NFS code needs to call some variant of "pfind()" while holding a mutex lock. The current _pfind() code uses the pidhashtbl_locks, which are "

Re: what do jails map 127.0.0.1 to?

2019-02-16 Thread Rick Macklem
Rodney W. Grimes wrote: [stuff snipped] > ipv4 127.0.0.1 == ipv6 ::1, see /etc/hosts Thanks. I've created D19218 with the patch for nfsuserd.c to both check the mapping of localhost and adding support for IPv6. I've listed bz@ as a reviewer, but if anyone else would like to review it, feel to do

Re: what do jails map 127.0.0.1 to?

2019-02-12 Thread Rick Macklem
Bjoern A. Zeeb wrote: [good stuff snipped] >Yes, could do easily but wouldn’t work for my above case, would it? I >can help you with the code for v4 and jails if you help me with the code >for IPv6? Well, not quite what you volunteered to do, but can you fill in the INET6 code for this code snippe

Re: what do jails map 127.0.0.1 to?

2019-02-12 Thread Rick Macklem
Bjoern A. Zeeb wrote: >On 11 Feb 2019, at 0:50, Rick Macklem wrote: >> I am finally back to looking at an old PR#205193. >> >> The problem is that the nfsuserd daemon expects upcalls from the >> kernel >> that are from localhost (127.0.0.1) and when jails are runni

what do jails map 127.0.0.1 to?

2019-02-10 Thread Rick Macklem
I am finally back to looking at an old PR#205193. The problem is that the nfsuserd daemon expects upcalls from the kernel that are from localhost (127.0.0.1) and when jails are running on the system, 127.0.0.1 is mapped to some other IP#. (I think it might be the address of the first net interface

possible POLA violation for NFS server to make it Linux compatible

2018-12-18 Thread Rick Macklem
It was my understanding that the NFSv4 working group believed that the requirement for the NFSv4 client to use a priviledged port# (< 1024) should not exist. As such, I coded the server to ignore the vfs.nfsd.nfs_privport sysctl and allow the mount for NFSv4. PR#234106 has reported this as a com

Re: which way to update export_args structure?

2018-10-22 Thread Rick Macklem
From: owner-freebsd-curr...@freebsd.org on behalf of Rick Macklem Sent: Monday, October 22, 2018 5:24 PM To: Brooks Davis Cc: FreeBSD Current; Josh Paetzel Subject: Re: which way to update export_args structure? Brooks Davis wrote: On Sat, Oct 20

Re: which way to update export_args structure?

2018-10-22 Thread Rick Macklem
Brooks Davis wrote: On Sat, Oct 20, 2018 at 01:17:37AM +, Rick Macklem wrote: [lots of stuff snipped] >> + if (error == 0) { >> + gotexp = 0; >> + memset(&export, 0, sizeof(export)); >> + if (vfs_getopt(mp->mnt_

Re: which way to update export_args structure?

2018-10-19 Thread Rick Macklem
Brooks Davis wrote: > Yes, I think that's the right way foward. Thanks for following up. >Rick Macklem wrote: >> Just in case you missed it in the email thread, in your general question >> below... >> Did you mean/suggest that the fields of "struct export

Re: which way to update export_args structure?

2018-10-03 Thread Rick Macklem
Brooks Davis wrote: >On Wed, Oct 03, 2018 at 12:40:27AM +0000, Rick Macklem wrote: >> Hi, >> >> I am working on updating "struct export_args" to fix/add a few things. >> One of these is that "ex_flags" is an int, but the flags are defined in >

Re: which way to update export_args structure?

2018-10-03 Thread Rick Macklem
Gary Jennejohn wrote: [stuff snipped] >In B, shouldn't ex_flags become uint32_t if all 32 bits can contain >flag bits? You could. For B) my intent was to leave the structure exactly the same as the old versions and only add new fields at the end. The current compatibility code in head does bcopy(

Re: which way to update export_args structure?

2018-10-02 Thread Rick Macklem
I wrote: >Hi, > >I am working on updating "struct export_args" to fix/add a few things. >One of these is that "ex_flags" is an int, but the flags are defined in mount.h >as MNT_xx bits that now exceed 32bits (mnt_flag is now uint64_t). >For now, this doesn't break anything, since the flags used by

which way to update export_args structure?

2018-10-02 Thread Rick Macklem
Hi, I am working on updating "struct export_args" to fix/add a few things. One of these is that "ex_flags" is an int, but the flags are defined in mount.h as MNT_xx bits that now exceed 32bits (mnt_flag is now uint64_t). For now, this doesn't break anything, since the flags used by ex_flags are al

Heads up: nfsuserd -use-udpsock option no longer used

2018-08-22 Thread Rick Macklem
As of r338192 the command line option "-use-udpsock" no longer exists, since r320757 has been reverted. The behaviour is now back to always using a UDP socket the same as stable/11. In other words, if you are using the nfsuserd with the "-use-udpsock" option, just get rid of the "-use-udpsock" op

Re: panic excl->shared for an AF_LOCAL socket

2018-08-21 Thread Rick Macklem
Matthew Macy wrote: [stuff snipped] >I don't know what's special in this case, but I did revamp the locking there >several >months back so I'll take a look next weekend. Thanks but don't worry about it for now. I think I figured out how the panic() occurred. If the nfsd was accessing /var/run/nfs

panic excl->shared for an AF_LOCAL socket

2018-08-19 Thread Rick Macklem
Hi, PR#230752 shows a witness panic() for "excl->shared" on a vnode lock. In this case, the kernel RPC code is trying to do a soconnect() on an AF_LOCAL socket. The code is unp_connectat() looks like is does a straightforward namei()/lookup(), so I am surprised to see this. Does anyone know if AF

Re: ffs_truncate3 panics

2018-08-17 Thread Rick Macklem
Just fyi, I have committed r337962 to head, which stops the pNFS service from creating non-zero length empty files. Since these files were the ones causing the "ffs_truncate3" panics, the panics should no longer occur. I will take a closer look at some point to see if I can spot why the panics() o

Re: kernel build failure

2018-08-13 Thread Rick Macklem
Rodney W. Grimes wrote: >> On Sun, 12 Aug 2018 14:39-0700, Matthew Macy wrote: >> >> > Sorry guys, last time I touched ZFS I tried to push to make it an option to >> > statically link and was actually told that it wasn't something anyone else >> > wanted. The issue comes from ZFS not being in NOTES

Re: ffs_truncate3 panics

2018-08-12 Thread Rick Macklem
Konstantin Belousov wrote: [stuff snipped] >Problem with this buffer is that BX_ALTDATA bit is not set. >This is the reason why vinvalbuf(V_ALT) skips it. [more stuff snipped] >The vnode is exclusively locked. Other thread must not be able to >instantiate a buffer under us. That's what I thought,

Re: ffs_truncate3 panics

2018-08-11 Thread Rick Macklem
Konstantin Belousov wrote: >On Thu, Aug 09, 2018 at 08:38:50PM +0000, Rick Macklem wrote: >> >BTW, does NFS server use extended attributes ? What for ? Can you, please, >> >point out the code which does this ? >> For the pNFS service, there are two system namesp

Re: ffs_truncate3 panics

2018-08-10 Thread Rick Macklem
Konstantin Belousov wrote: >On Thu, Aug 09, 2018 at 08:38:50PM +0000, Rick Macklem wrote: >> I did notice that my code locks the vnode first and then calls >> vn_start_write() >> for the vn_extattr_set() calls, whereas the syscall code locks the vnode >> afte

Re: ffs_truncate3 panics

2018-08-09 Thread Rick Macklem
Konstantin Belousov wrote: [stuff snipped] >I wrote: >> >> I can add printf()s anywhere you suggest, but I'm not sure how you would >> catch >> this case sooner? (For example, I could print out di_extsize at the >> beginning of >> ffs_truncate(), if that would help?) >May be, add a loop at the be

Re: ffs_truncate3 panics

2018-08-09 Thread Rick Macklem
Rodney W. Grimes wrote: [stuff snipped] >It should be possible to design a set of VM's using bhyve, xen or ones >favorite hypervisor/virtulization platform to do "more" pNFS testing. >If you could provide a rought machine set needed to have a functional >test bed, and what should be done to "test"

Re: ffs_truncate3 panics

2018-08-08 Thread Rick Macklem
Konstantin Belousov wrote: [stuff snipped] >> >Can you print the only buffer on the clean queue when the panic occur ? >> ffst3 vtyp=1 bodirty=0 boclean=1 >> buf at 0x428a110 >> b_flags = 0x20001020, b_xflags=0x2, b_vflags=0x0 >> b_error = 0, b_bufsize = 4096, b_bcount = 4096, b_resid = 0 >> b_bufo

Re: ffs_truncate3 panics

2018-08-08 Thread Rick Macklem
Konstantin Belousov wrote: >On Tue, Aug 07, 2018 at 12:28:33PM +0000, Rick Macklem wrote: >> Hi, >> >> During testing of the pNFS server I get an ffs_truncate3 panic every once in >> a while. >> A few things that might be relevant: >> - Seems to happen mo

ffs_truncate3 panics

2018-08-07 Thread Rick Macklem
Hi, During testing of the pNFS server I get an ffs_truncate3 panic every once in a while. A few things that might be relevant: - Seems to happen more often when soft update journaling is enabled, but will happen when it is disabled. - Normally happens when a fairly large subtree of the file sys

Re: NFSv4.1 server deficiencies fixed for ESXi client

2018-07-15 Thread Rick Macklem
I wrote: > (I am going to look and see what the Linux server does for this case.) I just looked and the Linux 4.17-rc2 kernel NFS server just returns NFS_OK for the rca_one_fs == TRUE case. I have given a patch that does the same thing for the FreeBSD server to Andreas and Daniel and think it is ok

Re: NFSv4.1 server deficiencies fixed for ESXi client

2018-07-15 Thread Rick Macklem
Rodney W. Grimes wrote: >Have you any contact with VMWare so that they might fix the issues >in thier code, rather than having to put hacks in FreeBSD for these >issues? Well, Jim White (who is not in their file system area) submitted a PR on their system and the response was along the lines of: S

Heads Up: rebuild nfsd if using the "-p" option

2018-07-02 Thread Rick Macklem
r335870 changes the interface between the nfsd and the kernel for the case where the "-p" option is used, so if you are using "nfsd -p" you should rebuild your nfsd from sources of r335871 or later. To be honest, I didn't see problems during testing, but I think that is because there are some byte

change of nfsd->kernel interface in head

2018-06-30 Thread Rick Macklem
r335012 (the big patch that added the pNFS server support) revised the nfsd->kernel nfssvc(2) syscall interface. It has compatibility code, so that old nfsd binaries still work. I now need to revise this interface again to add a new pNFS server feature. Since the revised interface is only in head

Re: nfsd kernel threads won't die via SIGKILL

2018-06-26 Thread Rick Macklem
Konstantin Belousov wrote: On Mon, Jun 25, 2018 at 02:04:32AM +, Rick Macklem wrote: > Konstantin Belousov wrote: > >On Sat, Jun 23, 2018 at 09:03:02PM +0000, Rick Macklem wrote: > >> During testing of the pNFS server I have been frequently > >> killing/restarti

Re: nfsd kernel threads won't die via SIGKILL

2018-06-24 Thread Rick Macklem
Konstantin Belousov wrote: >On Sat, Jun 23, 2018 at 09:03:02PM +0000, Rick Macklem wrote: >> During testing of the pNFS server I have been frequently killing/restarting >> the nfsd. >> Once in a while, the "slave" nfsd process doesn't terminate and a "p

nfsd kernel threads won't die via SIGKILL

2018-06-23 Thread Rick Macklem
During testing of the pNFS server I have been frequently killing/restarting the nfsd. Once in a while, the "slave" nfsd process doesn't terminate and a "ps axHl" shows: 0 48889 1 0 20 0 5884 812 svcexit D - 0:00.01 nfsd: server 0 48889 1 0 40 0 5884 812 rpcsvc

Re: head -r335568 breaks the ci.freebsd.org builds of the various FreeBSD-head-*-build

2018-06-22 Thread Rick Macklem
Mark Millard wrote: >from the likes of: > >https://ci.freebsd.org/job/FreeBSD-head-sparc64-build/8346/consoleText > >--- nfs_commonkrpc.o --- >/usr/src/sys/fs/nfs/nfs_commonkrpc.c: In function 'newnfs_request': >/usr/src/sys/fs/nfs/nfs_commonkrpc.c:813: error: 'ND_HASSLOTID' undeclared >(first >us

Re: ESXi NFSv4.1 client id is nasty

2018-06-19 Thread Rick Macklem
Steve Wills wrote: On 06/18/18 17:42, Rick Macklem wrote: >> Steve Wills wrote: >>> Would it be possible or reasonable to use the client ID to log a message >>> telling the admin to enable a sysctl to enable the hacks? >> Yes. However, this client implementation

Re: review of nfsd rc.d script patch

2018-06-19 Thread Rick Macklem
Don Lewis wrote: >On 15 Jun, Rick Macklem wrote: >> Hi, >> >> For the pNFS service MDS machine, the nfsd can't be started until all nfs >> mounts >> in /etc/fstab are done. >> I think that adding "mountcritremote" to the "# REQUIRE:"

Re: ESXi NFSv4.1 client id is nasty

2018-06-18 Thread Rick Macklem
essage and fail the mount, if the "hack" sysctl isn't set? rick [stuff snipped] From: Steve Wills Sent: Monday, June 18, 2018 5:21:10 PM To: Rick Macklem; freebsd-current@freebsd.org Cc: andreas.n...@frequentis.com Subject: Re: ESXi NFSv4

RFC: ESXi client is nasty, what should I do?

2018-06-18 Thread Rick Macklem
Hi, I realized that the subject line "ESXi NFSv4.1 client id is nasty" wouldn't have indicated that I was looking for comments w.r.t. how to handle this poorly behaved client. Please go to the "ESXi NFSv4.1 client id is nasty" thread and comment. (It should be in the archive, if you already delet

Re: NFSv4.1 server deficiencies fixed for ESXi client

2018-06-17 Thread Rick Macklem
Rodney W. Grimes wrote: [stuff snipped] >Have you any contact with VMWare so that they might fix the issues >in their code, rather than having to put hacks in FreeBSD for these >issues? Nope. I tried an email to nf...@ietf.org about one issue (that isn't yet resolved related to it complaining that

NFSv4.1 server deficiencies fixed for ESXi client

2018-06-17 Thread Rick Macklem
Hi, Since I posted w.r.t. issues that seem to violate the RFC, I figured I should also post ones that identified deficiencies in the FreeBSD server. These have either been patched in head/current or will be soon and will be MFC'd. - BindConnectiontoSession wasn't implemented. It is never used by

ESXi NFSv4.1 client id is nasty

2018-06-17 Thread Rick Macklem
Hi, Andreas Nagy has been doing a lot of testing of the NFSv4.1 client in ESXi 6.5u1 (VMware) against the FreeBSD server. I have given him a bunch of hackish patches to try and some of them do help. However not all issues are resolved. The problem is that these hacks pretty obviously violate the N

review of nfsd rc.d script patch

2018-06-15 Thread Rick Macklem
Hi, For the pNFS service MDS machine, the nfsd can't be started until all nfs mounts in /etc/fstab are done. I think that adding "mountcritremote" to the "# REQUIRE:" line is sufficient to do this? I don't think delaying the startup of the nfsd daemon until after any NFS mounts are done will do

Re: utilities in /usr/bin or /usr/sbin?

2018-06-14 Thread Rick Macklem
Rodney W. Grimes wrote: ?> On Thu, Jun 14, 2018 at 2:15 PM, Rick Macklem wrote: >> >> > Hi, >> > >> > I have three new utilities that are mainly useful for managing the pNFS >> > server >> > committed as r335130. >> > >> >

Re: utilities in /usr/bin or /usr/sbin?

2018-06-14 Thread Rick Macklem
I wrote: >I have three new utilities that are mainly useful for managing the pNFS server >committed as r335130. Oops, I meant r334930, although it doesn't really affect the question. >In the projects tree, I have them in /usr/bin and man section 1. However, >since they are mostly useful to a sysad

<    1   2   3   4   5   6   7   8   >