Hi Udo, how long have you been on vanilla Osol 2009.06 and how long on SRU u6 before updating to SRU u7? Is it possible that SRU u6 has the issue as well? I was suspicious about one integration which went to SRUu2 and to S10U8 and to snv_114, but the change is only on the nfs client side - this does not match the set-up described earlier by Jorgen: - clients are s10u5 - server/s10u8 (issue) - server/snv_117 (no issue)
I am not able to find out more from the available data - unless the problem is reproducible it is hard to diagnose. I have made one observation (not sure if it is useful) reopen with CLAIM_PREVIOUS (CT=P) fails with NFS4ERR_NO_GRACE, but reopen with CLAIM_NULL (CT=N) succeeds with NFS4_OK Pavel On 03/11/10 14:16, Udo Grabowski wrote: > Hi Pavel, > > both clients and server were updated (we always have a consistent > environment), and we came > from u6, to which we updated directly from a vanilla Osol 2009.06 before. The > Readmes don't > list any NFS patches there, so I suspect that our IDR30 patch carries > unwanted changes from > Solaris u8 into Opensolaris U7 which trigger this problem. Since we don't use > Solaris 10, I cannot > confirm that 10u7 did not have that problem, I just concluded it from the > initial post here (but > that conclusion maybe wrong, I admit). > We currently snoop the problem and catched some clues (maybe): Shortly > before, we > see CB_NULL and NULL4 exchanges, seemingly as a result of a client renewal > (we do not > catch everything before), both server and client seem to check their partners > callback capabilities: > > imksunxxx -> imksunyyy NFS C 4 (setclientid ) PUTROOTFH GETATTR 400 0 > SETCLIENTID Prog=1073741826 ID=tcp Addr=192.168.2.151.156.34 CBID=1073741... > imksunyyy -> imksunxxx NFS R 4 (setclientid ) NFS4_OK PUTROOTFH NFS4_OK > GETATTR NFS4_OK SETCLIENTID NFS4_OK CL=3144b6920ee CFV=00002AE100000314 > imksunxxx -> imksunyyy NFS C 4 (sclntid_conf) SETCLIENTID_CONFIRM > CL=3144b6920ee CFV=00002AE100000314 > imksunyyy -> imksunxxx NFS R 4 (sclntid_conf) NFS4_OK > SETCLIENTID_CONFIRM NFS4_OK > imksunyyy -> imksunxxx NFS C CB_NULL > imksunxxx -> imksunyyy NFS R CB_NULL > imksunxxx -> imksunyyy TCP D=2049 S=772 Ack=2315081869 Seq=1684524332 > Len=0 Win=64074 Options=<nop,nop,tstamp 449054982 188906496> > imksunyyy -> imksunxxx TCP D=39970 S=46007 Ack=2765004476 > Seq=2079880454 Len=0 Win=64074 Options=<nop,nop,tstamp 188906503 449054975> > imksunxxx -> imksunyyy TCP D=2049 S=35143 Syn Seq=3666179320 Len=0 > Win=64057 Options=<mss 1460,nop,nop,tstamp 449055009 0,nop,wscale > 4,nop,nop,sackOK> > imksunyyy -> imksunxxx TCP D=35143 S=2049 Syn Ack=3666179321 > Seq=3064768973 Len=0 Win=64074 Options=<nop,nop,tstamp 188906530 > 449055009,mss 1460,nop,wscale 4,nop,nop,sackOK> > imksunxxx -> imksunyyy TCP D=2049 S=35143 Ack=3064768974 Seq=3666179321 > Len=0 Win=64074 Options=<nop,nop,tstamp 449055009 188906530> > imksunxxx -> imksunyyy NFS C NULL4 > imksunyyy -> imksunxxx TCP D=35143 S=2049 Ack=3666179365 Seq=3064768974 > Len=0 Win=64074 Options=<nop,nop,tstamp 188906530 449055009> > imksunyyy -> imksunxxx NFS R NULL4 > imksunxxx -> imksunyyy TCP D=2049 S=35143 Ack=3064769002 Seq=3666179365 > Len=0 Win=64074 Options=<nop,nop,tstamp 449055009 188906530> > imksunxxx -> imksunyyy TCP D=2049 S=35143 Fin Ack=3064769002 > Seq=3666179365 Len=0 Win=64074 Options=<nop,nop,tstamp 449055009 188906530> > imksunyyy -> imksunxxx TCP D=35143 S=2049 Ack=3666179366 Seq=3064769002 > Len=0 Win=64074 Options=<nop,nop,tstamp 188906530 449055009> > imksunyyy -> imksunxxx TCP D=35143 S=2049 Fin Ack=3666179366 > Seq=3064769002 Len=0 Win=64074 Options=<nop,nop,tstamp 188906530 449055009> > imksunxxx -> imksunyyy TCP D=2049 S=35143 Ack=3064769003 Seq=3666179366 > Len=0 Win=64074 Options=<nop,nop,tstamp 449055009 188906530> > ****** here starts the reopen mess, clients hang ********** > imksunxxx -> imksunyyy NFS C 4 (reopen ) PUTFH FH=92CA OPEN OT=NC > SQ=21955 CT=P DT=N AC=R DN=N OO=0670 GETFH GETATTR 10011a b0a23a > imksunyyy -> imksunxxx NFS R 4 (reopen ) NFS4ERR_NO_GRACE PUTFH > NFS4_OK OPEN NFS4ERR_NO_GRACE > imksunxxx -> imksunyyy NFS C 4 (reopen ) PUTFH FH=95AC OPEN > start_version OT=NC SQ=21956 CT=N AC=R DN=N OO=0670 GETFH GETATTR 10011a > b0a23a > imksunyyy -> imksunxxx NFS R 4 (reopen ) NFS4_OK PUTFH NFS4_OK > OPEN NFS4_OK ST=150F:10979 RF=PL DT=N GETFH NFS4_OK FH=92CA GETATTR NFS4_OK > imksunxxx -> imksunyyy NFS C 4 (reopen ) PUTFH FH=92CA OPEN OT=NC > SQ=21955 CT=P DT=N AC=R DN=N OO=07D1 GETFH GETATTR 10011a b0a23a > imksunyyy -> imksunxxx NFS R 4 (reopen ) NFS4ERR_NO_GRACE PUTFH > NFS4_OK OPEN NFS4ERR_NO_GRACE > > Another one here, this time after the reopen: > > imksunzzz -> local-yyy NFS C 4 (reopen ) PUTFH FH=92CA OPEN OT=NC > SQ=42131 CT=P DT=N AC=R DN=N OO=065A GETFH GETATTR 10011a b0a23a > local-yyy -> imksunzzz NFS R 4 (reopen ) NFS4ERR_NO_GRACE PUTFH > NFS4_OK OPEN NFS4ERR_NO_GRACE > imksunzzz -> local-yyy NFS C 4 (reopen ) PUTFH FH=95AC OPEN > start_version OT=NC SQ=42132 CT=N AC=R DN=N OO=065A GETFH GETATTR 10011a > b0a23a > local-yyy -> imksunzzz NFS R 4 (reopen ) NFS4_OK PUTFH NFS4_OK > OPEN NFS4_OK ST=1B1B:21067 RF=PL DT=N GETFH NFS4_OK FH=92CA GETATTR NFS4_OK > imksunzzz -> local-yyy NFS C 4 (open ) PUTFH FH=95AC OPEN > start_version OT=NC SQ=1 CT=N AC=R DN=N OO=0F54 GETFH GETATTR 10011a b0a23a > local-yyy -> imksunzzz NFS R 4 (open ) NFS4ERR_STALE_CLIENTID > PUTFH NFS4_OK OPEN NFS4ERR_STALE_CLIENTID > imksunzzz -> local-yyy NFS C 4 (setclientid ) PUTROOTFH GETATTR 400 0 > SETCLIENTID Prog=1073741826 ID=tcp Addr=192.168.2.157.145.153 CBID=107374... > local-yyy -> imksunzzz NFS R 4 (setclientid ) NFS4_OK PUTROOTFH NFS4_OK > GETATTR NFS4_OK SETCLIENTID NFS4_OK CL=3154b6920ee CFV=0000524A00000315 > imksunzzz -> local-yyy NFS C 4 (sclntid_conf) SETCLIENTID_CONFIRM > CL=3154b6920ee CFV=0000524A00000315 > local-yyy -> imksunzzz NFS R 4 (sclntid_conf) NFS4_OK > SETCLIENTID_CONFIRM NFS4_OK > local-yyy -> imksunzzz NFS C CB_NULL > imksunzzz -> local-yyy NFS R CB_NULL > > and a third one with a couple of NFS4ERR_STALE_CLIENTID: > > imksunwww -> local-uuu NFS C 4 (setclientid ) PUTROOTFH GETATTR 400 0 > SETCLIENTID Prog=1073741825 ID=tcp Addr=192.168.2.158.211.129 CBID=107374... > local-uuu -> imksunwww NFS R 4 (setclientid ) NFS4_OK PUTROOTFH NFS4_OK > GETATTR NFS4_OK SETCLIENTID NFS4_OK CL=784b2b88bf CFV=000017B600000078 > imksunwww -> local-uuu NFS C 4 (sclntid_conf) SETCLIENTID_CONFIRM > CL=784b2b88bf CFV=000017B600000078 > local-uuu -> imksunwww NFS R 4 (sclntid_conf) NFS4_OK > SETCLIENTID_CONFIRM NFS4_OK > local-uuu -> imksunwww NFS C CB_NULL > imksunwww -> local-uuu NFS R CB_NULL > imksunwww -> local-uuu TCP D=2049 S=1014 Ack=2856458498 Seq=2503417192 > Len=0 Win=64074 Options=<nop,nop,tstamp 419650675 417669232> > local-uuu -> imksunwww TCP D=54145 S=44819 Ack=696198650 Seq=3514023629 > Len=0 Win=64074 Options=<nop,nop,tstamp 417669239 419650669> > imksunwww -> local-uuu NFS C 4 (reopen ) PUTFH FH=D915 OPEN OT=NC > SQ=12381 CT=P DT=N AC=R DN=N OO=0053 GETFH GETATTR 10011a b0a23a > local-uuu -> imksunwww NFS R 4 (reopen ) NFS4ERR_NO_GRACE PUTFH > NFS4_OK OPEN NFS4ERR_NO_GRACE > imksunwww -> local-uuu NFS C 4 (reopen ) PUTFH FH=CDFC OPEN > sge_shepherd OT=NC SQ=12382 CT=N AC=R DN=N OO=0053 GETFH GETATTR 10011a > b0a23a > local-uuu -> imksunwww NFS R 4 (reopen ) NFS4_OK PUTFH NFS4_OK > OPEN NFS4_OK ST=1B1B:6070 RF=PL DT=N GETFH NFS4_OK FH=D915 GETATTR NFS4_OK > imksunwww -> local-uuu NFS C 4 (reopen ) PUTFH FH=D8C0 OPEN OT=NC > SQ=12163 CT=P DT=N AC=R DN=N OO=005A GETFH GETATTR 10011a b0a23a > local-uuu -> imksunwww NFS R 4 (reopen ) NFS4ERR_NO_GRACE PUTFH > NFS4_OK OPEN NFS4ERR_NO_GRACE > imksunwww -> local-uuu NFS C 4 (reopen ) PUTFH FH=CDFC OPEN > sge_execd OT=NC SQ=12164 CT=N AC=R DN=N OO=005A GETFH GETATTR 10011a b0a23a > local-uuu -> imksunwww NFS R 4 (reopen ) NFS4_OK PUTFH NFS4_OK > OPEN NFS4_OK ST=1583:6071 RF=PL DT=N GETFH NFS4_OK FH=D8C0 GETATTR NFS4_OK > imksunwww -> local-uuu NFS C 4 (open ) PUTFH FH=C7A4 OPEN > settings.sh OT=NC SQ=1 CT=N AC=R DN=N OO=0C0D GETFH GETATTR 10011a b0a23a > imksunwww -> local-uuu NFS C 4 (open ) PUTFH FH=C7A4 OPEN > settings.sh OT=NC SQ=1 CT=N AC=R DN=N OO=0C5C GETFH GETATTR 10011a b0a23a > imksunwww -> local-uuu NFS C 4 (open ) PUTFH FH=C7A4 OPEN > settings.sh OT=NC SQ=1 CT=N AC=R DN=N OO=0C1B GETFH GETATTR 10011a b0a23a > imksunwww -> local-uuu NFS C 4 (open ) PUTFH FH=C7A4 OPEN > settings.sh OT=NC SQ=1 CT=N AC=R DN=N OO=0C69 GETFH GETATTR 10011a b0a23a > imksunwww -> local-uuu NFS C 4 (open ) PUTFH FH=C7A4 OPEN > settings.sh OT=NC SQ=1 CT=N AC=R DN=N OO=0FF5 GETFH GETATTR 10011a b0a23a > imksunwww -> local-uuu NFS C 4 (open ) PUTFH FH=C7A4 OPEN > settings.sh OT=NC SQ=1 CT=N AC=R DN=N OO=0C70 GETFH GETATTR 10011a b0a23a > imksunwww -> local-uuu NFS C 4 (open ) PUTFH FH=C7A4 OPEN > settings.sh OT=NC SQ=1 CT=N AC=R DN=N OO=0CAA GETFH GETATTR 10011a b0a23a > imksunwww -> local-uuu NFS C 4 (open ) PUTFH FH=C7A4 OPEN > settings.sh OT=NC SQ=1 CT=N AC=R DN=N OO=0FE3 GETFH GETATTR 10011a b0a23a > local-uuu -> imksunwww TCP D=1014 S=2049 Ack=2503418560 Seq=2856459314 > Len=0 Win=64074 Options=<nop,nop,tstamp 417669242 419650679> > local-uuu -> imksunwww TCP D=1014 S=2049 Ack=2503419056 Seq=2856459314 > Len=0 Win=64074 Options=<nop,nop,tstamp 417669242 419650679> > local-uuu -> imksunwww TCP D=1014 S=2049 Ack=2503419552 Seq=2856459314 > Len=0 Win=64074 Options=<nop,nop,tstamp 417669242 419650679> > local-uuu -> imksunwww TCP D=1014 S=2049 Ack=2503420048 Seq=2856459314 > Len=0 Win=64074 Options=<nop,nop,tstamp 417669242 419650679> > local-uuu -> imksunwww NFS R 4 (open ) NFS4ERR_STALE_CLIENTID > PUTFH NFS4_OK OPEN NFS4ERR_STALE_CLIENTID > local-uuu -> imksunwww NFS R 4 (open ) NFS4ERR_STALE_CLIENTID > PUTFH NFS4_OK OPEN NFS4ERR_STALE_CLIENTID > local-uuu -> imksunwww NFS R 4 (open ) NFS4ERR_STALE_CLIENTID > PUTFH NFS4_OK OPEN NFS4ERR_STALE_CLIENTID > local-uuu -> imksunwww NFS R 4 (open ) NFS4ERR_STALE_CLIENTID > PUTFH NFS4_OK OPEN NFS4ERR_STALE_CLIENTID > local-uuu -> imksunwww NFS R 4 (open ) NFS4ERR_STALE_CLIENTID > PUTFH NFS4_OK OPEN NFS4ERR_STALE_CLIENTID > imksunwww -> local-uuu TCP D=2049 S=1014 Ack=2856459450 Seq=2503420048 > Len=0 Win=64074 Options=<nop,nop,tstamp 419650680 417669242> > local-uuu -> imksunwww NFS R 4 (open ) NFS4ERR_STALE_CLIENTID > PUTFH NFS4_OK OPEN NFS4ERR_STALE_CLIENTID > imksunwww -> local-uuu TCP D=2049 S=1014 Ack=2856459586 Seq=2503420048 > Len=0 Win=64074 Options=<nop,nop,tstamp 419650680 417669242> > local-uuu -> imksunwww NFS R 4 (open ) NFS4ERR_STALE_CLIENTID > PUTFH NFS4_OK OPEN NFS4ERR_STALE_CLIENTID > imksunwww -> local-uuu TCP D=2049 S=1014 Ack=2856459722 Seq=2503420048 > Len=0 Win=64074 Options=<nop,nop,tstamp 419650680 417669242> > local-uuu -> imksunwww NFS R 4 (open ) NFS4ERR_STALE_CLIENTID > PUTFH NFS4_OK OPEN NFS4ERR_STALE_CLIENTID > imksunwww -> local-uuu TCP D=2049 S=1014 Ack=2856459858 Seq=2503420048 > Len=0 Win=64074 Options=<nop,nop,tstamp 419650680 417669242> >
