Re: [Gluster-devel] NetBSD regression tests not Initializing...
On Tue, Jul 07, 2015 at 06:04:44PM +0200, Niels de Vos wrote: On Tue, Jul 07, 2015 at 07:13:53PM +0530, Kaushal M wrote: I've taken this slave and one other offline and am rebooting it. Reminder that you do not need to take teh system offline for rebooting. I normally follow these steps to get hung systems back functional: 1. verify stuck job, NFS unmount related? 2. open http://build.gluster.org/view/Infra/job/reboot-vm/build 3. login on Jenkins 4. start the reboot-vm job for the stuck system 5. wait until the job finished 6. click the abort [x] link on the stuck job 7. retrigger the job after aborting has been done (reload page) These hangs do not seem to happen on tests from the master branch anymore, only on release-3.7. I think this is a confirmation that the reference counting for auth-cache structures in gluster/nfs is a working solution. We should backport these changes: - nfs: add a gf_lock_t for the auth_cache-cache_dict http://review.gluster.org/11021 - core: add gf_ref_t for common refcounting structures http://review.gluster.org/11022 (already done through http://review.gluster.org/11421) - nfs: refcount each auth_cache_entry and related data_t http://review.gluster.org/11023 - refcount: correct the documentation http://review.gluster.org/11328 I'll try to send backports later this week (maybe Thursday?), unless someone else beats me to it. Please reply to this thread if you file a bug for this and send some backports. The above backports have been posted. These should prevent the Gluster/NFS crashes in the regression tests, and therefor prevent the hanging of NetBSD on unmounting NFS (when the NFS-server died). Please check these patches, and merge them when ready: http://review.gluster.org/#/q/status:open+project:glusterfs+branch:release-3.7+topic:bug-1242515 Thanks, Niels Thanks, Niels On Tue, Jul 7, 2015 at 6:44 PM, Kotresh Hiremath Ravishankar khire...@redhat.com wrote: Hi Emmanuel, We are seeing these issues again on nbslave7h.cloud.gluster.org http://build.gluster.org/job/rackspace-netbsd7-regression-triggered/7974/console Thanks and Regards, Kotresh H R - Original Message - From: Emmanuel Dreyfus m...@netbsd.org To: Kotresh Hiremath Ravishankar khire...@redhat.com, Gluster Devel gluster-devel@gluster.org Sent: Sunday, July 5, 2015 12:52:23 AM Subject: Re: [Gluster-devel] NetBSD regression tests not Initializing... Kotresh Hiremath Ravishankar khire...@redhat.com wrote: Any help is appreciated. nbslave72 was sick indeed: it refused SSH connexions. I rebooted it and retiggered your change, but it went on another machine. -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] NetBSD regression tests not Initializing...
NetBSD tests arefailing again: http://build.gluster.org/job/rackspace-netbsd7-regression-triggered/8123/console Triggered by Gerrit:http://review.gluster.org/11616 in silent mode. Building remotely onnbslave74.cloud.gluster.org http://build.gluster.org/computer/nbslave74.cloud.gluster.org (netbsd7_regression) in workspace /home/jenkins/root/workspace/rackspace-netbsd7-regression-triggered git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository git config remote.origin.urlhttp://review.gluster.org/glusterfs.git # timeout=10 Fetching upstream changes fromhttp://review.gluster.org/glusterfs.git git --version # timeout=10 git -c core.askpass=true fetch --tags --progresshttp://review.gluster.org/glusterfs.git refs/changes/16/11616/1 ERROR: Error fetching remote repo 'origin' ERROR http://stacktrace.jenkins-ci.org/search?query=ERROR: Error fetching remote repo 'origin' Finished http://stacktrace.jenkins-ci.org/search?query=Finished: FAILURE Thanks, Vijay On Tuesday 07 July 2015 07:13 PM, Kaushal M wrote: I've taken this slave and one other offline and am rebooting it. On Tue, Jul 7, 2015 at 6:44 PM, Kotresh Hiremath Ravishankar khire...@redhat.com wrote: Hi Emmanuel, We are seeing these issues again on nbslave7h.cloud.gluster.org http://build.gluster.org/job/rackspace-netbsd7-regression-triggered/7974/console Thanks and Regards, Kotresh H R - Original Message - From: Emmanuel Dreyfus m...@netbsd.org To: Kotresh Hiremath Ravishankar khire...@redhat.com, Gluster Devel gluster-devel@gluster.org Sent: Sunday, July 5, 2015 12:52:23 AM Subject: Re: [Gluster-devel] NetBSD regression tests not Initializing... Kotresh Hiremath Ravishankar khire...@redhat.com wrote: Any help is appreciated. nbslave72 was sick indeed: it refused SSH connexions. I rebooted it and retiggered your change, but it went on another machine. -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] NetBSD regression tests not Initializing...
Vijaikumar M vmall...@redhat.com wrote: NetBSD tests arefailing again: (...) ERROR: Error fetching remote repo 'origin' Please reboot it. I amstill working on the infamous NFS unmount kernel bug, I hope the NetBSD slaves will behave better with the fix. -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] NetBSD regression tests not Initializing...
I've taken this slave and one other offline and am rebooting it. On Tue, Jul 7, 2015 at 6:44 PM, Kotresh Hiremath Ravishankar khire...@redhat.com wrote: Hi Emmanuel, We are seeing these issues again on nbslave7h.cloud.gluster.org http://build.gluster.org/job/rackspace-netbsd7-regression-triggered/7974/console Thanks and Regards, Kotresh H R - Original Message - From: Emmanuel Dreyfus m...@netbsd.org To: Kotresh Hiremath Ravishankar khire...@redhat.com, Gluster Devel gluster-devel@gluster.org Sent: Sunday, July 5, 2015 12:52:23 AM Subject: Re: [Gluster-devel] NetBSD regression tests not Initializing... Kotresh Hiremath Ravishankar khire...@redhat.com wrote: Any help is appreciated. nbslave72 was sick indeed: it refused SSH connexions. I rebooted it and retiggered your change, but it went on another machine. -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] NetBSD regression tests not Initializing...
Thanks Emmanuel. Thanks and Regards, Kotresh H R - Original Message - From: Emmanuel Dreyfus m...@netbsd.org To: Kotresh Hiremath Ravishankar khire...@redhat.com, Gluster Devel gluster-devel@gluster.org Sent: Sunday, July 5, 2015 12:52:23 AM Subject: Re: [Gluster-devel] NetBSD regression tests not Initializing... Kotresh Hiremath Ravishankar khire...@redhat.com wrote: Any help is appreciated. nbslave72 was sick indeed: it refused SSH connexions. I rebooted it and retiggered your change, but it went on another machine. -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] NetBSD regression tests not Initializing...
Kotresh Hiremath Ravishankar khire...@redhat.com wrote: Any help is appreciated. nbslave72 was sick indeed: it refused SSH connexions. I rebooted it and retiggered your change, but it went on another machine. -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] NetBSD regression tests not Initializing...
Hi NetBSD regressions are not initializing because of following error consistently with multiple re-triggers. I see the same error for quite a few patches. http://review.gluster.org/#/c/11443/ Building remotely on nbslave72.cloud.gluster.org (netbsd7_regression) in workspace /home/jenkins/root/workspace/rackspace-netbsd7-regression-triggered git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository git config remote.origin.url http://review.gluster.org/glusterfs.git # timeout=10 Fetching upstream changes from http://review.gluster.org/glusterfs.git git --version # timeout=10 git -c core.askpass=true fetch --tags --progress http://review.gluster.org/glusterfs.git refs/changes/43/11443/9 ERROR: Error fetching remote repo 'origin' ERROR: Error fetching remote repo 'origin' Finished: FAILURE Any help is appreciated. Thanks and Regards, Kotresh H R ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
