Re: [Gluster-infra] Possible Gluster outage? (was Re: [Gluster-devel] tests/bitrot/bug-1373520.t is failing multiple times)
It just didn't work for an hour or so. I didn't look into what exactly was the problem. The page was not even loading. On Mon, Jan 30, 2017 at 7:48 PM, Nigel Babu wrote: > Hi Pranith, > > What problems did you see? > * Did you have an Apache error? > * Did you have a possible DNS issue? > > On Sat, Jan 28, 2017 at 3:56 PM, Pranith Kumar Karampuri < > pkara...@redhat.com> wrote: > >> It is a bug in EC name heal code path. I sent a fix but >> review.gluster.org is not accessible now to paste the link here. Will >> send a mail again once it is accessible. >> > > -- > nigelb > -- Pranith ___ Gluster-infra mailing list Gluster-infra@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-infra
[Gluster-infra] [Bug 1416953] Request gerrit integration for gluster/ test github repository
https://bugzilla.redhat.com/show_bug.cgi?id=1416953 --- Comment #2 from Nigel Babu --- Okay so replication is setup and working. I've also cleared the cache, so now it should mostly work. I'm fixing up apache issues now -- You are receiving this mail because: You are on the CC list for the bug. Unsubscribe from this bug https://bugzilla.redhat.com/token.cgi?t=sJ3VZcVaOt&a=cc_unsubscribe ___ Gluster-infra mailing list Gluster-infra@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-infra
[Gluster-infra] [Bug 1416953] Request gerrit integration for gluster/ test github repository
https://bugzilla.redhat.com/show_bug.cgi?id=1416953 --- Comment #3 from Nigel Babu --- And Apache issues are now fixed up. -- You are receiving this mail because: You are on the CC list for the bug. Unsubscribe from this bug https://bugzilla.redhat.com/token.cgi?t=CqrllqYWI3&a=cc_unsubscribe ___ Gluster-infra mailing list Gluster-infra@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-infra
Re: [Gluster-infra] Possible Gluster outage? (was Re: [Gluster-devel] tests/bitrot/bug-1373520.t is failing multiple times)
Le mercredi 01 février 2017 à 14:11 +0530, Pranith Kumar Karampuri a écrit : > It just didn't work for an hour or so. I didn't look into what exactly was > the problem. The page was not even loading. I heard of issue on another server in the same location from people in india. What internet provider were you using ? What time did it start and stop ? Can you do a traceroute next time ? > On Mon, Jan 30, 2017 at 7:48 PM, Nigel Babu wrote: > > > Hi Pranith, > > > > What problems did you see? > > * Did you have an Apache error? > > * Did you have a possible DNS issue? > > > > On Sat, Jan 28, 2017 at 3:56 PM, Pranith Kumar Karampuri < > > pkara...@redhat.com> wrote: > > > >> It is a bug in EC name heal code path. I sent a fix but > >> review.gluster.org is not accessible now to paste the link here. Will > >> send a mail again once it is accessible. > >> > > > > -- > > nigelb > > > > > > ___ > Gluster-infra mailing list > Gluster-infra@gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-infra -- Michael Scherer Sysadmin, Community Infrastructure and Platform, OSAS signature.asc Description: This is a digitally signed message part ___ Gluster-infra mailing list Gluster-infra@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-infra
[Gluster-infra] [Bug 1418369] New: Need way to revert commits or mark tests bad without testing
https://bugzilla.redhat.com/show_bug.cgi?id=1418369 Bug ID: 1418369 Summary: Need way to revert commits or mark tests bad without testing Product: GlusterFS Version: mainline Component: project-infrastructure Assignee: b...@gluster.org Reporter: jda...@redhat.com CC: b...@gluster.org, gluster-infra@gluster.org As discussed in today's community meeting, we need a way to "dig ourselves out of the hole" quickly when work is blocked by sporadically failing tests. Specifically, there should be a way to exempt patches from testing which (a) revert a previous commit or (b) mark a test as bad. Gerrit has permissions that allow specified users to bypass tests by pushing to refs/heads/xxx instead of refs/for/xxx. This permission should be enabled for project architects (and probably not for others). If we can further extend or limit this ability by recognizing reverts and test markings for what they are, that's great but not immediately necessary. -- You are receiving this mail because: You are on the CC list for the bug. Unsubscribe from this bug https://bugzilla.redhat.com/token.cgi?t=Pegl66jblI&a=cc_unsubscribe ___ Gluster-infra mailing list Gluster-infra@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-infra
[Gluster-infra] [Bug 1418542] New: Multiple smoke failures observed since yesterday
https://bugzilla.redhat.com/show_bug.cgi?id=1418542 Bug ID: 1418542 Summary: Multiple smoke failures observed since yesterday Product: GlusterFS Version: mainline Component: project-infrastructure Assignee: b...@gluster.org Reporter: amukh...@redhat.com CC: b...@gluster.org, gluster-infra@gluster.org Description of problem: https://build.gluster.org/job/devrpm-el7/3055/console 07:13:33 Triggered by Gerrit: https://review.gluster.org/16502 07:13:33 Building remotely on slave25.cloud.gluster.org (smoke_tests rackspace_regression_2gb glusterfs-devrpms) in workspace /home/jenkins/root/workspace/devrpm-el7 07:13:33 Wiping out workspace first. 07:13:34 java.io.IOException: remote file operation failed: /home/jenkins/root/workspace/devrpm-el7 at hudson.remoting.Channel@148e532:slave25.cloud.gluster.org: java.nio.file.AccessDeniedException: /home/jenkins/root/workspace/devrpm-el7/RPMS/el7/x86_64/build.log 07:13:34 at hudson.FilePath.act(FilePath.java:986) 07:13:34 at hudson.FilePath.act(FilePath.java:968) 07:13:34 at hudson.FilePath.deleteContents(FilePath.java:1183) 07:13:34 at hudson.plugins.git.extensions.impl.WipeWorkspace.beforeCheckout(WipeWorkspace.java:28) 07:13:34 at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1094) 07:13:34 at hudson.scm.SCM.checkout(SCM.java:485) 07:13:34 at hudson.model.AbstractProject.checkout(AbstractProject.java:1269) 07:13:34 at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:607) 07:13:34 at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) 07:13:34 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529) 07:13:34 at hudson.model.Run.execute(Run.java:1738) 07:13:34 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) 07:13:34 at hudson.model.ResourceController.execute(ResourceController.java:98) 07:13:34 at hudson.model.Executor.run(Executor.java:410) 07:13:34 Caused by: java.nio.file.AccessDeniedException: /home/jenkins/root/workspace/devrpm-el7/RPMS/el7/x86_64/build.log 07:13:34 at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) 07:13:34 at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) 07:13:34 at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) 07:13:34 at sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244) 07:13:34 at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103) 07:13:34 at java.nio.file.Files.delete(Files.java:1079) 07:13:34 at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source) 07:13:34 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 07:13:34 at java.lang.reflect.Method.invoke(Method.java:606) 07:13:34 at hudson.Util.deleteFile(Util.java:255) 07:13:34 at hudson.FilePath.deleteRecursive(FilePath.java:1203) 07:13:34 at hudson.FilePath.deleteContentsRecursive(FilePath.java:1212) 07:13:34 at hudson.FilePath.deleteRecursive(FilePath.java:1194) 07:13:34 at hudson.FilePath.deleteContentsRecursive(FilePath.java:1212) 07:13:34 at hudson.FilePath.deleteRecursive(FilePath.java:1194) 07:13:34 at hudson.FilePath.deleteContentsRecursive(FilePath.java:1212) 07:13:34 at hudson.FilePath.deleteRecursive(FilePath.java:1194) 07:13:34 at hudson.FilePath.deleteContentsRecursive(FilePath.java:1212) 07:13:34 at hudson.FilePath.access$1100(FilePath.java:190) 07:13:34 at hudson.FilePath$15.invoke(FilePath.java:1186) 07:13:34 at hudson.FilePath$15.invoke(FilePath.java:1183) 07:13:34 at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2719) 07:13:34 at hudson.remoting.UserRequest.perform(UserRequest.java:120) 07:13:34 at hudson.remoting.UserRequest.perform(UserRequest.java:48) 07:13:34 at hudson.remoting.Request$2.run(Request.java:332) 07:13:34 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) 07:13:34 at java.util.concurrent.FutureTask.run(FutureTask.java:262) 07:13:34 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 07:13:34 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 07:13:34 at java.lang.Thread.run(Thread.java:745) 07:13:34 at ..remote call to slave25.cloud.gluster.org(Native Method) 07:13:34 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416) 07:13:34 at hudson.remoting.UserResponse.retrieve(UserRequest.java:220) 07:13:34 at hudson.remoting.Channel.call(Channel.java:781) 07:13:34 at hudson.FilePath.act(FilePath.java:979) 07:13:34 ... 13 more 07:13:34 Archiving artifacts 07:13:37 Finished: FAILURE Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1
[Gluster-infra] Postmortem for RPM build failures
Hello,
Some of you may have noticed that we've been particularly plagued with RPM
build failures for the past couple of days. The failure looks like a Java error
from Jenkins[1]. I first noticed the error on Tuesday, 31st January. I narrowed
down the issue with a problem with permissions. Jenkins cleans the workspace at
the start of every run and Jenkins was unable to delete the RPM folder for some
reason. My best guess is that something changed in the rpmbuild or mock package
in the last couple of days, which lead to this. We have not deployed any change
from the infra side which should have caused this.
I immediately ran an Ansible job that deleted all the RPM workspaces so that
fresh ones would work and I added this line at the end of every Jenkins job:
sudo chown -R jenkins:jenkins ${WORKSPACE}/RPMS
This seemed to work for the moment and I'd moved onto other things. Yesterday,
we saw quite a resurgence of the errors. Atin filed a bug this morning and I've
figured out what I did wrong. The permission change runs only when the job is
successful. Not exactly my best idea :)
I've now fixed it up by making it a post-job script[2] that runs the same
command
irrespective of whether the job failed or worked. That should fix up any
problems in the future. Apologies for the inconvenience. As always, please file
a bug when you notice any unexpected Jenkins behavior.
[1]: https://build.gluster.org/job/strfmt_errors/2789/console
[2]:
http://git.gluster.org/cgit/build-jobs.git/commit/?id=3b77dbdac288bf21f802be41019e2cd4d4dc3e3c
--
nigelb
signature.asc
Description: PGP signature
___
Gluster-infra mailing list
Gluster-infra@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-infra
[Gluster-infra] [Bug 1418542] Multiple smoke failures observed since yesterday
https://bugzilla.redhat.com/show_bug.cgi?id=1418542 Nigel Babu changed: What|Removed |Added Status|NEW |CLOSED CC||nig...@redhat.com Resolution|--- |CURRENTRELEASE Assignee|b...@gluster.org|nig...@redhat.com Last Closed||2017-02-02 02:35:04 --- Comment #1 from Nigel Babu --- Fixed. Post-mortem here: http://lists.gluster.org/pipermail/gluster-infra/2017-February/003153.html -- You are receiving this mail because: You are on the CC list for the bug. Unsubscribe from this bug https://bugzilla.redhat.com/token.cgi?t=zdTeTbaWX9&a=cc_unsubscribe ___ Gluster-infra mailing list Gluster-infra@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-infra
