Re: [Gluster-devel] Change in glusterfs[master]: Regression test portability: mktemp
Le jeudi 21 août 2014 à 16:37 +0100, Justin Clift a écrit : > On 21/08/2014, at 2:40 PM, Justin Clift wrote: > > On 21/08/2014, at 1:07 AM, Harshavardhana wrote: > >> On Wed, Aug 20, 2014 at 2:49 PM, Justin Clift wrote: > >>> Any ideas? > >> > >> Avati just fixed it > >> > >> We had to run 'git gc' directly on the gerrit server - gerrit gc > >> failed (could be a gerrit bug). > > > > Avati, did you make a backup of the /review/ directory before doing this? > > > > It looks like the "git gc" command was run as root instead of review user, > > so the permissions of the git dirs under /review/ have gone weird, and > > Gerrit looks to have done bad things to itself from there (eg internal > > Gerrit corruption) > > > > Misc is trying to resolve it atm, but a backup of /review/ dir would > > be a good thing to have atm. :) > > Doing a chown of the /review/ directory seems to have fixed the problem. > > I've made a tarball of the /review/ directory after it, and am rsync-ing > it from that box to my personal Linode one as a extra backup step. > > The review.gluster.org server seems to have a severe bandwidth problem > though, limiting connections to ~30kb/s. (my rsync will take up 12kb/s > of that for the next ~8 hours) > > This will make backups fun. ;) We just need to make backup of the gerrit db, no need for the git repo since they are already mirrored to 2 locations with enough bandwidth. -- Michael Scherer Open Source and Standards, Sysadmin signature.asc Description: This is a digitally signed message part ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Change in glusterfs[master]: Regression test portability: mktemp
On 21/08/2014, at 2:40 PM, Justin Clift wrote: > On 21/08/2014, at 1:07 AM, Harshavardhana wrote: >> On Wed, Aug 20, 2014 at 2:49 PM, Justin Clift wrote: >>> Any ideas? >> >> Avati just fixed it >> >> We had to run 'git gc' directly on the gerrit server - gerrit gc >> failed (could be a gerrit bug). > > Avati, did you make a backup of the /review/ directory before doing this? > > It looks like the "git gc" command was run as root instead of review user, > so the permissions of the git dirs under /review/ have gone weird, and > Gerrit looks to have done bad things to itself from there (eg internal > Gerrit corruption) > > Misc is trying to resolve it atm, but a backup of /review/ dir would > be a good thing to have atm. :) Doing a chown of the /review/ directory seems to have fixed the problem. I've made a tarball of the /review/ directory after it, and am rsync-ing it from that box to my personal Linode one as a extra backup step. The review.gluster.org server seems to have a severe bandwidth problem though, limiting connections to ~30kb/s. (my rsync will take up 12kb/s of that for the next ~8 hours) This will make backups fun. ;) + Justin -- GlusterFS - http://www.gluster.org An open source, distributed file system scaling to several petabytes, and handling thousands of clients. My personal twitter: twitter.com/realjustinclift ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Change in glusterfs[master]: Regression test portability: mktemp
On 21/08/2014, at 1:07 AM, Harshavardhana wrote: > On Wed, Aug 20, 2014 at 2:49 PM, Justin Clift wrote: >> Any ideas? >> >> > > Avati just fixed it > > We had to run 'git gc' directly on the gerrit server - gerrit gc > failed (could be a gerrit bug). Avati, did you make a backup of the /review/ directory before doing this? It looks like the "git gc" command was run as root instead of review user, so the permissions of the git dirs under /review/ have gone weird, and Gerrit looks to have done bad things to itself from there (eg internal Gerrit corruption) Misc is trying to resolve it atm, but a backup of /review/ dir would be a good thing to have atm. :) + Justin -- GlusterFS - http://www.gluster.org An open source, distributed file system scaling to several petabytes, and handling thousands of clients. My personal twitter: twitter.com/realjustinclift ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Change in glusterfs[master]: Regression test portability: mktemp
> Using git over ssh often fails, is it the same problem? I never > investigated, but using HTTP is reliable. I often have to run rfc.sh > three or four times before it works. > It seems like a 'gerrit' bug at the moment, its fixed with 'git gc' on the server no corruption reported :-). -- Religious confuse piety with mere ritual, the virtuous confuse regulation with outcomes ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Change in glusterfs[master]: Regression test portability: mktemp
Harshavardhana wrote: > ERROR: Could not clone repository Using git over ssh often fails, is it the same problem? I never investigated, but using HTTP is reliable. I often have to run rfc.sh three or four times before it works. -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Change in glusterfs[master]: Regression test portability: mktemp
Again with the unpack issue with git > > Build Failed > > http://build.gluster.org/job/glusterfs-rpms/556/ : SUCCESS > > http://build.gluster.org/job/glusterfs-rpms-el6/556/ : FAILURE > Receiving objects: 2% (2239/81742), 579.83 KiB | 854.00 KiB/s Receiving objects: 3% (2453/81742), 579.83 KiB | 854.00 KiB/s remote: internal server error Receiving objects: 4% (3270/81742), 579.83 KiB | 854.00 KiB/s fatal: early EOF fatal: index-pack failed at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:774) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.clone(CliGitAPIImpl.java:218) ... 11 more Trying next repository ERROR: Could not clone repository ... ... ... Finished: FAILURE If you remember this bogged us while setting up FreeBSD ?? - there seems something going wrong we have to verify as it may be disastrous in future. Perhaps a 'git gc' and we have to see if the git packs on the git.gluster.org are not corrupted. -- Religious confuse piety with mere ritual, the virtuous confuse regulation with outcomes ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
