Re: RFR: race with nested repos in hgforest.sh

2013-02-21 Thread Christos Zoulas
On Feb 6, 10:04am, chris.hega...@oracle.com (Chris Hegarty) wrote: -- Subject: Re: RFR: race with nested repos in hgforest.sh | Yes this is better, but the quotes are incorrect. It should simply be |for rc in ${tmp}/*.pid.rc ; do This will break if there is no match. christos

Re: RFR: race with nested repos in hgforest.sh

2013-02-06 Thread Chris Hegarty
On 6 Feb 2013, at 16:26, chris...@zoulas.com wrote: > On Feb 6, 10:04am, chris.hega...@oracle.com (Chris Hegarty) wrote: > -- Subject: Re: RFR: race with nested repos in hgforest.sh > > | Yes this is better, but the quotes are incorrect. It should simply be > |for rc in ${tmp

Re: RFR: race with nested repos in hgforest.sh

2013-02-06 Thread Chris Hegarty
Neil, I think a wider discussion needs to be had around how this script should operate, but I didn't want this specific issue to get side tracked by that. The hgforest script already clones/pulls in parallel, the problem here is that it just doesn't work with older versions of hg. And worse t

Re: RFR: race with nested repos in hgforest.sh

2013-02-06 Thread Neil Richards
fwiw, I've always found the attempts by hg_forest to process things in parallel cause me more pain than it's worth. Marching through the repo locations serially in order might take a few moments longer elapsed time, but creates instantly understandable status / output / log and points for returnin

Re: RFR: race with nested repos in hgforest.sh

2013-02-06 Thread Chris Hegarty
On 06/02/2013 10:24, David Holmes wrote: Thanks Chris. I missed a couple of obvious things there. :( One thing related to the repo name in the $(rc) variable. Somewhere the repo name gets mangled: + echo jdk/src/closed + sed -e s@./@@ -e s@/@_@g + repopidfile=jdsrc_closed the / to _ doesn't

Re: RFR: race with nested repos in hgforest.sh

2013-02-06 Thread David Holmes
Thanks Chris. I missed a couple of obvious things there. :( One thing related to the repo name in the $(rc) variable. Somewhere the repo name gets mangled: + echo jdk/src/closed + sed -e s@./@@ -e s@/@_@g + repopidfile=jdsrc_closed the / to _ doesn't work for the first / (not part of your fi

Re: RFR: race with nested repos in hgforest.sh

2013-02-06 Thread Chris Hegarty
Thank for the review. comments inline... On 06/02/2013 01:53, David Holmes wrote: 204 # Terminate with exit 0 only if all subprocesses were successful 205 ec=0 206 if [ -d ${tmp} ]; then 207 for rc in `ls ${tmp}/*.pid.rc` ; do I don't think you need to subshell an invocation of ls, simply do f

Re: RFR: race with nested repos in hgforest.sh

2013-02-05 Thread David Holmes
Hi Chris, I don't speak fluent sh but ... 204 # Terminate with exit 0 only if all subprocesses were successful 205 ec=0 206 if [ -d ${tmp} ]; then 207 for rc in `ls ${tmp}/*.pid.rc` ; do I don't think you need to subshell an invocation of ls, simply do for rc in "${tmp}/*.pid.rc" ; do

Re: RFR: race with nested repos in hgforest.sh

2013-02-05 Thread Chris Hegarty
On 02/05/2013 06:21 AM, David Holmes wrote: Chris, When these failures occur does the failure get reflected in an error exit code? Not currently, good point. I'm seeing hudson builds merilly buildinh OpenJDK instead of Oracle JDK because the closed repos were skipped due to this bug :( Thi

Re: RFR: race with nested repos in hgforest.sh

2013-02-04 Thread Fredrik Öhrström
Fixing get_source.sh problem: We can hardcode pre-creation of the closed repo locations as mkdir -p calls. We can do the openjdk clone first (in parllell) and when that is finished do the closedjdk clone. We can drop parallell clone altogether because it does not give the speedup we are looking fo

Re: RFR: race with nested repos in hgforest.sh

2013-02-04 Thread David Holmes
Chris, When these failures occur does the failure get reflected in an error exit code? I'm seeing hudson builds merilly buildinh OpenJDK instead of Oracle JDK because the closed repos were skipped due to this bug :( David On 2/02/2013 12:40 AM, Chris Hegarty wrote: [ to build-dev and core

Re: RFR: race with nested repos in hgforest.sh

2013-02-04 Thread Ioi Lam
How about adding a message to indicate the sleep, just in case the directory is never created for whatever reason. If you don't like too many such messages, you can print the message after the initial wait. 177 while [ ! -d "$path" ] ## nested repo, ensure containing dir exists 1

Re: RFR: race with nested repos in hgforest.sh

2013-02-04 Thread Chris Hegarty
On 4 Feb 2013, at 19:36, Ioi Lam wrote: > How about adding a message to indicate the sleep, just in case the directory > is never created for whatever reason. If you don't like too many such > messages, you can print the message after the initial wait. I understand the reason for your suggest

Re: RFR: race with nested repos in hgforest.sh

2013-02-04 Thread Kelly O'Hair
Looks fine to me. -kto On Feb 1, 2013, at 6:40 AM, Chris Hegarty wrote: > [ to build-dev and core-libs-dev, expect reviewer from either, but will > integrate through jdk8/tl ] > > This issue is mainly of interest to Oracle engineers, but it effects the > public hgforest script. > > When hgfo

Re: RFR: race with nested repos in hgforest.sh

2013-02-04 Thread Chris Hegarty
On 04/02/2013 10:04, Fredrik Öhrström wrote: 2013-02-04 10:48, Chris Hegarty skrev: There has been some discussion on this with David, but no other takers. Fredrik, am I right that you added the support for parallel clone/update? I believe the changes I have are correct, but maybe you had other

Re: RFR: race with nested repos in hgforest.sh

2013-02-04 Thread Fredrik Öhrström
2013-02-04 10:48, Chris Hegarty skrev: There has been some discussion on this with David, but no other takers. Fredrik, am I right that you added the support for parallel clone/update? I believe the changes I have are correct, but maybe you had other ideas about how to solve this issue? -Chr

Re: RFR: race with nested repos in hgforest.sh

2013-02-04 Thread Chris Hegarty
There has been some discussion on this with David, but no other takers. Fredrik, am I right that you added the support for parallel clone/update? I believe the changes I have are correct, but maybe you had other ideas about how to solve this issue? -Chris. On 01/02/2013 14:40, Chris Hegarty

Re: RFR: race with nested repos in hgforest.sh

2013-02-02 Thread Chris Hegarty
On 02/02/2013 12:37 AM, David Holmes wrote: ... I can see that happening if the sleeps are too short. But what I noticed from the logging was that the closed repos were being processed before the open ones - which shouldn't happen. I believe the output is misleading. I think the output from

Re: RFR: race with nested repos in hgforest.sh

2013-02-01 Thread David Holmes
On 2/02/2013 9:06 AM, Chris Hegarty wrote: On 1 Feb 2013, at 22:37, David Holmes wrote: Hi Chris, On 2/02/2013 12:40 AM, Chris Hegarty wrote: [ to build-dev and core-libs-dev, expect reviewer from either, but will integrate through jdk8/tl ] This issue is mainly of interest to Oracle engin

Re: RFR: race with nested repos in hgforest.sh

2013-02-01 Thread Chris Hegarty
On 1 Feb 2013, at 22:37, David Holmes wrote: > Hi Chris, > > On 2/02/2013 12:40 AM, Chris Hegarty wrote: >> [ to build-dev and core-libs-dev, expect reviewer from either, but will >> integrate through jdk8/tl ] >> >> This issue is mainly of interest to Oracle engineers, but it effects the >> p

Re: RFR: race with nested repos in hgforest.sh

2013-02-01 Thread David Holmes
Hi Chris, On 2/02/2013 12:40 AM, Chris Hegarty wrote: [ to build-dev and core-libs-dev, expect reviewer from either, but will integrate through jdk8/tl ] This issue is mainly of interest to Oracle engineers, but it effects the public hgforest script. When hgforest.sh is run with an addition ar

RFR: race with nested repos in hgforest.sh

2013-02-01 Thread Chris Hegarty
[ to build-dev and core-libs-dev, expect reviewer from either, but will integrate through jdk8/tl ] This issue is mainly of interest to Oracle engineers, but it effects the public hgforest script. When hgforest.sh is run with an addition argument to specify a closed server, there is a proble