Re: committing HADOOP-11746 test-patch improvements
On Wed, Apr 22, 2015 at 2:10 AM, Allen Wittenauer a...@altiscale.com wrote:
* There have been a few runs which seems to indicate that *something* is
destroying the artifact directory in the middle of runs…. which is very
very odd and something I hadn’t seen in any of my testing. In any case, I
clearly need to add some safety code here to report back that something
went awry and report back which node, console, etc this happened on.
Someone more familiar with the Jenkins setup might be able to shed some
light on why that might happen. All of these runs appear to be on H3, so
might be related? Impacted issues with this have been:
- HDFS-8200 (https://builds.apache.org/job/PreCommit-HDFS-Build/10335/)
- HDFS-8147 (https://builds.apache.org/job/PreCommit-HDFS-Build/10338/)
- YARN-3301 (https://builds.apache.org/job/PreCommit-YARN-Build/7441/)
From the HDFS precommit build:
PATCHPROCESS=${WORKSPACE}/../patchprocess
mkdir -p ${PATCHPROCESS}
Working on directories outside of the workspace for the job is not good,
though I'm not sure if that's the source of the issue. Do I need to
coordinate fixing this with anyone?
--
Sean
Re: committing HADOOP-11746 test-patch improvements
Err, first jira mentioned should be HADOOP-11861. On Apr 22, 2015, at 8:10 AM, Allen Wittenauer a...@altiscale.com wrote: Some status: * So far, HADOOP-11627 was filed which is luckily an extremely easy bug to fix. * There have been a few runs which seems to indicate that *something* is destroying the artifact directory in the middle of runs…. which is very very odd and something I hadn’t seen in any of my testing. In any case, I clearly need to add some safety code here to report back that something went awry and report back which node, console, etc this happened on. Someone more familiar with the Jenkins setup might be able to shed some light on why that might happen. All of these runs appear to be on H3, so might be related? Impacted issues with this have been: - HDFS-8200 (https://builds.apache.org/job/PreCommit-HDFS-Build/10335/) - HDFS-8147 (https://builds.apache.org/job/PreCommit-HDFS-Build/10338/) - YARN-3301 (https://builds.apache.org/job/PreCommit-YARN-Build/7441/) * Some sort of good news! The new mvn site test does appear to be capable of catching issues! YARN-3410 was committed just prior to the new test code and has markdown-to-html syntax issues. HADOOP-11627 (for example) shows the results. :D On Apr 21, 2015, at 10:48 PM, Allen Wittenauer a...@altiscale.com wrote: FYI, MAPREDUCE-6324 is the first JIRA to be tested with the new code in place if someone hasn’t seen the new output. On Apr 21, 2015, at 9:06 PM, Allen Wittenauer a...@altiscale.com wrote: Just a heads up that I’ll be committing this to trunk and branch-2 here in a bit. I’ll watch it over the next hour or two before heading off to bed. (I’m in London at the moment.) If there are any issues, revert and drop a note in HADOOP-11746 with any issues. Despite what will hopefully be obvious changes, a few things to keep your eyes out for: * shellcheck won’t work until shellcheck actually gets installed on the jenkins nodes. * checkstyle might give spurious errors until HADOOP-11778 is committed, since it currently NPEs, at least in trunk. On Apr 16, 2015, at 7:38 PM, Chris Nauroth cnaur...@hortonworks.com wrote: I'd like to thank Allen Wittenauer for his work on HADOOP-11746 to rewrite test-patch.sh. There is a lot of nice new functionality in there. My favorite part is that some patches will execute much faster, so I expect this will make the project more efficient overall at moving patches through the pre-commit process. I have +1'd the patch, but since this is a tool that we all use frequently, I'd like to delay a day before committing. Please comment on the jira if you have any additional feedback. We're aiming to commit on Friday, 4/17. Chris Nauroth Hortonworks http://hortonworks.com/
Re: committing HADOOP-11746 test-patch improvements
Some status: * So far, HADOOP-11627 was filed which is luckily an extremely easy bug to fix. * There have been a few runs which seems to indicate that *something* is destroying the artifact directory in the middle of runs…. which is very very odd and something I hadn’t seen in any of my testing. In any case, I clearly need to add some safety code here to report back that something went awry and report back which node, console, etc this happened on. Someone more familiar with the Jenkins setup might be able to shed some light on why that might happen. All of these runs appear to be on H3, so might be related? Impacted issues with this have been: - HDFS-8200 (https://builds.apache.org/job/PreCommit-HDFS-Build/10335/) - HDFS-8147 (https://builds.apache.org/job/PreCommit-HDFS-Build/10338/) - YARN-3301 (https://builds.apache.org/job/PreCommit-YARN-Build/7441/) * Some sort of good news! The new mvn site test does appear to be capable of catching issues! YARN-3410 was committed just prior to the new test code and has markdown-to-html syntax issues. HADOOP-11627 (for example) shows the results. :D On Apr 21, 2015, at 10:48 PM, Allen Wittenauer a...@altiscale.com wrote: FYI, MAPREDUCE-6324 is the first JIRA to be tested with the new code in place if someone hasn’t seen the new output. On Apr 21, 2015, at 9:06 PM, Allen Wittenauer a...@altiscale.com wrote: Just a heads up that I’ll be committing this to trunk and branch-2 here in a bit. I’ll watch it over the next hour or two before heading off to bed. (I’m in London at the moment.) If there are any issues, revert and drop a note in HADOOP-11746 with any issues. Despite what will hopefully be obvious changes, a few things to keep your eyes out for: * shellcheck won’t work until shellcheck actually gets installed on the jenkins nodes. * checkstyle might give spurious errors until HADOOP-11778 is committed, since it currently NPEs, at least in trunk. On Apr 16, 2015, at 7:38 PM, Chris Nauroth cnaur...@hortonworks.com wrote: I'd like to thank Allen Wittenauer for his work on HADOOP-11746 to rewrite test-patch.sh. There is a lot of nice new functionality in there. My favorite part is that some patches will execute much faster, so I expect this will make the project more efficient overall at moving patches through the pre-commit process. I have +1'd the patch, but since this is a tool that we all use frequently, I'd like to delay a day before committing. Please comment on the jira if you have any additional feedback. We're aiming to commit on Friday, 4/17. Chris Nauroth Hortonworks http://hortonworks.com/
Re: committing HADOOP-11746 test-patch improvements
FYI, MAPREDUCE-6324 is the first JIRA to be tested with the new code in place if someone hasn’t seen the new output. On Apr 21, 2015, at 9:06 PM, Allen Wittenauer a...@altiscale.com wrote: Just a heads up that I’ll be committing this to trunk and branch-2 here in a bit. I’ll watch it over the next hour or two before heading off to bed. (I’m in London at the moment.) If there are any issues, revert and drop a note in HADOOP-11746 with any issues. Despite what will hopefully be obvious changes, a few things to keep your eyes out for: * shellcheck won’t work until shellcheck actually gets installed on the jenkins nodes. * checkstyle might give spurious errors until HADOOP-11778 is committed, since it currently NPEs, at least in trunk. On Apr 16, 2015, at 7:38 PM, Chris Nauroth cnaur...@hortonworks.com wrote: I'd like to thank Allen Wittenauer for his work on HADOOP-11746 to rewrite test-patch.sh. There is a lot of nice new functionality in there. My favorite part is that some patches will execute much faster, so I expect this will make the project more efficient overall at moving patches through the pre-commit process. I have +1'd the patch, but since this is a tool that we all use frequently, I'd like to delay a day before committing. Please comment on the jira if you have any additional feedback. We're aiming to commit on Friday, 4/17. Chris Nauroth Hortonworks http://hortonworks.com/
Re: committing HADOOP-11746 test-patch improvements
Just a heads up that I’ll be committing this to trunk and branch-2 here in a bit. I’ll watch it over the next hour or two before heading off to bed. (I’m in London at the moment.) If there are any issues, revert and drop a note in HADOOP-11746 with any issues. Despite what will hopefully be obvious changes, a few things to keep your eyes out for: * shellcheck won’t work until shellcheck actually gets installed on the jenkins nodes. * checkstyle might give spurious errors until HADOOP-11778 is committed, since it currently NPEs, at least in trunk. On Apr 16, 2015, at 7:38 PM, Chris Nauroth cnaur...@hortonworks.com wrote: I'd like to thank Allen Wittenauer for his work on HADOOP-11746 to rewrite test-patch.sh. There is a lot of nice new functionality in there. My favorite part is that some patches will execute much faster, so I expect this will make the project more efficient overall at moving patches through the pre-commit process. I have +1'd the patch, but since this is a tool that we all use frequently, I'd like to delay a day before committing. Please comment on the jira if you have any additional feedback. We're aiming to commit on Friday, 4/17. Chris Nauroth Hortonworks http://hortonworks.com/
committing HADOOP-11746 test-patch improvements
I'd like to thank Allen Wittenauer for his work on HADOOP-11746 to rewrite test-patch.sh. There is a lot of nice new functionality in there. My favorite part is that some patches will execute much faster, so I expect this will make the project more efficient overall at moving patches through the pre-commit process. I have +1'd the patch, but since this is a tool that we all use frequently, I'd like to delay a day before committing. Please comment on the jira if you have any additional feedback. We're aiming to commit on Friday, 4/17. Chris Nauroth Hortonworks http://hortonworks.com/
