[jira] [Created] (HADOOP-11015) Http server/client utils to propagate and recreate Exceptions from server to client
Alejandro Abdelnur created HADOOP-11015: --- Summary: Http server/client utils to propagate and recreate Exceptions from server to client Key: HADOOP-11015 URL: https://issues.apache.org/jira/browse/HADOOP-11015 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.6.0 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur While doing HADOOP-10771, while discussing it with [~daryn], a suggested improvement was to propagate the server side exceptions to the client in the same way WebHDFS does it. This JIRA is to provide a utility class to do the same and refactor HttpFS and KMS to use it. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Git repo ready to use
We just got HADOOP-11001 in. If you have access, can you please try modifying the Jenkins jobs taking the patch on HADOOP-11001 into consideration. On Wed, Aug 27, 2014 at 4:38 PM, Ted Yu yuzhih...@gmail.com wrote: I have access. I can switch the repository if you think it is time to do so. On Wed, Aug 27, 2014 at 4:35 PM, Karthik Kambatla ka...@cloudera.com wrote: Thanks for reporting it, Ted. We are aware of it - second follow-up item in my earlier email. Unfortunately, I don't have access to the builds to fix them and don't quite know the procedure to get access either. I am waiting for someone with access to help us out. On Wed, Aug 27, 2014 at 3:45 PM, Ted Yu yuzhih...@gmail.com wrote: Precommit builds are still using svn : https://builds.apache.org/job/PreCommit-HDFS-Build/configure https://builds.apache.org/job/PreCommit-YARN-Build/configure FYI On Wed, Aug 27, 2014 at 7:00 AM, Ted Yu yuzhih...@gmail.com wrote: Currently Jenkins builds still use subversion as source. Should Jenkins point to git ? Cheers On Wed, Aug 27, 2014 at 1:40 AM, Karthik Kambatla ka...@cloudera.com wrote: Oh.. a couple more things. The git commit hashes have changed and are different from what we had on our github. This might interfere with any build automations that folks have. Another follow-up item: email and JIRA integration On Wed, Aug 27, 2014 at 1:33 AM, Karthik Kambatla ka...@cloudera.com wrote: Hi folks, I am very excited to let you know that the git repo is now writable. I committed a few changes (CHANGES.txt fixes and branching for 2.5.1) and everything looks good. Current status: 1. All branches have the same names, including trunk. 2. Force push is disabled on trunk, branch-2 and tags. 3. Even if you are experienced with git, take a look at https://wiki.apache.org/hadoop/HowToCommitWithGit . Particularly, let us avoid merge commits. Follow-up items: 1. Update rest of the wiki documentation 2. Update precommit Jenkins jobs and get HADOOP-11001 committed (reviews appreciated). Until this is done, the precommit jobs will run against our old svn repo. 3. git mirrors etc. to use the new repo instead of the old svn repo. Thanks again for your cooperation through the migration process. Please reach out to me (or the list) if you find anything missing or have suggestions. Cheers! Karthik
RE: Git repo ready to use
Thanks Karthik for making Git repo ready. Regards, Yi Liu -Original Message- From: Karthik Kambatla [mailto:ka...@cloudera.com] Sent: Thursday, August 28, 2014 2:26 PM To: common-dev@hadoop.apache.org Subject: Re: Git repo ready to use We just got HADOOP-11001 in. If you have access, can you please try modifying the Jenkins jobs taking the patch on HADOOP-11001 into consideration. On Wed, Aug 27, 2014 at 4:38 PM, Ted Yu yuzhih...@gmail.com wrote: I have access. I can switch the repository if you think it is time to do so. On Wed, Aug 27, 2014 at 4:35 PM, Karthik Kambatla ka...@cloudera.com wrote: Thanks for reporting it, Ted. We are aware of it - second follow-up item in my earlier email. Unfortunately, I don't have access to the builds to fix them and don't quite know the procedure to get access either. I am waiting for someone with access to help us out. On Wed, Aug 27, 2014 at 3:45 PM, Ted Yu yuzhih...@gmail.com wrote: Precommit builds are still using svn : https://builds.apache.org/job/PreCommit-HDFS-Build/configure https://builds.apache.org/job/PreCommit-YARN-Build/configure FYI On Wed, Aug 27, 2014 at 7:00 AM, Ted Yu yuzhih...@gmail.com wrote: Currently Jenkins builds still use subversion as source. Should Jenkins point to git ? Cheers On Wed, Aug 27, 2014 at 1:40 AM, Karthik Kambatla ka...@cloudera.com wrote: Oh.. a couple more things. The git commit hashes have changed and are different from what we had on our github. This might interfere with any build automations that folks have. Another follow-up item: email and JIRA integration On Wed, Aug 27, 2014 at 1:33 AM, Karthik Kambatla ka...@cloudera.com wrote: Hi folks, I am very excited to let you know that the git repo is now writable. I committed a few changes (CHANGES.txt fixes and branching for 2.5.1) and everything looks good. Current status: 1. All branches have the same names, including trunk. 2. Force push is disabled on trunk, branch-2 and tags. 3. Even if you are experienced with git, take a look at https://wiki.apache.org/hadoop/HowToCommitWithGit . Particularly, let us avoid merge commits. Follow-up items: 1. Update rest of the wiki documentation 2. Update precommit Jenkins jobs and get HADOOP-11001 committed (reviews appreciated). Until this is done, the precommit jobs will run against our old svn repo. 3. git mirrors etc. to use the new repo instead of the old svn repo. Thanks again for your cooperation through the migration process. Please reach out to me (or the list) if you find anything missing or have suggestions. Cheers! Karthik
Re: Git repo ready to use
I modified config for the following builds: https://builds.apache.org/job/PreCommit-HDFS-Build/build #7808 would be checking out trunk using git. https://builds.apache.org/job/PreCommit-yarn-Build/ https://builds.apache.org/job/PreCommit-mapreduce-Build/ Should I modify the other Jenkins jobs e.g.: https://builds.apache.org/job/Hadoop-Yarn-trunk/ Cheers On Wed, Aug 27, 2014 at 11:25 PM, Karthik Kambatla ka...@cloudera.com wrote: We just got HADOOP-11001 in. If you have access, can you please try modifying the Jenkins jobs taking the patch on HADOOP-11001 into consideration. On Wed, Aug 27, 2014 at 4:38 PM, Ted Yu yuzhih...@gmail.com wrote: I have access. I can switch the repository if you think it is time to do so. On Wed, Aug 27, 2014 at 4:35 PM, Karthik Kambatla ka...@cloudera.com wrote: Thanks for reporting it, Ted. We are aware of it - second follow-up item in my earlier email. Unfortunately, I don't have access to the builds to fix them and don't quite know the procedure to get access either. I am waiting for someone with access to help us out. On Wed, Aug 27, 2014 at 3:45 PM, Ted Yu yuzhih...@gmail.com wrote: Precommit builds are still using svn : https://builds.apache.org/job/PreCommit-HDFS-Build/configure https://builds.apache.org/job/PreCommit-YARN-Build/configure FYI On Wed, Aug 27, 2014 at 7:00 AM, Ted Yu yuzhih...@gmail.com wrote: Currently Jenkins builds still use subversion as source. Should Jenkins point to git ? Cheers On Wed, Aug 27, 2014 at 1:40 AM, Karthik Kambatla ka...@cloudera.com wrote: Oh.. a couple more things. The git commit hashes have changed and are different from what we had on our github. This might interfere with any build automations that folks have. Another follow-up item: email and JIRA integration On Wed, Aug 27, 2014 at 1:33 AM, Karthik Kambatla ka...@cloudera.com wrote: Hi folks, I am very excited to let you know that the git repo is now writable. I committed a few changes (CHANGES.txt fixes and branching for 2.5.1) and everything looks good. Current status: 1. All branches have the same names, including trunk. 2. Force push is disabled on trunk, branch-2 and tags. 3. Even if you are experienced with git, take a look at https://wiki.apache.org/hadoop/HowToCommitWithGit . Particularly, let us avoid merge commits. Follow-up items: 1. Update rest of the wiki documentation 2. Update precommit Jenkins jobs and get HADOOP-11001 committed (reviews appreciated). Until this is done, the precommit jobs will run against our old svn repo. 3. git mirrors etc. to use the new repo instead of the old svn repo. Thanks again for your cooperation through the migration process. Please reach out to me (or the list) if you find anything missing or have suggestions. Cheers! Karthik
Re: Git repo ready to use
build #7808 failed due to QA bot trying to apply the following as patch: http://issues.apache.org/jira/secure/attachment/12552318/dfsio-x86-trunk-vs-3529.png FYI On Thu, Aug 28, 2014 at 1:52 AM, Ted Yu yuzhih...@gmail.com wrote: I modified config for the following builds: https://builds.apache.org/job/PreCommit-HDFS-Build/build #7808 would be checking out trunk using git. https://builds.apache.org/job/PreCommit-yarn-Build/ https://builds.apache.org/job/PreCommit-mapreduce-Build/ Should I modify the other Jenkins jobs e.g.: https://builds.apache.org/job/Hadoop-Yarn-trunk/ Cheers On Wed, Aug 27, 2014 at 11:25 PM, Karthik Kambatla ka...@cloudera.com wrote: We just got HADOOP-11001 in. If you have access, can you please try modifying the Jenkins jobs taking the patch on HADOOP-11001 into consideration. On Wed, Aug 27, 2014 at 4:38 PM, Ted Yu yuzhih...@gmail.com wrote: I have access. I can switch the repository if you think it is time to do so. On Wed, Aug 27, 2014 at 4:35 PM, Karthik Kambatla ka...@cloudera.com wrote: Thanks for reporting it, Ted. We are aware of it - second follow-up item in my earlier email. Unfortunately, I don't have access to the builds to fix them and don't quite know the procedure to get access either. I am waiting for someone with access to help us out. On Wed, Aug 27, 2014 at 3:45 PM, Ted Yu yuzhih...@gmail.com wrote: Precommit builds are still using svn : https://builds.apache.org/job/PreCommit-HDFS-Build/configure https://builds.apache.org/job/PreCommit-YARN-Build/configure FYI On Wed, Aug 27, 2014 at 7:00 AM, Ted Yu yuzhih...@gmail.com wrote: Currently Jenkins builds still use subversion as source. Should Jenkins point to git ? Cheers On Wed, Aug 27, 2014 at 1:40 AM, Karthik Kambatla ka...@cloudera.com wrote: Oh.. a couple more things. The git commit hashes have changed and are different from what we had on our github. This might interfere with any build automations that folks have. Another follow-up item: email and JIRA integration On Wed, Aug 27, 2014 at 1:33 AM, Karthik Kambatla ka...@cloudera.com wrote: Hi folks, I am very excited to let you know that the git repo is now writable. I committed a few changes (CHANGES.txt fixes and branching for 2.5.1) and everything looks good. Current status: 1. All branches have the same names, including trunk. 2. Force push is disabled on trunk, branch-2 and tags. 3. Even if you are experienced with git, take a look at https://wiki.apache.org/hadoop/HowToCommitWithGit . Particularly, let us avoid merge commits. Follow-up items: 1. Update rest of the wiki documentation 2. Update precommit Jenkins jobs and get HADOOP-11001 committed (reviews appreciated). Until this is done, the precommit jobs will run against our old svn repo. 3. git mirrors etc. to use the new repo instead of the old svn repo. Thanks again for your cooperation through the migration process. Please reach out to me (or the list) if you find anything missing or have suggestions. Cheers! Karthik
Build failed in Jenkins: Hadoop-Common-0.23-Build #1055
See https://builds.apache.org/job/Hadoop-Common-0.23-Build/1055/ -- [...truncated 8263 lines...] Running org.apache.hadoop.io.file.tfile.TestTFileNoneCodecsJClassComparatorByteArrays Tests run: 25, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.551 sec Running org.apache.hadoop.io.file.tfile.TestTFileUnsortedByteArrays Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.673 sec Running org.apache.hadoop.io.file.tfile.TestTFileLzoCodecsStreams Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.275 sec Running org.apache.hadoop.io.file.tfile.TestTFileComparators Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.546 sec Running org.apache.hadoop.io.file.tfile.TestTFileSplit Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.775 sec Running org.apache.hadoop.io.TestIOUtils Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.282 sec Running org.apache.hadoop.io.retry.TestFailoverProxy Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.191 sec Running org.apache.hadoop.io.retry.TestRetryProxy Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.209 sec Running org.apache.hadoop.io.TestSortedMapWritable Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.196 sec Running org.apache.hadoop.io.TestSecureIOUtils Tests run: 4, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.495 sec Running org.apache.hadoop.io.TestEnumSetWritable Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.406 sec Running org.apache.hadoop.io.serializer.avro.TestAvroSerialization Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.538 sec Running org.apache.hadoop.io.serializer.TestWritableSerialization Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.316 sec Running org.apache.hadoop.io.serializer.TestSerializationFactory Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.279 sec Running org.apache.hadoop.io.TestArrayFile Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.287 sec Running org.apache.hadoop.util.TestProtoUtil Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.079 sec Running org.apache.hadoop.util.TestReflectionUtils Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.496 sec Running org.apache.hadoop.util.TestOptions Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.079 sec Running org.apache.hadoop.util.TestStringUtils Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.136 sec Running org.apache.hadoop.util.TestDataChecksum Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.189 sec Running org.apache.hadoop.util.TestShell Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.194 sec Running org.apache.hadoop.util.TestAsyncDiskService Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.123 sec Running org.apache.hadoop.util.TestIndexedSort Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.62 sec Running org.apache.hadoop.util.TestGenericOptionsParser Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.687 sec Running org.apache.hadoop.util.TestDiskChecker Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.494 sec Running org.apache.hadoop.util.TestShutdownHookManager Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.143 sec Running org.apache.hadoop.util.TestPureJavaCrc32 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.286 sec Running org.apache.hadoop.util.TestGenericsUtil Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.259 sec Running org.apache.hadoop.util.TestJarFinder Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.795 sec Running org.apache.hadoop.util.TestRunJar Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.127 sec Running org.apache.hadoop.util.TestStringInterner Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.114 sec Running org.apache.hadoop.util.TestHostsFileReader Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.186 sec Running org.apache.hadoop.cli.TestCLI Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.934 sec Running org.apache.hadoop.fs.TestLocalDirAllocator Tests run: 30, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.009 sec Running org.apache.hadoop.fs.TestLocalFSFileContextSymlink Tests run: 61, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 2.686 sec Running org.apache.hadoop.fs.TestFsShellCopy Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.367 sec Running org.apache.hadoop.fs.TestAvroFSInput Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.477 sec Running org.apache.hadoop.fs.TestLocal_S3FileContextURI Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.349 sec Running org.apache.hadoop.fs.TestFilterFs Tests run: 1,
Re: Git repo ready to use
I spent some time on PreCommit-hdfs-Build.
Looks like the following command was not effective:
mkdir -p ${WORKSPACE}/patchprocess
In build output, I saw:
/home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/jira:
No such file or directory
I will work with Giri in the morning.
Cheers
On Thu, Aug 28, 2014 at 2:04 AM, Ted Yu yuzhih...@gmail.com wrote:
build #7808 failed due to QA bot trying to apply the following as patch:
http://issues.apache.org/jira/secure/attachment/12552318/dfsio-x86-trunk-vs-3529.png
FYI
On Thu, Aug 28, 2014 at 1:52 AM, Ted Yu yuzhih...@gmail.com wrote:
I modified config for the following builds:
https://builds.apache.org/job/PreCommit-HDFS-Build/build #7808 would
be checking out trunk using git.
https://builds.apache.org/job/PreCommit-yarn-Build/
https://builds.apache.org/job/PreCommit-mapreduce-Build/
Should I modify the other Jenkins jobs e.g.:
https://builds.apache.org/job/Hadoop-Yarn-trunk/
Cheers
On Wed, Aug 27, 2014 at 11:25 PM, Karthik Kambatla ka...@cloudera.com
wrote:
We just got HADOOP-11001 in. If you have access, can you please try
modifying the Jenkins jobs taking the patch on HADOOP-11001 into
consideration.
On Wed, Aug 27, 2014 at 4:38 PM, Ted Yu yuzhih...@gmail.com wrote:
I have access.
I can switch the repository if you think it is time to do so.
On Wed, Aug 27, 2014 at 4:35 PM, Karthik Kambatla ka...@cloudera.com
wrote:
Thanks for reporting it, Ted. We are aware of it - second follow-up
item
in
my earlier email.
Unfortunately, I don't have access to the builds to fix them and
don't
quite know the procedure to get access either. I am waiting for
someone
with access to help us out.
On Wed, Aug 27, 2014 at 3:45 PM, Ted Yu yuzhih...@gmail.com wrote:
Precommit builds are still using svn :
https://builds.apache.org/job/PreCommit-HDFS-Build/configure
https://builds.apache.org/job/PreCommit-YARN-Build/configure
FYI
On Wed, Aug 27, 2014 at 7:00 AM, Ted Yu yuzhih...@gmail.com
wrote:
Currently Jenkins builds still use subversion as source.
Should Jenkins point to git ?
Cheers
On Wed, Aug 27, 2014 at 1:40 AM, Karthik Kambatla
ka...@cloudera.com
wrote:
Oh.. a couple more things.
The git commit hashes have changed and are different from what
we
had
on
our github. This might interfere with any build automations that
folks
have.
Another follow-up item: email and JIRA integration
On Wed, Aug 27, 2014 at 1:33 AM, Karthik Kambatla
ka...@cloudera.com
wrote:
Hi folks,
I am very excited to let you know that the git repo is now
writable. I
committed a few changes (CHANGES.txt fixes and branching for
2.5.1)
and
everything looks good.
Current status:
1. All branches have the same names, including trunk.
2. Force push is disabled on trunk, branch-2 and tags.
3. Even if you are experienced with git, take a look at
https://wiki.apache.org/hadoop/HowToCommitWithGit .
Particularly,
let
us avoid merge commits.
Follow-up items:
1. Update rest of the wiki documentation
2. Update precommit Jenkins jobs and get HADOOP-11001
committed
(reviews appreciated). Until this is done, the precommit
jobs
will
run
against our old svn repo.
3. git mirrors etc. to use the new repo instead of the old
svn
repo.
Thanks again for your cooperation through the migration
process.
Please
reach out to me (or the list) if you find anything missing or
have
suggestions.
Cheers!
Karthik
Re: Git repo ready to use
I'm looking into it.
-giri
On Thu, Aug 28, 2014 at 3:18 AM, Ted Yu yuzhih...@gmail.com wrote:
I spent some time on PreCommit-hdfs-Build.
Looks like the following command was not effective:
mkdir -p ${WORKSPACE}/patchprocess
In build output, I saw:
/home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/jira:
No such file or directory
I will work with Giri in the morning.
Cheers
On Thu, Aug 28, 2014 at 2:04 AM, Ted Yu yuzhih...@gmail.com wrote:
build #7808 failed due to QA bot trying to apply the following as patch:
http://issues.apache.org/jira/secure/attachment/12552318/dfsio-x86-trunk-vs-3529.png
FYI
On Thu, Aug 28, 2014 at 1:52 AM, Ted Yu yuzhih...@gmail.com wrote:
I modified config for the following builds:
https://builds.apache.org/job/PreCommit-HDFS-Build/build #7808
would
be checking out trunk using git.
https://builds.apache.org/job/PreCommit-yarn-Build/
https://builds.apache.org/job/PreCommit-mapreduce-Build/
Should I modify the other Jenkins jobs e.g.:
https://builds.apache.org/job/Hadoop-Yarn-trunk/
Cheers
On Wed, Aug 27, 2014 at 11:25 PM, Karthik Kambatla ka...@cloudera.com
wrote:
We just got HADOOP-11001 in. If you have access, can you please try
modifying the Jenkins jobs taking the patch on HADOOP-11001 into
consideration.
On Wed, Aug 27, 2014 at 4:38 PM, Ted Yu yuzhih...@gmail.com wrote:
I have access.
I can switch the repository if you think it is time to do so.
On Wed, Aug 27, 2014 at 4:35 PM, Karthik Kambatla
ka...@cloudera.com
wrote:
Thanks for reporting it, Ted. We are aware of it - second follow-up
item
in
my earlier email.
Unfortunately, I don't have access to the builds to fix them and
don't
quite know the procedure to get access either. I am waiting for
someone
with access to help us out.
On Wed, Aug 27, 2014 at 3:45 PM, Ted Yu yuzhih...@gmail.com
wrote:
Precommit builds are still using svn :
https://builds.apache.org/job/PreCommit-HDFS-Build/configure
https://builds.apache.org/job/PreCommit-YARN-Build/configure
FYI
On Wed, Aug 27, 2014 at 7:00 AM, Ted Yu yuzhih...@gmail.com
wrote:
Currently Jenkins builds still use subversion as source.
Should Jenkins point to git ?
Cheers
On Wed, Aug 27, 2014 at 1:40 AM, Karthik Kambatla
ka...@cloudera.com
wrote:
Oh.. a couple more things.
The git commit hashes have changed and are different from what
we
had
on
our github. This might interfere with any build automations
that
folks
have.
Another follow-up item: email and JIRA integration
On Wed, Aug 27, 2014 at 1:33 AM, Karthik Kambatla
ka...@cloudera.com
wrote:
Hi folks,
I am very excited to let you know that the git repo is now
writable. I
committed a few changes (CHANGES.txt fixes and branching for
2.5.1)
and
everything looks good.
Current status:
1. All branches have the same names, including trunk.
2. Force push is disabled on trunk, branch-2 and tags.
3. Even if you are experienced with git, take a look at
https://wiki.apache.org/hadoop/HowToCommitWithGit .
Particularly,
let
us avoid merge commits.
Follow-up items:
1. Update rest of the wiki documentation
2. Update precommit Jenkins jobs and get HADOOP-11001
committed
(reviews appreciated). Until this is done, the precommit
jobs
will
run
against our old svn repo.
3. git mirrors etc. to use the new repo instead of the
old
svn
repo.
Thanks again for your cooperation through the migration
process.
Please
reach out to me (or the list) if you find anything missing
or
have
suggestions.
Cheers!
Karthik
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Git repo ready to use
Fixed all the 3 pre-commit buids. test-patch's git reset --hard is removing
the patchprocess dir, so moved it off the workspace.
-giri
On Thu, Aug 28, 2014 at 8:48 AM, Giridharan Kesavan
gkesa...@hortonworks.com wrote:
I'm looking into it.
-giri
On Thu, Aug 28, 2014 at 3:18 AM, Ted Yu yuzhih...@gmail.com wrote:
I spent some time on PreCommit-hdfs-Build.
Looks like the following command was not effective:
mkdir -p ${WORKSPACE}/patchprocess
In build output, I saw:
/home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/jira:
No such file or directory
I will work with Giri in the morning.
Cheers
On Thu, Aug 28, 2014 at 2:04 AM, Ted Yu yuzhih...@gmail.com wrote:
build #7808 failed due to QA bot trying to apply the following as patch:
http://issues.apache.org/jira/secure/attachment/12552318/dfsio-x86-trunk-vs-3529.png
FYI
On Thu, Aug 28, 2014 at 1:52 AM, Ted Yu yuzhih...@gmail.com wrote:
I modified config for the following builds:
https://builds.apache.org/job/PreCommit-HDFS-Build/build #7808
would
be checking out trunk using git.
https://builds.apache.org/job/PreCommit-yarn-Build/
https://builds.apache.org/job/PreCommit-mapreduce-Build/
Should I modify the other Jenkins jobs e.g.:
https://builds.apache.org/job/Hadoop-Yarn-trunk/
Cheers
On Wed, Aug 27, 2014 at 11:25 PM, Karthik Kambatla ka...@cloudera.com
wrote:
We just got HADOOP-11001 in. If you have access, can you please try
modifying the Jenkins jobs taking the patch on HADOOP-11001 into
consideration.
On Wed, Aug 27, 2014 at 4:38 PM, Ted Yu yuzhih...@gmail.com wrote:
I have access.
I can switch the repository if you think it is time to do so.
On Wed, Aug 27, 2014 at 4:35 PM, Karthik Kambatla
ka...@cloudera.com
wrote:
Thanks for reporting it, Ted. We are aware of it - second
follow-up
item
in
my earlier email.
Unfortunately, I don't have access to the builds to fix them and
don't
quite know the procedure to get access either. I am waiting for
someone
with access to help us out.
On Wed, Aug 27, 2014 at 3:45 PM, Ted Yu yuzhih...@gmail.com
wrote:
Precommit builds are still using svn :
https://builds.apache.org/job/PreCommit-HDFS-Build/configure
https://builds.apache.org/job/PreCommit-YARN-Build/configure
FYI
On Wed, Aug 27, 2014 at 7:00 AM, Ted Yu yuzhih...@gmail.com
wrote:
Currently Jenkins builds still use subversion as source.
Should Jenkins point to git ?
Cheers
On Wed, Aug 27, 2014 at 1:40 AM, Karthik Kambatla
ka...@cloudera.com
wrote:
Oh.. a couple more things.
The git commit hashes have changed and are different from
what
we
had
on
our github. This might interfere with any build automations
that
folks
have.
Another follow-up item: email and JIRA integration
On Wed, Aug 27, 2014 at 1:33 AM, Karthik Kambatla
ka...@cloudera.com
wrote:
Hi folks,
I am very excited to let you know that the git repo is now
writable. I
committed a few changes (CHANGES.txt fixes and branching
for
2.5.1)
and
everything looks good.
Current status:
1. All branches have the same names, including trunk.
2. Force push is disabled on trunk, branch-2 and tags.
3. Even if you are experienced with git, take a look at
https://wiki.apache.org/hadoop/HowToCommitWithGit .
Particularly,
let
us avoid merge commits.
Follow-up items:
1. Update rest of the wiki documentation
2. Update precommit Jenkins jobs and get HADOOP-11001
committed
(reviews appreciated). Until this is done, the precommit
jobs
will
run
against our old svn repo.
3. git mirrors etc. to use the new repo instead of the
old
svn
repo.
Thanks again for your cooperation through the migration
process.
Please
reach out to me (or the list) if you find anything missing
or
have
suggestions.
Cheers!
Karthik
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Git repo ready to use
Thanks, Giri
I have switched the following builds to using git:
https://builds.apache.org/job/Hadoop-Yarn-trunk/
https://builds.apache.org/job/Hadoop-hdfs-trunk/
https://builds.apache.org/job/Hadoop-mapreduce-trunk/
Cheers
On Thu, Aug 28, 2014 at 9:07 AM, Giridharan Kesavan
gkesa...@hortonworks.com wrote:
Fixed all the 3 pre-commit buids. test-patch's git reset --hard is removing
the patchprocess dir, so moved it off the workspace.
-giri
On Thu, Aug 28, 2014 at 8:48 AM, Giridharan Kesavan
gkesa...@hortonworks.com wrote:
I'm looking into it.
-giri
On Thu, Aug 28, 2014 at 3:18 AM, Ted Yu yuzhih...@gmail.com wrote:
I spent some time on PreCommit-hdfs-Build.
Looks like the following command was not effective:
mkdir -p ${WORKSPACE}/patchprocess
In build output, I saw:
/home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/jira:
No such file or directory
I will work with Giri in the morning.
Cheers
On Thu, Aug 28, 2014 at 2:04 AM, Ted Yu yuzhih...@gmail.com wrote:
build #7808 failed due to QA bot trying to apply the following as
patch:
http://issues.apache.org/jira/secure/attachment/12552318/dfsio-x86-trunk-vs-3529.png
FYI
On Thu, Aug 28, 2014 at 1:52 AM, Ted Yu yuzhih...@gmail.com wrote:
I modified config for the following builds:
https://builds.apache.org/job/PreCommit-HDFS-Build/build #7808
would
be checking out trunk using git.
https://builds.apache.org/job/PreCommit-yarn-Build/
https://builds.apache.org/job/PreCommit-mapreduce-Build/
Should I modify the other Jenkins jobs e.g.:
https://builds.apache.org/job/Hadoop-Yarn-trunk/
Cheers
On Wed, Aug 27, 2014 at 11:25 PM, Karthik Kambatla
ka...@cloudera.com
wrote:
We just got HADOOP-11001 in. If you have access, can you please try
modifying the Jenkins jobs taking the patch on HADOOP-11001 into
consideration.
On Wed, Aug 27, 2014 at 4:38 PM, Ted Yu yuzhih...@gmail.com
wrote:
I have access.
I can switch the repository if you think it is time to do so.
On Wed, Aug 27, 2014 at 4:35 PM, Karthik Kambatla
ka...@cloudera.com
wrote:
Thanks for reporting it, Ted. We are aware of it - second
follow-up
item
in
my earlier email.
Unfortunately, I don't have access to the builds to fix them and
don't
quite know the procedure to get access either. I am waiting for
someone
with access to help us out.
On Wed, Aug 27, 2014 at 3:45 PM, Ted Yu yuzhih...@gmail.com
wrote:
Precommit builds are still using svn :
https://builds.apache.org/job/PreCommit-HDFS-Build/configure
https://builds.apache.org/job/PreCommit-YARN-Build/configure
FYI
On Wed, Aug 27, 2014 at 7:00 AM, Ted Yu yuzhih...@gmail.com
wrote:
Currently Jenkins builds still use subversion as source.
Should Jenkins point to git ?
Cheers
On Wed, Aug 27, 2014 at 1:40 AM, Karthik Kambatla
ka...@cloudera.com
wrote:
Oh.. a couple more things.
The git commit hashes have changed and are different from
what
we
had
on
our github. This might interfere with any build automations
that
folks
have.
Another follow-up item: email and JIRA integration
On Wed, Aug 27, 2014 at 1:33 AM, Karthik Kambatla
ka...@cloudera.com
wrote:
Hi folks,
I am very excited to let you know that the git repo is
now
writable. I
committed a few changes (CHANGES.txt fixes and branching
for
2.5.1)
and
everything looks good.
Current status:
1. All branches have the same names, including trunk.
2. Force push is disabled on trunk, branch-2 and tags.
3. Even if you are experienced with git, take a look
at
https://wiki.apache.org/hadoop/HowToCommitWithGit .
Particularly,
let
us avoid merge commits.
Follow-up items:
1. Update rest of the wiki documentation
2. Update precommit Jenkins jobs and get HADOOP-11001
committed
(reviews appreciated). Until this is done, the
precommit
jobs
will
run
against our old svn repo.
3. git mirrors etc. to use the new repo instead of the
old
svn
repo.
Thanks again for your cooperation through the migration
process.
Please
reach out to me (or the list) if you find anything
missing
or
have
suggestions.
Cheers!
Karthik
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the
Re: Git repo ready to use
On 8/28/2014 12:07 PM, Giridharan Kesavan wrote: Fixed all the 3 pre-commit buids. test-patch's git reset --hard is removing the patchprocess dir, so moved it off the workspace. Thanks Giri. Should I resubmit HDFS-6954's patch? I've gotten 3 or 4 jenkins messages that indicated the problem so something is resubmitting, but now that you've fixed it, should I resubmit it again? Charles
Re: Git repo ready to use
Charles: QA build is running for your JIRA: https://builds.apache.org/job/PreCommit-hdfs-Build/7828/parameters/ Cheers On Thu, Aug 28, 2014 at 9:41 AM, Charles Lamb cl...@cloudera.com wrote: On 8/28/2014 12:07 PM, Giridharan Kesavan wrote: Fixed all the 3 pre-commit buids. test-patch's git reset --hard is removing the patchprocess dir, so moved it off the workspace. Thanks Giri. Should I resubmit HDFS-6954's patch? I've gotten 3 or 4 jenkins messages that indicated the problem so something is resubmitting, but now that you've fixed it, should I resubmit it again? Charles
[jira] [Created] (HADOOP-11016) KMS should support signing cookies with zookeeper secret manager
Alejandro Abdelnur created HADOOP-11016: --- Summary: KMS should support signing cookies with zookeeper secret manager Key: HADOOP-11016 URL: https://issues.apache.org/jira/browse/HADOOP-11016 Project: Hadoop Common Issue Type: Improvement Components: security Affects Versions: 2.6.0 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur This will allow supporting multiple KMS instances behind a load-balancer. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HADOOP-11017) KMS delegation token secret manager should be able to use zookeeper as store
Alejandro Abdelnur created HADOOP-11017: --- Summary: KMS delegation token secret manager should be able to use zookeeper as store Key: HADOOP-11017 URL: https://issues.apache.org/jira/browse/HADOOP-11017 Project: Hadoop Common Issue Type: Improvement Components: security Affects Versions: 2.6.0 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur This will allow supporting multiple KMS instances behind a load balancer. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HADOOP-11019) Queued IPC calls are not aborted if the connection drops
Daryn Sharp created HADOOP-11019: Summary: Queued IPC calls are not aborted if the connection drops Key: HADOOP-11019 URL: https://issues.apache.org/jira/browse/HADOOP-11019 Project: Hadoop Common Issue Type: Bug Components: ipc Affects Versions: 2.0.0-alpha, 3.0.0 Reporter: Daryn Sharp Assignee: Daryn Sharp Once a call is read from the wire and queued, it will be executed even if the connection has already dropped. If the client closes the connection due to timeout, perhaps because the server is overloaded, the client's retry will only exacerbate the problem. One specific example is DNs with large block reports overwhelming an already unhealthy NN. Ideally calls should be cancelled when the connection is dropped and/or connection state should be checked when the call is extracted from the callq, prior to decoding and invoking the call. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HADOOP-11020) TestRefreshUserMappings fails
Chen He created HADOOP-11020: Summary: TestRefreshUserMappings fails Key: HADOOP-11020 URL: https://issues.apache.org/jira/browse/HADOOP-11020 Project: Hadoop Common Issue Type: Bug Reporter: Chen He Error Message /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build%402/trunk/hadoop-hdfs-project/hadoop-hdfs/target/test-classes/testGroupMappingRefresh_rsrc.xml (No such file or directory) Stacktrace java.io.FileNotFoundException: /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build%402/trunk/hadoop-hdfs-project/hadoop-hdfs/target/test-classes/testGroupMappingRefresh_rsrc.xml (No such file or directory) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.init(FileOutputStream.java:194) at java.io.FileOutputStream.init(FileOutputStream.java:84) at org.apache.hadoop.security.TestRefreshUserMappings.addNewConfigResource(TestRefreshUserMappings.java:242) at org.apache.hadoop.security.TestRefreshUserMappings.testRefreshSuperUserGroupsConfiguration(TestRefreshUserMappings.java:203) -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Git repo ready to use
Thanks Giri and Ted for fixing the builds. On Thu, Aug 28, 2014 at 9:49 AM, Ted Yu yuzhih...@gmail.com wrote: Charles: QA build is running for your JIRA: https://builds.apache.org/job/PreCommit-hdfs-Build/7828/parameters/ Cheers On Thu, Aug 28, 2014 at 9:41 AM, Charles Lamb cl...@cloudera.com wrote: On 8/28/2014 12:07 PM, Giridharan Kesavan wrote: Fixed all the 3 pre-commit buids. test-patch's git reset --hard is removing the patchprocess dir, so moved it off the workspace. Thanks Giri. Should I resubmit HDFS-6954's patch? I've gotten 3 or 4 jenkins messages that indicated the problem so something is resubmitting, but now that you've fixed it, should I resubmit it again? Charles
[jira] [Created] (HADOOP-11022) User replaced functions get lost 2-3 levels deep (e.g., sbin)
Allen Wittenauer created HADOOP-11022: - Summary: User replaced functions get lost 2-3 levels deep (e.g., sbin) Key: HADOOP-11022 URL: https://issues.apache.org/jira/browse/HADOOP-11022 Project: Hadoop Common Issue Type: Bug Components: scripts Affects Versions: 3.0.0 Reporter: Allen Wittenauer Priority: Critical The code that protects hadoop-env.sh from being re-executed is also causing functions that the user replaced to get overridden with the defaults. This typically happens when running commands that nest, such as most of the content in sbin. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HADOOP-11023) Example hadoop_log_rotate in hadoop-env.sh doesn't work
Allen Wittenauer created HADOOP-11023: - Summary: Example hadoop_log_rotate in hadoop-env.sh doesn't work Key: HADOOP-11023 URL: https://issues.apache.org/jira/browse/HADOOP-11023 Project: Hadoop Common Issue Type: Bug Components: scripts Affects Versions: 3.0.0 Reporter: Allen Wittenauer Silly errors in the example prevent it from properly rotating. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Git repo ready to use
I noticed that the JUnit test results aren't getting picked up anymore. I suspect we just need to update the path to the surefire output, but based on a quick examination I'm not sure what that is. Does someone mind taking another look? On Thu, Aug 28, 2014 at 4:21 PM, Karthik Kambatla ka...@cloudera.com wrote: Thanks Giri and Ted for fixing the builds. On Thu, Aug 28, 2014 at 9:49 AM, Ted Yu yuzhih...@gmail.com wrote: Charles: QA build is running for your JIRA: https://builds.apache.org/job/PreCommit-hdfs-Build/7828/parameters/ Cheers On Thu, Aug 28, 2014 at 9:41 AM, Charles Lamb cl...@cloudera.com wrote: On 8/28/2014 12:07 PM, Giridharan Kesavan wrote: Fixed all the 3 pre-commit buids. test-patch's git reset --hard is removing the patchprocess dir, so moved it off the workspace. Thanks Giri. Should I resubmit HDFS-6954's patch? I've gotten 3 or 4 jenkins messages that indicated the problem so something is resubmitting, but now that you've fixed it, should I resubmit it again? Charles
Re: Git repo ready to use
i'm also seeing broken links for javadocs warnings. Alejandro (phone typing) On Aug 28, 2014, at 20:00, Andrew Wang andrew.w...@cloudera.com wrote: I noticed that the JUnit test results aren't getting picked up anymore. I suspect we just need to update the path to the surefire output, but based on a quick examination I'm not sure what that is. Does someone mind taking another look? On Thu, Aug 28, 2014 at 4:21 PM, Karthik Kambatla ka...@cloudera.com wrote: Thanks Giri and Ted for fixing the builds. On Thu, Aug 28, 2014 at 9:49 AM, Ted Yu yuzhih...@gmail.com wrote: Charles: QA build is running for your JIRA: https://builds.apache.org/job/PreCommit-hdfs-Build/7828/parameters/ Cheers On Thu, Aug 28, 2014 at 9:41 AM, Charles Lamb cl...@cloudera.com wrote: On 8/28/2014 12:07 PM, Giridharan Kesavan wrote: Fixed all the 3 pre-commit buids. test-patch's git reset --hard is removing the patchprocess dir, so moved it off the workspace. Thanks Giri. Should I resubmit HDFS-6954's patch? I've gotten 3 or 4 jenkins messages that indicated the problem so something is resubmitting, but now that you've fixed it, should I resubmit it again? Charles
Re: Git repo ready to use
Currently patchprocess/ (contents shown below) is one level higher than
${WORKSPACE}
diffJavadocWarnings.txt newPatchFindbugsWarningshadoop-hdfs.html
patchFindBugsOutputhadoop-hdfs.txtpatchReleaseAuditOutput.txt
trunkJavadocWarnings.txt
filteredPatchJavacWarnings.txt newPatchFindbugsWarningshadoop-hdfs.xml
patchFindbugsWarningshadoop-hdfs.xml patchReleaseAuditWarnings.txt
filteredTrunkJavacWarnings.txt patch
patchJavacWarnings.txttestrun_hadoop-hdfs.txt
jirapatchEclipseOutput.txt
patchJavadocWarnings.txt trunkJavacWarnings.txt
Under Files to archive input box of PreCommit-HDFS-Build/configure, I saw:
‘../patchprocess/*’ doesn’t match anything, but ‘*’ does. Perhaps that’s
what you mean?
I guess once patchprocess is moved back under ${WORKSPACE}, a lot of things
would be back to normal.
Cheers
On Thu, Aug 28, 2014 at 9:16 PM, Alejandro Abdelnur t...@cloudera.com
wrote:
i'm also seeing broken links for javadocs warnings.
Alejandro
(phone typing)
On Aug 28, 2014, at 20:00, Andrew Wang andrew.w...@cloudera.com wrote:
I noticed that the JUnit test results aren't getting picked up anymore. I
suspect we just need to update the path to the surefire output, but based
on a quick examination I'm not sure what that is.
Does someone mind taking another look?
On Thu, Aug 28, 2014 at 4:21 PM, Karthik Kambatla ka...@cloudera.com
wrote:
Thanks Giri and Ted for fixing the builds.
On Thu, Aug 28, 2014 at 9:49 AM, Ted Yu yuzhih...@gmail.com wrote:
Charles:
QA build is running for your JIRA:
https://builds.apache.org/job/PreCommit-hdfs-Build/7828/parameters/
Cheers
On Thu, Aug 28, 2014 at 9:41 AM, Charles Lamb cl...@cloudera.com
wrote:
On 8/28/2014 12:07 PM, Giridharan Kesavan wrote:
Fixed all the 3 pre-commit buids. test-patch's git reset --hard is
removing
the patchprocess dir, so moved it off the workspace.
Thanks Giri. Should I resubmit HDFS-6954's patch? I've gotten 3 or 4
jenkins messages that indicated the problem so something is
resubmitting,
but now that you've fixed it, should I resubmit it again?
Charles
[jira] [Resolved] (HADOOP-11023) Example hadoop_log_rotate in hadoop-env.sh doesn't work
[ https://issues.apache.org/jira/browse/HADOOP-11023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HADOOP-11023. --- Resolution: Duplicate closing as dupe since it is fixed inside HADOOP-11022 Example hadoop_log_rotate in hadoop-env.sh doesn't work --- Key: HADOOP-11023 URL: https://issues.apache.org/jira/browse/HADOOP-11023 Project: Hadoop Common Issue Type: Bug Components: scripts Affects Versions: 3.0.0 Reporter: Allen Wittenauer Silly errors in the example prevent it from properly rotating. -- This message was sent by Atlassian JIRA (v6.2#6252)
