[jira] [Updated] (HADOOP-14208) Fix typo in the top page in branch-2.8
[
https://issues.apache.org/jira/browse/HADOOP-14208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Zhuge updated HADOOP-14208:
Component/s: documentation
common
> Fix typo in the top page in branch-2.8
> --
>
> Key: HADOOP-14208
> URL: https://issues.apache.org/jira/browse/HADOOP-14208
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, documentation
>Affects Versions: 2.8.0
>Reporter: Akira Ajisaka
>Assignee: Wenxin He
>Priority: Trivial
> Labels: newbie
> Fix For: 2.8.2
>
>
> There is a typo in the summary of the release.
> {noformat:title=index.md.vm}
> * Allow node labels get specificed in submitting MR jobs
> {noformat}
> specificed should be specified.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14690) RetryInvocationHandler$RetryInfo should override toString()
[
https://issues.apache.org/jira/browse/HADOOP-14690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106022#comment-16106022
]
Hadoop QA commented on HADOOP-14690:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 3s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 56m 55s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.security.TestRaceWhenRelogin |
| | hadoop.security.TestKDiag |
| | hadoop.net.TestDNS |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14690 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879475/HADOOP-14690-001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux e057b28f3608 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 890e14c |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12893/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12893/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12893/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> RetryInvocationHandler$RetryInfo should override toString()
> ---
>
> Key: HADOOP-14690
> URL: https://issues.apache.org/jira/browse/HADOOP-14690
> Project:
[jira] [Updated] (HADOOP-14690) RetryInvocationHandler$RetryInfo should override toString()
[
https://issues.apache.org/jira/browse/HADOOP-14690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yeliang Cang updated HADOOP-14690:
--
Assignee: Yeliang Cang
Status: Patch Available (was: Open)
> RetryInvocationHandler$RetryInfo should override toString()
> ---
>
> Key: HADOOP-14690
> URL: https://issues.apache.org/jira/browse/HADOOP-14690
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Akira Ajisaka
>Assignee: Yeliang Cang
>Priority: Minor
> Labels: newbie, supportability
> Attachments: HADOOP-14690-001.patch
>
>
> {code:title=RetryInvocationHandler.java}
> LOG.trace("#{} processRetryInfo: retryInfo={}, waitTime={}",
> callId, retryInfo, waitTime);
> {code}
> RetryInfo is used for logging but it does not output useful information.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14690) RetryInvocationHandler$RetryInfo should override toString()
[
https://issues.apache.org/jira/browse/HADOOP-14690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yeliang Cang updated HADOOP-14690:
--
Attachment: HADOOP-14690-001.patch
Hi [~ajisakaa], I submit a patch like you suggested. Please see it!
> RetryInvocationHandler$RetryInfo should override toString()
> ---
>
> Key: HADOOP-14690
> URL: https://issues.apache.org/jira/browse/HADOOP-14690
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Akira Ajisaka
>Priority: Minor
> Labels: newbie, supportability
> Attachments: HADOOP-14690-001.patch
>
>
> {code:title=RetryInvocationHandler.java}
> LOG.trace("#{} processRetryInfo: retryInfo={}, waitTime={}",
> callId, retryInfo, waitTime);
> {code}
> RetryInfo is used for logging but it does not output useful information.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14672) Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
[ https://issues.apache.org/jira/browse/HADOOP-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105904#comment-16105904 ] Junping Du commented on HADOOP-14672: - v5 patch LGTM too. [~arpitagarwal], please feel free to go ahead to commit this. > Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, > dom, etc. > -- > > Key: HADOOP-14672 > URL: https://issues.apache.org/jira/browse/HADOOP-14672 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Bharat Viswanadham >Priority: Blocker > Attachments: HADOOP-14672.02.patch, HADOOP-14672.03.patch, > HADOOP-14672.04.patch, HADOOP-14672.05.patch, HADOOP-14672.patch > > > The shaded hadoop-client-minicluster shouldn't include any unshaded > dependencies, but we can see: javax, dom, sax, etc. are all unshaded. > CC [~busbey] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14676) Wrong default value for "fs.df.interval"
[
https://issues.apache.org/jira/browse/HADOOP-14676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105846#comment-16105846
]
xiangguang zheng commented on HADOOP-14676:
---
The manual steps I did to verify is to compile and run the unit test towards
hadoop common module. When I run the test, there were 9 test failure and 1 test
error. But I have verified that those failure and errors existed in hadoop
trunk before I made any changes, and none of test failure/error was related to
this patch
> Wrong default value for "fs.df.interval"
>
>
> Key: HADOOP-14676
> URL: https://issues.apache.org/jira/browse/HADOOP-14676
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, conf, fs
>Affects Versions: 2.6.1
>Reporter: Konstantin Shvachko
>Assignee: xiangguang zheng
> Attachments: HADOOP-14676.001.patch
>
>
> According to {{core-default.xml}} the default value of {{fs.df.interval = 60
> sec}}. But the implementation of {{DF}} substitutes 3 sec instead. The
> problem is that {{DF}} uses outdated constant {{DF.DF_INTERVAL_DEFAULT}}
> instead of the correct one
> {{CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14698) Make copyFromLocal's -t option available for put as well
[
https://issues.apache.org/jira/browse/HADOOP-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105825#comment-16105825
]
Hadoop QA commented on HADOOP-14698:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
35s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 36s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 1 new + 44 unchanged - 0 fixed = 45 total (was 44) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 7s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 58m 40s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.net.TestDNS |
| | hadoop.cli.TestCLI |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14698 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879441/HADOOP-14698.01.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 2694976817ee 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 713349a |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12892/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12892/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12892/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12892/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Make copyFromLocal's -t option available for put as well
>
[jira] [Updated] (HADOOP-14701) Configuration can log misleading warnings about an attempt to override final parameter
[ https://issues.apache.org/jira/browse/HADOOP-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HADOOP-14701: Attachment: HADOOP-14701.1.patch > Configuration can log misleading warnings about an attempt to override final > parameter > -- > > Key: HADOOP-14701 > URL: https://issues.apache.org/jira/browse/HADOOP-14701 > Project: Hadoop Common > Issue Type: Bug > Components: conf >Reporter: Andrew Sherman > Attachments: HADOOP-14701.1.patch > > > The class Configuration (in hadoop-common-project) is used to hold > configuration properties. > The method loadResources() is used to load properties from a list of default > resources into a parameter called “properties” (which happens to always be > the field called “properties”). > This is done by calling the method loadResource() which is used to load > properties from a single resource into a parameter called “properties” > (which happens to always be the field called “properties”). > This is done by calling loadProperty() which is used to load a single > property into a parameter called “properties” (which is sometimes, but not > always, the field called “properties”). > In loadProperty() there is some logic for dealing with properties that are > marked as final. > (1) a final property is not set into the “properties” parameter. > (2) a warning is printed for a final property if the value of the property > with the same name in the “properties” parameter is not equal to the new > value, and the property name is in the finalParameters field. > When loadResource() is reading properties from a stream it creates a > temporary Properties object that it populates by calling loadProperty(). This > is the case where loadProperty() has a parameter called “properties” which > is not the same as the field called “properties”). > When loadProperty() is called on a final property then because of (1) it is > not set into the “properties” parameter. But sometimes the property name is > already in the finalParameters field and now (2) the warning is triggered > because the property is not present in the “properties” parameter (although > it IS present in the “properties” field). The warning is misleading and > confusing. > h3. Possible Fix > One way of looking at the problem is that loadProperty is using the > finalParameters field in combination with the “properties” parameter. Change > the code to only generate a warning for a final property if the value of the > property with the same name in the “properties” field is not equal to the new > value, and the property name is in the finalParameters field. > I’ve made a patch containing this tiny change. > Note that I have done very minimal testing of this fix > h3. Example warning: > 2017-07-25 12:14:24,639 WARN org.apache.hadoop.conf.Configuration: > [HiveServer2-Background-Pool: Thread-77]: > hdfs://asherman-cdh6-subset-parcel-1.gce.cloudera.com:8020/user/hive/.staging/job_1501008923167_0001/job.xml:an > attempt to override final parameter: > mapreduce.job.end-notification.max.retry.interval; Ignoring. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14699) Impersonation errors with UGI after second principal relogin
[
https://issues.apache.org/jira/browse/HADOOP-14699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Storck updated HADOOP-14699:
-
Description:
Multiple principals that are logged in using UGI instances that are
instantiated from a UGI class loaded by the same classloader will encounter
problems when the second principal attempts to relogin and perform an action
using a UGI.doAs(). An impersonation will occur and the operation attempted by
the second principal after relogging in will fail. There should not be an
implicit attempt to impersonate the second principal through the first
principal that logged in.
I have created a GitHub project that exhibits the impersonation error with
brief instructions on how to set up for the test and run it:
https://github.com/jtstorck/kerberos-examples/tree/master/hadoop/ugi-test
{quote}18:44:55.687 [pool-2-thread-2] WARN
h.u.u.ugirunnable.ugite...@example.com - Unexpected exception while performing
task for [ugite...@example.com (auth:KERBEROS)]
org.apache.hadoop.ipc.RemoteException: User: ugite...@example.com is not
allowed to impersonate ugite...@example.com
at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1481)
at org.apache.hadoop.ipc.Client.call(Client.java:1427)
at org.apache.hadoop.ipc.Client.call(Client.java:1337)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:787)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:398)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:335)
at com.sun.proxy.$Proxy10.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1700)
at
org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1436)
at
org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1433)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1448)
at
hadoop.ugitest.UgiTestMain$UgiRunnable.lambda$run$2(UgiTestMain.java:194)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807)
at hadoop.ugitest.UgiTestMain$UgiRunnable.run(UgiTestMain.java:194)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745){quote}
was:
Multiple principals that are logged in using UGI instances that are
instantiated from a UGI class loaded by the same classloader will encounter
problems when the second principal attempts to relogin and perform an action
using a UGI.doAs(). An impersonation will occur and the operation attempted by
the second principal after relogging in will fail. There should not be an
implicit attempt to impersonate the second principal through the first
principal that logged in.
I have created a GitHub project that exhibits the impersonation error with
brief instructions on how to set up for the test and run it:
https://github.com/jtstorck/kerberos-examples/tree/master/hadoop/ugi-test
{{18:44:55.687 [pool-2-thread-2] WARN
[jira] [Updated] (HADOOP-14699) Impersonation errors with UGI after second principal relogin
[
https://issues.apache.org/jira/browse/HADOOP-14699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Storck updated HADOOP-14699:
-
Description:
Multiple principals that are logged in using UGI instances that are
instantiated from a UGI class loaded by the same classloader will encounter
problems when the second principal attempts to relogin and perform an action
using a UGI.doAs(). An impersonation will occur and the operation attempted by
the second principal after relogging in will fail. There should not be an
implicit attempt to impersonate the second principal through the first
principal that logged in.
I have created a GitHub project that exhibits the impersonation error with
brief instructions on how to set up for the test and run it:
https://github.com/jtstorck/kerberos-examples/tree/master/hadoop/ugi-test
{{18:44:55.687 [pool-2-thread-2] WARN h.u.u.ugirunnable.ugite...@example.com -
Unexpected exception while performing task for [ugite...@example.com
(auth:KERBEROS)]
org.apache.hadoop.ipc.RemoteException: User: ugite...@example.com is not
allowed to impersonate ugite...@example.com
at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1481)
at org.apache.hadoop.ipc.Client.call(Client.java:1427)
at org.apache.hadoop.ipc.Client.call(Client.java:1337)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:787)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:398)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:335)
at com.sun.proxy.$Proxy10.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1700)
at
org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1436)
at
org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1433)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1448)
at
hadoop.ugitest.UgiTestMain$UgiRunnable.lambda$run$2(UgiTestMain.java:194)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807)
at hadoop.ugitest.UgiTestMain$UgiRunnable.run(UgiTestMain.java:194)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)}}
was:
Multiple principals that are logged in using UGI instances that are
instantiated from a UGI class loaded by the same classloader will encounter
problems when the second principal attempts to relogin and perform an action
using a UGI.doAs(). An impersonation will occur and the operation attempted by
the second principal after relogging in will fail.
I have created a GitHub project that exhibits the impersonation error with
brief instructions on how to set up for the test and run it:
https://github.com/jtstorck/kerberos-examples/tree/master/hadoop/ugi-test
> Impersonation errors with UGI after second principal relogin
>
>
> Key: HADOOP-14699
> URL:
[jira] [Updated] (HADOOP-14699) Impersonation errors with UGI after second principal relogin
[ https://issues.apache.org/jira/browse/HADOOP-14699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Storck updated HADOOP-14699: - Description: Multiple principals that are logged in using UGI instances that are instantiated from a UGI class loaded by the same classloader will encounter problems when the second principal attempts to relogin and perform an action using a UGI.doAs(). An impersonation will occur and the operation attempted by the second principal after relogging in will fail. I have created a GitHub project that exhibits the impersonation error with brief instructions on how to set up for the test and run it: https://github.com/jtstorck/kerberos-examples/tree/master/hadoop/ugi-test was: Multiple principals that are logged in using UGI instances that are instantiated from a UGI class loaded by the same classloader will encounter problems when the second principal attempts to relogin and perform an action using a UGI.doAs(). An impersonation will occur and the operation attempted by the second principal after relogging in will fail. I have created a GitHub project that exhibits the impersonation error with brief instructions on how to set up for the test and run it: https://github.com/jtstorck/kerberos-examples/tree/master/hadoop/ugi-test > Impersonation errors with UGI after second principal relogin > > > Key: HADOOP-14699 > URL: https://issues.apache.org/jira/browse/HADOOP-14699 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 2.6.2, 2.7.3, 2.8.1 >Reporter: Jeff Storck > > Multiple principals that are logged in using UGI instances that are > instantiated from a UGI class loaded by the same classloader will encounter > problems when the second principal attempts to relogin and perform an action > using a UGI.doAs(). An impersonation will occur and the operation attempted > by the second principal after relogging in will fail. > I have created a GitHub project that exhibits the impersonation error with > brief instructions on how to set up for the test and run it: > https://github.com/jtstorck/kerberos-examples/tree/master/hadoop/ugi-test -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14699) Impersonation errors with UGI after second principal relogin
[ https://issues.apache.org/jira/browse/HADOOP-14699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Storck updated HADOOP-14699: - Description: Multiple principals that are logged in using UGI instances that are instantiated from a UGI class loaded by the same classloader will encounter problems when the second principal attempts to relogin and perform an action using a UGI.doAs(). An impersonation will occur and the operation attempted by the second principal after relogging in will fail. I have created a GitHub project that exhibits the impersonation error with brief instructions on how to set up for the test and run it: https://github.com/jtstorck/kerberos-examples/tree/master/hadoop/ugi-test was: Multiple principals that are logged in using UGI instances that are instantiated from a UGI class loaded by the same classloader will encounter problems when the second principal attempts to relogin. An impersonation will occur and the operation attempted by the second principal after relogging in will fail. I have created a GitHub project that exhibits the impersonation error with brief instructions on how to set up for the test and run it: https://github.com/jtstorck/kerberos-examples/tree/master/hadoop/ugi-test > Impersonation errors with UGI after second principal relogin > > > Key: HADOOP-14699 > URL: https://issues.apache.org/jira/browse/HADOOP-14699 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 2.6.2, 2.7.3, 2.8.1 >Reporter: Jeff Storck > > Multiple principals that are logged in using UGI instances that are > instantiated from a UGI class loaded by the same classloader will encounter > problems when the second principal attempts to relogin and perform an action > using a UGI.doAs(). An impersonation will occur and the operation attempted > by the second principal after relogging in will fail. > I have created a GitHub project that exhibits the impersonation error with > brief instructions on how to set up for the test and run it: > https://github.com/jtstorck/kerberos-examples/tree/master/hadoop/ugi-test -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14676) Wrong default value for "fs.df.interval"
[
https://issues.apache.org/jira/browse/HADOOP-14676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105812#comment-16105812
]
Hadoop QA commented on HADOOP-14676:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 13m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} hadoop-common-project/hadoop-common: The patch
generated 0 new + 4 unchanged - 1 fixed = 4 total (was 5) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
25s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 75m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14676 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879434/HADOOP-14676.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 0323f0016078 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 713349a |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12889/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12889/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Wrong default value for "fs.df.interval"
>
>
> Key: HADOOP-14676
> URL: https://issues.apache.org/jira/browse/HADOOP-14676
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, conf, fs
>Affects Versions: 2.6.1
>Reporter: Konstantin Shvachko
>Assignee: xiangguang zheng
> Attachments: HADOOP-14676.001.patch
[jira] [Created] (HADOOP-14701) Configuration can log misleading warnings about an attempt to override final parameter
Andrew Sherman created HADOOP-14701: --- Summary: Configuration can log misleading warnings about an attempt to override final parameter Key: HADOOP-14701 URL: https://issues.apache.org/jira/browse/HADOOP-14701 Project: Hadoop Common Issue Type: Bug Components: conf Reporter: Andrew Sherman The class Configuration (in hadoop-common-project) is used to hold configuration properties. The method loadResources() is used to load properties from a list of default resources into a parameter called “properties” (which happens to always be the field called “properties”). This is done by calling the method loadResource() which is used to load properties from a single resource into a parameter called “properties” (which happens to always be the field called “properties”). This is done by calling loadProperty() which is used to load a single property into a parameter called “properties” (which is sometimes, but not always, the field called “properties”). In loadProperty() there is some logic for dealing with properties that are marked as final. (1) a final property is not set into the “properties” parameter. (2) a warning is printed for a final property if the value of the property with the same name in the “properties” parameter is not equal to the new value, and the property name is in the finalParameters field. When loadResource() is reading properties from a stream it creates a temporary Properties object that it populates by calling loadProperty(). This is the case where loadProperty() has a parameter called “properties” which is not the same as the field called “properties”). When loadProperty() is called on a final property then because of (1) it is not set into the “properties” parameter. But sometimes the property name is already in the finalParameters field and now (2) the warning is triggered because the property is not present in the “properties” parameter (although it IS present in the “properties” field). The warning is misleading and confusing. h3. Possible Fix One way of looking at the problem is that loadProperty is using the finalParameters field in combination with the “properties” parameter. Change the code to only generate a warning for a final property if the value of the property with the same name in the “properties” field is not equal to the new value, and the property name is in the finalParameters field. I’ve made a patch containing this tiny change. Note that I have done very minimal testing of this fix h3. Example warning: 2017-07-25 12:14:24,639 WARN org.apache.hadoop.conf.Configuration: [HiveServer2-Background-Pool: Thread-77]: hdfs://asherman-cdh6-subset-parcel-1.gce.cloudera.com:8020/user/hive/.staging/job_1501008923167_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14700) NativeAzureFileSystem.open() ignores blob container name
[
https://issues.apache.org/jira/browse/HADOOP-14700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cheng Lian updated HADOOP-14700:
Description:
{{NativeAzureFileSystem}} instances are associated with the blob container used
to initialize the file system. Assuming that a file system instance {{fs}} is
associated with a container {{A}}, when trying to access a blob inside another
container {{B}}, {{fs}} still tries to find the blob inside container {{A}}. If
there happens to be two blobs with the same name inside both containers, the
user may get a wrong result because {{fs}} reads the contents from the blob
inside container {{A}} instead of container {{B}}.
The following self-contained Scala code snippet illustrates this issue. You may
reproduce it by running the script inside the [Ammonite
REPL|http://ammonite.io/].
{code}
import $ivy.`com.jsuereth::scala-arm:2.0`
import $ivy.`com.microsoft.azure:azure-storage:5.2.0`
import $ivy.`org.apache.hadoop:hadoop-azure:3.0.0-alpha4`
import $ivy.`org.apache.hadoop:hadoop-common:3.0.0-alpha4`
import $ivy.`org.scalatest::scalatest:3.0.3`
import java.io.{BufferedReader, InputStreamReader}
import java.net.URI
import java.time.{Duration, Instant}
import java.util.{Date, EnumSet}
import com.microsoft.azure.storage.{CloudStorageAccount,
StorageCredentialsAccountAndKey}
import com.microsoft.azure.storage.blob.{SharedAccessBlobPermissions,
SharedAccessBlobPolicy}
import org.apache.hadoop.conf.Configuration
import org.apache.hadoop.fs.{FileSystem, Path}
import org.apache.hadoop.fs.azure.{AzureException, NativeAzureFileSystem}
import org.scalatest.Assertions._
import resource._
// Utility implicit conversion for auto resource management.
implicit def `Closable->Resource`[T <: { def close() }]: Resource[T] = new
Resource[T] {
override def close(closable: T): Unit = closable.close()
}
// Credentials information
val ACCOUNT = "** REDACTED **"
val ACCESS_KEY = "** REDACTED **"
// We'll create two different containers, both contain a blob named "test-blob"
but with different
// contents.
val CONTAINER_A = "container-a"
val CONTAINER_B = "container-b"
val TEST_BLOB = "test-blob"
val blobClient = {
val credentials = new StorageCredentialsAccountAndKey(ACCOUNT, ACCESS_KEY)
val account = new CloudStorageAccount(credentials, /* useHttps */ true)
account.createCloudBlobClient()
}
// Generates a read-only SAS key restricted within "container-a".
val sasKeyForContainerA = {
val since = Instant.now() minus Duration.ofMinutes(10)
val duration = Duration.ofHours(1)
val policy = new SharedAccessBlobPolicy()
policy.setSharedAccessStartTime(Date.from(since))
policy.setSharedAccessExpiryTime(Date.from(since plus duration))
policy.setPermissions(EnumSet.of(
SharedAccessBlobPermissions.READ,
SharedAccessBlobPermissions.LIST
))
blobClient
.getContainerReference(CONTAINER_A)
.generateSharedAccessSignature(policy, null)
}
// Sets up testing containers and blobs using the Azure storage SDK:
//
// container-a/test-blob => "foo"
// container-b/test-blob => "bar"
{
val containerARef = blobClient.getContainerReference(CONTAINER_A)
val containerBRef = blobClient.getContainerReference(CONTAINER_B)
containerARef.createIfNotExists()
containerARef.getBlockBlobReference(TEST_BLOB).uploadText("foo")
containerBRef.createIfNotExists()
containerBRef.getBlockBlobReference(TEST_BLOB).uploadText("bar")
}
val pathA = new
Path(s"wasbs://$CONTAINER_A@$ACCOUNT.blob.core.windows.net/$TEST_BLOB")
val pathB = new
Path(s"wasbs://$CONTAINER_A@$ACCOUNT.blob.core.windows.net/$TEST_BLOB")
for {
// Creates a file system associated with "container-a".
fs <- managed {
val conf = new Configuration
conf.set("fs.wasbs.impl", classOf[NativeAzureFileSystem].getName)
conf.set(s"fs.azure.sas.$CONTAINER_A.$ACCOUNT.blob.core.windows.net",
sasKeyForContainerA)
pathA.getFileSystem(conf)
}
// Opens a reader pointing to "container-a/test-blob". We expect to get the
string "foo" written
// to this blob previously.
readerA <- managed(new BufferedReader(new InputStreamReader(fs open pathA)))
// Opens a reader pointing to "container-b/test-blob". We expect to get an
exception since the SAS
// key used to create the `FileSystem` instance is restricted to
"container-a".
readerB <- managed(new BufferedReader(new InputStreamReader(fs open pathB)))
} {
// Should get "foo"
assert(readerA.readLine() == "foo")
// Should catch an exception ...
intercept[AzureException] {
// ... but instead, we get string "foo" here, which indicates that the
readerB was reading from
// "container-a" instead of "container-b".
val contents = readerB.readLine()
println(s"Should not reach here but we got $contents")
}
}
{code}
was:
{{NativeAzureFileSystem}} instances are associated with the blob container used
to initialize the file system. Assuming that a
[jira] [Created] (HADOOP-14700) NativeAzureFileSystem.open() ignores blob container name
Cheng Lian created HADOOP-14700:
---
Summary: NativeAzureFileSystem.open() ignores blob container name
Key: HADOOP-14700
URL: https://issues.apache.org/jira/browse/HADOOP-14700
Project: Hadoop Common
Issue Type: Sub-task
Components: fs
Affects Versions: 3.0.0-alpha4, 3.0.0-beta1
Reporter: Cheng Lian
{{NativeAzureFileSystem}} instances are associated with the blob container used
to initialize the file system. Assuming that a file system instance {{fs}} is
associated with a container {{A}}, when trying to access a blob inside another
container {{B}}, {{fs}} still tries to find the blob inside container {{A}}. If
there happens to be two blobs with the same name inside both containers, the
user may get a wrong result because {{fs}} reads the contents from the blob
inside container {{A}} instead of container {{B}}.
The following self-contained Scala code snippet illustrates this issue. You may
reproduce it by running the script inside the [Ammonite
REPL|http://ammonite.io/].
{code}
import $ivy.`com.jsuereth::scala-arm:2.0`
import $ivy.`com.microsoft.azure:azure-storage:5.2.0`
import $ivy.`org.apache.hadoop:hadoop-azure:3.0.0-alpha4`
import $ivy.`org.apache.hadoop:hadoop-common:3.0.0-alpha4`
import $ivy.`org.scalatest::scalatest:3.0.3`
import java.io.{BufferedReader, InputStreamReader}
import java.net.URI
import java.time.{Duration, Instant}
import java.util.{Date, EnumSet}
import com.microsoft.azure.storage.{CloudStorageAccount,
StorageCredentialsAccountAndKey}
import com.microsoft.azure.storage.blob.{SharedAccessBlobPermissions,
SharedAccessBlobPolicy}
import org.apache.hadoop.conf.Configuration
import org.apache.hadoop.fs.{FileSystem, Path}
import org.apache.hadoop.fs.azure.{AzureException, NativeAzureFileSystem}
import org.scalatest.Assertions._
import resource._
// Utility implicit conversion for auto resource management.
implicit def `Closable->Resource`[T <: { def close() }]: Resource[T] = new
Resource[T] {
override def close(closable: T): Unit = closable.close()
}
// Credentials information
val ACCOUNT = "** REDACTED **"
val ACCESS_KEY = "** REDACTED **"
// We'll create two different containers, both contain a blob named "test-blob"
but with different
// contents.
val CONTAINER_A = "container-a"
val CONTAINER_B = "container-b"
val TEST_BLOB = "test-blob"
val blobClient = {
val credentials = new StorageCredentialsAccountAndKey(ACCOUNT, ACCESS_KEY)
val account = new CloudStorageAccount(credentials, /* useHttps */ true)
account.createCloudBlobClient()
}
// Generates a read-only SAS key restricted within "container-a".
val sasKeyForContainerA = {
val since = Instant.now() minus Duration.ofMinutes(10)
val duration = Duration.ofHours(1)
val policy = new SharedAccessBlobPolicy()
policy.setSharedAccessStartTime(Date.from(since))
policy.setSharedAccessExpiryTime(Date.from(since plus duration))
policy.setPermissions(EnumSet.of(
SharedAccessBlobPermissions.READ,
SharedAccessBlobPermissions.LIST
))
blobClient
.getContainerReference(CONTAINER_A)
.generateSharedAccessSignature(policy, null)
}
// Sets up testing containers and blobs using the Azure storage SDK:
//
// container-a/test-blob => "foo"
// container-b/test-blob => "bar"
{
val containerARef = blobClient.getContainerReference(CONTAINER_A)
val containerBRef = blobClient.getContainerReference(CONTAINER_B)
containerARef.createIfNotExists()
containerARef.getBlockBlobReference(TEST_BLOB).uploadText("foo")
containerBRef.createIfNotExists()
containerBRef.getBlockBlobReference(TEST_BLOB).uploadText("bar")
}
val pathA = new
Path(s"wasbs://$CONTAINER_A@$ACCOUNT.blob.core.windows.net/$TEST_BLOB")
val pathB = new
Path(s"wasbs://$CONTAINER_A@$ACCOUNT.blob.core.windows.net/$TEST_BLOB")
for {
// Creates a file system associated with "container-a".
fs <- managed {
val conf = new Configuration
conf.set("fs.wasbs.impl", classOf[NativeAzureFileSystem].getName)
conf.set(s"fs.azure.sas.$CONTAINER_A.$ACCOUNT.blob.core.windows.net",
sasKeyForContainerA)
pathA.getFileSystem(conf)
}
// Opens a reader pointing to "container-a/test-blob". We expect to get the
string "foo" written
// to this blob previously.
readerA <- managed(new BufferedReader(new InputStreamReader(fs open pathA)))
// Opens a reader pointing to "container-b/test-blob". We expect to get an
exception since the SAS
// key used to create the `FileSystem` instance is restricted to
"container-a".
readerB <- managed(new BufferedReader(new InputStreamReader(fs open pathB)))
} {
// Should get "foo"
assert(readerA.readLine() == "foo")
// Should catch an exception ...
intercept[AzureException] {
// ... but instead, we get string "foo" here, which indicates that the
readerB was reading from
// "container-a" instead of "container-b".
val contents =
[jira] [Created] (HADOOP-14699) Impersonation errors with UGI after second principal relogin
Jeff Storck created HADOOP-14699: Summary: Impersonation errors with UGI after second principal relogin Key: HADOOP-14699 URL: https://issues.apache.org/jira/browse/HADOOP-14699 Project: Hadoop Common Issue Type: Bug Components: common Affects Versions: 2.8.1, 2.7.3, 2.6.2 Reporter: Jeff Storck Multiple principals that are logged in using UGI instances that are instantiated from a UGI class loaded by the same classloader will encounter problems when the second principal attempts to relogin. An impersonation will occur and the operation attempted by the second principal after relogging in will fail. I have created a GitHub project that exhibits the impersonation error with brief instructions on how to set up for the test and run it: https://github.com/jtstorck/kerberos-examples/tree/master/hadoop/ugi-test -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14698) Make copyFromLocal's -t option available for put as well
[ https://issues.apache.org/jira/browse/HADOOP-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Bokor updated HADOOP-14698: -- Description: After HDFS-11786 copyFromLocal and put are no longer identical. I do not see any reason why not to add the new feature to put as well. Being non-identical makes the understanding/usage of command more complicated from user point of view. was: After HDFS-11786 copyFromLocal and put are no longer identical. I do not see any reason why not to add the new feature to put as well. It makes the understanding/usage of command more complicated from user point of view. > Make copyFromLocal's -t option available for put as well > > > Key: HADOOP-14698 > URL: https://issues.apache.org/jira/browse/HADOOP-14698 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Andras Bokor >Assignee: Andras Bokor > Attachments: HADOOP-14698.01.patch > > > After HDFS-11786 copyFromLocal and put are no longer identical. > I do not see any reason why not to add the new feature to put as well. > Being non-identical makes the understanding/usage of command more complicated > from user point of view. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14672) Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
[ https://issues.apache.org/jira/browse/HADOOP-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105780#comment-16105780 ] Arpit Agarwal commented on HADOOP-14672: Thank you Sean for the review. +1 for the v5 patch. [~djp], do you have any comments? I'll hold off committing until Monday. > Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, > dom, etc. > -- > > Key: HADOOP-14672 > URL: https://issues.apache.org/jira/browse/HADOOP-14672 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Bharat Viswanadham >Priority: Blocker > Attachments: HADOOP-14672.02.patch, HADOOP-14672.03.patch, > HADOOP-14672.04.patch, HADOOP-14672.05.patch, HADOOP-14672.patch > > > The shaded hadoop-client-minicluster shouldn't include any unshaded > dependencies, but we can see: javax, dom, sax, etc. are all unshaded. > CC [~busbey] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14685) Test jars to exclude from hadoop-client-minicluster jar
[
https://issues.apache.org/jira/browse/HADOOP-14685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105775#comment-16105775
]
Hadoop QA commented on HADOOP-14685:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
15s{color} | {color:green} hadoop-client-minicluster in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14685 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879439/HADOOP-14685.01.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml |
| uname | Linux bced212d0d15 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26
18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 713349a |
| Default Java | 1.8.0_131 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12891/testReport/ |
| modules | C: hadoop-client-modules/hadoop-client-minicluster U:
hadoop-client-modules/hadoop-client-minicluster |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12891/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Test jars to exclude from hadoop-client-minicluster jar
> ---
>
> Key: HADOOP-14685
> URL: https://issues.apache.org/jira/browse/HADOOP-14685
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: common
>Affects Versions: 3.0.0-beta1
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
> Attachments: HADOOP-14685.01.patch, HADOOP-14685.patch
>
>
> This jira is to discuss, what test jars to be included/excluded from
> hadoop-client-minicluster
> Jars included/excluded when building hadoop-client-minicluster
> [INFO] --- maven-shade-plugin:2.4.3:shade (default) @
> hadoop-client-minicluster ---
> [INFO] Excluding org.apache.hadoop:hadoop-client-api:jar:3.0.0-beta1-SNAPSHOT
> from the shaded jar.
> [INFO] Excluding
>
[jira] [Commented] (HADOOP-7002) Wrong description of copyFromLocal and copyToLocal in documentation
[ https://issues.apache.org/jira/browse/HADOOP-7002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105774#comment-16105774 ] Andras Bokor commented on HADOOP-7002: -- [~raviprak] Thanks for the review, good catch. They were identical when I created this JIRA. The threading feature was added later. I proposed in HADOOP-14698 to make them identical again. I am setting HADOOP-14698 as "blocked by" for now. Thanks. > Wrong description of copyFromLocal and copyToLocal in documentation > --- > > Key: HADOOP-7002 > URL: https://issues.apache.org/jira/browse/HADOOP-7002 > Project: Hadoop Common > Issue Type: Bug >Reporter: Jingguo Yao >Assignee: Andras Bokor >Priority: Minor > Attachments: HADOOP-7002.01.patch > > Original Estimate: 20m > Remaining Estimate: 20m > > The descriptions of copyFromLocal and copyToLocal are wrong. > For copyFromLocal, the documentation says "Similar to put command, except > that the source is restricted to a local file reference." But from the source > code of FsShell.java, I can see that copyFromLocal is the sames as put. > For copyToLocal, the documentation says "Similar to get command, except that > the destination is restricted to a local file reference.". But from the > source code of FsShell.java, I can see that copyToLocal is the same as get. > And this problem exist for both English and Chinese documentation. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105773#comment-16105773 ] Aaron Fabbri commented on HADOOP-13345: --- FYI I started working on merging latest trunk into HADOOP-13345 feature branch. Right now I am looking into a 400 Bad Request error that is 100% reproduction on ITestS3AContractRootDir#testListEmptyRootDirectory, on trunk. > S3Guard: Improved Consistency for S3A > - > > Key: HADOOP-13345 > URL: https://issues.apache.org/jira/browse/HADOOP-13345 > Project: Hadoop Common > Issue Type: New Feature > Components: fs/s3 >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Attachments: HADOOP-13345.prototype1.patch, s3c.001.patch, > S3C-ConsistentListingonS3-Design.pdf, S3GuardImprovedConsistencyforS3A.pdf, > S3GuardImprovedConsistencyforS3AV2.pdf > > > This issue proposes S3Guard, a new feature of S3A, to provide an option for a > stronger consistency model than what is currently offered. The solution > coordinates with a strongly consistent external store to resolve > inconsistencies caused by the S3 eventual consistency model. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13948) Create automated scripts to update LICENSE/NOTICE
[ https://issues.apache.org/jira/browse/HADOOP-13948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105770#comment-16105770 ] Andrew Wang commented on HADOOP-13948: -- Hi Xiao, do you think we can get this done still for beta1? We might need another license sweep too. > Create automated scripts to update LICENSE/NOTICE > - > > Key: HADOOP-13948 > URL: https://issues.apache.org/jira/browse/HADOOP-13948 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Reporter: Xiao Chen >Assignee: Xiao Chen >Priority: Critical > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14364) refresh changelog/release notes with newer Apache Yetus build
[
https://issues.apache.org/jira/browse/HADOOP-14364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105760#comment-16105760
]
Hadoop QA commented on HADOOP-14364:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:red}-1{color} | {color:red} @author {color} | {color:red} 0m
0s{color} | {color:red} The patch appears to contain 2 @author tags which the
community has agreed to not allow in code contributions. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 13m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
7s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 6m
1s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 44 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
2s{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 13m 59s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}111m 43s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.security.TestKDiag |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14364 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12866528/HADOOP-14364.00.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml |
| uname | Linux b4da3c6db7f3 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 746189a |
| Default Java | 1.8.0_131 |
| @author |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12887/artifact/patchprocess/author-tags.txt
|
| mvnsite |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12887/artifact/patchprocess/patch-mvnsite-root.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12887/artifact/patchprocess/whitespace-eol.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12887/artifact/patchprocess/whitespace-tabs.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12887/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12887/testReport/ |
| modules | C:
[jira] [Commented] (HADOOP-14685) Test jars to exclude from hadoop-client-minicluster jar
[
https://issues.apache.org/jira/browse/HADOOP-14685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105755#comment-16105755
]
Hadoop QA commented on HADOOP-14685:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
12s{color} | {color:green} hadoop-client-minicluster in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14685 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879437/HADOOP-14685.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml |
| uname | Linux 7e15119bbb5f 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 713349a |
| Default Java | 1.8.0_131 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12890/testReport/ |
| modules | C: hadoop-client-modules/hadoop-client-minicluster U:
hadoop-client-modules/hadoop-client-minicluster |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12890/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Test jars to exclude from hadoop-client-minicluster jar
> ---
>
> Key: HADOOP-14685
> URL: https://issues.apache.org/jira/browse/HADOOP-14685
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: common
>Affects Versions: 3.0.0-beta1
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
> Attachments: HADOOP-14685.01.patch, HADOOP-14685.patch
>
>
> This jira is to discuss, what test jars to be included/excluded from
> hadoop-client-minicluster
> Jars included/excluded when building hadoop-client-minicluster
> [INFO] --- maven-shade-plugin:2.4.3:shade (default) @
> hadoop-client-minicluster ---
> [INFO] Excluding org.apache.hadoop:hadoop-client-api:jar:3.0.0-beta1-SNAPSHOT
> from the shaded jar.
> [INFO] Excluding
>
[jira] [Updated] (HADOOP-14698) Make copyFromLocal's -t option available for put as well
[ https://issues.apache.org/jira/browse/HADOOP-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Bokor updated HADOOP-14698: -- Status: Patch Available (was: Open) > Make copyFromLocal's -t option available for put as well > > > Key: HADOOP-14698 > URL: https://issues.apache.org/jira/browse/HADOOP-14698 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Andras Bokor >Assignee: Andras Bokor > Attachments: HADOOP-14698.01.patch > > > After HDFS-11786 copyFromLocal and put are no longer identical. > I do not see any reason why not to add the new feature to put as well. > It makes the understanding/usage of command more complicated from user point > of view. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14698) Make copyFromLocal's -t option available for put as well
[ https://issues.apache.org/jira/browse/HADOOP-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Bokor updated HADOOP-14698: -- Attachment: HADOOP-14698.01.patch > Make copyFromLocal's -t option available for put as well > > > Key: HADOOP-14698 > URL: https://issues.apache.org/jira/browse/HADOOP-14698 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Andras Bokor >Assignee: Andras Bokor > Attachments: HADOOP-14698.01.patch > > > After HDFS-11786 copyFromLocal and put are no longer identical. > I do not see any reason why not to add the new feature to put as well. > It makes the understanding/usage of command more complicated from user point > of view. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14698) Make copyFromLocal's -t option available for put as well
Andras Bokor created HADOOP-14698: - Summary: Make copyFromLocal's -t option available for put as well Key: HADOOP-14698 URL: https://issues.apache.org/jira/browse/HADOOP-14698 Project: Hadoop Common Issue Type: Improvement Reporter: Andras Bokor Assignee: Andras Bokor After HDFS-11786 copyFromLocal and put are no longer identical. I do not see any reason why not to add the new feature to put as well. It makes the understanding/usage of command more complicated from user point of view. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14685) Test jars to exclude from hadoop-client-minicluster jar
[ https://issues.apache.org/jira/browse/HADOOP-14685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105736#comment-16105736 ] Bharat Viswanadham edited comment on HADOOP-14685 at 7/28/17 9:46 PM: -- [~busbey] Updated the patch. Thank you for reviewing the changes. was (Author: bharatviswa): [~busbey] Updated the patch. Thank you for review. > Test jars to exclude from hadoop-client-minicluster jar > --- > > Key: HADOOP-14685 > URL: https://issues.apache.org/jira/browse/HADOOP-14685 > Project: Hadoop Common > Issue Type: Sub-task > Components: common >Affects Versions: 3.0.0-beta1 >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham > Attachments: HADOOP-14685.01.patch, HADOOP-14685.patch > > > This jira is to discuss, what test jars to be included/excluded from > hadoop-client-minicluster > Jars included/excluded when building hadoop-client-minicluster > [INFO] --- maven-shade-plugin:2.4.3:shade (default) @ > hadoop-client-minicluster --- > [INFO] Excluding org.apache.hadoop:hadoop-client-api:jar:3.0.0-beta1-SNAPSHOT > from the shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-client-runtime:jar:3.0.0-beta1-SNAPSHOT from the > shaded jar. > [INFO] Excluding org.apache.htrace:htrace-core4:jar:4.1.0-incubating from the > shaded jar. > [INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar. > [INFO] Excluding commons-logging:commons-logging:jar:1.1.3 from the shaded > jar. > [INFO] Excluding junit:junit:jar:4.11 from the shaded jar. > [INFO] Including org.hamcrest:hamcrest-core:jar:1.3 in the shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-annotations:jar:3.0.0-beta1-SNAPSHOT from the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-minicluster:jar:3.0.0-beta1-SNAPSHOT in the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-tests:test-jar:tests:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including de.ruedigermoeller:fst:jar:2.50 in the shaded jar. > [INFO] Including com.cedarsoftware:java-util:jar:1.9.0 in the shaded jar. > [INFO] Including com.cedarsoftware:json-io:jar:2.5.1 in the shaded jar. > [INFO] Including org.apache.curator:curator-test:jar:2.12.0 in the shaded jar. > [INFO] Including org.javassist:javassist:jar:3.18.1-GA in the shaded jar. > [INFO] Including org.apache.hadoop:hadoop-hdfs:jar:3.0.0-beta1-SNAPSHOT in > the shaded jar. > [INFO] Including org.eclipse.jetty:jetty-util-ajax:jar:9.3.11.v20160721 in > the shaded jar. > [INFO] Including commons-daemon:commons-daemon:jar:1.0.13 in the shaded jar. > [INFO] Including io.netty:netty-all:jar:4.0.23.Final in the shaded jar. > [INFO] Including xerces:xercesImpl:jar:2.9.1 in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-mapreduce-client-hs:jar:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-yarn-server-timelineservice:jar:3.0.0-beta1-SNAPSHOT > from the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-common:test-jar:tests:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-mapreduce-client-jobclient:test-jar:tests:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including com.sun.jersey:jersey-core:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-client:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-json:jar:1.19 in the shaded jar. > [INFO] Including org.codehaus.jettison:jettison:jar:1.1 in the shaded jar. > [INFO] Including com.sun.xml.bind:jaxb-impl:jar:2.2.3-1 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-server:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-servlet:jar:1.19 in the shaded jar. > [INFO] Including org.eclipse.jdt:core:jar:3.1.1 in the shaded jar. > [INFO] Including net.sf.kosmosfs:kfs:jar:0.3 in the shaded jar. > [INFO] Including net.java.dev.jets3t:jets3t:jar:0.9.0 in the shaded jar. > [INFO] Including com.jamesmurty.utils:java-xmlbuilder:jar:0.4 in the shaded > jar. > [INFO] Including com.jcraft:jsch:jar:0.1.54 in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:3.0.0-beta1-SNAPSHOT in > the shaded jar. > [INFO] Including com.codahale.metrics:metrics-core:jar:3.0.1 in the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:3.0.0-beta1-SNAPSHOT in > the shaded
[jira] [Commented] (HADOOP-14685) Test jars to exclude from hadoop-client-minicluster jar
[ https://issues.apache.org/jira/browse/HADOOP-14685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105736#comment-16105736 ] Bharat Viswanadham commented on HADOOP-14685: - [~busbey] Updated the patch. Thank you for review. > Test jars to exclude from hadoop-client-minicluster jar > --- > > Key: HADOOP-14685 > URL: https://issues.apache.org/jira/browse/HADOOP-14685 > Project: Hadoop Common > Issue Type: Sub-task > Components: common >Affects Versions: 3.0.0-beta1 >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham > Attachments: HADOOP-14685.01.patch, HADOOP-14685.patch > > > This jira is to discuss, what test jars to be included/excluded from > hadoop-client-minicluster > Jars included/excluded when building hadoop-client-minicluster > [INFO] --- maven-shade-plugin:2.4.3:shade (default) @ > hadoop-client-minicluster --- > [INFO] Excluding org.apache.hadoop:hadoop-client-api:jar:3.0.0-beta1-SNAPSHOT > from the shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-client-runtime:jar:3.0.0-beta1-SNAPSHOT from the > shaded jar. > [INFO] Excluding org.apache.htrace:htrace-core4:jar:4.1.0-incubating from the > shaded jar. > [INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar. > [INFO] Excluding commons-logging:commons-logging:jar:1.1.3 from the shaded > jar. > [INFO] Excluding junit:junit:jar:4.11 from the shaded jar. > [INFO] Including org.hamcrest:hamcrest-core:jar:1.3 in the shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-annotations:jar:3.0.0-beta1-SNAPSHOT from the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-minicluster:jar:3.0.0-beta1-SNAPSHOT in the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-tests:test-jar:tests:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including de.ruedigermoeller:fst:jar:2.50 in the shaded jar. > [INFO] Including com.cedarsoftware:java-util:jar:1.9.0 in the shaded jar. > [INFO] Including com.cedarsoftware:json-io:jar:2.5.1 in the shaded jar. > [INFO] Including org.apache.curator:curator-test:jar:2.12.0 in the shaded jar. > [INFO] Including org.javassist:javassist:jar:3.18.1-GA in the shaded jar. > [INFO] Including org.apache.hadoop:hadoop-hdfs:jar:3.0.0-beta1-SNAPSHOT in > the shaded jar. > [INFO] Including org.eclipse.jetty:jetty-util-ajax:jar:9.3.11.v20160721 in > the shaded jar. > [INFO] Including commons-daemon:commons-daemon:jar:1.0.13 in the shaded jar. > [INFO] Including io.netty:netty-all:jar:4.0.23.Final in the shaded jar. > [INFO] Including xerces:xercesImpl:jar:2.9.1 in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-mapreduce-client-hs:jar:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-yarn-server-timelineservice:jar:3.0.0-beta1-SNAPSHOT > from the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-common:test-jar:tests:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-mapreduce-client-jobclient:test-jar:tests:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including com.sun.jersey:jersey-core:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-client:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-json:jar:1.19 in the shaded jar. > [INFO] Including org.codehaus.jettison:jettison:jar:1.1 in the shaded jar. > [INFO] Including com.sun.xml.bind:jaxb-impl:jar:2.2.3-1 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-server:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-servlet:jar:1.19 in the shaded jar. > [INFO] Including org.eclipse.jdt:core:jar:3.1.1 in the shaded jar. > [INFO] Including net.sf.kosmosfs:kfs:jar:0.3 in the shaded jar. > [INFO] Including net.java.dev.jets3t:jets3t:jar:0.9.0 in the shaded jar. > [INFO] Including com.jamesmurty.utils:java-xmlbuilder:jar:0.4 in the shaded > jar. > [INFO] Including com.jcraft:jsch:jar:0.1.54 in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:3.0.0-beta1-SNAPSHOT in > the shaded jar. > [INFO] Including com.codahale.metrics:metrics-core:jar:3.0.1 in the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:3.0.0-beta1-SNAPSHOT in > the shaded jar. > [INFO] Including org.eclipse.jetty:jetty-server:jar:9.3.11.v20160721 in the > shaded jar. > [INFO] Including
[jira] [Updated] (HADOOP-14685) Test jars to exclude from hadoop-client-minicluster jar
[ https://issues.apache.org/jira/browse/HADOOP-14685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HADOOP-14685: Attachment: HADOOP-14685.01.patch > Test jars to exclude from hadoop-client-minicluster jar > --- > > Key: HADOOP-14685 > URL: https://issues.apache.org/jira/browse/HADOOP-14685 > Project: Hadoop Common > Issue Type: Sub-task > Components: common >Affects Versions: 3.0.0-beta1 >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham > Attachments: HADOOP-14685.01.patch, HADOOP-14685.patch > > > This jira is to discuss, what test jars to be included/excluded from > hadoop-client-minicluster > Jars included/excluded when building hadoop-client-minicluster > [INFO] --- maven-shade-plugin:2.4.3:shade (default) @ > hadoop-client-minicluster --- > [INFO] Excluding org.apache.hadoop:hadoop-client-api:jar:3.0.0-beta1-SNAPSHOT > from the shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-client-runtime:jar:3.0.0-beta1-SNAPSHOT from the > shaded jar. > [INFO] Excluding org.apache.htrace:htrace-core4:jar:4.1.0-incubating from the > shaded jar. > [INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar. > [INFO] Excluding commons-logging:commons-logging:jar:1.1.3 from the shaded > jar. > [INFO] Excluding junit:junit:jar:4.11 from the shaded jar. > [INFO] Including org.hamcrest:hamcrest-core:jar:1.3 in the shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-annotations:jar:3.0.0-beta1-SNAPSHOT from the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-minicluster:jar:3.0.0-beta1-SNAPSHOT in the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-tests:test-jar:tests:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including de.ruedigermoeller:fst:jar:2.50 in the shaded jar. > [INFO] Including com.cedarsoftware:java-util:jar:1.9.0 in the shaded jar. > [INFO] Including com.cedarsoftware:json-io:jar:2.5.1 in the shaded jar. > [INFO] Including org.apache.curator:curator-test:jar:2.12.0 in the shaded jar. > [INFO] Including org.javassist:javassist:jar:3.18.1-GA in the shaded jar. > [INFO] Including org.apache.hadoop:hadoop-hdfs:jar:3.0.0-beta1-SNAPSHOT in > the shaded jar. > [INFO] Including org.eclipse.jetty:jetty-util-ajax:jar:9.3.11.v20160721 in > the shaded jar. > [INFO] Including commons-daemon:commons-daemon:jar:1.0.13 in the shaded jar. > [INFO] Including io.netty:netty-all:jar:4.0.23.Final in the shaded jar. > [INFO] Including xerces:xercesImpl:jar:2.9.1 in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-mapreduce-client-hs:jar:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-yarn-server-timelineservice:jar:3.0.0-beta1-SNAPSHOT > from the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-common:test-jar:tests:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-mapreduce-client-jobclient:test-jar:tests:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including com.sun.jersey:jersey-core:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-client:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-json:jar:1.19 in the shaded jar. > [INFO] Including org.codehaus.jettison:jettison:jar:1.1 in the shaded jar. > [INFO] Including com.sun.xml.bind:jaxb-impl:jar:2.2.3-1 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-server:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-servlet:jar:1.19 in the shaded jar. > [INFO] Including org.eclipse.jdt:core:jar:3.1.1 in the shaded jar. > [INFO] Including net.sf.kosmosfs:kfs:jar:0.3 in the shaded jar. > [INFO] Including net.java.dev.jets3t:jets3t:jar:0.9.0 in the shaded jar. > [INFO] Including com.jamesmurty.utils:java-xmlbuilder:jar:0.4 in the shaded > jar. > [INFO] Including com.jcraft:jsch:jar:0.1.54 in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:3.0.0-beta1-SNAPSHOT in > the shaded jar. > [INFO] Including com.codahale.metrics:metrics-core:jar:3.0.1 in the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:3.0.0-beta1-SNAPSHOT in > the shaded jar. > [INFO] Including org.eclipse.jetty:jetty-server:jar:9.3.11.v20160721 in the > shaded jar. > [INFO] Including org.eclipse.jetty:jetty-http:jar:9.3.11.v20160721 in the > shaded jar. > [INFO]
[jira] [Created] (HADOOP-14697) Analyze the properties files which need to be included/excluded from hadoop-client-runtime and hadoop-client-minicluster
Bharat Viswanadham created HADOOP-14697: --- Summary: Analyze the properties files which need to be included/excluded from hadoop-client-runtime and hadoop-client-minicluster Key: HADOOP-14697 URL: https://issues.apache.org/jira/browse/HADOOP-14697 Project: Hadoop Common Issue Type: Sub-task Reporter: Bharat Viswanadham about.html capacity-scheduler.xml catalog.cat container-log4j.properties hdfs-default.xml java.policy javaee_5.xsd javaee_6.xsd javaee_web_services_client_1_2.xsd javaee_web_services_client_1_3.xsd jdtCompilerAdapter.jar jsp_2_1.xsd jsp_2_2.xsd krb5.conf log4j.properties plugin.properties plugin.xml web-app_2_5.xsd web-app_3_0.xsd web-common_3_0.xsd xml.xsd This issue is raised from [https://issues.apache.org/jira/browse/HADOOP-14685] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14685) Test jars to exclude from hadoop-client-minicluster jar
[ https://issues.apache.org/jira/browse/HADOOP-14685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105726#comment-16105726 ] Bharat Viswanadham edited comment on HADOOP-14685 at 7/28/17 9:38 PM: -- [~busbey] Updated to exclude the testjar and testshell classes. [~arpitagarwal][~djp] Could you please help in reviewing the patch. was (Author: bharatviswa): [~busbey] Updated to exclude the testjar and testshell classes. [~arpitagarwal][~djp] Could you pleas have a look in to the patch. > Test jars to exclude from hadoop-client-minicluster jar > --- > > Key: HADOOP-14685 > URL: https://issues.apache.org/jira/browse/HADOOP-14685 > Project: Hadoop Common > Issue Type: Sub-task > Components: common >Affects Versions: 3.0.0-beta1 >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham > Attachments: HADOOP-14685.patch > > > This jira is to discuss, what test jars to be included/excluded from > hadoop-client-minicluster > Jars included/excluded when building hadoop-client-minicluster > [INFO] --- maven-shade-plugin:2.4.3:shade (default) @ > hadoop-client-minicluster --- > [INFO] Excluding org.apache.hadoop:hadoop-client-api:jar:3.0.0-beta1-SNAPSHOT > from the shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-client-runtime:jar:3.0.0-beta1-SNAPSHOT from the > shaded jar. > [INFO] Excluding org.apache.htrace:htrace-core4:jar:4.1.0-incubating from the > shaded jar. > [INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar. > [INFO] Excluding commons-logging:commons-logging:jar:1.1.3 from the shaded > jar. > [INFO] Excluding junit:junit:jar:4.11 from the shaded jar. > [INFO] Including org.hamcrest:hamcrest-core:jar:1.3 in the shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-annotations:jar:3.0.0-beta1-SNAPSHOT from the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-minicluster:jar:3.0.0-beta1-SNAPSHOT in the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-tests:test-jar:tests:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including de.ruedigermoeller:fst:jar:2.50 in the shaded jar. > [INFO] Including com.cedarsoftware:java-util:jar:1.9.0 in the shaded jar. > [INFO] Including com.cedarsoftware:json-io:jar:2.5.1 in the shaded jar. > [INFO] Including org.apache.curator:curator-test:jar:2.12.0 in the shaded jar. > [INFO] Including org.javassist:javassist:jar:3.18.1-GA in the shaded jar. > [INFO] Including org.apache.hadoop:hadoop-hdfs:jar:3.0.0-beta1-SNAPSHOT in > the shaded jar. > [INFO] Including org.eclipse.jetty:jetty-util-ajax:jar:9.3.11.v20160721 in > the shaded jar. > [INFO] Including commons-daemon:commons-daemon:jar:1.0.13 in the shaded jar. > [INFO] Including io.netty:netty-all:jar:4.0.23.Final in the shaded jar. > [INFO] Including xerces:xercesImpl:jar:2.9.1 in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-mapreduce-client-hs:jar:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-yarn-server-timelineservice:jar:3.0.0-beta1-SNAPSHOT > from the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-common:test-jar:tests:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-mapreduce-client-jobclient:test-jar:tests:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including com.sun.jersey:jersey-core:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-client:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-json:jar:1.19 in the shaded jar. > [INFO] Including org.codehaus.jettison:jettison:jar:1.1 in the shaded jar. > [INFO] Including com.sun.xml.bind:jaxb-impl:jar:2.2.3-1 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-server:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-servlet:jar:1.19 in the shaded jar. > [INFO] Including org.eclipse.jdt:core:jar:3.1.1 in the shaded jar. > [INFO] Including net.sf.kosmosfs:kfs:jar:0.3 in the shaded jar. > [INFO] Including net.java.dev.jets3t:jets3t:jar:0.9.0 in the shaded jar. > [INFO] Including com.jamesmurty.utils:java-xmlbuilder:jar:0.4 in the shaded > jar. > [INFO] Including com.jcraft:jsch:jar:0.1.54 in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:3.0.0-beta1-SNAPSHOT in > the shaded jar. > [INFO] Including com.codahale.metrics:metrics-core:jar:3.0.1 in the
[jira] [Commented] (HADOOP-14685) Test jars to exclude from hadoop-client-minicluster jar
[
https://issues.apache.org/jira/browse/HADOOP-14685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105729#comment-16105729
]
Sean Busbey commented on HADOOP-14685:
--
{code}
641 testshell/*
{code}
Should be {{}}
> Test jars to exclude from hadoop-client-minicluster jar
> ---
>
> Key: HADOOP-14685
> URL: https://issues.apache.org/jira/browse/HADOOP-14685
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: common
>Affects Versions: 3.0.0-beta1
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
> Attachments: HADOOP-14685.patch
>
>
> This jira is to discuss, what test jars to be included/excluded from
> hadoop-client-minicluster
> Jars included/excluded when building hadoop-client-minicluster
> [INFO] --- maven-shade-plugin:2.4.3:shade (default) @
> hadoop-client-minicluster ---
> [INFO] Excluding org.apache.hadoop:hadoop-client-api:jar:3.0.0-beta1-SNAPSHOT
> from the shaded jar.
> [INFO] Excluding
> org.apache.hadoop:hadoop-client-runtime:jar:3.0.0-beta1-SNAPSHOT from the
> shaded jar.
> [INFO] Excluding org.apache.htrace:htrace-core4:jar:4.1.0-incubating from the
> shaded jar.
> [INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar.
> [INFO] Excluding commons-logging:commons-logging:jar:1.1.3 from the shaded
> jar.
> [INFO] Excluding junit:junit:jar:4.11 from the shaded jar.
> [INFO] Including org.hamcrest:hamcrest-core:jar:1.3 in the shaded jar.
> [INFO] Excluding
> org.apache.hadoop:hadoop-annotations:jar:3.0.0-beta1-SNAPSHOT from the shaded
> jar.
> [INFO] Including
> org.apache.hadoop:hadoop-minicluster:jar:3.0.0-beta1-SNAPSHOT in the shaded
> jar.
> [INFO] Including
> org.apache.hadoop:hadoop-yarn-server-tests:test-jar:tests:3.0.0-beta1-SNAPSHOT
> in the shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:3.0.0-beta1-SNAPSHOT
> in the shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:3.0.0-beta1-SNAPSHOT
> in the shaded jar.
> [INFO] Including de.ruedigermoeller:fst:jar:2.50 in the shaded jar.
> [INFO] Including com.cedarsoftware:java-util:jar:1.9.0 in the shaded jar.
> [INFO] Including com.cedarsoftware:json-io:jar:2.5.1 in the shaded jar.
> [INFO] Including org.apache.curator:curator-test:jar:2.12.0 in the shaded jar.
> [INFO] Including org.javassist:javassist:jar:3.18.1-GA in the shaded jar.
> [INFO] Including org.apache.hadoop:hadoop-hdfs:jar:3.0.0-beta1-SNAPSHOT in
> the shaded jar.
> [INFO] Including org.eclipse.jetty:jetty-util-ajax:jar:9.3.11.v20160721 in
> the shaded jar.
> [INFO] Including commons-daemon:commons-daemon:jar:1.0.13 in the shaded jar.
> [INFO] Including io.netty:netty-all:jar:4.0.23.Final in the shaded jar.
> [INFO] Including xerces:xercesImpl:jar:2.9.1 in the shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-mapreduce-client-hs:jar:3.0.0-beta1-SNAPSHOT in the
> shaded jar.
> [INFO] Excluding
> org.apache.hadoop:hadoop-yarn-server-timelineservice:jar:3.0.0-beta1-SNAPSHOT
> from the shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-common:test-jar:tests:3.0.0-beta1-SNAPSHOT in the
> shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0-beta1-SNAPSHOT in the
> shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-mapreduce-client-jobclient:test-jar:tests:3.0.0-beta1-SNAPSHOT
> in the shaded jar.
> [INFO] Including com.sun.jersey:jersey-core:jar:1.19 in the shaded jar.
> [INFO] Including com.sun.jersey:jersey-client:jar:1.19 in the shaded jar.
> [INFO] Including com.sun.jersey:jersey-json:jar:1.19 in the shaded jar.
> [INFO] Including org.codehaus.jettison:jettison:jar:1.1 in the shaded jar.
> [INFO] Including com.sun.xml.bind:jaxb-impl:jar:2.2.3-1 in the shaded jar.
> [INFO] Including com.sun.jersey:jersey-server:jar:1.19 in the shaded jar.
> [INFO] Including com.sun.jersey:jersey-servlet:jar:1.19 in the shaded jar.
> [INFO] Including org.eclipse.jdt:core:jar:3.1.1 in the shaded jar.
> [INFO] Including net.sf.kosmosfs:kfs:jar:0.3 in the shaded jar.
> [INFO] Including net.java.dev.jets3t:jets3t:jar:0.9.0 in the shaded jar.
> [INFO] Including com.jamesmurty.utils:java-xmlbuilder:jar:0.4 in the shaded
> jar.
> [INFO] Including com.jcraft:jsch:jar:0.1.54 in the shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:3.0.0-beta1-SNAPSHOT in
> the shaded jar.
> [INFO] Including com.codahale.metrics:metrics-core:jar:3.0.1 in the shaded
> jar.
> [INFO] Including
> org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:3.0.0-beta1-SNAPSHOT in
> the shaded jar.
> [INFO] Including org.eclipse.jetty:jetty-server:jar:9.3.11.v20160721 in the
> shaded jar.
> [INFO] Including
[jira] [Commented] (HADOOP-14685) Test jars to exclude from hadoop-client-minicluster jar
[ https://issues.apache.org/jira/browse/HADOOP-14685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105726#comment-16105726 ] Bharat Viswanadham commented on HADOOP-14685: - [~busbey] Updated to exclude the testjar and testshell classes. [~arpitagarwal][~djp] Could you pleas have a look in to the patch. > Test jars to exclude from hadoop-client-minicluster jar > --- > > Key: HADOOP-14685 > URL: https://issues.apache.org/jira/browse/HADOOP-14685 > Project: Hadoop Common > Issue Type: Sub-task > Components: common >Affects Versions: 3.0.0-beta1 >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham > Attachments: HADOOP-14685.patch > > > This jira is to discuss, what test jars to be included/excluded from > hadoop-client-minicluster > Jars included/excluded when building hadoop-client-minicluster > [INFO] --- maven-shade-plugin:2.4.3:shade (default) @ > hadoop-client-minicluster --- > [INFO] Excluding org.apache.hadoop:hadoop-client-api:jar:3.0.0-beta1-SNAPSHOT > from the shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-client-runtime:jar:3.0.0-beta1-SNAPSHOT from the > shaded jar. > [INFO] Excluding org.apache.htrace:htrace-core4:jar:4.1.0-incubating from the > shaded jar. > [INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar. > [INFO] Excluding commons-logging:commons-logging:jar:1.1.3 from the shaded > jar. > [INFO] Excluding junit:junit:jar:4.11 from the shaded jar. > [INFO] Including org.hamcrest:hamcrest-core:jar:1.3 in the shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-annotations:jar:3.0.0-beta1-SNAPSHOT from the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-minicluster:jar:3.0.0-beta1-SNAPSHOT in the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-tests:test-jar:tests:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including de.ruedigermoeller:fst:jar:2.50 in the shaded jar. > [INFO] Including com.cedarsoftware:java-util:jar:1.9.0 in the shaded jar. > [INFO] Including com.cedarsoftware:json-io:jar:2.5.1 in the shaded jar. > [INFO] Including org.apache.curator:curator-test:jar:2.12.0 in the shaded jar. > [INFO] Including org.javassist:javassist:jar:3.18.1-GA in the shaded jar. > [INFO] Including org.apache.hadoop:hadoop-hdfs:jar:3.0.0-beta1-SNAPSHOT in > the shaded jar. > [INFO] Including org.eclipse.jetty:jetty-util-ajax:jar:9.3.11.v20160721 in > the shaded jar. > [INFO] Including commons-daemon:commons-daemon:jar:1.0.13 in the shaded jar. > [INFO] Including io.netty:netty-all:jar:4.0.23.Final in the shaded jar. > [INFO] Including xerces:xercesImpl:jar:2.9.1 in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-mapreduce-client-hs:jar:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-yarn-server-timelineservice:jar:3.0.0-beta1-SNAPSHOT > from the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-common:test-jar:tests:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-mapreduce-client-jobclient:test-jar:tests:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including com.sun.jersey:jersey-core:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-client:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-json:jar:1.19 in the shaded jar. > [INFO] Including org.codehaus.jettison:jettison:jar:1.1 in the shaded jar. > [INFO] Including com.sun.xml.bind:jaxb-impl:jar:2.2.3-1 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-server:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-servlet:jar:1.19 in the shaded jar. > [INFO] Including org.eclipse.jdt:core:jar:3.1.1 in the shaded jar. > [INFO] Including net.sf.kosmosfs:kfs:jar:0.3 in the shaded jar. > [INFO] Including net.java.dev.jets3t:jets3t:jar:0.9.0 in the shaded jar. > [INFO] Including com.jamesmurty.utils:java-xmlbuilder:jar:0.4 in the shaded > jar. > [INFO] Including com.jcraft:jsch:jar:0.1.54 in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:3.0.0-beta1-SNAPSHOT in > the shaded jar. > [INFO] Including com.codahale.metrics:metrics-core:jar:3.0.1 in the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:3.0.0-beta1-SNAPSHOT in > the shaded jar. > [INFO] Including org.eclipse.jetty:jetty-server:jar:9.3.11.v20160721 in the >
[jira] [Commented] (HADOOP-14672) Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
[ https://issues.apache.org/jira/browse/HADOOP-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105727#comment-16105727 ] Sean Busbey commented on HADOOP-14672: -- +1 (non-binding) on v5. there'll be a test added by HADOOP-14089, so I think we're fine to skip one here. > Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, > dom, etc. > -- > > Key: HADOOP-14672 > URL: https://issues.apache.org/jira/browse/HADOOP-14672 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Bharat Viswanadham >Priority: Blocker > Attachments: HADOOP-14672.02.patch, HADOOP-14672.03.patch, > HADOOP-14672.04.patch, HADOOP-14672.05.patch, HADOOP-14672.patch > > > The shaded hadoop-client-minicluster shouldn't include any unshaded > dependencies, but we can see: javax, dom, sax, etc. are all unshaded. > CC [~busbey] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14685) Test jars to exclude from hadoop-client-minicluster jar
[ https://issues.apache.org/jira/browse/HADOOP-14685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HADOOP-14685: Status: Patch Available (was: Open) > Test jars to exclude from hadoop-client-minicluster jar > --- > > Key: HADOOP-14685 > URL: https://issues.apache.org/jira/browse/HADOOP-14685 > Project: Hadoop Common > Issue Type: Sub-task > Components: common >Affects Versions: 3.0.0-beta1 >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham > Attachments: HADOOP-14685.patch > > > This jira is to discuss, what test jars to be included/excluded from > hadoop-client-minicluster > Jars included/excluded when building hadoop-client-minicluster > [INFO] --- maven-shade-plugin:2.4.3:shade (default) @ > hadoop-client-minicluster --- > [INFO] Excluding org.apache.hadoop:hadoop-client-api:jar:3.0.0-beta1-SNAPSHOT > from the shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-client-runtime:jar:3.0.0-beta1-SNAPSHOT from the > shaded jar. > [INFO] Excluding org.apache.htrace:htrace-core4:jar:4.1.0-incubating from the > shaded jar. > [INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar. > [INFO] Excluding commons-logging:commons-logging:jar:1.1.3 from the shaded > jar. > [INFO] Excluding junit:junit:jar:4.11 from the shaded jar. > [INFO] Including org.hamcrest:hamcrest-core:jar:1.3 in the shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-annotations:jar:3.0.0-beta1-SNAPSHOT from the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-minicluster:jar:3.0.0-beta1-SNAPSHOT in the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-tests:test-jar:tests:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including de.ruedigermoeller:fst:jar:2.50 in the shaded jar. > [INFO] Including com.cedarsoftware:java-util:jar:1.9.0 in the shaded jar. > [INFO] Including com.cedarsoftware:json-io:jar:2.5.1 in the shaded jar. > [INFO] Including org.apache.curator:curator-test:jar:2.12.0 in the shaded jar. > [INFO] Including org.javassist:javassist:jar:3.18.1-GA in the shaded jar. > [INFO] Including org.apache.hadoop:hadoop-hdfs:jar:3.0.0-beta1-SNAPSHOT in > the shaded jar. > [INFO] Including org.eclipse.jetty:jetty-util-ajax:jar:9.3.11.v20160721 in > the shaded jar. > [INFO] Including commons-daemon:commons-daemon:jar:1.0.13 in the shaded jar. > [INFO] Including io.netty:netty-all:jar:4.0.23.Final in the shaded jar. > [INFO] Including xerces:xercesImpl:jar:2.9.1 in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-mapreduce-client-hs:jar:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-yarn-server-timelineservice:jar:3.0.0-beta1-SNAPSHOT > from the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-common:test-jar:tests:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-mapreduce-client-jobclient:test-jar:tests:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including com.sun.jersey:jersey-core:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-client:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-json:jar:1.19 in the shaded jar. > [INFO] Including org.codehaus.jettison:jettison:jar:1.1 in the shaded jar. > [INFO] Including com.sun.xml.bind:jaxb-impl:jar:2.2.3-1 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-server:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-servlet:jar:1.19 in the shaded jar. > [INFO] Including org.eclipse.jdt:core:jar:3.1.1 in the shaded jar. > [INFO] Including net.sf.kosmosfs:kfs:jar:0.3 in the shaded jar. > [INFO] Including net.java.dev.jets3t:jets3t:jar:0.9.0 in the shaded jar. > [INFO] Including com.jamesmurty.utils:java-xmlbuilder:jar:0.4 in the shaded > jar. > [INFO] Including com.jcraft:jsch:jar:0.1.54 in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:3.0.0-beta1-SNAPSHOT in > the shaded jar. > [INFO] Including com.codahale.metrics:metrics-core:jar:3.0.1 in the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:3.0.0-beta1-SNAPSHOT in > the shaded jar. > [INFO] Including org.eclipse.jetty:jetty-server:jar:9.3.11.v20160721 in the > shaded jar. > [INFO] Including org.eclipse.jetty:jetty-http:jar:9.3.11.v20160721 in the > shaded jar. > [INFO] Including
[jira] [Updated] (HADOOP-14685) Test jars to exclude from hadoop-client-minicluster jar
[ https://issues.apache.org/jira/browse/HADOOP-14685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HADOOP-14685: Attachment: HADOOP-14685.patch > Test jars to exclude from hadoop-client-minicluster jar > --- > > Key: HADOOP-14685 > URL: https://issues.apache.org/jira/browse/HADOOP-14685 > Project: Hadoop Common > Issue Type: Sub-task > Components: common >Affects Versions: 3.0.0-beta1 >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham > Attachments: HADOOP-14685.patch > > > This jira is to discuss, what test jars to be included/excluded from > hadoop-client-minicluster > Jars included/excluded when building hadoop-client-minicluster > [INFO] --- maven-shade-plugin:2.4.3:shade (default) @ > hadoop-client-minicluster --- > [INFO] Excluding org.apache.hadoop:hadoop-client-api:jar:3.0.0-beta1-SNAPSHOT > from the shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-client-runtime:jar:3.0.0-beta1-SNAPSHOT from the > shaded jar. > [INFO] Excluding org.apache.htrace:htrace-core4:jar:4.1.0-incubating from the > shaded jar. > [INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar. > [INFO] Excluding commons-logging:commons-logging:jar:1.1.3 from the shaded > jar. > [INFO] Excluding junit:junit:jar:4.11 from the shaded jar. > [INFO] Including org.hamcrest:hamcrest-core:jar:1.3 in the shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-annotations:jar:3.0.0-beta1-SNAPSHOT from the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-minicluster:jar:3.0.0-beta1-SNAPSHOT in the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-tests:test-jar:tests:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including de.ruedigermoeller:fst:jar:2.50 in the shaded jar. > [INFO] Including com.cedarsoftware:java-util:jar:1.9.0 in the shaded jar. > [INFO] Including com.cedarsoftware:json-io:jar:2.5.1 in the shaded jar. > [INFO] Including org.apache.curator:curator-test:jar:2.12.0 in the shaded jar. > [INFO] Including org.javassist:javassist:jar:3.18.1-GA in the shaded jar. > [INFO] Including org.apache.hadoop:hadoop-hdfs:jar:3.0.0-beta1-SNAPSHOT in > the shaded jar. > [INFO] Including org.eclipse.jetty:jetty-util-ajax:jar:9.3.11.v20160721 in > the shaded jar. > [INFO] Including commons-daemon:commons-daemon:jar:1.0.13 in the shaded jar. > [INFO] Including io.netty:netty-all:jar:4.0.23.Final in the shaded jar. > [INFO] Including xerces:xercesImpl:jar:2.9.1 in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-mapreduce-client-hs:jar:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-yarn-server-timelineservice:jar:3.0.0-beta1-SNAPSHOT > from the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-common:test-jar:tests:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-mapreduce-client-jobclient:test-jar:tests:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including com.sun.jersey:jersey-core:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-client:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-json:jar:1.19 in the shaded jar. > [INFO] Including org.codehaus.jettison:jettison:jar:1.1 in the shaded jar. > [INFO] Including com.sun.xml.bind:jaxb-impl:jar:2.2.3-1 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-server:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-servlet:jar:1.19 in the shaded jar. > [INFO] Including org.eclipse.jdt:core:jar:3.1.1 in the shaded jar. > [INFO] Including net.sf.kosmosfs:kfs:jar:0.3 in the shaded jar. > [INFO] Including net.java.dev.jets3t:jets3t:jar:0.9.0 in the shaded jar. > [INFO] Including com.jamesmurty.utils:java-xmlbuilder:jar:0.4 in the shaded > jar. > [INFO] Including com.jcraft:jsch:jar:0.1.54 in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:3.0.0-beta1-SNAPSHOT in > the shaded jar. > [INFO] Including com.codahale.metrics:metrics-core:jar:3.0.1 in the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:3.0.0-beta1-SNAPSHOT in > the shaded jar. > [INFO] Including org.eclipse.jetty:jetty-server:jar:9.3.11.v20160721 in the > shaded jar. > [INFO] Including org.eclipse.jetty:jetty-http:jar:9.3.11.v20160721 in the > shaded jar. > [INFO] Including
[jira] [Commented] (HADOOP-14672) Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
[
https://issues.apache.org/jira/browse/HADOOP-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105715#comment-16105715
]
Hadoop QA commented on HADOOP-14672:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
12s{color} | {color:green} hadoop-client-minicluster in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14672 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879427/HADOOP-14672.05.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml |
| uname | Linux 992960597a0d 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 746189a |
| Default Java | 1.8.0_131 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12888/testReport/ |
| modules | C: hadoop-client-modules/hadoop-client-minicluster U:
hadoop-client-modules/hadoop-client-minicluster |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12888/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax,
> dom, etc.
> --
>
> Key: HADOOP-14672
> URL: https://issues.apache.org/jira/browse/HADOOP-14672
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Junping Du
>Assignee: Bharat Viswanadham
>Priority: Blocker
> Attachments: HADOOP-14672.02.patch, HADOOP-14672.03.patch,
> HADOOP-14672.04.patch, HADOOP-14672.05.patch, HADOOP-14672.patch
>
>
> The shaded hadoop-client-minicluster shouldn't include any unshaded
> dependencies, but we can see: javax, dom, sax, etc. are all unshaded.
> CC [~busbey]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail:
[jira] [Updated] (HADOOP-14676) Wrong default value for "fs.df.interval"
[
https://issues.apache.org/jira/browse/HADOOP-14676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiangguang zheng updated HADOOP-14676:
--
Attachment: HADOOP-14676.001.patch
> Wrong default value for "fs.df.interval"
>
>
> Key: HADOOP-14676
> URL: https://issues.apache.org/jira/browse/HADOOP-14676
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, conf, fs
>Affects Versions: 2.6.1
>Reporter: Konstantin Shvachko
>Assignee: xiangguang zheng
> Attachments: HADOOP-14676.001.patch
>
>
> According to {{core-default.xml}} the default value of {{fs.df.interval = 60
> sec}}. But the implementation of {{DF}} substitutes 3 sec instead. The
> problem is that {{DF}} uses outdated constant {{DF.DF_INTERVAL_DEFAULT}}
> instead of the correct one
> {{CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14676) Wrong default value for "fs.df.interval"
[
https://issues.apache.org/jira/browse/HADOOP-14676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiangguang zheng updated HADOOP-14676:
--
Attachment: (was: HADOOP-14676.000.patch)
> Wrong default value for "fs.df.interval"
>
>
> Key: HADOOP-14676
> URL: https://issues.apache.org/jira/browse/HADOOP-14676
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, conf, fs
>Affects Versions: 2.6.1
>Reporter: Konstantin Shvachko
>Assignee: xiangguang zheng
> Attachments: HADOOP-14676.001.patch
>
>
> According to {{core-default.xml}} the default value of {{fs.df.interval = 60
> sec}}. But the implementation of {{DF}} substitutes 3 sec instead. The
> problem is that {{DF}} uses outdated constant {{DF.DF_INTERVAL_DEFAULT}}
> instead of the correct one
> {{CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14643) Clean up Test(HDFS|LocalFS)FileContextMainOperations and FileContextMainOperationsBaseTest
[
https://issues.apache.org/jira/browse/HADOOP-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105678#comment-16105678
]
Ravi Prakash commented on HADOOP-14643:
---
You can remove {{unwrapException}} now. Other than that, patch looks mostly
good. Can you please try getting a +1 from HadoopQA?
> Clean up Test(HDFS|LocalFS)FileContextMainOperations and
> FileContextMainOperationsBaseTest
> --
>
> Key: HADOOP-14643
> URL: https://issues.apache.org/jira/browse/HADOOP-14643
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Andras Bokor
>Assignee: Andras Bokor
>Priority: Minor
> Attachments: HADOOP-14643.01.patch, HADOOP-14643.02.patch
>
>
> I was working with classes in summary. It's good time to clean up them as
> "Boy Scout Rule"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14672) Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
[ https://issues.apache.org/jira/browse/HADOOP-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105672#comment-16105672 ] Bharat Viswanadham commented on HADOOP-14672: - [~djp] addressed the review comments from [~busbey]. could you have a look in to the patch. > Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, > dom, etc. > -- > > Key: HADOOP-14672 > URL: https://issues.apache.org/jira/browse/HADOOP-14672 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Bharat Viswanadham >Priority: Blocker > Attachments: HADOOP-14672.02.patch, HADOOP-14672.03.patch, > HADOOP-14672.04.patch, HADOOP-14672.05.patch, HADOOP-14672.patch > > > The shaded hadoop-client-minicluster shouldn't include any unshaded > dependencies, but we can see: javax, dom, sax, etc. are all unshaded. > CC [~busbey] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14672) Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
[ https://issues.apache.org/jira/browse/HADOOP-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HADOOP-14672: Attachment: HADOOP-14672.05.patch > Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, > dom, etc. > -- > > Key: HADOOP-14672 > URL: https://issues.apache.org/jira/browse/HADOOP-14672 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Bharat Viswanadham >Priority: Blocker > Attachments: HADOOP-14672.02.patch, HADOOP-14672.03.patch, > HADOOP-14672.04.patch, HADOOP-14672.05.patch, HADOOP-14672.patch > > > The shaded hadoop-client-minicluster shouldn't include any unshaded > dependencies, but we can see: javax, dom, sax, etc. are all unshaded. > CC [~busbey] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14672) Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
[ https://issues.apache.org/jira/browse/HADOOP-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105648#comment-16105648 ] Bharat Viswanadham commented on HADOOP-14672: - [~busbey] I am leaving OfflineEditViewer packages as it is. As in future the xerces will be removed, which will not cause any issue. But for now updated the patch to exclude xerces:xercesImpl. And as this is targeted for 3.0-beta1, by that time we will have other jira in, so it will not be any issue. > Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, > dom, etc. > -- > > Key: HADOOP-14672 > URL: https://issues.apache.org/jira/browse/HADOOP-14672 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Bharat Viswanadham >Priority: Blocker > Attachments: HADOOP-14672.02.patch, HADOOP-14672.03.patch, > HADOOP-14672.04.patch, HADOOP-14672.patch > > > The shaded hadoop-client-minicluster shouldn't include any unshaded > dependencies, but we can see: javax, dom, sax, etc. are all unshaded. > CC [~busbey] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14672) Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
[ https://issues.apache.org/jira/browse/HADOOP-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105628#comment-16105628 ] Lei (Eddy) Xu commented on HADOOP-14672: Hi, [~busbey]. Filed HDFS-12221 to track OEV change. > Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, > dom, etc. > -- > > Key: HADOOP-14672 > URL: https://issues.apache.org/jira/browse/HADOOP-14672 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Bharat Viswanadham >Priority: Blocker > Attachments: HADOOP-14672.02.patch, HADOOP-14672.03.patch, > HADOOP-14672.04.patch, HADOOP-14672.patch > > > The shaded hadoop-client-minicluster shouldn't include any unshaded > dependencies, but we can see: javax, dom, sax, etc. are all unshaded. > CC [~busbey] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14672) Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
[ https://issues.apache.org/jira/browse/HADOOP-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105623#comment-16105623 ] Sean Busbey commented on HADOOP-14672: -- Right, it used the old proprietary sun XML classes. those have all been replaced now with a standard interface for the JRE. the default implementation of that in openjdk is even a fork of xerces. > Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, > dom, etc. > -- > > Key: HADOOP-14672 > URL: https://issues.apache.org/jira/browse/HADOOP-14672 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Bharat Viswanadham >Priority: Blocker > Attachments: HADOOP-14672.02.patch, HADOOP-14672.03.patch, > HADOOP-14672.04.patch, HADOOP-14672.patch > > > The shaded hadoop-client-minicluster shouldn't include any unshaded > dependencies, but we can see: javax, dom, sax, etc. are all unshaded. > CC [~busbey] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13714) Tighten up our compatibility guidelines for Hadoop 3
[ https://issues.apache.org/jira/browse/HADOOP-13714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105615#comment-16105615 ] Andrew Wang commented on HADOOP-13714: -- Ping, next steps here? Beta1 is fast approaching. > Tighten up our compatibility guidelines for Hadoop 3 > > > Key: HADOOP-13714 > URL: https://issues.apache.org/jira/browse/HADOOP-13714 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 2.7.3 >Reporter: Karthik Kambatla >Assignee: Daniel Templeton >Priority: Blocker > Attachments: HADOOP-13714.001.patch, HADOOP-13714.WIP-001.patch > > > Our current compatibility guidelines are incomplete and loose. For many > categories, we do not have a policy. It would be nice to actually define > those policies so our users know what to expect and the developers know what > releases to target their changes. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14696) common's parallel tests don't work for Windows
[
https://issues.apache.org/jira/browse/HADOOP-14696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated HADOOP-14696:
--
Description:
If hadoop-common-project/hadoop-common is run with -Pparallel-tests flag, it
fails in create-parallel-tests-dirs from the pom.xml
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-antrun-plugin:1.7:run
(create-parallel-tests-dirs) on project hadoop-common: An Ant BuildException
has occured: Directory
F:\jenkins\jenkins-slave\workspace\hadoop-trunk-win\s\hadoop-common-project\hadoop-common\jenkinsjenkins-slaveworkspacehadoop-trunk-winshadoop-common-projecthadoop-common
arget\test\data\1 creation was not successful for an unknown reason
[ERROR] around Ant part
[jira] [Updated] (HADOOP-14696) common's parallel tests don't work for Windows
[
https://issues.apache.org/jira/browse/HADOOP-14696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated HADOOP-14696:
--
Description:
If hadoop-common-project/hadoop-common is run with the -Pparallel-tests flag,
it fails in create-parallel-tests-dirs from the pom.xml
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-antrun-plugin:1.7:run
(create-parallel-tests-dirs) on project hadoop-common: An Ant BuildException
has occured: Directory
F:\jenkins\jenkins-slave\workspace\hadoop-trunk-win\s\hadoop-common-project\hadoop-common\jenkinsjenkins-slaveworkspacehadoop-trunk-winshadoop-common-projecthadoop-common
arget\test\data\1 creation was not successful for an unknown reason
[ERROR] around Ant part
[jira] [Commented] (HADOOP-14672) Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
[ https://issues.apache.org/jira/browse/HADOOP-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105608#comment-16105608 ] Bharat Viswanadham commented on HADOOP-14672: - [~busbey] previously it is used with default XML classes from JDK. Due to class package naming is different with Open JDK and IBM JDK, xerces has been introduced to use. I am updating the patch to exclude offlineEditsViewer classes and test classes. > Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, > dom, etc. > -- > > Key: HADOOP-14672 > URL: https://issues.apache.org/jira/browse/HADOOP-14672 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Bharat Viswanadham >Priority: Blocker > Attachments: HADOOP-14672.02.patch, HADOOP-14672.03.patch, > HADOOP-14672.04.patch, HADOOP-14672.patch > > > The shaded hadoop-client-minicluster shouldn't include any unshaded > dependencies, but we can see: javax, dom, sax, etc. are all unshaded. > CC [~busbey] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14696) common's parallel tests don't work for Windows
Allen Wittenauer created HADOOP-14696:
-
Summary: common's parallel tests don't work for Windows
Key: HADOOP-14696
URL: https://issues.apache.org/jira/browse/HADOOP-14696
Project: Hadoop Common
Issue Type: Test
Components: test
Affects Versions: 3.0.0-beta1
Environment: Windows
Reporter: Allen Wittenauer
create-parallel-tests-dirs in hadoop-common-project/hadoop-common/pom.xml fail
with:
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-antrun-plugin:1.7:run
(create-parallel-tests-dirs) on project hadoop-common: An Ant BuildException
has occured: Directory
F:\jenkins\jenkins-slave\workspace\hadoop-trunk-win\s\hadoop-common-project\hadoop-common\jenkinsjenkins-slaveworkspacehadoop-trunk-winshadoop-common-projecthadoop-common
arget\test\data\1 creation was not successful for an unknown reason
[ERROR] around Ant part
[jira] [Commented] (HADOOP-11790) leveldb usage should be disabled by default or smarter about platforms
[ https://issues.apache.org/jira/browse/HADOOP-11790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105603#comment-16105603 ] Allen Wittenauer commented on HADOOP-11790: --- We didn't have a viable alternative until the licensing changed on rocksdb. Now that we do, 3.0 is the time to do it. > leveldb usage should be disabled by default or smarter about platforms > -- > > Key: HADOOP-11790 > URL: https://issues.apache.org/jira/browse/HADOOP-11790 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.6.0, 3.0.0-alpha3 > Environment: * any non-x86 > * any OS that isn't Linux, OSX, Windows >Reporter: Ayappan >Priority: Blocker > > The leveldbjni artifact in maven repository has been built for only x86 > architecture. Due to which some of the testcases are failing in PowerPC. The > leveldbjni community has no plans to support other platforms [ > https://github.com/fusesource/leveldbjni/issues/54 ]. Right now , the > approach is we need to locally built leveldbjni prior to running hadoop > testcases. Pushing a PowerPC-specific leveldbjni artifact in central maven > repository and making pom.xml to pickup it up while running in PowerPC is > another option but i don't know whether this is a suitable one . Any other > alternative/solution is there ? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11790) leveldb usage should be disabled by default or smarter about platforms
[ https://issues.apache.org/jira/browse/HADOOP-11790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105598#comment-16105598 ] Andrew Wang commented on HADOOP-11790: -- Why is this a blocker? We've had multiple releases since 2015 when this was filed, trying to understand why this is escalated now. > leveldb usage should be disabled by default or smarter about platforms > -- > > Key: HADOOP-11790 > URL: https://issues.apache.org/jira/browse/HADOOP-11790 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.6.0, 3.0.0-alpha3 > Environment: * any non-x86 > * any OS that isn't Linux, OSX, Windows >Reporter: Ayappan >Priority: Blocker > > The leveldbjni artifact in maven repository has been built for only x86 > architecture. Due to which some of the testcases are failing in PowerPC. The > leveldbjni community has no plans to support other platforms [ > https://github.com/fusesource/leveldbjni/issues/54 ]. Right now , the > approach is we need to locally built leveldbjni prior to running hadoop > testcases. Pushing a PowerPC-specific leveldbjni artifact in central maven > repository and making pom.xml to pickup it up while running in PowerPC is > another option but i don't know whether this is a suitable one . Any other > alternative/solution is there ? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14672) Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
[ https://issues.apache.org/jira/browse/HADOOP-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105596#comment-16105596 ] Sean Busbey commented on HADOOP-14672: -- let me know when there's a jira. Seems like we ought to be able to just use the XML stuff that comes in the JVM now that we don't have to use really old jdks. > Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, > dom, etc. > -- > > Key: HADOOP-14672 > URL: https://issues.apache.org/jira/browse/HADOOP-14672 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Bharat Viswanadham >Priority: Blocker > Attachments: HADOOP-14672.02.patch, HADOOP-14672.03.patch, > HADOOP-14672.04.patch, HADOOP-14672.patch > > > The shaded hadoop-client-minicluster shouldn't include any unshaded > dependencies, but we can see: javax, dom, sax, etc. are all unshaded. > CC [~busbey] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14364) refresh changelog/release notes with newer Apache Yetus build
[ https://issues.apache.org/jira/browse/HADOOP-14364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105593#comment-16105593 ] Andrew Wang commented on HADOOP-14364: -- Ping, should we just get this in? > refresh changelog/release notes with newer Apache Yetus build > - > > Key: HADOOP-14364 > URL: https://issues.apache.org/jira/browse/HADOOP-14364 > Project: Hadoop Common > Issue Type: Bug > Components: build, documentation >Affects Versions: 3.0.0-alpha4 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer >Priority: Blocker > Attachments: HADOOP-14364.00.patch > > > A lot of fixes went into Apache Yetus 0.4.0 wrt releasedocs and how it's > output gets rendered with mvn site. We should re-run releasedocs for all > releases and refresh the content to use the new formatting. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14672) Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
[ https://issues.apache.org/jira/browse/HADOOP-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105590#comment-16105590 ] Lei (Eddy) Xu commented on HADOOP-14672: [~andrew.wang] it should be easy to replace xerces with another XML serializer. Will file a JIRA for it > Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, > dom, etc. > -- > > Key: HADOOP-14672 > URL: https://issues.apache.org/jira/browse/HADOOP-14672 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Bharat Viswanadham >Priority: Blocker > Attachments: HADOOP-14672.02.patch, HADOOP-14672.03.patch, > HADOOP-14672.04.patch, HADOOP-14672.patch > > > The shaded hadoop-client-minicluster shouldn't include any unshaded > dependencies, but we can see: javax, dom, sax, etc. are all unshaded. > CC [~busbey] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14676) Wrong default value for "fs.df.interval"
[
https://issues.apache.org/jira/browse/HADOOP-14676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105579#comment-16105579
]
Hadoop QA commented on HADOOP-14676:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} hadoop-common-project/hadoop-common: The patch
generated 0 new + 4 unchanged - 1 fixed = 4 total (was 5) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
36s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 59m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14676 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879404/HADOOP-14676.000.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux f081ca315932 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 746189a |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| whitespace |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12886/artifact/patchprocess/whitespace-eol.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12886/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12886/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Wrong default value for "fs.df.interval"
>
>
> Key: HADOOP-14676
> URL: https://issues.apache.org/jira/browse/HADOOP-14676
> Project: Hadoop Common
> Issue Type:
[jira] [Commented] (HADOOP-14397) Pull up the builder pattern to FileSystem and add AbstractContractCreateTest for it
[
https://issues.apache.org/jira/browse/HADOOP-14397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105574#comment-16105574
]
Hadoop QA commented on HADOOP-14397:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
19s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
40s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs-client in trunk has 2
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
26s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
5s{color} | {color:green} root: The patch generated 0 new + 116 unchanged - 2
fixed = 116 total (was 118) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 45s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
25s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 94m 5s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ha.TestZKFailoverController |
| | hadoop.security.TestKDiag |
| | hadoop.fs.TestLocalFileSystem |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14397 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879395/HADOOP-14397.003.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux a2f3fa39d14d 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 77791e4 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12885/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html
|
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12885/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
[jira] [Commented] (HADOOP-14644) javadoc:test-javadoc fails with OutOfMemoryError
[
https://issues.apache.org/jira/browse/HADOOP-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105570#comment-16105570
]
Ravi Prakash commented on HADOOP-14644:
---
FWIW, it worked without any changes for me on Linux. I'll let some Mac users
comment.
> javadoc:test-javadoc fails with OutOfMemoryError
>
>
> Key: HADOOP-14644
> URL: https://issues.apache.org/jira/browse/HADOOP-14644
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
>Reporter: Andras Bokor
>Assignee: Andras Bokor
> Attachments: HADOOP-14644.01.patch
>
>
> It's easy to reproduce:
> {code}cd hadoop-common-project/hadoop-common/
> mvn javadoc:test-javadoc{code}
> The build will fail. I tried it on OSX (2 different ones).
> {code}ERROR] javadoc: error - Java heap space
> [ERROR]
> [ERROR] Command line was:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_74.jdk/Contents/Home/jre/../bin/javadoc
> -J-Xmx512m @options @packages{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-7002) Wrong description of copyFromLocal and copyToLocal in documentation
[ https://issues.apache.org/jira/browse/HADOOP-7002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105556#comment-16105556 ] Ravi Prakash commented on HADOOP-7002: -- CopyFromLocal seems to have a thread count which is not an option in Put. https://github.com/apache/hadoop/blob/746189ad8cdf90ab35baec9364b2e02956a1e70c/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CopyCommands.java#L311 > Wrong description of copyFromLocal and copyToLocal in documentation > --- > > Key: HADOOP-7002 > URL: https://issues.apache.org/jira/browse/HADOOP-7002 > Project: Hadoop Common > Issue Type: Bug >Reporter: Jingguo Yao >Assignee: Andras Bokor >Priority: Minor > Attachments: HADOOP-7002.01.patch > > Original Estimate: 20m > Remaining Estimate: 20m > > The descriptions of copyFromLocal and copyToLocal are wrong. > For copyFromLocal, the documentation says "Similar to put command, except > that the source is restricted to a local file reference." But from the source > code of FsShell.java, I can see that copyFromLocal is the sames as put. > For copyToLocal, the documentation says "Similar to get command, except that > the destination is restricted to a local file reference.". But from the > source code of FsShell.java, I can see that copyToLocal is the same as get. > And this problem exist for both English and Chinese documentation. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14229) hadoop.security.auth_to_local example is incorrect in the documentation
[
https://issues.apache.org/jira/browse/HADOOP-14229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105535#comment-16105535
]
Andras Bokor commented on HADOOP-14229:
---
Thanks [~raviprak]!
> hadoop.security.auth_to_local example is incorrect in the documentation
> ---
>
> Key: HADOOP-14229
> URL: https://issues.apache.org/jira/browse/HADOOP-14229
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Andras Bokor
>Assignee: Andras Bokor
> Fix For: 3.0.0-beta1
>
> Attachments: HADOOP-14229.01.patch, HADOOP-14229.02.patch,
> HADOOP-14229.03.patch
>
>
> Let's see jhs as example:
> {code}RULE:[2:$1@$0](jhs/.*@.*REALM.TLD)s/.*/mapred/{code}
> That means principal has 2 components (jhs/myhost@REALM).
> The second column converts this to jhs@REALM. So the regex will not match on
> this since regex expects / in the principal.
> My suggestion is
> {code}RULE:[2:$1](jhs)s/.*/mapred/{code}
> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SecureMode.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14229) hadoop.security.auth_to_local example is incorrect in the documentation
[
https://issues.apache.org/jira/browse/HADOOP-14229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105522#comment-16105522
]
Hudson commented on HADOOP-14229:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12070 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/12070/])
HADOOP-14229. hadoop.security.auth_to_local example is incorrect in the
(raviprak: rev 746189ad8cdf90ab35baec9364b2e02956a1e70c)
* (edit) hadoop-common-project/hadoop-common/src/site/markdown/SecureMode.md
> hadoop.security.auth_to_local example is incorrect in the documentation
> ---
>
> Key: HADOOP-14229
> URL: https://issues.apache.org/jira/browse/HADOOP-14229
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Andras Bokor
>Assignee: Andras Bokor
> Fix For: 3.0.0-beta1
>
> Attachments: HADOOP-14229.01.patch, HADOOP-14229.02.patch,
> HADOOP-14229.03.patch
>
>
> Let's see jhs as example:
> {code}RULE:[2:$1@$0](jhs/.*@.*REALM.TLD)s/.*/mapred/{code}
> That means principal has 2 components (jhs/myhost@REALM).
> The second column converts this to jhs@REALM. So the regex will not match on
> this since regex expects / in the principal.
> My suggestion is
> {code}RULE:[2:$1](jhs)s/.*/mapred/{code}
> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SecureMode.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14676) Wrong default value for "fs.df.interval"
[
https://issues.apache.org/jira/browse/HADOOP-14676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiangguang zheng updated HADOOP-14676:
--
Attachment: HADOOP-14676.000.patch
> Wrong default value for "fs.df.interval"
>
>
> Key: HADOOP-14676
> URL: https://issues.apache.org/jira/browse/HADOOP-14676
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, conf, fs
>Affects Versions: 2.6.1
>Reporter: Konstantin Shvachko
>Assignee: xiangguang zheng
> Attachments: HADOOP-14676.000.patch
>
>
> According to {{core-default.xml}} the default value of {{fs.df.interval = 60
> sec}}. But the implementation of {{DF}} substitutes 3 sec instead. The
> problem is that {{DF}} uses outdated constant {{DF.DF_INTERVAL_DEFAULT}}
> instead of the correct one
> {{CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14676) Wrong default value for "fs.df.interval"
[
https://issues.apache.org/jira/browse/HADOOP-14676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiangguang zheng updated HADOOP-14676:
--
Attachment: (was: HADOOP-14676.patch)
> Wrong default value for "fs.df.interval"
>
>
> Key: HADOOP-14676
> URL: https://issues.apache.org/jira/browse/HADOOP-14676
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, conf, fs
>Affects Versions: 2.6.1
>Reporter: Konstantin Shvachko
>Assignee: xiangguang zheng
>
> According to {{core-default.xml}} the default value of {{fs.df.interval = 60
> sec}}. But the implementation of {{DF}} substitutes 3 sec instead. The
> problem is that {{DF}} uses outdated constant {{DF.DF_INTERVAL_DEFAULT}}
> instead of the correct one
> {{CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14695) Allow disabling chunked encoding on S3 client
[
https://issues.apache.org/jira/browse/HADOOP-14695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Zhuge updated HADOOP-14695:
Summary: Allow disabling chunked encoding on S3 client (was: Allow
disabling chunked encoding)
> Allow disabling chunked encoding on S3 client
> -
>
> Key: HADOOP-14695
> URL: https://issues.apache.org/jira/browse/HADOOP-14695
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/s3
>Affects Versions: 2.8.0
>Reporter: John Zhuge
>Assignee: John Zhuge
>
> [Using the Amazon S3 Adapter for Snowball
> |http://docs.aws.amazon.com/snowball/latest/ug/using-adapter.html] indicates
> that we need to disable chunked coding and set path style access.
> HADOOP-12963 enables setting path style access.
> This JIRA will enable disabling chunked encoding. A new property
> {{fs.s3a.disable.chunked.encoding}} is proposed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Closed] (HADOOP-14229) hadoop.security.auth_to_local example is incorrect in the documentation
[
https://issues.apache.org/jira/browse/HADOOP-14229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Prakash closed HADOOP-14229.
-
> hadoop.security.auth_to_local example is incorrect in the documentation
> ---
>
> Key: HADOOP-14229
> URL: https://issues.apache.org/jira/browse/HADOOP-14229
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Andras Bokor
>Assignee: Andras Bokor
> Fix For: 3.0.0-beta1
>
> Attachments: HADOOP-14229.01.patch, HADOOP-14229.02.patch,
> HADOOP-14229.03.patch
>
>
> Let's see jhs as example:
> {code}RULE:[2:$1@$0](jhs/.*@.*REALM.TLD)s/.*/mapred/{code}
> That means principal has 2 components (jhs/myhost@REALM).
> The second column converts this to jhs@REALM. So the regex will not match on
> this since regex expects / in the principal.
> My suggestion is
> {code}RULE:[2:$1](jhs)s/.*/mapred/{code}
> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SecureMode.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14229) hadoop.security.auth_to_local example is incorrect in the documentation
[
https://issues.apache.org/jira/browse/HADOOP-14229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Prakash updated HADOOP-14229:
--
Resolution: Fixed
Fix Version/s: 3.0.0-beta1
Status: Resolved (was: Patch Available)
Committed to trunk.
> hadoop.security.auth_to_local example is incorrect in the documentation
> ---
>
> Key: HADOOP-14229
> URL: https://issues.apache.org/jira/browse/HADOOP-14229
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Andras Bokor
>Assignee: Andras Bokor
> Fix For: 3.0.0-beta1
>
> Attachments: HADOOP-14229.01.patch, HADOOP-14229.02.patch,
> HADOOP-14229.03.patch
>
>
> Let's see jhs as example:
> {code}RULE:[2:$1@$0](jhs/.*@.*REALM.TLD)s/.*/mapred/{code}
> That means principal has 2 components (jhs/myhost@REALM).
> The second column converts this to jhs@REALM. So the regex will not match on
> this since regex expects / in the principal.
> My suggestion is
> {code}RULE:[2:$1](jhs)s/.*/mapred/{code}
> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SecureMode.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14229) hadoop.security.auth_to_local example is incorrect in the documentation
[
https://issues.apache.org/jira/browse/HADOOP-14229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105477#comment-16105477
]
Ravi Prakash commented on HADOOP-14229:
---
Looks good to me. +1. Committing shortly. Thank you for the contribution Andras!
> hadoop.security.auth_to_local example is incorrect in the documentation
> ---
>
> Key: HADOOP-14229
> URL: https://issues.apache.org/jira/browse/HADOOP-14229
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Andras Bokor
>Assignee: Andras Bokor
> Attachments: HADOOP-14229.01.patch, HADOOP-14229.02.patch,
> HADOOP-14229.03.patch
>
>
> Let's see jhs as example:
> {code}RULE:[2:$1@$0](jhs/.*@.*REALM.TLD)s/.*/mapred/{code}
> That means principal has 2 components (jhs/myhost@REALM).
> The second column converts this to jhs@REALM. So the regex will not match on
> this since regex expects / in the principal.
> My suggestion is
> {code}RULE:[2:$1](jhs)s/.*/mapred/{code}
> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SecureMode.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14695) Allow disabling chunked encoding
John Zhuge created HADOOP-14695:
---
Summary: Allow disabling chunked encoding
Key: HADOOP-14695
URL: https://issues.apache.org/jira/browse/HADOOP-14695
Project: Hadoop Common
Issue Type: Improvement
Components: fs/s3
Affects Versions: 2.8.0
Reporter: John Zhuge
Assignee: John Zhuge
[Using the Amazon S3 Adapter for Snowball
|http://docs.aws.amazon.com/snowball/latest/ug/using-adapter.html] indicates
that we need to disable chunked coding and set path style access.
HADOOP-12963 enables setting path style access.
This JIRA will enable disabling chunked encoding. A new property
{{fs.s3a.disable.chunked.encoding}} is proposed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14672) Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
[ https://issues.apache.org/jira/browse/HADOOP-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105466#comment-16105466 ] Andrew Wang commented on HADOOP-14672: -- [~eddyxu] how feasible do you think it is to migrate oiv off xerces? This sounds like a bad dependency that we don't want in our codebase. > Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, > dom, etc. > -- > > Key: HADOOP-14672 > URL: https://issues.apache.org/jira/browse/HADOOP-14672 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Bharat Viswanadham >Priority: Blocker > Attachments: HADOOP-14672.02.patch, HADOOP-14672.03.patch, > HADOOP-14672.04.patch, HADOOP-14672.patch > > > The shaded hadoop-client-minicluster shouldn't include any unshaded > dependencies, but we can see: javax, dom, sax, etc. are all unshaded. > CC [~busbey] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14676) Wrong default value for "fs.df.interval"
[
https://issues.apache.org/jira/browse/HADOOP-14676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105433#comment-16105433
]
Hadoop QA commented on HADOOP-14676:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
13s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 35s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 1 new + 4 unchanged - 1 fixed = 5 total (was 5) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 58s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 61m 32s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.security.TestRaceWhenRelogin |
| | hadoop.security.TestKDiag |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14676 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879382/HADOOP-14676.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 27c090312e4b 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 9ea01fd |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12884/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12884/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12884/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12884/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Wrong default value for "fs.df.interval"
>
[jira] [Commented] (HADOOP-14685) Test jars to exclude from hadoop-client-minicluster jar
[
https://issues.apache.org/jira/browse/HADOOP-14685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105434#comment-16105434
]
Sean Busbey commented on HADOOP-14685:
--
I'm saying we expressly _just_ exclude the {{testjar}} and {{testshell}}
packages, because they aren't part of the minicluster stuff we are trying to
make available to downstream. we presumably need other things that come in from
the hadoop-mapreduce-client-jobclient test-jar, so we can leave it in place.
> Test jars to exclude from hadoop-client-minicluster jar
> ---
>
> Key: HADOOP-14685
> URL: https://issues.apache.org/jira/browse/HADOOP-14685
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: common
>Affects Versions: 3.0.0-beta1
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>
> This jira is to discuss, what test jars to be included/excluded from
> hadoop-client-minicluster
> Jars included/excluded when building hadoop-client-minicluster
> [INFO] --- maven-shade-plugin:2.4.3:shade (default) @
> hadoop-client-minicluster ---
> [INFO] Excluding org.apache.hadoop:hadoop-client-api:jar:3.0.0-beta1-SNAPSHOT
> from the shaded jar.
> [INFO] Excluding
> org.apache.hadoop:hadoop-client-runtime:jar:3.0.0-beta1-SNAPSHOT from the
> shaded jar.
> [INFO] Excluding org.apache.htrace:htrace-core4:jar:4.1.0-incubating from the
> shaded jar.
> [INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar.
> [INFO] Excluding commons-logging:commons-logging:jar:1.1.3 from the shaded
> jar.
> [INFO] Excluding junit:junit:jar:4.11 from the shaded jar.
> [INFO] Including org.hamcrest:hamcrest-core:jar:1.3 in the shaded jar.
> [INFO] Excluding
> org.apache.hadoop:hadoop-annotations:jar:3.0.0-beta1-SNAPSHOT from the shaded
> jar.
> [INFO] Including
> org.apache.hadoop:hadoop-minicluster:jar:3.0.0-beta1-SNAPSHOT in the shaded
> jar.
> [INFO] Including
> org.apache.hadoop:hadoop-yarn-server-tests:test-jar:tests:3.0.0-beta1-SNAPSHOT
> in the shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:3.0.0-beta1-SNAPSHOT
> in the shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:3.0.0-beta1-SNAPSHOT
> in the shaded jar.
> [INFO] Including de.ruedigermoeller:fst:jar:2.50 in the shaded jar.
> [INFO] Including com.cedarsoftware:java-util:jar:1.9.0 in the shaded jar.
> [INFO] Including com.cedarsoftware:json-io:jar:2.5.1 in the shaded jar.
> [INFO] Including org.apache.curator:curator-test:jar:2.12.0 in the shaded jar.
> [INFO] Including org.javassist:javassist:jar:3.18.1-GA in the shaded jar.
> [INFO] Including org.apache.hadoop:hadoop-hdfs:jar:3.0.0-beta1-SNAPSHOT in
> the shaded jar.
> [INFO] Including org.eclipse.jetty:jetty-util-ajax:jar:9.3.11.v20160721 in
> the shaded jar.
> [INFO] Including commons-daemon:commons-daemon:jar:1.0.13 in the shaded jar.
> [INFO] Including io.netty:netty-all:jar:4.0.23.Final in the shaded jar.
> [INFO] Including xerces:xercesImpl:jar:2.9.1 in the shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-mapreduce-client-hs:jar:3.0.0-beta1-SNAPSHOT in the
> shaded jar.
> [INFO] Excluding
> org.apache.hadoop:hadoop-yarn-server-timelineservice:jar:3.0.0-beta1-SNAPSHOT
> from the shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-common:test-jar:tests:3.0.0-beta1-SNAPSHOT in the
> shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0-beta1-SNAPSHOT in the
> shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-mapreduce-client-jobclient:test-jar:tests:3.0.0-beta1-SNAPSHOT
> in the shaded jar.
> [INFO] Including com.sun.jersey:jersey-core:jar:1.19 in the shaded jar.
> [INFO] Including com.sun.jersey:jersey-client:jar:1.19 in the shaded jar.
> [INFO] Including com.sun.jersey:jersey-json:jar:1.19 in the shaded jar.
> [INFO] Including org.codehaus.jettison:jettison:jar:1.1 in the shaded jar.
> [INFO] Including com.sun.xml.bind:jaxb-impl:jar:2.2.3-1 in the shaded jar.
> [INFO] Including com.sun.jersey:jersey-server:jar:1.19 in the shaded jar.
> [INFO] Including com.sun.jersey:jersey-servlet:jar:1.19 in the shaded jar.
> [INFO] Including org.eclipse.jdt:core:jar:3.1.1 in the shaded jar.
> [INFO] Including net.sf.kosmosfs:kfs:jar:0.3 in the shaded jar.
> [INFO] Including net.java.dev.jets3t:jets3t:jar:0.9.0 in the shaded jar.
> [INFO] Including com.jamesmurty.utils:java-xmlbuilder:jar:0.4 in the shaded
> jar.
> [INFO] Including com.jcraft:jsch:jar:0.1.54 in the shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:3.0.0-beta1-SNAPSHOT in
> the shaded jar.
> [INFO] Including com.codahale.metrics:metrics-core:jar:3.0.1 in the shaded
> jar.
> [INFO] Including
>
[jira] [Commented] (HADOOP-14672) Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
[ https://issues.apache.org/jira/browse/HADOOP-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105419#comment-16105419 ] Sean Busbey commented on HADOOP-14672: -- okay, we need plenty of things in hadoop-hdfs, but we needn't make the fsimage xml offline tool work. I'd say exclude it entirely. If we need to also exclude the fsimage xml offline tool as well, that's okay by me. > Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, > dom, etc. > -- > > Key: HADOOP-14672 > URL: https://issues.apache.org/jira/browse/HADOOP-14672 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Bharat Viswanadham >Priority: Blocker > Attachments: HADOOP-14672.02.patch, HADOOP-14672.03.patch, > HADOOP-14672.04.patch, HADOOP-14672.patch > > > The shaded hadoop-client-minicluster shouldn't include any unshaded > dependencies, but we can see: javax, dom, sax, etc. are all unshaded. > CC [~busbey] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14672) Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
[ https://issues.apache.org/jira/browse/HADOOP-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105419#comment-16105419 ] Sean Busbey edited comment on HADOOP-14672 at 7/28/17 6:14 PM: --- okay, we need plenty of things from hadoop-hdfs, but we needn't make the fsimage xml offline tool work. I'd say exclude it entirely. If we need to also exclude the fsimage xml offline tool as well, that's okay by me. was (Author: busbey): okay, we need plenty of things in hadoop-hdfs, but we needn't make the fsimage xml offline tool work. I'd say exclude it entirely. If we need to also exclude the fsimage xml offline tool as well, that's okay by me. > Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, > dom, etc. > -- > > Key: HADOOP-14672 > URL: https://issues.apache.org/jira/browse/HADOOP-14672 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Bharat Viswanadham >Priority: Blocker > Attachments: HADOOP-14672.02.patch, HADOOP-14672.03.patch, > HADOOP-14672.04.patch, HADOOP-14672.patch > > > The shaded hadoop-client-minicluster shouldn't include any unshaded > dependencies, but we can see: javax, dom, sax, etc. are all unshaded. > CC [~busbey] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14397) Pull up the builder pattern to FileSystem and add AbstractContractCreateTest for it
[
https://issues.apache.org/jira/browse/HADOOP-14397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei (Eddy) Xu updated HADOOP-14397:
---
Attachment: HADOOP-14397.003.patch
Thanks, [~manojg]
Fixed checkstyle and the error message in IOE.
> Pull up the builder pattern to FileSystem and add AbstractContractCreateTest
> for it
> ---
>
> Key: HADOOP-14397
> URL: https://issues.apache.org/jira/browse/HADOOP-14397
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: common, fs, hdfs-client
>Affects Versions: 2.9.0, 3.0.0-alpha3
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HADOOP-14397.000.patch, HADOOP-14397.001.patch,
> HADOOP-14397.002.patch, HADOOP-14397.003.patch
>
>
> After reach the stability of the Builder APIs, we should promote the API from
> {{DistributedFileSystem}} to {{FileSystem}}, and add necessary contract tests
> to cover the API for all file systems.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14397) Pull up the builder pattern to FileSystem and add AbstractContractCreateTest for it
[
https://issues.apache.org/jira/browse/HADOOP-14397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105395#comment-16105395
]
Manoj Govindassamy commented on HADOOP-14397:
-
Thanks for working on this [~eddyxu]. v02 patch looks good. +1, pending
following items.
1. New checkstyle issues by v02 patch. Can you please rectify them ?
2. Nit: throw new IOException("Must specify either create or append"); -- does
the error message have to say create, overwrite or append ?
> Pull up the builder pattern to FileSystem and add AbstractContractCreateTest
> for it
> ---
>
> Key: HADOOP-14397
> URL: https://issues.apache.org/jira/browse/HADOOP-14397
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: common, fs, hdfs-client
>Affects Versions: 2.9.0, 3.0.0-alpha3
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HADOOP-14397.000.patch, HADOOP-14397.001.patch,
> HADOOP-14397.002.patch
>
>
> After reach the stability of the Builder APIs, we should promote the API from
> {{DistributedFileSystem}} to {{FileSystem}}, and add necessary contract tests
> to cover the API for all file systems.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14676) Wrong default value for "fs.df.interval"
[
https://issues.apache.org/jira/browse/HADOOP-14676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105363#comment-16105363
]
Erik Krogen edited comment on HADOOP-14676 at 7/28/17 5:49 PM:
---
Hey [~zhengxg3], thanks for working on this. A few things:
* When you upload the patch you should put a version name in it, e.g.
HADOOP-14676.000.patch. This helps us to track the revisions of the patch as we
go through the review-update cycle. Also, I think it might be required for
Jenkins to run the precommit build against your patch.
* I don't think {{DF.DF_INTERVAL_DEFAULT}} is used anywhere anymore, can you
confirm this and if so then delete it altogether?
* Please don't use wildcard ({{*}}) imports.
* Keep line length below 80 characters to pass checkstyle.
was (Author: xkrogen):
Hey [~zhengxg3], thanks for working on this. A few things:
* When you upload the patch you should put a version name in it, e.g.
HADOOP-14676.000.patch. This helps us to track the revisions of the patch as we
go through the review-update cycle. Also, I think it might be required for
Jenkins to run the precommit build against your patch.
* I don't think {{DF.DF_INTERVAL_DEFAULT}} is used anywhere anymore, can you
confirm this and if so then delete it altogether?
* Please don't use wildcard ({{*}}) imports.
> Wrong default value for "fs.df.interval"
>
>
> Key: HADOOP-14676
> URL: https://issues.apache.org/jira/browse/HADOOP-14676
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, conf, fs
>Affects Versions: 2.6.1
>Reporter: Konstantin Shvachko
>Assignee: xiangguang zheng
> Attachments: HADOOP-14676.patch
>
>
> According to {{core-default.xml}} the default value of {{fs.df.interval = 60
> sec}}. But the implementation of {{DF}} substitutes 3 sec instead. The
> problem is that {{DF}} uses outdated constant {{DF.DF_INTERVAL_DEFAULT}}
> instead of the correct one
> {{CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14676) Wrong default value for "fs.df.interval"
[
https://issues.apache.org/jira/browse/HADOOP-14676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105363#comment-16105363
]
Erik Krogen commented on HADOOP-14676:
--
Hey [~zhengxg3], thanks for working on this. A few things:
* When you upload the patch you should put a version name in it, e.g.
HADOOP-14676.000.patch. This helps us to track the revisions of the patch as we
go through the review-update cycle. Also, I think it might be required for
Jenkins to run the precommit build against your patch.
* I don't think {{DF.DF_INTERVAL_DEFAULT}} is used anywhere anymore, can you
confirm this and if so then delete it altogether?
* Please don't use wildcard ({{*}}) imports.
> Wrong default value for "fs.df.interval"
>
>
> Key: HADOOP-14676
> URL: https://issues.apache.org/jira/browse/HADOOP-14676
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, conf, fs
>Affects Versions: 2.6.1
>Reporter: Konstantin Shvachko
>Assignee: xiangguang zheng
> Attachments: HADOOP-14676.patch
>
>
> According to {{core-default.xml}} the default value of {{fs.df.interval = 60
> sec}}. But the implementation of {{DF}} substitutes 3 sec instead. The
> problem is that {{DF}} uses outdated constant {{DF.DF_INTERVAL_DEFAULT}}
> instead of the correct one
> {{CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14685) Test jars to exclude from hadoop-client-minicluster jar
[ https://issues.apache.org/jira/browse/HADOOP-14685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104367#comment-16104367 ] Bharat Viswanadham edited comment on HADOOP-14685 at 7/28/17 5:42 PM: -- [~busbey] The testjar and testshell package classes are from hadoop-mapreduce-client-jobclient:test-jar. So, if you are saying this artifact is to include test jars, then above classes should be included in this artifact right? Let me know if i am missing something here. was (Author: bharatviswa): [~busbey] The testjar and testshell package classes are from hadoop-mapreduce-client-jobclient:test-jar. So, if you are saying this artifact is to include test jars, then above jars should be included in this artifact right? Let me know if i am missing something here. > Test jars to exclude from hadoop-client-minicluster jar > --- > > Key: HADOOP-14685 > URL: https://issues.apache.org/jira/browse/HADOOP-14685 > Project: Hadoop Common > Issue Type: Sub-task > Components: common >Affects Versions: 3.0.0-beta1 >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham > > This jira is to discuss, what test jars to be included/excluded from > hadoop-client-minicluster > Jars included/excluded when building hadoop-client-minicluster > [INFO] --- maven-shade-plugin:2.4.3:shade (default) @ > hadoop-client-minicluster --- > [INFO] Excluding org.apache.hadoop:hadoop-client-api:jar:3.0.0-beta1-SNAPSHOT > from the shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-client-runtime:jar:3.0.0-beta1-SNAPSHOT from the > shaded jar. > [INFO] Excluding org.apache.htrace:htrace-core4:jar:4.1.0-incubating from the > shaded jar. > [INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar. > [INFO] Excluding commons-logging:commons-logging:jar:1.1.3 from the shaded > jar. > [INFO] Excluding junit:junit:jar:4.11 from the shaded jar. > [INFO] Including org.hamcrest:hamcrest-core:jar:1.3 in the shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-annotations:jar:3.0.0-beta1-SNAPSHOT from the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-minicluster:jar:3.0.0-beta1-SNAPSHOT in the shaded > jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-tests:test-jar:tests:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including de.ruedigermoeller:fst:jar:2.50 in the shaded jar. > [INFO] Including com.cedarsoftware:java-util:jar:1.9.0 in the shaded jar. > [INFO] Including com.cedarsoftware:json-io:jar:2.5.1 in the shaded jar. > [INFO] Including org.apache.curator:curator-test:jar:2.12.0 in the shaded jar. > [INFO] Including org.javassist:javassist:jar:3.18.1-GA in the shaded jar. > [INFO] Including org.apache.hadoop:hadoop-hdfs:jar:3.0.0-beta1-SNAPSHOT in > the shaded jar. > [INFO] Including org.eclipse.jetty:jetty-util-ajax:jar:9.3.11.v20160721 in > the shaded jar. > [INFO] Including commons-daemon:commons-daemon:jar:1.0.13 in the shaded jar. > [INFO] Including io.netty:netty-all:jar:4.0.23.Final in the shaded jar. > [INFO] Including xerces:xercesImpl:jar:2.9.1 in the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-mapreduce-client-hs:jar:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Excluding > org.apache.hadoop:hadoop-yarn-server-timelineservice:jar:3.0.0-beta1-SNAPSHOT > from the shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-common:test-jar:tests:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0-beta1-SNAPSHOT in the > shaded jar. > [INFO] Including > org.apache.hadoop:hadoop-mapreduce-client-jobclient:test-jar:tests:3.0.0-beta1-SNAPSHOT > in the shaded jar. > [INFO] Including com.sun.jersey:jersey-core:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-client:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-json:jar:1.19 in the shaded jar. > [INFO] Including org.codehaus.jettison:jettison:jar:1.1 in the shaded jar. > [INFO] Including com.sun.xml.bind:jaxb-impl:jar:2.2.3-1 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-server:jar:1.19 in the shaded jar. > [INFO] Including com.sun.jersey:jersey-servlet:jar:1.19 in the shaded jar. > [INFO] Including org.eclipse.jdt:core:jar:3.1.1 in the shaded jar. > [INFO] Including net.sf.kosmosfs:kfs:jar:0.3 in the shaded jar. > [INFO] Including net.java.dev.jets3t:jets3t:jar:0.9.0 in the shaded jar. > [INFO] Including com.jamesmurty.utils:java-xmlbuilder:jar:0.4 in the shaded > jar. > [INFO]
[jira] [Commented] (HADOOP-14676) Wrong default value for "fs.df.interval"
[
https://issues.apache.org/jira/browse/HADOOP-14676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105321#comment-16105321
]
xiangguang zheng commented on HADOOP-14676:
---
Please review the code, I modified the DF.java to use the
CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT instead of outdated
DF.DF_INTERVAL_DEFAULT
> Wrong default value for "fs.df.interval"
>
>
> Key: HADOOP-14676
> URL: https://issues.apache.org/jira/browse/HADOOP-14676
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, conf, fs
>Affects Versions: 2.6.1
>Reporter: Konstantin Shvachko
>Assignee: xiangguang zheng
> Attachments: HADOOP-14676.patch
>
>
> According to {{core-default.xml}} the default value of {{fs.df.interval = 60
> sec}}. But the implementation of {{DF}} substitutes 3 sec instead. The
> problem is that {{DF}} uses outdated constant {{DF.DF_INTERVAL_DEFAULT}}
> instead of the correct one
> {{CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14678) AdlFilesystem#initialize swallows exception when getting user name
[ https://issues.apache.org/jira/browse/HADOOP-14678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105322#comment-16105322 ] Hudson commented on HADOOP-14678: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12066 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12066/]) HADOOP-14678. AdlFilesystem#initialize swallows exception when getting (jzhuge: rev f735ad1b67ed82d9b11b1afd7ae39035a6aed18b) * (edit) hadoop-tools/hadoop-azure-datalake/src/main/java/org/apache/hadoop/fs/adl/AdlFileSystem.java > AdlFilesystem#initialize swallows exception when getting user name > -- > > Key: HADOOP-14678 > URL: https://issues.apache.org/jira/browse/HADOOP-14678 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/adl >Affects Versions: 3.0.0-alpha2 >Reporter: John Zhuge >Assignee: John Zhuge >Priority: Minor > Labels: supportability > Fix For: 2.9.0, 3.0.0-beta1, 2.8.3 > > Attachments: HADOOP-14678.001.patch, HADOOP-14678.002.patch > > > https://github.com/apache/hadoop/blob/5c61ad24887f76dfc5a5935b2c5dceb6bfd99417/hadoop-tools/hadoop-azure-datalake/src/main/java/org/apache/hadoop/fs/adl/AdlFileSystem.java#L122 > It should log the exception. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14676) Wrong default value for "fs.df.interval"
[
https://issues.apache.org/jira/browse/HADOOP-14676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiangguang zheng updated HADOOP-14676:
--
Attachment: HADOOP-14676.patch
> Wrong default value for "fs.df.interval"
>
>
> Key: HADOOP-14676
> URL: https://issues.apache.org/jira/browse/HADOOP-14676
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, conf, fs
>Affects Versions: 2.6.1
>Reporter: Konstantin Shvachko
>Assignee: xiangguang zheng
> Attachments: HADOOP-14676.patch
>
>
> According to {{core-default.xml}} the default value of {{fs.df.interval = 60
> sec}}. But the implementation of {{DF}} substitutes 3 sec instead. The
> problem is that {{DF}} uses outdated constant {{DF.DF_INTERVAL_DEFAULT}}
> instead of the correct one
> {{CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (HADOOP-14676) Wrong default value for "fs.df.interval"
[
https://issues.apache.org/jira/browse/HADOOP-14676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiangguang zheng updated HADOOP-14676:
--
Comment: was deleted
(was: Replace DF.DF_INTERVAL_DEFAULT with
CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT in file
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/DF.java
. [^/Users/xnzheng/Documents/opensource/hadoop/HADOOP-14676.patch])
> Wrong default value for "fs.df.interval"
>
>
> Key: HADOOP-14676
> URL: https://issues.apache.org/jira/browse/HADOOP-14676
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, conf, fs
>Affects Versions: 2.6.1
>Reporter: Konstantin Shvachko
>Assignee: xiangguang zheng
>
> According to {{core-default.xml}} the default value of {{fs.df.interval = 60
> sec}}. But the implementation of {{DF}} substitutes 3 sec instead. The
> problem is that {{DF}} uses outdated constant {{DF.DF_INTERVAL_DEFAULT}}
> instead of the correct one
> {{CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14676) Wrong default value for "fs.df.interval"
[
https://issues.apache.org/jira/browse/HADOOP-14676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiangguang zheng updated HADOOP-14676:
--
Status: Patch Available (was: Open)
Replace DF.DF_INTERVAL_DEFAULT with
CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT in file
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/DF.java
. [^/Users/xnzheng/Documents/opensource/hadoop/HADOOP-14676.patch]
> Wrong default value for "fs.df.interval"
>
>
> Key: HADOOP-14676
> URL: https://issues.apache.org/jira/browse/HADOOP-14676
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, conf, fs
>Affects Versions: 2.6.1
>Reporter: Konstantin Shvachko
>Assignee: xiangguang zheng
>
> According to {{core-default.xml}} the default value of {{fs.df.interval = 60
> sec}}. But the implementation of {{DF}} substitutes 3 sec instead. The
> problem is that {{DF}} uses outdated constant {{DF.DF_INTERVAL_DEFAULT}}
> instead of the correct one
> {{CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (HADOOP-14676) Wrong default value for "fs.df.interval"
[
https://issues.apache.org/jira/browse/HADOOP-14676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiangguang zheng updated HADOOP-14676:
--
Comment: was deleted
(was: [^HADOOP-14676.patch])
> Wrong default value for "fs.df.interval"
>
>
> Key: HADOOP-14676
> URL: https://issues.apache.org/jira/browse/HADOOP-14676
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, conf, fs
>Affects Versions: 2.6.1
>Reporter: Konstantin Shvachko
>Assignee: xiangguang zheng
>
> According to {{core-default.xml}} the default value of {{fs.df.interval = 60
> sec}}. But the implementation of {{DF}} substitutes 3 sec instead. The
> problem is that {{DF}} uses outdated constant {{DF.DF_INTERVAL_DEFAULT}}
> instead of the correct one
> {{CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14676) Wrong default value for "fs.df.interval"
[
https://issues.apache.org/jira/browse/HADOOP-14676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105307#comment-16105307
]
xiangguang zheng commented on HADOOP-14676:
---
[^HADOOP-14676.patch]
> Wrong default value for "fs.df.interval"
>
>
> Key: HADOOP-14676
> URL: https://issues.apache.org/jira/browse/HADOOP-14676
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, conf, fs
>Affects Versions: 2.6.1
>Reporter: Konstantin Shvachko
>Assignee: xiangguang zheng
>
> According to {{core-default.xml}} the default value of {{fs.df.interval = 60
> sec}}. But the implementation of {{DF}} substitutes 3 sec instead. The
> problem is that {{DF}} uses outdated constant {{DF.DF_INTERVAL_DEFAULT}}
> instead of the correct one
> {{CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14678) AdlFilesystem#initialize swallows exception when getting user name
[ https://issues.apache.org/jira/browse/HADOOP-14678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-14678: Resolution: Fixed Fix Version/s: 2.8.3 3.0.0-beta1 2.9.0 Status: Resolved (was: Patch Available) Committed to trunk, branch-2, and branch-2.8. Thanks [~vishwajeet.dusane] and [~jojochuang] for the review! > AdlFilesystem#initialize swallows exception when getting user name > -- > > Key: HADOOP-14678 > URL: https://issues.apache.org/jira/browse/HADOOP-14678 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/adl >Affects Versions: 3.0.0-alpha2 >Reporter: John Zhuge >Assignee: John Zhuge >Priority: Minor > Labels: supportability > Fix For: 2.9.0, 3.0.0-beta1, 2.8.3 > > Attachments: HADOOP-14678.001.patch, HADOOP-14678.002.patch > > > https://github.com/apache/hadoop/blob/5c61ad24887f76dfc5a5935b2c5dceb6bfd99417/hadoop-tools/hadoop-azure-datalake/src/main/java/org/apache/hadoop/fs/adl/AdlFileSystem.java#L122 > It should log the exception. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-8690) Shell may remove a file without going to trash even if skipTrash is not enabled
[
https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105167#comment-16105167
]
Andras Bokor commented on HADOOP-8690:
--
JUnit failure is unrelated.
> Shell may remove a file without going to trash even if skipTrash is not
> enabled
> ---
>
> Key: HADOOP-8690
> URL: https://issues.apache.org/jira/browse/HADOOP-8690
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha
>Reporter: Eli Collins
>Assignee: Andras Bokor
>Priority: Minor
> Attachments: HADOOP-8690.01.patch
>
>
> Delete.java contains the following comment:
> {noformat}
> // TODO: if the user wants the trash to be used but there is any
> // problem (ie. creating the trash dir, moving the item to be deleted,
> // etc), then the path will just be deleted because moveToTrash returns
> // false and it falls thru to fs.delete. this doesn't seem right
> {noformat}
> If Trash#moveToAppropriateTrash returns false FsShell will delete the path
> even if skipTrash is not enabled. The comment isn't quite right as some of
> these failure scenarios result in exceptions not a false return value, and in
> the case of an exception we don't unconditionally delete the path.
> TrashPolicy#moveToTrash states that it only returns false if the item is
> already in the trash or trash is disabled, and the expected behavior for
> these cases is to just delete the path. However
> TrashPolicyDefault#moveToTrash also returns false if there's a problem
> creating the trash directory, so for this case I think we should throw an
> exception rather than return false (and delete the path bypassing trash).
> I also question the behavior of just deleting when the item is already in the
> trash as it may have changed since previously put in the trash and not been
> checkpointed yet. Seems like in this case we should move it to trash but with
> a file name suffix.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-8690) Shell may remove a file without going to trash even if skipTrash is not enabled
[
https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105148#comment-16105148
]
Hadoop QA commented on HADOOP-8690:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 1s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 57m 11s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.security.TestKDiag |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-8690 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879359/HADOOP-8690.01.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux f9b7df93d8cd 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 369f731 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12883/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12883/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12883/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Shell may remove a file without going to trash even if skipTrash is not
> enabled
> ---
>
> Key: HADOOP-8690
> URL: https://issues.apache.org/jira/browse/HADOOP-8690
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha
>Reporter: Eli Collins
>Assignee: Andras Bokor
>
[jira] [Updated] (HADOOP-8690) Shell may remove a file without going to trash even if skipTrash is not enabled
[
https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Bokor updated HADOOP-8690:
-
Target Version/s: 3.0.0-beta1 (was: 2.0.2-alpha)
> Shell may remove a file without going to trash even if skipTrash is not
> enabled
> ---
>
> Key: HADOOP-8690
> URL: https://issues.apache.org/jira/browse/HADOOP-8690
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha
>Reporter: Eli Collins
>Assignee: Andras Bokor
>Priority: Minor
> Attachments: HADOOP-8690.01.patch
>
>
> Delete.java contains the following comment:
> {noformat}
> // TODO: if the user wants the trash to be used but there is any
> // problem (ie. creating the trash dir, moving the item to be deleted,
> // etc), then the path will just be deleted because moveToTrash returns
> // false and it falls thru to fs.delete. this doesn't seem right
> {noformat}
> If Trash#moveToAppropriateTrash returns false FsShell will delete the path
> even if skipTrash is not enabled. The comment isn't quite right as some of
> these failure scenarios result in exceptions not a false return value, and in
> the case of an exception we don't unconditionally delete the path.
> TrashPolicy#moveToTrash states that it only returns false if the item is
> already in the trash or trash is disabled, and the expected behavior for
> these cases is to just delete the path. However
> TrashPolicyDefault#moveToTrash also returns false if there's a problem
> creating the trash directory, so for this case I think we should throw an
> exception rather than return false (and delete the path bypassing trash).
> I also question the behavior of just deleting when the item is already in the
> trash as it may have changed since previously put in the trash and not been
> checkpointed yet. Seems like in this case we should move it to trash but with
> a file name suffix.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14694) Clean up TestTrash
Andras Bokor created HADOOP-14694:
-
Summary: Clean up TestTrash
Key: HADOOP-14694
URL: https://issues.apache.org/jira/browse/HADOOP-14694
Project: Hadoop Common
Issue Type: Improvement
Reporter: Andras Bokor
Assignee: Andras Bokor
Clean up {{TestTrash.java}} after HADOOP-8690 as per Boy Scout Rule.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-8690) Shell may remove a file without going to trash even if skipTrash is not enabled
[
https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Bokor updated HADOOP-8690:
-
Status: Patch Available (was: Open)
> Shell may remove a file without going to trash even if skipTrash is not
> enabled
> ---
>
> Key: HADOOP-8690
> URL: https://issues.apache.org/jira/browse/HADOOP-8690
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha
>Reporter: Eli Collins
>Assignee: Andras Bokor
>Priority: Minor
> Attachments: HADOOP-8690.01.patch
>
>
> Delete.java contains the following comment:
> {noformat}
> // TODO: if the user wants the trash to be used but there is any
> // problem (ie. creating the trash dir, moving the item to be deleted,
> // etc), then the path will just be deleted because moveToTrash returns
> // false and it falls thru to fs.delete. this doesn't seem right
> {noformat}
> If Trash#moveToAppropriateTrash returns false FsShell will delete the path
> even if skipTrash is not enabled. The comment isn't quite right as some of
> these failure scenarios result in exceptions not a false return value, and in
> the case of an exception we don't unconditionally delete the path.
> TrashPolicy#moveToTrash states that it only returns false if the item is
> already in the trash or trash is disabled, and the expected behavior for
> these cases is to just delete the path. However
> TrashPolicyDefault#moveToTrash also returns false if there's a problem
> creating the trash directory, so for this case I think we should throw an
> exception rather than return false (and delete the path bypassing trash).
> I also question the behavior of just deleting when the item is already in the
> trash as it may have changed since previously put in the trash and not been
> checkpointed yet. Seems like in this case we should move it to trash but with
> a file name suffix.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-8690) Shell may remove a file without going to trash even if skipTrash is not enabled
[
https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Bokor updated HADOOP-8690:
-
Attachment: HADOOP-8690.01.patch
> Shell may remove a file without going to trash even if skipTrash is not
> enabled
> ---
>
> Key: HADOOP-8690
> URL: https://issues.apache.org/jira/browse/HADOOP-8690
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha
>Reporter: Eli Collins
>Assignee: Andras Bokor
>Priority: Minor
> Attachments: HADOOP-8690.01.patch
>
>
> Delete.java contains the following comment:
> {noformat}
> // TODO: if the user wants the trash to be used but there is any
> // problem (ie. creating the trash dir, moving the item to be deleted,
> // etc), then the path will just be deleted because moveToTrash returns
> // false and it falls thru to fs.delete. this doesn't seem right
> {noformat}
> If Trash#moveToAppropriateTrash returns false FsShell will delete the path
> even if skipTrash is not enabled. The comment isn't quite right as some of
> these failure scenarios result in exceptions not a false return value, and in
> the case of an exception we don't unconditionally delete the path.
> TrashPolicy#moveToTrash states that it only returns false if the item is
> already in the trash or trash is disabled, and the expected behavior for
> these cases is to just delete the path. However
> TrashPolicyDefault#moveToTrash also returns false if there's a problem
> creating the trash directory, so for this case I think we should throw an
> exception rather than return false (and delete the path bypassing trash).
> I also question the behavior of just deleting when the item is already in the
> trash as it may have changed since previously put in the trash and not been
> checkpointed yet. Seems like in this case we should move it to trash but with
> a file name suffix.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-8690) Shell may remove a file without going to trash even if skipTrash is not enabled
[
https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105035#comment-16105035
]
Andras Bokor commented on HADOOP-8690:
--
[~eli],
You are right and you are not at the same time.
The first part of your description is correct. If the trash directory cannot be
created the file will be deleted without moving into the trash. Please find my
PoC below.
The second part is not true. The code you linked here is a bit misleading. I
think your understanding is that it examines whether the file to delete (from
anywhere on the filesystem) has already been in the trash or not. But in fact,
it checks that you try to delete a file from trash.
So the condition is false if you delete /user/knoguchi/abcdef recreate then
delete again. But it will be true and the method will return false if you try
to delete /user/knoguchi/.Trash/Current/user/knoguchi/abcdef file. That is a
correct behavior.
My PoC: it has 4 asserts. I assert that the:
* trash is enabled
* rm command success
* file does not exist
* file is not in the trash which means potential dataloss
{code:title=TestTrash.java}
public void testFileNotDeletedWhenTrashCannotBeCreated()
throws Exception {
Configuration conf = new Configuration();
conf.set(FS_TRASH_INTERVAL_KEY, "10");
conf.setClass("fs.file.impl", TestLFS.class, FileSystem.class);
assertTrue(new Trash(conf).isEnabled());
FileSystem fs = FileSystem.getLocal(conf);
Path myFolder = new Path(GenericTestUtils.getRandomizedTempPath());
Path myFile = new Path(myFolder, "myFile");
writeFile(fs, myFile, 10);
FsShell shell = new FsShell(conf);
int val = -1;
FsPermission origPermission =
fs.getFileStatus(TEST_DIR.getParent()).getPermission();
try {
fs.setPermission(TEST_DIR.getParent(), new FsPermission("555"));
assertEquals(shell.run(new String[]{"-rm", myFile.toString()}), 0);
assertFalse(fs.exists(myFile));
checkNotInTrash(fs, shell.getCurrentTrashDir(), myFile.toString());
} catch (Exception e) {
System.err.println("Exception raised from Trash.run " +
e.getLocalizedMessage());
} finally {
fs.setPermission(TEST_DIR.getParent(), origPermission);
fs.delete(myFolder, true);
}
}
{code}
This code passes but it should fail.
I will upload a patch.
> Shell may remove a file without going to trash even if skipTrash is not
> enabled
> ---
>
> Key: HADOOP-8690
> URL: https://issues.apache.org/jira/browse/HADOOP-8690
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha
>Reporter: Eli Collins
>Assignee: Andras Bokor
>Priority: Minor
>
> Delete.java contains the following comment:
> {noformat}
> // TODO: if the user wants the trash to be used but there is any
> // problem (ie. creating the trash dir, moving the item to be deleted,
> // etc), then the path will just be deleted because moveToTrash returns
> // false and it falls thru to fs.delete. this doesn't seem right
> {noformat}
> If Trash#moveToAppropriateTrash returns false FsShell will delete the path
> even if skipTrash is not enabled. The comment isn't quite right as some of
> these failure scenarios result in exceptions not a false return value, and in
> the case of an exception we don't unconditionally delete the path.
> TrashPolicy#moveToTrash states that it only returns false if the item is
> already in the trash or trash is disabled, and the expected behavior for
> these cases is to just delete the path. However
> TrashPolicyDefault#moveToTrash also returns false if there's a problem
> creating the trash directory, so for this case I think we should throw an
> exception rather than return false (and delete the path bypassing trash).
> I also question the behavior of just deleting when the item is already in the
> trash as it may have changed since previously put in the trash and not been
> checkpointed yet. Seems like in this case we should move it to trash but with
> a file name suffix.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14163) Refactor existing hadoop site to use more usable static website generator
[ https://issues.apache.org/jira/browse/HADOOP-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105017#comment-16105017 ] Elek, Marton commented on HADOOP-14163: --- For me everything is acceptable. I would like to get it merged anyway... Separated repo could be more cleaner. But I think it's important that we are speaking about two repositories. One for the source of the site and one for the rendered site. Spark uses git (apache/spark-website) as the source but don't know where is the generated site (was here 1 year ago: https://svn.apache.org/repos/asf/spark) IMHO for hadoop both part should be migrated to git (to the same or different repositories) to avoid using different SCM just for releasing... BTW, I refreshed the proposed branch at https://github.com/elek/hadoop-site-proposal with 2.8.1, 3.0.0-alpha4 and the new PMC/commiters. So it's ready to move to somewhere... > Refactor existing hadoop site to use more usable static website generator > - > > Key: HADOOP-14163 > URL: https://issues.apache.org/jira/browse/HADOOP-14163 > Project: Hadoop Common > Issue Type: Improvement > Components: site >Reporter: Elek, Marton >Assignee: Elek, Marton > Attachments: HADOOP-14163-001.zip, HADOOP-14163-002.zip, > HADOOP-14163-003.zip, hadoop-site.tar.gz, hadop-site-rendered.tar.gz > > > From the dev mailing list: > "Publishing can be attacked via a mix of scripting and revamping the darned > website. Forrest is pretty bad compared to the newer static site generators > out there (e.g. need to write XML instead of markdown, it's hard to review a > staging site because of all the absolute links, hard to customize, did I > mention XML?), and the look and feel of the site is from the 00s. We don't > actually have that much site content, so it should be possible to migrate to > a new system." > This issue is find a solution to migrate the old site to a new modern static > site generator using a more contemprary theme. > Goals: > * existing links should work (or at least redirected) > * It should be easy to add more content required by a release automatically > (most probably with creating separated markdown files) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14089) Shaded Hadoop client runtime includes non-shaded classes
[
https://issues.apache.org/jira/browse/HADOOP-14089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104974#comment-16104974
]
Sean Busbey commented on HADOOP-14089:
--
Okay, I'm moving forward under the assumption that the proposed test is
sufficient. if anyone disagrees, please speak up.
> Shaded Hadoop client runtime includes non-shaded classes
>
>
> Key: HADOOP-14089
> URL: https://issues.apache.org/jira/browse/HADOOP-14089
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 3.0.0-alpha2
>Reporter: David Phillips
>Assignee: Sean Busbey
>Priority: Critical
> Attachments: HADOOP-14089.WIP.0.patch
>
>
> The jar includes things like {{assets}}, {{okio}}, {{javax/annotation}},
> {{javax/ws}}, {{mozilla}}, etc.
> An easy way to verify this is to look at the contents of the jar:
> {code}
> jar tf hadoop-client-runtime-xxx.jar | sort | grep -v '^org/apache/hadoop'
> {code}
> For standard dependencies, such as the JSR 305 {{javax.annotation}} or JAX-RS
> {{javax.ws}}, it makes sense for those to be normal dependencies in the POM
> -- they are standard, so version conflicts shouldn't be a problem. The JSR
> 305 annotations can be {{true}} since they aren't needed
> at runtime (this is what Guava does).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-8690) Shell may remove a file without going to trash even if skipTrash is not enabled
[
https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Bokor reassigned HADOOP-8690:
Assignee: Andras Bokor
> Shell may remove a file without going to trash even if skipTrash is not
> enabled
> ---
>
> Key: HADOOP-8690
> URL: https://issues.apache.org/jira/browse/HADOOP-8690
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha
>Reporter: Eli Collins
>Assignee: Andras Bokor
>Priority: Minor
>
> Delete.java contains the following comment:
> {noformat}
> // TODO: if the user wants the trash to be used but there is any
> // problem (ie. creating the trash dir, moving the item to be deleted,
> // etc), then the path will just be deleted because moveToTrash returns
> // false and it falls thru to fs.delete. this doesn't seem right
> {noformat}
> If Trash#moveToAppropriateTrash returns false FsShell will delete the path
> even if skipTrash is not enabled. The comment isn't quite right as some of
> these failure scenarios result in exceptions not a false return value, and in
> the case of an exception we don't unconditionally delete the path.
> TrashPolicy#moveToTrash states that it only returns false if the item is
> already in the trash or trash is disabled, and the expected behavior for
> these cases is to just delete the path. However
> TrashPolicyDefault#moveToTrash also returns false if there's a problem
> creating the trash directory, so for this case I think we should throw an
> exception rather than return false (and delete the path bypassing trash).
> I also question the behavior of just deleting when the item is already in the
> trash as it may have changed since previously put in the trash and not been
> checkpointed yet. Seems like in this case we should move it to trash but with
> a file name suffix.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14355) Update maven-war-plugin to 3.1.0
[
https://issues.apache.org/jira/browse/HADOOP-14355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104904#comment-16104904
]
Hadoop QA commented on HADOOP-14355:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
6s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14355 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12865880/HADOOP-14355.01.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml |
| uname | Linux 802c84a44718 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 369f731 |
| Default Java | 1.8.0_131 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12881/testReport/ |
| modules | C: hadoop-project U: hadoop-project |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12881/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Update maven-war-plugin to 3.1.0
>
>
> Key: HADOOP-14355
> URL: https://issues.apache.org/jira/browse/HADOOP-14355
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
> Attachments: HADOOP-14335.testing.patch, HADOOP-14355.01.patch
>
>
> Due to MWAR-405, build fails with Java 9. The issue is fixed in maven war
> plugin 3.1.0.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14355) Update maven-war-plugin to 3.1.0
[ https://issues.apache.org/jira/browse/HADOOP-14355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104891#comment-16104891 ] Akira Ajisaka commented on HADOOP-14355: Hi [~ste...@apache.org], would you review the latest patch? > Update maven-war-plugin to 3.1.0 > > > Key: HADOOP-14355 > URL: https://issues.apache.org/jira/browse/HADOOP-14355 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Attachments: HADOOP-14335.testing.patch, HADOOP-14355.01.patch > > > Due to MWAR-405, build fails with Java 9. The issue is fixed in maven war > plugin 3.1.0. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
