[jira] [Created] (HADOOP-14076) Allow Configuration to be persisted given path to file
Ted Yu created HADOOP-14076:
---
Summary: Allow Configuration to be persisted given path to file
Key: HADOOP-14076
URL: https://issues.apache.org/jira/browse/HADOOP-14076
Project: Hadoop Common
Issue Type: Improvement
Reporter: Ted Yu
Currently Configuration has the following methods for persistence:
{code}
public void writeXml(OutputStream out) throws IOException {
public void writeXml(Writer out) throws IOException {
{code}
Adding API for persisting to file given path would be useful:
{code}
public void writeXml(String path) throws IOException {
{code}
Background: I recently worked on exporting Configuration to a file using JNI.
Without the proposed API, I resorted to some trick such as the following:
http://www.kfu.com/~nsayer/Java/jni-filedesc.html
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14075) chown doesn't work with usernames containing '\' character
Attila Bukor created HADOOP-14075:
-
Summary: chown doesn't work with usernames containing '\' character
Key: HADOOP-14075
URL: https://issues.apache.org/jira/browse/HADOOP-14075
Project: Hadoop Common
Issue Type: Bug
Reporter: Attila Bukor
Usernames containing backslash (e.g. down-level logon names) seem to work fine
with Hadoop, except for chown.
{code}
$ HADOOP_USER_NAME="FOOBAR\\testuser" hdfs dfs -mkdir /test/testfile1
$ hdfs dfs -ls /test
Found 1 items
drwxrwxr-x - FOOBAR\testuser supergroup 0 2017-02-10 12:49
/test/testfile1
$ HADOOP_USER_NAME="testuser" hdfs dfs -mkdir /test/testfile2
$ HADOOP_USER_NAME="hdfs" hdfs dfs -chown "FOOBAR\\testuser" /test/testfile2
-chown: 'FOOBAR\testuser' does not match expected pattern for [owner][:group].
Usage: hadoop fs [generic options] -chown [-R] [OWNER][:[GROUP]] PATH...
$
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/ppc64le
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/225/ [Feb 9, 2017 4:04:28 PM] (kihwal) HADOOP-14032. Reduce fair call queue priority inversion. Contributed by [Feb 9, 2017 4:47:29 PM] (kihwal) HADOOP-14034. Allow ipc layer exceptions to selectively close [Feb 9, 2017 6:23:50 PM] (wangda) YARN-5889. Improve and refactor user-limit calculation in Capacity [Feb 9, 2017 9:09:32 PM] (kasha) YARN-6112. UpdateCallDuration is calculated only when debug logging is [Feb 9, 2017 10:17:24 PM] (kihwal) HADOOP-14033. Reduce fair call queue lock contention. Contributed by [Feb 9, 2017 10:45:04 PM] (kasha) YARN-6144. FairScheduler: preempted resources can become negative. [Feb 10, 2017 6:53:18 AM] (kai.zheng) HADOOP-13768. AliyunOSS: handle the failure in the batch delete [Feb 10, 2017 10:18:46 AM] (sunilg) YARN-6118. Add javadoc for Resources.isNone. Contributed by Andres -1 overall The following subsystems voted -1: compile unit The following subsystems voted -1 but were configured to be filtered/ignored: cc javac The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.hdfs.server.blockmanagement.TestPendingReconstruction hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.yarn.server.timeline.TestRollingLevelDB hadoop.yarn.server.timeline.TestTimelineDataManager hadoop.yarn.server.timeline.TestLeveldbTimelineStore hadoop.yarn.server.timeline.webapp.TestTimelineWebServices hadoop.yarn.server.timeline.recovery.TestLeveldbTimelineStateStore hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore hadoop.yarn.server.resourcemanager.TestRMRestart hadoop.yarn.server.TestMiniYarnClusterNodeUtilization hadoop.yarn.server.TestContainerManagerSecurity hadoop.yarn.server.timeline.TestLevelDBCacheTimelineStore hadoop.yarn.server.timeline.TestOverrideTimelineStoreYarnClient hadoop.yarn.server.timeline.TestEntityGroupFSTimelineStore hadoop.yarn.applications.distributedshell.TestDistributedShell hadoop.mapred.TestShuffleHandler hadoop.mapreduce.v2.hs.TestHistoryServerLeveldbStateStoreService Timed out junit tests : org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCache compile: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/225/artifact/out/patch-compile-root.txt [128K] cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/225/artifact/out/patch-compile-root.txt [128K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/225/artifact/out/patch-compile-root.txt [128K] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/225/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [412K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/225/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt [16K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/225/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt [56K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/225/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt [72K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/225/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt [324K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/225/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timeline-pluginstorage.txt [28K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/225/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell.txt [12K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/225/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-ui.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/225/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-shuffle.txt [8.0K]
[jira] [Resolved] (HADOOP-14074) --idle_query_timeout does not work in Impala
[ https://issues.apache.org/jira/browse/HADOOP-14074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton resolved HADOOP-14074. --- Resolution: Invalid Assignee: Daniel Templeton Please file this issue at https://issues.cloudera.org/browse/IMPALA > --idle_query_timeout does not work in Impala > > > Key: HADOOP-14074 > URL: https://issues.apache.org/jira/browse/HADOOP-14074 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: site, 1.3.0 > Environment: Oracle's Big data appliance using CDH 5.7.0 >Reporter: Naveen Kumar >Assignee: Daniel Templeton > > When a user submits query from hue Impala this query resids in memory even > after user gets his results. In this case simple select * query's running > since 13hrs and using some memory. when these queries are killed manually > then the memory is freed up. To avoid this I setup --idle_query_timeout=1800 > but I dont see any change after configuring this parameter. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14074) --idle_query_timeout does not work in Impala
Naveen Kumar created HADOOP-14074: - Summary: --idle_query_timeout does not work in Impala Key: HADOOP-14074 URL: https://issues.apache.org/jira/browse/HADOOP-14074 Project: Hadoop Common Issue Type: Bug Components: build Affects Versions: site, 1.3.0 Environment: Oracle's Big data appliance using CDH 5.7.0 Reporter: Naveen Kumar When a user submits query from hue Impala this query resids in memory even after user gets his results. In this case simple select * query's running since 13hrs and using some memory. when these queries are killed manually then the memory is freed up. To avoid this I setup --idle_query_timeout=1800 but I dont see any change after configuring this parameter. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14073) Document default HttpServer2 servlets
John Zhuge created HADOOP-14073: --- Summary: Document default HttpServer2 servlets Key: HADOOP-14073 URL: https://issues.apache.org/jira/browse/HADOOP-14073 Project: Hadoop Common Issue Type: Improvement Components: documentation Affects Versions: 3.0.0-alpha3 Reporter: John Zhuge Priority: Minor Since many components (NN Web UI, YARN AHS, KMS, HttpFS, etc) now use HttpServer2 which provides default servlets /conf, /jmx, /logLevel, /stacks, /logs, and /static, it'd nice to have an independent markdown doc to describe authentication and authorization of these servlets. Related components can just link to this markdown doc. Good summary of HTTP servlets by [~yuanbo] for HADOOP-13119: https://issues.apache.org/jira/browse/HADOOP-13119?focusedCommentId=15635132=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15635132. I also made a poor attempt in https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm#L1086-L1129. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13119) Web UI error accessing links which need authorization when Kerberos
[ https://issues.apache.org/jira/browse/HADOOP-13119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HADOOP-13119. --- Resolution: Fixed Fix Version/s: 3.0.0-alpha2 Re-resolving this as this was committed to 3.0.0-alpha2 as well, despite it missing from the fix field. Since it's already been committed and released, we can't revert it or re-open this JIRA. You'll need to open a new JIRA with a code fix. > Web UI error accessing links which need authorization when Kerberos > --- > > Key: HADOOP-13119 > URL: https://issues.apache.org/jira/browse/HADOOP-13119 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.8.0, 2.7.4 >Reporter: Jeffrey E Rodriguez >Assignee: Yuanbo Liu > Labels: security > Fix For: 2.7.4, 2.8.1, 3.0.0-alpha2 > > Attachments: HADOOP-13119.001.patch, HADOOP-13119.002.patch, > HADOOP-13119.003.patch, HADOOP-13119.004.patch, HADOOP-13119.005.patch, > HADOOP-13119.005.patch, screenshot-1.png > > > User Hadoop on secure mode. > login as kdc user, kinit. > start firefox and enable Kerberos > access http://localhost:50070/logs/ > Get 403 authorization errors. > only hdfs user could access logs. > Would expect as a user to be able to web interface logs link. > Same results if using curl: > curl -v --negotiate -u tester: http://localhost:50070/logs/ > HTTP/1.1 403 User tester is unauthorized to access this page. > so: > 1. either don't show links if hdfs user is able to access. > 2. provide mechanism to add users to web application realm. > 3. note that we are pass authentication so the issue is authorization to > /logs/ > suspect that /logs/ path is secure in webdescriptor so suspect users by > default don't have access to secure paths. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/313/ [Feb 9, 2017 10:00:22 AM] (kai.zheng) HADOOP-14065. AliyunOSS: oss directory filestatus should use meta time. [Feb 9, 2017 4:04:28 PM] (kihwal) HADOOP-14032. Reduce fair call queue priority inversion. Contributed by [Feb 9, 2017 4:47:29 PM] (kihwal) HADOOP-14034. Allow ipc layer exceptions to selectively close [Feb 9, 2017 6:23:50 PM] (wangda) YARN-5889. Improve and refactor user-limit calculation in Capacity [Feb 9, 2017 9:09:32 PM] (kasha) YARN-6112. UpdateCallDuration is calculated only when debug logging is [Feb 9, 2017 10:17:24 PM] (kihwal) HADOOP-14033. Reduce fair call queue lock contention. Contributed by [Feb 9, 2017 10:45:04 PM] (kasha) YARN-6144. FairScheduler: preempted resources can become negative. -1 overall The following subsystems voted -1: asflicense unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.security.TestKDiag hadoop.hdfs.TestErasureCodeBenchmarkThroughput hadoop.yarn.server.timeline.webapp.TestTimelineWebServices hadoop.yarn.server.resourcemanager.TestRMRestart hadoop.yarn.server.TestContainerManagerSecurity hadoop.yarn.server.TestMiniYarnClusterNodeUtilization cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/313/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/313/artifact/out/diff-compile-javac-root.txt [168K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/313/artifact/out/diff-checkstyle-root.txt [16M] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/313/artifact/out/diff-patch-pylint.txt [20K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/313/artifact/out/diff-patch-shellcheck.txt [24K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/313/artifact/out/diff-patch-shelldocs.txt [12K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/313/artifact/out/whitespace-eol.txt [11M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/313/artifact/out/whitespace-tabs.txt [1.3M] javadoc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/313/artifact/out/diff-javadoc-javadoc-root.txt [2.2M] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/313/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt [128K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/313/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [160K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/313/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt [12K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/313/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt [60K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/313/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt [324K] asflicense: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/313/artifact/out/patch-asflicense-problems.txt [4.0K] Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14072) AliyunOSS: Failed to read from stream when seek beyond the download size.
Genmao Yu created HADOOP-14072: -- Summary: AliyunOSS: Failed to read from stream when seek beyond the download size. Key: HADOOP-14072 URL: https://issues.apache.org/jira/browse/HADOOP-14072 Project: Hadoop Common Issue Type: Sub-task Components: fs/oss Affects Versions: 3.0.0-alpha2 Reporter: Genmao Yu Assignee: Genmao Yu -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
