[jira] [Updated] (HADOOP-9897) Add method to get path start position without drive specifier in o.a.h.fs.Path
[
https://issues.apache.org/jira/browse/HADOOP-9897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Binglin Chang updated HADOOP-9897:
--
Attachment: HADOOP-9897.v2.patch
Thanks for the review and comments [~cnauroth]. Attach new version addressing
the comments.
> Add method to get path start position without drive specifier in
> o.a.h.fs.Path
>
>
> Key: HADOOP-9897
> URL: https://issues.apache.org/jira/browse/HADOOP-9897
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HADOOP-9897.v1.patch, HADOOP-9897.v2.patch
>
>
> There are a lot of code in Path to get start position after skipping drive
> specifier, like:
> {code}
> int start = hasWindowsDrive(uri.getPath()) ? 3 : 0;
> {code}
> Also there is a minor bug in mergePaths:
> mergePath("/", "/foo") will yield Path("//foo") which will be parsed as uri
> authority, not path.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9733) a node's stack trace output to log file should be controllable
[ https://issues.apache.org/jira/browse/HADOOP-9733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shinichi Yamashita updated HADOOP-9733: --- Status: Patch Available (was: Open) > a node's stack trace output to log file should be controllable > -- > > Key: HADOOP-9733 > URL: https://issues.apache.org/jira/browse/HADOOP-9733 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Shinichi Yamashita >Priority: Minor > Attachments: HADOOP-9733.patch > > > We can confirm the stack trace of node when we access the node's web > interface "/stacks". > And the stack trace is output not only browser but also log file of node. > Considering the cluster's management policy (e.g. log monitoring), it should > be controllable. > At now implementation, stack trace output controls HttpServer class. > If we change log level, other messages does not output by HttpServer class. > So, we control to output the stacktrace to log file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9733) a node's stack trace output to log file should be controllable
[ https://issues.apache.org/jira/browse/HADOOP-9733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shinichi Yamashita updated HADOOP-9733: --- Attachment: HADOOP-9733.patch My idea is to prepare Log class instance for stack trace at HttpServer class. I attach the improvement patch file. > a node's stack trace output to log file should be controllable > -- > > Key: HADOOP-9733 > URL: https://issues.apache.org/jira/browse/HADOOP-9733 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Shinichi Yamashita >Priority: Minor > Attachments: HADOOP-9733.patch > > > We can confirm the stack trace of node when we access the node's web > interface "/stacks". > And the stack trace is output not only browser but also log file of node. > Considering the cluster's management policy (e.g. log monitoring), it should > be controllable. > At now implementation, stack trace output controls HttpServer class. > If we change log level, other messages does not output by HttpServer class. > So, we control to output the stacktrace to log file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9733) a node's stack trace output to log file should be controllable
[ https://issues.apache.org/jira/browse/HADOOP-9733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shinichi Yamashita updated HADOOP-9733: --- Description: We can confirm the stack trace of node when we access the node's web interface "/stacks". And the stack trace is output not only browser but also log file of node. Considering the cluster's management policy (e.g. log monitoring), it should be controllable. At now implementation, stack trace output controls HttpServer class. If we change log level, other messages does not output by HttpServer class. So, we control to output the stacktrace to log file. was: "/stacks" which is a function of Web UI uses StackServlet class. StackServlet class outputs the stack traces in a screen and the log file of the node. When it outputs stack trace in log, log file size increases and becomes hard to look.Therefore it controls the output to log of the stack trace. > a node's stack trace output to log file should be controllable > -- > > Key: HADOOP-9733 > URL: https://issues.apache.org/jira/browse/HADOOP-9733 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Shinichi Yamashita >Priority: Minor > > We can confirm the stack trace of node when we access the node's web > interface "/stacks". > And the stack trace is output not only browser but also log file of node. > Considering the cluster's management policy (e.g. log monitoring), it should > be controllable. > At now implementation, stack trace output controls HttpServer class. > If we change log level, other messages does not output by HttpServer class. > So, we control to output the stacktrace to log file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9733) a node's stack trace output to log file should be controllable
[ https://issues.apache.org/jira/browse/HADOOP-9733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shinichi Yamashita updated HADOOP-9733: --- Summary: a node's stack trace output to log file should be controllable (was: Control the output to log of the stack trace) > a node's stack trace output to log file should be controllable > -- > > Key: HADOOP-9733 > URL: https://issues.apache.org/jira/browse/HADOOP-9733 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Shinichi Yamashita >Priority: Minor > > "/stacks" which is a function of Web UI uses StackServlet class. StackServlet > class outputs the stack traces in a screen and the log file of the node. > When it outputs stack trace in log, log file size increases and becomes hard > to look.Therefore it controls the output to log of the stack trace. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9883) Local mode FileNotFoundException: File does not exist
[ https://issues.apache.org/jira/browse/HADOOP-9883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abin Shahab updated HADOOP-9883: Attachment: HADOOP-9883.patch > Local mode FileNotFoundException: File does not exist > - > > Key: HADOOP-9883 > URL: https://issues.apache.org/jira/browse/HADOOP-9883 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.0.2-alpha > Environment: Centos 6.3 > Hadoop 2.0.2-alpha > Hive 0.10.0 >Reporter: Abin Shahab >Priority: Critical > Attachments: HADOOP-9883.patch, HADOOP-9883.patch, HADOOP-9883.patch, > HADOOP-9883.patch, HADOOP-9883.patch > > > Hive jobs in local mode fail with the error posted below. The jar file that's > not being found exists and has the following access: > > ls -l hive-0.10.0/lib/hive-builtins-0.10.0.jar > rw-rw-r-- 1 ashahab ashahab 3914 Dec 18 2012 > hive-0.10.0/lib/hive-builtins-0.10.0.jar > Steps to reproduce > [vcc_chaiken@HadoopDesktop0 ~]$ hive > Logging initialized using configuration in > jar:file:/opt/hive/lib/hive-common-0.10.0.jar!/hive-log4j.properties > Hive history > file=/disk1/hive/log/vcc_chaiken/hive_job_log_vcc_chaiken_201307162119_876702406.txt > hive> create database chaiken_test_00; > OK > Time taken: 1.675 seconds > hive> use chaiken_test_00; > OK > Time taken: 0.029 seconds > hive> create table chaiken_test_table(foo INT); > OK > Time taken: 0.301 seconds > hive> select count(*) from chaiken_test_table; > Total MapReduce jobs = 1 > Launching Job 1 out of 1 > Number of reduce tasks determined at compile time: 1 > In order to change the average load for a reducer (in bytes): > set hive.exec.reducers.bytes.per.reducer= > In order to limit the maximum number of reducers: > set hive.exec.reducers.max= > In order to set a constant number of reducers: > set mapred.reduce.tasks= > Starting Job = job_1373902166027_0061, Tracking URL = > http://100-01-09.sc1.verticloud.com:8088/proxy/application_1373902166027_0061/ > Kill Command = /opt/hadoop/bin/hadoop job -kill job_1373902166027_0061 > Hadoop job information for Stage-1: number of mappers: 1; number of reducers: > 1 > 2013-07-16 21:20:25,617 Stage-1 map = 0%, reduce = 0% > 2013-07-16 21:20:30,026 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 1.13 > sec > 2013-07-16 21:20:31,110 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 1.13 > sec > 2013-07-16 21:20:32,188 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 1.13 > sec > 2013-07-16 21:20:33,270 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 1.13 > sec > 2013-07-16 21:20:34,356 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 1.13 > sec > 2013-07-16 21:20:35,455 Stage-1 map = 100%, reduce = 100%, Cumulative CPU > 3.4 sec > MapReduce Total cumulative CPU time: 3 seconds 400 msec > Ended Job = job_1373902166027_0061 > MapReduce Jobs Launched: > Job 0: Map: 1 Reduce: 1 Cumulative CPU: 3.4 sec HDFS Read: 246 HDFS > Write: 2 SUCCESS > Total MapReduce CPU Time Spent: 3 seconds 400 msec > OK > 0 > Time taken: 20.627 seconds > hive> set hive.exec.mode.local.auto; > hive.exec.mode.local.auto=false > hive> set hive.exec.mode.local.auto=true; > hive> set hive.exec.mode.local.auto; > hive.exec.mode.local.auto=true > hive> select count(*) from chaiken_test_table; > Automatically selecting local only mode for query > Total MapReduce jobs = 1 > Launching Job 1 out of 1 > Number of reduce tasks determined at compile time: 1 > In order to change the average load for a reducer (in bytes): > set hive.exec.reducers.bytes.per.reducer= > In order to limit the maximum number of reducers: > set hive.exec.reducers.max= > In order to set a constant number of reducers: > set mapred.reduce.tasks= > 13/07/16 21:20:49 WARN conf.Configuration: > file:/disk1/hive/scratch/vcc_chaiken/hive_2013-07-16_21-20-47_210_4351529322776236119/-local-10002/jobconf.xml:an > attempt to override final parameter: > mapreduce.job.end-notification.max.retry.interval; Ignoring. > 13/07/16 21:20:49 WARN conf.Configuration: > file:/disk1/hive/scratch/vcc_chaiken/hive_2013-07-16_21-20-47_210_4351529322776236119/-local-10002/jobconf.xml:an > attempt to override final parameter: > mapreduce.job.end-notification.max.attempts; Ignoring. > WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use > org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files. > Execution log at: > /tmp/vcc_chaiken/vcc_chaiken_20130716212020_4db219e0-cf40-4e73-ac0d-a1d2eaca934e.log > java.io.FileNotFoundException: File does not exist: > /opt/hive/lib/hive-builtins-0.10.0.jar > at > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:782) > at > org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.ja
[jira] [Created] (HADOOP-9900) Backport HADOOP-9582(Non-existent file to "hadoop fs -conf" doesn't throw error) to branch-1
Akira AJISAKA created HADOOP-9900: - Summary: Backport HADOOP-9582(Non-existent file to "hadoop fs -conf" doesn't throw error) to branch-1 Key: HADOOP-9900 URL: https://issues.apache.org/jira/browse/HADOOP-9900 Project: Hadoop Common Issue Type: Bug Components: conf Affects Versions: 1.3.0 Reporter: Akira AJISAKA When we run : hadoop fs -conf NON_EXISTENT_FILE -ls / hadoop (trunk, branch-2, branch-0.23) throws an RuntimeException, but hadoop (branch-1) doesn't. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9774) RawLocalFileSystem.listStatus() return absolute paths when input path is relative on Windows
[ https://issues.apache.org/jira/browse/HADOOP-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749060#comment-13749060 ] shanyu zhao commented on HADOOP-9774: - Hi Ivan, would you please run that patch against hdfs, mapreduce and a few other projects? I'm still not able to run unit tests even WITHOUT this patch - I got the VM crash error here and there, so I even couldn't finish one project. This is completely irrelevant to this bug. I even created a bug for the VM crash - HADOOP-9896. > RawLocalFileSystem.listStatus() return absolute paths when input path is > relative on Windows > > > Key: HADOOP-9774 > URL: https://issues.apache.org/jira/browse/HADOOP-9774 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 3.0.0, 2.1.0-beta >Reporter: shanyu zhao >Assignee: shanyu zhao > Attachments: HADOOP-9774-2.patch, HADOOP-9774-3.patch, > HADOOP-9774-4.patch, HADOOP-9774-5.patch, HADOOP-9774.patch > > > On Windows, when using RawLocalFileSystem.listStatus() to enumerate a > relative path (without drive spec), e.g., "file:///mydata", the resulting > paths become absolute paths, e.g., ["file://E:/mydata/t1.txt", > "file://E:/mydata/t2.txt"...]. > Note that if we use it to enumerate an absolute path, e.g., > "file://E:/mydata" then the we get the same results as above. > This breaks some hive unit tests which uses local file system to simulate > HDFS when testing, therefore the drive spec is removed. Then after > listStatus() the path is changed to absolute path, hive failed to find the > path in its map reduce job. > You'll see the following exception: > [junit] java.io.IOException: cannot find dir = > pfile:/E:/GitHub/hive-monarch/build/ql/test/data/warehouse/src/kv1.txt in > pathToPartitionInfo: > [pfile:/GitHub/hive-monarch/build/ql/test/data/warehouse/src] > [junit] at > org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getPartitionDescFromPathRecursively(HiveFileFormatUtils.java:298) > This problem is introduced by this JIRA: > HADOOP-8962 > Prior to the fix for HADOOP-8962 (merged in 0.23.5), the resulting paths are > relative paths if the parent paths are relative, e.g., > ["file:///mydata/t1.txt", "file:///mydata/t2.txt"...] > This behavior change is a side effect of the fix in HADOOP-8962, not an > intended change. The resulting behavior, even though is legitimate from a > function point of view, break consistency from the caller's point of view. > When the caller use a relative path (without drive spec) to do listStatus() > the resulting path should be relative. Therefore, I think this should be > fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9896) TestIPC fail with VM crash or System.exit
[ https://issues.apache.org/jira/browse/HADOOP-9896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749058#comment-13749058 ] shanyu zhao commented on HADOOP-9896: - More information. I was using Hyper-V on Windows 8 running Ubuntu 12.04 VM. I found out that it was due to TestIPC#testSocketLeaks got stuck. However, if I just run that one test case, it will pass for me. But if I run all TestIPC unit tests, it will get stuck. > TestIPC fail with VM crash or System.exit > - > > Key: HADOOP-9896 > URL: https://issues.apache.org/jira/browse/HADOOP-9896 > Project: Hadoop Common > Issue Type: Bug > Components: ipc >Affects Versions: 2.0.5-alpha >Reporter: shanyu zhao > Attachments: org.apache.hadoop.ipc.TestIPC-output.txt > > > I'm running hadoop unit tests on a Ubuntu 12.04 virtual machine, every time I > try to run all unit tests with command "mvn test", the TestIPC unit test will > fail, the console will show "The forked VM terminated without saying properly > goodbye. VM crash or System.exit called?" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9897) Add method to get path start position without drive specifier in o.a.h.fs.Path
[
https://issues.apache.org/jira/browse/HADOOP-9897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13748911#comment-13748911
]
Chris Nauroth commented on HADOOP-9897:
---
It looks like this patch won't work now that HADOOP-9887 has been committed.
That patch added another call to {{Path#isWindowsAbsolutePath}}.
This is a public static method on a {{Public}} and {{Stable}} annotated class,
so changing this method signature would be backwards-incompatible. A safer
change would be to handle this just as an internal refactoring within {{Path}},
and keep the public method signatures the same.
> Add method to get path start position without drive specifier in
> o.a.h.fs.Path
>
>
> Key: HADOOP-9897
> URL: https://issues.apache.org/jira/browse/HADOOP-9897
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HADOOP-9897.v1.patch
>
>
> There are a lot of code in Path to get start position after skipping drive
> specifier, like:
> {code}
> int start = hasWindowsDrive(uri.getPath()) ? 3 : 0;
> {code}
> Also there is a minor bug in mergePaths:
> mergePath("/", "/foo") will yield Path("//foo") which will be parsed as uri
> authority, not path.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9889) Refresh the Krb5 configuration when creating a new kdc in Hadoop-MiniKDC
[ https://issues.apache.org/jira/browse/HADOOP-9889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated HADOOP-9889: Attachment: HAOOP-9889.patch just trigger the jenkins > Refresh the Krb5 configuration when creating a new kdc in Hadoop-MiniKDC > > > Key: HADOOP-9889 > URL: https://issues.apache.org/jira/browse/HADOOP-9889 > Project: Hadoop Common > Issue Type: Bug >Reporter: Wei Yan >Assignee: Wei Yan > Attachments: HAOOP-9889.patch, HAOOP-9889.patch > > > Krb5 Config uses a singleton and once initialized it does not refresh > automatically. Without refresh, there are failures if you are using MiniKDCs > with different configurations (such as different realms) within the same test > run or if the Krb5 Config singleton is called before the MiniKDC is started > for the first time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8704) add request logging to jetty/httpserver
[ https://issues.apache.org/jira/browse/HADOOP-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13748740#comment-13748740 ] Kihwal Lee commented on HADOOP-8704: bq. Does this mean failed requests won't get logged? A better question will be: does any handler call setHandled(true) in any case before reaching the log handler? > add request logging to jetty/httpserver > --- > > Key: HADOOP-8704 > URL: https://issues.apache.org/jira/browse/HADOOP-8704 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Thomas Graves >Assignee: Jonathan Eagles > Attachments: HADOOP-8704.patch > > > We have been requested to log all the requests coming into Jetty/HttpServer > for security and auditing purposes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8704) add request logging to jetty/httpserver
[
https://issues.apache.org/jira/browse/HADOOP-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13748726#comment-13748726
]
Kihwal Lee commented on HADOOP-8704:
{noformat} handlers.setHandlers(new Handler[] {contexts, requestLogHandler});
{noformat}
Does this mean failed requests won't get logged?
> add request logging to jetty/httpserver
> ---
>
> Key: HADOOP-8704
> URL: https://issues.apache.org/jira/browse/HADOOP-8704
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 0.23.3, 2.0.0-alpha
>Reporter: Thomas Graves
>Assignee: Jonathan Eagles
> Attachments: HADOOP-8704.patch
>
>
> We have been requested to log all the requests coming into Jetty/HttpServer
> for security and auditing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9887) globStatus does not correctly handle paths starting with a drive spec on Windows
[
https://issues.apache.org/jira/browse/HADOOP-9887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13748588#comment-13748588
]
Hudson commented on HADOOP-9887:
FAILURE: Integrated in Hadoop-Mapreduce-trunk #1527 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1527/])
HADOOP-9887. globStatus does not correctly handle paths starting with a drive
spec on Windows. Contributed by Chuan Liu. (cnauroth:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1516608)
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java
> globStatus does not correctly handle paths starting with a drive spec on
> Windows
>
>
> Key: HADOOP-9887
> URL: https://issues.apache.org/jira/browse/HADOOP-9887
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Chris Nauroth
>Assignee: Chuan Liu
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HADOOP-9887-trunk.2.patch, HADOOP-9887-trunk.patch
>
>
> Recent file system changes have caused globStatus to stop working for paths
> starting with a drive spec on Windows. The problem is most easily visible by
> running {{TestFileUtil#createJarWithClassPath}} on Windows. This method
> attempts a globStatus with pattern {{*\{.jar,.JAR\}}}, and it no longer
> correctly identifies files at the path ending in .jar or .JAR.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-9715) Unintentional debug statement leftout in KerberosAutheticator#sendToken
[ https://issues.apache.org/jira/browse/HADOOP-9715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G resolved HADOOP-9715. - Resolution: Duplicate > Unintentional debug statement leftout in KerberosAutheticator#sendToken > --- > > Key: HADOOP-9715 > URL: https://issues.apache.org/jira/browse/HADOOP-9715 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.1.0-beta >Reporter: Uma Maheswara Rao G >Assignee: Uma Maheswara Rao G >Priority: Minor > > I saw this is not there in trunk. This is left while merging of the JIRA > HADOOP-8855 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9887) globStatus does not correctly handle paths starting with a drive spec on Windows
[
https://issues.apache.org/jira/browse/HADOOP-9887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13748534#comment-13748534
]
Hudson commented on HADOOP-9887:
SUCCESS: Integrated in Hadoop-Hdfs-trunk #1500 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1500/])
HADOOP-9887. globStatus does not correctly handle paths starting with a drive
spec on Windows. Contributed by Chuan Liu. (cnauroth:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1516608)
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java
> globStatus does not correctly handle paths starting with a drive spec on
> Windows
>
>
> Key: HADOOP-9887
> URL: https://issues.apache.org/jira/browse/HADOOP-9887
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Chris Nauroth
>Assignee: Chuan Liu
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HADOOP-9887-trunk.2.patch, HADOOP-9887-trunk.patch
>
>
> Recent file system changes have caused globStatus to stop working for paths
> starting with a drive spec on Windows. The problem is most easily visible by
> running {{TestFileUtil#createJarWithClassPath}} on Windows. This method
> attempts a globStatus with pattern {{*\{.jar,.JAR\}}}, and it no longer
> correctly identifies files at the path ending in .jar or .JAR.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9887) globStatus does not correctly handle paths starting with a drive spec on Windows
[
https://issues.apache.org/jira/browse/HADOOP-9887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13748451#comment-13748451
]
Hudson commented on HADOOP-9887:
SUCCESS: Integrated in Hadoop-Yarn-trunk #310 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/310/])
HADOOP-9887. globStatus does not correctly handle paths starting with a drive
spec on Windows. Contributed by Chuan Liu. (cnauroth:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1516608)
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java
> globStatus does not correctly handle paths starting with a drive spec on
> Windows
>
>
> Key: HADOOP-9887
> URL: https://issues.apache.org/jira/browse/HADOOP-9887
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Chris Nauroth
>Assignee: Chuan Liu
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HADOOP-9887-trunk.2.patch, HADOOP-9887-trunk.patch
>
>
> Recent file system changes have caused globStatus to stop working for paths
> starting with a drive spec on Windows. The problem is most easily visible by
> running {{TestFileUtil#createJarWithClassPath}} on Windows. This method
> attempts a globStatus with pattern {{*\{.jar,.JAR\}}}, and it no longer
> correctly identifies files at the path ending in .jar or .JAR.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9899) Remove the debug message added by HADOOP-8855
[
https://issues.apache.org/jira/browse/HADOOP-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo (Nicholas), SZE updated HADOOP-9899:
---
Resolution: Fixed
Fix Version/s: 2.1.1-beta
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks Suresh for reviewing the patch.
I have committed this.
> Remove the debug message added by HADOOP-8855
> -
>
> Key: HADOOP-9899
> URL: https://issues.apache.org/jira/browse/HADOOP-9899
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Tsz Wo (Nicholas), SZE
>Assignee: Tsz Wo (Nicholas), SZE
>Priority: Minor
> Fix For: 2.1.1-beta
>
> Attachments: c9899_20130823.patch
>
>
> HADOOP-8855 added a debug message which was printed to System.out.
> {code}
> //KerberosAuthenticator.java
> private void sendToken(byte[] outToken) throws IOException,
> AuthenticationException {
> new Exception("sendToken").printStackTrace(System.out);
> ...
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9899) Remove the debug message added by HADOOP-8855
[
https://issues.apache.org/jira/browse/HADOOP-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13748398#comment-13748398
]
Tsz Wo (Nicholas), SZE commented on HADOOP-9899:
Uma, you are right that this is the same as HADOOP-9715. Thanks for marking it
as duplicated.
> Remove the debug message added by HADOOP-8855
> -
>
> Key: HADOOP-9899
> URL: https://issues.apache.org/jira/browse/HADOOP-9899
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Tsz Wo (Nicholas), SZE
>Assignee: Tsz Wo (Nicholas), SZE
>Priority: Minor
> Attachments: c9899_20130823.patch
>
>
> HADOOP-8855 added a debug message which was printed to System.out.
> {code}
> //KerberosAuthenticator.java
> private void sendToken(byte[] outToken) throws IOException,
> AuthenticationException {
> new Exception("sendToken").printStackTrace(System.out);
> ...
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9899) Remove the debug message added by HADOOP-8855
[
https://issues.apache.org/jira/browse/HADOOP-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13748395#comment-13748395
]
Uma Maheswara Rao G commented on HADOOP-9899:
-
I think this is same as HADOOP-9715, I will mark that as duplicate as you
already put the patch here.
> Remove the debug message added by HADOOP-8855
> -
>
> Key: HADOOP-9899
> URL: https://issues.apache.org/jira/browse/HADOOP-9899
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Tsz Wo (Nicholas), SZE
>Assignee: Tsz Wo (Nicholas), SZE
>Priority: Minor
> Attachments: c9899_20130823.patch
>
>
> HADOOP-8855 added a debug message which was printed to System.out.
> {code}
> //KerberosAuthenticator.java
> private void sendToken(byte[] outToken) throws IOException,
> AuthenticationException {
> new Exception("sendToken").printStackTrace(System.out);
> ...
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9899) Remove the debug message added by HADOOP-8855
[
https://issues.apache.org/jira/browse/HADOOP-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13748391#comment-13748391
]
Tsz Wo (Nicholas), SZE commented on HADOOP-9899:
I should have mentioned that [~yeshavora] found this problem when testing
hftp://.
{code}
RUNNING: /usr/bin/hdfs dfs -ls hftp://NN:50070/user/hrt_qa/hdfsRegressionData/ls
Found 4 items
-rwx-- 3 hrt_qa hrt_qa 95 2013-08-21 20:03
hftp://NN:50070/user/hrt_qa/hdfsRegressionData/ls/helloWorld755
-rwx-- 3 hrt_qa hrt_qa 2255 2013-08-21 20:03
hftp://NN:50070/user/hrt_qa/hdfsRegressionData/ls/smallRDFile755
-rwx-- 3 hrt_qa hrt_qa 2367 2013-08-21 20:03
hftp://NN:50070/user/hrt_qa/hdfsRegressionData/ls/smallRXFile755
-rwx-- 3 hrt_qa hrt_qa 2278 2013-08-21 20:03
hftp://NN:50070/user/hrt_qa/hdfsRegressionData/ls/smallWRFile755
java.lang.Exception: sendToken
at
org.apache.hadoop.security.authentication.client.KerberosAuthenticator.sendToken(KerberosAuthenticator.java:284)
at
org.apache.hadoop.security.authentication.client.KerberosAuthenticator.access$200(KerberosAuthenticator.java:48)
at
org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:254)
at
org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:228)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:228)
at
org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:169)
at
org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:232)
at
org.apache.hadoop.security.SecurityUtil.openSecureHttpConnection(SecurityUtil.java:512)
at
org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.cancelDelegationToken(DelegationTokenFetcher.java:354)
at
org.apache.hadoop.hdfs.HftpFileSystem$TokenManager.cancel(HftpFileSystem.java:724)
at org.apache.hadoop.security.token.Token.cancel(Token.java:382)
at
org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.cancel(DelegationTokenRenewer.java:152)
at
org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.access$200(DelegationTokenRenewer.java:58)
at
org.apache.hadoop.fs.DelegationTokenRenewer.removeRenewAction(DelegationTokenRenewer.java:241)
at org.apache.hadoop.hdfs.HftpFileSystem.close(HftpFileSystem.java:413)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2520)
at
org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2537)
at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
{code}
> Remove the debug message added by HADOOP-8855
> -
>
> Key: HADOOP-9899
> URL: https://issues.apache.org/jira/browse/HADOOP-9899
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Tsz Wo (Nicholas), SZE
>Assignee: Tsz Wo (Nicholas), SZE
>Priority: Minor
> Attachments: c9899_20130823.patch
>
>
> HADOOP-8855 added a debug message which was printed to System.out.
> {code}
> //KerberosAuthenticator.java
> private void sendToken(byte[] outToken) throws IOException,
> AuthenticationException {
> new Exception("sendToken").printStackTrace(System.out);
> ...
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9899) Remove the debug message added by HADOOP-8855
[
https://issues.apache.org/jira/browse/HADOOP-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13748376#comment-13748376
]
Suresh Srinivas commented on HADOOP-9899:
-
+1 for the patch
> Remove the debug message added by HADOOP-8855
> -
>
> Key: HADOOP-9899
> URL: https://issues.apache.org/jira/browse/HADOOP-9899
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Tsz Wo (Nicholas), SZE
>Assignee: Tsz Wo (Nicholas), SZE
>Priority: Minor
> Attachments: c9899_20130823.patch
>
>
> HADOOP-8855 added a debug message which was printed to System.out.
> {code}
> //KerberosAuthenticator.java
> private void sendToken(byte[] outToken) throws IOException,
> AuthenticationException {
> new Exception("sendToken").printStackTrace(System.out);
> ...
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
