[jira] [Commented] (HADOOP-9774) RawLocalFileSystem.listStatus() return absolute paths when input path is relative on Windows
[ https://issues.apache.org/jira/browse/HADOOP-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749853#comment-13749853 ] Ivan Mitic commented on HADOOP-9774: Sure Shanyu, Chris. Let me kick off the tests tonight and I'll reply back. > RawLocalFileSystem.listStatus() return absolute paths when input path is > relative on Windows > > > Key: HADOOP-9774 > URL: https://issues.apache.org/jira/browse/HADOOP-9774 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 3.0.0, 2.1.0-beta >Reporter: shanyu zhao >Assignee: shanyu zhao > Attachments: HADOOP-9774-2.patch, HADOOP-9774-3.patch, > HADOOP-9774-4.patch, HADOOP-9774-5.patch, HADOOP-9774.patch > > > On Windows, when using RawLocalFileSystem.listStatus() to enumerate a > relative path (without drive spec), e.g., "file:///mydata", the resulting > paths become absolute paths, e.g., ["file://E:/mydata/t1.txt", > "file://E:/mydata/t2.txt"...]. > Note that if we use it to enumerate an absolute path, e.g., > "file://E:/mydata" then the we get the same results as above. > This breaks some hive unit tests which uses local file system to simulate > HDFS when testing, therefore the drive spec is removed. Then after > listStatus() the path is changed to absolute path, hive failed to find the > path in its map reduce job. > You'll see the following exception: > [junit] java.io.IOException: cannot find dir = > pfile:/E:/GitHub/hive-monarch/build/ql/test/data/warehouse/src/kv1.txt in > pathToPartitionInfo: > [pfile:/GitHub/hive-monarch/build/ql/test/data/warehouse/src] > [junit] at > org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getPartitionDescFromPathRecursively(HiveFileFormatUtils.java:298) > This problem is introduced by this JIRA: > HADOOP-8962 > Prior to the fix for HADOOP-8962 (merged in 0.23.5), the resulting paths are > relative paths if the parent paths are relative, e.g., > ["file:///mydata/t1.txt", "file:///mydata/t2.txt"...] > This behavior change is a side effect of the fix in HADOOP-8962, not an > intended change. The resulting behavior, even though is legitimate from a > function point of view, break consistency from the caller's point of view. > When the caller use a relative path (without drive spec) to do listStatus() > the resulting path should be relative. Therefore, I think this should be > fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9044) add FindClass main class to provide classpath checking of installations
[
https://issues.apache.org/jira/browse/HADOOP-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749816#comment-13749816
]
Hadoop QA commented on HADOOP-9044:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12599864/HADOOP-9044-002.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/3018//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/3018//console
This message is automatically generated.
> add FindClass main class to provide classpath checking of installations
> ---
>
> Key: HADOOP-9044
> URL: https://issues.apache.org/jira/browse/HADOOP-9044
> Project: Hadoop Common
> Issue Type: New Feature
> Components: util
>Affects Versions: 3.0.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Minor
> Attachments: HADOOP-9044-002.patch, HADOOP-9044.patch,
> HADOOP-9044.patch
>
>
> It's useful in postflight checking of a hadoop installation to verify that
> classes load, especially codes with external JARs and native codecs.
> An entry point designed to load a named class and create an instance of that
> class can do this -and be invoked from any shell script or tool that does the
> installation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9044) add FindClass main class to provide classpath checking of installations
[
https://issues.apache.org/jira/browse/HADOOP-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-9044:
---
Attachment: HADOOP-9044-002.patch
This patch (which applies to trunk)
# adds a {{printresource}} option that prints the resource to stdout.
# adds a test for it by locating log4j properties, capturing the output (a
testing method to redefine the output/err streams was added for this),
verifying it contains "Apache", which is of course in the headers of all of the
ASF supplied files.
As all errors are printed to stderr, it should be usable in operations where
the result is streamed to a new file
{code}
FindClass printresource /log4j.properties > log4j.txt
{code}
> add FindClass main class to provide classpath checking of installations
> ---
>
> Key: HADOOP-9044
> URL: https://issues.apache.org/jira/browse/HADOOP-9044
> Project: Hadoop Common
> Issue Type: New Feature
> Components: util
>Affects Versions: 1.1.0, 2.0.3-alpha
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Minor
> Attachments: HADOOP-9044-002.patch, HADOOP-9044.patch,
> HADOOP-9044.patch
>
>
> It's useful in postflight checking of a hadoop installation to verify that
> classes load, especially codes with external JARs and native codecs.
> An entry point designed to load a named class and create an instance of that
> class can do this -and be invoked from any shell script or tool that does the
> installation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9044) add FindClass main class to provide classpath checking of installations
[ https://issues.apache.org/jira/browse/HADOOP-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-9044: --- Status: Open (was: Patch Available) > add FindClass main class to provide classpath checking of installations > --- > > Key: HADOOP-9044 > URL: https://issues.apache.org/jira/browse/HADOOP-9044 > Project: Hadoop Common > Issue Type: New Feature > Components: util >Affects Versions: 2.0.3-alpha, 1.1.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > Attachments: HADOOP-9044-002.patch, HADOOP-9044.patch, > HADOOP-9044.patch > > > It's useful in postflight checking of a hadoop installation to verify that > classes load, especially codes with external JARs and native codecs. > An entry point designed to load a named class and create an instance of that > class can do this -and be invoked from any shell script or tool that does the > installation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9044) add FindClass main class to provide classpath checking of installations
[ https://issues.apache.org/jira/browse/HADOOP-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-9044: --- Status: Patch Available (was: Open) > add FindClass main class to provide classpath checking of installations > --- > > Key: HADOOP-9044 > URL: https://issues.apache.org/jira/browse/HADOOP-9044 > Project: Hadoop Common > Issue Type: New Feature > Components: util >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > Attachments: HADOOP-9044-002.patch, HADOOP-9044.patch, > HADOOP-9044.patch > > > It's useful in postflight checking of a hadoop installation to verify that > classes load, especially codes with external JARs and native codecs. > An entry point designed to load a named class and create an instance of that > class can do this -and be invoked from any shell script or tool that does the > installation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9044) add FindClass main class to provide classpath checking of installations
[ https://issues.apache.org/jira/browse/HADOOP-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-9044: --- Affects Version/s: (was: 2.0.3-alpha) (was: 1.1.0) 3.0.0 > add FindClass main class to provide classpath checking of installations > --- > > Key: HADOOP-9044 > URL: https://issues.apache.org/jira/browse/HADOOP-9044 > Project: Hadoop Common > Issue Type: New Feature > Components: util >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > Attachments: HADOOP-9044-002.patch, HADOOP-9044.patch, > HADOOP-9044.patch > > > It's useful in postflight checking of a hadoop installation to verify that > classes load, especially codes with external JARs and native codecs. > An entry point designed to load a named class and create an instance of that > class can do this -and be invoked from any shell script or tool that does the > installation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9889) Refresh the Krb5 configuration when creating a new kdc in Hadoop-MiniKDC
[ https://issues.apache.org/jira/browse/HADOOP-9889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749753#comment-13749753 ] Wei Yan commented on HADOOP-9889: - Sure, I'll take care of it. > Refresh the Krb5 configuration when creating a new kdc in Hadoop-MiniKDC > > > Key: HADOOP-9889 > URL: https://issues.apache.org/jira/browse/HADOOP-9889 > Project: Hadoop Common > Issue Type: Bug >Reporter: Wei Yan >Assignee: Wei Yan > Attachments: HAOOP-9889.patch, HAOOP-9889.patch > > > Krb5 Config uses a singleton and once initialized it does not refresh > automatically. Without refresh, there are failures if you are using MiniKDCs > with different configurations (such as different realms) within the same test > run or if the Krb5 Config singleton is called before the MiniKDC is started > for the first time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9889) Refresh the Krb5 configuration when creating a new kdc in Hadoop-MiniKDC
[ https://issues.apache.org/jira/browse/HADOOP-9889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749716#comment-13749716 ] Sandy Ryza commented on HADOOP-9889: [~ywskycn], are you able to fix the javac and javadoc warnings? > Refresh the Krb5 configuration when creating a new kdc in Hadoop-MiniKDC > > > Key: HADOOP-9889 > URL: https://issues.apache.org/jira/browse/HADOOP-9889 > Project: Hadoop Common > Issue Type: Bug >Reporter: Wei Yan >Assignee: Wei Yan > Attachments: HAOOP-9889.patch, HAOOP-9889.patch > > > Krb5 Config uses a singleton and once initialized it does not refresh > automatically. Without refresh, there are failures if you are using MiniKDCs > with different configurations (such as different realms) within the same test > run or if the Krb5 Config singleton is called before the MiniKDC is started > for the first time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9897) Add method to get path start position without drive specifier in o.a.h.fs.Path
[
https://issues.apache.org/jira/browse/HADOOP-9897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749688#comment-13749688
]
Hadoop QA commented on HADOOP-9897:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12599838/HADOOP-9897.v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/3017//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/3017//console
This message is automatically generated.
> Add method to get path start position without drive specifier in
> o.a.h.fs.Path
>
>
> Key: HADOOP-9897
> URL: https://issues.apache.org/jira/browse/HADOOP-9897
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HADOOP-9897.v1.patch, HADOOP-9897.v2.patch,
> HADOOP-9897.v2.patch
>
>
> There are a lot of code in Path to get start position after skipping drive
> specifier, like:
> {code}
> int start = hasWindowsDrive(uri.getPath()) ? 3 : 0;
> {code}
> Also there is a minor bug in mergePaths:
> mergePath("/", "/foo") will yield Path("//foo") which will be parsed as uri
> authority, not path.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9897) Add method to get path start position without drive specifier in o.a.h.fs.Path
[
https://issues.apache.org/jira/browse/HADOOP-9897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Binglin Chang updated HADOOP-9897:
--
Attachment: HADOOP-9897.v2.patch
> Add method to get path start position without drive specifier in
> o.a.h.fs.Path
>
>
> Key: HADOOP-9897
> URL: https://issues.apache.org/jira/browse/HADOOP-9897
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HADOOP-9897.v1.patch, HADOOP-9897.v2.patch,
> HADOOP-9897.v2.patch
>
>
> There are a lot of code in Path to get start position after skipping drive
> specifier, like:
> {code}
> int start = hasWindowsDrive(uri.getPath()) ? 3 : 0;
> {code}
> Also there is a minor bug in mergePaths:
> mergePath("/", "/foo") will yield Path("//foo") which will be parsed as uri
> authority, not path.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9902) Shell script rewrite
[ https://issues.apache.org/jira/browse/HADOOP-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749657#comment-13749657 ] Allen Wittenauer commented on HADOOP-9902: -- Adding a bunch of links to JIRAs for xref to when various things got added. A quick read leaves me with one impression: YARN is incredibly inconsistent and it's attempts to "make things easier" have actually made things harder for both the user and the developer. Worse, a lot of the stuff is completely undocumented outside of JIRAs. I don't know if this situation is salvageable without undoing some of this nonsense. > Shell script rewrite > > > Key: HADOOP-9902 > URL: https://issues.apache.org/jira/browse/HADOOP-9902 > Project: Hadoop Common > Issue Type: Improvement > Components: scripts >Affects Versions: 2.1.1-beta >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: more-info.txt, scripts.tgz > > > Umbrella JIRA for shell script rewrite. See more-info.txt for more details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9894) Race condition in Shell leads to logged error stream handling exceptions
[
https://issues.apache.org/jira/browse/HADOOP-9894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749640#comment-13749640
]
Hadoop QA commented on HADOOP-9894:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12599348/hadoop-9894.01.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/3016//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/3016//console
This message is automatically generated.
> Race condition in Shell leads to logged error stream handling exceptions
>
>
> Key: HADOOP-9894
> URL: https://issues.apache.org/jira/browse/HADOOP-9894
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.1.0-beta
>Reporter: Jason Lowe
> Attachments: hadoop-9894.01.patch
>
>
> Shell.runCommand starts an error stream handling thread and normally joins
> with it before closing the error stream. However if parseExecResult throws
> an exception (e.g.: like Stat.parseExecResult does for FileNotFoundException)
> then the error thread is not joined and the error stream can be closed before
> the error stream handling thread is finished. This causes the error stream
> handling thread to log an exception backtrace for a "normal" situation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9897) Add method to get path start position without drive specifier in o.a.h.fs.Path
[
https://issues.apache.org/jira/browse/HADOOP-9897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749639#comment-13749639
]
Binglin Chang commented on HADOOP-9897:
---
The failed test is probably due to timeout, it passed on my machine.
> Add method to get path start position without drive specifier in
> o.a.h.fs.Path
>
>
> Key: HADOOP-9897
> URL: https://issues.apache.org/jira/browse/HADOOP-9897
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HADOOP-9897.v1.patch, HADOOP-9897.v2.patch
>
>
> There are a lot of code in Path to get start position after skipping drive
> specifier, like:
> {code}
> int start = hasWindowsDrive(uri.getPath()) ? 3 : 0;
> {code}
> Also there is a minor bug in mergePaths:
> mergePath("/", "/foo") will yield Path("//foo") which will be parsed as uri
> authority, not path.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9883) Local mode FileNotFoundException: File does not exist
[
https://issues.apache.org/jira/browse/HADOOP-9883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749608#comment-13749608
]
Hadoop QA commented on HADOOP-9883:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12599745/HADOOP-9883.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:
org.apache.hadoop.mapreduce.security.TestJHSSecurity
The following test timeouts occurred in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:
org.apache.hadoop.mapreduce.v2.TestUberAM
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/3013//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/3013//console
This message is automatically generated.
> Local mode FileNotFoundException: File does not exist
> -
>
> Key: HADOOP-9883
> URL: https://issues.apache.org/jira/browse/HADOOP-9883
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.0.2-alpha
> Environment: Centos 6.3
> Hadoop 2.0.2-alpha
> Hive 0.10.0
>Reporter: Abin Shahab
>Priority: Critical
> Attachments: HADOOP-9883.patch, HADOOP-9883.patch, HADOOP-9883.patch,
> HADOOP-9883.patch, HADOOP-9883.patch
>
>
> Hive jobs in local mode fail with the error posted below. The jar file that's
> not being found exists and has the following access:
> > ls -l hive-0.10.0/lib/hive-builtins-0.10.0.jar
> rw-rw-r-- 1 ashahab ashahab 3914 Dec 18 2012
> hive-0.10.0/lib/hive-builtins-0.10.0.jar
> Steps to reproduce
> [vcc_chaiken@HadoopDesktop0 ~]$ hive
> Logging initialized using configuration in
> jar:file:/opt/hive/lib/hive-common-0.10.0.jar!/hive-log4j.properties
> Hive history
> file=/disk1/hive/log/vcc_chaiken/hive_job_log_vcc_chaiken_201307162119_876702406.txt
> hive> create database chaiken_test_00;
> OK
> Time taken: 1.675 seconds
> hive> use chaiken_test_00;
> OK
> Time taken: 0.029 seconds
> hive> create table chaiken_test_table(foo INT);
> OK
> Time taken: 0.301 seconds
> hive> select count(*) from chaiken_test_table;
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks determined at compile time: 1
> In order to change the average load for a reducer (in bytes):
> set hive.exec.reducers.bytes.per.reducer=
> In order to limit the maximum number of reducers:
> set hive.exec.reducers.max=
> In order to set a constant number of reducers:
> set mapred.reduce.tasks=
> Starting Job = job_1373902166027_0061, Tracking URL =
> http://100-01-09.sc1.verticloud.com:8088/proxy/application_1373902166027_0061/
> Kill Command = /opt/hadoop/bin/hadoop job -kill job_1373902166027_0061
> Hadoop job information for Stage-1: number of mappers: 1; number of reducers:
> 1
> 2013-07-16 21:20:25,617 Stage-1 map = 0%, reduce = 0%
> 2013-07-16 21:20:30,026 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 1.13
> sec
> 2013-07-16 21:20:31,110 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 1.13
> sec
> 2013-07-16 21:20:32,188 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 1.13
> sec
> 2013-07-16 21:20:33,270 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 1.13
> sec
> 2013-07-16 21:20:34,356 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 1.13
> sec
> 2013-07-16 21:20:35,455 Stage-1 map = 100%, reduce = 100%, Cumulative CPU
> 3.4 sec
> MapReduce Total cumulative CPU time: 3 seconds 400 msec
> Ended Job = job_1373902166027_0061
> MapReduce Jobs Launched:
> Job 0: Map: 1 Reduce: 1 Cumulative CPU: 3.4 sec HDFS Read: 246 HDFS
> Write: 2 SUCCESS
> Total MapReduce CPU Time Spent: 3 seconds
[jira] [Commented] (HADOOP-8704) add request logging to jetty/httpserver
[
https://issues.apache.org/jira/browse/HADOOP-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749599#comment-13749599
]
Hadoop QA commented on HADOOP-8704:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12599265/HADOOP-8704.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/3015//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/3015//console
This message is automatically generated.
> add request logging to jetty/httpserver
> ---
>
> Key: HADOOP-8704
> URL: https://issues.apache.org/jira/browse/HADOOP-8704
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 0.23.3, 2.0.0-alpha
>Reporter: Thomas Graves
>Assignee: Jonathan Eagles
> Attachments: HADOOP-8704.patch
>
>
> We have been requested to log all the requests coming into Jetty/HttpServer
> for security and auditing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9889) Refresh the Krb5 configuration when creating a new kdc in Hadoop-MiniKDC
[
https://issues.apache.org/jira/browse/HADOOP-9889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749595#comment-13749595
]
Hadoop QA commented on HADOOP-9889:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12599692/HAOOP-9889.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:red}-1 javac{color}. The applied patch generated 1151 javac
compiler warnings (more than the trunk's current 1149 warnings).
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-minikdc.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/3014//testReport/
Javac warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/3014//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/3014//console
This message is automatically generated.
> Refresh the Krb5 configuration when creating a new kdc in Hadoop-MiniKDC
>
>
> Key: HADOOP-9889
> URL: https://issues.apache.org/jira/browse/HADOOP-9889
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Wei Yan
>Assignee: Wei Yan
> Attachments: HAOOP-9889.patch, HAOOP-9889.patch
>
>
> Krb5 Config uses a singleton and once initialized it does not refresh
> automatically. Without refresh, there are failures if you are using MiniKDCs
> with different configurations (such as different realms) within the same test
> run or if the Krb5 Config singleton is called before the MiniKDC is started
> for the first time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9897) Add method to get path start position without drive specifier in o.a.h.fs.Path
[
https://issues.apache.org/jira/browse/HADOOP-9897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749588#comment-13749588
]
Hadoop QA commented on HADOOP-9897:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12599776/HADOOP-9897.v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The following test timeouts occurred in
hadoop-common-project/hadoop-common:
org.apache.hadoop.ipc.TestIPC
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/3011//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/3011//console
This message is automatically generated.
> Add method to get path start position without drive specifier in
> o.a.h.fs.Path
>
>
> Key: HADOOP-9897
> URL: https://issues.apache.org/jira/browse/HADOOP-9897
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HADOOP-9897.v1.patch, HADOOP-9897.v2.patch
>
>
> There are a lot of code in Path to get start position after skipping drive
> specifier, like:
> {code}
> int start = hasWindowsDrive(uri.getPath()) ? 3 : 0;
> {code}
> Also there is a minor bug in mergePaths:
> mergePath("/", "/foo") will yield Path("//foo") which will be parsed as uri
> authority, not path.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9733) a node's stack trace output to log file should be controllable
[
https://issues.apache.org/jira/browse/HADOOP-9733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749587#comment-13749587
]
Hadoop QA commented on HADOOP-9733:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12599759/HADOOP-9733.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/3012//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/3012//console
This message is automatically generated.
> a node's stack trace output to log file should be controllable
> --
>
> Key: HADOOP-9733
> URL: https://issues.apache.org/jira/browse/HADOOP-9733
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Shinichi Yamashita
>Priority: Minor
> Attachments: HADOOP-9733.patch
>
>
> We can confirm the stack trace of node when we access the node's web
> interface "/stacks".
> And the stack trace is output not only browser but also log file of node.
> Considering the cluster's management policy (e.g. log monitoring), it should
> be controllable.
> At now implementation, stack trace output controls HttpServer class.
> If we change log level, other messages does not output by HttpServer class.
> So, we control to output the stacktrace to log file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
