[jira] Commented: (HIVE-1330) fatal error check omitted for reducer-side operators
[ https://issues.apache.org/jira/browse/HIVE-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862106#action_12862106 ] Namit Jain commented on HIVE-1330: -- +1 looks good > fatal error check omitted for reducer-side operators > > > Key: HIVE-1330 > URL: https://issues.apache.org/jira/browse/HIVE-1330 > Project: Hadoop Hive > Issue Type: Bug >Affects Versions: 0.6.0 >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.6.0 > > Attachments: HIVE-1330.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1330) fatal error check omitted for reducer-side operators
[ https://issues.apache.org/jira/browse/HIVE-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang updated HIVE-1330: - Attachment: HIVE-1330.patch > fatal error check omitted for reducer-side operators > > > Key: HIVE-1330 > URL: https://issues.apache.org/jira/browse/HIVE-1330 > Project: Hadoop Hive > Issue Type: Bug >Affects Versions: 0.6.0 >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.6.0 > > Attachments: HIVE-1330.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1330) fatal error check omitted for reducer-side operators
[ https://issues.apache.org/jira/browse/HIVE-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang updated HIVE-1330: - Status: Patch Available (was: Open) Affects Version/s: 0.6.0 Fix Version/s: 0.6.0 > fatal error check omitted for reducer-side operators > > > Key: HIVE-1330 > URL: https://issues.apache.org/jira/browse/HIVE-1330 > Project: Hadoop Hive > Issue Type: Bug >Affects Versions: 0.6.0 >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.6.0 > > Attachments: HIVE-1330.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HIVE-1330) fatal error check omitted for reducer-side operators
[ https://issues.apache.org/jira/browse/HIVE-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang reassigned HIVE-1330: Assignee: Ning Zhang > fatal error check omitted for reducer-side operators > > > Key: HIVE-1330 > URL: https://issues.apache.org/jira/browse/HIVE-1330 > Project: Hadoop Hive > Issue Type: Bug >Reporter: Ning Zhang >Assignee: Ning Zhang > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1330) fatal error check omitted for reducer-side operators
fatal error check omitted for reducer-side operators Key: HIVE-1330 URL: https://issues.apache.org/jira/browse/HIVE-1330 Project: Hadoop Hive Issue Type: Bug Reporter: Ning Zhang -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1328) make mapred.input.dir.recursive work for select *
[
https://issues.apache.org/jira/browse/HIVE-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862077#action_12862077
]
John Sichi commented on HIVE-1328:
--
Hi Ed,
This is not a new feature--this is an inconsistency in an existing feature when
a particular Hadoop parameter is enabled (it should not matter whether you use
select * vs a more complex select, you should get the same results).
In general, prioritization is driven by a number of factors such as the overall
project roadmap, quality, and the use cases which the developer wants or needs
to make work (this one happens to be important for Facebook, which is why I'm
working on it at the moment); if the ones you mention are high priority for
you, please submit patches for them so we can get them resolved.
Regardless of that, thanks for all the bug reports that you have
submitted--they're very valuable in themselves, and we want to get them all
fixed too.
> make mapred.input.dir.recursive work for select *
> -
>
> Key: HIVE-1328
> URL: https://issues.apache.org/jira/browse/HIVE-1328
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
>Affects Versions: 0.6.0
>Reporter: John Sichi
>Assignee: John Sichi
> Fix For: 0.6.0
>
>
> For the script below, we would like the behavior from MAPREDUCE-1501 to apply
> so that the select * returns two rows instead of none.
> create table fact_daily(x int)
> partitioned by (ds string);
> create table fact_tz(x int)
> partitioned by (ds string, hr string, gmtoffset string);
> alter table fact_tz
> add partition (ds='2010-01-03', hr='1', gmtoffset='-8');
> insert overwrite table fact_tz
> partition (ds='2010-01-03', hr='1', gmtoffset='-8')
> select key+11 from src where key=484;
> alter table fact_tz
> add partition (ds='2010-01-03', hr='2', gmtoffset='-7');
> insert overwrite table fact_tz
> partition (ds='2010-01-03', hr='2', gmtoffset='-7')
> select key+12 from src where key=484;
> alter table fact_daily
> set tblproperties('EXTERNAL'='TRUE');
> alter table fact_daily
> add partition (ds='2010-01-03')
> location '/user/hive/warehouse/fact_tz/ds=2010-01-03';
> set mapred.input.dir.recursive=true;
> select * from fact_daily where ds='2010-01-03';
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1328) make mapred.input.dir.recursive work for select *
[
https://issues.apache.org/jira/browse/HIVE-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862074#action_12862074
]
Edward Capriolo commented on HIVE-1328:
---
Can we look at HIVE-1318 and maybe HIVE-1303 first. Already the external
partitions seem to have bugs can we get them working properly before more
features are added?
> make mapred.input.dir.recursive work for select *
> -
>
> Key: HIVE-1328
> URL: https://issues.apache.org/jira/browse/HIVE-1328
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
>Affects Versions: 0.6.0
>Reporter: John Sichi
>Assignee: John Sichi
> Fix For: 0.6.0
>
>
> For the script below, we would like the behavior from MAPREDUCE-1501 to apply
> so that the select * returns two rows instead of none.
> create table fact_daily(x int)
> partitioned by (ds string);
> create table fact_tz(x int)
> partitioned by (ds string, hr string, gmtoffset string);
> alter table fact_tz
> add partition (ds='2010-01-03', hr='1', gmtoffset='-8');
> insert overwrite table fact_tz
> partition (ds='2010-01-03', hr='1', gmtoffset='-8')
> select key+11 from src where key=484;
> alter table fact_tz
> add partition (ds='2010-01-03', hr='2', gmtoffset='-7');
> insert overwrite table fact_tz
> partition (ds='2010-01-03', hr='2', gmtoffset='-7')
> select key+12 from src where key=484;
> alter table fact_daily
> set tblproperties('EXTERNAL'='TRUE');
> alter table fact_daily
> add partition (ds='2010-01-03')
> location '/user/hive/warehouse/fact_tz/ds=2010-01-03';
> set mapred.input.dir.recursive=true;
> select * from fact_daily where ds='2010-01-03';
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when hadoop.version=0.20.1
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862054#action_12862054 ] Carl Steinbach commented on HIVE-1192: -- @John: I attached a correctly formatted version of the md5 checksum file for the 0.20.1 tarball. > Build fails when hadoop.version=0.20.1 > -- > > Key: HIVE-1192 > URL: https://issues.apache.org/jira/browse/HIVE-1192 > Project: Hadoop Hive > Issue Type: Bug > Components: Build Infrastructure >Reporter: Carl Steinbach > Attachments: hadoop-0.20.1.tar.gz.md5 > > > Setting hadoop.version=0.20.1 causes the build to fail since > mirror.facebook.net/facebook/hive-deps does not have 0.20.1 > (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0). > Suggested fix: > * remove/ignore the hadoop.version configuration parameter > or > * Remove the patch numbers from these archives and use only the major.minor > numbers specified by the user to locate the appropriate tarball to download, > so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz. > * Optionally create new tarballs that only contain the components that are > actually needed for the build (Hadoop jars), and remove things that aren't > needed (all of the source files). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1192) Build fails when hadoop.version=0.20.1
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1192: - Attachment: hadoop-0.20.1.tar.gz.md5 > Build fails when hadoop.version=0.20.1 > -- > > Key: HIVE-1192 > URL: https://issues.apache.org/jira/browse/HIVE-1192 > Project: Hadoop Hive > Issue Type: Bug > Components: Build Infrastructure >Reporter: Carl Steinbach > Attachments: hadoop-0.20.1.tar.gz.md5 > > > Setting hadoop.version=0.20.1 causes the build to fail since > mirror.facebook.net/facebook/hive-deps does not have 0.20.1 > (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0). > Suggested fix: > * remove/ignore the hadoop.version configuration parameter > or > * Remove the patch numbers from these archives and use only the major.minor > numbers specified by the user to locate the appropriate tarball to download, > so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz. > * Optionally create new tarballs that only contain the components that are > actually needed for the build (Hadoop jars), and remove things that aren't > needed (all of the source files). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when hadoop.version=0.20.1
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862033#action_12862033 ] John Sichi commented on HIVE-1192: -- If someone regenerates the correctly formatted .md5 file and tests it, I can get the bad one replaced on mirror.facebook.net. > Build fails when hadoop.version=0.20.1 > -- > > Key: HIVE-1192 > URL: https://issues.apache.org/jira/browse/HIVE-1192 > Project: Hadoop Hive > Issue Type: Bug > Components: Build Infrastructure >Reporter: Carl Steinbach > > Setting hadoop.version=0.20.1 causes the build to fail since > mirror.facebook.net/facebook/hive-deps does not have 0.20.1 > (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0). > Suggested fix: > * remove/ignore the hadoop.version configuration parameter > or > * Remove the patch numbers from these archives and use only the major.minor > numbers specified by the user to locate the appropriate tarball to download, > so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz. > * Optionally create new tarballs that only contain the components that are > actually needed for the build (Hadoop jars), and remove things that aren't > needed (all of the source files). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when hadoop.version=0.20.1
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862005#action_12862005 ] Carl Steinbach commented on HIVE-1192: -- John/Bill: Ivy can't download the 0.20.1 tarball off of archive.apache.org because the associated md5 checksum file does not adhere to convention, i.e. it was not created by running 'md5sum' on the tarball. For example, compare the following two files: http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-0.20.0/hadoop-0.20.0.tar.gz.md5 http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-0.20.1/hadoop-0.20.1.tar.gz.md5 The checksum file for 0.20.0 contains the output of md5sum. The checksum file for 0.20.1 contains a bunch of checksums generated using some other tool. Ivy barfs when it tries to located the md5 checksum in this mess. > Build fails when hadoop.version=0.20.1 > -- > > Key: HIVE-1192 > URL: https://issues.apache.org/jira/browse/HIVE-1192 > Project: Hadoop Hive > Issue Type: Bug > Components: Build Infrastructure >Reporter: Carl Steinbach > > Setting hadoop.version=0.20.1 causes the build to fail since > mirror.facebook.net/facebook/hive-deps does not have 0.20.1 > (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0). > Suggested fix: > * remove/ignore the hadoop.version configuration parameter > or > * Remove the patch numbers from these archives and use only the major.minor > numbers specified by the user to locate the appropriate tarball to download, > so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz. > * Optionally create new tarballs that only contain the components that are > actually needed for the build (Hadoop jars), and remove things that aren't > needed (all of the source files). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when hadoop.version=0.20.1
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861996#action_12861996 ] John Sichi commented on HIVE-1192: -- 0.20.1 is there now...Bill, can you try it? > Build fails when hadoop.version=0.20.1 > -- > > Key: HIVE-1192 > URL: https://issues.apache.org/jira/browse/HIVE-1192 > Project: Hadoop Hive > Issue Type: Bug > Components: Build Infrastructure >Reporter: Carl Steinbach > > Setting hadoop.version=0.20.1 causes the build to fail since > mirror.facebook.net/facebook/hive-deps does not have 0.20.1 > (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0). > Suggested fix: > * remove/ignore the hadoop.version configuration parameter > or > * Remove the patch numbers from these archives and use only the major.minor > numbers specified by the user to locate the appropriate tarball to download, > so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz. > * Optionally create new tarballs that only contain the components that are > actually needed for the build (Hadoop jars), and remove things that aren't > needed (all of the source files). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1329) for ALTER TABLE t SET TBLPROPERTIES ('EXTERNAL'='TRUE'), change TBL_TYPE attribute from MANAGED_TABLE to EXTERNAL_TABLE
for ALTER TABLE t SET TBLPROPERTIES ('EXTERNAL'='TRUE'), change TBL_TYPE
attribute from MANAGED_TABLE to EXTERNAL_TABLE
---
Key: HIVE-1329
URL: https://issues.apache.org/jira/browse/HIVE-1329
Project: Hadoop Hive
Issue Type: Improvement
Components: Metastore
Affects Versions: 0.6.0
Reporter: John Sichi
Fix For: 0.6.0
Currently they are left inconsistent.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1329) for ALTER TABLE t SET TBLPROPERTIES ('EXTERNAL'='TRUE'), change TBL_TYPE attribute from MANAGED_TABLE to EXTERNAL_TABLE
[
https://issues.apache.org/jira/browse/HIVE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861959#action_12861959
]
John Sichi commented on HIVE-1329:
--
And vice versa for setting 'EXTERNAL' = 'FALSE' (or removing the 'EXTERNAL'
property if we allow that).
> for ALTER TABLE t SET TBLPROPERTIES ('EXTERNAL'='TRUE'), change TBL_TYPE
> attribute from MANAGED_TABLE to EXTERNAL_TABLE
> ---
>
> Key: HIVE-1329
> URL: https://issues.apache.org/jira/browse/HIVE-1329
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Metastore
>Affects Versions: 0.6.0
>Reporter: John Sichi
> Fix For: 0.6.0
>
>
> Currently they are left inconsistent.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1328) make mapred.input.dir.recursive work for select *
make mapred.input.dir.recursive work for select *
-
Key: HIVE-1328
URL: https://issues.apache.org/jira/browse/HIVE-1328
Project: Hadoop Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.6.0
Reporter: John Sichi
Assignee: John Sichi
Fix For: 0.6.0
For the script below, we would like the behavior from MAPREDUCE-1501 to apply
so that the select * returns two rows instead of none.
create table fact_daily(x int)
partitioned by (ds string);
create table fact_tz(x int)
partitioned by (ds string, hr string, gmtoffset string);
alter table fact_tz
add partition (ds='2010-01-03', hr='1', gmtoffset='-8');
insert overwrite table fact_tz
partition (ds='2010-01-03', hr='1', gmtoffset='-8')
select key+11 from src where key=484;
alter table fact_tz
add partition (ds='2010-01-03', hr='2', gmtoffset='-7');
insert overwrite table fact_tz
partition (ds='2010-01-03', hr='2', gmtoffset='-7')
select key+12 from src where key=484;
alter table fact_daily
set tblproperties('EXTERNAL'='TRUE');
alter table fact_daily
add partition (ds='2010-01-03')
location '/user/hive/warehouse/fact_tz/ds=2010-01-03';
set mapred.input.dir.recursive=true;
select * from fact_daily where ds='2010-01-03';
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when hadoop.version=0.20.1
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861926#action_12861926 ] John Sichi commented on HIVE-1192: -- Meanwhile, I'm pinging Facebook ops to see if we can get 0.20.1 added to mirror.facebook.net/facebook/hive-deps. > Build fails when hadoop.version=0.20.1 > -- > > Key: HIVE-1192 > URL: https://issues.apache.org/jira/browse/HIVE-1192 > Project: Hadoop Hive > Issue Type: Bug > Components: Build Infrastructure >Reporter: Carl Steinbach > > Setting hadoop.version=0.20.1 causes the build to fail since > mirror.facebook.net/facebook/hive-deps does not have 0.20.1 > (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0). > Suggested fix: > * remove/ignore the hadoop.version configuration parameter > or > * Remove the patch numbers from these archives and use only the major.minor > numbers specified by the user to locate the appropriate tarball to download, > so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz. > * Optionally create new tarballs that only contain the components that are > actually needed for the build (Hadoop jars), and remove things that aren't > needed (all of the source files). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1317) CombineHiveInputFormat throws exception when partition name contains special characters to URI
[ https://issues.apache.org/jira/browse/HIVE-1317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861925#action_12861925 ] Ning Zhang commented on HIVE-1317: -- Found a bug in my local testing, I will update a new patch once it is fixed. > CombineHiveInputFormat throws exception when partition name contains special > characters to URI > -- > > Key: HIVE-1317 > URL: https://issues.apache.org/jira/browse/HIVE-1317 > Project: Hadoop Hive > Issue Type: Bug >Affects Versions: 0.6.0 >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.6.0 > > Attachments: HIVE-1317.2.patch, HIVE-1317.patch > > > If a partition name contains characters such as ':' and '|' which have > special meaning in URI (hdfs uses URI internally for Path), > CombineHiveInputFormat throws an exception. URI was created in > CombineHiveInputFormat to compare a path belongs to a partition in > partitionToPathInfo. We should bypass URI creation by just string > comparisons. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1317) CombineHiveInputFormat throws exception when partition name contains special characters to URI
[ https://issues.apache.org/jira/browse/HIVE-1317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang updated HIVE-1317: - Status: Open (was: Patch Available) > CombineHiveInputFormat throws exception when partition name contains special > characters to URI > -- > > Key: HIVE-1317 > URL: https://issues.apache.org/jira/browse/HIVE-1317 > Project: Hadoop Hive > Issue Type: Bug >Affects Versions: 0.6.0 >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.6.0 > > Attachments: HIVE-1317.2.patch, HIVE-1317.patch > > > If a partition name contains characters such as ':' and '|' which have > special meaning in URI (hdfs uses URI internally for Path), > CombineHiveInputFormat throws an exception. URI was created in > CombineHiveInputFormat to compare a path belongs to a partition in > partitionToPathInfo. We should bypass URI creation by just string > comparisons. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when hadoop.version=0.20.1
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861922#action_12861922 ] John Sichi commented on HIVE-1192: -- Hmmm, looks like the mirrors have been updated to 0.20.2. Facebook hasn't yet gotten around to provisioning more archival versions under mirror.facebook.net/facebook/hive-deps. If you can't use 0.20.2, you could * retry with archive.apache.org (sometimes it is under too much load) * or see if you can google for a mirror which didn't get refreshed * or else try downloading it through some other means such as wget (ivy seems to be very sensitive to network problems) > Build fails when hadoop.version=0.20.1 > -- > > Key: HIVE-1192 > URL: https://issues.apache.org/jira/browse/HIVE-1192 > Project: Hadoop Hive > Issue Type: Bug > Components: Build Infrastructure >Reporter: Carl Steinbach > > Setting hadoop.version=0.20.1 causes the build to fail since > mirror.facebook.net/facebook/hive-deps does not have 0.20.1 > (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0). > Suggested fix: > * remove/ignore the hadoop.version configuration parameter > or > * Remove the patch numbers from these archives and use only the major.minor > numbers specified by the user to locate the appropriate tarball to download, > so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz. > * Optionally create new tarballs that only contain the components that are > actually needed for the build (Hadoop jars), and remove things that aren't > needed (all of the source files). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1083) allow sub-directories for an external table/partition
[ https://issues.apache.org/jira/browse/HIVE-1083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861920#action_12861920 ] John Sichi commented on HIVE-1083: -- Correction: local file system is probably OK; I just realized that when I tested, I was using the stock hadoop 0.20 version which does not have MAPREDUCE-1501 in it. > allow sub-directories for an external table/partition > - > > Key: HIVE-1083 > URL: https://issues.apache.org/jira/browse/HIVE-1083 > Project: Hadoop Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.6.0 >Reporter: Namit Jain >Assignee: Zheng Shao > Fix For: 0.6.0 > > > Sometimes users want to define an external table/partition based on all files > (recursively) inside a directory. > Currently most of the Hadoop InputFormat classes do not support that. We > should extract all files recursively in the directory, and add them to the > input path of the job. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1327) Group by partition column returns wrong results
Group by partition column returns wrong results --- Key: HIVE-1327 URL: https://issues.apache.org/jira/browse/HIVE-1327 Project: Hadoop Hive Issue Type: Bug Affects Versions: 0.6.0 Reporter: Ning Zhang Fix For: 0.6.0 hive> show partitions nzhang_part7; show partitions nzhang_part7; OK ds=2010-01-11 ds=2010-01-23 ds=2010-04-03 ds=2010-04-19 ds=2010-04-22 Time taken: 0.431 seconds [nzh...@dev303 /tmp] dfs -ls /user/facebook/warehouse/nzhang_part7/* -rw-r--r-- 3 nzhang supergroup1756123 2010-04-28 11:54 /user/facebook/warehouse/nzhang_part7/ds=2010-01-11/attempt_201004162336_176893_r_00_0.gz -rw-r--r-- 3 nzhang supergroup1758227 2010-04-28 11:54 /user/facebook/warehouse/nzhang_part7/ds=2010-01-11/attempt_201004162336_176893_r_01_0.gz -rw-r--r-- 3 nzhang supergroup1915969 2010-04-28 11:54 /user/facebook/warehouse/nzhang_part7/ds=2010-01-23/attempt_201004162336_176893_r_00_0.gz -rw-r--r-- 3 nzhang supergroup1943830 2010-04-28 11:54 /user/facebook/warehouse/nzhang_part7/ds=2010-01-23/attempt_201004162336_176893_r_01_0.gz -rw-r--r-- 3 nzhang supergroup1646739 2010-04-28 11:54 /user/facebook/warehouse/nzhang_part7/ds=2010-04-03/attempt_201004162336_176893_r_00_0.gz -rw-r--r-- 3 nzhang supergroup1641052 2010-04-28 11:54 /user/facebook/warehouse/nzhang_part7/ds=2010-04-03/attempt_201004162336_176893_r_01_0.gz -rw-r--r-- 3 nzhang supergroup 58601 2010-04-28 11:54 /user/facebook/warehouse/nzhang_part7/ds=2010-04-19/attempt_201004162336_176893_r_00_0.gz -rw-r--r-- 3 nzhang supergroup 57465 2010-04-28 11:54 /user/facebook/warehouse/nzhang_part7/ds=2010-04-19/attempt_201004162336_176893_r_01_0.gz -rw-r--r-- 3 nzhang supergroup1064491 2010-04-28 11:54 /user/facebook/warehouse/nzhang_part7/ds=2010-04-22/attempt_201004162336_176893_r_00_0.gz -rw-r--r-- 3 nzhang supergroup1070580 2010-04-28 11:54 /user/facebook/warehouse/nzhang_part7/ds=2010-04-22/attempt_201004162336_176893_r_01_0.gz hive> select ds, count(1) from nzhang_part7 where ds is not null group by ds; 2010-04-03 1761129 Time taken: 187.692 seconds -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when hadoop.version=0.20.1
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861889#action_12861889 ] Bill Au commented on HIVE-1192: --- I am trying to build 0.5.0 for Hadoop 0.20.1. It looks like 0.20.1 is only available on http://archive.apache.org/dist. So I set hadoop.mirror to that but I am getting the same error as in HIVE-984: https://issues.apache.org/jira/browse/HIVE-984 I can connect and download the file but there is a problem with the checksum. What do I need to do to proceed? > Build fails when hadoop.version=0.20.1 > -- > > Key: HIVE-1192 > URL: https://issues.apache.org/jira/browse/HIVE-1192 > Project: Hadoop Hive > Issue Type: Bug > Components: Build Infrastructure >Reporter: Carl Steinbach > > Setting hadoop.version=0.20.1 causes the build to fail since > mirror.facebook.net/facebook/hive-deps does not have 0.20.1 > (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0). > Suggested fix: > * remove/ignore the hadoop.version configuration parameter > or > * Remove the patch numbers from these archives and use only the major.minor > numbers specified by the user to locate the appropriate tarball to download, > so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz. > * Optionally create new tarballs that only contain the components that are > actually needed for the build (Hadoop jars), and remove things that aren't > needed (all of the source files). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1317) CombineHiveInputFormat throws exception when partition name contains special characters to URI
[ https://issues.apache.org/jira/browse/HIVE-1317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang updated HIVE-1317: - Attachment: HIVE-1317.2.patch Updated HIVE-1317.2.patch with the fix. > CombineHiveInputFormat throws exception when partition name contains special > characters to URI > -- > > Key: HIVE-1317 > URL: https://issues.apache.org/jira/browse/HIVE-1317 > Project: Hadoop Hive > Issue Type: Bug >Affects Versions: 0.6.0 >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.6.0 > > Attachments: HIVE-1317.2.patch, HIVE-1317.patch > > > If a partition name contains characters such as ':' and '|' which have > special meaning in URI (hdfs uses URI internally for Path), > CombineHiveInputFormat throws an exception. URI was created in > CombineHiveInputFormat to compare a path belongs to a partition in > partitionToPathInfo. We should bypass URI creation by just string > comparisons. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1317) CombineHiveInputFormat throws exception when partition name contains special characters to URI
[ https://issues.apache.org/jira/browse/HIVE-1317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang updated HIVE-1317: - Attachment: (was: HIVE-1317.2.patch) > CombineHiveInputFormat throws exception when partition name contains special > characters to URI > -- > > Key: HIVE-1317 > URL: https://issues.apache.org/jira/browse/HIVE-1317 > Project: Hadoop Hive > Issue Type: Bug >Affects Versions: 0.6.0 >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.6.0 > > Attachments: HIVE-1317.patch > > > If a partition name contains characters such as ':' and '|' which have > special meaning in URI (hdfs uses URI internally for Path), > CombineHiveInputFormat throws an exception. URI was created in > CombineHiveInputFormat to compare a path belongs to a partition in > partitionToPathInfo. We should bypass URI creation by just string > comparisons. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1317) CombineHiveInputFormat throws exception when partition name contains special characters to URI
[ https://issues.apache.org/jira/browse/HIVE-1317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861824#action_12861824 ] Namit Jain commented on HIVE-1317: -- Ning, I am getting a compilation error after applying the patch - [javac] symbol : variable File [javac] location: class org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.CombineFilter [javac] pString = p.toString() + File.separator; [javac]^ [javac] Note: Some input files use or override a deprecated API. [javac] Note: Recompile with -Xlint:deprecation for details. [javac] Note: Some input files use unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. [javac] 1 error BUILD FAILED > CombineHiveInputFormat throws exception when partition name contains special > characters to URI > -- > > Key: HIVE-1317 > URL: https://issues.apache.org/jira/browse/HIVE-1317 > Project: Hadoop Hive > Issue Type: Bug >Affects Versions: 0.6.0 >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.6.0 > > Attachments: HIVE-1317.2.patch, HIVE-1317.patch > > > If a partition name contains characters such as ':' and '|' which have > special meaning in URI (hdfs uses URI internally for Path), > CombineHiveInputFormat throws an exception. URI was created in > CombineHiveInputFormat to compare a path belongs to a partition in > partitionToPathInfo. We should bypass URI creation by just string > comparisons. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
