[jira] [Updated] (HIVE-2848) Add validation to HiveConf ConfVars
[ https://issues.apache.org/jira/browse/HIVE-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2848: Status: Patch Available (was: Open) > Add validation to HiveConf ConfVars > --- > > Key: HIVE-2848 > URL: https://issues.apache.org/jira/browse/HIVE-2848 > Project: Hive > Issue Type: Improvement >Affects Versions: 0.9.0 >Reporter: Kevin Wilfong >Assignee: Navis > Attachments: HIVE-2848.D2313.1.patch, HIVE-2848.D2313.2.patch, > HIVE-2848.D2313.3.patch, HIVE-2848.D2313.4.patch, HIVE-2848.D2313.5.patch > > > It would be good if we could add some validation to ConfVars in the HiveConf > when they are set. > At least if a variable is supposed to be boolean, numeric, or > strict/nonstrict, it shouldn't be too difficult to check for these. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2848) Add validation to HiveConf ConfVars
[ https://issues.apache.org/jira/browse/HIVE-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2848: -- Attachment: HIVE-2848.D2313.6.patch navis updated the revision "HIVE-2848 [jira] Add validation to HiveConf ConfVars". Reviewers: JIRA, cwsteinbach 1. Changed to use CamelCase (VAR_TYPE to VarType) 2. Added test explanation 3. Fixed error message I've missed some messages from phabricator. Sorry for that. REVISION DETAIL https://reviews.facebook.net/D2313 AFFECTED FILES cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ql/src/java/org/apache/hadoop/hive/ql/processors/SetProcessor.java ql/src/test/queries/clientnegative/set_hiveconf_validation0.q ql/src/test/queries/clientnegative/set_hiveconf_validation1.q ql/src/test/queries/clientpositive/udaf_context_ngrams.q ql/src/test/queries/clientpositive/udaf_ngrams.q ql/src/test/queries/clientpositive/udaf_percentile_approx.q ql/src/test/results/clientnegative/set_hiveconf_validation0.q.out ql/src/test/results/clientnegative/set_hiveconf_validation1.q.out ql/src/test/results/clientpositive/udaf_context_ngrams.q.out ql/src/test/results/clientpositive/udaf_ngrams.q.out ql/src/test/results/clientpositive/udaf_percentile_approx.q.out > Add validation to HiveConf ConfVars > --- > > Key: HIVE-2848 > URL: https://issues.apache.org/jira/browse/HIVE-2848 > Project: Hive > Issue Type: Improvement >Affects Versions: 0.9.0 >Reporter: Kevin Wilfong >Assignee: Navis > Attachments: HIVE-2848.D2313.1.patch, HIVE-2848.D2313.2.patch, > HIVE-2848.D2313.3.patch, HIVE-2848.D2313.4.patch, HIVE-2848.D2313.5.patch, > HIVE-2848.D2313.6.patch > > > It would be good if we could add some validation to ConfVars in the HiveConf > when they are set. > At least if a variable is supposed to be boolean, numeric, or > strict/nonstrict, it shouldn't be too difficult to check for these. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2961) Remove need for storage descriptors for view partitions
[ https://issues.apache.org/jira/browse/HIVE-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257330#comment-13257330 ] Kevin Wilfong commented on HIVE-2961: - We can revert those two patches instead. This patch does fix a null pointer exception that occurs for DESCRIBE FORMATTED on a view partition. But I can submit that in a separate patch. > Remove need for storage descriptors for view partitions > --- > > Key: HIVE-2961 > URL: https://issues.apache.org/jira/browse/HIVE-2961 > Project: Hive > Issue Type: Improvement >Affects Versions: 0.9.0 >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2961.D2877.1.patch > > > Storage descriptors were introduced for view partitions as part of HIVE-2795. > This was to allow view partitions to have the concept of a region as well as > to fix a NPE that resulted from calling describe formatted on them. > Since regions are no longer necessary for view partitions and the NPE can be > fixed by not displaying storage information for view partitions (or > displaying the view's storage information if this is preferred, although, > since a view partition is purely metadata, this does not seem necessary), > these are no longer needed. > This also means the Python script added which retroactively adds storage > descriptors to existing view partitions can be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2961) Remove need for storage descriptors for view partitions
[ https://issues.apache.org/jira/browse/HIVE-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257329#comment-13257329 ] Kevin Wilfong commented on HIVE-2961: - We can revert those two patches instead. This patch does fix a null pointer exception that occurs for DESCRIBE FORMATTED on a view partition. But I can submit that in a separate patch. > Remove need for storage descriptors for view partitions > --- > > Key: HIVE-2961 > URL: https://issues.apache.org/jira/browse/HIVE-2961 > Project: Hive > Issue Type: Improvement >Affects Versions: 0.9.0 >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2961.D2877.1.patch > > > Storage descriptors were introduced for view partitions as part of HIVE-2795. > This was to allow view partitions to have the concept of a region as well as > to fix a NPE that resulted from calling describe formatted on them. > Since regions are no longer necessary for view partitions and the NPE can be > fixed by not displaying storage information for view partitions (or > displaying the view's storage information if this is preferred, although, > since a view partition is purely metadata, this does not seem necessary), > these are no longer needed. > This also means the Python script added which retroactively adds storage > descriptors to existing view partitions can be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2964) Provide object inspectors of all visible columns for TOK_FUNCTIONSTAR type UDAF
Provide object inspectors of all visible columns for TOK_FUNCTIONSTAR type UDAF --- Key: HIVE-2964 URL: https://issues.apache.org/jira/browse/HIVE-2964 Project: Hive Issue Type: Improvement Reporter: Navis Assignee: Navis Priority: Minor Currently, TOK_FUNCTIONSTAR type UDAFs are provided with empty OI at init time and is sufficient for sum, the only UDAF accepts START as a parameter. But for some UDAFs, it is needed to know all OIs that can be used to access values of a row. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2964) Provide object inspectors of all visible columns for TOK_FUNCTIONSTAR type UDAF
[ https://issues.apache.org/jira/browse/HIVE-2964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2964: -- Attachment: HIVE-2964.D2895.1.patch navis requested code review of "HIVE-2964 [jira] Provide object inspectors of all visible columns for TOK_FUNCTIONSTAR type UDAF". Reviewers: JIRA DPAL-1123 Provide object inspectors of all visible columns for TOK_FUNCTIONSTAR type UDAF Currently, TOK_FUNCTIONSTAR type UDAFs are provided with empty OI at init time and is sufficient for sum, the only UDAF accepts START as a parameter. But for some UDAFs, it is needed to know all OIs that can be used to access values of a row. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D2895 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/6609/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. > Provide object inspectors of all visible columns for TOK_FUNCTIONSTAR type > UDAF > --- > > Key: HIVE-2964 > URL: https://issues.apache.org/jira/browse/HIVE-2964 > Project: Hive > Issue Type: Improvement >Reporter: Navis >Assignee: Navis >Priority: Minor > Attachments: HIVE-2964.D2895.1.patch > > > Currently, TOK_FUNCTIONSTAR type UDAFs are provided with empty OI at init > time and is sufficient for sum, the only UDAF accepts START as a parameter. > But for some UDAFs, it is needed to know all OIs that can be used to access > values of a row. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257474#comment-13257474 ] Thomas Weise commented on HIVE-2646: The patch needs to be applied through arc: arc patch D2883 chmod u+x testutils/hadoop svn propset svn:executable yes testutils/hadoop Al least for me, arc patch will choke on ivy/common-configurations.xml - download the file from phabricator directly. I had the problem with Andrew's patch also, so I'm not sure whether this is just me or an arc issue. > Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs > > > Key: HIVE-2646 > URL: https://issues.apache.org/jira/browse/HIVE-2646 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 0.8.0 >Reporter: Andrew Bayer >Assignee: Andrew Bayer >Priority: Critical > Fix For: 0.9.0 > > Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, > HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, > HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, > HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, > HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, > HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, > HIVE-2646.diff.txt > > > The current Hive Ivy dependency logic for its Hadoop dependencies is > problematic - depending on the tarball and extracting the jars from there, > rather than depending on the jars directly. It'd be great if this was fixed > to actually have the jar dependencies defined directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated HIVE-2646: --- Attachment: (was: HIVE-2646-fixtests.patch) > Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs > > > Key: HIVE-2646 > URL: https://issues.apache.org/jira/browse/HIVE-2646 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 0.8.0 >Reporter: Andrew Bayer >Assignee: Andrew Bayer >Priority: Critical > Fix For: 0.9.0 > > Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, > HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, > HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, > HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, > HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, > HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.diff.txt > > > The current Hive Ivy dependency logic for its Hadoop dependencies is > problematic - depending on the tarball and extracting the jars from there, > rather than depending on the jars directly. It'd be great if this was fixed > to actually have the jar dependencies defined directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated HIVE-2646: --- Attachment: HIVE-2646-fixtests.patch Patch file produced using arc export --unified --revision 2883 then replaced ivy/common-configurations section with output from svn diff ivy/common-configurations.xml > Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs > > > Key: HIVE-2646 > URL: https://issues.apache.org/jira/browse/HIVE-2646 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 0.8.0 >Reporter: Andrew Bayer >Assignee: Andrew Bayer >Priority: Critical > Fix For: 0.9.0 > > Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, > HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, > HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, > HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, > HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, > HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, > HIVE-2646.diff.txt > > > The current Hive Ivy dependency logic for its Hadoop dependencies is > problematic - depending on the tarball and extracting the jars from there, > rather than depending on the jars directly. It'd be great if this was fixed > to actually have the jar dependencies defined directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [VOTE] Apache Hive 0.9.0 Release Candidate 0
This vote stands cancelled because of various problems people have found in RC0. Thanks to all who tried RC0. I will respin RC1 shortly. Thanks, Ashutosh On Fri, Apr 13, 2012 at 11:50, Ashutosh Chauhan wrote: > Hey all, > > Apache Hive 0.9.0-rc0 is out and available at > http://people.apache.org/~hashutosh/hive-0.9.0-rc0/ > > Release notes are available at: > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310843&version=12317742 > > Please give it a try, let us know. > > Hive PMC members: Please test and vote. > > Thanks, > Ashutosh >
[jira] [Commented] (HIVE-2961) Remove need for storage descriptors for view partitions
[ https://issues.apache.org/jira/browse/HIVE-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257593#comment-13257593 ] Ashutosh Chauhan commented on HIVE-2961: bq. But if we end up taking that route, I think we should leave the upgrade-0.8.0-to-0.9.0.xxx.sql scripts for the sake of consistency. Do you mean delete the content of the file, thus effectively leaving empty files, since there is no upgrade needed when we revert those patches? > Remove need for storage descriptors for view partitions > --- > > Key: HIVE-2961 > URL: https://issues.apache.org/jira/browse/HIVE-2961 > Project: Hive > Issue Type: Improvement >Affects Versions: 0.9.0 >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2961.D2877.1.patch > > > Storage descriptors were introduced for view partitions as part of HIVE-2795. > This was to allow view partitions to have the concept of a region as well as > to fix a NPE that resulted from calling describe formatted on them. > Since regions are no longer necessary for view partitions and the NPE can be > fixed by not displaying storage information for view partitions (or > displaying the view's storage information if this is preferred, although, > since a view partition is purely metadata, this does not seem necessary), > these are no longer needed. > This also means the Python script added which retroactively adds storage > descriptors to existing view partitions can be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2965) Revert HIVE-2612
Revert HIVE-2612 Key: HIVE-2965 URL: https://issues.apache.org/jira/browse/HIVE-2965 Project: Hive Issue Type: Task Reporter: Ashutosh Chauhan Priority: Blocker Fix For: 0.9.0 In 4/19 contrib meeting it was decided to revert HIVE-2612. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2966) Revert HIVE-2795
Revert HIVE-2795 Key: HIVE-2966 URL: https://issues.apache.org/jira/browse/HIVE-2966 Project: Hive Issue Type: Task Components: Metastore Affects Versions: 0.9.0 Reporter: Ashutosh Chauhan Priority: Blocker Fix For: 0.9.0 In 4/18/12 contrib meeting, it was decided to revert HIVE-2795 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2961) Remove need for storage descriptors for view partitions
[ https://issues.apache.org/jira/browse/HIVE-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2961: --- Status: Open (was: Patch Available) Canceling patch since current patch may not be valid after HIVE-2795 is reverted. > Remove need for storage descriptors for view partitions > --- > > Key: HIVE-2961 > URL: https://issues.apache.org/jira/browse/HIVE-2961 > Project: Hive > Issue Type: Improvement >Affects Versions: 0.9.0 >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2961.D2877.1.patch > > > Storage descriptors were introduced for view partitions as part of HIVE-2795. > This was to allow view partitions to have the concept of a region as well as > to fix a NPE that resulted from calling describe formatted on them. > Since regions are no longer necessary for view partitions and the NPE can be > fixed by not displaying storage information for view partitions (or > displaying the view's storage information if this is preferred, although, > since a view partition is purely metadata, this does not seem necessary), > these are no longer needed. > This also means the Python script added which retroactively adds storage > descriptors to existing view partitions can be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
hive 0.9.0 RC1
RC0 failed to pass because of variety of reasons. In the meantime, various folks have requested for inclusion of other fixes in 0.9. Following is the list. Following list is for committers to review these patches and to get them committed in 0.9 branch. These are in "Patch Available" status. HIVE-2958 HIVE-2777 HIVE-2646 HIVE-2883 HIVE-2585 HIVE-538 HIVE-2904 Following list is for contributors/committers to contribute patches. These are in "Open" status. HIVE-2961 HIVE-2965 HIVE-2966 Thanks, Ashutosh
[jira] [Assigned] (HIVE-2804) Task log retrieval fails on secure cluster
[ https://issues.apache.org/jira/browse/HIVE-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenxiao Luo reassigned HIVE-2804: -- Assignee: Zhenxiao Luo > Task log retrieval fails on secure cluster > -- > > Key: HIVE-2804 > URL: https://issues.apache.org/jira/browse/HIVE-2804 > Project: Hive > Issue Type: Bug > Components: Diagnosability, Query Processor, Security >Reporter: Carl Steinbach >Assignee: Zhenxiao Luo > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2804) Task log retrieval fails on secure cluster
[ https://issues.apache.org/jira/browse/HIVE-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257616#comment-13257616 ] Zhenxiao Luo commented on HIVE-2804: Also relates to the following: http://search-hadoop.com/m/LR0vP1Y5227&subj=Hive+Hadoop+Log+Retrieval+Problem > Task log retrieval fails on secure cluster > -- > > Key: HIVE-2804 > URL: https://issues.apache.org/jira/browse/HIVE-2804 > Project: Hive > Issue Type: Bug > Components: Diagnosability, Query Processor, Security >Reporter: Carl Steinbach > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-0.8.1-SNAPSHOT-h0.21 - Build # 258 - Fixed
Changes for Build #257 Changes for Build #258 All tests passed The Apache Jenkins build system has built Hive-0.8.1-SNAPSHOT-h0.21 (build #258) Status: Fixed Check console output at https://builds.apache.org/job/Hive-0.8.1-SNAPSHOT-h0.21/258/ to view the results.
[jira] [Commented] (HIVE-2961) Remove need for storage descriptors for view partitions
[ https://issues.apache.org/jira/browse/HIVE-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257772#comment-13257772 ] Carl Steinbach commented on HIVE-2961: -- bq. Do you mean delete the content of the file, thus effectively leaving empty files, since there is no upgrade needed when we revert those patches? Yup, that's basically what I meant, but now that I think more about it I agree that it doesn't make much sense. > Remove need for storage descriptors for view partitions > --- > > Key: HIVE-2961 > URL: https://issues.apache.org/jira/browse/HIVE-2961 > Project: Hive > Issue Type: Improvement >Affects Versions: 0.9.0 >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2961.D2877.1.patch > > > Storage descriptors were introduced for view partitions as part of HIVE-2795. > This was to allow view partitions to have the concept of a region as well as > to fix a NPE that resulted from calling describe formatted on them. > Since regions are no longer necessary for view partitions and the NPE can be > fixed by not displaying storage information for view partitions (or > displaying the view's storage information if this is preferred, although, > since a view partition is purely metadata, this does not seem necessary), > these are no longer needed. > This also means the Python script added which retroactively adds storage > descriptors to existing view partitions can be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257833#comment-13257833 ] Thomas Weise commented on HIVE-2646: All test pass in latest run: Tests FailuresErrors Success rateTime 16780 0 100.00% 19245.326 > Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs > > > Key: HIVE-2646 > URL: https://issues.apache.org/jira/browse/HIVE-2646 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 0.8.0 >Reporter: Andrew Bayer >Assignee: Andrew Bayer >Priority: Critical > Fix For: 0.9.0 > > Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, > HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, > HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, > HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, > HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, > HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, > HIVE-2646.diff.txt > > > The current Hive Ivy dependency logic for its Hadoop dependencies is > problematic - depending on the tarball and extracting the jars from there, > rather than depending on the jars directly. It'd be great if this was fixed > to actually have the jar dependencies defined directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2967) Statement.updateQuery() returns Method Not Supported exception
Statement.updateQuery() returns Method Not Supported exception -- Key: HIVE-2967 URL: https://issues.apache.org/jira/browse/HIVE-2967 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.8.0 Environment: Remotely connecting from a Windows machine via JDBC to Hive installed on Linux Kernel-2.6.18-glibc-2.5 x86_64 64bit Reporter: Alex Piyevsky When our software sends a CREATE TABLE command to HIVE via JDBC, a Statement.updateQuery() statement is issued which results in a Method Not Supported exception. We did not get this error in the previous version of Hive (0.7.1). Please advise, did something change within the 0.8 JDBC driver, is a statement like this no longer supported? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Meta data didn't rollback after accessing mysql fail
Hi Kai, I agree that this sounds like a bug. Can you describe the steps required to reproduce this behavior on 0.7.1? Thanks. Carl On Wed, Apr 18, 2012 at 11:03 PM, Zhang Kai wrote: > Hi all, > > I got a problem when using Hive 0.7.1. > After I created a table with JDO failure, I can still find the table by > using 'show tables;'. > > I grep the code and find there is a transaction mechanism when hive > interacts with metastore. > However, it seems that the transaction doesn't work. > > I have tested this for several times. > I manually throw an exception before ObjectStore commit the 'CREATE > TABLE' transaction. > But I can find that a new record in TBLS in mysql. > So I guess meta data didn't rollback correctly. > > I also notice this could also happen on Hive 0.8 and 0.9. > However, I haven't tested it. > > I think this is a fatal error and can make hive metastore unavailable. > Is there anyone can help me solve this problem? > > Thanks. > > Kai Zhang >
Re: Problems with Arc/Phabricator
Hit a new problem with arc today: Fatal error: Uncaught exception 'Exception' with message 'Host returned HTTP/200, but invalid JSON data in response to a Conduit method call: Warning: Unknown: POST Content-Length of 9079953 bytes exceeds the limit of 8388608 bytes in Unknown on line 0 for(;;);{"result":null,"error_code":"ERR-INVALID-SESSION","error_info":"Session key is not present."}' in /Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitFuture.php:48 Stack trace: #0 /Users/ashutosh/work/hive/libphutil/src/future/proxy/FutureProxy.php(62): ConduitFuture->didReceiveResult(Array) #1 /Users/ashutosh/work/hive/libphutil/src/future/proxy/FutureProxy.php(39): FutureProxy->getResult() #2 /Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitClient.php(52): FutureProxy->resolve() #3 /Users/ashutosh/work/hive/arcanist/src/workflow/diff/ArcanistDiffWorkflow.php(341): ConduitClient->callMethodSynchronous('differential.cr...', Array) #4 /Users/ashutosh/work/hive/arcanist/scripts/arcanist.php(266): ArcanistDiffWo in /Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitFuture.php on line 48 Any ideas how to solve this? Thanks, Ashutosh On Wed, Apr 11, 2012 at 18:37, Edward Capriolo wrote: > I think the most practical solution is try and use arc/phab and then > if there is a problem fall back to Jira and do it the old way. > > Edward > > On Wed, Apr 11, 2012 at 7:17 PM, Carl Steinbach wrote: > > +1 to switching over to Git. > > > > As for the rest of the Phabricator/Gerrit/Reviewboard discussion, I think > > we should pick this up again at the contributor meeting on Wednesday. > > > > Thanks. > > > > Carl > > > > On Wed, Apr 11, 2012 at 12:19 PM, Ashutosh Chauhan >wrote: > > > >> +1 on moving away from arc/phabricator. It works great when it works, > but > >> most of the time it doesnt work. > >> > >> Ashutosh > >> > >> On Wed, Apr 11, 2012 at 11:57, Owen O'Malley > wrote: > >> > >> > On Wed, Apr 11, 2012 at 11:48 AM, Edward Capriolo < > edlinuxg...@gmail.com > >> > > >> > wrote: > >> > > If we are going to switch from fabricator we just might as well go > >> > > back to not using anything. Review board was really clunky and > >> > > confusing. > >> > > >> > I'm mostly +1 to that. If no one is supporting phabricator, then it > >> > won't work for long. Personally, I'd love it if we could move Hive to > >> > git completely. Has anyone used gerrit? The videos of it make it look > >> > better than sliced bread. > >> > > >> > -- Owen > >> > > >> >
[jira] [Updated] (HIVE-2965) Revert HIVE-2612
[ https://issues.apache.org/jira/browse/HIVE-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2965: --- Attachment: hive-2765.patch Arc is unwieldy again. Can't convince it to submit the patch. So, uploading directly, patch reverting HIVE-2612 generated using svn merge command. > Revert HIVE-2612 > > > Key: HIVE-2965 > URL: https://issues.apache.org/jira/browse/HIVE-2965 > Project: Hive > Issue Type: Task >Reporter: Ashutosh Chauhan >Priority: Blocker > Fix For: 0.9.0 > > Attachments: hive-2765.patch > > > In 4/19 contrib meeting it was decided to revert HIVE-2612. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2965) Revert HIVE-2612
[ https://issues.apache.org/jira/browse/HIVE-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2965: --- Status: Patch Available (was: Open) Ready for review. > Revert HIVE-2612 > > > Key: HIVE-2965 > URL: https://issues.apache.org/jira/browse/HIVE-2965 > Project: Hive > Issue Type: Task >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Blocker > Fix For: 0.9.0 > > Attachments: hive-2765.patch > > > In 4/19 contrib meeting it was decided to revert HIVE-2612. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2965) Revert HIVE-2612
[ https://issues.apache.org/jira/browse/HIVE-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan reassigned HIVE-2965: -- Assignee: Ashutosh Chauhan > Revert HIVE-2612 > > > Key: HIVE-2965 > URL: https://issues.apache.org/jira/browse/HIVE-2965 > Project: Hive > Issue Type: Task >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Blocker > Fix For: 0.9.0 > > Attachments: hive-2765.patch > > > In 4/19 contrib meeting it was decided to revert HIVE-2612. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Problems with Arc/Phabricator
I think as an Apache project, using code review tools outside apache should never be a "requirement". I agree that arc/phab are great tools, and they help the patch/review process a lot. But I think what is essential for long term sustainability as a project is to lower the entry barriers for contributing. Not every developer who wants to contribute some code will be willing to work with phab/rb. What I would suggest is to always have the option to submit patches via jira, and make rb / phab optional. Just my 2 cents, Enis On Thu, Apr 19, 2012 at 4:35 PM, Ashutosh Chauhan wrote: > Hit a new problem with arc today: > > Fatal error: Uncaught exception 'Exception' with message 'Host returned > HTTP/200, but invalid JSON data in response to a Conduit method call: > > Warning: Unknown: POST Content-Length of 9079953 bytes exceeds the > limit of 8388608 bytes in Unknown on line 0 > > for(;;);{"result":null,"error_code":"ERR-INVALID-SESSION","error_info":"Session > key is not present."}' in > /Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitFuture.php:48 > Stack trace: > #0 > /Users/ashutosh/work/hive/libphutil/src/future/proxy/FutureProxy.php(62): > ConduitFuture->didReceiveResult(Array) > #1 > /Users/ashutosh/work/hive/libphutil/src/future/proxy/FutureProxy.php(39): > FutureProxy->getResult() > #2 > > /Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitClient.php(52): > FutureProxy->resolve() > #3 > > /Users/ashutosh/work/hive/arcanist/src/workflow/diff/ArcanistDiffWorkflow.php(341): > ConduitClient->callMethodSynchronous('differential.cr...', Array) > #4 /Users/ashutosh/work/hive/arcanist/scripts/arcanist.php(266): > ArcanistDiffWo in > /Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitFuture.php on > line 48 > > > Any ideas how to solve this? > > Thanks, > Ashutosh > > On Wed, Apr 11, 2012 at 18:37, Edward Capriolo >wrote: > > > I think the most practical solution is try and use arc/phab and then > > if there is a problem fall back to Jira and do it the old way. > > > > Edward > > > > On Wed, Apr 11, 2012 at 7:17 PM, Carl Steinbach > wrote: > > > +1 to switching over to Git. > > > > > > As for the rest of the Phabricator/Gerrit/Reviewboard discussion, I > think > > > we should pick this up again at the contributor meeting on Wednesday. > > > > > > Thanks. > > > > > > Carl > > > > > > On Wed, Apr 11, 2012 at 12:19 PM, Ashutosh Chauhan < > hashut...@apache.org > > >wrote: > > > > > >> +1 on moving away from arc/phabricator. It works great when it works, > > but > > >> most of the time it doesnt work. > > >> > > >> Ashutosh > > >> > > >> On Wed, Apr 11, 2012 at 11:57, Owen O'Malley > > wrote: > > >> > > >> > On Wed, Apr 11, 2012 at 11:48 AM, Edward Capriolo < > > edlinuxg...@gmail.com > > >> > > > >> > wrote: > > >> > > If we are going to switch from fabricator we just might as well go > > >> > > back to not using anything. Review board was really clunky and > > >> > > confusing. > > >> > > > >> > I'm mostly +1 to that. If no one is supporting phabricator, then it > > >> > won't work for long. Personally, I'd love it if we could move Hive > to > > >> > git completely. Has anyone used gerrit? The videos of it make it > look > > >> > better than sliced bread. > > >> > > > >> > -- Owen > > >> > > > >> > > >
[jira] [Updated] (HIVE-2966) Revert HIVE-2795
[ https://issues.apache.org/jira/browse/HIVE-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-2966: Attachment: HIVE-2966.1.patch HIVE-2966.1.patch - Attaching the patch directly, as arc command didn't do it for me. Here is the phabricator link - https://reviews.facebook.net/D2907 > Revert HIVE-2795 > > > Key: HIVE-2966 > URL: https://issues.apache.org/jira/browse/HIVE-2966 > Project: Hive > Issue Type: Task > Components: Metastore >Affects Versions: 0.9.0 >Reporter: Ashutosh Chauhan >Priority: Blocker > Fix For: 0.9.0 > > Attachments: HIVE-2966.1.patch > > > In 4/18/12 contrib meeting, it was decided to revert HIVE-2795 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Problems with Arc/Phabricator
Just throwing this out there. The phabricator IRC has more people and is usually more active then Hive IRC. #JustSaying... On Thu, Apr 19, 2012 at 7:35 PM, Ashutosh Chauhan wrote: > Hit a new problem with arc today: > > Fatal error: Uncaught exception 'Exception' with message 'Host returned > HTTP/200, but invalid JSON data in response to a Conduit method call: > > Warning: Unknown: POST Content-Length of 9079953 bytes exceeds the > limit of 8388608 bytes in Unknown on line 0 > for(;;);{"result":null,"error_code":"ERR-INVALID-SESSION","error_info":"Session > key is not present."}' in > /Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitFuture.php:48 > Stack trace: > #0 > /Users/ashutosh/work/hive/libphutil/src/future/proxy/FutureProxy.php(62): > ConduitFuture->didReceiveResult(Array) > #1 > /Users/ashutosh/work/hive/libphutil/src/future/proxy/FutureProxy.php(39): > FutureProxy->getResult() > #2 > /Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitClient.php(52): > FutureProxy->resolve() > #3 > /Users/ashutosh/work/hive/arcanist/src/workflow/diff/ArcanistDiffWorkflow.php(341): > ConduitClient->callMethodSynchronous('differential.cr...', Array) > #4 /Users/ashutosh/work/hive/arcanist/scripts/arcanist.php(266): > ArcanistDiffWo in > /Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitFuture.php on > line 48 > > > Any ideas how to solve this? > > Thanks, > Ashutosh > > On Wed, Apr 11, 2012 at 18:37, Edward Capriolo wrote: > >> I think the most practical solution is try and use arc/phab and then >> if there is a problem fall back to Jira and do it the old way. >> >> Edward >> >> On Wed, Apr 11, 2012 at 7:17 PM, Carl Steinbach wrote: >> > +1 to switching over to Git. >> > >> > As for the rest of the Phabricator/Gerrit/Reviewboard discussion, I think >> > we should pick this up again at the contributor meeting on Wednesday. >> > >> > Thanks. >> > >> > Carl >> > >> > On Wed, Apr 11, 2012 at 12:19 PM, Ashutosh Chauhan > >wrote: >> > >> >> +1 on moving away from arc/phabricator. It works great when it works, >> but >> >> most of the time it doesnt work. >> >> >> >> Ashutosh >> >> >> >> On Wed, Apr 11, 2012 at 11:57, Owen O'Malley >> wrote: >> >> >> >> > On Wed, Apr 11, 2012 at 11:48 AM, Edward Capriolo < >> edlinuxg...@gmail.com >> >> > >> >> > wrote: >> >> > > If we are going to switch from fabricator we just might as well go >> >> > > back to not using anything. Review board was really clunky and >> >> > > confusing. >> >> > >> >> > I'm mostly +1 to that. If no one is supporting phabricator, then it >> >> > won't work for long. Personally, I'd love it if we could move Hive to >> >> > git completely. Has anyone used gerrit? The videos of it make it look >> >> > better than sliced bread. >> >> > >> >> > -- Owen >> >> > >> >> >>
Re: Problems with Arc/Phabricator
Problem while using arc on ubuntu $ arc patch D2871 ARC: Cannot mix P and A UNIX: No such file or directory Any ideas whats up there. Thanks, Ashutosh On Thu, Apr 19, 2012 at 17:19, Edward Capriolo wrote: > Just throwing this out there. The phabricator IRC has more people and > is usually more active then Hive IRC. > > #JustSaying... > > On Thu, Apr 19, 2012 at 7:35 PM, Ashutosh Chauhan > wrote: > > Hit a new problem with arc today: > > > > Fatal error: Uncaught exception 'Exception' with message 'Host returned > > HTTP/200, but invalid JSON data in response to a Conduit method call: > > > > Warning: Unknown: POST Content-Length of 9079953 bytes exceeds > the > > limit of 8388608 bytes in Unknown on line 0 > > > for(;;);{"result":null,"error_code":"ERR-INVALID-SESSION","error_info":"Session > > key is not present."}' in > > > /Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitFuture.php:48 > > Stack trace: > > #0 > > /Users/ashutosh/work/hive/libphutil/src/future/proxy/FutureProxy.php(62): > > ConduitFuture->didReceiveResult(Array) > > #1 > > /Users/ashutosh/work/hive/libphutil/src/future/proxy/FutureProxy.php(39): > > FutureProxy->getResult() > > #2 > > > /Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitClient.php(52): > > FutureProxy->resolve() > > #3 > > > /Users/ashutosh/work/hive/arcanist/src/workflow/diff/ArcanistDiffWorkflow.php(341): > > ConduitClient->callMethodSynchronous('differential.cr...', Array) > > #4 /Users/ashutosh/work/hive/arcanist/scripts/arcanist.php(266): > > ArcanistDiffWo in > > /Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitFuture.php > on > > line 48 > > > > > > Any ideas how to solve this? > > > > Thanks, > > Ashutosh > > > > On Wed, Apr 11, 2012 at 18:37, Edward Capriolo >wrote: > > > >> I think the most practical solution is try and use arc/phab and then > >> if there is a problem fall back to Jira and do it the old way. > >> > >> Edward > >> > >> On Wed, Apr 11, 2012 at 7:17 PM, Carl Steinbach > wrote: > >> > +1 to switching over to Git. > >> > > >> > As for the rest of the Phabricator/Gerrit/Reviewboard discussion, I > think > >> > we should pick this up again at the contributor meeting on Wednesday. > >> > > >> > Thanks. > >> > > >> > Carl > >> > > >> > On Wed, Apr 11, 2012 at 12:19 PM, Ashutosh Chauhan < > hashut...@apache.org > >> >wrote: > >> > > >> >> +1 on moving away from arc/phabricator. It works great when it works, > >> but > >> >> most of the time it doesnt work. > >> >> > >> >> Ashutosh > >> >> > >> >> On Wed, Apr 11, 2012 at 11:57, Owen O'Malley > >> wrote: > >> >> > >> >> > On Wed, Apr 11, 2012 at 11:48 AM, Edward Capriolo < > >> edlinuxg...@gmail.com > >> >> > > >> >> > wrote: > >> >> > > If we are going to switch from fabricator we just might as well > go > >> >> > > back to not using anything. Review board was really clunky and > >> >> > > confusing. > >> >> > > >> >> > I'm mostly +1 to that. If no one is supporting phabricator, then it > >> >> > won't work for long. Personally, I'd love it if we could move Hive > to > >> >> > git completely. Has anyone used gerrit? The videos of it make it > look > >> >> > better than sliced bread. > >> >> > > >> >> > -- Owen > >> >> > > >> >> > >> >
Re: hive 0.9.0 RC1
Am I missing something about? https://issues.apache.org/jira/browse/HIVE-2777 I see no way to access this feature from the hive QL language. Should we delay releases on features not usable? Edward On Thu, Apr 19, 2012 at 1:28 PM, Ashutosh Chauhan wrote: > RC0 failed to pass because of variety of reasons. In the meantime, various > folks have requested for inclusion of other fixes in 0.9. Following is the > list. > > Following list is for committers to review these patches and to get them > committed in 0.9 branch. These are in "Patch Available" status. > HIVE-2958 > HIVE-2777 > HIVE-2646 > HIVE-2883 > HIVE-2585 > HIVE-538 > HIVE-2904 > > Following list is for contributors/committers to contribute patches. These > are in "Open" status. > HIVE-2961 > HIVE-2965 > HIVE-2966 > > Thanks, > Ashutosh
[jira] [Commented] (HIVE-2958) GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger]
[ https://issues.apache.org/jira/browse/HIVE-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257950#comment-13257950 ] Phabricator commented on HIVE-2958: --- ashutoshc has accepted the revision "HIVE-2958 [jira] GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger]". +1 REVISION DETAIL https://reviews.facebook.net/D2871 BRANCH DPAL- > GROUP BY causing ClassCastException [LazyDioInteger cannot be cast > LazyInteger] > --- > > Key: HIVE-2958 > URL: https://issues.apache.org/jira/browse/HIVE-2958 > Project: Hive > Issue Type: Bug > Components: HBase Handler >Affects Versions: 0.9.0 > Environment: HBase 0.90.4, Hive 0.90 snapshot (trunk) built today >Reporter: Tim Robertson >Assignee: Navis >Priority: Blocker > Attachments: HIVE-2958.D2871.1.patch > > > This relates to https://issues.apache.org/jira/browse/HIVE-1634. > The following work fine: > {code} > CREATE EXTERNAL TABLE tim_hbase_occurrence ( > id int, > scientific_name string, > data_resource_id int > ) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH > SERDEPROPERTIES ( > "hbase.columns.mapping" = ":key#b,v:scientific_name#s,v:data_resource_id#b" > ) TBLPROPERTIES( > "hbase.table.name" = "mini_occurrences", > "hbase.table.default.storage.type" = "binary" > ); > SELECT * FROM tim_hbase_occurrence LIMIT 3; > SELECT * FROM tim_hbase_occurrence WHERE data_resource_id=1081 LIMIT 3; > {code} > However, the following fails: > {code} > SELECT data_resource_id, count(*) FROM tim_hbase_occurrence GROUP BY > data_resource_id; > {code} > The error given: > {code} > 0 TS > 2012-04-17 16:58:45,693 INFO org.apache.hadoop.hive.ql.exec.MapOperator: > Initialization Done 7 MAP > 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: > Processing alias tim_hbase_occurrence for file > hdfs://c1n2.gbif.org/user/hive/warehouse/tim_hbase_occurrence > 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7 > forwarding 1 rows > 2012-04-17 16:58:45,714 INFO > org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows > 2012-04-17 16:58:45,716 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 > forwarding 1 rows > 2012-04-17 16:58:45,723 FATAL ExecMapper: > org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while > processing row {"id":1444,"scientific_name":null,"data_resource_id":1081} > at > org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548) > at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) > at org.apache.hadoop.mapred.Child$4.run(Child.java:270) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157) > at org.apache.hadoop.mapred.Child.main(Child.java:264) > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.ClassCastException: > org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to > org.apache.hadoop.hive.serde2.lazy.LazyInteger > at > org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:737) > at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) > at > org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84) > at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) > at > org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83) > at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) > at > org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529) > ... 9 more > Caused by: java.lang.ClassCastException: > org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to > org.apache.hadoop.hive.serde2.lazy.LazyInteger > at > org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyIntObjectInspector.copyObject(LazyIntObjectInspector.java:43) > at > org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:239) > at > org.apache.hadoop
[jira] [Updated] (HIVE-2958) GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger]
[ https://issues.apache.org/jira/browse/HIVE-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2958: --- Resolution: Fixed Fix Version/s: 0.9.0 Status: Resolved (was: Patch Available) Committed to trunk and 0.9. Thanks, Navis! > GROUP BY causing ClassCastException [LazyDioInteger cannot be cast > LazyInteger] > --- > > Key: HIVE-2958 > URL: https://issues.apache.org/jira/browse/HIVE-2958 > Project: Hive > Issue Type: Bug > Components: HBase Handler >Affects Versions: 0.9.0 > Environment: HBase 0.90.4, Hive 0.90 snapshot (trunk) built today >Reporter: Tim Robertson >Assignee: Navis >Priority: Blocker > Fix For: 0.9.0 > > Attachments: HIVE-2958.D2871.1.patch > > > This relates to https://issues.apache.org/jira/browse/HIVE-1634. > The following work fine: > {code} > CREATE EXTERNAL TABLE tim_hbase_occurrence ( > id int, > scientific_name string, > data_resource_id int > ) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH > SERDEPROPERTIES ( > "hbase.columns.mapping" = ":key#b,v:scientific_name#s,v:data_resource_id#b" > ) TBLPROPERTIES( > "hbase.table.name" = "mini_occurrences", > "hbase.table.default.storage.type" = "binary" > ); > SELECT * FROM tim_hbase_occurrence LIMIT 3; > SELECT * FROM tim_hbase_occurrence WHERE data_resource_id=1081 LIMIT 3; > {code} > However, the following fails: > {code} > SELECT data_resource_id, count(*) FROM tim_hbase_occurrence GROUP BY > data_resource_id; > {code} > The error given: > {code} > 0 TS > 2012-04-17 16:58:45,693 INFO org.apache.hadoop.hive.ql.exec.MapOperator: > Initialization Done 7 MAP > 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: > Processing alias tim_hbase_occurrence for file > hdfs://c1n2.gbif.org/user/hive/warehouse/tim_hbase_occurrence > 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7 > forwarding 1 rows > 2012-04-17 16:58:45,714 INFO > org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows > 2012-04-17 16:58:45,716 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 > forwarding 1 rows > 2012-04-17 16:58:45,723 FATAL ExecMapper: > org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while > processing row {"id":1444,"scientific_name":null,"data_resource_id":1081} > at > org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548) > at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) > at org.apache.hadoop.mapred.Child$4.run(Child.java:270) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157) > at org.apache.hadoop.mapred.Child.main(Child.java:264) > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.ClassCastException: > org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to > org.apache.hadoop.hive.serde2.lazy.LazyInteger > at > org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:737) > at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) > at > org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84) > at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) > at > org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83) > at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) > at > org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529) > ... 9 more > Caused by: java.lang.ClassCastException: > org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to > org.apache.hadoop.hive.serde2.lazy.LazyInteger > at > org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyIntObjectInspector.copyObject(LazyIntObjectInspector.java:43) > at > org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:239) > at > org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:1
[jira] [Commented] (HIVE-2958) GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger]
[ https://issues.apache.org/jira/browse/HIVE-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257957#comment-13257957 ] Phabricator commented on HIVE-2958: --- navis has committed the revision "HIVE-2958 [jira] GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger]". Change committed by hashutosh. REVISION DETAIL https://reviews.facebook.net/D2871 COMMIT https://reviews.facebook.net/rHIVE1328157 > GROUP BY causing ClassCastException [LazyDioInteger cannot be cast > LazyInteger] > --- > > Key: HIVE-2958 > URL: https://issues.apache.org/jira/browse/HIVE-2958 > Project: Hive > Issue Type: Bug > Components: HBase Handler >Affects Versions: 0.9.0 > Environment: HBase 0.90.4, Hive 0.90 snapshot (trunk) built today >Reporter: Tim Robertson >Assignee: Navis >Priority: Blocker > Fix For: 0.9.0 > > Attachments: HIVE-2958.D2871.1.patch > > > This relates to https://issues.apache.org/jira/browse/HIVE-1634. > The following work fine: > {code} > CREATE EXTERNAL TABLE tim_hbase_occurrence ( > id int, > scientific_name string, > data_resource_id int > ) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH > SERDEPROPERTIES ( > "hbase.columns.mapping" = ":key#b,v:scientific_name#s,v:data_resource_id#b" > ) TBLPROPERTIES( > "hbase.table.name" = "mini_occurrences", > "hbase.table.default.storage.type" = "binary" > ); > SELECT * FROM tim_hbase_occurrence LIMIT 3; > SELECT * FROM tim_hbase_occurrence WHERE data_resource_id=1081 LIMIT 3; > {code} > However, the following fails: > {code} > SELECT data_resource_id, count(*) FROM tim_hbase_occurrence GROUP BY > data_resource_id; > {code} > The error given: > {code} > 0 TS > 2012-04-17 16:58:45,693 INFO org.apache.hadoop.hive.ql.exec.MapOperator: > Initialization Done 7 MAP > 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: > Processing alias tim_hbase_occurrence for file > hdfs://c1n2.gbif.org/user/hive/warehouse/tim_hbase_occurrence > 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7 > forwarding 1 rows > 2012-04-17 16:58:45,714 INFO > org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows > 2012-04-17 16:58:45,716 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 > forwarding 1 rows > 2012-04-17 16:58:45,723 FATAL ExecMapper: > org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while > processing row {"id":1444,"scientific_name":null,"data_resource_id":1081} > at > org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548) > at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) > at org.apache.hadoop.mapred.Child$4.run(Child.java:270) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157) > at org.apache.hadoop.mapred.Child.main(Child.java:264) > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.ClassCastException: > org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to > org.apache.hadoop.hive.serde2.lazy.LazyInteger > at > org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:737) > at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) > at > org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84) > at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) > at > org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83) > at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) > at > org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529) > ... 9 more > Caused by: java.lang.ClassCastException: > org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to > org.apache.hadoop.hive.serde2.lazy.LazyInteger > at > org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyIntObjectInspector.copyObject(LazyIntObjectInspector.java:43) > at > org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorU
Re: hive 0.9.0 RC1
Release is not going to be blocked on it. If the patch gets committed on trunk by the time I roll RC1 I will merge it in. But, first step is to have it in trunk. Thanks, Ashutosh On Thu, Apr 19, 2012 at 17:32, Edward Capriolo wrote: > Am I missing something about? > > https://issues.apache.org/jira/browse/HIVE-2777 > > I see no way to access this feature from the hive QL language. Should > we delay releases on features not usable? > > Edward > > > On Thu, Apr 19, 2012 at 1:28 PM, Ashutosh Chauhan > wrote: > > RC0 failed to pass because of variety of reasons. In the meantime, > various > > folks have requested for inclusion of other fixes in 0.9. Following is > the > > list. > > > > Following list is for committers to review these patches and to get them > > committed in 0.9 branch. These are in "Patch Available" status. > > HIVE-2958 > > HIVE-2777 > > HIVE-2646 > > HIVE-2883 > > HIVE-2585 > > HIVE-538 > > HIVE-2904 > > > > Following list is for contributors/committers to contribute patches. > These > > are in "Open" status. > > HIVE-2961 > > HIVE-2965 > > HIVE-2966 > > > > Thanks, > > Ashutosh >
[jira] [Created] (HIVE-2968) Add char(n), character(n), character varying(n), and varchar(n) as datatype aliases for string.
Add char(n), character(n), character varying(n), and varchar(n) as datatype aliases for string. --- Key: HIVE-2968 URL: https://issues.apache.org/jira/browse/HIVE-2968 Project: Hive Issue Type: Improvement Reporter: Joey Echeverria MySQL and Postgres support char(n), varchar(n), character(n), and character varying(n) as data types. In Hive, those would map to strings. It would be useful to support these types as aliases when creating a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2968) Add char(n), character(n), character varying(n), and varchar(n) as datatype aliases for string.
[ https://issues.apache.org/jira/browse/HIVE-2968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joey Echeverria updated HIVE-2968: -- Description: MySQL and Postgres support char\(n), varchar\(n), character\(n), and character varying\(n) as data types. In Hive, those would map to strings. It would be useful to support these types as aliases when creating a table. (was: MySQL and Postgres support char(n), varchar(n), character(n), and character varying(n) as data types. In Hive, those would map to strings. It would be useful to support these types as aliases when creating a table.) > Add char(n), character(n), character varying(n), and varchar(n) as datatype > aliases for string. > --- > > Key: HIVE-2968 > URL: https://issues.apache.org/jira/browse/HIVE-2968 > Project: Hive > Issue Type: Improvement >Reporter: Joey Echeverria > > MySQL and Postgres support char\(n), varchar\(n), character\(n), and > character varying\(n) as data types. In Hive, those would map to strings. It > would be useful to support these types as aliases when creating a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2804) Task log retrieval fails on secure cluster
[ https://issues.apache.org/jira/browse/HIVE-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257964#comment-13257964 ] Edward Capriolo commented on HIVE-2804: --- It seems like you will have to make a shim or variable that can be used to generate a version dependent URL. > Task log retrieval fails on secure cluster > -- > > Key: HIVE-2804 > URL: https://issues.apache.org/jira/browse/HIVE-2804 > Project: Hive > Issue Type: Bug > Components: Diagnosability, Query Processor, Security >Reporter: Carl Steinbach >Assignee: Zhenxiao Luo > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Meta data didn't rollback after accessing mysql fail
Hi Carl To reproduce it, I added one line throw new MetaException("test"); at line 628 in ObjectStore.java and comment the following line //commited = commitTransaction(); I also added some codes for tracing the behaviors of open/commit/rollback transaction and executeWithRetry. Here are the results I saw in my console: hive> > create table test(str string); 12/04/20 11:01:18 ERROR metastore.HiveMetaStore: execute with retry: 0 12/04/20 11:01:18 ERROR metastore.ObjectStore: enter open transaction 12/04/20 11:01:18 ERROR metastore.ObjectStore: opening 12/04/20 11:01:18 ERROR metastore.ObjectStore: enter open transaction 12/04/20 11:01:18 ERROR metastore.ObjectStore: enter open transaction 12/04/20 11:01:18 ERROR metastore.ObjectStore: enter commit transaction 12/04/20 11:01:18 ERROR metastore.ObjectStore: enter commit transaction 12/04/20 11:01:18 ERROR metastore.ObjectStore: enter open transaction 12/04/20 11:01:18 ERROR metastore.ObjectStore: enter open transaction 12/04/20 11:01:18 ERROR metastore.ObjectStore: enter commit transaction 12/04/20 11:01:18 ERROR metastore.ObjectStore: enter rollback transaction 12/04/20 11:01:18 ERROR metastore.ObjectStore: rolling back 12/04/20 11:01:18 ERROR metastore.ObjectStore: enter rollback transaction FAILED: Error in metadata: MetaException(message:test) 12/04/20 11:01:18 ERROR exec.DDLTask: FAILED: Error in metadata: MetaException(message:test) org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:test) And then I use "describe test;", I got this: hive> describe test; 12/04/20 11:11:31 ERROR metastore.HiveMetaStore: execute with retry: 0 12/04/20 11:11:31 ERROR metastore.ObjectStore: enter open transaction 12/04/20 11:11:31 ERROR metastore.ObjectStore: opening 12/04/20 11:11:31 ERROR metastore.ObjectStore: enter open transaction 12/04/20 11:11:31 ERROR metastore.ObjectStore: enter commit transaction 12/04/20 11:11:31 ERROR metastore.ObjectStore: enter commit transaction 12/04/20 11:11:31 ERROR metastore.ObjectStore: commiting OK str string Time taken: 0.132 seconds It seems the table has been created successfully although the transaction has never been committed and has been rolled back. I think this could relate to JDO and MySQL. Because I cannot reproduce it when using Derby. My MySQL version is 5.1.18 Thanks. Kai Zhang 2012/4/20 Carl Steinbach > Hi Kai, > > I agree that this sounds like a bug. Can you describe the steps required > to reproduce this behavior on 0.7.1? > > Thanks. > > Carl > > > On Wed, Apr 18, 2012 at 11:03 PM, Zhang Kai wrote: > >> Hi all, >> >> I got a problem when using Hive 0.7.1. >> After I created a table with JDO failure, I can still find the table by >> using 'show tables;'. >> >> I grep the code and find there is a transaction mechanism when hive >> interacts with metastore. >> However, it seems that the transaction doesn't work. >> >> I have tested this for several times. >> I manually throw an exception before ObjectStore commit the 'CREATE >> TABLE' transaction. >> But I can find that a new record in TBLS in mysql. >> So I guess meta data didn't rollback correctly. >> >> I also notice this could also happen on Hive 0.8 and 0.9. >> However, I haven't tested it. >> >> I think this is a fatal error and can make hive metastore unavailable. >> Is there anyone can help me solve this problem? >> >> Thanks. >> >> Kai Zhang >> > >
[jira] [Updated] (HIVE-2933) analyze command throw NPE when table doesn't exists
[ https://issues.apache.org/jira/browse/HIVE-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ransom.hezhiqiang updated HIVE-2933: Attachment: (was: HIVE-2933-0.8.1-2.patch) > analyze command throw NPE when table doesn't exists > --- > > Key: HIVE-2933 > URL: https://issues.apache.org/jira/browse/HIVE-2933 > Project: Hive > Issue Type: Bug > Components: Statistics >Affects Versions: 0.8.1 >Reporter: alex gemini >Priority: Minor > > analyze command throw NPE when table doesn't exists -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2933) analyze command throw NPE when table doesn't exists
[ https://issues.apache.org/jira/browse/HIVE-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ransom.hezhiqiang updated HIVE-2933: Attachment: (was: HIVE-2933-0.8.1-1.patch) > analyze command throw NPE when table doesn't exists > --- > > Key: HIVE-2933 > URL: https://issues.apache.org/jira/browse/HIVE-2933 > Project: Hive > Issue Type: Bug > Components: Statistics >Affects Versions: 0.8.1 >Reporter: alex gemini >Priority: Minor > > analyze command throw NPE when table doesn't exists -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2933) analyze command throw NPE when table doesn't exists
[ https://issues.apache.org/jira/browse/HIVE-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ransom.hezhiqiang updated HIVE-2933: Attachment: HIVE-2933-0.8.1-2.patch > analyze command throw NPE when table doesn't exists > --- > > Key: HIVE-2933 > URL: https://issues.apache.org/jira/browse/HIVE-2933 > Project: Hive > Issue Type: Bug > Components: Statistics >Affects Versions: 0.8.1 >Reporter: alex gemini >Priority: Minor > Attachments: HIVE-2933-0.8.1-2.patch > > > analyze command throw NPE when table doesn't exists -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1384 - Failure
Changes for Build #1384 [hashutosh] HIVE-2958 [jira] GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger] (Navis Ryu via Ashutosh Chauhan) Summary: DPAL- GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger] This relates to https://issues.apache.org/jira/browse/HIVE-1634. The following work fine: CREATE EXTERNAL TABLE tim_hbase_occurrence ( id int, scientific_name string, data_resource_id int ) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ( "hbase.columns.mapping" = ":key#b,v:scientific_name#s,v:data_resource_id#b" ) TBLPROPERTIES( "hbase.table.name" = "mini_occurrences", "hbase.table.default.storage.type" = "binary" ); SELECT * FROM tim_hbase_occurrence LIMIT 3; SELECT * FROM tim_hbase_occurrence WHERE data_resource_id=1081 LIMIT 3; However, the following fails: SELECT data_resource_id, count(*) FROM tim_hbase_occurrence GROUP BY data_resource_id; The error given: 0 TS 2012-04-17 16:58:45,693 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Initialization Done 7 MAP 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Processing alias tim_hbase_occurrence for file hdfs://c1n2.gbif.org/user/hive/warehouse/tim_hbase_occurrence 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7 forwarding 1 rows 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows 2012-04-17 16:58:45,716 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 forwarding 1 rows 2012-04-17 16:58:45,723 FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"id":1444,"scientific_name":null,"data_resource_id":1081} at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548) at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:270) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157) at org.apache.hadoop.mapred.Child.main(Child.java:264) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to org.apache.hadoop.hive.serde2.lazy.LazyInteger at org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:737) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529) ... 9 more Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to org.apache.hadoop.hive.serde2.lazy.LazyInteger at org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyIntObjectInspector.copyObject(LazyIntObjectInspector.java:43) at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:239) at org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:150) at org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:142) at org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.copyKey(KeyWrapperFactory.java:119) at org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:750) at org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:722) ... 18 more Test Plan: EMPTY Reviewers: JIRA, ashutoshc Reviewed By: ashutoshc Differential Revision: https://reviews.facebook.net/D2871 1 tests failed. REGRESSION: org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See bu
[jira] [Commented] (HIVE-1634) Allow access to Primitive types stored in binary format in HBase
[ https://issues.apache.org/jira/browse/HIVE-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258036#comment-13258036 ] Hudson commented on HIVE-1634: -- Integrated in Hive-trunk-h0.21 #1384 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1384/]) HIVE-2958 [jira] GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger] (Navis Ryu via Ashutosh Chauhan) Summary: DPAL- GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger] This relates to https://issues.apache.org/jira/browse/HIVE-1634. The following work fine: CREATE EXTERNAL TABLE tim_hbase_occurrence ( id int, scientific_name string, data_resource_id int ) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ( "hbase.columns.mapping" = ":key#b,v:scientific_name#s,v:data_resource_id#b" ) TBLPROPERTIES( "hbase.table.name" = "mini_occurrences", "hbase.table.default.storage.type" = "binary" ); SELECT * FROM tim_hbase_occurrence LIMIT 3; SELECT * FROM tim_hbase_occurrence WHERE data_resource_id=1081 LIMIT 3; However, the following fails: SELECT data_resource_id, count(*) FROM tim_hbase_occurrence GROUP BY data_resource_id; The error given: 0 TS 2012-04-17 16:58:45,693 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Initialization Done 7 MAP 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Processing alias tim_hbase_occurrence for file hdfs://c1n2.gbif.org/user/hive/warehouse/tim_hbase_occurrence 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7 forwarding 1 rows 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows 2012-04-17 16:58:45,716 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 forwarding 1 rows 2012-04-17 16:58:45,723 FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"id":1444,"scientific_name":null,"data_resource_id":1081} at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548) at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:270) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157) at org.apache.hadoop.mapred.Child.main(Child.java:264) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to org.apache.hadoop.hive.serde2.lazy.LazyInteger at org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:737) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529) ... 9 more Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to org.apache.hadoop.hive.serde2.lazy.LazyInteger at org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyIntObjectInspector.copyObject(LazyIntObjectInspector.java:43) at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:239) at org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:150) at org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:142) at org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.copyKey(KeyWrapperFactory.java:119) at org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:750) at org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:722) ... 18 more Test Plan: EMPTY Reviewers: JIRA, ashutoshc Reviewed By: ashutoshc Differential Revision: https://reviews.facebook.net/D2871 (Revision 1328157)
[jira] [Commented] (HIVE-2958) GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger]
[ https://issues.apache.org/jira/browse/HIVE-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258037#comment-13258037 ] Hudson commented on HIVE-2958: -- Integrated in Hive-trunk-h0.21 #1384 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1384/]) HIVE-2958 [jira] GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger] (Navis Ryu via Ashutosh Chauhan) Summary: DPAL- GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger] This relates to https://issues.apache.org/jira/browse/HIVE-1634. The following work fine: CREATE EXTERNAL TABLE tim_hbase_occurrence ( id int, scientific_name string, data_resource_id int ) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ( "hbase.columns.mapping" = ":key#b,v:scientific_name#s,v:data_resource_id#b" ) TBLPROPERTIES( "hbase.table.name" = "mini_occurrences", "hbase.table.default.storage.type" = "binary" ); SELECT * FROM tim_hbase_occurrence LIMIT 3; SELECT * FROM tim_hbase_occurrence WHERE data_resource_id=1081 LIMIT 3; However, the following fails: SELECT data_resource_id, count(*) FROM tim_hbase_occurrence GROUP BY data_resource_id; The error given: 0 TS 2012-04-17 16:58:45,693 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Initialization Done 7 MAP 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Processing alias tim_hbase_occurrence for file hdfs://c1n2.gbif.org/user/hive/warehouse/tim_hbase_occurrence 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7 forwarding 1 rows 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows 2012-04-17 16:58:45,716 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 forwarding 1 rows 2012-04-17 16:58:45,723 FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"id":1444,"scientific_name":null,"data_resource_id":1081} at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548) at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:270) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157) at org.apache.hadoop.mapred.Child.main(Child.java:264) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to org.apache.hadoop.hive.serde2.lazy.LazyInteger at org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:737) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529) ... 9 more Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to org.apache.hadoop.hive.serde2.lazy.LazyInteger at org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyIntObjectInspector.copyObject(LazyIntObjectInspector.java:43) at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:239) at org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:150) at org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:142) at org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.copyKey(KeyWrapperFactory.java:119) at org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:750) at org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:722) ... 18 more Test Plan: EMPTY Reviewers: JIRA, ashutoshc Reviewed By: ashutoshc Differential Revision: https://reviews.facebook.net/D2871 (Revision 1328157)
POST limit
Ashutosh, are you submitting an exceptionally large patch of some kind? http://stackoverflow.com/questions/6279897/php-post-content-length-of-11933650-bytes-exceeds-the-limit-of-8388608-bytes We could try bumping up that limit on the server side, but first it would be good to find out whether that is really the problem (and if so what is contributing to such a big size). JVS On Thu, Apr 19, 2012 at 7:35 PM, Ashutosh Chauhan wrote: > Hit a new problem with arc today: > > Fatal error: Uncaught exception 'Exception' with message 'Host returned > HTTP/200, but invalid JSON data in response to a Conduit method call: > > Warning: Unknown: POST Content-Length of 9079953 bytes exceeds the > limit of 8388608 bytes in Unknown on line 0 > for(;;);{"result":null,"error_code":"ERR-INVALID-SESSION","error_info":"Session > key is not present."}' in > /Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitFuture.php:48 > Stack trace: > #0 > /Users/ashutosh/work/hive/libphutil/src/future/proxy/FutureProxy.php(62): > ConduitFuture->didReceiveResult(Array) > #1 > /Users/ashutosh/work/hive/libphutil/src/future/proxy/FutureProxy.php(39): > FutureProxy->getResult() > #2 > /Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitClient.php(52): > FutureProxy->resolve() > #3 > /Users/ashutosh/work/hive/arcanist/src/workflow/diff/ArcanistDiffWorkflow.php(341): > ConduitClient->callMethodSynchronous('differential.cr...', Array) > #4 /Users/ashutosh/work/hive/arcanist/scripts/arcanist.php(266): > ArcanistDiffWo in > /Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitFuture.php on > line 48 > > > Any ideas how to solve this? > > Thanks, > Ashutosh