[jira] Updated: (HIVE-1697) Migration scripts should increase size of PARAM_VALUE in PARTITION_PARAMS
[ https://issues.apache.org/jira/browse/HIVE-1697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1697: Status: Patch Available (was: Open) > Migration scripts should increase size of PARAM_VALUE in PARTITION_PARAMS > - > > Key: HIVE-1697 > URL: https://issues.apache.org/jira/browse/HIVE-1697 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.7.0 >Reporter: Paul Yang >Assignee: Paul Yang > Attachments: HIVE-1697.1.patch > > > The migration scripts should increase the size of column PARAM_VALUE in the > table PARTITION_PARAMS to 4000 chars to follow the description in package.jdo. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1697) Migration scripts should increase size of PARAM_VALUE in PARTITION_PARAMS
[ https://issues.apache.org/jira/browse/HIVE-1697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1697: Attachment: HIVE-1697.1.patch > Migration scripts should increase size of PARAM_VALUE in PARTITION_PARAMS > - > > Key: HIVE-1697 > URL: https://issues.apache.org/jira/browse/HIVE-1697 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.7.0 >Reporter: Paul Yang >Assignee: Paul Yang > Attachments: HIVE-1697.1.patch > > > The migration scripts should increase the size of column PARAM_VALUE in the > table PARTITION_PARAMS to 4000 chars to follow the description in package.jdo. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1697) Migration scripts should increase size of PARAM_VALUE in PARTITION_PARAMS
Migration scripts should increase size of PARAM_VALUE in PARTITION_PARAMS - Key: HIVE-1697 URL: https://issues.apache.org/jira/browse/HIVE-1697 Project: Hadoop Hive Issue Type: Bug Components: Metastore Affects Versions: 0.7.0 Reporter: Paul Yang Assignee: Paul Yang The migration scripts should increase the size of column PARAM_VALUE in the table PARTITION_PARAMS to 4000 chars to follow the description in package.jdo. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1427) Provide metastore schema migration scripts (0.5 -> 0.6)
[ https://issues.apache.org/jira/browse/HIVE-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12918657#action_12918657 ] Paul Yang commented on HIVE-1427: - Actually indexing wasn't present in 0.5 so that column doesn't need to be in the script. > Provide metastore schema migration scripts (0.5 -> 0.6) > --- > > Key: HIVE-1427 > URL: https://issues.apache.org/jira/browse/HIVE-1427 > Project: Hadoop Hive > Issue Type: Task > Components: Metastore >Reporter: Carl Steinbach >Assignee: Carl Steinbach > Fix For: 0.6.0 > > Attachments: HIVE-1427.1.patch.txt > > > At a minimum this ticket covers packaging up example MySQL migration scripts > (cumulative across all schema changes from 0.5 to 0.6) and explaining what to > do with them in the release notes. > This is also probably a good point at which to decide and clearly state which > Metastore DBs we officially support in production, e.g. do we need to provide > migration scripts for Derby? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1427) Provide metastore schema migration scripts (0.5 -> 0.6)
[ https://issues.apache.org/jira/browse/HIVE-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1427: Status: Open (was: Patch Available) > Provide metastore schema migration scripts (0.5 -> 0.6) > --- > > Key: HIVE-1427 > URL: https://issues.apache.org/jira/browse/HIVE-1427 > Project: Hadoop Hive > Issue Type: Task > Components: Metastore >Reporter: Carl Steinbach >Assignee: Carl Steinbach > Fix For: 0.6.0 > > Attachments: HIVE-1427.1.patch.txt > > > At a minimum this ticket covers packaging up example MySQL migration scripts > (cumulative across all schema changes from 0.5 to 0.6) and explaining what to > do with them in the release notes. > This is also probably a good point at which to decide and clearly state which > Metastore DBs we officially support in production, e.g. do we need to provide > migration scripts for Derby? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1427) Provide metastore schema migration scripts (0.5 -> 0.6)
[ https://issues.apache.org/jira/browse/HIVE-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12918642#action_12918642 ] Paul Yang commented on HIVE-1427: - >From looking at HIVE-1364 - it looked like some size changes were missing: Column INDEX_HANDLER_CLASS in INDXS Column PARAM_VALUE in PARTITION_PARAMS @Carl, can you verify those are missing from the upgrade script and add if necessary? > Provide metastore schema migration scripts (0.5 -> 0.6) > --- > > Key: HIVE-1427 > URL: https://issues.apache.org/jira/browse/HIVE-1427 > Project: Hadoop Hive > Issue Type: Task > Components: Metastore >Reporter: Carl Steinbach >Assignee: Carl Steinbach > Fix For: 0.6.0 > > Attachments: HIVE-1427.1.patch.txt > > > At a minimum this ticket covers packaging up example MySQL migration scripts > (cumulative across all schema changes from 0.5 to 0.6) and explaining what to > do with them in the release notes. > This is also probably a good point at which to decide and clearly state which > Metastore DBs we officially support in production, e.g. do we need to provide > migration scripts for Derby? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1658) Fix describe [extended] column formatting
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913315#action_12913315 ] Paul Yang commented on HIVE-1658: - Maybe another option is to have a pretty qualifier? To get your output, the user would have to type something like 'pretty describe '. Otherwise everything defaults to normal. > Fix describe [extended] column formatting > - > > Key: HIVE-1658 > URL: https://issues.apache.org/jira/browse/HIVE-1658 > Project: Hadoop Hive > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Paul Yang >Assignee: Thiruvel Thirumoolan > > When displaying the column schema, the formatting should follow should be > nametypecomment > to be inline with the previous formatting style for backward compatibility. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1609) Support partition filtering in metastore
[ https://issues.apache.org/jira/browse/HIVE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913239#action_12913239 ] Paul Yang commented on HIVE-1609: - @Yongqiang - The new thrift function seems benign - what problems are you running into? > Support partition filtering in metastore > > > Key: HIVE-1609 > URL: https://issues.apache.org/jira/browse/HIVE-1609 > Project: Hadoop Hive > Issue Type: New Feature > Components: Metastore >Reporter: Ajay Kidave >Assignee: Ajay Kidave > Fix For: 0.7.0 > > Attachments: hive_1609.patch, hive_1609_2.patch, hive_1609_3.patch > > > The metastore needs to have support for returning a list of partitions based > on user specified filter conditions. This will be useful for tools which need > to do partition pruning. Howl is one such use case. The way partition pruning > is done during hive query execution need not be changed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1609) Support partition filtering in metastore
[ https://issues.apache.org/jira/browse/HIVE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12912812#action_12912812 ] Paul Yang commented on HIVE-1609: - Looks good to me +1 @Ajay - can you create that JIRA and assign it to me? > Support partition filtering in metastore > > > Key: HIVE-1609 > URL: https://issues.apache.org/jira/browse/HIVE-1609 > Project: Hadoop Hive > Issue Type: New Feature > Components: Metastore >Reporter: Ajay Kidave >Assignee: Ajay Kidave > Fix For: 0.7.0 > > Attachments: hive_1609.patch, hive_1609_2.patch, hive_1609_3.patch > > > The metastore needs to have support for returning a list of partitions based > on user specified filter conditions. This will be useful for tools which need > to do partition pruning. Howl is one such use case. The way partition pruning > is done during hive query execution need not be changed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1658) Fix describe [extended] column formatting
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12912809#action_12912809 ] Paul Yang commented on HIVE-1658: - Actually, this JIRA is more about handling backward compatibility. It's possible that there things (JDBC? scripts?) that parsed the text output and depended on the old style. > Fix describe [extended] column formatting > - > > Key: HIVE-1658 > URL: https://issues.apache.org/jira/browse/HIVE-1658 > Project: Hadoop Hive > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Paul Yang >Assignee: Thiruvel Thirumoolan > > When displaying the column schema, the formatting should follow should be > nametypecomment > to be inline with the previous formatting style for backward compatibility. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1658) Fix describe [extended] column formatting
Fix describe [extended] column formatting - Key: HIVE-1658 URL: https://issues.apache.org/jira/browse/HIVE-1658 Project: Hadoop Hive Issue Type: Bug Affects Versions: 0.7.0 Reporter: Paul Yang Assignee: Thiruvel Thirumoolan When displaying the column schema, the formatting should follow should be nametypecomment to be inline with the previous formatting style for backward compatibility. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-558) describe extended table/partition output is cryptic
[ https://issues.apache.org/jira/browse/HIVE-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12912695#action_12912695 ] Paul Yang commented on HIVE-558: As an extensive user of describe extended, this helps a lot! Thanks. > describe extended table/partition output is cryptic > --- > > Key: HIVE-558 > URL: https://issues.apache.org/jira/browse/HIVE-558 > Project: Hadoop Hive > Issue Type: Improvement > Components: Metastore >Reporter: Prasad Chakka >Assignee: Thiruvel Thirumoolan > Fix For: 0.7.0 > > Attachments: HIVE-558.3.patch, HIVE-558.4.patch, HIVE-558.patch, > HIVE-558.patch, HIVE-558_PrelimPatch.patch, SampleOutputDescribe.txt > > > describe extended table prints out the Thrift metadata object directly. The > information from it is not easy to read or parse. Output should be easily > read and can be simple parsed to get table location etc by programs. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1609) Support partition filtering in metastore
[ https://issues.apache.org/jira/browse/HIVE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12912635#action_12912635 ] Paul Yang commented on HIVE-1609: - I'm taking another look - should be ready later today > Support partition filtering in metastore > > > Key: HIVE-1609 > URL: https://issues.apache.org/jira/browse/HIVE-1609 > Project: Hadoop Hive > Issue Type: New Feature > Components: Metastore >Reporter: Ajay Kidave >Assignee: Ajay Kidave > Fix For: 0.7.0 > > Attachments: hive_1609.patch, hive_1609_2.patch, hive_1609_3.patch > > > The metastore needs to have support for returning a list of partitions based > on user specified filter conditions. This will be useful for tools which need > to do partition pruning. Howl is one such use case. The way partition pruning > is done during hive query execution need not be changed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1617) ScriptOperator's AutoProgressor can lead to an infinite loop
[ https://issues.apache.org/jira/browse/HIVE-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12910839#action_12910839 ] Paul Yang commented on HIVE-1617: - I've tried creating a test for this but it seems like for local mode, the task timeouts don't work. Unless we change how hadoop's local mode works, we might have to go without a test case. > ScriptOperator's AutoProgressor can lead to an infinite loop > > > Key: HIVE-1617 > URL: https://issues.apache.org/jira/browse/HIVE-1617 > Project: Hadoop Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Paul Yang > Fix For: 0.7.0 > > Attachments: HIVE-1617.1.patch > > > In the default settings, the auto progressor can result in a infinite loop. > There should be another configurable parameter which stops the auto progress > if the script has not made an progress. > The default can be an hour or so - this way we will not get indefinitely stuck -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-558) describe extended table/partition output is cryptic
[ https://issues.apache.org/jira/browse/HIVE-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12910775#action_12910775 ] Paul Yang commented on HIVE-558: Looks good +1 > describe extended table/partition output is cryptic > --- > > Key: HIVE-558 > URL: https://issues.apache.org/jira/browse/HIVE-558 > Project: Hadoop Hive > Issue Type: Improvement > Components: Metastore >Reporter: Prasad Chakka >Assignee: Thiruvel Thirumoolan > Attachments: HIVE-558.3.patch, HIVE-558.patch, HIVE-558.patch, > HIVE-558_PrelimPatch.patch, SampleOutputDescribe.txt > > > describe extended table prints out the Thrift metadata object directly. The > information from it is not easy to read or parse. Output should be easily > read and can be simple parsed to get table location etc by programs. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1617) ScriptOperator's AutoProgressor can lead to an infinite loop
[ https://issues.apache.org/jira/browse/HIVE-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1617: Attachment: HIVE-1617.1.patch This patch can be tested with the following commands: set hive.auto.progress.timeout=1200; set hive.exec.script.allow.partial.consumption=true; set hive.exec.mode.local.auto=false; set hive.script.auto.progress=true; select transform(*) using 'sleep 3600' from src; The logs should show reporter messages at regular intervals, up to the 20 minute mark. Then the task should time out. > ScriptOperator's AutoProgressor can lead to an infinite loop > > > Key: HIVE-1617 > URL: https://issues.apache.org/jira/browse/HIVE-1617 > Project: Hadoop Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Paul Yang > Fix For: 0.7.0 > > Attachments: HIVE-1617.1.patch > > > In the default settings, the auto progressor can result in a infinite loop. > There should be another configurable parameter which stops the auto progress > if the script has not made an progress. > The default can be an hour or so - this way we will not get indefinitely stuck -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1617) ScriptOperator's AutoProgressor can lead to an infinite loop
[ https://issues.apache.org/jira/browse/HIVE-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1617: Status: Patch Available (was: Open) > ScriptOperator's AutoProgressor can lead to an infinite loop > > > Key: HIVE-1617 > URL: https://issues.apache.org/jira/browse/HIVE-1617 > Project: Hadoop Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Paul Yang > Fix For: 0.7.0 > > Attachments: HIVE-1617.1.patch > > > In the default settings, the auto progressor can result in a infinite loop. > There should be another configurable parameter which stops the auto progress > if the script has not made an progress. > The default can be an hour or so - this way we will not get indefinitely stuck -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1609) Support partition filtering in metastore
[ https://issues.apache.org/jira/browse/HIVE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12909930#action_12909930 ] Paul Yang commented on HIVE-1609: - A couple of suggestions: 1. Update get_partitions_ps() to use get_partitions_by_filter(). By doing this, we should see some speedup when using the Hive CLI for tables with large number of partitions. (May require #2 for full benefits) 2. could we add a listPartitionNamesByFilter() as well to RawStore/ObjectStore? This would allow partition filtering for get_partition_names_ps(). This could be done in a separate JIRA, if necessary. 3. Add test cases to TestHiveMetastore for the new method(s). I'll take another look once the antlr stuff is done, but looks good. > Support partition filtering in metastore > > > Key: HIVE-1609 > URL: https://issues.apache.org/jira/browse/HIVE-1609 > Project: Hadoop Hive > Issue Type: New Feature > Components: Metastore >Reporter: Ajay Kidave >Assignee: Ajay Kidave > Fix For: 0.7.0 > > Attachments: hive_1609.patch, hive_1609_2.patch > > > The metastore needs to have support for returning a list of partitions based > on user specified filter conditions. This will be useful for tools which need > to do partition pruning. Howl is one such use case. The way partition pruning > is done during hive query execution need not be changed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-675) add database/schema support Hive QL
[ https://issues.apache.org/jira/browse/HIVE-675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907044#action_12907044 ] Paul Yang commented on HIVE-675: @Carl, do you happen to have a MySQL upgrade script for adding the new tables / columns required to support this feature? > add database/schema support Hive QL > --- > > Key: HIVE-675 > URL: https://issues.apache.org/jira/browse/HIVE-675 > Project: Hadoop Hive > Issue Type: New Feature > Components: Metastore, Query Processor >Reporter: Prasad Chakka >Assignee: Carl Steinbach > Fix For: 0.6.0, 0.7.0 > > Attachments: hive-675-2009-9-16.patch, hive-675-2009-9-19.patch, > hive-675-2009-9-21.patch, hive-675-2009-9-23.patch, hive-675-2009-9-7.patch, > hive-675-2009-9-8.patch, HIVE-675-2010-08-16.patch.txt, > HIVE-675-2010-7-16.patch.txt, HIVE-675-2010-8-4.patch.txt, > HIVE-675.10.patch.txt, HIVE-675.11.patch.txt, HIVE-675.12.patch.txt, > HIVE-675.13.patch.txt > > > Currently all Hive tables reside in single namespace (default). Hive should > support multiple namespaces (databases or schemas) such that users can create > tables in their specific namespaces. These name spaces can have different > warehouse directories (with a default naming scheme) and possibly different > properties. > There is already some support for this in metastore but Hive query parser > should have this feature as well. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HIVE-1617) ScriptOperator's AutoProgressor can lead to an infinite loop
[ https://issues.apache.org/jira/browse/HIVE-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang reassigned HIVE-1617: --- Assignee: Paul Yang > ScriptOperator's AutoProgressor can lead to an infinite loop > > > Key: HIVE-1617 > URL: https://issues.apache.org/jira/browse/HIVE-1617 > Project: Hadoop Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Paul Yang > Fix For: 0.7.0 > > > In the default settings, the auto progressor can result in a infinite loop. > There should be another configurable parameter which stops the auto progress > if the script has not made an progress. > The default can be an hour or so - this way we will not get indefinitely stuck -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1618) Expression in sort by throws NPE
Expression in sort by throws NPE Key: HIVE-1618 URL: https://issues.apache.org/jira/browse/HIVE-1618 Project: Hadoop Hive Issue Type: Bug Affects Versions: 0.7.0 Reporter: Paul Yang A query such as: SELECT count(1) FROM src SORT BY count(1); Will throw a null pointer exception: 2010-09-07 15:27:27,059 ERROR ql.Driver (SessionState.java:printError(277)) - FAILED: Hive Internal Error: java.lang.NullPointerException(null) java.lang.NullPointerException at org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:153) at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:587) at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:708) at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:89) at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:88) at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:128) at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:102) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:6418) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genReduceSinkPlan(SemanticAnalyzer.java:4029) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:5148) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:5661) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:6229) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:240) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:326) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-558) describe extended table/partition output is cryptic
[ https://issues.apache.org/jira/browse/HIVE-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906015#action_12906015 ] Paul Yang commented on HIVE-558: Also, this change will break a lot of unit tests - have you had a chance to run the full suite? In many unit tests, we do a describe extended that prints a single line with all the attributes. The entire line was usually ignored as the location attribute had a 'file:/'. With your change, we will now start comparing all the attributes, some of which which may vary from run to run, e.g. table create time. Those attributes will need to be added to the ignore list in QTestUtil.java:checkCliDriverResults() > describe extended table/partition output is cryptic > --- > > Key: HIVE-558 > URL: https://issues.apache.org/jira/browse/HIVE-558 > Project: Hadoop Hive > Issue Type: Improvement > Components: Metastore >Reporter: Prasad Chakka >Assignee: Thiruvel Thirumoolan > Attachments: HIVE-558.patch, HIVE-558.patch, > HIVE-558_PrelimPatch.patch, SampleOutputDescribe.txt > > > describe extended table prints out the Thrift metadata object directly. The > information from it is not easy to read or parse. Output should be easily > read and can be simple parsed to get table location etc by programs. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-558) describe extended table/partition output is cryptic
[ https://issues.apache.org/jira/browse/HIVE-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904371#action_12904371 ] Paul Yang commented on HIVE-558: Looks pretty good but in DDLTask, the updated code does not make use of field List cols and instead just relies on the table object. Since cols could have been formed via Hive.getFieldsFromDeserializer() (DDLTask.java 1553) your change might give different results. Also in formatOutput(), why is the field width 15 in one instance but 20 for the rest? Also, could you move the width to a static constant? > describe extended table/partition output is cryptic > --- > > Key: HIVE-558 > URL: https://issues.apache.org/jira/browse/HIVE-558 > Project: Hadoop Hive > Issue Type: Improvement > Components: Metastore >Reporter: Prasad Chakka >Assignee: Thiruvel Thirumoolan > Attachments: HIVE-558.patch, HIVE-558_PrelimPatch.patch, > SampleOutputDescribe.txt > > > describe extended table prints out the Thrift metadata object directly. The > information from it is not easy to read or parse. Output should be easily > read and can be simple parsed to get table location etc by programs. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1593) udtf_explode.q is an empty file
[ https://issues.apache.org/jira/browse/HIVE-1593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902269#action_12902269 ] Paul Yang commented on HIVE-1593: - Odd, this file had tests in it before... https://issues.apache.org/jira/secure/attachment/12427672/HIVE-935.8.patch > udtf_explode.q is an empty file > --- > > Key: HIVE-1593 > URL: https://issues.apache.org/jira/browse/HIVE-1593 > Project: Hadoop Hive > Issue Type: Bug > Components: Testing Infrastructure >Affects Versions: 0.6.0 >Reporter: John Sichi >Assignee: Paul Yang >Priority: Minor > Fix For: 0.7.0 > > > jsichi-mac:clientpositive jsichi$ pwd > /Users/jsichi/open/hive-trunk/ql/src/test/queries/clientpositive > jsichi-mac:clientpositive jsichi$ cat udtf_explode.q > jsichi-mac:clientpositive jsichi$ -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1578) Add conf. property hive.exec.show.job.failure.debug.info to enable/disable displaying link to the task with most failures
[ https://issues.apache.org/jira/browse/HIVE-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12901207#action_12901207 ] Paul Yang commented on HIVE-1578: - @Carl The message to the user about the conf var is a good idea. I can put info level logging statements, but I don't think it's possible to know the number of task completion events before retrieving them, so there won't be a % complete message. > Add conf. property hive.exec.show.job.failure.debug.info to enable/disable > displaying link to the task with most failures > - > > Key: HIVE-1578 > URL: https://issues.apache.org/jira/browse/HIVE-1578 > Project: Hadoop Hive > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Paul Yang >Assignee: Paul Yang > Fix For: 0.7.0 > > Attachments: HIVE-1578.1.patch > > > If a job fails, Hive currently displays a link to the task with the most > number of failures for easy access to the error logs. However, generating the > link may require many RPC's to get all the task completion events, adding a > delay of up to 30 minutes. This patch adds a configuration variable to > control whether the link is generated. Turning off this feature would also > disable automatic debugging tips generated by heuristics reading from the > error logs. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1578) Add conf. variable for displaying link to the task with most failures
[ https://issues.apache.org/jira/browse/HIVE-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1578: Description: If a job fails, Hive currently displays a link to the task with the most number of failures for easy access to the error logs. However, generating the link may require many RPC's to get all the task completion events, adding a delay of up to 30 minutes. This patch adds a configuration variable to control whether the link is generated. Turning off this feature would also disable automatic debugging tips generated by heuristics reading from the error logs. (was: If a job fails, Hive currently displays a link to the task with the most number of failures, for easy access to the error logs. However, generating the link may require many RPC calls to get all the task completion events, adding a delay of up to 30 minutes. This patch adds a configuration variable to control whether the link is generated. Turning off this feature would also disable automatic debugging tips generated by heuristics reading from the error logs.) > Add conf. variable for displaying link to the task with most failures > - > > Key: HIVE-1578 > URL: https://issues.apache.org/jira/browse/HIVE-1578 > Project: Hadoop Hive > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Paul Yang >Assignee: Paul Yang > Fix For: 0.7.0 > > Attachments: HIVE-1578.1.patch > > > If a job fails, Hive currently displays a link to the task with the most > number of failures for easy access to the error logs. However, generating the > link may require many RPC's to get all the task completion events, adding a > delay of up to 30 minutes. This patch adds a configuration variable to > control whether the link is generated. Turning off this feature would also > disable automatic debugging tips generated by heuristics reading from the > error logs. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1578) Add conf. variable for displaying link to the task with most failures
[ https://issues.apache.org/jira/browse/HIVE-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1578: Status: Patch Available (was: Open) > Add conf. variable for displaying link to the task with most failures > - > > Key: HIVE-1578 > URL: https://issues.apache.org/jira/browse/HIVE-1578 > Project: Hadoop Hive > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Paul Yang >Assignee: Paul Yang > Fix For: 0.7.0 > > Attachments: HIVE-1578.1.patch > > > If a job fails, Hive currently displays a link to the task with the most > number of failures for easy access to the error logs. However, generating the > link may require many RPC's to get all the task completion events, adding a > delay of up to 30 minutes. This patch adds a configuration variable to > control whether the link is generated. Turning off this feature would also > disable automatic debugging tips generated by heuristics reading from the > error logs. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1578) Add conf. variable for displaying link to the task with most failures
[ https://issues.apache.org/jira/browse/HIVE-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1578: Attachment: HIVE-1578.1.patch > Add conf. variable for displaying link to the task with most failures > - > > Key: HIVE-1578 > URL: https://issues.apache.org/jira/browse/HIVE-1578 > Project: Hadoop Hive > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Paul Yang >Assignee: Paul Yang > Fix For: 0.7.0 > > Attachments: HIVE-1578.1.patch > > > If a job fails, Hive currently displays a link to the task with the most > number of failures, for easy access to the error logs. However, generating > the link may require many RPC calls to get all the task completion events, > adding a delay of up to 30 minutes. This patch adds a configuration variable > to control whether the link is generated. Turning off this feature would also > disable automatic debugging tips generated by heuristics reading from the > error logs. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1578) Add conf. variable for displaying link to the task with most failures
Add conf. variable for displaying link to the task with most failures - Key: HIVE-1578 URL: https://issues.apache.org/jira/browse/HIVE-1578 Project: Hadoop Hive Issue Type: Bug Affects Versions: 0.7.0 Reporter: Paul Yang Assignee: Paul Yang Fix For: 0.7.0 If a job fails, Hive currently displays a link to the task with the most number of failures, for easy access to the error logs. However, generating the link may require many RPC calls to get all the task completion events, adding a delay of up to 30 minutes. This patch adds a configuration variable to control whether the link is generated. Turning off this feature would also disable automatic debugging tips generated by heuristics reading from the error logs. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1547) Unarchiving operation throws NPE
[ https://issues.apache.org/jira/browse/HIVE-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1547: Status: Patch Available (was: Open) > Unarchiving operation throws NPE > > > Key: HIVE-1547 > URL: https://issues.apache.org/jira/browse/HIVE-1547 > Project: Hadoop Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.7.0 >Reporter: Paul Yang >Assignee: Paul Yang > Fix For: 0.7.0 > > Attachments: HIVE-1547.1.patch > > > Unarchiving a partition throws a null pointer exception similar to the > following: > 2010-08-16 12:44:18,801 ERROR exec.DDLTask > (SessionState.java:printError(277)) - Failed with exception null > java.lang.NullPointerException > at org.apache.hadoop.hive.ql.exec.DDLTask.unarchive(DDLTask.java:729) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:195) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:108) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:609) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:478) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:356) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:140) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:199) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:351) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > This error seems to be DFS specific, as local file system in the unit tests > don't catch this. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1547) Unarchiving operation throws NPE
[ https://issues.apache.org/jira/browse/HIVE-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1547: Attachment: HIVE-1547.1.patch > Unarchiving operation throws NPE > > > Key: HIVE-1547 > URL: https://issues.apache.org/jira/browse/HIVE-1547 > Project: Hadoop Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.7.0 >Reporter: Paul Yang >Assignee: Paul Yang > Fix For: 0.7.0 > > Attachments: HIVE-1547.1.patch > > > Unarchiving a partition throws a null pointer exception similar to the > following: > 2010-08-16 12:44:18,801 ERROR exec.DDLTask > (SessionState.java:printError(277)) - Failed with exception null > java.lang.NullPointerException > at org.apache.hadoop.hive.ql.exec.DDLTask.unarchive(DDLTask.java:729) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:195) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:108) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:609) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:478) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:356) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:140) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:199) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:351) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > This error seems to be DFS specific, as local file system in the unit tests > don't catch this. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1547) Unarchiving operation throws NPE
Unarchiving operation throws NPE Key: HIVE-1547 URL: https://issues.apache.org/jira/browse/HIVE-1547 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.0 Reporter: Paul Yang Assignee: Paul Yang Fix For: 0.7.0 Unarchiving a partition throws a null pointer exception similar to the following: 2010-08-16 12:44:18,801 ERROR exec.DDLTask (SessionState.java:printError(277)) - Failed with exception null java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.DDLTask.unarchive(DDLTask.java:729) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:195) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:108) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:609) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:478) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:356) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:140) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:199) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:351) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) This error seems to be DFS specific, as local file system in the unit tests don't catch this. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1539) Concurrent metastore threading problem
[ https://issues.apache.org/jira/browse/HIVE-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899157#action_12899157 ] Paul Yang commented on HIVE-1539: - >From looking at the Hive class, we are creating a ThreadLocal instance of Hive >when we call get(). Since the HiveMetaStoreClient is a member of Hive, >shouldn't the client be accessed by only 1 thread at a time? > Concurrent metastore threading problem > --- > > Key: HIVE-1539 > URL: https://issues.apache.org/jira/browse/HIVE-1539 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.7.0 >Reporter: Bennie Schut >Assignee: Bennie Schut > Attachments: thread_dump_hanging.txt > > > When running hive as a service and running a high number of queries > concurrently I end up with multiple threads running at 100% cpu without any > progress. > Looking at these threads I notice this thread(484e): > at > org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:598) > But on a different thread(63a2): > at > org.apache.hadoop.hive.metastore.model.MStorageDescriptor.jdoReplaceField(MStorageDescriptor.java) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1528) JSON UDTF function
[
https://issues.apache.org/jira/browse/HIVE-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898034#action_12898034
]
Paul Yang commented on HIVE-1528:
-
Looks good +1
> JSON UDTF function
> --
>
> Key: HIVE-1528
> URL: https://issues.apache.org/jira/browse/HIVE-1528
> Project: Hadoop Hive
> Issue Type: New Feature
>Affects Versions: 0.7.0
>Reporter: Ning Zhang
>Assignee: Ning Zhang
> Fix For: 0.7.0
>
> Attachments: HIVE-1528.2.patch, HIVE-1528.patch
>
>
> Currently the only way to evaluate a path expression on a JSON object is
> through get_json_object. If there are many fields in the JSON object need to
> be extract, we have to call this UDF multiple times.
> There are many use cases that get_json_object needs to be called many times
> in one query to convert the JSON object to a relational schema. It would be
> much desirable if we have a JSON UDTF that supports the following syntax:
> {code}
> select a.id, b.*
> from a lateral view json_table(a.json_object, '$.f1', '$.f2', ..., '$.fn') b
> as f1, f2, ..., fn
> {code}
> where the json_table function only scans the json_object once and return a
> set of tuple of (f1, f2,..., fn).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1515) archive is not working when multiple partitions inside one table are archived.
[ https://issues.apache.org/jira/browse/HIVE-1515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1515: Status: Open (was: Patch Available) See comments on reviewboard. > archive is not working when multiple partitions inside one table are archived. > -- > > Key: HIVE-1515 > URL: https://issues.apache.org/jira/browse/HIVE-1515 > Project: Hadoop Hive > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: He Yongqiang >Assignee: He Yongqiang > Attachments: hive-1515.1.patch > > > set hive.exec.compress.output = true; > set hive.input.format=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat; > set mapred.min.split.size=256; > set mapred.min.split.size.per.node=256; > set mapred.min.split.size.per.rack=256; > set mapred.max.split.size=256; > set hive.archive.enabled = true; > drop table combine_3_srcpart_seq_rc; > create table combine_3_srcpart_seq_rc (key int , value string) partitioned by > (ds string, hr string) stored as sequencefile; > insert overwrite table combine_3_srcpart_seq_rc partition (ds="2010-08-03", > hr="00") select * from src; > insert overwrite table combine_3_srcpart_seq_rc partition (ds="2010-08-03", > hr="001") select * from src; > ALTER TABLE combine_3_srcpart_seq_rc ARCHIVE PARTITION (ds="2010-08-03", > hr="00"); > ALTER TABLE combine_3_srcpart_seq_rc ARCHIVE PARTITION (ds="2010-08-03", > hr="001"); > select key, value, ds, hr from combine_3_srcpart_seq_rc where ds="2010-08-03" > order by key, hr limit 30; > drop table combine_3_srcpart_seq_rc; > will fail. > java.io.IOException: Invalid file name: > har:/data/users/heyongqiang/hive-trunk-clean/build/ql/test/data/warehouse/combine_3_srcpart_seq_rc/ds=2010-08-03/hr=001/data.har/data/users/heyongqiang/hive-trunk-clean/build/ql/test/data/warehouse/combine_3_srcpart_seq_rc/ds=2010-08-03/hr=001 > in > har:/data/users/heyongqiang/hive-trunk-clean/build/ql/test/data/warehouse/combine_3_srcpart_seq_rc/ds=2010-08-03/hr=00/data.har > The reason it fails is because: > there are 2 input paths (one for each partition) for the above query: > 1): > har:/Users/heyongqiang/Documents/workspace/Hive-Index/build/ql/test/data/warehouse/combine_3_srcpart_seq_rc/ds=2010-08-03/hr=00/data.har/Users/heyongqiang/Documents/workspace/Hive-Index/build/ql/test/data/warehouse/combine_3_srcpart_seq_rc/ds=2010-08-03/hr=00 > 2): > har:/Users/heyongqiang/Documents/workspace/Hive-Index/build/ql/test/data/warehouse/combine_3_srcpart_seq_rc/ds=2010-08-03/hr=001/data.har/Users/heyongqiang/Documents/workspace/Hive-Index/build/ql/test/data/warehouse/combine_3_srcpart_seq_rc/ds=2010-08-03/hr=001 > But when doing path.getFileSystem() for these 2 input paths. they both return > same one file system instance which points the first caller, in this case > which is > har:/Users/heyongqiang/Documents/workspace/Hive-Index/build/ql/test/data/warehouse/combine_3_srcpart_seq_rc/ds=2010-08-03/hr=00/data.har > The reason here is Hadoop's FileSystem has a global cache, and when trying to > load a FileSystem instance from a given path, it only take the path's scheme > and username to lookup the cache. So when we do Path.getFileSystem for the > second har path, it actually returns the file system handle for the first > path. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1489) TestCliDriver -Doverwrite=true does not put the file in the correct directory
[ https://issues.apache.org/jira/browse/HIVE-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12892603#action_12892603 ] Paul Yang commented on HIVE-1489: - Works as advertised. +1 > TestCliDriver -Doverwrite=true does not put the file in the correct directory > - > > Key: HIVE-1489 > URL: https://issues.apache.org/jira/browse/HIVE-1489 > Project: Hadoop Hive > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.7.0 > > Attachments: HIVE-1489.patch > > > When adding a new file in clientpositive with -Doverwrite=true, the output > file was in ql/ rather than ql/src/test/results/clientpositive. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1411) DataNucleus throws NucleusException if core-3.1.1 JAR appears more than once on CLASSPATH
[
https://issues.apache.org/jira/browse/HIVE-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12891392#action_12891392
]
Paul Yang commented on HIVE-1411:
-
About #4, I was able to reproduce the exception and verify the fix by creating
a copy of the datanucleus core jar in hive's lib directory.
Makes sense to me. Now that you mention it, I think I ran into a similar
problem while using the 'hadoop jar' command. It would unpack the jar, but
include both the jar and the unpacked classes in the classpath. If the jar
contained datanucleus stuff, this would throw an exception.
Anyway, the fix looks good. +1
> DataNucleus throws NucleusException if core-3.1.1 JAR appears more than once
> on CLASSPATH
> -
>
> Key: HIVE-1411
> URL: https://issues.apache.org/jira/browse/HIVE-1411
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 0.4.0, 0.4.1, 0.5.0
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Fix For: 0.6.0, 0.7.0
>
> Attachments: HIVE-1411.patch.txt
>
>
> DataNucleus barfs when the core-3.1.1 JAR file appears more than once on the
> CLASSPATH:
> {code}
> 2010-03-06 12:33:25,565 ERROR exec.DDLTask
> (SessionState.java:printError(279)) - FAILED: Error in metadata:
> javax.jdo.JDOFatalInter
> nalException: Unexpected exception caught.
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> org.apache.hadoop.hive.ql.metadata.HiveException:
> javax.jdo.JDOFatalInternalException: Unexpected exception caught.
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:258)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:879)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:103)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:379)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:285)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: javax.jdo.JDOFatalInternalException: Unexpected exception caught.
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> at
> javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1186)
> at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:803)
> at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:698)
> at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:164)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:181)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:125)
> at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:104)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:130)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:146)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:118)
>
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.(HiveMetaStore.java:100)
>
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:74)
>
> at
> org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:783)
> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:794)
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:252)
> ... 12 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at javax.jdo.JDOHelper$16.run(JDOHelper.java:1956)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.jdo.JDOHelper.invoke(JDOHelper.java:1951)
> at
> javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnI
[jira] Commented: (HIVE-1411) DataNucleus barfs if JAR appears more than once in CLASSPATH
[
https://issues.apache.org/jira/browse/HIVE-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12891307#action_12891307
]
Paul Yang commented on HIVE-1411:
-
@Carl - what is the motivation for turning off this exception? Wouldn't it be
good to know if 2 jars were on the CP so that the user would know for sure
which version was in use?
> DataNucleus barfs if JAR appears more than once in CLASSPATH
>
>
> Key: HIVE-1411
> URL: https://issues.apache.org/jira/browse/HIVE-1411
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 0.4.0, 0.4.1, 0.5.0
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Fix For: 0.6.0, 0.7.0
>
> Attachments: HIVE-1411.patch.txt
>
>
> DataNucleus barfs when more than one JAR with the same name appears on the
> CLASSPATH:
> {code}
> 2010-03-06 12:33:25,565 ERROR exec.DDLTask
> (SessionState.java:printError(279)) - FAILED: Error in metadata:
> javax.jdo.JDOFatalInter
> nalException: Unexpected exception caught.
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> org.apache.hadoop.hive.ql.metadata.HiveException:
> javax.jdo.JDOFatalInternalException: Unexpected exception caught.
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:258)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:879)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:103)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:379)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:285)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: javax.jdo.JDOFatalInternalException: Unexpected exception caught.
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> at
> javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1186)
> at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:803)
> at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:698)
> at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:164)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:181)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:125)
> at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:104)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:130)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:146)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:118)
>
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.(HiveMetaStore.java:100)
>
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:74)
>
> at
> org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:783)
> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:794)
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:252)
> ... 12 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at javax.jdo.JDOHelper$16.run(JDOHelper.java:1956)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.jdo.JDOHelper.invoke(JDOHelper.java:1951)
> at
> javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1159)
> ... 28 more
> Caused by: org.datanucleus.exceptions.NucleusException: Plugin (Bundle)
> "org.eclipse.jdt.core" is already registered. Ensure you do
> nt have multiple JAR versions of the same plugin in the classpath. The URL
> "file:/Users/hadop/hadoop-0.20.1+152/build/ivy/lib/Hadoo
> p/common/cor
[jira] Commented: (HIVE-1411) DataNucleus barfs if JAR appears more than once in CLASSPATH
[
https://issues.apache.org/jira/browse/HIVE-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12891301#action_12891301
]
Paul Yang commented on HIVE-1411:
-
Whoops, got side tracked by another datanucleus issue. I'll review it today.
> DataNucleus barfs if JAR appears more than once in CLASSPATH
>
>
> Key: HIVE-1411
> URL: https://issues.apache.org/jira/browse/HIVE-1411
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 0.4.0, 0.4.1, 0.5.0
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Fix For: 0.6.0, 0.7.0
>
> Attachments: HIVE-1411.patch.txt
>
>
> DataNucleus barfs when more than one JAR with the same name appears on the
> CLASSPATH:
> {code}
> 2010-03-06 12:33:25,565 ERROR exec.DDLTask
> (SessionState.java:printError(279)) - FAILED: Error in metadata:
> javax.jdo.JDOFatalInter
> nalException: Unexpected exception caught.
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> org.apache.hadoop.hive.ql.metadata.HiveException:
> javax.jdo.JDOFatalInternalException: Unexpected exception caught.
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:258)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:879)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:103)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:379)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:285)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: javax.jdo.JDOFatalInternalException: Unexpected exception caught.
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> at
> javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1186)
> at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:803)
> at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:698)
> at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:164)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:181)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:125)
> at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:104)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:130)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:146)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:118)
>
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.(HiveMetaStore.java:100)
>
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:74)
>
> at
> org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:783)
> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:794)
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:252)
> ... 12 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at javax.jdo.JDOHelper$16.run(JDOHelper.java:1956)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.jdo.JDOHelper.invoke(JDOHelper.java:1951)
> at
> javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1159)
> ... 28 more
> Caused by: org.datanucleus.exceptions.NucleusException: Plugin (Bundle)
> "org.eclipse.jdt.core" is already registered. Ensure you do
> nt have multiple JAR versions of the same plugin in the classpath. The URL
> "file:/Users/hadop/hadoop-0.20.1+152/build/ivy/lib/Hadoo
> p/common/core-3.1.1.jar" is already registered, and you are trying to
> register an identical plugin located at URL
[jira] Created: (HIVE-1478) Non-boolean expression in WHERE clause throws exception
Non-boolean expression in WHERE clause throws exception
---
Key: HIVE-1478
URL: https://issues.apache.org/jira/browse/HIVE-1478
Project: Hadoop Hive
Issue Type: Bug
Affects Versions: 0.7.0
Reporter: Paul Yang
Priority: Minor
If the expression in the where clause does not evaluate to a boolean, the job
will fail with the following exception in the task logs:
Query:
SELECT key FROM src WHERE 1;
Exception in mapper:
2010-07-21 17:00:31,460 FATAL ExecMapper:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {"key":"238","value":"val_238"}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:417)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:180)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:159)
Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to
java.lang.Boolean
at
org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:45)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:400)
... 5 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1455) lateral view does not work with column pruning
[ https://issues.apache.org/jira/browse/HIVE-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1455: Attachment: HIVE-1455.4.patch > lateral view does not work with column pruning > --- > > Key: HIVE-1455 > URL: https://issues.apache.org/jira/browse/HIVE-1455 > Project: Hadoop Hive > Issue Type: Bug >Reporter: He Yongqiang >Assignee: Paul Yang > Fix For: 0.6.0, 0.7.0 > > Attachments: HIVE-1455.3.patch, HIVE-1455.4.patch, hive.1455.1.patch, > hive.1455.2.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1455) lateral view does not work with column pruning
[ https://issues.apache.org/jira/browse/HIVE-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1455: Attachment: (was: HIVE-1455.4.patch) > lateral view does not work with column pruning > --- > > Key: HIVE-1455 > URL: https://issues.apache.org/jira/browse/HIVE-1455 > Project: Hadoop Hive > Issue Type: Bug >Reporter: He Yongqiang >Assignee: Paul Yang > Fix For: 0.6.0, 0.7.0 > > Attachments: HIVE-1455.3.patch, HIVE-1455.4.patch, hive.1455.1.patch, > hive.1455.2.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1455) lateral view does not work with column pruning
[ https://issues.apache.org/jira/browse/HIVE-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1455: Attachment: HIVE-1455.4.patch Added an additional test case where we do a lateral view on a subquery. > lateral view does not work with column pruning > --- > > Key: HIVE-1455 > URL: https://issues.apache.org/jira/browse/HIVE-1455 > Project: Hadoop Hive > Issue Type: Bug >Reporter: He Yongqiang >Assignee: Paul Yang > Fix For: 0.6.0, 0.7.0 > > Attachments: HIVE-1455.3.patch, HIVE-1455.4.patch, hive.1455.1.patch, > hive.1455.2.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1455) lateral view does not work with column pruning
[ https://issues.apache.org/jira/browse/HIVE-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1455: Attachment: (was: HIVE-1455.3.patch) > lateral view does not work with column pruning > --- > > Key: HIVE-1455 > URL: https://issues.apache.org/jira/browse/HIVE-1455 > Project: Hadoop Hive > Issue Type: Bug >Reporter: He Yongqiang >Assignee: Paul Yang > Fix For: 0.6.0, 0.7.0 > > Attachments: HIVE-1455.3.patch, hive.1455.1.patch, hive.1455.2.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1455) lateral view does not work with column pruning
[ https://issues.apache.org/jira/browse/HIVE-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1455: Attachment: HIVE-1455.3.patch > lateral view does not work with column pruning > --- > > Key: HIVE-1455 > URL: https://issues.apache.org/jira/browse/HIVE-1455 > Project: Hadoop Hive > Issue Type: Bug >Reporter: He Yongqiang >Assignee: Paul Yang > Fix For: 0.6.0, 0.7.0 > > Attachments: HIVE-1455.3.patch, hive.1455.1.patch, hive.1455.2.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HIVE-1455) lateral view does not work with column pruning
[ https://issues.apache.org/jira/browse/HIVE-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang reassigned HIVE-1455: --- Assignee: Paul Yang (was: He Yongqiang) > lateral view does not work with column pruning > --- > > Key: HIVE-1455 > URL: https://issues.apache.org/jira/browse/HIVE-1455 > Project: Hadoop Hive > Issue Type: Bug >Reporter: He Yongqiang >Assignee: Paul Yang > Fix For: 0.6.0, 0.7.0 > > Attachments: HIVE-1455.3.patch, hive.1455.1.patch, hive.1455.2.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1455) lateral view does not work with column pruning
[ https://issues.apache.org/jira/browse/HIVE-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1455: Attachment: HIVE-1455.3.patch This patch modifies the column pruner to create a special case for the lateral view join operator. This will result in more pruned columns. > lateral view does not work with column pruning > --- > > Key: HIVE-1455 > URL: https://issues.apache.org/jira/browse/HIVE-1455 > Project: Hadoop Hive > Issue Type: Bug >Reporter: He Yongqiang >Assignee: He Yongqiang > Fix For: 0.6.0, 0.7.0 > > Attachments: HIVE-1455.3.patch, hive.1455.1.patch, hive.1455.2.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1056) Predicate push down does not work with UDTF's
[
https://issues.apache.org/jira/browse/HIVE-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Yang updated HIVE-1056:
Attachment: HIVE-1056.4.patch
Added comment for LVF PPD rule.
> Predicate push down does not work with UDTF's
> -
>
> Key: HIVE-1056
> URL: https://issues.apache.org/jira/browse/HIVE-1056
> Project: Hadoop Hive
> Issue Type: Bug
>Affects Versions: 0.5.0, 0.6.0
>Reporter: Paul Yang
>Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1056.1.patch, HIVE-1056.2.patch, HIVE-1056.3.patch,
> HIVE-1056.4.patch
>
>
> Predicate push down does not work with UDTF's in lateral views
> {code}
> hive> SELECT * FROM src LATERAL VIEW explode(array(1,2,3)) myTable AS k WHERE
> k=1;
> FAILED: Unknown exception: null
> hive>
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1056) Predicate push down does not work with UDTF's
[
https://issues.apache.org/jira/browse/HIVE-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Yang updated HIVE-1056:
Attachment: HIVE-1056.3.patch
Regenerated again.
> Predicate push down does not work with UDTF's
> -
>
> Key: HIVE-1056
> URL: https://issues.apache.org/jira/browse/HIVE-1056
> Project: Hadoop Hive
> Issue Type: Bug
>Affects Versions: 0.5.0, 0.6.0
>Reporter: Paul Yang
>Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1056.1.patch, HIVE-1056.2.patch, HIVE-1056.3.patch
>
>
> Predicate push down does not work with UDTF's in lateral views
> {code}
> hive> SELECT * FROM src LATERAL VIEW explode(array(1,2,3)) myTable AS k WHERE
> k=1;
> FAILED: Unknown exception: null
> hive>
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1461) Clean up references to 'hive.metastore.local'
Clean up references to 'hive.metastore.local' - Key: HIVE-1461 URL: https://issues.apache.org/jira/browse/HIVE-1461 Project: Hadoop Hive Issue Type: Bug Affects Versions: 0.7.0 Reporter: Paul Yang Priority: Minor 'hive.metastore.local' should not be referred directly as a string. Instead, a HiveConf.ConfVar entry should be created and used. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1428) ALTER TABLE ADD PARTITION fails with a remote Thirft metastore
[ https://issues.apache.org/jira/browse/HIVE-1428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12888011#action_12888011 ] Paul Yang commented on HIVE-1428: - "hive.metastore.local" should be made into a HiveConf, but that can be done in a separate JIRA since it's probably used elsewhere as well. Looks good +1. > ALTER TABLE ADD PARTITION fails with a remote Thirft metastore > -- > > Key: HIVE-1428 > URL: https://issues.apache.org/jira/browse/HIVE-1428 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.6.0, 0.7.0 >Reporter: Paul Yang >Assignee: Pradeep Kamath > Attachments: HIVE-1428-2.patch, HIVE-1428-3.patch, HIVE-1428.patch, > TestHiveMetaStoreRemote.java > > > If the hive cli is configured to use a remote metastore, ALTER TABLE ... ADD > PARTITION commands will fail with an error similar to the following: > [prade...@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER TABLE > mytable add partition(datestamp = '20091101', srcid = '10',action) location > '/user/pradeepk/mytable/20091101/10';" > 10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found > in the classpath. Usage of hadoop-site.xml is deprecated. Instead use > core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of > core-default.xml, mapred-default.xml and hdfs-default.xml respectively > Hive history > file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt > FAILED: Error in metadata: org.apache.thrift.TApplicationException: > get_partition failed: unknown result > FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask > [prade...@chargesize:~/dev/howl] > This is due to a check that tries to retrieve the partition to see if it > exists. If it does not, an attempt is made to pass a null value from the > metastore. Since thrift does not support null return values, an exception is > thrown. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1056) Predicate push down does not work with UDTF's
[
https://issues.apache.org/jira/browse/HIVE-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Yang updated HIVE-1056:
Attachment: HIVE-1056.2.patch
Regenerated patch.
> Predicate push down does not work with UDTF's
> -
>
> Key: HIVE-1056
> URL: https://issues.apache.org/jira/browse/HIVE-1056
> Project: Hadoop Hive
> Issue Type: Bug
>Affects Versions: 0.5.0, 0.6.0
>Reporter: Paul Yang
>Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1056.1.patch, HIVE-1056.2.patch
>
>
> Predicate push down does not work with UDTF's in lateral views
> {code}
> hive> SELECT * FROM src LATERAL VIEW explode(array(1,2,3)) myTable AS k WHERE
> k=1;
> FAILED: Unknown exception: null
> hive>
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HIVE-1458) Table aliases are case sensitive.
[ https://issues.apache.org/jira/browse/HIVE-1458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang reassigned HIVE-1458: --- Assignee: Paul Yang > Table aliases are case sensitive. > - > > Key: HIVE-1458 > URL: https://issues.apache.org/jira/browse/HIVE-1458 > Project: Hadoop Hive > Issue Type: Bug >Reporter: Jonathan Chang >Assignee: Paul Yang >Priority: Minor > > This query: > SELECT exploded, COUNT(1) > > FROM ( > > SELECT SPLIT(value, "0") AS value > > FROM tmp_jonchang_hive_test > > ) B LATERAL VIEW explode(value) C AS exploded > > GROUP BY exploded > produces the error: > FAILED: Error in semantic analysis: line 7:9 Invalid Table Alias or Column > Reference exploded > However, when B is changed to lowercase, the query works as expected: > SELECT exploded, COUNT(1) > > FROM ( > > SELECT SPLIT(value, "0") AS value > > FROM tmp_jonchang_hive_test > > ) b LATERAL VIEW explode(value) C AS exploded > > GROUP BY exploded > Table aliases shouldn't be case sensitive AFAIK. And even if they are, I > can't see the cause for the error. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1428) ALTER TABLE ADD PARTITION fails with a remote Thirft metastore
[ https://issues.apache.org/jira/browse/HIVE-1428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1428: Status: Open (was: Patch Available) > ALTER TABLE ADD PARTITION fails with a remote Thirft metastore > -- > > Key: HIVE-1428 > URL: https://issues.apache.org/jira/browse/HIVE-1428 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.6.0, 0.7.0 >Reporter: Paul Yang >Assignee: Pradeep Kamath > Attachments: HIVE-1428-2.patch, HIVE-1428.patch, > TestHiveMetaStoreRemote.java > > > If the hive cli is configured to use a remote metastore, ALTER TABLE ... ADD > PARTITION commands will fail with an error similar to the following: > [prade...@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER TABLE > mytable add partition(datestamp = '20091101', srcid = '10',action) location > '/user/pradeepk/mytable/20091101/10';" > 10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found > in the classpath. Usage of hadoop-site.xml is deprecated. Instead use > core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of > core-default.xml, mapred-default.xml and hdfs-default.xml respectively > Hive history > file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt > FAILED: Error in metadata: org.apache.thrift.TApplicationException: > get_partition failed: unknown result > FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask > [prade...@chargesize:~/dev/howl] > This is due to a check that tries to retrieve the partition to see if it > exists. If it does not, an attempt is made to pass a null value from the > metastore. Since thrift does not support null return values, an exception is > thrown. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1428) ALTER TABLE ADD PARTITION fails with a remote Thirft metastore
[ https://issues.apache.org/jira/browse/HIVE-1428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886559#action_12886559 ] Paul Yang commented on HIVE-1428: - Fix looks pretty good - two things though: 1. The field identifiers for NoSuchObjectException and MetaException in hive_metastore.thrift should be swapped - this is because thrift uses those identifiers for versioning and we want to be consistent. 2. TestHiveMetaStoreRemote and TestHiveMetaStore share quite a bit of code. Can you extract this out to a separate class? Or maybe roll the remote metastore client into TestHiveMetastore. > ALTER TABLE ADD PARTITION fails with a remote Thirft metastore > -- > > Key: HIVE-1428 > URL: https://issues.apache.org/jira/browse/HIVE-1428 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.6.0, 0.7.0 >Reporter: Paul Yang >Assignee: Pradeep Kamath > Attachments: HIVE-1428-2.patch, HIVE-1428.patch, > TestHiveMetaStoreRemote.java > > > If the hive cli is configured to use a remote metastore, ALTER TABLE ... ADD > PARTITION commands will fail with an error similar to the following: > [prade...@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER TABLE > mytable add partition(datestamp = '20091101', srcid = '10',action) location > '/user/pradeepk/mytable/20091101/10';" > 10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found > in the classpath. Usage of hadoop-site.xml is deprecated. Instead use > core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of > core-default.xml, mapred-default.xml and hdfs-default.xml respectively > Hive history > file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt > FAILED: Error in metadata: org.apache.thrift.TApplicationException: > get_partition failed: unknown result > FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask > [prade...@chargesize:~/dev/howl] > This is due to a check that tries to retrieve the partition to see if it > exists. If it does not, an attempt is made to pass a null value from the > metastore. Since thrift does not support null return values, an exception is > thrown. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1428) ALTER TABLE ADD PARTITION fails with a remote Thirft metastore
[ https://issues.apache.org/jira/browse/HIVE-1428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12884437#action_12884437 ] Paul Yang commented on HIVE-1428: - Meant to add - By using a thread, you can avoid having to set up an environment, classpath, etc. An important line is "HiveConf hiveConf = new HiveConf(this.getClass());", which allows you to load the configuration files that are in the classpath defined for testing. And the command to run the test should be 'ant test -Dtestcase=TestHiveMetastoreRemote' > ALTER TABLE ADD PARTITION fails with a remote Thirft metastore > -- > > Key: HIVE-1428 > URL: https://issues.apache.org/jira/browse/HIVE-1428 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.6.0, 0.7.0 >Reporter: Paul Yang > Attachments: HIVE-1428.patch, TestHiveMetaStoreRemote.java > > > If the hive cli is configured to use a remote metastore, ALTER TABLE ... ADD > PARTITION commands will fail with an error similar to the following: > [prade...@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER TABLE > mytable add partition(datestamp = '20091101', srcid = '10',action) location > '/user/pradeepk/mytable/20091101/10';" > 10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found > in the classpath. Usage of hadoop-site.xml is deprecated. Instead use > core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of > core-default.xml, mapred-default.xml and hdfs-default.xml respectively > Hive history > file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt > FAILED: Error in metadata: org.apache.thrift.TApplicationException: > get_partition failed: unknown result > FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask > [prade...@chargesize:~/dev/howl] > This is due to a check that tries to retrieve the partition to see if it > exists. If it does not, an attempt is made to pass a null value from the > metastore. Since thrift does not support null return values, an exception is > thrown. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1428) ALTER TABLE ADD PARTITION fails with a remote Thirft metastore
[
https://issues.apache.org/jira/browse/HIVE-1428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12884383#action_12884383
]
Paul Yang commented on HIVE-1428:
-
Can you try using a thread to create a HiveMetaStore instance?
e.g.
{code}
private static class RunMS implements Runnable {
@Override
public void run() {
System.out.println("Running metastore!");
String [] args = new String [0];
HiveMetaStore.main(args);
}
}
@Override
protected void setUp() throws Exception {
super.setUp();
Thread t = new Thread(new RunMS());
t.start();
// Wait a little bit for the metastore to start. Should probably have
// a better way of detecting if the metastore has started?
Thread.sleep(5000);
// Set conf to connect to the local metastore.
HiveConf hiveConf = new HiveConf(this.getClass());
// hive.metastore.local should be defined in HiveConf
hiveConf.set("hive.metastore.local", "false");
hiveConf.setVar(HiveConf.ConfVars.METASTOREURIS, "thrift://localhost:9083");
// There's a bug in HiveConf that needs to be fixed before this next line
// will work
// Change METATORETHRIFTRETRIES("hive.metastore.connect.retries", ""),
// To METATORETHRIFTRETRIES("hive.metastore.connect.retries", 3),
hiveConf.setIntVar(HiveConf.ConfVars.METATORETHRIFTRETRIES, 3);
client = new HiveMetaStoreClient(hiveConf);
// Now you have the client - run necessary tests.
}
{code}
> ALTER TABLE ADD PARTITION fails with a remote Thirft metastore
> --
>
> Key: HIVE-1428
> URL: https://issues.apache.org/jira/browse/HIVE-1428
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 0.6.0, 0.7.0
>Reporter: Paul Yang
> Attachments: HIVE-1428.patch, TestHiveMetaStoreRemote.java
>
>
> If the hive cli is configured to use a remote metastore, ALTER TABLE ... ADD
> PARTITION commands will fail with an error similar to the following:
> [prade...@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER TABLE
> mytable add partition(datestamp = '20091101', srcid = '10',action) location
> '/user/pradeepk/mytable/20091101/10';"
> 10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found
> in the classpath. Usage of hadoop-site.xml is deprecated. Instead use
> core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of
> core-default.xml, mapred-default.xml and hdfs-default.xml respectively
> Hive history
> file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt
> FAILED: Error in metadata: org.apache.thrift.TApplicationException:
> get_partition failed: unknown result
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask
> [prade...@chargesize:~/dev/howl]
> This is due to a check that tries to retrieve the partition to see if it
> exists. If it does not, an attempt is made to pass a null value from the
> metastore. Since thrift does not support null return values, an exception is
> thrown.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1445) Missing command type definitions for some alter table commands
[
https://issues.apache.org/jira/browse/HIVE-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Yang updated HIVE-1445:
Description:
Some alter table command don't have their command type defined in
SemanticAnalyzerFactory, leading to entries like
{code}
POSTHOOK: type: null
{code}
in test outputs. Examples are ALTER TABLE...CLUSTER BY and ALTER TABLE... SET
FILEFORMAT
was:
Some alter table command don't have their command type defined in
SemanticAnalyzerFactory, leading to entries like
{code}
POSTHOOK: type: null
{code}
in test outputs.
> Missing command type definitions for some alter table commands
> --
>
> Key: HIVE-1445
> URL: https://issues.apache.org/jira/browse/HIVE-1445
> Project: Hadoop Hive
> Issue Type: Bug
>Reporter: Paul Yang
>Assignee: Paul Yang
> Fix For: 0.7.0
>
>
> Some alter table command don't have their command type defined in
> SemanticAnalyzerFactory, leading to entries like
> {code}
> POSTHOOK: type: null
> {code}
> in test outputs. Examples are ALTER TABLE...CLUSTER BY and ALTER TABLE... SET
> FILEFORMAT
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1445) Missing command type definitions for some alter table commands
Missing command type definitions for some alter table commands
--
Key: HIVE-1445
URL: https://issues.apache.org/jira/browse/HIVE-1445
Project: Hadoop Hive
Issue Type: Bug
Reporter: Paul Yang
Assignee: Paul Yang
Fix For: 0.7.0
Some alter table command don't have their command type defined in
SemanticAnalyzerFactory, leading to entries like
{code}
POSTHOOK: type: null
{code}
in test outputs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1443) Add support to turn off bucketing with ALTER TABLE
[ https://issues.apache.org/jira/browse/HIVE-1443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1443: Status: Patch Available (was: Open) > Add support to turn off bucketing with ALTER TABLE > -- > > Key: HIVE-1443 > URL: https://issues.apache.org/jira/browse/HIVE-1443 > Project: Hadoop Hive > Issue Type: Improvement >Reporter: Paul Yang >Assignee: Paul Yang > Fix For: 0.7.0 > > Attachments: HIVE-1443.1.patch > > > Currently, there is an alter table command that can change the bucketing / > sort columns, as well as the number of buckets (ALTER TABLE table_name > CLUSTERED BY (col_name, col_name, ...) [SORTED BY (col_name, ...)] INTO > num_buckets BUCKETS). However, this command does not provide a means to > disable bucketing. This proposes to introduce a syntax like > ALTER TABLE src NOT CLUSTERED; > that would turn off bucketing. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1443) Add support to turn off bucketing with ALTER TABLE
[ https://issues.apache.org/jira/browse/HIVE-1443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1443: Attachment: HIVE-1443.1.patch > Add support to turn off bucketing with ALTER TABLE > -- > > Key: HIVE-1443 > URL: https://issues.apache.org/jira/browse/HIVE-1443 > Project: Hadoop Hive > Issue Type: Improvement >Reporter: Paul Yang >Assignee: Paul Yang > Fix For: 0.7.0 > > Attachments: HIVE-1443.1.patch > > > Currently, there is an alter table command that can change the bucketing / > sort columns, as well as the number of buckets (ALTER TABLE table_name > CLUSTERED BY (col_name, col_name, ...) [SORTED BY (col_name, ...)] INTO > num_buckets BUCKETS). However, this command does not provide a means to > disable bucketing. This proposes to introduce a syntax like > ALTER TABLE src NOT CLUSTERED; > that would turn off bucketing. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1443) Add support to turn off bucketing with ALTER TABLE
Add support to turn off bucketing with ALTER TABLE -- Key: HIVE-1443 URL: https://issues.apache.org/jira/browse/HIVE-1443 Project: Hadoop Hive Issue Type: Improvement Reporter: Paul Yang Assignee: Paul Yang Fix For: 0.7.0 Currently, there is an alter table command that can change the bucketing / sort columns, as well as the number of buckets (ALTER TABLE table_name CLUSTERED BY (col_name, col_name, ...) [SORTED BY (col_name, ...)] INTO num_buckets BUCKETS). However, this command does not provide a means to disable bucketing. This proposes to introduce a syntax like ALTER TABLE src NOT CLUSTERED; that would turn off bucketing. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-1439) Alter the number of buckets for a table
[
https://issues.apache.org/jira/browse/HIVE-1439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Yang resolved HIVE-1439.
-
Fix Version/s: 0.6.0
Resolution: Not A Problem
> Alter the number of buckets for a table
> ---
>
> Key: HIVE-1439
> URL: https://issues.apache.org/jira/browse/HIVE-1439
> Project: Hadoop Hive
> Issue Type: New Feature
>Affects Versions: 0.6.0, 0.7.0
>Reporter: Paul Yang
>Assignee: Paul Yang
> Fix For: 0.6.0
>
>
> Add an alter table command to change the number of buckets for a table.
> e.g.
> {code}
> ALTER TABLE mytabl SET NUMBUCKETS 64;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1439) Alter the number of buckets for a table
[
https://issues.apache.org/jira/browse/HIVE-1439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12883732#action_12883732
]
Paul Yang commented on HIVE-1439:
-
This is actually supported with ALTER TABLE table_name CLUSTERED BY (col_name,
col_name, ...) [SORTED BY (col_name, ...)] INTO num_buckets BUCKETS. However
that command doesn't support turning off bucketing. Will file a JIRA for that.
> Alter the number of buckets for a table
> ---
>
> Key: HIVE-1439
> URL: https://issues.apache.org/jira/browse/HIVE-1439
> Project: Hadoop Hive
> Issue Type: New Feature
>Affects Versions: 0.6.0, 0.7.0
>Reporter: Paul Yang
>Assignee: Paul Yang
> Fix For: 0.6.0
>
>
> Add an alter table command to change the number of buckets for a table.
> e.g.
> {code}
> ALTER TABLE mytabl SET NUMBUCKETS 64;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1428) ALTER TABLE ADD PARTITION fails with a remote Thirft metastore
[
https://issues.apache.org/jira/browse/HIVE-1428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882718#action_12882718
]
Paul Yang commented on HIVE-1428:
-
The unit tests that we have for the metastore run locally, so I don't think
there's a way to use the existing framework for the thrift command - take a
look at TestHiveMetastore.java. To test this, you would have to create a new
JUnit test (or modify the existing one) to sets up a metastore server and
issues commands (create table, drop table, etc) to it. Does that seem doable?
> ALTER TABLE ADD PARTITION fails with a remote Thirft metastore
> --
>
> Key: HIVE-1428
> URL: https://issues.apache.org/jira/browse/HIVE-1428
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 0.6.0, 0.7.0
>Reporter: Paul Yang
> Attachments: HIVE-1428.patch
>
>
> If the hive cli is configured to use a remote metastore, ALTER TABLE ... ADD
> PARTITION commands will fail with an error similar to the following:
> {code}
> [prade...@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER TABLE
> mytable add partition(datestamp = '20091101', srcid = '10',action) location
> '/user/pradeepk/mytable/20091101/10';"
> 10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found
> in the classpath. Usage of hadoop-site.xml is deprecated. Instead use
> core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of
> core-default.xml, mapred-default.xml and hdfs-default.xml respectively
> Hive history
> file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt
> FAILED: Error in metadata: org.apache.thrift.TApplicationException:
> get_partition failed: unknown result
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask
> [prade...@chargesize:~/dev/howl]
> {code}
> This is due to a check that tries to retrieve the partition to see if it
> exists. If it does not, an attempt is made to pass a null value from the
> metastore. Since thrift does not support null return values, an exception is
> thrown.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1428) ALTER TABLE ADD PARTITION fails with a remote Thirft metastore
[
https://issues.apache.org/jira/browse/HIVE-1428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Yang updated HIVE-1428:
Description:
If the hive cli is configured to use a remote metastore, ALTER TABLE ... ADD
PARTITION commands will fail with an error similar to the following:
[prade...@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER TABLE
mytable add partition(datestamp = '20091101', srcid = '10',action) location
'/user/pradeepk/mytable/20091101/10';"
10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in
the classpath. Usage of hadoop-site.xml is deprecated. Instead use
core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of
core-default.xml, mapred-default.xml and hdfs-default.xml respectively
Hive history
file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt
FAILED: Error in metadata: org.apache.thrift.TApplicationException:
get_partition failed: unknown result
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask
[prade...@chargesize:~/dev/howl]
This is due to a check that tries to retrieve the partition to see if it
exists. If it does not, an attempt is made to pass a null value from the
metastore. Since thrift does not support null return values, an exception is
thrown.
was:
If the hive cli is configured to use a remote metastore, ALTER TABLE ... ADD
PARTITION commands will fail with an error similar to the following:
{code}
[prade...@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER TABLE
mytable add partition(datestamp = '20091101', srcid = '10',action) location
'/user/pradeepk/mytable/20091101/10';"
10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in
the classpath. Usage of hadoop-site.xml is deprecated. Instead use
core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of
core-default.xml, mapred-default.xml and hdfs-default.xml respectively
Hive history
file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt
FAILED: Error in metadata: org.apache.thrift.TApplicationException:
get_partition failed: unknown result
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask
[prade...@chargesize:~/dev/howl]
{code}
This is due to a check that tries to retrieve the partition to see if it
exists. If it does not, an attempt is made to pass a null value from the
metastore. Since thrift does not support null return values, an exception is
thrown.
> ALTER TABLE ADD PARTITION fails with a remote Thirft metastore
> --
>
> Key: HIVE-1428
> URL: https://issues.apache.org/jira/browse/HIVE-1428
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 0.6.0, 0.7.0
>Reporter: Paul Yang
> Attachments: HIVE-1428.patch
>
>
> If the hive cli is configured to use a remote metastore, ALTER TABLE ... ADD
> PARTITION commands will fail with an error similar to the following:
> [prade...@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER TABLE
> mytable add partition(datestamp = '20091101', srcid = '10',action) location
> '/user/pradeepk/mytable/20091101/10';"
> 10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found
> in the classpath. Usage of hadoop-site.xml is deprecated. Instead use
> core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of
> core-default.xml, mapred-default.xml and hdfs-default.xml respectively
> Hive history
> file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt
> FAILED: Error in metadata: org.apache.thrift.TApplicationException:
> get_partition failed: unknown result
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask
> [prade...@chargesize:~/dev/howl]
> This is due to a check that tries to retrieve the partition to see if it
> exists. If it does not, an attempt is made to pass a null value from the
> metastore. Since thrift does not support null return values, an exception is
> thrown.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1439) Alter the number of buckets for a table
Alter the number of buckets for a table
---
Key: HIVE-1439
URL: https://issues.apache.org/jira/browse/HIVE-1439
Project: Hadoop Hive
Issue Type: New Feature
Affects Versions: 0.6.0, 0.7.0
Reporter: Paul Yang
Assignee: Paul Yang
Add an alter table command to change the number of buckets for a table.
e.g.
{code}
ALTER TABLE mytabl SET NUMBUCKETS 64;
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1435) Upgraded naming scheme causes JDO exceptions
[ https://issues.apache.org/jira/browse/HIVE-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1435: Status: Patch Available (was: Open) > Upgraded naming scheme causes JDO exceptions > > > Key: HIVE-1435 > URL: https://issues.apache.org/jira/browse/HIVE-1435 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.6.0, 0.7.0 >Reporter: Paul Yang >Assignee: Paul Yang > Attachments: HIVE-1435.1.patch > > > We recently upgraded from Datanucleus 1.0 to 2.0, which changed some of the > defaults for how field names get mapped to datastore identifiers. Because of > this change, connecting to an existing database would throw exceptions such > as: > 2010-06-24 17:59:09,854 ERROR exec.DDLTask > (SessionState.java:printError(277)) - FAILED: Error in metadata: > javax.jdo.JDODataStoreException: Insert of object > "org.apache.hadoop.hive.metastore.model.mstoragedescrip...@4ccd21c" using > statement "INSERT INTO `SDS` > (`SD_ID`,`NUM_BUCKETS`,`INPUT_FORMAT`,`OUTPUT_FORMAT`,`LOCATION`,`SERDE_ID`,`ISCOMPRESSED`) > VALUES (?,?,?,?,?,?,?)" failed : Unknown column 'ISCOMPRESSED' in 'field > list' > NestedThrowables: > com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column > 'ISCOMPRESSED' in 'field list' > org.apache.hadoop.hive.ql.metadata.HiveException: > javax.jdo.JDODataStoreException: Insert of object > "org.apache.hadoop.hive.metastore.model.mstoragedescrip...@4ccd21c" using > statement "INSERT INTO `SDS` > (`SD_ID`,`NUM_BUCKETS`,`INPUT_FORMAT`,`OUTPUT_FORMAT`,`LOCATION`,`SERDE_ID`,`ISCOMPRESSED`) > VALUES (?,?,?,?,?,?,?)" failed : Unknown column 'ISCOMPRESSED' in 'field > list' > NestedThrowables: > com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column > 'ISCOMPRESSED' in 'field list' > at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:325) > at > org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:2012) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:144) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:633) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:506) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:384) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:302) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1435) Upgraded naming scheme causes JDO exceptions
[
https://issues.apache.org/jira/browse/HIVE-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882433#action_12882433
]
Paul Yang commented on HIVE-1435:
-
This patch was tested with a metastore database created prior to the
datanucleus upgrade. In our configuration, we have
{code}
datanucleus.autoCreateSchema
false
datanucleus.fixedDatastore
true
{code}
so that no automatic schema changes will occur. The following commands were run
without errors: create table, add partition, create view, create view, drop
table, and drop view.
> Upgraded naming scheme causes JDO exceptions
>
>
> Key: HIVE-1435
> URL: https://issues.apache.org/jira/browse/HIVE-1435
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 0.6.0, 0.7.0
>Reporter: Paul Yang
>Assignee: Paul Yang
> Attachments: HIVE-1435.1.patch
>
>
> We recently upgraded from Datanucleus 1.0 to 2.0, which changed some of the
> defaults for how field names get mapped to datastore identifiers. Because of
> this change, connecting to an existing database would throw exceptions such
> as:
> 2010-06-24 17:59:09,854 ERROR exec.DDLTask
> (SessionState.java:printError(277)) - FAILED: Error in metadata:
> javax.jdo.JDODataStoreException: Insert of object
> "org.apache.hadoop.hive.metastore.model.mstoragedescrip...@4ccd21c" using
> statement "INSERT INTO `SDS`
> (`SD_ID`,`NUM_BUCKETS`,`INPUT_FORMAT`,`OUTPUT_FORMAT`,`LOCATION`,`SERDE_ID`,`ISCOMPRESSED`)
> VALUES (?,?,?,?,?,?,?)" failed : Unknown column 'ISCOMPRESSED' in 'field
> list'
> NestedThrowables:
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column
> 'ISCOMPRESSED' in 'field list'
> org.apache.hadoop.hive.ql.metadata.HiveException:
> javax.jdo.JDODataStoreException: Insert of object
> "org.apache.hadoop.hive.metastore.model.mstoragedescrip...@4ccd21c" using
> statement "INSERT INTO `SDS`
> (`SD_ID`,`NUM_BUCKETS`,`INPUT_FORMAT`,`OUTPUT_FORMAT`,`LOCATION`,`SERDE_ID`,`ISCOMPRESSED`)
> VALUES (?,?,?,?,?,?,?)" failed : Unknown column 'ISCOMPRESSED' in 'field
> list'
> NestedThrowables:
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column
> 'ISCOMPRESSED' in 'field list'
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:325)
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:2012)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:144)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:633)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:506)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:384)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:302)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1435) Upgraded naming scheme causes JDO exceptions
[ https://issues.apache.org/jira/browse/HIVE-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1435: Attachment: HIVE-1435.1.patch This reverts the identifier factory to the same as datanucleus 1.1. > Upgraded naming scheme causes JDO exceptions > > > Key: HIVE-1435 > URL: https://issues.apache.org/jira/browse/HIVE-1435 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.6.0, 0.7.0 >Reporter: Paul Yang >Assignee: Paul Yang > Attachments: HIVE-1435.1.patch > > > We recently upgraded from Datanucleus 1.0 to 2.0, which changed some of the > defaults for how field names get mapped to datastore identifiers. Because of > this change, connecting to an existing database would throw exceptions such > as: > 2010-06-24 17:59:09,854 ERROR exec.DDLTask > (SessionState.java:printError(277)) - FAILED: Error in metadata: > javax.jdo.JDODataStoreException: Insert of object > "org.apache.hadoop.hive.metastore.model.mstoragedescrip...@4ccd21c" using > statement "INSERT INTO `SDS` > (`SD_ID`,`NUM_BUCKETS`,`INPUT_FORMAT`,`OUTPUT_FORMAT`,`LOCATION`,`SERDE_ID`,`ISCOMPRESSED`) > VALUES (?,?,?,?,?,?,?)" failed : Unknown column 'ISCOMPRESSED' in 'field > list' > NestedThrowables: > com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column > 'ISCOMPRESSED' in 'field list' > org.apache.hadoop.hive.ql.metadata.HiveException: > javax.jdo.JDODataStoreException: Insert of object > "org.apache.hadoop.hive.metastore.model.mstoragedescrip...@4ccd21c" using > statement "INSERT INTO `SDS` > (`SD_ID`,`NUM_BUCKETS`,`INPUT_FORMAT`,`OUTPUT_FORMAT`,`LOCATION`,`SERDE_ID`,`ISCOMPRESSED`) > VALUES (?,?,?,?,?,?,?)" failed : Unknown column 'ISCOMPRESSED' in 'field > list' > NestedThrowables: > com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column > 'ISCOMPRESSED' in 'field list' > at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:325) > at > org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:2012) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:144) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:633) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:506) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:384) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:302) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1435) Upgraded naming scheme causes JDO exceptions
Upgraded naming scheme causes JDO exceptions
Key: HIVE-1435
URL: https://issues.apache.org/jira/browse/HIVE-1435
Project: Hadoop Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.6.0, 0.7.0
Reporter: Paul Yang
Assignee: Paul Yang
We recently upgraded from Datanucleus 1.0 to 2.0, which changed some of the
defaults for how field names get mapped to datastore identifiers. Because of
this change, connecting to an existing database would throw exceptions such as:
{code}
2010-06-24 17:59:09,854 ERROR exec.DDLTask (SessionState.java:printError(277))
- FAILED: Error in metadata: javax.jdo.JDODataStoreException: Insert of object
"org.apache.hadoop.hive.metastore.model.mstoragedescrip...@4ccd21c" using
statement "INSERT INTO `SDS`
(`SD_ID`,`NUM_BUCKETS`,`INPUT_FORMAT`,`OUTPUT_FORMAT`,`LOCATION`,`SERDE_ID`,`ISCOMPRESSED`)
VALUES (?,?,?,?,?,?,?)" failed : Unknown column 'ISCOMPRESSED' in 'field list'
NestedThrowables:
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column
'ISCOMPRESSED' in 'field list'
org.apache.hadoop.hive.ql.metadata.HiveException:
javax.jdo.JDODataStoreException: Insert of object
"org.apache.hadoop.hive.metastore.model.mstoragedescrip...@4ccd21c" using
statement "INSERT INTO `SDS`
(`SD_ID`,`NUM_BUCKETS`,`INPUT_FORMAT`,`OUTPUT_FORMAT`,`LOCATION`,`SERDE_ID`,`ISCOMPRESSED`)
VALUES (?,?,?,?,?,?,?)" failed : Unknown column 'ISCOMPRESSED' in 'field list'
NestedThrowables:
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column
'ISCOMPRESSED' in 'field list'
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:325)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:2012)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:144)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:633)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:506)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:384)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:302)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1435) Upgraded naming scheme causes JDO exceptions
[
https://issues.apache.org/jira/browse/HIVE-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Yang updated HIVE-1435:
Description:
We recently upgraded from Datanucleus 1.0 to 2.0, which changed some of the
defaults for how field names get mapped to datastore identifiers. Because of
this change, connecting to an existing database would throw exceptions such as:
2010-06-24 17:59:09,854 ERROR exec.DDLTask (SessionState.java:printError(277))
- FAILED: Error in metadata: javax.jdo.JDODataStoreException: Insert of object
"org.apache.hadoop.hive.metastore.model.mstoragedescrip...@4ccd21c" using
statement "INSERT INTO `SDS`
(`SD_ID`,`NUM_BUCKETS`,`INPUT_FORMAT`,`OUTPUT_FORMAT`,`LOCATION`,`SERDE_ID`,`ISCOMPRESSED`)
VALUES (?,?,?,?,?,?,?)" failed : Unknown column 'ISCOMPRESSED' in 'field list'
NestedThrowables:
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column
'ISCOMPRESSED' in 'field list'
org.apache.hadoop.hive.ql.metadata.HiveException:
javax.jdo.JDODataStoreException: Insert of object
"org.apache.hadoop.hive.metastore.model.mstoragedescrip...@4ccd21c" using
statement "INSERT INTO `SDS`
(`SD_ID`,`NUM_BUCKETS`,`INPUT_FORMAT`,`OUTPUT_FORMAT`,`LOCATION`,`SERDE_ID`,`ISCOMPRESSED`)
VALUES (?,?,?,?,?,?,?)" failed : Unknown column 'ISCOMPRESSED' in 'field list'
NestedThrowables:
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column
'ISCOMPRESSED' in 'field list'
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:325)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:2012)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:144)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:633)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:506)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:384)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:302)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
was:
We recently upgraded from Datanucleus 1.0 to 2.0, which changed some of the
defaults for how field names get mapped to datastore identifiers. Because of
this change, connecting to an existing database would throw exceptions such as:
{code}
2010-06-24 17:59:09,854 ERROR exec.DDLTask (SessionState.java:printError(277))
- FAILED: Error in metadata: javax.jdo.JDODataStoreException: Insert of object
"org.apache.hadoop.hive.metastore.model.mstoragedescrip...@4ccd21c" using
statement "INSERT INTO `SDS`
(`SD_ID`,`NUM_BUCKETS`,`INPUT_FORMAT`,`OUTPUT_FORMAT`,`LOCATION`,`SERDE_ID`,`ISCOMPRESSED`)
VALUES (?,?,?,?,?,?,?)" failed : Unknown column 'ISCOMPRESSED' in 'field list'
NestedThrowables:
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column
'ISCOMPRESSED' in 'field list'
org.apache.hadoop.hive.ql.metadata.HiveException:
javax.jdo.JDODataStoreException: Insert of object
"org.apache.hadoop.hive.metastore.model.mstoragedescrip...@4ccd21c" using
statement "INSERT INTO `SDS`
(`SD_ID`,`NUM_BUCKETS`,`INPUT_FORMAT`,`OUTPUT_FORMAT`,`LOCATION`,`SERDE_ID`,`ISCOMPRESSED`)
VALUES (?,?,?,?,?,?,?)" failed : Unknown column 'ISCOMPRESSED' in 'field list'
NestedThrowables:
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column
'ISCOMPRESSED' in 'field list'
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:325)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:2012)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:144)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:633)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:506)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:384)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:302)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(
[jira] Commented: (HIVE-1176) 'create if not exists' fails for a table name with 'select' in it
[ https://issues.apache.org/jira/browse/HIVE-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881572#action_12881572 ] Paul Yang commented on HIVE-1176: - Looks good to me +1 > 'create if not exists' fails for a table name with 'select' in it > - > > Key: HIVE-1176 > URL: https://issues.apache.org/jira/browse/HIVE-1176 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore, Query Processor >Reporter: Prasad Chakka >Assignee: Arvind Prabhakar > Fix For: 0.6.0 > > Attachments: HIVE-1176-1.patch, HIVE-1176-2.patch, HIVE-1176-3.patch, > HIVE-1176-4.patch, HIVE-1176.lib-files.tar.gz, HIVE-1176.patch > > > hive> create table if not exists tmp_select(s string, c string, n int); > org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Got > exception: javax.jdo.JDOUserException JDOQL Single-String query should always > start with SELECT) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:441) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesByPattern(Hive.java:423) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:5538) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:5192) > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:105) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:275) > at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > Caused by: MetaException(message:Got exception: javax.jdo.JDOUserException > JDOQL Single-String query should always start with SELECT) > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:612) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:450) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:439) > ... 15 more -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1176) 'create if not exists' fails for a table name with 'select' in it
[ https://issues.apache.org/jira/browse/HIVE-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881568#action_12881568 ] Paul Yang commented on HIVE-1176: - @John - good point! In which case, we'll probably need to fix up some of the other unit tests with a create table. > 'create if not exists' fails for a table name with 'select' in it > - > > Key: HIVE-1176 > URL: https://issues.apache.org/jira/browse/HIVE-1176 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore, Query Processor >Reporter: Prasad Chakka >Assignee: Arvind Prabhakar > Fix For: 0.6.0 > > Attachments: HIVE-1176-1.patch, HIVE-1176-2.patch, HIVE-1176-3.patch, > HIVE-1176-4.patch, HIVE-1176.lib-files.tar.gz, HIVE-1176.patch > > > hive> create table if not exists tmp_select(s string, c string, n int); > org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Got > exception: javax.jdo.JDOUserException JDOQL Single-String query should always > start with SELECT) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:441) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesByPattern(Hive.java:423) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:5538) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:5192) > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:105) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:275) > at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > Caused by: MetaException(message:Got exception: javax.jdo.JDOUserException > JDOQL Single-String query should always start with SELECT) > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:612) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:450) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:439) > ... 15 more -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1176) 'create if not exists' fails for a table name with 'select' in it
[ https://issues.apache.org/jira/browse/HIVE-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881559#action_12881559 ] Paul Yang commented on HIVE-1176: - @Arvind - While the test that you added is thorough, I think for this case, we might be able to get by with something simpler / faster? e.g. * Use one of the existing source files or insert from an existing table like 'src'. * Limit to one select statement / table as we test select more extensively in other unit tests. * Remove the drop table statements in the beginning, as other tests aren't supposed to leave tables around at the end of their run. * Rename the query file to something more descriptive. This would help developers browsing the list of tests. We don't want to sacrifice coverage, but at the same time, the whole suite of tests take quite a while to execute. > 'create if not exists' fails for a table name with 'select' in it > - > > Key: HIVE-1176 > URL: https://issues.apache.org/jira/browse/HIVE-1176 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore, Query Processor >Reporter: Prasad Chakka >Assignee: Arvind Prabhakar > Fix For: 0.6.0 > > Attachments: HIVE-1176-1.patch, HIVE-1176-2.patch, HIVE-1176-3.patch, > HIVE-1176.lib-files.tar.gz, HIVE-1176.patch > > > hive> create table if not exists tmp_select(s string, c string, n int); > org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Got > exception: javax.jdo.JDOUserException JDOQL Single-String query should always > start with SELECT) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:441) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesByPattern(Hive.java:423) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:5538) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:5192) > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:105) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:275) > at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > Caused by: MetaException(message:Got exception: javax.jdo.JDOUserException > JDOQL Single-String query should always start with SELECT) > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:612) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:450) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:439) > ... 15 more -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1428) ALTER TABLE ADD PARTITION fails with a remote Thirft metastore
[
https://issues.apache.org/jira/browse/HIVE-1428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Yang updated HIVE-1428:
Description:
If the hive cli is configured to use a remote metastore, ALTER TABLE ... ADD
PARTITION commands will fail with an error similar to the following:
{code}
[prade...@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER TABLE
mytable add partition(datestamp = '20091101', srcid = '10',action) location
'/user/pradeepk/mytable/20091101/10';"
10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in
the classpath. Usage of hadoop-site.xml is deprecated. Instead use
core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of
core-default.xml, mapred-default.xml and hdfs-default.xml respectively
Hive history
file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt
FAILED: Error in metadata: org.apache.thrift.TApplicationException:
get_partition failed: unknown result
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask
[prade...@chargesize:~/dev/howl]
{code}
This is due to a check that tries to retrieve the partition to see if it
exists. If it does not, an attempt is made to pass a null value from the
metastore. Since thrift does not support null return values, an exception is
thrown.
was:
If the hive cli is configured to use a remote metastore, ALTER TABLE ... ADD
PARTITION commands will fail with an error similar to the following:
{code}
[prade...@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER TABLE
mytable add partition(datestamp = '20091101', srcid = '10',action) location
'/user/pradeepk/mytable/20091101/10';"
10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in
the classpath. Usage of hadoop-site.xml is deprecated. Instead use
core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of
core-default.xml, mapred-default.xml and hdfs-default.xml respectively
Hive history
file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt
FAILED: Error in metadata: org.apache.thrift.TApplicationException:
get_partition failed: unknown result
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask
[prade...@chargesize:~/dev/howl]
{code}
This is due to a check that tries to retrieve the partition to check if it
exists. If it does not, an attempt is made to pass a null partition value from
the metastore. Since thrift does not support null return values, an exception
is thrown when the CLI is configured to use a remote metastore.
> ALTER TABLE ADD PARTITION fails with a remote Thirft metastore
> --
>
> Key: HIVE-1428
> URL: https://issues.apache.org/jira/browse/HIVE-1428
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 0.6.0, 0.7.0
>Reporter: Paul Yang
>
> If the hive cli is configured to use a remote metastore, ALTER TABLE ... ADD
> PARTITION commands will fail with an error similar to the following:
> {code}
> [prade...@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER TABLE
> mytable add partition(datestamp = '20091101', srcid = '10',action) location
> '/user/pradeepk/mytable/20091101/10';"
> 10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found
> in the classpath. Usage of hadoop-site.xml is deprecated. Instead use
> core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of
> core-default.xml, mapred-default.xml and hdfs-default.xml respectively
> Hive history
> file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt
> FAILED: Error in metadata: org.apache.thrift.TApplicationException:
> get_partition failed: unknown result
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask
> [prade...@chargesize:~/dev/howl]
> {code}
> This is due to a check that tries to retrieve the partition to see if it
> exists. If it does not, an attempt is made to pass a null value from the
> metastore. Since thrift does not support null return values, an exception is
> thrown.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1428) ALTER TABLE ADD PARTITION fails with a remote Thirft metastore
ALTER TABLE ADD PARTITION fails with a remote Thirft metastore
--
Key: HIVE-1428
URL: https://issues.apache.org/jira/browse/HIVE-1428
Project: Hadoop Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.6.0, 0.7.0
Reporter: Paul Yang
If the hive cli is configured to use a remote metastore, ALTER TABLE ... ADD
PARTITION commands will fail with an error similar to the following:
{code}
[prade...@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER TABLE
mytable add partition(datestamp = '20091101', srcid = '10',action) location
'/user/pradeepk/mytable/20091101/10';"
10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in
the classpath. Usage of hadoop-site.xml is deprecated. Instead use
core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of
core-default.xml, mapred-default.xml and hdfs-default.xml respectively
Hive history
file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt
FAILED: Error in metadata: org.apache.thrift.TApplicationException:
get_partition failed: unknown result
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask
[prade...@chargesize:~/dev/howl]
{code}
This is due to a check that tries to retrieve the partition to check if it
exists. If it does not, an attempt is made to pass a null partition value from
the metastore. Since thrift does not support null return values, an exception
is thrown when the CLI is configured to use a remote metastore.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1176) 'create if not exists' fails for a table name with 'select' in it
[ https://issues.apache.org/jira/browse/HIVE-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881497#action_12881497 ] Paul Yang commented on HIVE-1176: - Oh, but I thought the original problem (as per title) was an exception with 'create table if not exists tmp_select(s string, c string, n int)'? So maybe something like: CREATE TABLE IF NOT EXISTS tmp_select(s STRING, c STRING, n INT); DROP TABLE tmp_select; > 'create if not exists' fails for a table name with 'select' in it > - > > Key: HIVE-1176 > URL: https://issues.apache.org/jira/browse/HIVE-1176 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore, Query Processor >Reporter: Prasad Chakka >Assignee: Arvind Prabhakar > Fix For: 0.6.0 > > Attachments: HIVE-1176-1.patch, HIVE-1176-2.patch, > HIVE-1176.lib-files.tar.gz, HIVE-1176.patch > > > hive> create table if not exists tmp_select(s string, c string, n int); > org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Got > exception: javax.jdo.JDOUserException JDOQL Single-String query should always > start with SELECT) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:441) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesByPattern(Hive.java:423) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:5538) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:5192) > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:105) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:275) > at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > Caused by: MetaException(message:Got exception: javax.jdo.JDOUserException > JDOQL Single-String query should always start with SELECT) > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:612) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:450) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:439) > ... 15 more -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1176) 'create if not exists' fails for a table name with 'select' in it
[ https://issues.apache.org/jira/browse/HIVE-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881484#action_12881484 ] Paul Yang commented on HIVE-1176: - One last thing, can you include a unit test to verify the fix? > 'create if not exists' fails for a table name with 'select' in it > - > > Key: HIVE-1176 > URL: https://issues.apache.org/jira/browse/HIVE-1176 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore, Query Processor >Reporter: Prasad Chakka >Assignee: Arvind Prabhakar > Fix For: 0.6.0 > > Attachments: HIVE-1176-1.patch, HIVE-1176-2.patch, > HIVE-1176.lib-files.tar.gz, HIVE-1176.patch > > > hive> create table if not exists tmp_select(s string, c string, n int); > org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Got > exception: javax.jdo.JDOUserException JDOQL Single-String query should always > start with SELECT) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:441) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesByPattern(Hive.java:423) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:5538) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:5192) > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:105) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:275) > at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > Caused by: MetaException(message:Got exception: javax.jdo.JDOUserException > JDOQL Single-String query should always start with SELECT) > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:612) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:450) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:439) > ... 15 more -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1176) 'create if not exists' fails for a table name with 'select' in it
[ https://issues.apache.org/jira/browse/HIVE-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881446#action_12881446 ] Paul Yang commented on HIVE-1176: - Tried it again, and yep, it works. Not sure what happened before. Anyway though, yeah I do see the NPE's that you mentioned. Why did this not occur before? Just changes in behavior between major versions? > 'create if not exists' fails for a table name with 'select' in it > - > > Key: HIVE-1176 > URL: https://issues.apache.org/jira/browse/HIVE-1176 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore, Query Processor >Reporter: Prasad Chakka >Assignee: Arvind Prabhakar > Fix For: 0.6.0 > > Attachments: HIVE-1176-1.patch, HIVE-1176-2.patch, > HIVE-1176.lib-files.tar.gz, HIVE-1176.patch > > > hive> create table if not exists tmp_select(s string, c string, n int); > org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Got > exception: javax.jdo.JDOUserException JDOQL Single-String query should always > start with SELECT) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:441) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesByPattern(Hive.java:423) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:5538) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:5192) > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:105) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:275) > at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > Caused by: MetaException(message:Got exception: javax.jdo.JDOUserException > JDOQL Single-String query should always start with SELECT) > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:612) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:450) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:439) > ... 15 more -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1176) 'create if not exists' fails for a table name with 'select' in it
[ https://issues.apache.org/jira/browse/HIVE-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881304#action_12881304 ] Paul Yang commented on HIVE-1176: - Nevermind - JVS pointed out that jars are skipped as they are binary files. > 'create if not exists' fails for a table name with 'select' in it > - > > Key: HIVE-1176 > URL: https://issues.apache.org/jira/browse/HIVE-1176 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore, Query Processor >Reporter: Prasad Chakka >Assignee: Arvind Prabhakar > Fix For: 0.6.0 > > Attachments: HIVE-1176-1.patch, HIVE-1176-2.patch, > HIVE-1176.lib-files.tar.gz, HIVE-1176.patch > > > hive> create table if not exists tmp_select(s string, c string, n int); > org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Got > exception: javax.jdo.JDOUserException JDOQL Single-String query should always > start with SELECT) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:441) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesByPattern(Hive.java:423) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:5538) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:5192) > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:105) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:275) > at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > Caused by: MetaException(message:Got exception: javax.jdo.JDOUserException > JDOQL Single-String query should always start with SELECT) > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:612) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:450) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:439) > ... 15 more -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1176) 'create if not exists' fails for a table name with 'select' in it
[ https://issues.apache.org/jira/browse/HIVE-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881296#action_12881296 ] Paul Yang commented on HIVE-1176: - To clarify, I am referring to the jars under /lib that should be deleted along with the *.LICENSE files. > 'create if not exists' fails for a table name with 'select' in it > - > > Key: HIVE-1176 > URL: https://issues.apache.org/jira/browse/HIVE-1176 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore, Query Processor >Reporter: Prasad Chakka >Assignee: Arvind Prabhakar > Fix For: 0.6.0 > > Attachments: HIVE-1176-1.patch, HIVE-1176-2.patch, > HIVE-1176.lib-files.tar.gz, HIVE-1176.patch > > > hive> create table if not exists tmp_select(s string, c string, n int); > org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Got > exception: javax.jdo.JDOUserException JDOQL Single-String query should always > start with SELECT) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:441) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesByPattern(Hive.java:423) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:5538) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:5192) > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:105) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:275) > at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > Caused by: MetaException(message:Got exception: javax.jdo.JDOUserException > JDOQL Single-String query should always start with SELECT) > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:612) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:450) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:439) > ... 15 more -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1176) 'create if not exists' fails for a table name with 'select' in it
[
https://issues.apache.org/jira/browse/HIVE-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881293#action_12881293
]
Paul Yang commented on HIVE-1176:
-
For some reason, I don't see the JDO files being deleted after applying the
patch:
{code}
? build.xml.orig
? HIVE-1176-2.patch
? test.log
M eclipse-templates/.classpath
M build.properties
M build.xml
? metastore/test.log
M metastore/ivy.xml
M
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
! lib/jdo2-api-2.3-SNAPSHOT.LICENSE
! lib/datanucleus-rdbms-1.1.2.LICENSE
! lib/datanucleus-enhancer-1.1.2.LICENSE
! lib/datanucleus-core-1.1.2.LICENSE
M ivy/ivysettings.xml
{code}
Also, the patch works for branch 0.6 but not for trunk. Can you regenerate it?
> 'create if not exists' fails for a table name with 'select' in it
> -
>
> Key: HIVE-1176
> URL: https://issues.apache.org/jira/browse/HIVE-1176
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Metastore, Query Processor
>Reporter: Prasad Chakka
>Assignee: Arvind Prabhakar
> Fix For: 0.6.0
>
> Attachments: HIVE-1176-1.patch, HIVE-1176-2.patch,
> HIVE-1176.lib-files.tar.gz, HIVE-1176.patch
>
>
> hive> create table if not exists tmp_select(s string, c string, n int);
> org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Got
> exception: javax.jdo.JDOUserException JDOQL Single-String query should always
> start with SELECT)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:441)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.getTablesByPattern(Hive.java:423)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:5538)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:5192)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:105)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:275)
> at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: MetaException(message:Got exception: javax.jdo.JDOUserException
> JDOQL Single-String query should always start with SELECT)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:612)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:450)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:439)
> ... 15 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1176) 'create if not exists' fails for a table name with 'select' in it
[ https://issues.apache.org/jira/browse/HIVE-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881026#action_12881026 ] Paul Yang commented on HIVE-1176: - I was going to look at it again today, but looks like I'll get to it around mid-day tomorrow? Will keep this posted. > 'create if not exists' fails for a table name with 'select' in it > - > > Key: HIVE-1176 > URL: https://issues.apache.org/jira/browse/HIVE-1176 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore, Query Processor >Reporter: Prasad Chakka >Assignee: Arvind Prabhakar > Fix For: 0.6.0 > > Attachments: HIVE-1176-1.patch, HIVE-1176-2.patch, > HIVE-1176.lib-files.tar.gz, HIVE-1176.patch > > > hive> create table if not exists tmp_select(s string, c string, n int); > org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Got > exception: javax.jdo.JDOUserException JDOQL Single-String query should always > start with SELECT) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:441) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesByPattern(Hive.java:423) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:5538) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:5192) > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:105) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:275) > at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > Caused by: MetaException(message:Got exception: javax.jdo.JDOUserException > JDOQL Single-String query should always start with SELECT) > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:612) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:450) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:439) > ... 15 more -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1417) Archived partitions throw error with queries calling getContentSummary
[
https://issues.apache.org/jira/browse/HIVE-1417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Yang updated HIVE-1417:
Attachment: HIVE-1417.1.patch
HIVE-1417.branch-0.6.1.patch
Expanded test coverage to include join and group by, but this bug doesn't show
up during unit tests because the underlying filesystem is not HDFS.
> Archived partitions throw error with queries calling getContentSummary
> --
>
> Key: HIVE-1417
> URL: https://issues.apache.org/jira/browse/HIVE-1417
> Project: Hadoop Hive
> Issue Type: Bug
>Affects Versions: 0.6.0, 0.7.0
>Reporter: Paul Yang
>Assignee: Paul Yang
> Attachments: HIVE-1417.1.patch, HIVE-1417.branch-0.6.1.patch
>
>
> Assuming you have a src table with a ds='1' partition that is archived in
> HDFS, the following query will throw an exception
> {code}
> select count(1) from src where ds='1' group by key;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1417) Archived partitions throw error with queries calling getContentSummary
[
https://issues.apache.org/jira/browse/HIVE-1417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Yang updated HIVE-1417:
Status: Patch Available (was: Open)
Affects Version/s: 0.7.0
> Archived partitions throw error with queries calling getContentSummary
> --
>
> Key: HIVE-1417
> URL: https://issues.apache.org/jira/browse/HIVE-1417
> Project: Hadoop Hive
> Issue Type: Bug
>Affects Versions: 0.6.0, 0.7.0
>Reporter: Paul Yang
>Assignee: Paul Yang
> Attachments: HIVE-1417.1.patch, HIVE-1417.branch-0.6.1.patch
>
>
> Assuming you have a src table with a ds='1' partition that is archived in
> HDFS, the following query will throw an exception
> {code}
> select count(1) from src where ds='1' group by key;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1417) Archived partitions throw error with queries calling getContentSummary
[
https://issues.apache.org/jira/browse/HIVE-1417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Yang updated HIVE-1417:
Description:
Assuming you have a src table with a ds='1' partition that is archived in HDFS,
the following query will throw an exception
{code}
select count(1) from src where ds='1' group by key;
{code}
was:
Assuming you have a src table with a ds='1' partition that is archived, the
following table will throw an exception
{code}
select count(1) from src where ds='1' group by key;
{code}
> Archived partitions throw error with queries calling getContentSummary
> --
>
> Key: HIVE-1417
> URL: https://issues.apache.org/jira/browse/HIVE-1417
> Project: Hadoop Hive
> Issue Type: Bug
>Affects Versions: 0.6.0, 0.7.0
>Reporter: Paul Yang
>Assignee: Paul Yang
> Attachments: HIVE-1417.1.patch, HIVE-1417.branch-0.6.1.patch
>
>
> Assuming you have a src table with a ds='1' partition that is archived in
> HDFS, the following query will throw an exception
> {code}
> select count(1) from src where ds='1' group by key;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1417) Archived partitions throw error with queries calling getContentSummary
Archived partitions throw error with queries calling getContentSummary
--
Key: HIVE-1417
URL: https://issues.apache.org/jira/browse/HIVE-1417
Project: Hadoop Hive
Issue Type: Bug
Affects Versions: 0.6.0
Reporter: Paul Yang
Assignee: Paul Yang
Assuming you have a src table with a ds='1' partition that is archived, the
following table will throw an exception
{code}
select count(1) from src where ds='1' group by key;
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1255) Add mathamatical UDFs PI, E, degrees, radians, tan, sign, and atan
[ https://issues.apache.org/jira/browse/HIVE-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879423#action_12879423 ] Paul Yang commented on HIVE-1255: - Looks good +1 > Add mathamatical UDFs PI, E, degrees, radians, tan, sign, and atan > -- > > Key: HIVE-1255 > URL: https://issues.apache.org/jira/browse/HIVE-1255 > Project: Hadoop Hive > Issue Type: New Feature > Components: Query Processor >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: hive-1255-patch-2.txt, hive-1255-patch-3.txt, > hive-1255-patch-4.txt, hive-1255-patch.txt > > > Add support for PI, E, degrees, radians, tan, sign and atan -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1411) DataNucleus barfs if JAR appears more than once in CLASSPATH
[
https://issues.apache.org/jira/browse/HIVE-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879218#action_12879218
]
Paul Yang commented on HIVE-1411:
-
Taking a look.
> DataNucleus barfs if JAR appears more than once in CLASSPATH
>
>
> Key: HIVE-1411
> URL: https://issues.apache.org/jira/browse/HIVE-1411
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 0.4.0, 0.4.1, 0.5.0
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Fix For: 0.6.0
>
> Attachments: HIVE-1411.patch.txt
>
>
> DataNucleus barfs when more than one JAR with the same name appears on the
> CLASSPATH:
> {code}
> 2010-03-06 12:33:25,565 ERROR exec.DDLTask
> (SessionState.java:printError(279)) - FAILED: Error in metadata:
> javax.jdo.JDOFatalInter
> nalException: Unexpected exception caught.
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> org.apache.hadoop.hive.ql.metadata.HiveException:
> javax.jdo.JDOFatalInternalException: Unexpected exception caught.
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:258)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:879)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:103)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:379)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:285)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: javax.jdo.JDOFatalInternalException: Unexpected exception caught.
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> at
> javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1186)
> at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:803)
> at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:698)
> at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:164)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:181)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:125)
> at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:104)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:130)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:146)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:118)
>
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.(HiveMetaStore.java:100)
>
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:74)
>
> at
> org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:783)
> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:794)
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:252)
> ... 12 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at javax.jdo.JDOHelper$16.run(JDOHelper.java:1956)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.jdo.JDOHelper.invoke(JDOHelper.java:1951)
> at
> javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1159)
> ... 28 more
> Caused by: org.datanucleus.exceptions.NucleusException: Plugin (Bundle)
> "org.eclipse.jdt.core" is already registered. Ensure you do
> nt have multiple JAR versions of the same plugin in the classpath. The URL
> "file:/Users/hadop/hadoop-0.20.1+152/build/ivy/lib/Hadoo
> p/common/core-3.1.1.jar" is already registered, and you are trying to
> register an identical plugin located at URL "file:/Users/hado
> p/hadoop-0.20.1+152/lib/core-3.1.1.jar."
> at
[jira] Updated: (HIVE-1255) Add mathamatical UDFs PI, E, degrees, radians, tan, sign, and atan
[ https://issues.apache.org/jira/browse/HIVE-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1255: Status: Open (was: Patch Available) > Add mathamatical UDFs PI, E, degrees, radians, tan, sign, and atan > -- > > Key: HIVE-1255 > URL: https://issues.apache.org/jira/browse/HIVE-1255 > Project: Hadoop Hive > Issue Type: New Feature > Components: Query Processor >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: hive-1255-patch-2.txt, hive-1255-patch-3.txt, > hive-1255-patch.txt > > > Add support for PI, E, degrees, radians, tan, sign and atan -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1255) Add mathamatical UDFs PI, E, degrees, radians, tan, sign, and atan
[ https://issues.apache.org/jira/browse/HIVE-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879216#action_12879216 ] Paul Yang commented on HIVE-1255: - Can you update show_functions.q? That test will fail with the addition of new UDF's. > Add mathamatical UDFs PI, E, degrees, radians, tan, sign, and atan > -- > > Key: HIVE-1255 > URL: https://issues.apache.org/jira/browse/HIVE-1255 > Project: Hadoop Hive > Issue Type: New Feature > Components: Query Processor >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: hive-1255-patch-2.txt, hive-1255-patch-3.txt, > hive-1255-patch.txt > > > Add support for PI, E, degrees, radians, tan, sign and atan -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1255) Add mathamatical UDFs PI, E, degrees, radians, tan, sign, and atan
[ https://issues.apache.org/jira/browse/HIVE-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879212#action_12879212 ] Paul Yang commented on HIVE-1255: - Taking a look. > Add mathamatical UDFs PI, E, degrees, radians, tan, sign, and atan > -- > > Key: HIVE-1255 > URL: https://issues.apache.org/jira/browse/HIVE-1255 > Project: Hadoop Hive > Issue Type: New Feature > Components: Query Processor >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: hive-1255-patch-2.txt, hive-1255-patch-3.txt, > hive-1255-patch.txt > > > Add support for PI, E, degrees, radians, tan, sign and atan -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1176) 'create if not exists' fails for a table name with 'select' in it
[ https://issues.apache.org/jira/browse/HIVE-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879208#action_12879208 ] Paul Yang commented on HIVE-1176: - Can you elaborate on what you mean by 'some collections were being fetched as semi-populated proxies with missing session context leading to NPEs'? Is there something I can do to reproduce this? > 'create if not exists' fails for a table name with 'select' in it > - > > Key: HIVE-1176 > URL: https://issues.apache.org/jira/browse/HIVE-1176 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore, Query Processor >Reporter: Prasad Chakka >Assignee: Arvind Prabhakar > Attachments: HIVE-1176-1.patch, HIVE-1176.lib-files.tar.gz, > HIVE-1176.patch > > > hive> create table if not exists tmp_select(s string, c string, n int); > org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Got > exception: javax.jdo.JDOUserException JDOQL Single-String query should always > start with SELECT) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:441) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesByPattern(Hive.java:423) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:5538) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:5192) > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:105) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:275) > at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > Caused by: MetaException(message:Got exception: javax.jdo.JDOUserException > JDOQL Single-String query should always start with SELECT) > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:612) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:450) > at > org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:439) > ... 15 more -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1410) Add TCP keepalive option for the metastore server
[ https://issues.apache.org/jira/browse/HIVE-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1410: Status: Patch Available (was: Open) > Add TCP keepalive option for the metastore server > - > > Key: HIVE-1410 > URL: https://issues.apache.org/jira/browse/HIVE-1410 > Project: Hadoop Hive > Issue Type: New Feature > Components: Metastore >Affects Versions: 0.6.0 >Reporter: Paul Yang >Assignee: Paul Yang > Attachments: HIVE-1410.1.patch > > > In production, we have noticed that the metastore server tends to accumulate > half-open TCP connections when it has been running for a long time. By > half-open, I am referring to idle connections where there is an established > TCP connection on the metastore server machine, but no corresponding > connection on the client machine. This could be the result of network > disconnects or crashed clients. > This patch will add an option to turn on TCP keepalive so that these > half-open connections will get cleaned up. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1410) Add TCP keepalive option for the metastore server
[ https://issues.apache.org/jira/browse/HIVE-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1410: Attachment: HIVE-1410.1.patch No unit test for this one, but I ran 2 metastore servers, one with and one without this patch to verify that the one with the patch did not accumulate half-open connections. > Add TCP keepalive option for the metastore server > - > > Key: HIVE-1410 > URL: https://issues.apache.org/jira/browse/HIVE-1410 > Project: Hadoop Hive > Issue Type: New Feature > Components: Metastore >Affects Versions: 0.6.0 >Reporter: Paul Yang >Assignee: Paul Yang > Attachments: HIVE-1410.1.patch > > > In production, we have noticed that the metastore server tends to accumulate > half-open TCP connections when it has been running for a long time. By > half-open, I am referring to idle connections where there is an established > TCP connection on the metastore server machine, but no corresponding > connection on the client machine. This could be the result of network > disconnects or crashed clients. > This patch will add an option to turn on TCP keepalive so that these > half-open connections will get cleaned up. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1410) Add TCP keepalive option for the metastore server
Add TCP keepalive option for the metastore server - Key: HIVE-1410 URL: https://issues.apache.org/jira/browse/HIVE-1410 Project: Hadoop Hive Issue Type: New Feature Components: Metastore Affects Versions: 0.6.0 Reporter: Paul Yang Assignee: Paul Yang In production, we have noticed that the metastore server tends to accumulate half-open TCP connections when it has been running for a long time. By half-open, I am referring to idle connections where there is an established TCP connection on the metastore server machine, but no corresponding connection on the client machine. This could be the result of network disconnects or crashed clients. This patch will add an option to turn on TCP keepalive so that these half-open connections will get cleaned up. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1409) File format information is retrieved from first partition
[
https://issues.apache.org/jira/browse/HIVE-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Yang updated HIVE-1409:
Attachment: HIVE-1409.2.patch
* Drop table added
> File format information is retrieved from first partition
> -
>
> Key: HIVE-1409
> URL: https://issues.apache.org/jira/browse/HIVE-1409
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 0.6.0
>Reporter: Paul Yang
>Assignee: Paul Yang
> Attachments: HIVE-1409.1.patch, HIVE-1409.2.patch
>
>
> Currently, if no partitions match the partition predicate, the first
> partition is used to retrieve the file format. This can cause an problem if
> the table is set to use RCFile, but the first partition uses SequenceFile:
> {code}
> java.lang.RuntimeException: java.lang.NoSuchMethodException:
> org.apache.hadoop.hive.ql.io.RCFile$KeyBuffer.()
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at
> org.apache.hadoop.mapred.SequenceFileRecordReader.createKey(SequenceFileRecordReader.java:65)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.createKey(CombineHiveRecordReader.java:76)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.createKey(CombineHiveRecordReader.java:42)
> at
> org.apache.hadoop.hive.shims.Hadoop20Shims$CombineFileRecordReader.createKey(Hadoop20Shims.java:212)
> at
> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.createKey(MapTask.java:167)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:45)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.NoSuchMethodException:
> org.apache.hadoop.hive.ql.io.RCFile$KeyBuffer.()
> at java.lang.Class.getConstructor0(Class.java:2706)
> at java.lang.Class.getDeclaredConstructor(Class.java:1985)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:109)
> ... 9 more
> {code}
> The proposed change is to use the table's metadata in such cases.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1409) File format information is retrieved from first partition
[
https://issues.apache.org/jira/browse/HIVE-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Yang updated HIVE-1409:
Status: Patch Available (was: Open)
> File format information is retrieved from first partition
> -
>
> Key: HIVE-1409
> URL: https://issues.apache.org/jira/browse/HIVE-1409
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 0.6.0
>Reporter: Paul Yang
>Assignee: Paul Yang
> Attachments: HIVE-1409.1.patch, HIVE-1409.2.patch
>
>
> Currently, if no partitions match the partition predicate, the first
> partition is used to retrieve the file format. This can cause an problem if
> the table is set to use RCFile, but the first partition uses SequenceFile:
> {code}
> java.lang.RuntimeException: java.lang.NoSuchMethodException:
> org.apache.hadoop.hive.ql.io.RCFile$KeyBuffer.()
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at
> org.apache.hadoop.mapred.SequenceFileRecordReader.createKey(SequenceFileRecordReader.java:65)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.createKey(CombineHiveRecordReader.java:76)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.createKey(CombineHiveRecordReader.java:42)
> at
> org.apache.hadoop.hive.shims.Hadoop20Shims$CombineFileRecordReader.createKey(Hadoop20Shims.java:212)
> at
> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.createKey(MapTask.java:167)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:45)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.NoSuchMethodException:
> org.apache.hadoop.hive.ql.io.RCFile$KeyBuffer.()
> at java.lang.Class.getConstructor0(Class.java:2706)
> at java.lang.Class.getDeclaredConstructor(Class.java:1985)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:109)
> ... 9 more
> {code}
> The proposed change is to use the table's metadata in such cases.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1409) File format information is retrieved from first partition
[
https://issues.apache.org/jira/browse/HIVE-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Yang updated HIVE-1409:
Status: Patch Available (was: Open)
> File format information is retrieved from first partition
> -
>
> Key: HIVE-1409
> URL: https://issues.apache.org/jira/browse/HIVE-1409
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 0.6.0
>Reporter: Paul Yang
>Assignee: Paul Yang
> Attachments: HIVE-1409.1.patch
>
>
> Currently, if no partitions match the partition predicate, the first
> partition is used to retrieve the file format. This can cause an problem if
> the table is set to use RCFile, but the first partition uses SequenceFile:
> {code}
> java.lang.RuntimeException: java.lang.NoSuchMethodException:
> org.apache.hadoop.hive.ql.io.RCFile$KeyBuffer.()
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at
> org.apache.hadoop.mapred.SequenceFileRecordReader.createKey(SequenceFileRecordReader.java:65)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.createKey(CombineHiveRecordReader.java:76)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.createKey(CombineHiveRecordReader.java:42)
> at
> org.apache.hadoop.hive.shims.Hadoop20Shims$CombineFileRecordReader.createKey(Hadoop20Shims.java:212)
> at
> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.createKey(MapTask.java:167)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:45)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.NoSuchMethodException:
> org.apache.hadoop.hive.ql.io.RCFile$KeyBuffer.()
> at java.lang.Class.getConstructor0(Class.java:2706)
> at java.lang.Class.getDeclaredConstructor(Class.java:1985)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:109)
> ... 9 more
> {code}
> The proposed change is to use the table's metadata in such cases.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
