[jira] [Commented] (HIVE-2468) Make Hive compile against Hadoop 0.23
[ https://issues.apache.org/jira/browse/HIVE-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160712#comment-13160712 ] Hudson commented on HIVE-2468: -- Integrated in Hive-trunk-h0.21 #1115 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1115/]) HIVE-2468. Make Hive compile against Hadoop 0.23 (Tom White via cws) cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1208940 Files : * /hive/trunk/bin/hive * /hive/trunk/build-common.xml * /hive/trunk/build.properties * /hive/trunk/build.xml * /hive/trunk/conf/hive-default.xml * /hive/trunk/contrib/build.xml * /hive/trunk/hbase-handler/build.xml * /hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java * /hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableOutputFormat.java * /hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHFileOutputFormat.java * /hive/trunk/hwi/build.xml * /hive/trunk/jdbc/build.xml * /hive/trunk/ql/build.xml * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/JobTrackerURLResolver.java * /hive/trunk/service/build.xml * /hive/trunk/service/src/java/org/apache/hadoop/hive/service/HiveServer.java * /hive/trunk/shims/build.xml * /hive/trunk/shims/src/0.20/java/org/apache/hadoop/fs/ProxyFileSystem.java * /hive/trunk/shims/src/0.20/java/org/apache/hadoop/fs/ProxyLocalFileSystem.java * /hive/trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java * /hive/trunk/shims/src/0.23 * /hive/trunk/shims/src/0.23/java * /hive/trunk/shims/src/0.23/java/org * /hive/trunk/shims/src/0.23/java/org/apache * /hive/trunk/shims/src/0.23/java/org/apache/hadoop * /hive/trunk/shims/src/0.23/java/org/apache/hadoop/hive * /hive/trunk/shims/src/0.23/java/org/apache/hadoop/hive/shims * /hive/trunk/shims/src/0.23/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java * /hive/trunk/shims/src/0.23/java/org/apache/hadoop/hive/shims/Jetty23Shims.java * /hive/trunk/shims/src/0.23/java/org/apache/hadoop/hive/thrift * /hive/trunk/shims/src/0.23/java/org/apache/hadoop/hive/thrift/DelegationTokenIdentifier23.java * /hive/trunk/shims/src/0.23/java/org/apache/hadoop/hive/thrift/DelegationTokenSelector23.java * /hive/trunk/shims/src/common/java/org/apache/hadoop/fs * /hive/trunk/shims/src/common/java/org/apache/hadoop/fs/ProxyFileSystem.java * /hive/trunk/shims/src/common/java/org/apache/hadoop/fs/ProxyLocalFileSystem.java * /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java * /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/shims/ShimLoader.java > Make Hive compile against Hadoop 0.23 > - > > Key: HIVE-2468 > URL: https://issues.apache.org/jira/browse/HIVE-2468 > Project: Hive > Issue Type: Task >Reporter: Konstantin Shvachko >Assignee: Carl Steinbach > Fix For: 0.8.0, 0.9.0 > > Attachments: HIVE-2468.patch, HIVE-2468.patch, HIVE-2468.patch, > HIVE-2468.patch, HIVE-2468.patch > > > Due to restructure of Hadoop 0.22 branch compared to Hadoop 0.20 Hive does > not compile against 0.22 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2097) Explore mechanisms for better compression with RC Files
[
https://issues.apache.org/jira/browse/HIVE-2097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160726#comment-13160726
]
alex gemini commented on HIVE-2097:
---
another issue is about efficient serialization/deserialization,for the same
example above,assume every gender,age,region have 100 message equally store in
one dfs block,in gender column we store value like
this:{'male'}[1-60k]{'female'}[60k+1 - 120k],age column look like
this:{21}[1-3k]{22}[3k+1 - 6k]{23}[6k+1 - 9k],and region column is
like:{'LA'}[1-300]{'NY'}[301-600].
When we issue a query on a single table like :select sum(age) from logs where
region='LA' and age=30,we count every column represented at
'select,where,group' clause,so we know the last column means lowest
selectivity(in this example is region),we find the region
value={'LA'}[(1-300),(30k+1 - 30k+300),(60k+1 -60k+300)] and 'NY'
value={'NY'}[(301-600),(30k+301 - 30k+600),(60k+301 -60k+600)]
we just need to deserialization it but don't need to decompression it because
we know the lowest selectivity column,then we organize inputSplit's key like
{[age='21'][region='LA']} and value is {(1-300),(30k+1 - 30k+300),(60k+1
-60k+300)},this inputSplit key and value is unique per dfs block because we
already sort column by selectivity,the lowest selectivity column presented at
(select,where,group) must be unique.
> Explore mechanisms for better compression with RC Files
> ---
>
> Key: HIVE-2097
> URL: https://issues.apache.org/jira/browse/HIVE-2097
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor, Serializers/Deserializers
>Reporter: Krishna Kumar
>Assignee: Krishna Kumar
>Priority: Minor
>
> Optimization of the compression mechanisms used by RC File to be explored.
> Some initial ideas
>
> 1. More efficient serialization/deserialization based on type-specific and
> storage-specific knowledge.
>
>For instance, storing sorted numeric values efficiently using some delta
> coding techniques
> 2. More efficient compression based on type-specific and storage-specific
> knowledge
>Enable compression codecs to be specified based on types or individual
> columns
> 3. Reordering the on-disk storage for better compression efficiency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2097) Explore mechanisms for better compression with RC Files
[ https://issues.apache.org/jira/browse/HIVE-2097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160734#comment-13160734 ] alex gemini commented on HIVE-2097: --- I'm not quit sure I explain the columnar database execution strategy clearly,I hope the following material will help: #1 http://www.infoq.com/news/2011/09/nosqlnow-columnar-databases #2 http://www.oscon.com/oscon2010/public/schedule/detail/15561 #3 http://www.vertica.com/2010/05/26/why-verticas-compression-is-better/ #4 http://www.vertica.com/2011/09/01/the-power-of-projections-part-1/ Good luck. > Explore mechanisms for better compression with RC Files > --- > > Key: HIVE-2097 > URL: https://issues.apache.org/jira/browse/HIVE-2097 > Project: Hive > Issue Type: Improvement > Components: Query Processor, Serializers/Deserializers >Reporter: Krishna Kumar >Assignee: Krishna Kumar >Priority: Minor > > Optimization of the compression mechanisms used by RC File to be explored. > Some initial ideas > > 1. More efficient serialization/deserialization based on type-specific and > storage-specific knowledge. > >For instance, storing sorted numeric values efficiently using some delta > coding techniques > 2. More efficient compression based on type-specific and storage-specific > knowledge >Enable compression codecs to be specified based on types or individual > columns > 3. Reordering the on-disk storage for better compression efficiency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2388) Facing issues while executing commands on hive shell. The system throws following error: only on Windows Cygwin setup
[ https://issues.apache.org/jira/browse/HIVE-2388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160839#comment-13160839 ] Bhavesh Shah commented on HIVE-2388: Initially it works with bin/start.sh but after closing cygwin it is not working. It is not showing any kind of output. And one more problem is with select query , that it is giving Execution error on Cygwin. Pls help me. > Facing issues while executing commands on hive shell. The system throws > following error: only on Windows Cygwin setup > - > > Key: HIVE-2388 > URL: https://issues.apache.org/jira/browse/HIVE-2388 > Project: Hive > Issue Type: Bug > Components: CLI, Query Processor >Affects Versions: 0.7.1 > Environment: Cygwin Windows >Reporter: Siddharth tiwari >Priority: Critical > Labels: patch > Fix For: 0.7.1 > > Attachments: start.sh, start.sh > > Original Estimate: 456h > Remaining Estimate: 456h > > DDL runs well but the following command describes throw error pls help with > resolution and how to get about it > hive> show tables > > ; > FAILED: Hive Internal Error: > java.lang.IllegalArgumentException(java.net.URISyntaxException: Relative path > in absolute URI: file:C:/cygwin/tmp//siddharth/hive_2011-08-18_ > 03-11-05_208_1818592223695168110) > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: > file:C:/cygwin/tmp//siddharth/hive_2011-08-18_03-11-05_208_181859222369516 > 8110 > at org.apache.hadoop.fs.Path.initialize(Path.java:140) > at org.apache.hadoop.fs.Path.(Path.java:132) > at org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:142) > at > org.apache.hadoop.hive.ql.Context.getLocalScratchDir(Context.java:168) > at > org.apache.hadoop.hive.ql.Context.getLocalTmpFileURI(Context.java:282) > at > org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:205) > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > Caused by: java.net.URISyntaxException: Relative path in absolute URI: > file:C:/cygwin/tmp//siddharth/hive_2011-08-18_03-11-05_208_1818592223695168110 > at java.net.URI.checkPath(URI.java:1787) > at java.net.URI.(URI.java:735) > at org.apache.hadoop.fs.Path.initialize(Path.java:137) > ... 16 more -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2620) LIKE incorrectly transforms expression to regex (does not escape "+" and possibly other special chars)
LIKE incorrectly transforms expression to regex (does not escape "+" and
possibly other special chars)
--
Key: HIVE-2620
URL: https://issues.apache.org/jira/browse/HIVE-2620
Project: Hive
Issue Type: Bug
Components: UDF
Reporter: Mitja Trampus
Whenever you have a LIKE expression that contains "|+" (the culprit) and "%"
(so it gets converted to regex), hive throws an exception that crashes the
whole job.
hive> select 'foo |+18| bar' like 'foo |+18% bar' from akramer_one_row;
FAILED: Error in semantic analysis: Line 1:7 Wrong arguments ''foo |+18% bar'':
org.apache.hadoop.hive.ql.metadata.HiveException: Unable to execute method
public org.apache.hadoop.io.BooleanWritable
org.apache.hadoop.hive.ql.udf.UDFLike.evaluate(org.apache.hadoop.io.Text,org.apache.hadoop.io.Text)
on object org.apache.hadoop.hive.ql.udf.UDFLike@292e2fba of class
org.apache.hadoop.hive.ql.udf.UDFLike with arguments {foo |+18|
bar:org.apache.hadoop.io.Text, foo |+% bar:org.apache.hadoop.io.Text} of size 2
Stack trace from the real world example with which I found this:
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to execute
method public org.apache.hadoop.io.BooleanWritable
org.apache.hadoop.hive.ql.udf.UDFLike.evaluate(org.apache.hadoop.io.Text,org.apache.hadoop.io.Text)
on object org.apache.hadoop.hive.ql.udf.UDFLike@4a7baf7d of class
org.apache.hadoop.hive.ql.udf.UDFLike with arguments {ewt.arkadaslar
pazartesinden sonra ozel escortlar sayfamızı zıyaret etcek lutfn kaba
dawranmıyalım escortlarımız resmlı olcak sız begenıceksınız escortunuzu
escortlarımı ıl ıl olacktır bılgnıze:org.apache.hadoop.io.Text,
%çıtıR%kızLar%escort%kızLarı%burda%|+%18%|%:org.apache.hadoop.io.Text} of size 2
at
org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:836)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.evaluate(GenericUDFBridge.java:180)
at
org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.evaluate(ExprNodeGenericFuncEvaluator.java:163)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.updateAggregations(GroupByOperator.java:575)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:767)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:722)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:765)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:765)
at
org.apache.hadoop.hive.ql.exec.UnionOperator.processOp(UnionOperator.java:129)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:765)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:531)
... 5 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:812)
... 19 more
Caused by: java.util.regex.PatternSyntaxException: Dangling meta character '+'
near index 42
.*çıtıR.*kızLar.*escort.*kızLarı.*burda.*|+.*18.*|.*
^
at java.util.regex.Pattern.error(Pattern.java:1713)
at java.util.regex.Pattern.sequence(Pattern.java:1878)
at java.util.regex.Pattern.expr(Pattern.java:1752)
at java.util.regex.Pattern.compile(Pattern.java:1460)
at java.util.regex.Pattern.(Pattern.java:1133)
at java.util.regex.Pattern.compile(Pattern.java:823)
at org.apache.hadoop.hive.ql.udf.UDFLike.evaluate(UDFLike.java:186)
... 23 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2620) LIKE incorrectly transforms expression to regex (does not escape "+" and possibly other special chars)
[
https://issues.apache.org/jira/browse/HIVE-2620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mitja Trampus updated HIVE-2620:
Description:
Whenever you have a LIKE expression that contains "|+" (the culprit) and "%"
(so it gets converted to regex), hive throws an exception that crashes the
whole job.
Possibly related: https://issues.apache.org/jira/browse/HIVE-2594
{noformat}
hive> select 'foo |+18| bar' like 'foo |+18% bar' from akramer_one_row;
FAILED: Error in semantic analysis: Line 1:7 Wrong arguments ''foo |+18% bar'':
org.apache.hadoop.hive.ql.metadata.HiveException: Unable to execute method
public org.apache.hadoop.io.BooleanWritable
org.apache.hadoop.hive.ql.udf.UDFLike.evaluate(org.apache.hadoop.io.Text,org.apache.hadoop.io.Text)
on object org.apache.hadoop.hive.ql.udf.UDFLike@292e2fba of class
org.apache.hadoop.hive.ql.udf.UDFLike with arguments {foo |+18|
bar:org.apache.hadoop.io.Text, foo |+% bar:org.apache.hadoop.io.Text} of size 2
{noformat}
Stack trace from the real world example with which I found this:
{noformat}
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to execute
method public org.apache.hadoop.io.BooleanWritable
org.apache.hadoop.hive.ql.udf.UDFLike.evaluate(org.apache.hadoop.io.Text,org.apache.hadoop.io.Text)
on object org.apache.hadoop.hive.ql.udf.UDFLike@4a7baf7d of class
org.apache.hadoop.hive.ql.udf.UDFLike with arguments {ewt.arkadaslar
pazartesinden sonra ozel escortlar sayfamızı zıyaret etcek lutfn kaba
dawranmıyalım escortlarımız resmlı olcak sız begenıceksınız escortunuzu
escortlarımı ıl ıl olacktır bılgnıze:org.apache.hadoop.io.Text,
%çıtıR%kızLar%escort%kızLarı%burda%|+%18%|%:org.apache.hadoop.io.Text} of size 2
at
org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:836)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.evaluate(GenericUDFBridge.java:180)
at
org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.evaluate(ExprNodeGenericFuncEvaluator.java:163)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.updateAggregations(GroupByOperator.java:575)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:767)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:722)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:765)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:765)
at
org.apache.hadoop.hive.ql.exec.UnionOperator.processOp(UnionOperator.java:129)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:765)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:531)
... 5 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:812)
... 19 more
Caused by: java.util.regex.PatternSyntaxException: Dangling meta character '+'
near index 42
.*çıtıR.*kızLar.*escort.*kızLarı.*burda.*|+.*18.*|.*
^
at java.util.regex.Pattern.error(Pattern.java:1713)
at java.util.regex.Pattern.sequence(Pattern.java:1878)
at java.util.regex.Pattern.expr(Pattern.java:1752)
at java.util.regex.Pattern.compile(Pattern.java:1460)
at java.util.regex.Pattern.(Pattern.java:1133)
at java.util.regex.Pattern.compile(Pattern.java:823)
at org.apache.hadoop.hive.ql.udf.UDFLike.evaluate(UDFLike.java:186)
... 23 more
{noformat}
was:
Whenever you have a LIKE expression that contains "|+" (the culprit) and "%"
(so it gets converted to regex), hive throws an exception that crashes the
whole job.
hive> select 'foo |+18| bar' like 'foo |+18% bar' from akramer_one_row;
FAILED: Error in semantic analysis: Line 1:7 Wrong arguments ''foo |+18% bar'':
org.apache.hadoop.hive.ql.metadata.HiveException: Unable to execute method
public org.apache.hadoop.io.BooleanWritable
org.apache.hadoop.hive.ql.udf.UDFLike.evaluate(org.apache.hadoop.io.Text,org.apache.hadoop.io.Text)
on object org.apache.hadoop.hive.ql.udf.UDFLike@292e2fba of class
org.apache.hadoop.hive.ql.udf.UDFLike with arguments {foo |+18|
bar:org.apache.hadoop.io.Text, foo |+% bar:org.apache.hadoop.io.Text} of size 2
Stack t
[jira] [Updated] (HIVE-2619) Add hook to run in meatastore's endFunction which can collect more fb303 counters
[ https://issues.apache.org/jira/browse/HIVE-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2619: -- Attachment: HIVE-2619.D555.1.patch kevinwilfong requested code review of "HIVE-2619 [jira] Add hook to run in meatastore's endFunction which can collect more fb303 counters". Reviewers: njain, heyongqiang, JIRA I added a new type of listener called MetaStoreEndFunctionListener, which is called by endFunction in HMSHandler and takes the method's name, and whether it succeeded. I also overrode FacebookBase's getCounters method to include counters from these listeners. These new listeners are configurable in the same way as EventListeners. I had to modify endFunction and all calls to it to pass in whether or not the method succeeded. TEST PLAN I created an implementation of MetaStoreEndFunctionListener and verified it ran without exceptions, and it could add fb303 counters. I also ran the unit tests. REVISION DETAIL https://reviews.facebook.net/D555 AFFECTED FILES conf/hive-default.xml metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEndFunctionListener.java metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java common/src/java/org/apache/hadoop/hive/conf/HiveConf.java > Add hook to run in meatastore's endFunction which can collect more fb303 > counters > - > > Key: HIVE-2619 > URL: https://issues.apache.org/jira/browse/HIVE-2619 > Project: Hive > Issue Type: Improvement >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2619.1.patch.txt, HIVE-2619.D555.1.patch, > HIVE-2619.D561.1.patch > > > Create the potential for hooks to run in the endFunction method of HMSHandler > which take the name of a function and whether or not it succeeded. Also, > override getCounters from fb303 to allow these hooks to add counters which > they collect, should this be desired. These hooks can be similar to > EventListeners, but they should be more generic. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2619) Add hook to run in meatastore's endFunction which can collect more fb303 counters
[ https://issues.apache.org/jira/browse/HIVE-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2619: -- Attachment: HIVE-2619.D555.2.patch kevinwilfong updated the revision "HIVE-2619 [jira] Add hook to run in meatastore's endFunction which can collect more fb303 counters". Reviewers: JIRA Namit made the following comment here https://reviews.facebook.net/D561 "Looks good - do you want to pass a metastoreContext to the endFunction. This can be a empty class right now, but in future if we need to add anything, it would be useful" That's a good idea, we can put perf data there later. REVISION DETAIL https://reviews.facebook.net/D555 AFFECTED FILES conf/hive-default.xml metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEndFunctionContext.java metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEndFunctionListener.java metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java common/src/java/org/apache/hadoop/hive/conf/HiveConf.java > Add hook to run in meatastore's endFunction which can collect more fb303 > counters > - > > Key: HIVE-2619 > URL: https://issues.apache.org/jira/browse/HIVE-2619 > Project: Hive > Issue Type: Improvement >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2619.1.patch.txt, HIVE-2619.D555.1.patch, > HIVE-2619.D555.2.patch, HIVE-2619.D561.1.patch > > > Create the potential for hooks to run in the endFunction method of HMSHandler > which take the name of a function and whether or not it succeeded. Also, > override getCounters from fb303 to allow these hooks to add counters which > they collect, should this be desired. These hooks can be similar to > EventListeners, but they should be more generic. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HIVE-2388) Facing issues while executing commands on hive shell. The system throws following error: only on Windows Cygwin setup
[ https://issues.apache.org/jira/browse/HIVE-2388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reopened HIVE-2388: -- > Facing issues while executing commands on hive shell. The system throws > following error: only on Windows Cygwin setup > - > > Key: HIVE-2388 > URL: https://issues.apache.org/jira/browse/HIVE-2388 > Project: Hive > Issue Type: Bug > Components: CLI, Query Processor >Affects Versions: 0.7.1 > Environment: Cygwin Windows >Reporter: Siddharth tiwari >Priority: Critical > Labels: patch > Fix For: 0.7.1 > > Attachments: start.sh, start.sh > > Original Estimate: 456h > Remaining Estimate: 456h > > DDL runs well but the following command describes throw error pls help with > resolution and how to get about it > hive> show tables > > ; > FAILED: Hive Internal Error: > java.lang.IllegalArgumentException(java.net.URISyntaxException: Relative path > in absolute URI: file:C:/cygwin/tmp//siddharth/hive_2011-08-18_ > 03-11-05_208_1818592223695168110) > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: > file:C:/cygwin/tmp//siddharth/hive_2011-08-18_03-11-05_208_181859222369516 > 8110 > at org.apache.hadoop.fs.Path.initialize(Path.java:140) > at org.apache.hadoop.fs.Path.(Path.java:132) > at org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:142) > at > org.apache.hadoop.hive.ql.Context.getLocalScratchDir(Context.java:168) > at > org.apache.hadoop.hive.ql.Context.getLocalTmpFileURI(Context.java:282) > at > org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:205) > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > Caused by: java.net.URISyntaxException: Relative path in absolute URI: > file:C:/cygwin/tmp//siddharth/hive_2011-08-18_03-11-05_208_1818592223695168110 > at java.net.URI.checkPath(URI.java:1787) > at java.net.URI.(URI.java:735) > at org.apache.hadoop.fs.Path.initialize(Path.java:137) > ... 16 more -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2388) Facing issues while executing commands on hive shell. The system throws following error: only on Windows Cygwin setup
[ https://issues.apache.org/jira/browse/HIVE-2388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-2388. -- Resolution: Won't Fix Running Hive on Cygwin is not a supported configuration. In the past some people have gotten this to work by tweaking different parts of the code, but we don't test Hive on Cygwin, and consequently have no way of insuring that this configuration works. Please run Hive on Linux instead. > Facing issues while executing commands on hive shell. The system throws > following error: only on Windows Cygwin setup > - > > Key: HIVE-2388 > URL: https://issues.apache.org/jira/browse/HIVE-2388 > Project: Hive > Issue Type: Bug > Components: CLI, Query Processor >Affects Versions: 0.7.1 > Environment: Cygwin Windows >Reporter: Siddharth tiwari >Priority: Critical > Labels: patch > Fix For: 0.7.1 > > Attachments: start.sh, start.sh > > Original Estimate: 456h > Remaining Estimate: 456h > > DDL runs well but the following command describes throw error pls help with > resolution and how to get about it > hive> show tables > > ; > FAILED: Hive Internal Error: > java.lang.IllegalArgumentException(java.net.URISyntaxException: Relative path > in absolute URI: file:C:/cygwin/tmp//siddharth/hive_2011-08-18_ > 03-11-05_208_1818592223695168110) > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: > file:C:/cygwin/tmp//siddharth/hive_2011-08-18_03-11-05_208_181859222369516 > 8110 > at org.apache.hadoop.fs.Path.initialize(Path.java:140) > at org.apache.hadoop.fs.Path.(Path.java:132) > at org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:142) > at > org.apache.hadoop.hive.ql.Context.getLocalScratchDir(Context.java:168) > at > org.apache.hadoop.hive.ql.Context.getLocalTmpFileURI(Context.java:282) > at > org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:205) > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > Caused by: java.net.URISyntaxException: Relative path in absolute URI: > file:C:/cygwin/tmp//siddharth/hive_2011-08-18_03-11-05_208_1818592223695168110 > at java.net.URI.checkPath(URI.java:1787) > at java.net.URI.(URI.java:735) > at org.apache.hadoop.fs.Path.initialize(Path.java:137) > ... 16 more -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1116 - Still Failing
Changes for Build #1114 Changes for Build #1115 [cws] HIVE-2468. Make Hive compile against Hadoop 0.23 (Tom White via cws) Changes for Build #1116 3 tests failed. REGRESSION: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat Error Message: Unexpected exception See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat(TestMinimrCliDriver.java:194) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) REGRESSION: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin6 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin6(TestMinimrCliDriver.java:242) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) REGRESSION: org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 Error Message: One or more queries failed Stack Trace: junit.framework.AssertionFailedError: One or more queries failed at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1(TestMTQueries.java:37) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.an
Hive ODBC
I wanted to connect to Hive server with Hive ODBC driver (to try Tableau). I noticed there is Hive ODBC driver in Apache. Cloudera created a closed-source proprietary ODBC driver instead. Why Cloudera didn't contribute to open source? I understand Cloudera has become proprietary (SCM express is proprietary, Hive ODBC driver is proprietary, new management tools are proprietary). But in this case there was already Apache open source code for Hive ODBC driver and Cloudera chose not to contribute.
[jira] [Updated] (HIVE-2253) Merge failing of join tree in exceptional case
[ https://issues.apache.org/jira/browse/HIVE-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi updated HIVE-2253: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) +1, passed tests, committed to trunk. Thanks Navis! > Merge failing of join tree in exceptional case > -- > > Key: HIVE-2253 > URL: https://issues.apache.org/jira/browse/HIVE-2253 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.9.0 > Environment: hadoop 0.20.2, hive 0.7.0 >Reporter: Navis >Assignee: Navis >Priority: Minor > Fix For: 0.9.0 > > Attachments: HIVE-2253-0.8.0.patch, HIVE-2253.1.patch.txt > > > In some very exceptional cases, SemanticAnayzer fails to merge join tree. > Example is below. > create table a (val1 int, val2 int) > create table b (val1 int, val2 int) > create table c (val1 int, val2 int) > create table d (val1 int, val2 int) > create table e (val1 int, val2 int) > 1. all same(single) join key --> one MR, good > select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on > a.val1=d.val1 join e on a.val1=e.val1 > 2. two join keys --> expected to have two MR, but resulted to three MR > select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on > a.val1=d.val1 join e on a.val2=e.val2 > 3. by changing the join order, we could attain two MR as first-expectation. > select * from a join e on a.val2=e.val2 join c on a.val1=c.val1 join d on > a.val1=d.val1 join b on a.val1=b.val1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive ODBC driver - Cloudera not contributing
I wanted to connect to Hive server with Hive ODBC driver (I am evaluating a BI application). I noticed that the Hive source includes an ODBC driver, but that Cloudera created a closed-source proprietary ODBC driver instead. Why Cloudera didn't contribute to open source? I understand Cloudera has become proprietary in last few years (SCM express is proprietary, Hive ODBC driver is proprietary, management tools are proprietary). But in this case there was already Apache open source code for Hive ODBC driver and Cloudera chose not to contribute. Why?
[jira] [Commented] (HIVE-2619) Add hook to run in meatastore's endFunction which can collect more fb303 counters
[ https://issues.apache.org/jira/browse/HIVE-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161261#comment-13161261 ] Namit Jain commented on HIVE-2619: -- +1 > Add hook to run in meatastore's endFunction which can collect more fb303 > counters > - > > Key: HIVE-2619 > URL: https://issues.apache.org/jira/browse/HIVE-2619 > Project: Hive > Issue Type: Improvement >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2619.1.patch.txt, HIVE-2619.D555.1.patch, > HIVE-2619.D555.2.patch, HIVE-2619.D561.1.patch > > > Create the potential for hooks to run in the endFunction method of HMSHandler > which take the name of a function and whether or not it succeeded. Also, > override getCounters from fb303 to allow these hooks to add counters which > they collect, should this be desired. These hooks can be similar to > EventListeners, but they should be more generic. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2523) add a new builtins subproject
[ https://issues.apache.org/jira/browse/HIVE-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161308#comment-13161308 ] Phabricator commented on HIVE-2523: --- cwsteinbach has accepted the revision "HIVE-2523 [jira] add a new builtins subproject". +1. Will commit if tests pass. REVISION DETAIL https://reviews.facebook.net/D267 > add a new builtins subproject > - > > Key: HIVE-2523 > URL: https://issues.apache.org/jira/browse/HIVE-2523 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: John Sichi >Assignee: John Sichi > Fix For: 0.9.0 > > Attachments: D267.1.patch, D267.2.patch, HIVE-2523.D267.3.patch, > HIVE-2523.D267.4.patch > > > Now that we have a PDK, we can make it easier to add builtin functions to > Hive by putting them in a plugin which automatically gets loaded by Hive. > This issue will add the necessary framework and one example function; then > new functions can be added here, and over time we could migrate old ones here > if desired. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1117 - Still Failing
Changes for Build #1114 Changes for Build #1115 [cws] HIVE-2468. Make Hive compile against Hadoop 0.23 (Tom White via cws) Changes for Build #1116 Changes for Build #1117 [jvs] HIVE-2253. Merge failing of join tree in exceptional case (Navis Ryu via jvs) 1 tests failed. REGRESSION: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppr_pushdown Error Message: Unexpected exception See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppr_pushdown(TestCliDriver.java:25455) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1117) Status: Still Failing Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1117/ to view the results.
[jira] [Commented] (HIVE-2253) Merge failing of join tree in exceptional case
[ https://issues.apache.org/jira/browse/HIVE-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161334#comment-13161334 ] Hudson commented on HIVE-2253: -- Integrated in Hive-trunk-h0.21 #1117 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1117/]) HIVE-2253. Merge failing of join tree in exceptional case (Navis Ryu via jvs) jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1209226 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java * /hive/trunk/ql/src/test/queries/clientpositive/mergejoins.q * /hive/trunk/ql/src/test/results/clientpositive/mergejoins.q.out > Merge failing of join tree in exceptional case > -- > > Key: HIVE-2253 > URL: https://issues.apache.org/jira/browse/HIVE-2253 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.9.0 > Environment: hadoop 0.20.2, hive 0.7.0 >Reporter: Navis >Assignee: Navis >Priority: Minor > Fix For: 0.9.0 > > Attachments: HIVE-2253-0.8.0.patch, HIVE-2253.1.patch.txt > > > In some very exceptional cases, SemanticAnayzer fails to merge join tree. > Example is below. > create table a (val1 int, val2 int) > create table b (val1 int, val2 int) > create table c (val1 int, val2 int) > create table d (val1 int, val2 int) > create table e (val1 int, val2 int) > 1. all same(single) join key --> one MR, good > select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on > a.val1=d.val1 join e on a.val1=e.val1 > 2. two join keys --> expected to have two MR, but resulted to three MR > select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on > a.val1=d.val1 join e on a.val2=e.val2 > 3. by changing the join order, we could attain two MR as first-expectation. > select * from a join e on a.val2=e.val2 join c on a.val1=c.val1 join d on > a.val1=d.val1 join b on a.val1=b.val1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2391) published POMs in Maven repo are incorrect
[ https://issues.apache.org/jira/browse/HIVE-2391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161361#comment-13161361 ] Mithun Radhakrishnan commented on HIVE-2391: The currently published pom.xml seems to indicate the following: (from https://repository.apache.org/content/groups/snapshots/org/apache/hive/hive-metastore/0.8.0-SNAPSHOT/hive-metastore-0.8.0-2030.194742-149.pom) hadoop core 0.20.1 compile Could the pom please be corrected to reference the hadoop-core jar correctly? (Unless of course I'm missing something, in which case, may I please know which repository this artifact might be pulled from?) > published POMs in Maven repo are incorrect > -- > > Key: HIVE-2391 > URL: https://issues.apache.org/jira/browse/HIVE-2391 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 0.7.1 >Reporter: Alejandro Abdelnur >Assignee: Carl Steinbach >Priority: Critical > Fix For: 0.8.0 > > Attachments: HIVE-2391.1.patch.txt, HIVE-2391.2.patch.txt, > HIVE-2391.3.patch.txt, HIVE-2391.4.patch.txt, HIVE-2391.5.patch.txt, > HIVE-2391.wip.1.patch.txt > > > The Hive artifacts published in Apache Maven SNAPSHOTS repo are incorrect. > Dependencies are not complete. > Even after adding as dependencies ALL the Hive artifacts it is not possible > to compile a project using Hive JARs (I'm trying to integrate Oozie Hive > Action using Apache Hive). > As a reference the Hive CDH POMs dependencies could be used (Using those > artifacts I'm able to compile/test/run Hive from within Oozie). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2621) Allow multiple group bys with the same input data and spray keys to be run on the same reducer.
Allow multiple group bys with the same input data and spray keys to be run on the same reducer. --- Key: HIVE-2621 URL: https://issues.apache.org/jira/browse/HIVE-2621 Project: Hive Issue Type: New Feature Reporter: Kevin Wilfong Assignee: Kevin Wilfong Currently, when a user runs a query, such as a multi-insert, where each insertion subclause consists of a simple query followed by a group by, the group bys for each clause are run on a separate reducer. This requires writing the data for each group by clause to an intermediate file, and then reading it back. This uses a significant amount of the total CPU consumed by the query for an otherwise simple query. If the subclauses are grouped by their distinct expressions and group by keys, with all of the group by expressions for a group of subclauses run on a single reducer, this would reduce the amount of reading/writing to intermediate files for some queries. To do this, for each group of subclauses, in the mapper we would execute a the filters for each subclause 'or'd together (provided each subclause has a filter) followed by a reduce sink. In the reducer, the child operators would be each subclauses filter followed by the group by and any subsequent operations. Note that this would require turning off map aggregation, so we would need to make using this type of plan configurable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2622) Hive POMs reference the wrong Hadoop artifacts
Hive POMs reference the wrong Hadoop artifacts -- Key: HIVE-2622 URL: https://issues.apache.org/jira/browse/HIVE-2622 Project: Hive Issue Type: Bug Components: Build Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Fix For: 0.8.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2622) Hive POMs reference the wrong Hadoop artifacts
[
https://issues.apache.org/jira/browse/HIVE-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161364#comment-13161364
]
Carl Steinbach commented on HIVE-2622:
--
Comment from HIVE-2391:
{quote}
The currently published pom.xml seems to indicate the following:
(from
https://repository.apache.org/content/groups/snapshots/org/apache/hive/hive-metastore/0.8.0-SNAPSHOT/hive-metastore-0.8.0-2030.194742-149.pom)
hadoop
core
0.20.1
compile
Could the pom please be corrected to reference the hadoop-core jar correctly?
(Unless of course I'm missing something, in which case, may I please know which
repository this artifact might be pulled from?)
{quote}
> Hive POMs reference the wrong Hadoop artifacts
> --
>
> Key: HIVE-2622
> URL: https://issues.apache.org/jira/browse/HIVE-2622
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Fix For: 0.8.0
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2391) published POMs in Maven repo are incorrect
[ https://issues.apache.org/jira/browse/HIVE-2391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161365#comment-13161365 ] Carl Steinbach commented on HIVE-2391: -- @Mithun: Thanks for pointing this out. I filed HIVE-2622 to track this issue. > published POMs in Maven repo are incorrect > -- > > Key: HIVE-2391 > URL: https://issues.apache.org/jira/browse/HIVE-2391 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 0.7.1 >Reporter: Alejandro Abdelnur >Assignee: Carl Steinbach >Priority: Critical > Fix For: 0.8.0 > > Attachments: HIVE-2391.1.patch.txt, HIVE-2391.2.patch.txt, > HIVE-2391.3.patch.txt, HIVE-2391.4.patch.txt, HIVE-2391.5.patch.txt, > HIVE-2391.wip.1.patch.txt > > > The Hive artifacts published in Apache Maven SNAPSHOTS repo are incorrect. > Dependencies are not complete. > Even after adding as dependencies ALL the Hive artifacts it is not possible > to compile a project using Hive JARs (I'm trying to integrate Oozie Hive > Action using Apache Hive). > As a reference the Hive CDH POMs dependencies could be used (Using those > artifacts I'm able to compile/test/run Hive from within Oozie). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2621) Allow multiple group bys with the same input data and spray keys to be run on the same reducer.
[ https://issues.apache.org/jira/browse/HIVE-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-2621: Attachment: HIVE-2621.1.patch.txt > Allow multiple group bys with the same input data and spray keys to be run on > the same reducer. > --- > > Key: HIVE-2621 > URL: https://issues.apache.org/jira/browse/HIVE-2621 > Project: Hive > Issue Type: New Feature >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2621.1.patch.txt, HIVE-2621.D567.1.patch > > > Currently, when a user runs a query, such as a multi-insert, where each > insertion subclause consists of a simple query followed by a group by, the > group bys for each clause are run on a separate reducer. This requires > writing the data for each group by clause to an intermediate file, and then > reading it back. This uses a significant amount of the total CPU consumed by > the query for an otherwise simple query. > If the subclauses are grouped by their distinct expressions and group by > keys, with all of the group by expressions for a group of subclauses run on a > single reducer, this would reduce the amount of reading/writing to > intermediate files for some queries. > To do this, for each group of subclauses, in the mapper we would execute a > the filters for each subclause 'or'd together (provided each subclause has a > filter) followed by a reduce sink. In the reducer, the child operators would > be each subclauses filter followed by the group by and any subsequent > operations. > Note that this would require turning off map aggregation, so we would need to > make using this type of plan configurable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2621) Allow multiple group bys with the same input data and spray keys to be run on the same reducer.
[ https://issues.apache.org/jira/browse/HIVE-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2621: -- Attachment: HIVE-2621.D567.1.patch kevinwilfong requested code review of "HIVE-2621 [jira] Allow multiple group bys with the same input data and spray keys to be run on the same reducer.". Reviewers: JIRA The meaningful changes are all in how the plan is generated. If the conf variable has been set, the subclauses are first grouped by their group by keys and distinct keys. To facilitate this I added a wrapper class to ExprNodeDesc which makes equals like the isSame method. If the conf variable is not set, I create a single group of all the subqueries. Then, provided certain conditions are met, e.g. the conf variable is set, there is a group by and there are aggregations, the skew conf variable hasn't been set, I create the new plan for each group, otherwise the old plan is produced. To start I generate the common filter by 'or'ing the group's clauses' filters. This goes into a select operator, which goes into a new reduce operator. The reduce operator is like the typical 1 MR group by reduce operator, except that to generate the reduce values it loops over each of the group's subclauses' aggregations and the columns used in the where clauses. This goes into a forward operator and for each subclause the forward operator has a child filter operator, if the subclause has a filter, and a group by operator. Each group by operator is followed by the operators which would normally follow it in a plan. TEST PLAN I added some unit tests. I verified these unit tests and the old unit tests all passed. I created a sample query which consisted of a multi-insert from a table with 1,000,000 rows, going into 6 tables, each of which's subclause consisted of a group by, and a count distinct, as well as some other aggregations and havings. The subclauses were constructed such that they could be grouped into two reducers using the new plan. I also ensured that the data was such that map aggregation was turned of early using the existing plan. I verified that this query saw a significant improvement in its CPU usage. REVISION DETAIL https://reviews.facebook.net/D567 AFFECTED FILES conf/hive-default.xml common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ql/src/test/results/clientpositive/groupby7_noskew_multi_single_reducer.q.out ql/src/test/results/clientpositive/groupby_multi_single_reducer.q.out ql/src/test/results/clientpositive/groupby_complex_types_multi_single_reducer.q.out ql/src/test/queries/clientpositive/groupby_multi_single_reducer.q ql/src/test/queries/clientpositive/groupby7_noskew_multi_single_reducer.q ql/src/test/queries/clientpositive/groupby_complex_types_multi_single_reducer.q ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDesc.java ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/1269/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. > Allow multiple group bys with the same input data and spray keys to be run on > the same reducer. > --- > > Key: HIVE-2621 > URL: https://issues.apache.org/jira/browse/HIVE-2621 > Project: Hive > Issue Type: New Feature >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2621.1.patch.txt, HIVE-2621.D567.1.patch > > > Currently, when a user runs a query, such as a multi-insert, where each > insertion subclause consists of a simple query followed by a group by, the > group bys for each clause are run on a separate reducer. This requires > writing the data for each group by clause to an intermediate file, and then > reading it back. This uses a significant amount of the total CPU consumed by > the query for an otherwise simple query. > If the subclauses are grouped by their distinct expressions and group by > keys, with all of the group by expressions for a group of subclauses run on a > single reducer, this would reduce the amount of reading/writing to > intermediate files for some queries. > To do this, for each group of subclauses, in the mapper we would execute a > the filters for each subclause 'or'd together (provided each subclause has a > filter) followed by a reduce sink. In the reducer, the child operators would > be each subclauses filter followed by the group by and any subsequent > operations. > Note that this would require turning off map aggregation, so we would need to > make using this type of plan configurable. -- This message is automatically g
[jira] [Commented] (HIVE-2621) Allow multiple group bys with the same input data and spray keys to be run on the same reducer.
[ https://issues.apache.org/jira/browse/HIVE-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161369#comment-13161369 ] Kevin Wilfong commented on HIVE-2621: - diff is here https://reviews.facebook.net/D567 > Allow multiple group bys with the same input data and spray keys to be run on > the same reducer. > --- > > Key: HIVE-2621 > URL: https://issues.apache.org/jira/browse/HIVE-2621 > Project: Hive > Issue Type: New Feature >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2621.1.patch.txt, HIVE-2621.D567.1.patch > > > Currently, when a user runs a query, such as a multi-insert, where each > insertion subclause consists of a simple query followed by a group by, the > group bys for each clause are run on a separate reducer. This requires > writing the data for each group by clause to an intermediate file, and then > reading it back. This uses a significant amount of the total CPU consumed by > the query for an otherwise simple query. > If the subclauses are grouped by their distinct expressions and group by > keys, with all of the group by expressions for a group of subclauses run on a > single reducer, this would reduce the amount of reading/writing to > intermediate files for some queries. > To do this, for each group of subclauses, in the mapper we would execute a > the filters for each subclause 'or'd together (provided each subclause has a > filter) followed by a reduce sink. In the reducer, the child operators would > be each subclauses filter followed by the group by and any subsequent > operations. > Note that this would require turning off map aggregation, so we would need to > make using this type of plan configurable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2621) Allow multiple group bys with the same input data and spray keys to be run on the same reducer.
[ https://issues.apache.org/jira/browse/HIVE-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161371#comment-13161371 ] Phabricator commented on HIVE-2621: --- kevinwilfong has commented on the revision "HIVE-2621 [jira] Allow multiple group bys with the same input data and spray keys to be run on the same reducer.". INLINE COMMENTS ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:6296 The code in this method should be the same as what followed the code to generate a group by plan in the existing code. The diff just didn't seem to match them up. ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:5693 The string was not being used in this method. REVISION DETAIL https://reviews.facebook.net/D567 > Allow multiple group bys with the same input data and spray keys to be run on > the same reducer. > --- > > Key: HIVE-2621 > URL: https://issues.apache.org/jira/browse/HIVE-2621 > Project: Hive > Issue Type: New Feature >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2621.1.patch.txt, HIVE-2621.D567.1.patch > > > Currently, when a user runs a query, such as a multi-insert, where each > insertion subclause consists of a simple query followed by a group by, the > group bys for each clause are run on a separate reducer. This requires > writing the data for each group by clause to an intermediate file, and then > reading it back. This uses a significant amount of the total CPU consumed by > the query for an otherwise simple query. > If the subclauses are grouped by their distinct expressions and group by > keys, with all of the group by expressions for a group of subclauses run on a > single reducer, this would reduce the amount of reading/writing to > intermediate files for some queries. > To do this, for each group of subclauses, in the mapper we would execute a > the filters for each subclause 'or'd together (provided each subclause has a > filter) followed by a reduce sink. In the reducer, the child operators would > be each subclauses filter followed by the group by and any subsequent > operations. > Note that this would require turning off map aggregation, so we would need to > make using this type of plan configurable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1040) use sed rather than diff for masking out noise in diff-based tests
[ https://issues.apache.org/jira/browse/HIVE-1040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161438#comment-13161438 ] Marek Sapota commented on HIVE-1040: There is a small problem with masking - some of the tests generate lines in random order, `-I` completely removes the ignored lines, so masked some valid line some valid line masked are the same to `diff -I` but will differ when using plain `diff`. We could do several things to make it work: - use `diff -I masked` (only full line masking then), for some reason it fails, probably because `diff` claims "some valid line" are the lines not matching and doesn't use the `-I` switch, man says "Ignore changes whose lines all match RE", anyone knows if this really means make a diff and remove changes matching RE? I expected it to be apply `-I` first and then do the diff. - remove a line if the whole line was masked (masking inside a line possible but would be hard to tell what was removed from the output file) - if `diff -I '^masked$'` works it could be a win, but has the same problem as above For example TestNegativeCliDriver create_view_failure2.q has this problem. Ideas? > use sed rather than diff for masking out noise in diff-based tests > -- > > Key: HIVE-1040 > URL: https://issues.apache.org/jira/browse/HIVE-1040 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Affects Versions: 0.4.1 >Reporter: John Sichi >Assignee: Marek Sapota >Priority: Minor > > The current diff -I approach has two problems: (1) it does not allow > resolution finer than line-level, so it's impossible to mask out pattern > occurrences within a line, and (2) it produces unmasked files, so if you run > diff on the command line to compare the result .q.out with the checked-in > file, you see the noise. > My suggestion is to first run sed to replace noise patterns with an > unlikely-to-occur string like ZYZZYZVA, and then diff the pre-masked files > without using any -I. > This would require a one-time hit to update all existing .q.out files so that > they would contain the pre-masked results. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2619) Add hook to run in metastore's endFunction which can collect more fb303 counters
[ https://issues.apache.org/jira/browse/HIVE-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain resolved HIVE-2619. -- Resolution: Fixed Hadoop Flags: Reviewed Committed. Thanks Kevin > Add hook to run in metastore's endFunction which can collect more fb303 > counters > > > Key: HIVE-2619 > URL: https://issues.apache.org/jira/browse/HIVE-2619 > Project: Hive > Issue Type: Improvement >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2619.1.patch.txt, HIVE-2619.D555.1.patch, > HIVE-2619.D555.2.patch, HIVE-2619.D561.1.patch > > > Create the potential for hooks to run in the endFunction method of HMSHandler > which take the name of a function and whether or not it succeeded. Also, > override getCounters from fb303 to allow these hooks to add counters which > they collect, should this be desired. These hooks can be similar to > EventListeners, but they should be more generic. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2619) Add hook to run in metastore's endFunction which can collect more fb303 counters
[ https://issues.apache.org/jira/browse/HIVE-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2619: - Summary: Add hook to run in metastore's endFunction which can collect more fb303 counters (was: Add hook to run in meatastore's endFunction which can collect more fb303 counters) > Add hook to run in metastore's endFunction which can collect more fb303 > counters > > > Key: HIVE-2619 > URL: https://issues.apache.org/jira/browse/HIVE-2619 > Project: Hive > Issue Type: Improvement >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2619.1.patch.txt, HIVE-2619.D555.1.patch, > HIVE-2619.D555.2.patch, HIVE-2619.D561.1.patch > > > Create the potential for hooks to run in the endFunction method of HMSHandler > which take the name of a function and whether or not it succeeded. Also, > override getCounters from fb303 to allow these hooks to add counters which > they collect, should this be desired. These hooks can be similar to > EventListeners, but they should be more generic. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Problem in running Hive Queries
Hi, I am new to Hive. I have configured Hadoop and Hive on Windows 7 through Cygwin. Currently I am running hive queries on Hive CLI. I want to ask question that: When I enter the hive queries I get the following errors as: Query: bin/hive -e "insert overwrite table pokes select a.* from invites a where a.ds='2008-08-15';" Errors: bin/hive -e "insert overwrite table pokes select a.* from invites a where a.ds='2008-08-15';" Hive history file=/tmp/Bhavesh.Shah/hive_job_log_Bhavesh.Shah_201112021007_2120318983.txt Total MapReduce jobs = 2 Launching Job 1 out of 2 Number of reduce tasks is set to 0 since there's no reduce operator Starting Job = job_201112011620_0004, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201112011620_0004 Kill Command = C:\cygwin\home\Bhavesh.Shah\hadoop-0.20.2\/bin/hadoop job -Dmapred.job.tracker=localhost:9101 -kill job_201112011620_0004 2011-12-02 10:07:30,777 Stage-1 map = 0%, reduce = 0% 2011-12-02 10:07:57,796 Stage-1 map = 100%, reduce = 100% Ended Job = job_201112011620_0004 with errors FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask So what could be the solution? Pls suggest me. Thanks and Regards, Bhavesh Shah.
Re: Problem in running Hive Queries
you should open /tmp/Bhavesh.Shah/hive_job_log_Bhavesh.Shah_201112021007_2120318983.txt. That will give you more hints. Paste some lines from there on this thread. Thanks, Aniket On Thu, Dec 1, 2011 at 10:11 PM, Bhavesh Shah wrote: > Hi, > > I am new to Hive. I have configured Hadoop and Hive on Windows 7 through > Cygwin. > > Currently I am running hive queries on Hive CLI. > > I want to ask question that: > > When I enter the hive queries I get the following errors as: > > Query: > > bin/hive -e "insert overwrite table pokes select a.* from invites a where > a.ds='2008-08-15';" > > > > Errors: > > bin/hive -e "insert overwrite table pokes select a.* from invites a where > a.ds='2008-08-15';" > > Hive history > > file=/tmp/Bhavesh.Shah/hive_job_log_Bhavesh.Shah_201112021007_2120318983.txt > > Total MapReduce jobs = 2 > > Launching Job 1 out of 2 > > Number of reduce tasks is set to 0 since there's no reduce operator > > Starting Job = job_201112011620_0004, Tracking URL = > http://localhost:50030/jobdetails.jsp?jobid=job_201112011620_0004 > > Kill Command = C:\cygwin\home\Bhavesh.Shah\hadoop-0.20.2\/bin/hadoop job > -Dmapred.job.tracker=localhost:9101 -kill job_201112011620_0004 > > 2011-12-02 10:07:30,777 Stage-1 map = 0%, reduce = 0% > > 2011-12-02 10:07:57,796 Stage-1 map = 100%, reduce = 100% > > Ended Job = job_201112011620_0004 with errors > > FAILED: Execution Error, return code 2 from > org.apache.hadoop.hive.ql.exec.MapRedTask > > > > > > So what could be the solution? Pls suggest me. > > > > > > Thanks and Regards, > > Bhavesh Shah. > > -- "...:::Aniket:::... Quetzalco@tl"
RE: Problem in running Hive Queries
Hi,
Tthe content of file is too large, that's why I am attaching file.
Pls check it out.
Thanks.
-Original Message-
From: Aniket Mokashi [mailto:aniket...@gmail.com]
Sent: Friday, December 02, 2011 11:46 AM
To: dev@hive.apache.org
Subject: Re: Problem in running Hive Queries
you should open
/tmp/Bhavesh.Shah/hive_job_log_Bhavesh.Shah_201112021007_2120318983.txt.
That will give you more hints. Paste some lines from there on this thread.
Thanks,
Aniket
On Thu, Dec 1, 2011 at 10:11 PM, Bhavesh Shah
wrote:
> Hi,
>
> I am new to Hive. I have configured Hadoop and Hive on Windows 7
> through Cygwin.
>
> Currently I am running hive queries on Hive CLI.
>
> I want to ask question that:
>
> When I enter the hive queries I get the following errors as:
>
> Query:
>
> bin/hive -e "insert overwrite table pokes select a.* from invites a
> where a.ds='2008-08-15';"
>
>
>
> Errors:
>
> bin/hive -e "insert overwrite table pokes select a.* from invites a
> where a.ds='2008-08-15';"
>
> Hive history
>
> file=/tmp/Bhavesh.Shah/hive_job_log_Bhavesh.Shah_201112021007_21203189
> 83.txt
>
> Total MapReduce jobs = 2
>
> Launching Job 1 out of 2
>
> Number of reduce tasks is set to 0 since there's no reduce operator
>
> Starting Job = job_201112011620_0004, Tracking URL =
> http://localhost:50030/jobdetails.jsp?jobid=job_201112011620_0004
>
> Kill Command = C:\cygwin\home\Bhavesh.Shah\hadoop-0.20.2\/bin/hadoop
> job
> -Dmapred.job.tracker=localhost:9101 -kill job_201112011620_0004
>
> 2011-12-02 10:07:30,777 Stage-1 map = 0%, reduce = 0%
>
> 2011-12-02 10:07:57,796 Stage-1 map = 100%, reduce = 100%
>
> Ended Job = job_201112011620_0004 with errors
>
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.MapRedTask
>
>
>
>
>
> So what could be the solution? Pls suggest me.
>
>
>
>
>
> Thanks and Regards,
>
> Bhavesh Shah.
>
>
--
"...:::Aniket:::... Quetzalco@tl"
SessionStart SESSION_ID="Bhavesh.Shah_201112021007" TIME="1322800632978"
QueryStart QUERY_STRING="insert overwrite table pokes select a.* from invites a
where a.ds='2008-08-15'"
QUERY_ID="Bhavesh.Shah_20111202100707_87687ccb-9acf-4089-a391-0a56c4e04dd0"
TIME="1322800636979"
Counters
plan="{"queryId":"Bhavesh.Shah_20111202100707_87687ccb-9acf-4089-a391-0a56c4e04dd0","queryType":null,"queryAttributes":{"queryString":"insert
overwrite table pokes select a.* from invites a where
a.ds='2008-08-15'"},"queryCounters":"null","stageGraph":{"nodeType":"STAGE","roots":"null","adjacencyList":[{"node":"Stage-1","children":["Stage-5"],"adjacencyType":"CONJUNCTIVE"},{"node":"Stage-5","children":["Stage-4","Stage-3"],"adjacencyType":"DISJUNCTIVE"},{"node":"Stage-4","children":["Stage-0"],"adjacencyType":"CONJUNCTIVE"},{"node":"Stage-3","children":["Stage-0"],"adjacencyType":"CONJUNCTIVE"},{"node":"Stage-0","children":["Stage-2"],"adjacencyType":"CONJUNCTIVE"},{"node":"Stage-0","children":["Stage-2"],"adjacencyType":"CONJUNCTIVE"}]},"stageList":[{"stageId":"Stage-1","stageType":"MAPRED","stageAttributes":"null","stageCounters":"}","taskList":[{"taskId":"Stage-1_MAP","taskType":"MAP","taskAttributes":"null","taskCounters":"null","operatorGraph":{"nodeType":"OPERATOR","roots":"null","adjacencyList":[{"node":"TS_0","children":["FIL_1"],"adjacencyType":"CONJUNCTIVE"},{"node":"FIL_1","children":["SEL_2"],"adjacencyType":"CONJUNCTIVE"},{"node":"SEL_2","children":["SEL_3"],"adjacencyType":"CONJUNCTIVE"},{"node":"SEL_3","children":["FS_4"],"adjacencyType":"CONJUNCTIVE"}]},"operatorList":[{"operatorId":"TS_0","operatorType":"TABLESCAN","operatorAttributes":"null","operatorCounters":"null","done":"false","started":"false"},{"operatorId":"FIL_1","operatorType":"FILTER","operatorAttributes":"null","operatorCounters":"null","done":"false","started":"false"},{"operatorId":"SEL_2","operatorType":"SELECT","operatorAttributes":"null","operatorCounters":"null","done":"false","started":"false"},{"operatorId":"SEL_3","operatorType":"SELECT","operatorAttributes":"null","operatorCounters":"null","done":"false","started":"false"},{"operatorId":"FS_4","operatorType":"FILESINK","operatorAttributes":"null","operatorCounters":"null","done":"false","started":"false"}],"done":"false","started":"false"}],"done":"false","started":"false"},{"stageId":"Stage-5","stageType":"CONDITIONAL","stageAttributes":"null","stageCounters":"}","taskList":[{"taskId":"Stage-5_OTHER","taskType":"OTHER","taskAttributes":"null","taskCounters":"null","operatorGraph":"null","operatorList":"]","done":"false","started":"false"}],"done":"false","started":"false"},{"stageId":"Stage-4","stageType":"MOVE","stageAttributes":"null","stageCounters":"}","taskList":[{"taskId":"Stage-4_OTHER","taskType":"OTHER","taskAttributes":"null","taskCounters":"null","operatorGraph":"null","operatorList":"]","done":"false","started":"false"}],"done":"false","started":"false"},{"stageId":"Stage-3","stageType":"MAPRED","stageAttributes":"null","stageCounters":"}","taskList":[{"taskId":"Stage-3_MAP","taskType":"MAP",
Re: Problem in running Hive Queries
I guess this is not the right file. You should look into hive app logs.. You can paste the full exception stack. Thanks, Aniket On Thu, Dec 1, 2011 at 10:22 PM, Bhavesh Shah wrote: > Hi, > Tthe content of file is too large, that's why I am attaching file. > Pls check it out. > Thanks. > > -Original Message- > From: Aniket Mokashi [mailto:aniket...@gmail.com] > Sent: Friday, December 02, 2011 11:46 AM > To: dev@hive.apache.org > Subject: Re: Problem in running Hive Queries > > you should open > /tmp/Bhavesh.Shah/hive_job_log_Bhavesh.Shah_201112021007_2120318983.txt. > That will give you more hints. Paste some lines from there on this thread. > > Thanks, > Aniket > > On Thu, Dec 1, 2011 at 10:11 PM, Bhavesh Shah > wrote: > > > Hi, > > > > I am new to Hive. I have configured Hadoop and Hive on Windows 7 > > through Cygwin. > > > > Currently I am running hive queries on Hive CLI. > > > > I want to ask question that: > > > > When I enter the hive queries I get the following errors as: > > > > Query: > > > > bin/hive -e "insert overwrite table pokes select a.* from invites a > > where a.ds='2008-08-15';" > > > > > > > > Errors: > > > > bin/hive -e "insert overwrite table pokes select a.* from invites a > > where a.ds='2008-08-15';" > > > > Hive history > > > > file=/tmp/Bhavesh.Shah/hive_job_log_Bhavesh.Shah_201112021007_21203189 > > 83.txt > > > > Total MapReduce jobs = 2 > > > > Launching Job 1 out of 2 > > > > Number of reduce tasks is set to 0 since there's no reduce operator > > > > Starting Job = job_201112011620_0004, Tracking URL = > > http://localhost:50030/jobdetails.jsp?jobid=job_201112011620_0004 > > > > Kill Command = C:\cygwin\home\Bhavesh.Shah\hadoop-0.20.2\/bin/hadoop > > job > > -Dmapred.job.tracker=localhost:9101 -kill job_201112011620_0004 > > > > 2011-12-02 10:07:30,777 Stage-1 map = 0%, reduce = 0% > > > > 2011-12-02 10:07:57,796 Stage-1 map = 100%, reduce = 100% > > > > Ended Job = job_201112011620_0004 with errors > > > > FAILED: Execution Error, return code 2 from > > org.apache.hadoop.hive.ql.exec.MapRedTask > > > > > > > > > > > > So what could be the solution? Pls suggest me. > > > > > > > > > > > > Thanks and Regards, > > > > Bhavesh Shah. > > > > > > > -- > "...:::Aniket:::... Quetzalco@tl" > -- "...:::Aniket:::... Quetzalco@tl"
Re: Problem in running Hive Queries
Then where should I get that log file? I am not getting. On Fri, Dec 2, 2011 at 11:54 AM, Aniket Mokashi wrote: > I guess this is not the right file. You should look into hive app logs.. > You can paste the full exception stack. > > Thanks, > Aniket > > On Thu, Dec 1, 2011 at 10:22 PM, Bhavesh Shah wrote: > >> Hi, >> Tthe content of file is too large, that's why I am attaching file. >> Pls check it out. >> Thanks. >> >> -Original Message- >> From: Aniket Mokashi [mailto:aniket...@gmail.com] >> Sent: Friday, December 02, 2011 11:46 AM >> To: dev@hive.apache.org >> Subject: Re: Problem in running Hive Queries >> >> you should open >> /tmp/Bhavesh.Shah/hive_job_log_Bhavesh.Shah_201112021007_2120318983.txt. >> That will give you more hints. Paste some lines from there on this thread. >> >> Thanks, >> Aniket >> >> On Thu, Dec 1, 2011 at 10:11 PM, Bhavesh Shah >> wrote: >> >> > Hi, >> > >> > I am new to Hive. I have configured Hadoop and Hive on Windows 7 >> > through Cygwin. >> > >> > Currently I am running hive queries on Hive CLI. >> > >> > I want to ask question that: >> > >> > When I enter the hive queries I get the following errors as: >> > >> > Query: >> > >> > bin/hive -e "insert overwrite table pokes select a.* from invites a >> > where a.ds='2008-08-15';" >> > >> > >> > >> > Errors: >> > >> > bin/hive -e "insert overwrite table pokes select a.* from invites a >> > where a.ds='2008-08-15';" >> > >> > Hive history >> > >> > file=/tmp/Bhavesh.Shah/hive_job_log_Bhavesh.Shah_201112021007_21203189 >> > 83.txt >> > >> > Total MapReduce jobs = 2 >> > >> > Launching Job 1 out of 2 >> > >> > Number of reduce tasks is set to 0 since there's no reduce operator >> > >> > Starting Job = job_201112011620_0004, Tracking URL = >> > http://localhost:50030/jobdetails.jsp?jobid=job_201112011620_0004 >> > >> > Kill Command = C:\cygwin\home\Bhavesh.Shah\hadoop-0.20.2\/bin/hadoop >> > job >> > -Dmapred.job.tracker=localhost:9101 -kill job_201112011620_0004 >> > >> > 2011-12-02 10:07:30,777 Stage-1 map = 0%, reduce = 0% >> > >> > 2011-12-02 10:07:57,796 Stage-1 map = 100%, reduce = 100% >> > >> > Ended Job = job_201112011620_0004 with errors >> > >> > FAILED: Execution Error, return code 2 from >> > org.apache.hadoop.hive.ql.exec.MapRedTask >> > >> > >> > >> > >> > >> > So what could be the solution? Pls suggest me. >> > >> > >> > >> > >> > >> > Thanks and Regards, >> > >> > Bhavesh Shah. >> > >> > >> >> >> -- >> "...:::Aniket:::... Quetzalco@tl" >> > > > > -- > "...:::Aniket:::... Quetzalco@tl" -- Regards, Bhavesh Shah
Re: Problem in running Hive Queries
Hi Aniket, I just got it and pasted below: 2011-12-02 12:10:20,379 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires "org.eclipse.core.resources" but it cannot be resolved. 2011-12-02 12:10:20,379 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires "org.eclipse.core.resources" but it cannot be resolved. 2011-12-02 12:10:20,395 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires "org.eclipse.core.runtime" but it cannot be resolved. 2011-12-02 12:10:20,395 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires "org.eclipse.core.runtime" but it cannot be resolved. 2011-12-02 12:10:20,395 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires "org.eclipse.text" but it cannot be resolved. 2011-12-02 12:10:20,395 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires "org.eclipse.text" but it cannot be resolved. 2011-12-02 12:10:23,339 ERROR ql.Driver (SessionState.java:printError(343)) - FAILED: Error in semantic analysis: Line 1:23 Cannot insert into target table because column number/types are different pokes: Table insclause-0 has 3 columns, but query has 1 columns. org.apache.hadoop.hive.ql.parse.SemanticException: Line 1:23 Cannot insert into target table because column number/types are different pokes: Table insclause-0 has 3 columns, but query has 1 columns. at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genConversionSelectOperator(SemanticAnalyzer.java:3859) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(SemanticAnalyzer.java:3763) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:5480) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6016) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:6601) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:425) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) 2011-12-02 12:10:45,159 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires "org.eclipse.core.resources" but it cannot be resolved. 2011-12-02 12:10:45,159 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires "org.eclipse.core.resources" but it cannot be resolved. 2011-12-02 12:10:45,159 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires "org.eclipse.core.runtime" but it cannot be resolved. 2011-12-02 12:10:45,159 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires "org.eclipse.core.runtime" but it cannot be resolved. 2011-12-02 12:10:45,159 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires "org.eclipse.text" but it cannot be resolved. 2011-12-02 12:10:45,159 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires "org.eclipse.text" but it cannot be resolved. 2011-12-02 12:10:48,820 WARN mapred.JobClient (JobClient.java:configureCommandLineOptions(539)) - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 2011-12-02 12:11:23,287 ERROR exec.MapRedTask (SessionState.java:printError(343)) - Ended Job = job_201112011620_0005 with errors 2011-12-02 12:11:23,739 ERROR ql.Driver (SessionState.java:printError(343)) - FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask On Fri, Dec 2, 2011 at 11:54 AM, Aniket Mokashi wrote: > I guess this is not the right file. You should look into hive app logs.. > You can paste the full exception stack. > > Thanks, > Aniket > > On Thu, Dec 1, 2011 at 10:22 PM, Bhavesh Shah wrote: > >> Hi, >> Tthe content of file is too large, that's why I am attaching file. >> Pls check it out. >> Thanks. >> >> -Original Message- >> From: Aniket Mokashi [mailto:aniket...@gmail.com] >> Sent: Friday, December 02, 2011 11:46 AM >> To: dev@hive.apache.org >> Subject: Re: Problem in running H
Hive-trunk-h0.21 - Build # 1118 - Still Failing
Changes for Build #1114 Changes for Build #1115 [cws] HIVE-2468. Make Hive compile against Hadoop 0.23 (Tom White via cws) Changes for Build #1116 Changes for Build #1117 [jvs] HIVE-2253. Merge failing of join tree in exceptional case (Navis Ryu via jvs) Changes for Build #1118 [namit] HIVE-2619 Add hook to run in metastore's endFunction which can collect more fb303 counters (Kevin Wilfong via namit) No tests ran. The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1118) Status: Still Failing Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1118/ to view the results.
[jira] [Commented] (HIVE-2619) Add hook to run in metastore's endFunction which can collect more fb303 counters
[ https://issues.apache.org/jira/browse/HIVE-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161453#comment-13161453 ] Hudson commented on HIVE-2619: -- Integrated in Hive-trunk-h0.21 #1118 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1118/]) HIVE-2619 Add hook to run in metastore's endFunction which can collect more fb303 counters (Kevin Wilfong via namit) namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1209368 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/conf/hive-default.xml * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEndFunctionContext.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEndFunctionListener.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java > Add hook to run in metastore's endFunction which can collect more fb303 > counters > > > Key: HIVE-2619 > URL: https://issues.apache.org/jira/browse/HIVE-2619 > Project: Hive > Issue Type: Improvement >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2619.1.patch.txt, HIVE-2619.D555.1.patch, > HIVE-2619.D555.2.patch, HIVE-2619.D561.1.patch > > > Create the potential for hooks to run in the endFunction method of HMSHandler > which take the name of a function and whether or not it succeeded. Also, > override getCounters from fb303 to allow these hooks to add counters which > they collect, should this be desired. These hooks can be similar to > EventListeners, but they should be more generic. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2602) add support for insert partition overwrite(...) if not exists
[ https://issues.apache.org/jira/browse/HIVE-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161457#comment-13161457 ] Namit Jain commented on HIVE-2602: -- https://reviews.facebook.net/D579 Thanks Chinna. I will take a look > add support for insert partition overwrite(...) if not exists > - > > Key: HIVE-2602 > URL: https://issues.apache.org/jira/browse/HIVE-2602 > Project: Hive > Issue Type: New Feature >Reporter: Namit Jain >Assignee: Chinna Rao Lalam > Attachments: HIVE-2602.D579.1.patch, HIVE-2602.patch > > > INSERT OVERWRITE TABLE X PARTITION (a=b, c=d) IF NOT EXISTS ... > The partition should be created and written if and only if it's not there > already. > The support can be added for dynamic partitions in the future, but this jira > is for adding this support for static partitions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2602) add support for insert partition overwrite(...) if not exists
[ https://issues.apache.org/jira/browse/HIVE-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2602: -- Attachment: HIVE-2602.D579.1.patch njain requested code review of "HIVE-2602 [jira] add support for insert partition overwrite(...) if not exists". Reviewers: JIRA HIVE-2602 INSERT OVERWRITE TABLE X PARTITION (a=b, c=d) IF NOT EXISTS ... The partition should be created and written if and only if it's not there already. The support can be added for dynamic partitions in the future, but this jira is for adding this support for static partitions. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D579 AFFECTED FILES ql/src/test/results/clientpositive/insert1_overwrite_partitions.q.out ql/src/test/queries/clientpositive/insert1_overwrite_partitions.q ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/1281/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. > add support for insert partition overwrite(...) if not exists > - > > Key: HIVE-2602 > URL: https://issues.apache.org/jira/browse/HIVE-2602 > Project: Hive > Issue Type: New Feature >Reporter: Namit Jain >Assignee: Chinna Rao Lalam > Attachments: HIVE-2602.D579.1.patch, HIVE-2602.patch > > > INSERT OVERWRITE TABLE X PARTITION (a=b, c=d) IF NOT EXISTS ... > The partition should be created and written if and only if it's not there > already. > The support can be added for dynamic partitions in the future, but this jira > is for adding this support for static partitions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
