Re: [ANNOUNCE] New Hive Committer - Eugene Koifman
Congrats, Eugene! On Fri, Sep 12, 2014 at 3:35 PM, Carl Steinbach wrote: > The Apache Hive PMC has voted to make Eugene Koifman a committer on the > Apache Hive Project. > > Please join me in congratulating Eugene! > > Thanks. > > - Carl > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Review Request 25781: Windows: alter table ..set location from hcatalog failed with NullPointerException
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25781/ --- Review request for hive. Repository: hive-git Description --- Steps to reproduce the issue: Save the above query as HIVE-8169.hql, run it by >python $HIVE_HOME/hcatalog/bin/hcat.py -f HIVE-8169.hql, it pops up FAILED: SemanticException org.apache.hadoop.hive.ql.parse.SemanticException: java.lang.Null. drop table if exists hcat_altertable_16; create table hcat_altertable_16(a int, b string) stored as textfile; show table extended like hcat_altertable_16; alter table hcat_altertable_16 set location 'hdfs:///tmp/table_has_moved/'; show table extended like hcat_altertable_16; Stack trace: org.apache.hadoop.hive.ql.parse.SemanticException: org.apache.hadoop.hive.ql.parse.SemanticException: java.lang.NullPointerException at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzer.postAnalyze(HCatSemanticAnalyzer.java:264) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:406) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:299) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:993) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1063) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:930) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:920) at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43) at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291) at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245) at org.apache.hive.hcatalog.cli.HCatCli.processFile(HCatCli.java:264) at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:188) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: org.apache.hadoop.hive.ql.parse.SemanticException: java.lang.NullPointerException at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzerBase.authorizeDDL(HCatSemanticAnalyzerBase.java:114) at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzer.postAnalyze(HCatSemanticAnalyzer.java:259) ... 17 more Caused by: java.lang.NullPointerException at org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider.authorize(StorageBasedAuthorizationProvider.java:178) at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzerBase.authorize(HCatSemanticAnalyzerBase.java:161) at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzer.authorizeDDLWork(HCatSemanticAnalyzer.java:381) at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzerBase.authorizeDDL(HCatSemanticAnalyzerBase.java:105) ... 18 more Diffs - hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzer.java 6c54c05 Diff: https://reviews.apache.org/r/25781/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 25781: Windows: alter table ..set location from hcatalog failed with NullPointerException
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25781/ --- (Updated Sept. 18, 2014, 5:04 p.m.) Review request for hive. Repository: hive-git Description --- Steps to reproduce the issue: Save the above query as HIVE-8169.hql, run it by >python $HIVE_HOME/hcatalog/bin/hcat.py -f HIVE-8169.hql, it pops up FAILED: SemanticException org.apache.hadoop.hive.ql.parse.SemanticException: java.lang.Null. drop table if exists hcat_altertable_16; create table hcat_altertable_16(a int, b string) stored as textfile; show table extended like hcat_altertable_16; alter table hcat_altertable_16 set location 'hdfs:///tmp/table_has_moved/'; show table extended like hcat_altertable_16; Stack trace: org.apache.hadoop.hive.ql.parse.SemanticException: org.apache.hadoop.hive.ql.parse.SemanticException: java.lang.NullPointerException at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzer.postAnalyze(HCatSemanticAnalyzer.java:264) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:406) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:299) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:993) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1063) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:930) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:920) at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43) at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291) at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245) at org.apache.hive.hcatalog.cli.HCatCli.processFile(HCatCli.java:264) at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:188) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: org.apache.hadoop.hive.ql.parse.SemanticException: java.lang.NullPointerException at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzerBase.authorizeDDL(HCatSemanticAnalyzerBase.java:114) at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzer.postAnalyze(HCatSemanticAnalyzer.java:259) ... 17 more Caused by: java.lang.NullPointerException at org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider.authorize(StorageBasedAuthorizationProvider.java:178) at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzerBase.authorize(HCatSemanticAnalyzerBase.java:161) at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzer.authorizeDDLWork(HCatSemanticAnalyzer.java:381) at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzerBase.authorizeDDL(HCatSemanticAnalyzerBase.java:105) ... 18 more Diffs - hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzer.java 6c54c05 Diff: https://reviews.apache.org/r/25781/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 25781: Windows: alter table ..set location from hcatalog failed with NullPointerException
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25781/ --- (Updated Sept. 18, 2014, 5:05 p.m.) Review request for hive. Repository: hive-git Description --- Steps to reproduce the issue: Save the above query as HIVE-8169.hql, run it by >python $HIVE_HOME/hcatalog/bin/hcat.py -f HIVE-8169.hql, it pops up FAILED: SemanticException org.apache.hadoop.hive.ql.parse.SemanticException: java.lang.Null. drop table if exists hcat_altertable_16; create table hcat_altertable_16(a int, b string) stored as textfile; show table extended like hcat_altertable_16; alter table hcat_altertable_16 set location 'hdfs:///tmp/table_has_moved/'; show table extended like hcat_altertable_16; Stack trace: org.apache.hadoop.hive.ql.parse.SemanticException: org.apache.hadoop.hive.ql.parse.SemanticException: java.lang.NullPointerException at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzer.postAnalyze(HCatSemanticAnalyzer.java:264) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:406) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:299) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:993) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1063) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:930) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:920) at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43) at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291) at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245) at org.apache.hive.hcatalog.cli.HCatCli.processFile(HCatCli.java:264) at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:188) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: org.apache.hadoop.hive.ql.parse.SemanticException: java.lang.NullPointerException at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzerBase.authorizeDDL(HCatSemanticAnalyzerBase.java:114) at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzer.postAnalyze(HCatSemanticAnalyzer.java:259) ... 17 more Caused by: java.lang.NullPointerException at org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider.authorize(StorageBasedAuthorizationProvider.java:178) at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzerBase.authorize(HCatSemanticAnalyzerBase.java:161) at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzer.authorizeDDLWork(HCatSemanticAnalyzer.java:381) at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzerBase.authorizeDDL(HCatSemanticAnalyzerBase.java:105) ... 18 more Diffs - hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzer.java 6c54c05 Diff: https://reviews.apache.org/r/25781/diff/ Testing --- Thanks, XIAOBING ZHOU
Review Request 25784: Hive select query hang when fields contain map
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25784/ --- Review request for hive. Repository: hive-git Description --- refer to: https://issues.apache.org/jira/browse/HIVE-8115 Diffs - serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyMap.java 686fc76 Diff: https://reviews.apache.org/r/25784/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 25784: Hive select query hang when fields contain map
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25784/ --- (Updated Sept. 18, 2014, 5:40 p.m.) Review request for hive. Repository: hive-git Description (updated) --- SEE ALSO: https://issues.apache.org/jira/browse/HIVE-8115 Diffs - serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyMap.java 686fc76 Diff: https://reviews.apache.org/r/25784/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 25784: Hive select query hang when fields contain map
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25784/ --- (Updated Sept. 22, 2014, 10:23 p.m.) Review request for hive. Repository: hive-git Description --- SEE ALSO: https://issues.apache.org/jira/browse/HIVE-8115 Diffs (updated) - serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyMap.java 686fc76 Diff: https://reviews.apache.org/r/25784/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 25784: Hive select query hang when fields contain map
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25784/ --- (Updated Sept. 22, 2014, 10:41 p.m.) Review request for hive. Repository: hive-git Description --- SEE ALSO: https://issues.apache.org/jira/browse/HIVE-8115 Diffs (updated) - serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyMap.java 686fc76 Diff: https://reviews.apache.org/r/25784/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 25784: Hive select query hang when fields contain map
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25784/ --- (Updated Sept. 23, 2014, 8:45 p.m.) Review request for hive. Repository: hive-git Description --- SEE ALSO: https://issues.apache.org/jira/browse/HIVE-8115 Diffs (updated) - serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyMap.java 686fc76 serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazyArrayMapStruct.java ac0b583 Diff: https://reviews.apache.org/r/25784/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 25784: Hive select query hang when fields contain map
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25784/ --- (Updated Sept. 23, 2014, 8:51 p.m.) Review request for hive. Repository: hive-git Description --- SEE ALSO: https://issues.apache.org/jira/browse/HIVE-8115 Diffs (updated) - serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyMap.java 686fc76 serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazyArrayMapStruct.java ac0b583 Diff: https://reviews.apache.org/r/25784/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 25784: Hive select query hang when fields contain map
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25784/ --- (Updated Sept. 23, 2014, 8:54 p.m.) Review request for hive. Repository: hive-git Description --- SEE ALSO: https://issues.apache.org/jira/browse/HIVE-8115 Diffs (updated) - serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyMap.java 686fc76 serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazyArrayMapStruct.java ac0b583 Diff: https://reviews.apache.org/r/25784/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 25781: Windows: alter table ..set location from hcatalog failed with NullPointerException
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25781/ --- (Updated Sept. 24, 2014, 1:17 a.m.) Review request for hive. Repository: hive-git Description --- Steps to reproduce the issue: Save the above query as HIVE-8169.hql, run it by >python $HIVE_HOME/hcatalog/bin/hcat.py -f HIVE-8169.hql, it pops up FAILED: SemanticException org.apache.hadoop.hive.ql.parse.SemanticException: java.lang.Null. drop table if exists hcat_altertable_16; create table hcat_altertable_16(a int, b string) stored as textfile; show table extended like hcat_altertable_16; alter table hcat_altertable_16 set location 'hdfs:///tmp/table_has_moved/'; show table extended like hcat_altertable_16; Stack trace: org.apache.hadoop.hive.ql.parse.SemanticException: org.apache.hadoop.hive.ql.parse.SemanticException: java.lang.NullPointerException at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzer.postAnalyze(HCatSemanticAnalyzer.java:264) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:406) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:299) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:993) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1063) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:930) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:920) at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43) at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291) at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245) at org.apache.hive.hcatalog.cli.HCatCli.processFile(HCatCli.java:264) at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:188) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: org.apache.hadoop.hive.ql.parse.SemanticException: java.lang.NullPointerException at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzerBase.authorizeDDL(HCatSemanticAnalyzerBase.java:114) at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzer.postAnalyze(HCatSemanticAnalyzer.java:259) ... 17 more Caused by: java.lang.NullPointerException at org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider.authorize(StorageBasedAuthorizationProvider.java:178) at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzerBase.authorize(HCatSemanticAnalyzerBase.java:161) at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzer.authorizeDDLWork(HCatSemanticAnalyzer.java:381) at org.apache.hive.hcatalog.cli.SemanticAnalysis.HCatSemanticAnalyzerBase.authorizeDDL(HCatSemanticAnalyzerBase.java:105) ... 18 more Diffs (updated) - hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzer.java 6c54c05 ql/src/test/queries/clientpositive/alter_table_location.q PRE-CREATION ql/src/test/results/clientpositive/alter_table_location.q.out PRE-CREATION Diff: https://reviews.apache.org/r/25781/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: [ANNOUNCE] New Hive Committer - John Pullokkaran
Congrats John! On Wed, Oct 1, 2014 at 12:54 PM, Matthew McCline wrote: > Congrats! > > On Wed, Oct 1, 2014 at 8:19 AM, Carl Steinbach wrote: > > > The Apache Hive PMC has voted to make John Pullokkaran a committer on the > > Apache Hive Project. > > > > Please join me in congratulating John! > > > > Thanks. > > > > - Carl > > > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Review Request 26321: HiveServer2 service doesn't stop backend jvm process, which prevents follow-up service start.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26321/ --- Review request for hive. Repository: hive-git Description --- On stopping the HS2 service from the services tab, it only kills the root process and does not kill the child java process. As a result resources are not freed and this throws an error on restarting from command line. Diffs - bin/ext/hiveserver2.cmd a5f3bb5 bin/hive.cmd c2e9853 common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 3fe67b2 ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 79da5a0 Diff: https://reviews.apache.org/r/26321/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 26321: HiveServer2 service doesn't stop backend jvm process, which prevents follow-up service start.
> On Oct. 3, 2014, 7:06 p.m., Vaibhav Gumashta wrote: > > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 1726 > > <https://reviews.apache.org/r/26321/diff/1/?file=713544#file713544line1726> > > > > Can you add a description for this new param (for why this is required) For Windows OS, we need to pass HIVE_HADOOP_CLASSPATH java parameter while starting hiveserver2 using "-hiveconf hive.hadoop.classpath=%HIVE_LIB%". This is where it's defined. > On Oct. 3, 2014, 7:06 p.m., Vaibhav Gumashta wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java, line 241 > > <https://reviews.apache.org/r/26321/diff/1/?file=713545#file713545line241> > > > > Can you add a comment here explaining the intent of using > > HIVE_HADOOP_CLASSPATH? For Windows OS, we need to pass HIVE_HADOOP_CLASSPATH java parameter while starting hiveserver2 using "-hiveconf hive.hadoop.classpath=%HIVE_LIB%". This is how it's used. - XIAOBING --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26321/#review55378 --- On Oct. 3, 2014, 6:27 p.m., XIAOBING ZHOU wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/26321/ > --- > > (Updated Oct. 3, 2014, 6:27 p.m.) > > > Review request for hive. > > > Repository: hive-git > > > Description > --- > > On stopping the HS2 service from the services tab, it only kills the root > process and does not kill the child java process. As a result resources are > not freed and this throws an error on restarting from command line. > > > Diffs > - > > bin/ext/hiveserver2.cmd a5f3bb5 > bin/hive.cmd c2e9853 > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 3fe67b2 > ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 79da5a0 > > Diff: https://reviews.apache.org/r/26321/diff/ > > > Testing > --- > > > Thanks, > > XIAOBING ZHOU > >
Re: Review Request 26321: HiveServer2 service doesn't stop backend jvm process, which prevents follow-up service start.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26321/ --- (Updated Oct. 4, 2014, 5:11 a.m.) Review request for hive. Repository: hive-git Description --- On stopping the HS2 service from the services tab, it only kills the root process and does not kill the child java process. As a result resources are not freed and this throws an error on restarting from command line. Diffs (updated) - bin/ext/hiveserver2.cmd a5f3bb5 bin/hive.cmd c2e9853 common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 3fe67b2 ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 79da5a0 Diff: https://reviews.apache.org/r/26321/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 26321: HiveServer2 service doesn't stop backend jvm process, which prevents follow-up service start.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26321/ --- (Updated Oct. 4, 2014, 5:20 a.m.) Review request for hive. Repository: hive-git Description --- On stopping the HS2 service from the services tab, it only kills the root process and does not kill the child java process. As a result resources are not freed and this throws an error on restarting from command line. Diffs (updated) - bin/ext/hiveserver2.cmd a5f3bb5 bin/hive.cmd c2e9853 common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 3fe67b2 ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 79da5a0 Diff: https://reviews.apache.org/r/26321/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 26321: HiveServer2 service doesn't stop backend jvm process, which prevents follow-up service start.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26321/ --- (Updated Oct. 4, 2014, 5:21 a.m.) Review request for hive. Repository: hive-git Description --- On stopping the HS2 service from the services tab, it only kills the root process and does not kill the child java process. As a result resources are not freed and this throws an error on restarting from command line. Diffs (updated) - bin/ext/hiveserver2.cmd a5f3bb5 bin/hive.cmd c2e9853 common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 3fe67b2 ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 79da5a0 Diff: https://reviews.apache.org/r/26321/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 22191: HIVE-7167: Hive Metastore fails to start with SQLServerException
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22191/ --- (Updated Oct. 6, 2014, 10:14 p.m.) Review request for hive. Bugs: HIVE-7167 https://issues.apache.org/jira/browse/HIVE-7167 Repository: hive-git Description --- In the case that hiveserver2 uses embedded metastore and hiveserver uses remote metastore, this exception comes up when hiveserver2 and hiveserver are started simultaneously. See the bug link: https://issues.apache.org/jira/browse/HIVE-7167 The idea is to catch JDOException from concurrent defaultDB and roles initializations or privilege insertions, and try createDefaultDB, createDefaultRoles and addAdminUsers one more time, if still fails, simply throw up exceptions. Diffs (updated) - metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 47eca29 Diff: https://reviews.apache.org/r/22191/diff/ Testing --- done Thanks, XIAOBING ZHOU
Review Request 26457: OrcFileMergeOperator only close last orc file it opened, which resulted in stale data in table directory
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26457/ --- Review request for hive. Repository: hive-git Description --- run the test mvn -Phadoop-2 test -Dtest=TestCliDriver -Dqfile=alter_merge_2_orc.q to reproduce it. Simply, this query does three data loads which generates three orc files, ALTER TABLE CONCATENATE tries to merge orc pieces into a single one which is final file to queried. Output \hive\itests\qtest\target\qfile-results\clientpositive\alter_merge_2_orc.q.out shows # records as 600 that is wrong as opposed to 610 expected. Because OrcFileMergeOperator only closes last orc file, the 1st and 2nd orc files still remain in table directory due to failure of deleting unclosed file for old data clean when MoveTask tries to copy merged orc file from scratch dir to table dir. Eventually the query goes to old data(1st and 2nd orc files). Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/OrcFileMergeOperator.java 0fb1c57 Diff: https://reviews.apache.org/r/26457/diff/ Testing --- Passed a bunch of orc file merge UT falures. Thanks, XIAOBING ZHOU
Re: Review Request 26457: OrcFileMergeOperator only close last orc file it opened, which resulted in stale data in table directory
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26457/ --- (Updated Oct. 8, 2014, 9 p.m.) Review request for hive. Repository: hive-git Description --- run the test mvn -Phadoop-2 test -Dtest=TestCliDriver -Dqfile=alter_merge_2_orc.q to reproduce it. Simply, this query does three data loads which generates three orc files, ALTER TABLE CONCATENATE tries to merge orc pieces into a single one which is final file to queried. Output \hive\itests\qtest\target\qfile-results\clientpositive\alter_merge_2_orc.q.out shows # records as 600 that is wrong as opposed to 610 expected. Because OrcFileMergeOperator only closes last orc file, the 1st and 2nd orc files still remain in table directory due to failure of deleting unclosed file for old data clean when MoveTask tries to copy merged orc file from scratch dir to table dir. Eventually the query goes to old data(1st and 2nd orc files). Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/OrcFileMergeOperator.java 0fb1c57 Diff: https://reviews.apache.org/r/26457/diff/ Testing (updated) --- Cleared a bunch of orc file merge UT falures. Thanks, XIAOBING ZHOU
Re: Review Request 25784: Hive select query hang when fields contain map
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25784/ --- (Updated Oct. 10, 2014, 8:15 p.m.) Review request for hive. Repository: hive-git Description --- SEE ALSO: https://issues.apache.org/jira/browse/HIVE-8115 Diffs (updated) - serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyMap.java 686fc76 serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazyArrayMapStruct.java ac0b583 Diff: https://reviews.apache.org/r/25784/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 25784: Hive select query hang when fields contain map
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25784/ --- (Updated Oct. 10, 2014, 9:49 p.m.) Review request for hive. Repository: hive-git Description --- SEE ALSO: https://issues.apache.org/jira/browse/HIVE-8115 Diffs (updated) - serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyMap.java 686fc76 serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazyArrayMapStruct.java ac0b583 Diff: https://reviews.apache.org/r/25784/diff/ Testing --- Thanks, XIAOBING ZHOU
Review Request 26887: StatsNoJobTask doesn't close RecordReader, FSDataInputStream of which keeps open to prevent stale data clean
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26887/ --- Review request for hive. Repository: hive-git Description --- run the test mvn -Phadoop-2 test -Dtest=TestCliDriver -Dqfile=alter_merge_stats_orc.q to reproduce it. Simply, this query does three data loads which generates three base orc files. ANALYZE TABLE...COMPUTE STATISTICS NOSCAN will execute StatsNoJobTask to get stats, where file handle is held so as not able to clean base file. As a result, after running ALTER TABLE..CONCATENATE, follow-up queries go to stale base file and merged file. Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/StatsNoJobTask.java 07efb8f Diff: https://reviews.apache.org/r/26887/diff/ Testing --- passed alter_merge_stats_orc.q Thanks, XIAOBING ZHOU
Re: Review Request 26887: StatsNoJobTask doesn't close RecordReader, FSDataInputStream of which keeps open to prevent stale data clean
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26887/ --- (Updated Oct. 17, 2014, 7:10 p.m.) Review request for hive. Repository: hive-git Description --- run the test mvn -Phadoop-2 test -Dtest=TestCliDriver -Dqfile=alter_merge_stats_orc.q to reproduce it. Simply, this query does three data loads which generates three base orc files. ANALYZE TABLE...COMPUTE STATISTICS NOSCAN will execute StatsNoJobTask to get stats, where file handle is held so as not able to clean base file. As a result, after running ALTER TABLE..CONCATENATE, follow-up queries go to stale base file and merged file. Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/StatsNoJobTask.java 07efb8f Diff: https://reviews.apache.org/r/26887/diff/ Testing --- passed alter_merge_stats_orc.q Thanks, XIAOBING ZHOU
Review Request 21642: HIVE-5764: Stopping Metastore and HiveServer2 from command line
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21642/ --- Review request for hive, Hari Sankar Sivarama Subramaniyan, Harish Butani, Thejas Nair, and Vaibhav Gumashta. Repository: hive Description --- Corresponding to JIRA HIVE-5764(https://issues.apache.org/jira/browse/HIVE-5764) This patch is to support stopping service by command line. So far only hiveserver2 and metastore are supported, but maintains back compatibility to other services. bin/hive --service hiveserver2 --stop bin/hive --service metastore --stop Only one instance of hiveserver2 or metastore is allowed to be launched in individual node, but many instances allowed before applying this patch. Diffs - Diff: https://reviews.apache.org/r/21642/diff/ Testing --- bin/hive --service hiveserver2 bin/hive --service hiveserver2 bin/hive --service hiveserver2 --stop bin/hive --service hiveserver2 --stop bin/hive --service hiveserver2 bin/hive --service hiveserver2 --stop bin/hive --service hiveserver2 --help bin/hive --service hiveserver2 -H bin/hive --service metastore bin/hive --service metastore bin/hive --service metastore --stop bin/hive --service metastore --stop bin/hive --service metastore bin/hive --service metastore --stop bin/hive --service metastore --help bin/hive --service metastore -h Thanks, XIAOBING ZHOU
Re: Review Request 21642: HIVE-5764: Stopping Metastore and HiveServer2 from command line
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21642/ --- (Updated May 19, 2014, 6:12 p.m.) Review request for hive, Hari Sankar Sivarama Subramaniyan, Harish Butani, Thejas Nair, and Vaibhav Gumashta. Changes --- upload patch diff file Repository: hive Description --- Corresponding to JIRA HIVE-5764(https://issues.apache.org/jira/browse/HIVE-5764) This patch is to support stopping service by command line. So far only hiveserver2 and metastore are supported, but maintains back compatibility to other services. bin/hive --service hiveserver2 --stop bin/hive --service metastore --stop Only one instance of hiveserver2 or metastore is allowed to be launched in individual node, but many instances allowed before applying this patch. Diffs - Diff: https://reviews.apache.org/r/21642/diff/ Testing --- bin/hive --service hiveserver2 bin/hive --service hiveserver2 bin/hive --service hiveserver2 --stop bin/hive --service hiveserver2 --stop bin/hive --service hiveserver2 bin/hive --service hiveserver2 --stop bin/hive --service hiveserver2 --help bin/hive --service hiveserver2 -H bin/hive --service metastore bin/hive --service metastore bin/hive --service metastore --stop bin/hive --service metastore --stop bin/hive --service metastore bin/hive --service metastore --stop bin/hive --service metastore --help bin/hive --service metastore -h File Attachments (updated) DIFF FILE https://reviews.apache.org/media/uploaded/files/2014/05/19/a23cb744-2051-4b2b-ba32-5eb71d1f9c65__HIVE-5764.patch Thanks, XIAOBING ZHOU
Re: Review Request 21642: HIVE-5764: Stopping Metastore and HiveServer2 from command line
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21642/ --- (Updated May 19, 2014, 10:19 p.m.) Review request for hive, Hari Sankar Sivarama Subramaniyan, Harish Butani, Thejas Nair, and Vaibhav Gumashta. Repository: hive Description --- Corresponding to JIRA HIVE-5764(https://issues.apache.org/jira/browse/HIVE-5764) This patch is to support stopping service by command line. So far only hiveserver2 and metastore are supported, but maintains back compatibility to other services. bin/hive --service hiveserver2 --stop bin/hive --service metastore --stop Only one instance of hiveserver2 or metastore is allowed to be launched in individual node, but many instances allowed before applying this patch. Diffs (updated) - http://svn.apache.org/repos/asf/hive/trunk/bin/ext/hiveserver2.sh 1596074 http://svn.apache.org/repos/asf/hive/trunk/bin/ext/metastore.sh 1596074 http://svn.apache.org/repos/asf/hive/trunk/bin/hive 1596074 http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1596074 http://svn.apache.org/repos/asf/hive/trunk/service/src/java/org/apache/hive/service/server/ServerOptionsProcessor.java 1596074 Diff: https://reviews.apache.org/r/21642/diff/ Testing --- bin/hive --service hiveserver2 bin/hive --service hiveserver2 bin/hive --service hiveserver2 --stop bin/hive --service hiveserver2 --stop bin/hive --service hiveserver2 bin/hive --service hiveserver2 --stop bin/hive --service hiveserver2 --help bin/hive --service hiveserver2 -H bin/hive --service metastore bin/hive --service metastore bin/hive --service metastore --stop bin/hive --service metastore --stop bin/hive --service metastore bin/hive --service metastore --stop bin/hive --service metastore --help bin/hive --service metastore -h File Attachments DIFF FILE https://reviews.apache.org/media/uploaded/files/2014/05/19/a23cb744-2051-4b2b-ba32-5eb71d1f9c65__HIVE-5764.patch Thanks, XIAOBING ZHOU
Re: Review Request 21642: HIVE-5764: Stopping Metastore and HiveServer2 from command line
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21642/ --- (Updated May 19, 2014, 10:56 p.m.) Review request for hive, Hari Sankar Sivarama Subramaniyan, Harish Butani, Thejas Nair, and Vaibhav Gumashta. Bugs: HIVE-5764 https://issues.apache.org/jira/browse/HIVE-5764 Repository: hive Description --- Corresponding to JIRA HIVE-5764(https://issues.apache.org/jira/browse/HIVE-5764) This patch is to support stopping service by command line. So far only hiveserver2 and metastore are supported, but maintains back compatibility to other services. bin/hive --service hiveserver2 --stop bin/hive --service metastore --stop Only one instance of hiveserver2 or metastore is allowed to be launched in individual node, but many instances allowed before applying this patch. Diffs - http://svn.apache.org/repos/asf/hive/trunk/bin/ext/hiveserver2.sh 1596074 http://svn.apache.org/repos/asf/hive/trunk/bin/ext/metastore.sh 1596074 http://svn.apache.org/repos/asf/hive/trunk/bin/hive 1596074 http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1596074 http://svn.apache.org/repos/asf/hive/trunk/service/src/java/org/apache/hive/service/server/ServerOptionsProcessor.java 1596074 Diff: https://reviews.apache.org/r/21642/diff/ Testing --- bin/hive --service hiveserver2 bin/hive --service hiveserver2 bin/hive --service hiveserver2 --stop bin/hive --service hiveserver2 --stop bin/hive --service hiveserver2 bin/hive --service hiveserver2 --stop bin/hive --service hiveserver2 --help bin/hive --service hiveserver2 -H bin/hive --service metastore bin/hive --service metastore bin/hive --service metastore --stop bin/hive --service metastore --stop bin/hive --service metastore bin/hive --service metastore --stop bin/hive --service metastore --help bin/hive --service metastore -h File Attachments DIFF FILE https://reviews.apache.org/media/uploaded/files/2014/05/19/a23cb744-2051-4b2b-ba32-5eb71d1f9c65__HIVE-5764.patch Thanks, XIAOBING ZHOU
Review Request 22073: HIVE-405 Hive Metastore fails to start with SQLServerException
d) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.ja va:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocket Impl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java :182) at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:157) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.thrift.transport.TSocket.open(TSocket.java:180) ... 19 more ) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaSto reClient.java:382) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaS toreClient.java:214) ... 17 more Because of this issue, we are seeing many failures in hive related system tests. Attached is hive.log from asvhive21 cluster. Diffs - metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java a6afd8c Diff: https://reviews.apache.org/r/22073/diff/ Testing --- done Thanks, XIAOBING ZHOU
Re: Review Request 22073: HIVE-405 Hive Metastore fails to start with SQLServerException
5) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) Caused by: java.net.ConnectException: Connection refused: connect at java.net.TwoStacksPlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.ja va:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocket Impl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java :182) at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:157) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.thrift.transport.TSocket.open(TSocket.java:180) ... 19 more ) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaSto reClient.java:382) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaS toreClient.java:214) ... 17 more Because of this issue, we are seeing many failures in hive related system tests. Attached is hive.log from asvhive21 cluster. Diffs (updated) - metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java a6afd8c Diff: https://reviews.apache.org/r/22073/diff/ Testing --- done Thanks, XIAOBING ZHOU
Re: Review Request 22073: HIVE-405 Hive Metastore fails to start with SQLServerException
nectException: Connection refused: connect at java.net.TwoStacksPlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.ja va:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocket Impl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java :182) at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:157) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.thrift.transport.TSocket.open(TSocket.java:180) ... 19 more ) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaSto reClient.java:382) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaS toreClient.java:214) ... 17 more Because of this issue, we are seeing many failures in hive related system tests. Attached is hive.log from asvhive21 cluster. Diffs - metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java a6afd8c Diff: https://reviews.apache.org/r/22073/diff/ Testing --- done Thanks, XIAOBING ZHOU
Re: Review Request 22073: HIVE-405 Hive Metastore fails to start with SQLServerException
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22073/ --- (Updated May 30, 2014, 6:01 a.m.) Review request for hive, Hari Sankar Sivarama Subramaniyan, Thejas Nair, and Vaibhav Gumashta. Bugs: HIVE-405 https://issues.apache.org/jira/browse/HIVE-405 Repository: hive-git Description (updated) --- see also jira link: https://hwxmonarch.atlassian.net/browse/HIVE-405 The idea is to catch any exceptions from concurrent defaultDB and roles initializations or privilege insertions, and try createDefaultDB, createDefaultRoles and addAdminUsers one more time, if still fails, simply return by printing out error msg and stack trace. Since exceptions are suppressed and consumed, other kind of issues like connection failure will not be catchable by HiveMetaStore caller(s), this could lead to some problems. Diffs - metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java a6afd8c Diff: https://reviews.apache.org/r/22073/diff/ Testing --- done Thanks, XIAOBING ZHOU
Review Request 22191: HIVE-7167: Hive Metastore fails to start with SQLServerException
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22191/ --- Review request for hive. Bugs: HIVE-7167 https://issues.apache.org/jira/browse/HIVE-7167 Repository: hive-git Description --- In the case that hiveserver2 uses embedded metastore and hiveserver uses remote metastore, this exception comes up when hiveserver2 and hiveserver are started simultaneously. See the bug link: https://issues.apache.org/jira/browse/HIVE-7167 Diffs - hcatalog/core/.gitignore 0a7a9c5 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java a6afd8c Diff: https://reviews.apache.org/r/22191/diff/ Testing --- done Thanks, XIAOBING ZHOU
Re: Review Request 22191: HIVE-7167: Hive Metastore fails to start with SQLServerException
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22191/ --- (Updated June 3, 2014, 6:20 a.m.) Review request for hive. Bugs: HIVE-7167 https://issues.apache.org/jira/browse/HIVE-7167 Repository: hive-git Description (updated) --- In the case that hiveserver2 uses embedded metastore and hiveserver uses remote metastore, this exception comes up when hiveserver2 and hiveserver are started simultaneously. See the bug link: https://issues.apache.org/jira/browse/HIVE-7167 The idea is to catch any exceptions from concurrent defaultDB and roles initializations or privilege insertions, and try createDefaultDB, createDefaultRoles and addAdminUsers one more time, if still fails, simply return by printing out error msg and stack trace. Since exceptions are suppressed and consumed, other kind of issues like connection failure will not be catchable by HiveMetaStore caller(s), this could lead to some problems. Diffs - hcatalog/core/.gitignore 0a7a9c5 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java a6afd8c Diff: https://reviews.apache.org/r/22191/diff/ Testing --- done Thanks, XIAOBING ZHOU
Re: Review Request 22191: HIVE-7167: Hive Metastore fails to start with SQLServerException
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22191/ --- (Updated June 3, 2014, 6:22 a.m.) Review request for hive. Bugs: HIVE-7167 https://issues.apache.org/jira/browse/HIVE-7167 Repository: hive-git Description (updated) --- In the case that hiveserver2 uses embedded metastore and hiveserver uses remote metastore, this exception comes up when hiveserver2 and hiveserver are started simultaneously. See the bug link: https://issues.apache.org/jira/browse/HIVE-7167 The idea is to catch JDOException from concurrent defaultDB and roles initializations or privilege insertions, and try createDefaultDB, createDefaultRoles and addAdminUsers one more time, if still fails, simply throw up exceptions. Diffs - hcatalog/core/.gitignore 0a7a9c5 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java a6afd8c Diff: https://reviews.apache.org/r/22191/diff/ Testing --- done Thanks, XIAOBING ZHOU
Re: Review Request 22191: HIVE-7167: Hive Metastore fails to start with SQLServerException
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22191/ --- (Updated June 20, 2014, 8:55 p.m.) Review request for hive. Bugs: HIVE-7167 https://issues.apache.org/jira/browse/HIVE-7167 Repository: hive-git Description --- In the case that hiveserver2 uses embedded metastore and hiveserver uses remote metastore, this exception comes up when hiveserver2 and hiveserver are started simultaneously. See the bug link: https://issues.apache.org/jira/browse/HIVE-7167 The idea is to catch JDOException from concurrent defaultDB and roles initializations or privilege insertions, and try createDefaultDB, createDefaultRoles and addAdminUsers one more time, if still fails, simply throw up exceptions. Diffs - metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 487d292 Diff: https://reviews.apache.org/r/22191/diff/ Testing --- done Thanks, XIAOBING ZHOU
Re: Review Request 22191: HIVE-7167: Hive Metastore fails to start with SQLServerException
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22191/ --- (Updated June 20, 2014, 8:55 p.m.) Review request for hive. Changes --- new patch HIVE-7167.2.patch is made on trunk and attached. Bugs: HIVE-7167 https://issues.apache.org/jira/browse/HIVE-7167 Repository: hive-git Description --- In the case that hiveserver2 uses embedded metastore and hiveserver uses remote metastore, this exception comes up when hiveserver2 and hiveserver are started simultaneously. See the bug link: https://issues.apache.org/jira/browse/HIVE-7167 The idea is to catch JDOException from concurrent defaultDB and roles initializations or privilege insertions, and try createDefaultDB, createDefaultRoles and addAdminUsers one more time, if still fails, simply throw up exceptions. Diffs (updated) - metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 487d292 Diff: https://reviews.apache.org/r/22191/diff/ Testing --- done Thanks, XIAOBING ZHOU
Re: [ANNOUNCE] New Hive Committers - Gopal Vijayaraghavan and Szehon Ho
Congrats! On Mon, Jun 23, 2014 at 9:52 AM, Vaibhav Gumashta wrote: > Congrats Gopal and Szehon! > > --Vaibhav > > > On Mon, Jun 23, 2014 at 8:48 AM, Szehon Ho wrote: > > > Thank you all very much, and congrats Gopal! > > Szehon > > > > > > On Sun, Jun 22, 2014 at 8:42 PM, Carl Steinbach wrote: > > > > > The Apache Hive PMC has voted to make Gopal Vijayaraghavan and Szehon > Ho > > > committers on the Apache Hive Project. > > > > > > Please join me in congratulating Gopal and Szehon! > > > > > > Thanks. > > > > > > - Carl > > > > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Error running unit tests from eclipse (weird classpath issue)
Hi Pavel, I usually do interactive debugging by using -Dmaven.surefire.debug, e.g. mvn -Dmaven.surefire.debug test -Phadoop-2 -Dhadoop-23.version=2.4.0.2081 -Dtest=TestHiveHBaseStorageHandler#testTableCreateExternal. Maven will launch a VM running as a server and listen any incoming debug requests on port 5005, and then create a debug config in eclipse on [Remote Java Application] with socket(localhost:5005) attached to launch debugging. I did tried your approach, but not working out, and I got jar dependency problems, why not to adopt the way mentioned above? You got full control over. Hopefully it helps, thanks! -Xiaobing On Wed, Jul 23, 2014 at 6:40 AM, Nick Dimiduk wrote: > I'm sure google knows; I don't. > > For what it's worth, I run tests via maven and attach the debugger to the > remote process when necessary. > > -n > > On Tuesday, July 22, 2014, Pavel Chadnov wrote: > > > How can I do it in eclipse. I do this in console and it works > > > > On Tuesday, July 22, 2014, Nick Dimiduk > > wrote: > > > >> Are you specifying a hadoop profile via eclipse? Ie, from maven, > >> -Phadoop-2. > >> > >> > >> On Tue, Jul 22, 2014 at 4:03 PM, Pavel Chadnov > >> wrote: > >> > >>> Hey Guys, > >>> > >>> > >>> I'm trying to run Hive unit tests on eclipse and have few failures. One > >>> of > >>> the interesting one is throwing this exception as shown below when ran > >>> from > >>> eclipse, this one passes fine from the console. > >>> > >>> > >>> java.lang.IncompatibleClassChangeError: Implementing class > >>> > >>> at java.lang.ClassLoader.defineClass1(Native Method) > >>> > >>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800) > >>> > >>> at > >>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) > >>> > >>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) > >>> > >>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) > >>> > >>> ... > >>> > >>> ... > >>> > >>> at java.lang.Class.forName(Class.java:190) > >>> > >>> at > >>> org.apache.hadoop.hive.shims.ShimLoader.createShim(ShimLoader.java:120) > >>> > >>> at > org.apache.hadoop.hive.shims.ShimLoader.loadShims(ShimLoader.java:115) > >>> > >>> at > >>> > >>> > org.apache.hadoop.hive.shims.ShimLoader.getHadoopShims(ShimLoader.java:80) > >>> > >>> at > >>> > org.apache.hadoop.hive.conf.HiveConf$ConfVars.(HiveConf.java:254) > >>> > >>> at > >>> > org.apache.hadoop.hive.ql.exec.Utilities.getPlanPath(Utilities.java:652) > >>> > >>> at > >>> > org.apache.hadoop.hive.ql.exec.Utilities.setPlanPath(Utilities.java:641) > >>> > >>> at > >>> > org.apache.hadoop.hive.ql.exec.Utilities.setBaseWork(Utilities.java:584) > >>> > >>> at > >>> org.apache.hadoop.hive.ql.exec.Utilities.setMapWork(Utilities.java:575) > >>> > >>> at > >>> > >>> > org.apache.hadoop.hive.ql.exec.Utilities.setMapRedWork(Utilities.java:568) > >>> > >>> at > >>> > >>> > org.apache.hadoop.hive.ql.io.TestSymlinkTextInputFormat.setUp(TestSymlinkTextInputFormat.java:84) > >>> > >>> at junit.framework.TestCase.runBare(TestCase.java:132) > >>> > >>> > >>> I tried adding hadoop-shims project in the classpath by manually adding > >>> them but no luck. Would really appreciate any help here. > >>> > >>> > >>> Thanks, > >>> > >>> Pavel > >>> > >> > >> > > > > -- > > Regards, > > Pavel Chadnov > > > > > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Review Request 23907: Hive: output is incorrect if there are UTF-8 characters in where clause of a hive select query.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23907/ --- Review request for hive. Repository: hive-git Description --- When we put UTF-8 characters in where clause of a hive query the results are empty for "where content like '%?%'" and results contain all rows for "where content not like '%?%';" even when few rows contain this character. Steps to reproduce: 1. Save a file called data.txt in the root container. The contents of the files are as follows. 190 ?f??c??h?c? 899 d???geg??ea?eead?e 137 ??h?ge??g?? 21 ??e?c??d?? 767 ?c?g? 281 ???aga?c?e?? 573 ??hc?b??hc? 966 e?eb??cga?? 565 bb?ehd?ea?? 778 ?bbea??a? 363 gd?a?a?b??fg? 822 a???h?e?h?gacb 338 b??ff?e?e?ba? 2. Execute the following queries to setup the table. a. CREATE TABLE hivetable(row INT, content STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY ' t' LOCATION '/hivetable'; b. LOAD DATA INPATH 'wasb:///data.txt' OVERWRITE INTO TABLE hivetable; 3. create a query file query.hql with following contents INSERT OVERWRITE DIRECTORY 'wasb:///hiveoutput' select * from hivetable where content like '%?%'; 4. even though few rows contains this character the output is empty. 5. change the contents of query.hql to INSERT OVERWRITE DIRECTORY 'wasb:///hiveoutput' select * from hivetable where content not like '%?%'; 6. The output contains all rows including those containing the given character. 7. Similar results are observed when using "where content = '?f??c??h?c?'; " 8. We get expected results when using "where content like '%a%'; " Diffs - cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 3cdedba Diff: https://reviews.apache.org/r/23907/diff/ Testing --- Tested, resolved the issue. Thanks, XIAOBING ZHOU
Re: [ANNOUNCE] New Hive PMC Member - Alan Gates
Alan, congrats! On Mon, Oct 27, 2014 at 3:45 PM, Xuefu Zhang wrote: > Congratulations, Alan! > > On Mon, Oct 27, 2014 at 3:43 PM, Matthew McCline > > wrote: > > > Congratulations! > > > > On Mon, Oct 27, 2014 at 3:38 PM, Carl Steinbach wrote: > > > > > I am pleased to announce that Alan Gates has been elected to the Hive > > > Project Management Committee. Please join me in congratulating Alan! > > > > > > Thanks. > > > > > > - Carl > > > > > > > -- > > CONFIDENTIALITY NOTICE > > NOTICE: This message is intended for the use of the individual or entity > to > > which it is addressed and may contain information that is confidential, > > privileged and exempt from disclosure under applicable law. If the reader > > of this message is not the intended recipient, you are hereby notified > that > > any printing, copying, dissemination, distribution, disclosure or > > forwarding of this communication is strictly prohibited. If you have > > received this communication in error, please contact the sender > immediately > > and delete it from your system. Thank You. > > > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Review Request 27313: Hive cannot load data into partitioned table with Unicode key
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27313/ --- Review request for hive. Repository: hive-git Description --- Steps to reproduce: 1) Copy the file partitioned.txt to the root folder of your HDFS root dir. Copy the two hql files to your local directory. 2) Open Hive CLI. 3) Run: hive> source ; 4) Run hive> source ; The following error will be shown: hive> source C:\Scripts\partition\LoadIntoPartitionedTable.hql; Loading data to table default.mypartitioned partition (tag=䶵) Failed with exception null FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask Diffs - metastore/scripts/upgrade/mssql/004-HIVE-8550.mssql.sql PRE-CREATION metastore/scripts/upgrade/mssql/upgrade-0.13.0-to-0.14.0.mssql.sql 1bda6d5 metastore/src/model/package.jdo f8936dc Diff: https://reviews.apache.org/r/27313/diff/ Testing --- 1. stop all services 2. drop HIVE DB 2. enable datanucleus.autoCreateSchema 3. start all services 4. run 004-HIVE-8550.mssql.sql It fixed the issue. Thanks, XIAOBING ZHOU
Re: Review Request 27313: Hive cannot load data into partitioned table with Unicode key
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27313/ --- (Updated Oct. 28, 2014, 10:27 p.m.) Review request for hive. Repository: hive-git Description --- Steps to reproduce: 1) Copy the file partitioned.txt to the root folder of your HDFS root dir. Copy the two hql files to your local directory. 2) Open Hive CLI. 3) Run: hive> source ; 4) Run hive> source ; The following error will be shown: hive> source C:\Scripts\partition\LoadIntoPartitionedTable.hql; Loading data to table default.mypartitioned partition (tag=䶵) Failed with exception null FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask Diffs (updated) - metastore/scripts/upgrade/mssql/004-HIVE-8550.mssql.sql PRE-CREATION metastore/scripts/upgrade/mssql/hive-schema-0.14.0.mssql.sql 174ed39 metastore/scripts/upgrade/mssql/upgrade-0.13.0-to-0.14.0.mssql.sql 1bda6d5 metastore/src/model/package.jdo f8936dc Diff: https://reviews.apache.org/r/27313/diff/ Testing --- 1. stop all services 2. drop HIVE DB 2. enable datanucleus.autoCreateSchema 3. start all services 4. run 004-HIVE-8550.mssql.sql It fixed the issue. Thanks, XIAOBING ZHOU
Re: Review Request 27313: Hive cannot load data into partitioned table with Unicode key
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27313/ --- (Updated Oct. 28, 2014, 11:25 p.m.) Review request for hive. Repository: hive-git Description --- Steps to reproduce: 1) Copy the file partitioned.txt to the root folder of your HDFS root dir. Copy the two hql files to your local directory. 2) Open Hive CLI. 3) Run: hive> source ; 4) Run hive> source ; The following error will be shown: hive> source C:\Scripts\partition\LoadIntoPartitionedTable.hql; Loading data to table default.mypartitioned partition (tag=䶵) Failed with exception null FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask Diffs - metastore/scripts/upgrade/mssql/004-HIVE-8550.mssql.sql PRE-CREATION metastore/scripts/upgrade/mssql/hive-schema-0.14.0.mssql.sql 174ed39 metastore/scripts/upgrade/mssql/upgrade-0.13.0-to-0.14.0.mssql.sql 1bda6d5 metastore/src/model/package.jdo f8936dc Diff: https://reviews.apache.org/r/27313/diff/ Testing (updated) --- Legacy system upgrading: 1. stop all services 2. drop HIVE DB 3. manually create HIVE DB 4. enable datanucleus.autoCreateSchema 5. start all services 6. all tables are created by autoCreateSchema 7. run 004-HIVE-8550.mssql.sql to do upgrade Fresh install: 1. stop all services 2. drop HIVE DB 3. manually create HIVE DB 4. run hive-schema-0.14.0.mssql.sql to create all tables 5. disable datanucleus.autoCreateSchema 6. start all services It fixed the issue. Thanks, XIAOBING ZHOU
Re: Review Request 27313: Hive cannot load data into partitioned table with Unicode key
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27313/ --- (Updated Oct. 28, 2014, 11:25 p.m.) Review request for hive. Repository: hive-git Description --- Steps to reproduce: 1) Copy the file partitioned.txt to the root folder of your HDFS root dir. Copy the two hql files to your local directory. 2) Open Hive CLI. 3) Run: hive> source ; 4) Run hive> source ; The following error will be shown: hive> source C:\Scripts\partition\LoadIntoPartitionedTable.hql; Loading data to table default.mypartitioned partition (tag=䶵) Failed with exception null FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask Diffs - metastore/scripts/upgrade/mssql/004-HIVE-8550.mssql.sql PRE-CREATION metastore/scripts/upgrade/mssql/hive-schema-0.14.0.mssql.sql 174ed39 metastore/scripts/upgrade/mssql/upgrade-0.13.0-to-0.14.0.mssql.sql 1bda6d5 metastore/src/model/package.jdo f8936dc Diff: https://reviews.apache.org/r/27313/diff/ Testing (updated) --- Legacy system upgrading: 1. stop all services 2. drop HIVE DB 3. manually create HIVE DB 4. enable datanucleus.autoCreateSchema 5. start all services 6. all tables are created by autoCreateSchema 7. run 004-HIVE-8550.mssql.sql to do upgrade Fresh install: 1. stop all services 2. drop HIVE DB 3. manually create HIVE DB 4. run hive-schema-0.14.0.mssql.sql to create all tables 5. disable datanucleus.autoCreateSchema 6. start all services It fixed the issue. Thanks, XIAOBING ZHOU
Review Request 27602: setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles on Windows
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27602/ --- Review request for hive. Repository: hive-git Description --- repro: mvn test -Phadoop-2 -Dmaven.surefire.debug -Dtest=TestCliDriverMethods#testprocessInitFiles setEnv tries to do JVM wide system variables changes, previous approach is not portable. Diffs - cli/src/test/org/apache/hadoop/hive/cli/TestCliDriverMethods.java 63668bc Diff: https://reviews.apache.org/r/27602/diff/ Testing --- Tested on both Linux and Windows. Fixed the issue. Thanks, XIAOBING ZHOU
Re: Review Request 27602: setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles on Windows
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27602/ --- (Updated Nov. 5, 2014, 1:07 a.m.) Review request for hive. Repository: hive-git Description --- repro: mvn test -Phadoop-2 -Dmaven.surefire.debug -Dtest=TestCliDriverMethods#testprocessInitFiles setEnv tries to do JVM wide system variables changes, previous approach is not portable. Diffs (updated) - cli/src/test/org/apache/hadoop/hive/cli/TestCliDriverMethods.java 63668bc Diff: https://reviews.apache.org/r/27602/diff/ Testing --- Tested on both Linux and Windows. Fixed the issue. Thanks, XIAOBING ZHOU
Re: Review Request 27602: setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles on Windows
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27602/ --- (Updated Nov. 5, 2014, 11:47 p.m.) Review request for hive. Repository: hive-git Description --- repro: mvn test -Phadoop-2 -Dmaven.surefire.debug -Dtest=TestCliDriverMethods#testprocessInitFiles setEnv tries to do JVM wide system variables changes, previous approach is not portable. Diffs (updated) - cli/src/test/org/apache/hadoop/hive/cli/TestCliDriverMethods.java 63668bc Diff: https://reviews.apache.org/r/27602/diff/ Testing --- Tested on both Linux and Windows. Fixed the issue. Thanks, XIAOBING ZHOU
Review Request 30335: Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- Review request for hive. Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 23d77ca ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDayBase.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFirstDay.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLastDay.java 911ee46 ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in GenericUDFFirstDay. Thanks, XIAOBING ZHOU
Re: Review Request 30335: Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Jan. 27, 2015, 10:53 p.m.) Review request for hive. Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 23d77ca ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDayBase.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFirstDay.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLastDay.java 911ee46 ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in GenericUDFFirstDay. Thanks, XIAOBING ZHOU
Review Request 30336: Make Hive support unicode with MSSQL as Metastore backend
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30336/ --- Review request for hive. Repository: hive-git Description --- There are significant issues when Hive uses MSSQL as metastore backend to support unicode, since MSSQL handles varchar and nvarchar datatypes differently. Hive 0.14 metastore mssql script DDL was using varchar as datatype, which can't handle multi-bytes/unicode characters, e.g., Chinese chars. This JIRA is going to track implementation of unicode support in that case. Diffs - metastore/scripts/upgrade/mssql/006-HIVE-9456.mssql.sql PRE-CREATION metastore/scripts/upgrade/mssql/hive-schema-0.15.0.mssql.sql 7ca9b24 metastore/scripts/upgrade/mssql/upgrade-0.14.0-to-0.15.0.mssql.sql 36a5210 Diff: https://reviews.apache.org/r/30336/diff/ Testing --- 1. New empty metastore's created with 0.14 schema. 2. upgrade-0.14.0-to-0.15.0.mssql.sql is run manually to get 0.14 metastore upgraded to 0.15. 3. Double check it to make sure every orginal varchar column changed to nvarchar. Thanks, XIAOBING ZHOU
Re: Review Request 30336: Make Hive support unicode with MSSQL as Metastore backend
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30336/ --- (Updated Jan. 27, 2015, 11:05 p.m.) Review request for hive. Repository: hive-git Description (updated) --- There are significant issues when Hive uses MSSQL as metastore backend to support unicode, since MSSQL handles varchar and nvarchar datatypes differently. Hive 0.14 metastore mssql script DDL was using varchar as datatype, which can't handle multi-bytes/unicode characters, e.g., Chinese chars. This JIRA is going to track implementation of unicode support in that case. https://issues.apache.org/jira/browse/HIVE-9456 Diffs - metastore/scripts/upgrade/mssql/006-HIVE-9456.mssql.sql PRE-CREATION metastore/scripts/upgrade/mssql/hive-schema-0.15.0.mssql.sql 7ca9b24 metastore/scripts/upgrade/mssql/upgrade-0.14.0-to-0.15.0.mssql.sql 36a5210 Diff: https://reviews.apache.org/r/30336/diff/ Testing --- 1. New empty metastore's created with 0.14 schema. 2. upgrade-0.14.0-to-0.15.0.mssql.sql is run manually to get 0.14 metastore upgraded to 0.15. 3. Double check it to make sure every orginal varchar column changed to nvarchar. Thanks, XIAOBING ZHOU
Re: Review Request 30335: Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Jan. 27, 2015, 11:06 p.m.) Review request for hive. Repository: hive-git Description (updated) --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 23d77ca ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDayBase.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFirstDay.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLastDay.java 911ee46 ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in GenericUDFFirstDay. Thanks, XIAOBING ZHOU
Review Request 30437: Implement MONTHS_BETWEEN aligned with Oracle one
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30437/ --- Review request for hive. Repository: hive-git Description --- This is used to track work to build Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. https://issues.apache.org/jira/browse/HIVE-9518 Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 23d77ca ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateDiff.java 1ecd835 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateDiffBase.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMonthsBetween.java PRE-CREATION ql/src/test/TestGenericUDFMonthsBetween.java PRE-CREATION Diff: https://reviews.apache.org/r/30437/diff/ Testing --- SEE ALSO ql/src/test/TestGenericUDFMonthsBetween.java Thanks, XIAOBING ZHOU
Re: Review Request 30335: Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Jan. 30, 2015, 2:03 a.m.) Review request for hive. Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 23d77ca ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDayBase.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFirstDay.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLastDay.java 911ee46 ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in GenericUDFFirstDay. Thanks, XIAOBING ZHOU
Re: Review Request 30335: Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Jan. 30, 2015, 2:04 a.m.) Review request for hive. Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 23d77ca ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDayBase.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFirstDay.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLastDay.java 911ee46 ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing (updated) --- Unit tests done in ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java Thanks, XIAOBING ZHOU
Re: Review Request 30437: Implement MONTHS_BETWEEN aligned with Oracle one
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30437/ --- (Updated Feb. 4, 2015, 10:30 p.m.) Review request for hive. Repository: hive-git Description --- This is used to track work to build Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. https://issues.apache.org/jira/browse/HIVE-9518 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfb4dc2 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateDiff.java 1ecd835 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateDiffBase.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMonthsBetween.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFMonthsBetween.java PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error2.q PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error3.q PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error4.q PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error5.q PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error6.q PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error7.q PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error8.q PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error9.q PRE-CREATION ql/src/test/queries/clientpositive/udf_months_between.q PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error2.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error3.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error4.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error5.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error6.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error7.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error8.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error9.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out e21b54b ql/src/test/results/clientpositive/udf_months_between.q.out PRE-CREATION Diff: https://reviews.apache.org/r/30437/diff/ Testing --- SEE ALSO ql/src/test/TestGenericUDFMonthsBetween.java Thanks, XIAOBING ZHOU
Re: Review Request 30335: Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Feb. 5, 2015, 11:27 p.m.) Review request for hive. Summary (updated) - Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfb4dc2 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrunc.java PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error2.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error3.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error4.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error5.q PRE-CREATION ql/src/test/queries/clientpositive/udf_trunc.q PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error2.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error3.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error4.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error5.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out e21b54b ql/src/test/results/clientpositive/udf_trunc.q.out PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java Thanks, XIAOBING ZHOU
Re: Review Request 30335: Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Feb. 5, 2015, 11:27 p.m.) Review request for hive. Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfb4dc2 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrunc.java PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error2.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error3.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error4.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error5.q PRE-CREATION ql/src/test/queries/clientpositive/udf_trunc.q PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error2.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error3.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error4.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error5.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out e21b54b ql/src/test/results/clientpositive/udf_trunc.q.out PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java Thanks, XIAOBING ZHOU
Re: Review Request 30335: Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Feb. 5, 2015, 11:33 p.m.) Review request for hive. Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfb4dc2 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrunc.java PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error2.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error3.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error4.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error5.q PRE-CREATION ql/src/test/queries/clientpositive/udf_trunc.q PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error2.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error3.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error4.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error5.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out e21b54b ql/src/test/results/clientpositive/udf_trunc.q.out PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java Thanks, XIAOBING ZHOU
Re: Review Request 30335: Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Feb. 20, 2015, 1:18 a.m.) Review request for hive. Changes --- Patch V5 Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfb4dc2 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrunc.java PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error2.q PRE-CREATION ql/src/test/queries/clientpositive/udf_trunc.q PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error2.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out e21b54b ql/src/test/results/clientpositive/udf_trunc.q.out PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java Thanks, XIAOBING ZHOU
Re: Review Request 30335: Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Feb. 21, 2015, 12:49 a.m.) Review request for hive. Changes --- patch V6, did a rebase. Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfeb33c ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrunc.java PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error2.q PRE-CREATION ql/src/test/queries/clientpositive/udf_trunc.q PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error2.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out d4b0650 ql/src/test/results/clientpositive/udf_trunc.q.out PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java Thanks, XIAOBING ZHOU
Review Request 31249: HIVE-9741: Refactor MetaStoreDirectSql by using getProductName instead of querying DB to determine DbType
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31249/ --- Review request for hive. Repository: hive-git Description --- MetaStoreDirectSql constructor is querying DB to determine dbType. which leads to too many DB queries to make metastore slow or hanging if MetaStoreDirectSql constructor is frequently called. This is to propose getProductName to get dbType info. Diffs - metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 574141c Diff: https://reviews.apache.org/r/31249/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 30335: Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Feb. 21, 2015, 2:54 a.m.) Review request for hive. Changes --- V7 addressed latest comments. Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfeb33c ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrunc.java PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error2.q PRE-CREATION ql/src/test/queries/clientpositive/udf_trunc.q PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error2.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out d4b0650 ql/src/test/results/clientpositive/udf_trunc.q.out PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java Thanks, XIAOBING ZHOU
Re: Review Request 31249: Refactor MetaStoreDirectSql constructor
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31249/ --- (Updated Feb. 23, 2015, 10:08 p.m.) Review request for hive. Changes --- Two points: 1. Using getProductName based JDBC driver to get dbType info. 2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit() and runTestQuery() in order to avoid DB queries within critical section of setConf. Summary (updated) - Refactor MetaStoreDirectSql constructor Repository: hive-git Description (updated) --- MetaStoreDirectSql constructor is querying DB to determine dbType, which leads to too many DB queries to make megastore slow as ObjectStore.setConf might be called frequently. Moreover, ObjectStore.setConf begins/ends with lock acquire/release, if the underlying DB hangs somehow, lock is never released and all hereafter incoming requests are blocked. Two points: 1. Using getProductName based JDBC driver to get dbType info. 2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit() and runTestQuery() in order to avoid DB queries within critical section of setConf. Here’s stack trace: MetaStoreDirectSql.determineDbType(...) MetaStoreDirectSql.MetaStoreDirectSql(...) ObjectStore.initialize(...) ObjectStore.setConf(…) Diffs (updated) - metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 574141c metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java fcaffc7 Diff: https://reviews.apache.org/r/31249/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31249/ --- (Updated Feb. 24, 2015, 12:49 a.m.) Review request for hive. Summary (updated) - Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section Repository: hive-git Description --- MetaStoreDirectSql constructor is querying DB to determine dbType, which leads to too many DB queries to make megastore slow as ObjectStore.setConf might be called frequently. Moreover, ObjectStore.setConf begins/ends with lock acquire/release, if the underlying DB hangs somehow, lock is never released and all hereafter incoming requests are blocked. Two points: 1. Using getProductName based JDBC driver to get dbType info. 2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit() and runTestQuery() in order to avoid DB queries within critical section of setConf. Here’s stack trace: MetaStoreDirectSql.determineDbType(...) MetaStoreDirectSql.MetaStoreDirectSql(...) ObjectStore.initialize(...) ObjectStore.setConf(…) Diffs - metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 574141c metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java fcaffc7 Diff: https://reviews.apache.org/r/31249/diff/ Testing --- Thanks, XIAOBING ZHOU
review HIVE-9741: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section
Hi team, Can you review the latest patch? https://issues.apache.org/jira/browse/HIVE-9741 https://reviews.apache.org/r/31249/
Review Request 31333: Killed job is not displayed correctly by failure hook
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31333/ --- Review request for hive. Repository: hive-git Description --- PROBLEM: After killing the job throug CLI by crtl+c, we are seeing the following exception. Also the job is shown as running with timer still counting on the UI Interrupting... Be patient, this might take some time. Press Ctrl+C again to kill JVM Map 1: 20/40 Reducer 2: 0/3 Reducer 3: 0/1 Map 1: 22/40 Reducer 2: 0/3 Reducer 3: 0/1 Map 1: 25/40 Reducer 2: 0/3 Reducer 3: 0/1 Map 1: 26/40 Reducer 2: 0/3 Reducer 3: 0/1 Exiting the JVM Trying to shutdown DAG Shutting down tez session. Status: Killed Job received Kill while in RUNNING state. Vertex killed, vertexName=Reducer 3, vertexId=vertex_1421602925091_3868_2_00, diagnostics=Vertex received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0 Vertex killed, vertexName=Map 1, vertexId=vertex_1421602925091_3868_2_02, diagnostics=Vertex received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0 Vertex killed, vertexName=Reducer 2, vertexId=vertex_1421602925091_3868_2_01, diagnostics=Vertex received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0 DAG killed due to user-initiated kill. failedVertices:0 killedVertices:3 FAILED: Hive Internal Error: java.lang.IllegalStateException(Shutdown in progress) java.lang.IllegalStateException: Shutdown in progress at java.lang.ApplicationShutdownHooks.add(ApplicationShutdownHooks.java:66) at java.lang.Runtime.addShutdownHook(Runtime.java:211) at org.apache.hadoop.hive.ql.hooks.ATSHook.(ATSHook.java:71) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at java.lang.Class.newInstance(Class.java:374) at org.apache.hadoop.hive.ql.hooks.HookUtils.getHooks(HookUtils.java:59) at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1177) at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1161) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1316) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1093) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) hive> EL Info: 2015-01-21 16:20:51.483-ServerSession(1006268460)-EclipseLink, version: Eclipse Persistence Services - 2.5.1.v20130918-f2b9fc5 EL Info: connection: 2015-01-21 16:20:52.025-ServerSession(1006268460)-file:/usr/lib/hive/lib/xasecure-audit.jar_xa_server_url=jdbc:mysql://bkcttplpd036.verizon.com/xasecure_user=xalogger login successful EL Info: connection: 2015-01-21 16:20:52.082-ServerSession(1006268460)-file:/usr/lib/hive/lib/xasecure-audit.jar_xa_server_url=jdbc:mysql://bkcttplpd036.verizon.com/xasecure_user=xalogger logout successful Diffs - ql/src/java/org/apache/hadoop/hive/ql/hooks/ATSHook.java 513a2fa Diff: https://reviews.apache.org/r/31333/diff/ Testing --- Thanks, XIAOBING ZHOU
can you review HIVE-9760: Killed job is not displayed correctly by failure hook
Hi team, Can anyone review this patch V1? Thanks! https://issues.apache.org/jira/browse/HIVE-9760 https://reviews.apache.org/r/31333/
Re: Review Request 31333: Killed job is not display correctly by ATS hook failure
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31333/ --- (Updated Feb. 24, 2015, 1:19 a.m.) Review request for hive. Summary (updated) - Killed job is not display correctly by ATS hook failure Repository: hive-git Description --- PROBLEM: After killing the job throug CLI by crtl+c, we are seeing the following exception. Also the job is shown as running with timer still counting on the UI Interrupting... Be patient, this might take some time. Press Ctrl+C again to kill JVM Map 1: 20/40 Reducer 2: 0/3 Reducer 3: 0/1 Map 1: 22/40 Reducer 2: 0/3 Reducer 3: 0/1 Map 1: 25/40 Reducer 2: 0/3 Reducer 3: 0/1 Map 1: 26/40 Reducer 2: 0/3 Reducer 3: 0/1 Exiting the JVM Trying to shutdown DAG Shutting down tez session. Status: Killed Job received Kill while in RUNNING state. Vertex killed, vertexName=Reducer 3, vertexId=vertex_1421602925091_3868_2_00, diagnostics=Vertex received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0 Vertex killed, vertexName=Map 1, vertexId=vertex_1421602925091_3868_2_02, diagnostics=Vertex received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0 Vertex killed, vertexName=Reducer 2, vertexId=vertex_1421602925091_3868_2_01, diagnostics=Vertex received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0 DAG killed due to user-initiated kill. failedVertices:0 killedVertices:3 FAILED: Hive Internal Error: java.lang.IllegalStateException(Shutdown in progress) java.lang.IllegalStateException: Shutdown in progress at java.lang.ApplicationShutdownHooks.add(ApplicationShutdownHooks.java:66) at java.lang.Runtime.addShutdownHook(Runtime.java:211) at org.apache.hadoop.hive.ql.hooks.ATSHook.(ATSHook.java:71) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at java.lang.Class.newInstance(Class.java:374) at org.apache.hadoop.hive.ql.hooks.HookUtils.getHooks(HookUtils.java:59) at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1177) at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1161) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1316) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1093) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) hive> EL Info: 2015-01-21 16:20:51.483-ServerSession(1006268460)-EclipseLink, version: Eclipse Persistence Services - 2.5.1.v20130918-f2b9fc5 EL Info: connection: 2015-01-21 16:20:52.025-ServerSession(1006268460)-file:/usr/lib/hive/lib/xasecure-audit.jar_xa_server_url=jdbc:mysql://bkcttplpd036.verizon.com/xasecure_user=xalogger login successful EL Info: connection: 2015-01-21 16:20:52.082-ServerSession(1006268460)-file:/usr/lib/hive/lib/xasecure-audit.jar_xa_server_url=jdbc:mysql://bkcttplpd036.verizon.com/xasecure_user=xalogger logout successful Diffs - ql/src/java/org/apache/hadoop/hive/ql/hooks/ATSHook.java 513a2fa Diff: https://reviews.apache.org/r/31333/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31249/ --- (Updated Feb. 24, 2015, 10:24 p.m.) Review request for hive. Changes --- moved setDbANSIMode call to MetaStoreDirectSql Repository: hive-git Description --- MetaStoreDirectSql constructor is querying DB to determine dbType, which leads to too many DB queries to make megastore slow as ObjectStore.setConf might be called frequently. Moreover, ObjectStore.setConf begins/ends with lock acquire/release, if the underlying DB hangs somehow, lock is never released and all hereafter incoming requests are blocked. Two points: 1. Using getProductName based JDBC driver to get dbType info. 2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit() and runTestQuery() in order to avoid DB queries within critical section of setConf. Here’s stack trace: MetaStoreDirectSql.determineDbType(...) MetaStoreDirectSql.MetaStoreDirectSql(...) ObjectStore.initialize(...) ObjectStore.setConf(…) Diffs (updated) - metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 574141c Diff: https://reviews.apache.org/r/31249/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31249/ --- (Updated Feb. 24, 2015, 10:34 p.m.) Review request for hive. Changes --- removed empty spaces. Repository: hive-git Description --- MetaStoreDirectSql constructor is querying DB to determine dbType, which leads to too many DB queries to make megastore slow as ObjectStore.setConf might be called frequently. Moreover, ObjectStore.setConf begins/ends with lock acquire/release, if the underlying DB hangs somehow, lock is never released and all hereafter incoming requests are blocked. Two points: 1. Using getProductName based JDBC driver to get dbType info. 2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit() and runTestQuery() in order to avoid DB queries within critical section of setConf. Here’s stack trace: MetaStoreDirectSql.determineDbType(...) MetaStoreDirectSql.MetaStoreDirectSql(...) ObjectStore.initialize(...) ObjectStore.setConf(…) Diffs (updated) - metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 574141c Diff: https://reviews.apache.org/r/31249/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 31333: Killed job is not display correctly by ATS hook failure
ve/ql/lockmgr/zookeeper/CuratorFrameworkSingleton.java fbf2a01 service/src/java/org/apache/hive/service/server/HiveServer2.java 21025a2 testutils/ptest2/src/main/java/org/apache/hive/ptest/api/server/ExecutionController.java 279cd72 Diff: https://reviews.apache.org/r/31333/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 31333: Killed job is not display correctly by ATS hook failure
ing --- Thanks, XIAOBING ZHOU
Review Request 31395: HIVE-9642: Hive metastore client retries don't happen consistently for all api calls
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31395/ --- Review request for hive. Repository: hive-git Description --- When org.apache.thrift.transport.TTransportException is thrown for issues like socket timeout, the retry via RetryingMetaStoreClient happens only in certain cases. Retry happens for the getDatabase call in but not for getAllDatabases(). The reason is RetryingMetaStoreClient checks for TTransportException being the cause for InvocationTargetException. But in case of some calls such as getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a MetaException. We should remove this unnecessary wrapping of exceptions for certain functions in HMC. Diffs - metastore/src/java/org/apache/hadoop/hive/metastore/RetryingMetaStoreClient.java b4f02fc Diff: https://reviews.apache.org/r/31395/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 31395: HIVE-9642: Hive metastore client retries don't happen consistently for all api calls
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31395/ --- (Updated Feb. 25, 2015, 7:23 p.m.) Review request for hive. Changes --- Patch V2. removd the MetaException Wrapping Repository: hive-git Description --- When org.apache.thrift.transport.TTransportException is thrown for issues like socket timeout, the retry via RetryingMetaStoreClient happens only in certain cases. Retry happens for the getDatabase call in but not for getAllDatabases(). The reason is RetryingMetaStoreClient checks for TTransportException being the cause for InvocationTargetException. But in case of some calls such as getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a MetaException. We should remove this unnecessary wrapping of exceptions for certain functions in HMC. Diffs (updated) - metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 4778bb6 Diff: https://reviews.apache.org/r/31395/diff/ Testing --- Thanks, XIAOBING ZHOU
Call for review on HIVE-9642: Hive metastore client retries don't happen consistently for all api calls
Hi team, Could anyone review HIVE-9642: Hive metastore client retries don't happen consistently for all api calls? Thanks! https://issues.apache.org/jira/browse/HIVE-9642 https://reviews.apache.org/r/31395/
Re: Review Request 31333: Killed job is not display correctly by ATS hook failure
tps://reviews.apache.org/r/31333/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31249/ --- (Updated Feb. 25, 2015, 9:24 p.m.) Review request for hive. Repository: hive-git Description (updated) --- MetaStoreDirectSql constructor is querying DB to determine dbType, which leads to too many DB queries to make megastore slow as ObjectStore.setConf might be called frequently. Moreover, ObjectStore.setConf begins/ends with lock acquire/release, if the underlying DB hangs somehow, lock is never released and all hereafter incoming requests are blocked. Two points: 1. Using getProductName based JDBC driver to get dbType info. 2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit() and runTestQuery() in order to avoid DB queries within critical section of setConf. Here’s stack trace: MetaStoreDirectSql.determineDbType(...) MetaStoreDirectSql.MetaStoreDirectSql(...) ObjectStore.initialize(...) ObjectStore.setConf(…) https://issues.apache.org/jira/browse/HIVE-9741 Diffs - metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 574141c Diff: https://reviews.apache.org/r/31249/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31249/ --- (Updated Feb. 25, 2015, 9:37 p.m.) Review request for hive. Changes --- 1. moved setDbANSIMode call 2. removed logging in comment Repository: hive-git Description --- MetaStoreDirectSql constructor is querying DB to determine dbType, which leads to too many DB queries to make megastore slow as ObjectStore.setConf might be called frequently. Moreover, ObjectStore.setConf begins/ends with lock acquire/release, if the underlying DB hangs somehow, lock is never released and all hereafter incoming requests are blocked. Two points: 1. Using getProductName based JDBC driver to get dbType info. 2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit() and runTestQuery() in order to avoid DB queries within critical section of setConf. Here’s stack trace: MetaStoreDirectSql.determineDbType(...) MetaStoreDirectSql.MetaStoreDirectSql(...) ObjectStore.initialize(...) ObjectStore.setConf(…) https://issues.apache.org/jira/browse/HIVE-9741 Diffs (updated) - metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 574141c Diff: https://reviews.apache.org/r/31249/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 30335: Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
> On Feb. 20, 2015, 9:21 p.m., Jason Dere wrote: > > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java, > > line 94 > > <https://reviews.apache.org/r/30335/diff/5/?file=870059#file870059line94> > > > > We do this same check in so many different date functions .. we should > > eventually add a utility method to do this date type parameter checking. We > > don't have to do it in this Jira, it can be future work. > > Alexander Pivovarov wrote: > working on it https://issues.apache.org/jira/browse/HIVE-9744 add relates-to link to HIVE-9744 - XIAOBING --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/#review73309 ------- On Feb. 25, 2015, 11:35 p.m., XIAOBING ZHOU wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/30335/ > --- > > (Updated Feb. 25, 2015, 11:35 p.m.) > > > Review request for hive. > > > Repository: hive-git > > > Description > --- > > Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to > do date/timestamp related computation. This JIRA is to track such an > implementation. > > https://issues.apache.org/jira/browse/HIVE-9480 > > > Diffs > - > > ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfeb33c > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java > PRE-CREATION > ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrunc.java > PRE-CREATION > ql/src/test/queries/clientnegative/udf_trunc_error1.q PRE-CREATION > ql/src/test/queries/clientnegative/udf_trunc_error2.q PRE-CREATION > ql/src/test/queries/clientpositive/udf_trunc.q PRE-CREATION > ql/src/test/results/clientnegative/udf_trunc_error1.q.out PRE-CREATION > ql/src/test/results/clientnegative/udf_trunc_error2.q.out PRE-CREATION > ql/src/test/results/clientpositive/show_functions.q.out d4b0650 > ql/src/test/results/clientpositive/udf_trunc.q.out PRE-CREATION > > Diff: https://reviews.apache.org/r/30335/diff/ > > > Testing > --- > > Unit tests done in > ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java > > > Thanks, > > XIAOBING ZHOU > >
Re: Review Request 30335: Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Feb. 25, 2015, 11:35 p.m.) Review request for hive. Changes --- patch V8 Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfeb33c ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrunc.java PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error2.q PRE-CREATION ql/src/test/queries/clientpositive/udf_trunc.q PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error2.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out d4b0650 ql/src/test/results/clientpositive/udf_trunc.q.out PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java Thanks, XIAOBING ZHOU
Re: Review Request 30335: Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
> On Feb. 20, 2015, 1:38 a.m., Alexander Pivovarov wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java,
> > line 143
> > <https://reviews.apache.org/r/30335/diff/5/?file=870059#file870059line143>
> >
> > In 99% of the cases arg[1] will be constant
> > you can get arg[1] avalue in initialize() instead of doing it in
> > evaluate() for each row
> >
> > if (arguments[1] instanceof ConstantObjectInspector) {
> > String fmt = ((ConstantObjectInspector)
> > arguments[1]).getWritableConstantValue()
> > .toString();
> > }
e.g. WritableTimestampObjectInspector is not ConstantObjectInspector. So init
once in evaluate function
- XIAOBING
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30335/#review73240
---
On Feb. 25, 2015, 11:35 p.m., XIAOBING ZHOU wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30335/
> ---
>
> (Updated Feb. 25, 2015, 11:35 p.m.)
>
>
> Review request for hive.
>
>
> Repository: hive-git
>
>
> Description
> ---
>
> Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
> do date/timestamp related computation. This JIRA is to track such an
> implementation.
>
> https://issues.apache.org/jira/browse/HIVE-9480
>
>
> Diffs
> -
>
> ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfeb33c
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java
> PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrunc.java
> PRE-CREATION
> ql/src/test/queries/clientnegative/udf_trunc_error1.q PRE-CREATION
> ql/src/test/queries/clientnegative/udf_trunc_error2.q PRE-CREATION
> ql/src/test/queries/clientpositive/udf_trunc.q PRE-CREATION
> ql/src/test/results/clientnegative/udf_trunc_error1.q.out PRE-CREATION
> ql/src/test/results/clientnegative/udf_trunc_error2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/show_functions.q.out d4b0650
> ql/src/test/results/clientpositive/udf_trunc.q.out PRE-CREATION
>
> Diff: https://reviews.apache.org/r/30335/diff/
>
>
> Testing
> ---
>
> Unit tests done in
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java
>
>
> Thanks,
>
> XIAOBING ZHOU
>
>
Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31249/ --- (Updated Feb. 25, 2015, 11:58 p.m.) Review request for hive. Changes --- remved setDbANSIMode Repository: hive-git Description --- MetaStoreDirectSql constructor is querying DB to determine dbType, which leads to too many DB queries to make megastore slow as ObjectStore.setConf might be called frequently. Moreover, ObjectStore.setConf begins/ends with lock acquire/release, if the underlying DB hangs somehow, lock is never released and all hereafter incoming requests are blocked. Two points: 1. Using getProductName based JDBC driver to get dbType info. 2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit() and runTestQuery() in order to avoid DB queries within critical section of setConf. Here’s stack trace: MetaStoreDirectSql.determineDbType(...) MetaStoreDirectSql.MetaStoreDirectSql(...) ObjectStore.initialize(...) ObjectStore.setConf(…) https://issues.apache.org/jira/browse/HIVE-9741 Diffs (updated) - metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 574141c Diff: https://reviews.apache.org/r/31249/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: [ANNOUNCE] New Hive PMC Member - Sergey Shelukhin
Congrats Sergey! On Feb 25, 2015, at 1:56 PM, Prasanth Jayachandran wrote: > Congrats Sergey! > > On Feb 25, 2015, at 1:50 PM, Alexander Pivovarov > mailto:apivova...@gmail.com>> wrote: > > Congrats! > > On Wed, Feb 25, 2015 at 12:33 PM, Vaibhav Gumashta > mailto:vgumas...@hortonworks.com>> wrote: > Congrats Sergey! > > On 2/25/15, 9:06 AM, "Vikram Dixit" > mailto:vik...@hortonworks.com>> wrote: > >> Congrats Sergey! >> >> On 2/25/15, 8:43 AM, "Carl Steinbach" >> mailto:c...@apache.org>> wrote: >> >>> I am pleased to announce that Sergey Shelukhin has been elected to the >>> Hive >>> Project Management Committee. Please join me in congratulating Sergey! >>> >>> Thanks. >>> >>> - Carl >> > > >
Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31249/ --- (Updated Feb. 26, 2015, 8:53 p.m.) Review request for hive. Changes --- added doDbSpecificInitializationsBeforeQuery in runTestQuery Repository: hive-git Description --- MetaStoreDirectSql constructor is querying DB to determine dbType, which leads to too many DB queries to make megastore slow as ObjectStore.setConf might be called frequently. Moreover, ObjectStore.setConf begins/ends with lock acquire/release, if the underlying DB hangs somehow, lock is never released and all hereafter incoming requests are blocked. Two points: 1. Using getProductName based JDBC driver to get dbType info. 2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit() and runTestQuery() in order to avoid DB queries within critical section of setConf. Here’s stack trace: MetaStoreDirectSql.determineDbType(...) MetaStoreDirectSql.MetaStoreDirectSql(...) ObjectStore.initialize(...) ObjectStore.setConf(…) https://issues.apache.org/jira/browse/HIVE-9741 Diffs (updated) - metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 574141c Diff: https://reviews.apache.org/r/31249/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 30437: Implement MONTHS_BETWEEN aligned with Oracle one
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30437/ --- (Updated Feb. 27, 2015, 6:45 p.m.) Review request for hive. Changes --- patch V4, addressed most recent comments. Repository: hive-git Description --- This is used to track work to build Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. https://issues.apache.org/jira/browse/HIVE-9518 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfeb33c ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateDiff.java 403cf11 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateDiffBase.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMonthsBetween.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFMonthsBetween.java PRE-CREATION ql/src/test/queries/clientnegative/udf_datediff_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_datediff_error2.q PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error2.q PRE-CREATION ql/src/test/queries/clientpositive/udf_datediff.q 30718fe ql/src/test/queries/clientpositive/udf_months_between.q PRE-CREATION ql/src/test/results/clientnegative/udf_datediff_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_datediff_error2.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error2.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out d4b0650 ql/src/test/results/clientpositive/show_functions.q.out.orig d79106f ql/src/test/results/clientpositive/udf_datediff.q.out 86f30f6 ql/src/test/results/clientpositive/udf_months_between.q.out PRE-CREATION Diff: https://reviews.apache.org/r/30437/diff/ Testing --- SEE ALSO ql/src/test/TestGenericUDFMonthsBetween.java Thanks, XIAOBING ZHOU
Re: Review Request 30335: Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Feb. 27, 2015, 7:22 p.m.) Review request for hive. Changes --- 1. removed commented code block 2. remove redundant code, arguments[1] != null Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfeb33c ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrunc.java PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error2.q PRE-CREATION ql/src/test/queries/clientpositive/udf_trunc.q PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error2.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out d4b0650 ql/src/test/results/clientpositive/udf_trunc.q.out PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java Thanks, XIAOBING ZHOU
[jira] [Updated] (HIVE-7297) org.apache.hadoop.hive.ql.WindowsPathUtil is not easy to be reused across different hive projects
[ https://issues.apache.org/jira/browse/HIVE-7297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-7297: Assignee: Hari Sankar Sivarama Subramaniyan (was: Xiaobing Zhou) > org.apache.hadoop.hive.ql.WindowsPathUtil is not easy to be reused across > different hive projects > - > > Key: HIVE-7297 > URL: https://issues.apache.org/jira/browse/HIVE-7297 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure, Tests >Affects Versions: 0.13.0 > Reporter: Xiaobing Zhou >Assignee: Hari Sankar Sivarama Subramaniyan >Priority: Minor > > WindowsPathUtil converts Windows path notation(c:\Uses\xzhou) to HDFS > compatible Unix notation(/Users/xzhou). It was used in several places, will > be reused by other fixes like patches for HIVE-7276 and HIVE-7295. There's > need to change those influenced projects pom.xml to include dependency. Or > could pushed WindowsPathUtil down to hive-exec library code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-8115) Hive select query hang when fields contain map
Xiaobing Zhou created HIVE-8115: --- Summary: Hive select query hang when fields contain map Key: HIVE-8115 URL: https://issues.apache.org/jira/browse/HIVE-8115 Project: Hive Issue Type: Bug Affects Versions: 0.13.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Fix For: 0.14.0 Attached the repro of the issue. When creating an table loading the data attached, all hive query with hangs even just select * from the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8115) Hive select query hang when fields contain map
[ https://issues.apache.org/jira/browse/HIVE-8115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-8115: Attachment: createTable.hql data > Hive select query hang when fields contain map > -- > > Key: HIVE-8115 > URL: https://issues.apache.org/jira/browse/HIVE-8115 > Project: Hive > Issue Type: Bug >Affects Versions: 0.13.0 > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > Fix For: 0.14.0 > > Attachments: createTable.hql, data > > > Attached the repro of the issue. When creating an table loading the data > attached, all hive query with hangs even just select * from the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8115) Hive select query hang when fields contain map
[ https://issues.apache.org/jira/browse/HIVE-8115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-8115: Attachment: (was: createTable.hql) > Hive select query hang when fields contain map > -- > > Key: HIVE-8115 > URL: https://issues.apache.org/jira/browse/HIVE-8115 > Project: Hive > Issue Type: Bug >Affects Versions: 0.13.0 > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > Fix For: 0.14.0 > > Attachments: createTable.hql, data > > > Attached the repro of the issue. When creating an table loading the data > attached, all hive query with hangs even just select * from the table. > repro steps: > 1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8115) Hive select query hang when fields contain map
[ https://issues.apache.org/jira/browse/HIVE-8115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-8115: Attachment: createTable.hql > Hive select query hang when fields contain map > -- > > Key: HIVE-8115 > URL: https://issues.apache.org/jira/browse/HIVE-8115 > Project: Hive > Issue Type: Bug >Affects Versions: 0.13.0 > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > Fix For: 0.14.0 > > Attachments: createTable.hql, data > > > Attached the repro of the issue. When creating an table loading the data > attached, all hive query with hangs even just select * from the table. > repro steps: > 1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8115) Hive select query hang when fields contain map
[ https://issues.apache.org/jira/browse/HIVE-8115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-8115: Description: Attached the repro of the issue. When creating an table loading the data attached, all hive query with hangs even just select * from the table. repro steps: 1. was:Attached the repro of the issue. When creating an table loading the data attached, all hive query with hangs even just select * from the table. > Hive select query hang when fields contain map > -- > > Key: HIVE-8115 > URL: https://issues.apache.org/jira/browse/HIVE-8115 > Project: Hive > Issue Type: Bug >Affects Versions: 0.13.0 > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > Fix For: 0.14.0 > > Attachments: createTable.hql, data > > > Attached the repro of the issue. When creating an table loading the data > attached, all hive query with hangs even just select * from the table. > repro steps: > 1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8115) Hive select query hang when fields contain map
[ https://issues.apache.org/jira/browse/HIVE-8115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-8115: Description: Attached the repro of the issue. When creating an table loading the data attached, all hive query with hangs even just select * from the table. repro steps: 1. run createTable.hql 2. hadoop fs ls -put data /data 3. LOAD DATA INPATH '/data' OVERWRITE INTO TABLE testtable; 4. SELECT * FROM testtable; was: Attached the repro of the issue. When creating an table loading the data attached, all hive query with hangs even just select * from the table. repro steps: 1. > Hive select query hang when fields contain map > -- > > Key: HIVE-8115 > URL: https://issues.apache.org/jira/browse/HIVE-8115 > Project: Hive > Issue Type: Bug >Affects Versions: 0.13.0 >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Fix For: 0.14.0 > > Attachments: createTable.hql, data > > > Attached the repro of the issue. When creating an table loading the data > attached, all hive query with hangs even just select * from the table. > repro steps: > 1. run createTable.hql > 2. hadoop fs ls -put data /data > 3. LOAD DATA INPATH '/data' OVERWRITE INTO TABLE testtable; > 4. SELECT * FROM testtable; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8115) Hive select query hang when fields contain map
[ https://issues.apache.org/jira/browse/HIVE-8115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14134807#comment-14134807 ] Xiaobing Zhou commented on HIVE-8115: - wedcsProperties column starts with [SOH][ETX][STX], where SOH is start of heading with value '\001' in terms of OCT, ETX is end of text with end of text with value '\003' , and STX is start of text with value '\002'. [STX][ETX] is the format good for map's key/value pair representation. In the data file, [STX][ETX] means empty map pair, this will make query hanging forever by complaining nothing. Of course '[' and ']' don't exist, only for notation purpose in the comment. Be sure to use Notepad++ to view these special chars. > Hive select query hang when fields contain map > -- > > Key: HIVE-8115 > URL: https://issues.apache.org/jira/browse/HIVE-8115 > Project: Hive > Issue Type: Bug > Affects Versions: 0.13.0 >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Fix For: 0.14.0 > > Attachments: createTable.hql, data > > > Attached the repro of the issue. When creating an table loading the data > attached, all hive query with hangs even just select * from the table. > repro steps: > 1. run createTable.hql > 2. hadoop fs ls -put data /data > 3. LOAD DATA INPATH '/data' OVERWRITE INTO TABLE testtable; > 4. SELECT * FROM testtable; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8115) Hive select query hang when fields contain map
[ https://issues.apache.org/jira/browse/HIVE-8115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14134809#comment-14134809 ] Xiaobing Zhou commented on HIVE-8115: - Just verified, [STX]my-key[ETX] is fine for empty value with specific key, however, [STX][EXT]my-value doesn't really work. So the conclusion is empty key will hang the query. > Hive select query hang when fields contain map > -- > > Key: HIVE-8115 > URL: https://issues.apache.org/jira/browse/HIVE-8115 > Project: Hive > Issue Type: Bug >Affects Versions: 0.13.0 >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Fix For: 0.14.0 > > Attachments: createTable.hql, data > > > Attached the repro of the issue. When creating an table loading the data > attached, all hive query with hangs even just select * from the table. > repro steps: > 1. run createTable.hql > 2. hadoop fs ls -put data /data > 3. LOAD DATA INPATH '/data' OVERWRITE INTO TABLE testtable; > 4. SELECT * FROM testtable; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8115) Hive select query hang when fields contain map
[ https://issues.apache.org/jira/browse/HIVE-8115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14134810#comment-14134810 ] Xiaobing Zhou commented on HIVE-8115: - By debugging, it enters an indefinite loop, LazyMap.parse() line: 173 LazyMap.getMap() line: 320 LazyMapObjectInspector.getMap(Object) line: 104 SerDeUtils.buildJSONString(StringBuilder, Object, ObjectInspector, String) line: 317 SerDeUtils.getJSONString(Object, ObjectInspector, String) line: 197 DelimitedJSONSerDe.serializeField(ByteStream$Output, Object, ObjectInspector, LazySimpleSerDe$SerDeParameters) line: 60 DelimitedJSONSerDe(LazySimpleSerDe).doSerialize(Object, ObjectInspector) line: 422 DelimitedJSONSerDe(AbstractEncodingAwareSerDe).serialize(Object, ObjectInspector) line: 50 DefaultFetchFormatter.convert(Object, ObjectInspector) line: 71 DefaultFetchFormatter.convert(Object, ObjectInspector) line: 40 ListSinkOperator.processOp(Object, int) line: 87 SelectOperator(Operator).forward(Object, ObjectInspector) line: 800 SelectOperator.processOp(Object, int) line: 84 TableScanOperator(Operator).forward(Object, ObjectInspector) line: 800 TableScanOperator.processOp(Object, int) line: 95 FetchOperator.pushRow(InspectableObject) line: 547 FetchOperator.pushRow() line: 539 FetchTask.fetch(List) line: 138 Driver.getResults(List) line: 1620 CliDriver.processLocalCmd(String, CommandProcessor, CliSessionState) line: 266 CliDriver.processCmd(String) line: 198 CliDriver.processLine(String, boolean) line: 408 CliDriver.executeDriver(CliSessionState, HiveConf, OptionsProcessor) line: 781 CliDriver.run(String[]) line: 675 CliDriver.main(String[]) line: 614 NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method] NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57 DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43 Method.invoke(Object, Object...) line: 601 RunJar.run(String[]) line: 221 RunJar.main(String[]) line: 136 > Hive select query hang when fields contain map > -- > > Key: HIVE-8115 > URL: https://issues.apache.org/jira/browse/HIVE-8115 > Project: Hive > Issue Type: Bug >Affects Versions: 0.13.0 >Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > Fix For: 0.14.0 > > Attachments: createTable.hql, data > > > Attached the repro of the issue. When creating an table loading the data > attached, all hive query with hangs even just select * from the table. > repro steps: > 1. run createTable.hql > 2. hadoop fs ls -put data /data > 3. LOAD DATA INPATH '/data' OVERWRITE INTO TABLE testtable; > 4. SELECT * FROM testtable; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8115) Hive select query hang when fields contain map
[ https://issues.apache.org/jira/browse/HIVE-8115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14134811#comment-14134811 ] Xiaobing Zhou commented on HIVE-8115: - LazyMap was commented as "LazyMap does not deal with the case of a NULL map. That is handled by the parent LazyObject.", so it'd better to output some warning message regarding empty key or pair in line 173 of LazyMap.parse(). > Hive select query hang when fields contain map > -- > > Key: HIVE-8115 > URL: https://issues.apache.org/jira/browse/HIVE-8115 > Project: Hive > Issue Type: Bug >Affects Versions: 0.13.0 >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Fix For: 0.14.0 > > Attachments: createTable.hql, data > > > Attached the repro of the issue. When creating an table loading the data > attached, all hive query with hangs even just select * from the table. > repro steps: > 1. run createTable.hql > 2. hadoop fs ls -put data /data > 3. LOAD DATA INPATH '/data' OVERWRITE INTO TABLE testtable; > 4. SELECT * FROM testtable; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
