Hudson build is back to normal : Hive-trunk-h0.17 #507
See http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.17/507/changes
Build failed in Hudson: Hive-trunk-h0.18 #509
See http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/509/changes Changes: [jvs] HIVE-417. Implement Indexing in Hive (He Yongqiang via jvs) [jvs] HIVE-1126. Missiong some Jdbc functionality like getTables getColumns and HiveResultSet.get* methods based on column name. (Bennie Schut via jvs) [heyongqiang] HIVE-1492. FileSinkOperator should remove duplicated files from the same task based on file sizes.(Ning Zhang via He Yongqiang) -- [...truncated 13336 lines...] [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/data/files/srcbucket22.txt [junit] Loading data to table srcbucket2 [junit] POSTHOOK: Output: defa...@srcbucket2 [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/data/files/srcbucket23.txt [junit] Loading data to table srcbucket2 [junit] POSTHOOK: Output: defa...@srcbucket2 [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/data/files/kv1.txt [junit] Loading data to table src [junit] POSTHOOK: Output: defa...@src [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/data/files/kv3.txt [junit] Loading data to table src1 [junit] POSTHOOK: Output: defa...@src1 [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/data/files/kv1.seq [junit] Loading data to table src_sequencefile [junit] POSTHOOK: Output: defa...@src_sequencefile [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/data/files/complex.seq [junit] Loading data to table src_thrift [junit] POSTHOOK: Output: defa...@src_thrift [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/data/files/json.txt [junit] Loading data to table src_json [junit] POSTHOOK: Output: defa...@src_json [junit] OK [junit] diff http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/build/ql/test/logs/negative/unknown_table1.q.out http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/src/test/results/compiler/errors/unknown_table1.q.out [junit] Done query: unknown_table1.q [junit] Begin query: unknown_table2.q [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/data/files/kv1.txt [junit] Loading data to table srcpart partition (ds=2008-04-08, hr=11) [junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=11 [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/data/files/kv1.txt [junit] Loading data to table srcpart partition (ds=2008-04-08, hr=12) [junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=12 [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/data/files/kv1.txt [junit] Loading data to table srcpart partition (ds=2008-04-09, hr=11) [junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=11 [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/data/files/kv1.txt [junit] Loading data to table srcpart partition (ds=2008-04-09, hr=12) [junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=12 [junit] OK [junit] POSTHOOK: Output: defa...@srcbucket [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/data/files/srcbucket0.txt [junit] Loading data to table srcbucket [junit] POSTHOOK: Output: defa...@srcbucket [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/data/files/srcbucket1.txt [junit] Loading data to table srcbucket [junit] POSTHOOK: Output: defa...@srcbucket [junit] OK [junit] POSTHOOK: Output: defa...@srcbucket2 [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/data/files/srcbucket20.txt [junit] Loading data to table srcbucket2 [junit] POSTHOOK: Output: defa...@srcbucket2 [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/data/files/srcbucket21.txt [junit] Loading data to table srcbucket2 [junit] POSTHOOK: Output: defa...@srcbucket2 [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/data/files/srcbucket22.txt [junit] Loading data to table srcbucket2 [junit] POSTHOOK: Output: defa...@srcbucket2 [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/data/files/srcbucket23.txt [junit] Loading data
Build failed in Hudson: Hive-trunk-h0.19 #509
See http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/509/changes Changes: [jvs] HIVE-417. Implement Indexing in Hive (He Yongqiang via jvs) [jvs] HIVE-1126. Missiong some Jdbc functionality like getTables getColumns and HiveResultSet.get* methods based on column name. (Bennie Schut via jvs) -- [...truncated 13411 lines...] [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/data/files/srcbucket22.txt [junit] Loading data to table srcbucket2 [junit] POSTHOOK: Output: defa...@srcbucket2 [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/data/files/srcbucket23.txt [junit] Loading data to table srcbucket2 [junit] POSTHOOK: Output: defa...@srcbucket2 [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/data/files/kv1.txt [junit] Loading data to table src [junit] POSTHOOK: Output: defa...@src [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/data/files/kv3.txt [junit] Loading data to table src1 [junit] POSTHOOK: Output: defa...@src1 [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/data/files/kv1.seq [junit] Loading data to table src_sequencefile [junit] POSTHOOK: Output: defa...@src_sequencefile [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/data/files/complex.seq [junit] Loading data to table src_thrift [junit] POSTHOOK: Output: defa...@src_thrift [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/data/files/json.txt [junit] Loading data to table src_json [junit] POSTHOOK: Output: defa...@src_json [junit] OK [junit] diff http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/build/ql/test/logs/negative/unknown_table1.q.out http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/src/test/results/compiler/errors/unknown_table1.q.out [junit] Done query: unknown_table1.q [junit] Begin query: unknown_table2.q [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/data/files/kv1.txt [junit] Loading data to table srcpart partition (ds=2008-04-08, hr=11) [junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=11 [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/data/files/kv1.txt [junit] Loading data to table srcpart partition (ds=2008-04-08, hr=12) [junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=12 [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/data/files/kv1.txt [junit] Loading data to table srcpart partition (ds=2008-04-09, hr=11) [junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=11 [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/data/files/kv1.txt [junit] Loading data to table srcpart partition (ds=2008-04-09, hr=12) [junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=12 [junit] OK [junit] POSTHOOK: Output: defa...@srcbucket [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/data/files/srcbucket0.txt [junit] Loading data to table srcbucket [junit] POSTHOOK: Output: defa...@srcbucket [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/data/files/srcbucket1.txt [junit] Loading data to table srcbucket [junit] POSTHOOK: Output: defa...@srcbucket [junit] OK [junit] POSTHOOK: Output: defa...@srcbucket2 [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/data/files/srcbucket20.txt [junit] Loading data to table srcbucket2 [junit] POSTHOOK: Output: defa...@srcbucket2 [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/data/files/srcbucket21.txt [junit] Loading data to table srcbucket2 [junit] POSTHOOK: Output: defa...@srcbucket2 [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/data/files/srcbucket22.txt [junit] Loading data to table srcbucket2 [junit] POSTHOOK: Output: defa...@srcbucket2 [junit] OK [junit] Copying data from http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/data/files/srcbucket23.txt [junit] Loading data to table srcbucket2 [junit] POSTHOOK: Output: defa...@srcbucket2 [junit] OK [junit] Copying data from
[jira] Updated: (HIVE-1491) fix or disable loadpart_err.q
[ https://issues.apache.org/jira/browse/HIVE-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Yongqiang updated HIVE-1491: --- Status: Resolved (was: Patch Available) Resolution: Fixed Committed. Thanks John! fix or disable loadpart_err.q - Key: HIVE-1491 URL: https://issues.apache.org/jira/browse/HIVE-1491 Project: Hadoop Hive Issue Type: Bug Components: Testing Infrastructure Affects Versions: 0.6.0 Reporter: John Sichi Assignee: John Sichi Fix For: 0.7.0 Attachments: HIVE-1491.patch This test fails sporadically due to a race condition, which is annoying since it hinders pre-commit testing of patches. I'm going to disable it unless someone has a fix. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1441) Extend ivy offline mode to cover metastore downloads
[ https://issues.apache.org/jira/browse/HIVE-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12894119#action_12894119 ] John Sichi commented on HIVE-1441: -- What is the failure? Look in build/metastore/test/TEST-org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote.xml And does it occur only with this patch applied? Offline mode only works after everything has been downloaded for the first time in online mode, so be sure to get a clean test run first in online mode. Extend ivy offline mode to cover metastore downloads Key: HIVE-1441 URL: https://issues.apache.org/jira/browse/HIVE-1441 Project: Hadoop Hive Issue Type: Improvement Components: Build Infrastructure Affects Versions: 0.6.0 Reporter: John Sichi Assignee: John Sichi Fix For: 0.7.0 Attachments: HIVE-1441.1.patch We recently started downloading datanucleus jars via ivy, and the existing ivy offilne mode doesn't cover this, so we still end up trying to contact the ivy repository even with offline mode enabled. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1411) DataNucleus throws NucleusException if core-3.1.1 JAR appears more than once on CLASSPATH
[
https://issues.apache.org/jira/browse/HIVE-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12894120#action_12894120
]
John Sichi commented on HIVE-1411:
--
Carl, when you supply patches, please make sure they can be applied with patch
-p0. This one needed patch -p1.
DataNucleus throws NucleusException if core-3.1.1 JAR appears more than once
on CLASSPATH
-
Key: HIVE-1411
URL: https://issues.apache.org/jira/browse/HIVE-1411
Project: Hadoop Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.4.0, 0.4.1, 0.5.0
Reporter: Carl Steinbach
Assignee: Carl Steinbach
Fix For: 0.6.0, 0.7.0
Attachments: HIVE-1411.patch.txt
DataNucleus barfs when the core-3.1.1 JAR file appears more than once on the
CLASSPATH:
{code}
2010-03-06 12:33:25,565 ERROR exec.DDLTask
(SessionState.java:printError(279)) - FAILED: Error in metadata:
javax.jdo.JDOFatalInter
nalException: Unexpected exception caught.
NestedThrowables:
java.lang.reflect.InvocationTargetException
org.apache.hadoop.hive.ql.metadata.HiveException:
javax.jdo.JDOFatalInternalException: Unexpected exception caught.
NestedThrowables:
java.lang.reflect.InvocationTargetException
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:258)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:879)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:103)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:379)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:285)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: javax.jdo.JDOFatalInternalException: Unexpected exception caught.
NestedThrowables:
java.lang.reflect.InvocationTargetException
at
javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1186)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:803)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:698)
at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:164)
at
org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:181)
at
org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:125)
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:104)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:130)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:146)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:118)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.(HiveMetaStore.java:100)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:74)
at
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:783)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:794)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:252)
... 12 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at javax.jdo.JDOHelper$16.run(JDOHelper.java:1956)
at java.security.AccessController.doPrivileged(Native Method)
at javax.jdo.JDOHelper.invoke(JDOHelper.java:1951)
at
javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1159)
... 28 more
Caused by: org.datanucleus.exceptions.NucleusException: Plugin (Bundle)
org.eclipse.jdt.core is already registered. Ensure you do
nt have multiple JAR versions of the same plugin in the classpath. The URL
file:/Users/hadop/hadoop-0.20.1+152/build/ivy/lib/Hadoo
p/common/core-3.1.1.jar is already registered, and you are trying to
register an identical plugin located at URL
[jira] Commented: (HIVE-1294) HIVE_AUX_JARS_PATH interferes with startup of Hive Web Interface
[
https://issues.apache.org/jira/browse/HIVE-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12894122#action_12894122
]
John Sichi commented on HIVE-1294:
--
+1. Will commit when tests pass.
HIVE_AUX_JARS_PATH interferes with startup of Hive Web Interface
Key: HIVE-1294
URL: https://issues.apache.org/jira/browse/HIVE-1294
Project: Hadoop Hive
Issue Type: Bug
Components: Web UI
Affects Versions: 0.5.0
Reporter: Dilip Joseph
Assignee: Edward Capriolo
Priority: Blocker
Fix For: 0.6.0
Attachments: hive-1294.patch.txt
The Hive Webserver fails to startup with the following error message, if
HIVE_AUX_JARS_PATH environment variable is set (works fine if unset).
$ build/dist/bin/hive --service hwi
Exception in thread main java.io.IOException: Error opening job jar:
-libjars
at org.apache.hadoop.util.RunJar.main(RunJar.java:90)
Caused by: java.util.zip.ZipException: error in opening zip file
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.init(ZipFile.java:114)
at java.util.jar.JarFile.init(JarFile.java:133)
at java.util.jar.JarFile.init(JarFile.java:70)
at org.apache.hadoop.util.RunJar.main(RunJar.java:88)
Slightly modifying the command line to launch hadoop in hwi.sh solves the
problem:
$ diff bin/ext/hwi.sh /tmp/new-hwi.sh
28c28
exec $HADOOP jar $AUX_JARS_CMD_LINE ${HWI_JAR_FILE} $CLASS $HIVE_OPTS $@
---
exec $HADOOP jar ${HWI_JAR_FILE} $CLASS $AUX_JARS_CMD_LINE $HIVE_OPTS
$@
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1422) skip counter update when RunningJob.getCounters() returns null
[ https://issues.apache.org/jira/browse/HIVE-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12894126#action_12894126 ] Joydeep Sen Sarma commented on HIVE-1422: - more hadoop goriness - i think John your fix was pretty spot on: - there are three levels of job storage: a. fully in memory (can get status and counters) b. partially in memory (a la retired - can get status and not counters) c. on disk (completed jobs) so what is happening is that we are hitting case b. jobstatus is available - but not counters. we should probably anticipate the null jobstatus (which we used to get in 0.17 before b. and c. were available). what is the effect of not having final counter values available in Hive? Local mode also doesn't report counters i think. skip counter update when RunningJob.getCounters() returns null -- Key: HIVE-1422 URL: https://issues.apache.org/jira/browse/HIVE-1422 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.6.0 Reporter: John Sichi Assignee: Joydeep Sen Sarma Fix For: 0.7.0 Attachments: HIVE-1422.1.patch Under heavy load circumstances on some Hadoop versions, we may get a NPE from trying to dereference a null Counters object. I don't have a unit test which can reproduce it, but here's an example stack from a production cluster we saw today: 10/06/21 13:01:10 ERROR exec.ExecDriver: Ended Job = job_201005200457_701060 with exception 'java.lang.NullPointerException(null)' java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.Operator.updateCounters(Operator.java:999) at org.apache.hadoop.hive.ql.exec.ExecDriver.updateCounters(ExecDriver.java:503) at org.apache.hadoop.hive.ql.exec.ExecDriver.progress(ExecDriver.java:390) at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:697) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55) at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:47) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1294) HIVE_AUX_JARS_PATH interferes with startup of Hive Web Interface
[
https://issues.apache.org/jira/browse/HIVE-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12894127#action_12894127
]
John Sichi commented on HIVE-1294:
--
Actually I guess tests are pretty much guaranteed to pass since we don't have
any.
HIVE_AUX_JARS_PATH interferes with startup of Hive Web Interface
Key: HIVE-1294
URL: https://issues.apache.org/jira/browse/HIVE-1294
Project: Hadoop Hive
Issue Type: Bug
Components: Web UI
Affects Versions: 0.5.0
Reporter: Dilip Joseph
Assignee: Edward Capriolo
Priority: Blocker
Fix For: 0.6.0
Attachments: hive-1294.patch.txt
The Hive Webserver fails to startup with the following error message, if
HIVE_AUX_JARS_PATH environment variable is set (works fine if unset).
$ build/dist/bin/hive --service hwi
Exception in thread main java.io.IOException: Error opening job jar:
-libjars
at org.apache.hadoop.util.RunJar.main(RunJar.java:90)
Caused by: java.util.zip.ZipException: error in opening zip file
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.init(ZipFile.java:114)
at java.util.jar.JarFile.init(JarFile.java:133)
at java.util.jar.JarFile.init(JarFile.java:70)
at org.apache.hadoop.util.RunJar.main(RunJar.java:88)
Slightly modifying the command line to launch hadoop in hwi.sh solves the
problem:
$ diff bin/ext/hwi.sh /tmp/new-hwi.sh
28c28
exec $HADOOP jar $AUX_JARS_CMD_LINE ${HWI_JAR_FILE} $CLASS $HIVE_OPTS $@
---
exec $HADOOP jar ${HWI_JAR_FILE} $CLASS $AUX_JARS_CMD_LINE $HIVE_OPTS
$@
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1441) Extend ivy offline mode to cover metastore downloads
[
https://issues.apache.org/jira/browse/HIVE-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12894133#action_12894133
]
Edward Capriolo commented on HIVE-1441:
---
Fresh checkout before any after patch. Still looking into it.
{noformat}
/properties
testcase
classname=org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote
name=testPartition time=8.242
error message=Could not connect to meta store using any of the URIs
provided type=org.apache.hadoop.hive.metastore.api.MetaExceptionM
etaException(message:Could not connect to meta store using any of the URIs
provided)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:160)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lt;initgt;(HiveMetaStoreClient.java:128)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lt;initgt;(HiveMetaStoreClient.java:71)
at
org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote.setUp(TestHiveMetaStoreRemote.java:64)
at junit.framework.TestCase.runBare(TestCase.java:125)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at junit.framework.TestResult.runProtected(TestResult.java:124)
at junit.framework.TestResult.run(TestResult.java:109)
at junit.framework.TestCase.run(TestCase.java:118)
at junit.framework.TestSuite.runTest(TestSuite.java:208)
at junit.framework.TestSuite.run(TestSuite.java:203)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)
/error
/testcase
system-out![CDATA[Running metastore!
]]/system-out
system-err![CDATA[]]/system-err
/testsuite
From eclipse:
Running metastore!
MetaException(message:hive.metastore.warehouse.dir is not set in the config or
blank)
at org.apache.hadoop.hive.metastore.Warehouse.init(Warehouse.java:58)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:155)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:125)
at
org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:1965)
at
org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote$RunMS.run(TestHiveMetaStoreRemote.java:39)
at java.lang.Thread.run(Thread.java:619)
10/07/30 16:03:22 ERROR metastore.HiveMetaStore: Metastore Thrift Server threw
an exception. Exiting...
10/07/30 16:03:22 ERROR metastore.HiveMetaStore:
MetaException(message:hive.metastore.warehouse.dir is not set in the config or
blank)
at org.apache.hadoop.hive.metastore.Warehouse.init(Warehouse.java:58)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:155)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:125)
at
org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:1965)
at
org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote$RunMS.run(TestHiveMetaStoreRemote.java:39)
at java.lang.Thread.run(Thread.java:619)
{noformat}
Extend ivy offline mode to cover metastore downloads
Key: HIVE-1441
URL: https://issues.apache.org/jira/browse/HIVE-1441
Project: Hadoop Hive
Issue Type: Improvement
Components: Build Infrastructure
Affects Versions: 0.6.0
Reporter: John Sichi
Assignee: John Sichi
Fix For: 0.7.0
Attachments: HIVE-1441.1.patch
We recently started downloading datanucleus jars via ivy, and the existing
ivy offilne mode doesn't cover this, so we still end up trying to contact the
ivy repository even with offline mode enabled.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1422) skip counter update when RunningJob.getCounters() returns null
[ https://issues.apache.org/jira/browse/HIVE-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12894136#action_12894136 ] Ning Zhang commented on HIVE-1422: -- Counters are used in for reporting fatal errors from tasks to Driver so that it can kill the job. Currently fatal errors can be thrown from dynamic partition (when there are too many partitions created by a mapper) or map-side join when the threshold of # of rows of the small table has encountered. I think the fatal error mechanism is only enabled for the first case (DP) by default. skip counter update when RunningJob.getCounters() returns null -- Key: HIVE-1422 URL: https://issues.apache.org/jira/browse/HIVE-1422 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.6.0 Reporter: John Sichi Assignee: Joydeep Sen Sarma Fix For: 0.7.0 Attachments: HIVE-1422.1.patch Under heavy load circumstances on some Hadoop versions, we may get a NPE from trying to dereference a null Counters object. I don't have a unit test which can reproduce it, but here's an example stack from a production cluster we saw today: 10/06/21 13:01:10 ERROR exec.ExecDriver: Ended Job = job_201005200457_701060 with exception 'java.lang.NullPointerException(null)' java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.Operator.updateCounters(Operator.java:999) at org.apache.hadoop.hive.ql.exec.ExecDriver.updateCounters(ExecDriver.java:503) at org.apache.hadoop.hive.ql.exec.ExecDriver.progress(ExecDriver.java:390) at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:697) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55) at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:47) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1411) DataNucleus throws NucleusException if core-3.1.1 JAR appears more than once on CLASSPATH
[
https://issues.apache.org/jira/browse/HIVE-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Sichi updated HIVE-1411:
-
Status: Resolved (was: Patch Available)
Hadoop Flags: [Reviewed]
Resolution: Fixed
Committed to branch and trunk. Thanks Carl!
DataNucleus throws NucleusException if core-3.1.1 JAR appears more than once
on CLASSPATH
-
Key: HIVE-1411
URL: https://issues.apache.org/jira/browse/HIVE-1411
Project: Hadoop Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.4.0, 0.4.1, 0.5.0
Reporter: Carl Steinbach
Assignee: Carl Steinbach
Fix For: 0.6.0, 0.7.0
Attachments: HIVE-1411.patch.txt
DataNucleus barfs when the core-3.1.1 JAR file appears more than once on the
CLASSPATH:
{code}
2010-03-06 12:33:25,565 ERROR exec.DDLTask
(SessionState.java:printError(279)) - FAILED: Error in metadata:
javax.jdo.JDOFatalInter
nalException: Unexpected exception caught.
NestedThrowables:
java.lang.reflect.InvocationTargetException
org.apache.hadoop.hive.ql.metadata.HiveException:
javax.jdo.JDOFatalInternalException: Unexpected exception caught.
NestedThrowables:
java.lang.reflect.InvocationTargetException
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:258)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:879)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:103)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:379)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:285)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: javax.jdo.JDOFatalInternalException: Unexpected exception caught.
NestedThrowables:
java.lang.reflect.InvocationTargetException
at
javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1186)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:803)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:698)
at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:164)
at
org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:181)
at
org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:125)
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:104)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:130)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:146)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:118)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.(HiveMetaStore.java:100)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:74)
at
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:783)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:794)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:252)
... 12 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at javax.jdo.JDOHelper$16.run(JDOHelper.java:1956)
at java.security.AccessController.doPrivileged(Native Method)
at javax.jdo.JDOHelper.invoke(JDOHelper.java:1951)
at
javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1159)
... 28 more
Caused by: org.datanucleus.exceptions.NucleusException: Plugin (Bundle)
org.eclipse.jdt.core is already registered. Ensure you do
nt have multiple JAR versions of the same plugin in the classpath. The URL
file:/Users/hadop/hadoop-0.20.1+152/build/ivy/lib/Hadoo
p/common/core-3.1.1.jar is already registered, and you are trying to
register an identical plugin located at URL file:/Users/hado
[jira] Created: (HIVE-1494) Index followup: remove sort by clause and fix a bug in collect_set udaf
Index followup: remove sort by clause and fix a bug in collect_set udaf --- Key: HIVE-1494 URL: https://issues.apache.org/jira/browse/HIVE-1494 Project: Hadoop Hive Issue Type: Bug Affects Versions: 0.7.0 Reporter: He Yongqiang Assignee: He Yongqiang Fix For: 0.7.0 Attachments: hive-1494.1.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1494) Index followup: remove sort by clause and fix a bug in collect_set udaf
[ https://issues.apache.org/jira/browse/HIVE-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Yongqiang updated HIVE-1494: --- Attachment: hive-1494.1.patch Index followup: remove sort by clause and fix a bug in collect_set udaf --- Key: HIVE-1494 URL: https://issues.apache.org/jira/browse/HIVE-1494 Project: Hadoop Hive Issue Type: Bug Affects Versions: 0.7.0 Reporter: He Yongqiang Assignee: He Yongqiang Fix For: 0.7.0 Attachments: hive-1494.1.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1401) Web Interface can ony browse default
[ https://issues.apache.org/jira/browse/HIVE-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12894199#action_12894199 ] John Sichi commented on HIVE-1401: -- +1. Will commit when tests pass. Web Interface can ony browse default Key: HIVE-1401 URL: https://issues.apache.org/jira/browse/HIVE-1401 Project: Hadoop Hive Issue Type: New Feature Components: Web UI Affects Versions: 0.5.0 Reporter: Edward Capriolo Assignee: Edward Capriolo Fix For: 0.6.0, 0.7.0 Attachments: HIVE-1401-1-patch.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1494) Index followup: remove sort by clause and fix a bug in collect_set udaf
[ https://issues.apache.org/jira/browse/HIVE-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12894203#action_12894203 ] John Sichi commented on HIVE-1494: -- For the new test, could you add ORDER BY for determinism? I know that the GROUP BY does it implicitly currently, but we shouldn't rely on that. Index followup: remove sort by clause and fix a bug in collect_set udaf --- Key: HIVE-1494 URL: https://issues.apache.org/jira/browse/HIVE-1494 Project: Hadoop Hive Issue Type: Bug Affects Versions: 0.7.0 Reporter: He Yongqiang Assignee: He Yongqiang Fix For: 0.7.0 Attachments: hive-1494.1.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1422) skip counter update when RunningJob.getCounters() returns null
[ https://issues.apache.org/jira/browse/HIVE-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joydeep Sen Sarma updated HIVE-1422: Attachment: 1422.2.patch - cleaned up ExecDriver a bit - removed some dead code, some unnecessary global vars and throw better exception if JT has lost job - fixed HIVE-1493 here as well. it's a one line fix in a printf running tests. skip counter update when RunningJob.getCounters() returns null -- Key: HIVE-1422 URL: https://issues.apache.org/jira/browse/HIVE-1422 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.6.0 Reporter: John Sichi Assignee: Joydeep Sen Sarma Fix For: 0.7.0 Attachments: 1422.2.patch, HIVE-1422.1.patch Under heavy load circumstances on some Hadoop versions, we may get a NPE from trying to dereference a null Counters object. I don't have a unit test which can reproduce it, but here's an example stack from a production cluster we saw today: 10/06/21 13:01:10 ERROR exec.ExecDriver: Ended Job = job_201005200457_701060 with exception 'java.lang.NullPointerException(null)' java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.Operator.updateCounters(Operator.java:999) at org.apache.hadoop.hive.ql.exec.ExecDriver.updateCounters(ExecDriver.java:503) at org.apache.hadoop.hive.ql.exec.ExecDriver.progress(ExecDriver.java:390) at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:697) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55) at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:47) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1494) Index followup: remove sort by clause and fix a bug in collect_set udaf
[ https://issues.apache.org/jira/browse/HIVE-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Yongqiang updated HIVE-1494: --- Attachment: hive-1494.2.patch A new patch added the ORDER BY clause. Index followup: remove sort by clause and fix a bug in collect_set udaf --- Key: HIVE-1494 URL: https://issues.apache.org/jira/browse/HIVE-1494 Project: Hadoop Hive Issue Type: Bug Affects Versions: 0.7.0 Reporter: He Yongqiang Assignee: He Yongqiang Fix For: 0.7.0 Attachments: hive-1494.1.patch, hive-1494.2.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1494) Index followup: remove sort by clause and fix a bug in collect_set udaf
[ https://issues.apache.org/jira/browse/HIVE-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12894212#action_12894212 ] John Sichi commented on HIVE-1494: -- +1. Will commit when tests pass. Index followup: remove sort by clause and fix a bug in collect_set udaf --- Key: HIVE-1494 URL: https://issues.apache.org/jira/browse/HIVE-1494 Project: Hadoop Hive Issue Type: Bug Affects Versions: 0.7.0 Reporter: He Yongqiang Assignee: He Yongqiang Fix For: 0.7.0 Attachments: hive-1494.1.patch, hive-1494.2.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1494) Index followup: remove sort by clause and fix a bug in collect_set udaf
[ https://issues.apache.org/jira/browse/HIVE-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Yongqiang updated HIVE-1494: --- Status: Patch Available (was: Open) Index followup: remove sort by clause and fix a bug in collect_set udaf --- Key: HIVE-1494 URL: https://issues.apache.org/jira/browse/HIVE-1494 Project: Hadoop Hive Issue Type: Bug Affects Versions: 0.7.0 Reporter: He Yongqiang Assignee: He Yongqiang Fix For: 0.7.0 Attachments: hive-1494.1.patch, hive-1494.2.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1495) supply correct information to hooks and lineage for index rebuild
supply correct information to hooks and lineage for index rebuild - Key: HIVE-1495 URL: https://issues.apache.org/jira/browse/HIVE-1495 Project: Hadoop Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.7.0 Reporter: John Sichi Assignee: He Yongqiang Fix For: 0.7.0 This is a followup for HIVE-417. Ashish can probably help on how this should work. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1496) enhance CREATE INDEX to support immediate index build
enhance CREATE INDEX to support immediate index build - Key: HIVE-1496 URL: https://issues.apache.org/jira/browse/HIVE-1496 Project: Hadoop Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.7.0 Reporter: John Sichi Fix For: 0.7.0 Currently we only support WITH DEFERRED REBUILD. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1496) enhance CREATE INDEX to support immediate index build
[ https://issues.apache.org/jira/browse/HIVE-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi updated HIVE-1496: - Assignee: He Yongqiang enhance CREATE INDEX to support immediate index build - Key: HIVE-1496 URL: https://issues.apache.org/jira/browse/HIVE-1496 Project: Hadoop Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.7.0 Reporter: John Sichi Assignee: He Yongqiang Fix For: 0.7.0 Currently we only support WITH DEFERRED REBUILD. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1497) support COMMENT clause on CREATE INDEX, and add new commands for SHOW/DESCRIBE indexes
support COMMENT clause on CREATE INDEX, and add new commands for SHOW/DESCRIBE indexes -- Key: HIVE-1497 URL: https://issues.apache.org/jira/browse/HIVE-1497 Project: Hadoop Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.7.0 Reporter: John Sichi Assignee: He Yongqiang Fix For: 0.7.0 We need to work out the syntax for SHOW/DESCRIBE, taking partitioning into account. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-417) Implement Indexing in Hive
[ https://issues.apache.org/jira/browse/HIVE-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12894221#action_12894221 ] He Yongqiang commented on HIVE-417: --- For mysql metastore upgrade, please refer to http://wiki.apache.org/hadoop/Hive/IndexDev#Metastore_Upgrades Implement Indexing in Hive -- Key: HIVE-417 URL: https://issues.apache.org/jira/browse/HIVE-417 Project: Hadoop Hive Issue Type: New Feature Components: Metastore, Query Processor Affects Versions: 0.3.0, 0.3.1, 0.4.0, 0.6.0 Reporter: Prasad Chakka Assignee: He Yongqiang Fix For: 0.7.0 Attachments: hive-417.proto.patch, hive-417－2009-07-18.patch, hive-indexing-8-thrift-metastore-remodel.patch, hive-indexing.3.patch, hive-indexing.5.thrift.patch, hive.indexing.11.patch, hive.indexing.12.patch, hive.indexing.13.patch, idx2.png, indexing_with_ql_rewrites_trunk_953221.patch Implement indexing on Hive so that lookup and range queries are efficient. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1498) support IDXPROPERTIES on CREATE/ALTER INDEX
support IDXPROPERTIES on CREATE/ALTER INDEX --- Key: HIVE-1498 URL: https://issues.apache.org/jira/browse/HIVE-1498 Project: Hadoop Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.7.0 Reporter: John Sichi Assignee: He Yongqiang Fix For: 0.7.0 It's partially there in the grammar but not hooked in; should work pretty much the same as TBLPROPERTIES. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1499) allow index partitioning to be only a subset of base table partitioning
allow index partitioning to be only a subset of base table partitioning --- Key: HIVE-1499 URL: https://issues.apache.org/jira/browse/HIVE-1499 Project: Hadoop Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.7.0 Reporter: John Sichi Assignee: He Yongqiang Fix For: 0.7.0 This includes providing control over what to do when fine-grained partitions are dropped: * do nothing (but support an ALTER INDEX DROP PARTITION to allow the user to clean up manually) * cascade to drop the entire index partition * wait for the last fine-grained partition to go away and then auto-drop -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1500) explicitly model index rebuild time
explicitly model index rebuild time --- Key: HIVE-1500 URL: https://issues.apache.org/jira/browse/HIVE-1500 Project: Hadoop Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.7.0 Reporter: John Sichi Assignee: He Yongqiang Fix For: 0.7.0 Currently we do this implicitly for indexes stored as tables (in the partitions of the index table), and not at all for indexes not stored in tables. We should do it explicitly via new metastore model instead, at index-partition level. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1501) when generating reentrant INSERT for index rebuild, quote identifiers using backticks
when generating reentrant INSERT for index rebuild, quote identifiers using backticks - Key: HIVE-1501 URL: https://issues.apache.org/jira/browse/HIVE-1501 Project: Hadoop Hive Issue Type: Bug Components: Indexing Affects Versions: 0.7.0 Reporter: John Sichi Assignee: He Yongqiang Fix For: 0.7.0 Yongqiang, you mentioned that you weren't able to do this due to SORT BY not accepting them. The SORT BY is gone now as of HIVE-1494 (and SORT BY needs to be fixed anyway). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1502) finish the index design doc
finish the index design doc --- Key: HIVE-1502 URL: https://issues.apache.org/jira/browse/HIVE-1502 Project: Hadoop Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.7.0 Reporter: John Sichi Assignee: He Yongqiang Fix For: 0.7.0 * regenerate the metastore schema diagram to match the final column names * update Plugin Interface section to match the latest code * fill out TBD's in the Reference Implementation section * resolve other TBD's or link them to followup JIRA's -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1503) additional validation for CREATE INDEX
additional validation for CREATE INDEX -- Key: HIVE-1503 URL: https://issues.apache.org/jira/browse/HIVE-1503 Project: Hadoop Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.7.0 Reporter: John Sichi Assignee: He Yongqiang Fix For: 0.7.0 Cases that should fail: * handler has usesIndexTable=false, and user supplies table name/format * handler has usesIndexTable=true, and user supplies table format that is incompatible with handler (does compact handler have any restrictions?) * user tries to create index on external table (?) * user tries to create index on index table * user tries to create index on column of unsupported datatype (e.g. array/map/struct probably does not make sense) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1503) additional validation for CREATE INDEX
[ https://issues.apache.org/jira/browse/HIVE-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12894235#action_12894235 ] John Sichi commented on HIVE-1503: -- Another possible case: user tries to create index on base table which is incompatible with handler (does compact handler have any restrictions here?) additional validation for CREATE INDEX -- Key: HIVE-1503 URL: https://issues.apache.org/jira/browse/HIVE-1503 Project: Hadoop Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.7.0 Reporter: John Sichi Assignee: He Yongqiang Fix For: 0.7.0 Cases that should fail: * handler has usesIndexTable=false, and user supplies table name/format * handler has usesIndexTable=true, and user supplies table format that is incompatible with handler (does compact handler have any restrictions?) * user tries to create index on external table (?) * user tries to create index on index table * user tries to create index on column of unsupported datatype (e.g. array/map/struct probably does not make sense) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1494) Index followup: remove sort by clause and fix a bug in collect_set udaf
[ https://issues.apache.org/jira/browse/HIVE-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi updated HIVE-1494: - Status: Resolved (was: Patch Available) Hadoop Flags: [Reviewed] Resolution: Fixed Committed. Thanks Yongqiang! Index followup: remove sort by clause and fix a bug in collect_set udaf --- Key: HIVE-1494 URL: https://issues.apache.org/jira/browse/HIVE-1494 Project: Hadoop Hive Issue Type: Bug Affects Versions: 0.7.0 Reporter: He Yongqiang Assignee: He Yongqiang Fix For: 0.7.0 Attachments: hive-1494.1.patch, hive-1494.2.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1401) Web Interface can ony browse default
[ https://issues.apache.org/jira/browse/HIVE-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi updated HIVE-1401: - Status: Resolved (was: Patch Available) Hadoop Flags: [Reviewed] Resolution: Fixed Committed to branch and trunk. Thanks Edward! Web Interface can ony browse default Key: HIVE-1401 URL: https://issues.apache.org/jira/browse/HIVE-1401 Project: Hadoop Hive Issue Type: New Feature Components: Web UI Affects Versions: 0.5.0 Reporter: Edward Capriolo Assignee: Edward Capriolo Fix For: 0.6.0, 0.7.0 Attachments: HIVE-1401-1-patch.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1294) HIVE_AUX_JARS_PATH interferes with startup of Hive Web Interface
[
https://issues.apache.org/jira/browse/HIVE-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Sichi updated HIVE-1294:
-
Status: Resolved (was: Patch Available)
Hadoop Flags: [Reviewed]
Resolution: Fixed
Committed to branch and trunk. Thanks Edward!
HIVE_AUX_JARS_PATH interferes with startup of Hive Web Interface
Key: HIVE-1294
URL: https://issues.apache.org/jira/browse/HIVE-1294
Project: Hadoop Hive
Issue Type: Bug
Components: Web UI
Affects Versions: 0.5.0
Reporter: Dilip Joseph
Assignee: Edward Capriolo
Priority: Blocker
Fix For: 0.6.0
Attachments: hive-1294.patch.txt
The Hive Webserver fails to startup with the following error message, if
HIVE_AUX_JARS_PATH environment variable is set (works fine if unset).
$ build/dist/bin/hive --service hwi
Exception in thread main java.io.IOException: Error opening job jar:
-libjars
at org.apache.hadoop.util.RunJar.main(RunJar.java:90)
Caused by: java.util.zip.ZipException: error in opening zip file
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.init(ZipFile.java:114)
at java.util.jar.JarFile.init(JarFile.java:133)
at java.util.jar.JarFile.init(JarFile.java:70)
at org.apache.hadoop.util.RunJar.main(RunJar.java:88)
Slightly modifying the command line to launch hadoop in hwi.sh solves the
problem:
$ diff bin/ext/hwi.sh /tmp/new-hwi.sh
28c28
exec $HADOOP jar $AUX_JARS_CMD_LINE ${HWI_JAR_FILE} $CLASS $HIVE_OPTS $@
---
exec $HADOOP jar ${HWI_JAR_FILE} $CLASS $AUX_JARS_CMD_LINE $HIVE_OPTS
$@
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
