[jira] [Commented] (HIVE-4251) Indices can't be built on tables whose schema info comes from SerDe
[
https://issues.apache.org/jira/browse/HIVE-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13708712#comment-13708712
]
Hudson commented on HIVE-4251:
--
FAILURE: Integrated in Hive-trunk-hadoop2-ptest #14 (See
[https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/14/])
HIVE-4251 : Indices can't be built on tables whose schema info comes from SerDe
(Mark Wagner via Ashutosh Chauhan) (hashutosh:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1500452)
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
* /hive/trunk/ql/src/test/queries/clientpositive/index_serde.q
* /hive/trunk/ql/src/test/results/clientpositive/index_serde.q.out
Indices can't be built on tables whose schema info comes from SerDe
---
Key: HIVE-4251
URL: https://issues.apache.org/jira/browse/HIVE-4251
Project: Hive
Issue Type: Bug
Affects Versions: 0.10.0, 0.10.1, 0.11.0
Reporter: Mark Wagner
Assignee: Mark Wagner
Fix For: 0.12.0
Attachments: HIVE-4251.1.patch, HIVE-4251.2.patch
Building indices on tables who get the schema information from the
deserializer (e.g. Avro backed tables) doesn't work because when the column
is checked to exist, the correct API isn't used.
{code}
hive describe doctors;
OK
# col_namedata_type comment
numberint from deserializer
first_namestring from deserializer
last_name string from deserializer
Time taken: 0.215 seconds, Fetched: 5 row(s)
hive create index doctors_index on table doctors(number) as 'compact' with
deferred rebuild;
FAILED: Error in metadata: java.lang.RuntimeException: Check the index
columns, they should appear in the table being indexed.
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4251) Indices can't be built on tables whose schema info comes from SerDe
[
https://issues.apache.org/jira/browse/HIVE-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13701810#comment-13701810
]
Hudson commented on HIVE-4251:
--
Integrated in Hive-trunk-hadoop2 #276 (See
[https://builds.apache.org/job/Hive-trunk-hadoop2/276/])
HIVE-4251 : Indices can't be built on tables whose schema info comes from
SerDe (Mark Wagner via Ashutosh Chauhan) (Revision 1500452)
Result = FAILURE
hashutosh :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1500452
Files :
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
* /hive/trunk/ql/src/test/queries/clientpositive/index_serde.q
* /hive/trunk/ql/src/test/results/clientpositive/index_serde.q.out
Indices can't be built on tables whose schema info comes from SerDe
---
Key: HIVE-4251
URL: https://issues.apache.org/jira/browse/HIVE-4251
Project: Hive
Issue Type: Bug
Affects Versions: 0.10.0, 0.10.1, 0.11.0
Reporter: Mark Wagner
Assignee: Mark Wagner
Fix For: 0.12.0
Attachments: HIVE-4251.1.patch, HIVE-4251.2.patch
Building indices on tables who get the schema information from the
deserializer (e.g. Avro backed tables) doesn't work because when the column
is checked to exist, the correct API isn't used.
{code}
hive describe doctors;
OK
# col_namedata_type comment
numberint from deserializer
first_namestring from deserializer
last_name string from deserializer
Time taken: 0.215 seconds, Fetched: 5 row(s)
hive create index doctors_index on table doctors(number) as 'compact' with
deferred rebuild;
FAILED: Error in metadata: java.lang.RuntimeException: Check the index
columns, they should appear in the table being indexed.
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4251) Indices can't be built on tables whose schema info comes from SerDe
[
https://issues.apache.org/jira/browse/HIVE-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13701627#comment-13701627
]
Hudson commented on HIVE-4251:
--
Integrated in Hive-trunk-hadoop1-ptest #41 (See
[https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/41/])
HIVE-4251 : Indices can't be built on tables whose schema info comes from
SerDe (Mark Wagner via Ashutosh Chauhan) (Revision 1500452)
Result = FAILURE
hashutosh :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1500452
Files :
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
* /hive/trunk/ql/src/test/queries/clientpositive/index_serde.q
* /hive/trunk/ql/src/test/results/clientpositive/index_serde.q.out
Indices can't be built on tables whose schema info comes from SerDe
---
Key: HIVE-4251
URL: https://issues.apache.org/jira/browse/HIVE-4251
Project: Hive
Issue Type: Bug
Affects Versions: 0.10.0, 0.10.1, 0.11.0
Reporter: Mark Wagner
Assignee: Mark Wagner
Fix For: 0.12.0
Attachments: HIVE-4251.1.patch, HIVE-4251.2.patch
Building indices on tables who get the schema information from the
deserializer (e.g. Avro backed tables) doesn't work because when the column
is checked to exist, the correct API isn't used.
{code}
hive describe doctors;
OK
# col_namedata_type comment
numberint from deserializer
first_namestring from deserializer
last_name string from deserializer
Time taken: 0.215 seconds, Fetched: 5 row(s)
hive create index doctors_index on table doctors(number) as 'compact' with
deferred rebuild;
FAILED: Error in metadata: java.lang.RuntimeException: Check the index
columns, they should appear in the table being indexed.
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4251) Indices can't be built on tables whose schema info comes from SerDe
[
https://issues.apache.org/jira/browse/HIVE-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13701766#comment-13701766
]
Hudson commented on HIVE-4251:
--
Integrated in Hive-trunk-h0.21 #2185 (See
[https://builds.apache.org/job/Hive-trunk-h0.21/2185/])
HIVE-4251 : Indices can't be built on tables whose schema info comes from
SerDe (Mark Wagner via Ashutosh Chauhan) (Revision 1500452)
Result = FAILURE
hashutosh :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1500452
Files :
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
* /hive/trunk/ql/src/test/queries/clientpositive/index_serde.q
* /hive/trunk/ql/src/test/results/clientpositive/index_serde.q.out
Indices can't be built on tables whose schema info comes from SerDe
---
Key: HIVE-4251
URL: https://issues.apache.org/jira/browse/HIVE-4251
Project: Hive
Issue Type: Bug
Affects Versions: 0.10.0, 0.10.1, 0.11.0
Reporter: Mark Wagner
Assignee: Mark Wagner
Fix For: 0.12.0
Attachments: HIVE-4251.1.patch, HIVE-4251.2.patch
Building indices on tables who get the schema information from the
deserializer (e.g. Avro backed tables) doesn't work because when the column
is checked to exist, the correct API isn't used.
{code}
hive describe doctors;
OK
# col_namedata_type comment
numberint from deserializer
first_namestring from deserializer
last_name string from deserializer
Time taken: 0.215 seconds, Fetched: 5 row(s)
hive create index doctors_index on table doctors(number) as 'compact' with
deferred rebuild;
FAILED: Error in metadata: java.lang.RuntimeException: Check the index
columns, they should appear in the table being indexed.
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4251) Indices can't be built on tables whose schema info comes from SerDe
[
https://issues.apache.org/jira/browse/HIVE-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13701439#comment-13701439
]
Ashutosh Chauhan commented on HIVE-4251:
+1
Indices can't be built on tables whose schema info comes from SerDe
---
Key: HIVE-4251
URL: https://issues.apache.org/jira/browse/HIVE-4251
Project: Hive
Issue Type: Bug
Affects Versions: 0.10.0, 0.10.1, 0.11.0
Reporter: Mark Wagner
Assignee: Mark Wagner
Attachments: HIVE-4251.1.patch, HIVE-4251.2.patch
Building indices on tables who get the schema information from the
deserializer (e.g. Avro backed tables) doesn't work because when the column
is checked to exist, the correct API isn't used.
{code}
hive describe doctors;
OK
# col_namedata_type comment
numberint from deserializer
first_namestring from deserializer
last_name string from deserializer
Time taken: 0.215 seconds, Fetched: 5 row(s)
hive create index doctors_index on table doctors(number) as 'compact' with
deferred rebuild;
FAILED: Error in metadata: java.lang.RuntimeException: Check the index
columns, they should appear in the table being indexed.
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4251) Indices can't be built on tables whose schema info comes from SerDe
[
https://issues.apache.org/jira/browse/HIVE-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13648665#comment-13648665
]
Mark Wagner commented on HIVE-4251:
---
Hi Steven,
Indexing on the field of a record/struct isn't supported yet. That's also the
case for other metadata like cluster, sort, and skew columns. I've been taking
a look at that recently, and will open up a JIRA to discuss/track. I tried your
second case and got the same issue as you. It seems to be an unrelated issue
that is preventing group by using a struct as a key. These are both issues that
affect all storage formats though, so we should discuss them in their own JIRAs.
Can you confirm that you're able to create indices on top level primitive
columns of Avro tables with this patch?
Thanks,
Mark
Indices can't be built on tables whose schema info comes from SerDe
---
Key: HIVE-4251
URL: https://issues.apache.org/jira/browse/HIVE-4251
Project: Hive
Issue Type: Bug
Affects Versions: 0.10.0, 0.11.0, 0.10.1
Reporter: Mark Wagner
Assignee: Mark Wagner
Fix For: 0.11.0, 0.10.1
Attachments: HIVE-4251.1.patch, HIVE-4251.2.patch
Building indices on tables who get the schema information from the
deserializer (e.g. Avro backed tables) doesn't work because when the column
is checked to exist, the correct API isn't used.
{code}
hive describe doctors;
OK
# col_namedata_type comment
numberint from deserializer
first_namestring from deserializer
last_name string from deserializer
Time taken: 0.215 seconds, Fetched: 5 row(s)
hive create index doctors_index on table doctors(number) as 'compact' with
deferred rebuild;
FAILED: Error in metadata: java.lang.RuntimeException: Check the index
columns, they should appear in the table being indexed.
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4251) Indices can't be built on tables whose schema info comes from SerDe
[
https://issues.apache.org/jira/browse/HIVE-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639120#comment-13639120
]
Steven Willis commented on HIVE-4251:
-
Just tried your patch and it kind of works for me, however I'm running into a
possibly related problem. I have a table called {{clicks}} that uses the avro
serde, it has a column called {{url}} which is an avro record containing a
field called {{domain}}. I'd like to index my {{clicks}} table on
{{url.domain}} with:
{code}
CREATE INDEX domainIndex
ON TABLE clicks(url.domain)
AS 'org.apache.hadoop.hive.ql.index.compact.CompactIndexHandler'
WITH DEFERRED REBUILD;
{code}
And I get:
{code}
FAILED: ParseException line 2:19 mismatched input '.' expecting ) near 'url' in
create index statement
{code}
I also tried using {{`url`.domain}} and {{`url.domain`}} but those don't work
either. I then tried making an index on the full {{url}} field. But then I just
get the following error in the MR job:
{code}
java.lang.RuntimeException: Hash code on complex types not supported yet.
{code}
Any ideas?
Indices can't be built on tables whose schema info comes from SerDe
---
Key: HIVE-4251
URL: https://issues.apache.org/jira/browse/HIVE-4251
Project: Hive
Issue Type: Bug
Affects Versions: 0.10.0, 0.11.0, 0.10.1
Reporter: Mark Wagner
Assignee: Mark Wagner
Fix For: 0.11.0, 0.10.1
Attachments: HIVE-4251.1.patch, HIVE-4251.2.patch
Building indices on tables who get the schema information from the
deserializer (e.g. Avro backed tables) doesn't work because when the column
is checked to exist, the correct API isn't used.
{code}
hive describe doctors;
OK
# col_namedata_type comment
numberint from deserializer
first_namestring from deserializer
last_name string from deserializer
Time taken: 0.215 seconds, Fetched: 5 row(s)
hive create index doctors_index on table doctors(number) as 'compact' with
deferred rebuild;
FAILED: Error in metadata: java.lang.RuntimeException: Check the index
columns, they should appear in the table being indexed.
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
