[jira] [Commented] (HIVE-5845) CTAS failed on vectorized code path

2013-11-20 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-5845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13828393#comment-13828393
 ] 

Hive QA commented on HIVE-5845:
---



{color:green}Overall{color}: +1 all checks pass

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12614892/HIVE-5845.1.patch

{color:green}SUCCESS:{color} +1 4679 tests passed

Test results: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/378/testReport
Console output: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/378/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12614892

> CTAS failed on vectorized code path
> ---
>
> Key: HIVE-5845
> URL: https://issues.apache.org/jira/browse/HIVE-5845
> Project: Hive
>  Issue Type: Bug
>Reporter: Ashutosh Chauhan
>Assignee: Remus Rusanu
> Attachments: HIVE-5845.1.patch
>
>
> Following query fails:
>  create table store_sales_2 stored as orc as select * from alltypesorc;



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5845) CTAS failed on vectorized code path

2013-11-20 Thread Ashutosh Chauhan (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-5845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13828056#comment-13828056
 ] 

Ashutosh Chauhan commented on HIVE-5845:


+1

> CTAS failed on vectorized code path
> ---
>
> Key: HIVE-5845
> URL: https://issues.apache.org/jira/browse/HIVE-5845
> Project: Hive
>  Issue Type: Bug
>Reporter: Ashutosh Chauhan
>Assignee: Remus Rusanu
> Attachments: HIVE-5845.1.patch
>
>
> Following query fails:
>  create table store_sales_2 stored as orc as select * from alltypesorc;



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5845) CTAS failed on vectorized code path

2013-11-20 Thread Remus Rusanu (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-5845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13827703#comment-13827703
 ] 

Remus Rusanu commented on HIVE-5845:


The uploaded fix addresses all the issues I found:

 - changes the OrcStruct serialization to use the correct object inspector (the 
one that writes into the OrcStruct fields)
 - the VectorExpressionWriterFactory was refactored to handle all 
assignments/writes and new object creation via the object inspectors, rather 
than assumming Writable types
 - a new API is available on the VectorExpressionWritter: setValue, which 
writes the value into the passed in object rather that returning a mutated the 
per-writer singleton. This was necessary for the OrcStruct vector 
serialization, which would end up reusing the same instance on all rows in the 
vector otherwise
 - changed the VectorExpressionWriter to use TypeInfo Category and 
PrimitiveCategory rather than type name string comparison
 - Have the VectorExpressionWriter generate writers from ObjectInspector and 
derive the OI from ExprNodeDesc rather than the other way around
 - extended the TestVectorExpressionWriter unit tests to cover the setValue API 
and struct fields assignment

> CTAS failed on vectorized code path
> ---
>
> Key: HIVE-5845
> URL: https://issues.apache.org/jira/browse/HIVE-5845
> Project: Hive
>  Issue Type: Bug
>Reporter: Ashutosh Chauhan
>Assignee: Remus Rusanu
> Attachments: HIVE-5845.1.patch
>
>
> Following query fails:
>  create table store_sales_2 stored as orc as select * from alltypesorc;



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5845) CTAS failed on vectorized code path

2013-11-20 Thread Remus Rusanu (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-5845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13827694#comment-13827694
 ] 

Remus Rusanu commented on HIVE-5845:


https://reviews.apache.org/r/15716/

> CTAS failed on vectorized code path
> ---
>
> Key: HIVE-5845
> URL: https://issues.apache.org/jira/browse/HIVE-5845
> Project: Hive
>  Issue Type: Bug
>Reporter: Ashutosh Chauhan
>Assignee: Remus Rusanu
> Attachments: HIVE-5845.1.patch
>
>
> Following query fails:
>  create table store_sales_2 stored as orc as select * from alltypesorc;



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5845) CTAS failed on vectorized code path

2013-11-20 Thread Remus Rusanu (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-5845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13827437#comment-13827437
 ] 

Remus Rusanu commented on HIVE-5845:


The root cause is not in OrcStruct/OrcSerde, but instead is in the 
VectorExpressionWriterFactory which does not honor properly the object 
inspectors passed in, assumes always to be a WritableXXObjectInspector and 
creates an WritableXXX object value. I am fixing this.

Why this was exposed is that OrcStruct.VectorExpressionWriterFactory creates 
writable object inspectors for most primitives, but for TIMESTAMP uses a native 
Java object inspector (also for DATE). I don't know why Orc does that, but none 
the less the VectorExpressionWriterFactory should handle this in a robust 
manner.

> CTAS failed on vectorized code path
> ---
>
> Key: HIVE-5845
> URL: https://issues.apache.org/jira/browse/HIVE-5845
> Project: Hive
>  Issue Type: Bug
>Reporter: Ashutosh Chauhan
>Assignee: Remus Rusanu
>
> Following query fails:
>  create table store_sales_2 stored as orc as select * from alltypesorc;



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5845) CTAS failed on vectorized code path

2013-11-20 Thread Remus Rusanu (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-5845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13827438#comment-13827438
 ] 

Remus Rusanu commented on HIVE-5845:


 OrcStruct.createObjectInspector(TypeInfo info) that is (Eclipse copy/paste ...)

> CTAS failed on vectorized code path
> ---
>
> Key: HIVE-5845
> URL: https://issues.apache.org/jira/browse/HIVE-5845
> Project: Hive
>  Issue Type: Bug
>Reporter: Ashutosh Chauhan
>Assignee: Remus Rusanu
>
> Following query fails:
>  create table store_sales_2 stored as orc as select * from alltypesorc;



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5845) CTAS failed on vectorized code path

2013-11-19 Thread Ashutosh Chauhan (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-5845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13826881#comment-13826881
 ] 

Ashutosh Chauhan commented on HIVE-5845:


 [~owen.omalley] is obvious expert on Orc. Lets tap into him for some advice 
here.


> CTAS failed on vectorized code path
> ---
>
> Key: HIVE-5845
> URL: https://issues.apache.org/jira/browse/HIVE-5845
> Project: Hive
>  Issue Type: Bug
>Reporter: Ashutosh Chauhan
>Assignee: Remus Rusanu
>
> Following query fails:
>  create table store_sales_2 stored as orc as select * from alltypesorc;



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5845) CTAS failed on vectorized code path

2013-11-19 Thread Remus Rusanu (JIRA) 
sday, November 19, 2013 1:11 AM
To: Remus Rusanu
Subject: [jira] [Commented] (HIVE-5845) CTAS failed on vectorized code path

[https://issues.apache.org/jira/secure/useravatar?avatarId=10452]

Ashutosh 
Chauhan<https://issues.apache.org/jira/secure/ViewProfile.jspa?name=ashutoshc> 
commented on an issue




Re: CTAS failed on vectorized code 
path<https://issues.apache.org/jira/browse/HIVE-5845>



Stack-trace:



Caused by: java.lang.ClassCastException: 
org.apache.hadoop.hive.ql.io.orc.OrcStruct cannot be cast to [Ljava.lang.Object;

at 
org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldData(StandardStructObjectInspector.java:173)

at 
org.apache.hadoop.hive.ql.io.orc.WriterImpl$StructTreeWriter.write(WriterImpl.java:1349)

at 
org.apache.hadoop.hive.ql.io.orc.WriterImpl.addRow(WriterImpl.java:1962)

at 
org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat$OrcRecordWriter.write(OrcOutputFormat.java:78)

at 
org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator.processOp(VectorFileSinkOperator.java:159)

at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:489)

at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:827)

at 
org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:129)

at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:489)

at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:827)

at 
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:91)

at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:489)

at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:827)

at 
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:43)


[Add Comment]<https://issues.apache.org/jira/browse/HIVE-5845#add-comment>

Add Comment<https://issues.apache.org/jira/browse/HIVE-5845#add-comment>






Hive<https://issues.apache.org/jira/browse/HIVE> / [Bug] 
<https://issues.apache.org/jira/browse/HIVE-5845> 
HIVE-5845<https://issues.apache.org/jira/browse/HIVE-5845>

CTAS failed on vectorized code 
path<https://issues.apache.org/jira/browse/HIVE-5845>


Following query fails:
 create table store_sales_2 stored as orc as select * from alltypesorc;



This message was sent by Atlassian JIRA (v6.1#6144-sha1:2e50328)

[Atlassian logo]






> CTAS failed on vectorized code path
> ---
>
> Key: HIVE-5845
> URL: https://issues.apache.org/jira/browse/HIVE-5845
> Project: Hive
>  Issue Type: Bug
>Reporter: Ashutosh Chauhan
>Assignee: Remus Rusanu
>
> Following query fails:
>  create table store_sales_2 stored as orc as select * from alltypesorc;



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5845) CTAS failed on vectorized code path

2013-11-19 Thread Remus Rusanu (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-5845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13826577#comment-13826577
 ] 

Remus Rusanu commented on HIVE-5845:


The VectorFileSinkOperator uses an optimized path for the VectorizedSerde which 
creates OrcStruct values, but attaches to it the Standard input object 
inspector. When it comes to consume this value/inspector combo, the code bombs 
as the inspector is not actually appropriate to crack an OrcStruct

> CTAS failed on vectorized code path
> ---
>
> Key: HIVE-5845
> URL: https://issues.apache.org/jira/browse/HIVE-5845
> Project: Hive
>  Issue Type: Bug
>Reporter: Ashutosh Chauhan
>Assignee: Remus Rusanu
>
> Following query fails:
>  create table store_sales_2 stored as orc as select * from alltypesorc;



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5845) CTAS failed on vectorized code path

2013-11-18 Thread Ashutosh Chauhan (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-5845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13825924#comment-13825924
 ] 

Ashutosh Chauhan commented on HIVE-5845:


Stack-trace:
{code}
Caused by: java.lang.ClassCastException: 
org.apache.hadoop.hive.ql.io.orc.OrcStruct cannot be cast to [Ljava.lang.Object;
at 
org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldData(StandardStructObjectInspector.java:173)
at 
org.apache.hadoop.hive.ql.io.orc.WriterImpl$StructTreeWriter.write(WriterImpl.java:1349)
at 
org.apache.hadoop.hive.ql.io.orc.WriterImpl.addRow(WriterImpl.java:1962)
at 
org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat$OrcRecordWriter.write(OrcOutputFormat.java:78)
at 
org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator.processOp(VectorFileSinkOperator.java:159)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:489)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:827)
at 
org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:129)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:489)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:827)
at 
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:91)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:489)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:827)
at 
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:43)
{code}

> CTAS failed on vectorized code path
> ---
>
> Key: HIVE-5845
> URL: https://issues.apache.org/jira/browse/HIVE-5845
> Project: Hive
>  Issue Type: Bug
>Reporter: Ashutosh Chauhan
>
> Following query fails:
>  create table store_sales_2 stored as orc as select * from alltypesorc;



--
This message was sent by Atlassian JIRA
(v6.1#6144)