[jira] [Commented] (HIVE-7017) Insertion into Parquet tables fails under Tez

2014-07-24 Thread Thejas M Nair (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14074098#comment-14074098
 ] 

Thejas M Nair commented on HIVE-7017:
-

+1

> Insertion into Parquet tables fails under Tez
> -
>
> Key: HIVE-7017
> URL: https://issues.apache.org/jira/browse/HIVE-7017
> Project: Hive
>  Issue Type: Bug
>  Components: Tez
>Affects Versions: 0.13.0
> Environment: Hive 0.13.0, CentOS 6
>Reporter: Craig Condit
> Attachments: HIVE-7017.1.patch.txt
>
>
> It seems Parquet tables cannot be written to in Tez mode. CREATE TABLE foo 
> STORED AS PARQUET SELECT ... queries fail with:
> {noformat}
>   java.lang.IllegalArgumentException: TaskAttemptId string : 
> task1396892688715_80817_m_76_3 is not properly formed
>   at 
> org.apache.hadoop.mapreduce.TaskAttemptID.forName(TaskAttemptID.java:201)
>   at 
> org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.(ParquetRecordWriterWrapper.java:49)
> {noformat}
> The same queries work fine after setting hive.execution.engine=mr.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7017) Insertion into Parquet tables fails under Tez

2014-07-24 Thread Craig Condit (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14073503#comment-14073503
 ] 

Craig Condit commented on HIVE-7017:


Attached patch seems to fix the issue.

> Insertion into Parquet tables fails under Tez
> -
>
> Key: HIVE-7017
> URL: https://issues.apache.org/jira/browse/HIVE-7017
> Project: Hive
>  Issue Type: Bug
>  Components: Tez
>Affects Versions: 0.13.0
> Environment: Hive 0.13.0, CentOS 6
>Reporter: Craig Condit
>Assignee: Craig Condit
> Attachments: HIVE-7017.1.patch.txt
>
>
> It seems Parquet tables cannot be written to in Tez mode. CREATE TABLE foo 
> STORED AS PARQUET SELECT ... queries fail with:
> {noformat}
>   java.lang.IllegalArgumentException: TaskAttemptId string : 
> task1396892688715_80817_m_76_3 is not properly formed
>   at 
> org.apache.hadoop.mapreduce.TaskAttemptID.forName(TaskAttemptID.java:201)
>   at 
> org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.(ParquetRecordWriterWrapper.java:49)
> {noformat}
> The same queries work fine after setting hive.execution.engine=mr.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7017) Insertion into Parquet tables fails under Tez

2014-07-24 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14073403#comment-14073403
 ] 

Hive QA commented on HIVE-7017:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12657543/HIVE-7017.1.patch.txt

{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 5756 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_join_hash
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/41/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/41/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-41/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12657543

> Insertion into Parquet tables fails under Tez
> -
>
> Key: HIVE-7017
> URL: https://issues.apache.org/jira/browse/HIVE-7017
> Project: Hive
>  Issue Type: Bug
>  Components: Tez
>Affects Versions: 0.13.0
> Environment: Hive 0.13.0, CentOS 6
>Reporter: Craig Condit
>Assignee: Craig Condit
> Attachments: HIVE-7017.1.patch.txt
>
>
> It seems Parquet tables cannot be written to in Tez mode. CREATE TABLE foo 
> STORED AS PARQUET SELECT ... queries fail with:
> {noformat}
>   java.lang.IllegalArgumentException: TaskAttemptId string : 
> task1396892688715_80817_m_76_3 is not properly formed
>   at 
> org.apache.hadoop.mapreduce.TaskAttemptID.forName(TaskAttemptID.java:201)
>   at 
> org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.(ParquetRecordWriterWrapper.java:49)
> {noformat}
> The same queries work fine after setting hive.execution.engine=mr.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7017) Insertion into Parquet tables fails under Tez

2014-05-06 Thread Craig Condit (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13991025#comment-13991025
 ] 

Craig Condit commented on HIVE-7017:


I mistakenly assumed that code came from TEZ, when it in fact exists in Hive...

https://github.com/apache/hive/blob/022ee59b8cb9161996310861d4fbf59801d4b9fe/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezProcessor.java#L103

Should probably be:

{noformat}
StringBuilder taskAttemptIdBuilder = new StringBuilder("attempt_");
{noformat}

instead of:

{noformat}
StringBuilder taskAttemptIdBuilder = new StringBuilder("task");
{noformat}




> Insertion into Parquet tables fails under Tez
> -
>
> Key: HIVE-7017
> URL: https://issues.apache.org/jira/browse/HIVE-7017
> Project: Hive
>  Issue Type: Bug
>  Components: Tez
>Affects Versions: 0.13.0
> Environment: Hive 0.13.0, CentOS 6
>Reporter: Craig Condit
>
> It seems Parquet tables cannot be written to in Tez mode. CREATE TABLE foo 
> STORED AS PARQUET SELECT ... queries fail with:
> {noformat}
>   java.lang.IllegalArgumentException: TaskAttemptId string : 
> task1396892688715_80817_m_76_3 is not properly formed
>   at 
> org.apache.hadoop.mapreduce.TaskAttemptID.forName(TaskAttemptID.java:201)
>   at 
> org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.(ParquetRecordWriterWrapper.java:49)
> {noformat}
> The same queries work fine after setting hive.execution.engine=mr.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7017) Insertion into Parquet tables fails under Tez

2014-05-06 Thread Craig Condit (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990997#comment-13990997
 ] 

Craig Condit commented on HIVE-7017:


It's not obvious what the proper thing to do in this case is. The existing ID 
could be parsed and reformatted, or Tez could be modified to generate 
TaskAttemptID-compatible identifiers. I have created 
https://issues.apache.org/jira/browse/TEZ-1104 to track the issue on that end.

> Insertion into Parquet tables fails under Tez
> -
>
> Key: HIVE-7017
> URL: https://issues.apache.org/jira/browse/HIVE-7017
> Project: Hive
>  Issue Type: Bug
>  Components: Tez
>Affects Versions: 0.13.0
> Environment: Hive 0.13.0, CentOS 6
>Reporter: Craig Condit
>
> It seems Parquet tables cannot be written to in Tez mode. CREATE TABLE foo 
> STORED AS PARQUET SELECT ... queries fail with:
> {noformat}
>   java.lang.IllegalArgumentException: TaskAttemptId string : 
> task1396892688715_80817_m_76_3 is not properly formed
>   at 
> org.apache.hadoop.mapreduce.TaskAttemptID.forName(TaskAttemptID.java:201)
>   at 
> org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.(ParquetRecordWriterWrapper.java:49)
> {noformat}
> The same queries work fine after setting hive.execution.engine=mr.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7017) Insertion into Parquet tables fails under Tez

2014-05-05 Thread Gopal V (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990266#comment-13990266
 ] 

Gopal V commented on HIVE-7017:
---

Looks like it comes from Parquet code assuming MR only?

https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/ParquetRecordWriterWrapper.java#L49

> Insertion into Parquet tables fails under Tez
> -
>
> Key: HIVE-7017
> URL: https://issues.apache.org/jira/browse/HIVE-7017
> Project: Hive
>  Issue Type: Bug
>  Components: Tez
>Affects Versions: 0.13.0
> Environment: Hive 0.13.0, CentOS 6
>Reporter: Craig Condit
>
> It seems Parquet tables cannot be written to in Tez mode. CREATE TABLE foo 
> STORED AS PARQUET SELECT ... queries fail with:
> {noformat}
>   java.lang.IllegalArgumentException: TaskAttemptId string : 
> task1396892688715_80817_m_76_3 is not properly formed
>   at 
> org.apache.hadoop.mapreduce.TaskAttemptID.forName(TaskAttemptID.java:201)
>   at 
> org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.(ParquetRecordWriterWrapper.java:49)
> {noformat}
> The same queries work fine after setting hive.execution.engine=mr.



--
This message was sent by Atlassian JIRA
(v6.2#6252)