[jira] [Commented] (HIVE-12061) add file type support to file metadata by expr call

2015-10-28 Thread Lefty Leverenz (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-12061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14977848#comment-14977848
 ] 

Lefty Leverenz commented on HIVE-12061:
---

Does this need any documentation in the wiki?  I'm guessing not, but want to be 
sure.

> add file type support to file metadata by expr call
> ---
>
> Key: HIVE-12061
> URL: https://issues.apache.org/jira/browse/HIVE-12061
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Fix For: 2.0.0
>
> Attachments: HIVE-12061.01.nogen.patch, HIVE-12061.01.patch, 
> HIVE-12061.02.patch, HIVE-12061.03.nogen.patch, HIVE-12061.03.patch, 
> HIVE-12061.04.patch, HIVE-12061.nogen.patch, HIVE-12061.patch
>
>
> Expr filtering, automatic caching, etc. should be aware of file types for 
> advanced features. For now only ORC is supported, but I want to add a 
> boundary between ORC-specific and general metastore code, that could later be 
> used for other formats if needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-12061) add file type support to file metadata by expr call

2015-10-28 Thread Sergey Shelukhin (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-12061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978906#comment-14978906
 ] 

Sergey Shelukhin commented on HIVE-12061:
-

Not really. The parent issue might need docs, this is just a minor change to an 
unreleased API.

> add file type support to file metadata by expr call
> ---
>
> Key: HIVE-12061
> URL: https://issues.apache.org/jira/browse/HIVE-12061
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Fix For: 2.0.0
>
> Attachments: HIVE-12061.01.nogen.patch, HIVE-12061.01.patch, 
> HIVE-12061.02.patch, HIVE-12061.03.nogen.patch, HIVE-12061.03.patch, 
> HIVE-12061.04.patch, HIVE-12061.nogen.patch, HIVE-12061.patch
>
>
> Expr filtering, automatic caching, etc. should be aware of file types for 
> advanced features. For now only ORC is supported, but I want to add a 
> boundary between ORC-specific and general metastore code, that could later be 
> used for other formats if needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-12061) add file type support to file metadata by expr call

2015-10-28 Thread Lefty Leverenz (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-12061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14979666#comment-14979666
 ] 

Lefty Leverenz commented on HIVE-12061:
---

Thanks Sergey.

> add file type support to file metadata by expr call
> ---
>
> Key: HIVE-12061
> URL: https://issues.apache.org/jira/browse/HIVE-12061
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Fix For: 2.0.0
>
> Attachments: HIVE-12061.01.nogen.patch, HIVE-12061.01.patch, 
> HIVE-12061.02.patch, HIVE-12061.03.nogen.patch, HIVE-12061.03.patch, 
> HIVE-12061.04.patch, HIVE-12061.nogen.patch, HIVE-12061.patch
>
>
> Expr filtering, automatic caching, etc. should be aware of file types for 
> advanced features. For now only ORC is supported, but I want to add a 
> boundary between ORC-specific and general metastore code, that could later be 
> used for other formats if needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-12061) add file type support to file metadata by expr call

2015-10-27 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-12061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14976387#comment-14976387
 ] 

Hive QA commented on HIVE-12061:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12768853/HIVE-12061.04.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 9708 tests executed
*Failed tests:*
{noformat}
TestCustomAuthentication - did not produce a TEST-*.xml file
org.apache.hadoop.hive.hwi.TestHWISessionManager.testHiveDriver
org.apache.hadoop.hive.ql.io.orc.TestJsonFileDump.testJsonDump
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5811/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5811/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5811/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12768853 - PreCommit-HIVE-TRUNK-Build

> add file type support to file metadata by expr call
> ---
>
> Key: HIVE-12061
> URL: https://issues.apache.org/jira/browse/HIVE-12061
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Fix For: 2.0.0
>
> Attachments: HIVE-12061.01.nogen.patch, HIVE-12061.01.patch, 
> HIVE-12061.02.patch, HIVE-12061.03.nogen.patch, HIVE-12061.03.patch, 
> HIVE-12061.04.patch, HIVE-12061.nogen.patch, HIVE-12061.patch
>
>
> Expr filtering, automatic caching, etc. should be aware of file types for 
> advanced features. For now only ORC is supported, but I want to add a 
> boundary between ORC-specific and general metastore code, that could later be 
> used for other formats if needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-12061) add file type support to file metadata by expr call

2015-10-23 Thread Sergey Shelukhin (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-12061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972151#comment-14972151
 ] 

Sergey Shelukhin commented on HIVE-12061:
-

I revered the patch. Looks like the hadoop-1 build is broken... cannot wait 
till we get rid of it :) Will investigate and update patch later.

> add file type support to file metadata by expr call
> ---
>
> Key: HIVE-12061
> URL: https://issues.apache.org/jira/browse/HIVE-12061
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Fix For: 2.0.0
>
> Attachments: HIVE-12061.01.nogen.patch, HIVE-12061.01.patch, 
> HIVE-12061.02.patch, HIVE-12061.nogen.patch, HIVE-12061.patch
>
>
> Expr filtering, automatic caching, etc. should be aware of file types for 
> advanced features. For now only ORC is supported, but I want to add a 
> boundary between ORC-specific and general metastore code, that could later be 
> used for other formats if needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-12061) add file type support to file metadata by expr call

2015-10-21 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-12061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967660#comment-14967660
 ] 

Hive QA commented on HIVE-12061:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12767824/HIVE-12061.02.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5724/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5724/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5724/

Messages:
{noformat}
 This message was trimmed, see log for full details 
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseConnection.java:[68,13]
 error: cannot find symbol
[ERROR] interface HBaseConnection
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseConnection.java:[84,2]
 error: cannot find symbol
[ERROR] interface HBaseConnection
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseConnection.java:[94,2]
 error: cannot find symbol
[ERROR] interface HBaseConnection
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[29,30]
 error: package org.apache.hadoop.hbase does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[30,30]
 error: package org.apache.hadoop.hbase does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[31,30]
 error: package org.apache.hadoop.hbase does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[32,37]
 error: package org.apache.hadoop.hbase.client does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[33,37]
 error: package org.apache.hadoop.hbase.client does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[34,37]
 error: package org.apache.hadoop.hbase.client does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[35,37]
 error: package org.apache.hadoop.hbase.client does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[36,37]
 error: package org.apache.hadoop.hbase.client does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[37,37]
 error: package org.apache.hadoop.hbase.client does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[38,37]
 error: package org.apache.hadoop.hbase.client does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[39,37]
 error: package org.apache.hadoop.hbase.client does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[40,37]
 error: package org.apache.hadoop.hbase.filter does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[41,37]
 error: package org.apache.hadoop.hbase.filter does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[42,37]
 error: package org.apache.hadoop.hbase.filter does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[43,37]
 error: package org.apache.hadoop.hbase.filter does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[44,62]
 error: package org.apache.hadoop.hbase.protobuf.generated.ClientProtos does 
not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/PartitionKeyComparator.java:[30,37]

[jira] [Commented] (HIVE-12061) add file type support to file metadata by expr call

2015-10-21 Thread Alan Gates (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-12061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967862#comment-14967862
 ] 

Alan Gates commented on HIVE-12061:
---

I definitely like decoupling this from ORC.  The only concern I have is that I 
never intended HBaseReadWrite to be available outside the package, which this 
forces it to be (I think it already was for some tool or another, but that's 
only because Java's packaging is borked).  But I don't see a way around it, so, 
oh well.

+1

> add file type support to file metadata by expr call
> ---
>
> Key: HIVE-12061
> URL: https://issues.apache.org/jira/browse/HIVE-12061
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-12061.01.nogen.patch, HIVE-12061.01.patch, 
> HIVE-12061.02.patch, HIVE-12061.nogen.patch, HIVE-12061.patch
>
>
> Expr filtering, automatic caching, etc. should be aware of file types for 
> advanced features. For now only ORC is supported, but I want to add a 
> boundary between ORC-specific and general metastore code, that could later be 
> used for other formats if needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-12061) add file type support to file metadata by expr call

2015-10-17 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-12061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14962136#comment-14962136
 ] 

Hive QA commented on HIVE-12061:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12767137/HIVE-12061.01.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5697/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5697/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5697/

Messages:
{noformat}
 This message was trimmed, see log for full details 
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseConnection.java:[68,13]
 error: cannot find symbol
[ERROR] interface HBaseConnection
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseConnection.java:[84,2]
 error: cannot find symbol
[ERROR] interface HBaseConnection
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseConnection.java:[94,2]
 error: cannot find symbol
[ERROR] interface HBaseConnection
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[29,30]
 error: package org.apache.hadoop.hbase does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[30,30]
 error: package org.apache.hadoop.hbase does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[31,30]
 error: package org.apache.hadoop.hbase does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[32,37]
 error: package org.apache.hadoop.hbase.client does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[33,37]
 error: package org.apache.hadoop.hbase.client does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[34,37]
 error: package org.apache.hadoop.hbase.client does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[35,37]
 error: package org.apache.hadoop.hbase.client does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[36,37]
 error: package org.apache.hadoop.hbase.client does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[37,37]
 error: package org.apache.hadoop.hbase.client does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[38,37]
 error: package org.apache.hadoop.hbase.client does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[39,37]
 error: package org.apache.hadoop.hbase.client does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[40,37]
 error: package org.apache.hadoop.hbase.filter does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[41,37]
 error: package org.apache.hadoop.hbase.filter does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[42,37]
 error: package org.apache.hadoop.hbase.filter does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[43,37]
 error: package org.apache.hadoop.hbase.filter does not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java:[44,62]
 error: package org.apache.hadoop.hbase.protobuf.generated.ClientProtos does 
not exist
[ERROR] 
/data/hive-ptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/hbase/PartitionKeyComparator.java:[30,37]