[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065302#comment-14065302
]
Gautam Kowshik commented on HIVE-6411:
--
Have we tried to back port this to Hive .13 .. This is a very useful feature to
have for hive over hbase in the current stable version as well. If not i can
try looking into this in a separate jira.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Labels: TODOC14
> Fix For: 0.14.0
>
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.10.patch.txt,
> HIVE-6411.11.patch.txt, HIVE-6411.2.patch.txt, HIVE-6411.3.patch.txt,
> HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt, HIVE-6411.6.patch.txt,
> HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt, HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14030506#comment-14030506
]
Lefty Leverenz commented on HIVE-6411:
--
The release note says this should be documented at the Hive-HBase Integration
page, which is in the Design Docs:
* [Design Docs -- Completed: Hive HBase Integration |
https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration]
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Labels: TODOC14
> Fix For: 0.14.0
>
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.10.patch.txt,
> HIVE-6411.11.patch.txt, HIVE-6411.2.patch.txt, HIVE-6411.3.patch.txt,
> HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt, HIVE-6411.6.patch.txt,
> HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt, HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13995651#comment-13995651
]
Swarnim Kulkarni commented on HIVE-6411:
[~xuefuz] Done. I have also linked to this newly created issue.
[1] https://issues.apache.org/jira/browse/HIVE-7048
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.10.patch.txt,
> HIVE-6411.11.patch.txt, HIVE-6411.2.patch.txt, HIVE-6411.3.patch.txt,
> HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt, HIVE-6411.6.patch.txt,
> HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt, HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996823#comment-13996823
]
Swarnim Kulkarni commented on HIVE-6411:
Thanks for the review Xuefu. I think we can now mark HIVE-6290 as resolved as
well as the patch for that was included as a part of this JIRA.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Fix For: 0.14.0
>
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.10.patch.txt,
> HIVE-6411.11.patch.txt, HIVE-6411.2.patch.txt, HIVE-6411.3.patch.txt,
> HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt, HIVE-6411.6.patch.txt,
> HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt, HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13992856#comment-13992856
]
Swarnim Kulkarni commented on HIVE-6411:
RB updated.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.10.patch.txt,
> HIVE-6411.11.patch.txt, HIVE-6411.2.patch.txt, HIVE-6411.3.patch.txt,
> HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt, HIVE-6411.6.patch.txt,
> HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt, HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13995423#comment-13995423
]
Xuefu Zhang commented on HIVE-6411:
---
+1
@Swarnim K Please log a followup JIRA to track the FamilyFilter issue and link
it here.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.10.patch.txt,
> HIVE-6411.11.patch.txt, HIVE-6411.2.patch.txt, HIVE-6411.3.patch.txt,
> HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt, HIVE-6411.6.patch.txt,
> HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt, HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13994808#comment-13994808
]
Swarnim Kulkarni commented on HIVE-6411:
[~xuefuz] I have responded to your comments on the RB.
[~navis] If you get a chance, would you mind telling us about your choice of a
FamilyFilter here[1]. Use of family filter for row key scans doesn't seem
appropriate to us.
[1] https://reviews.apache.org/r/21138/diff/1/?file=575776#file575776line132
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.10.patch.txt,
> HIVE-6411.11.patch.txt, HIVE-6411.2.patch.txt, HIVE-6411.3.patch.txt,
> HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt, HIVE-6411.6.patch.txt,
> HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt, HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13994317#comment-13994317
]
Hive QA commented on HIVE-6411:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12643959/HIVE-6411.11.patch.txt
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 5438 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/166/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/166/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12643959
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.10.patch.txt,
> HIVE-6411.11.patch.txt, HIVE-6411.2.patch.txt, HIVE-6411.3.patch.txt,
> HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt, HIVE-6411.6.patch.txt,
> HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt, HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13991332#comment-13991332
]
Swarnim Kulkarni commented on HIVE-6411:
Unfortunately since I couldn't update the old RB, I created a new one[1].
[~xuefuz] I think I addressed all concerns on the patch that you had with this
new one. Please let me know if I missed something.
[1] https://reviews.apache.org/r/21138/
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.10.patch.txt,
> HIVE-6411.2.patch.txt, HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt,
> HIVE-6411.5.patch.txt, HIVE-6411.6.patch.txt, HIVE-6411.7.patch.txt,
> HIVE-6411.8.patch.txt, HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13987318#comment-13987318
]
Swarnim Kulkarni commented on HIVE-6411:
Sounds good to me.
[~navis] Please do let me know if you will get a chance to look at it or else I
can pick it up over the weekend and start addressing remaining comments on the
RB. :)
Thanks!
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt, HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt,
> HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986972#comment-13986972
]
Xuefu Zhang commented on HIVE-6411:
---
[~swarnim] It seems Navis might not be available for this at the moment. If you
wish, you could help update the patch. Please communicate if you choose to do
so.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt, HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt,
> HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986858#comment-13986858
]
Swarnim Kulkarni commented on HIVE-6411:
Is it possible for us to get this in by the end of the week or so? The only
reason I ask is I have a bunch of other patches as well that I wanted to submit
but are currently blocked by this JIRA. :). Thanks!
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt, HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt,
> HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13984782#comment-13984782
]
Swarnim Kulkarni commented on HIVE-6411:
I added comments on the RB.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt, HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt,
> HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13984727#comment-13984727
]
Xuefu Zhang commented on HIVE-6411:
---
[~swarnim] If you look at the latest review comment, you will see one main item
on which your help might be helpful. It's about the changes made when
processing exprnode. I don't quite follow the change, so some documentation
might help.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt, HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt,
> HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13984718#comment-13984718
]
Swarnim Kulkarni commented on HIVE-6411:
[~navis] Please do let me know if there is anything that I can help out with. I
think we should move ahead with this soon :)
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt, HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt,
> HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975809#comment-13975809
]
Xuefu Zhang commented on HIVE-6411:
---
I believe there are some minor items to be resolved on review board.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt, HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt,
> HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975374#comment-13975374
]
Swarnim Kulkarni commented on HIVE-6411:
[~xuefuz], [~navis] Should we move ahead with this or is there something
blocking us?
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt, HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt,
> HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13960976#comment-13960976
]
Xuefu Zhang commented on HIVE-6411:
---
Actually I was referring to the changes made on the second page of review diff,
such as serialization a filter object. I also saw some seemingly dead code. I
guess I just need to spend a little bit more time understanding this.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt, HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt,
> HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13960308#comment-13960308
]
Swarnim Kulkarni commented on HIVE-6411:
I think part of that came from merging of my patch from
https://issues.apache.org/jira/browse/HIVE-6290 which added a support for
filter pushdown for structs.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt, HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt,
> HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13960268#comment-13960268
]
Xuefu Zhang commented on HIVE-6411:
---
The only thing that's holding this is time: I haven't got a chance to go over
the latest patch yet. One thing that I'm unclear about is the predicate
pushdown related changes, per my previous comments.
{quote}
One thing that I don't quite get is the changes related to predicate pushdown.
Is this necessary?
{quote}
An explanation of the nature of the change should help. Otherwise, it may take
more of my time digging around.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt, HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt,
> HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13960208#comment-13960208
]
Swarnim Kulkarni commented on HIVE-6411:
[~xuefuz][~navis] I was wondering if there is something that is still holding
us back on this JIRA or can we get this in?
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt, HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt,
> HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13957947#comment-13957947
]
Hive QA commented on HIVE-6411:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12638202/HIVE-6411.9.patch.txt
{color:green}SUCCESS:{color} +1 5541 tests passed
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2081/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2081/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12638202
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt, HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt,
> HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13957664#comment-13957664
]
Swarnim Kulkarni commented on HIVE-6411:
+1 from me.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt, HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt,
> HIVE-6411.9.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13956456#comment-13956456
]
Hive QA commented on HIVE-6411:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12637969/HIVE-6411.8.patch.txt
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 5515 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2063/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2063/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12637969
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt, HIVE-6411.7.patch.txt, HIVE-6411.8.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13946931#comment-13946931
]
Xuefu Zhang commented on HIVE-6411:
---
Thanks for the great work, [~navis]. The patch looks good. I left some minor
comments on the RB.
One thing that I don't quite get is the changes related to predicate pushdown.
Is this necessary? It would be great if someone can take a look at that.
Otherwise, I'll dig more.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt, HIVE-6411.7.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13946273#comment-13946273
]
Navis commented on HIVE-6411:
-
[~swarnim] [~xuefuz] I've done rebasing a few days ago, with some refactorings.
I think I've done a little much of that. Anyway, will update review board
shortly after.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt, HIVE-6411.7.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13945561#comment-13945561
]
Swarnim Kulkarni commented on HIVE-6411:
[~navis] If you are fine with it, I can pick this up from where you left it
off. Let me know.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13945282#comment-13945282
]
Xuefu Zhang commented on HIVE-6411:
---
Because of HIVE-6677, this patch needs some rework anyway, I'd think we it
might be better we just get it done in one chunk. It seems Navis isn't
available for this, but if anyone would like to take this forward, please
communicate and feel free to do so.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13945213#comment-13945213
]
Swarnim Kulkarni commented on HIVE-6411:
Would it make sense at all to make sense to log another bug for the refactoring
changes and moving forward with this patch minus the refactoring that needs to
be done?
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13942413#comment-13942413
]
Xuefu Zhang commented on HIVE-6411:
---
[~navis] Given that HIVE-6677 is in trunk, I'm wondering if you'd like to move
this forward. I understand that the patch will need rebasing and somewhat
refactoring, but I think either [~swarnim] or I can put some help on that if
needed. Thanks for your effort on this.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13933970#comment-13933970
]
Swarnim Kulkarni commented on HIVE-6411:
I think there are some RB comments that need to be addressed on this with some
refactoring. [~navis] I know you are already working on this patch. Please so
let me know if you need some assistance from my side on this. Thanks.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13933880#comment-13933880
]
Xuefu Zhang commented on HIVE-6411:
---
[~navis] [~swarnim] I'm wondering if we could move this forward?
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13926242#comment-13926242
]
Xuefu Zhang commented on HIVE-6411:
---
I left more comments on RB.
BTW, I noticed that RB has a new patch, which isn't attached here to run test.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13924970#comment-13924970
]
Hive QA commented on HIVE-6411:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12633301/HIVE-6411.6.patch.txt
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 5375 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1662/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1662/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12633301
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923614#comment-13923614
]
Xuefu Zhang commented on HIVE-6411:
---
Thanks for the nice work. I haven't finished review yet, but posted some
comments on RB.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt,
> HIVE-6411.6.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923375#comment-13923375
]
Hive QA commented on HIVE-6411:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12632985/HIVE-6411.5.patch.txt
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 5360 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_custom_key2
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_quotedid_smb
org.apache.hive.beeline.TestSchemaTool.testSchemaInit
org.apache.hive.beeline.TestSchemaTool.testSchemaUpgrade
org.apache.hive.service.cli.TestEmbeddedThriftBinaryCLIService.testExecuteStatementAsync
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1642/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1642/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12632985
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13922962#comment-13922962
]
Xuefu Zhang commented on HIVE-6411:
---
[~navis], would you mind update the review board with the latest patch? Thanks.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt, HIVE-6411.4.patch.txt, HIVE-6411.5.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13921569#comment-13921569
]
Hive QA commented on HIVE-6411:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12632761/HIVE-6411.4.patch.txt
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1634/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1634/console
Messages:
{noformat}
This message was trimmed, see log for full details
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @
hive-cli ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/data/hive-ptest/working/apache-svn-trunk-source/cli/src/test/resources
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-cli ---
[INFO] Executing tasks
main:
[mkdir] Created dir:
/data/hive-ptest/working/apache-svn-trunk-source/cli/target/tmp
[mkdir] Created dir:
/data/hive-ptest/working/apache-svn-trunk-source/cli/target/warehouse
[mkdir] Created dir:
/data/hive-ptest/working/apache-svn-trunk-source/cli/target/tmp/conf
[copy] Copying 5 files to
/data/hive-ptest/working/apache-svn-trunk-source/cli/target/tmp/conf
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @
hive-cli ---
[INFO] Compiling 4 source files to
/data/hive-ptest/working/apache-svn-trunk-source/cli/target/test-classes
[WARNING] Note:
/data/hive-ptest/working/apache-svn-trunk-source/cli/src/test/org/apache/hadoop/hive/cli/TestCliDriverMethods.java
uses unchecked or unsafe operations.
[WARNING] Note: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-surefire-plugin:2.16:test (default-test) @ hive-cli ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-cli ---
[INFO] Building jar:
/data/hive-ptest/working/apache-svn-trunk-source/cli/target/hive-cli-0.14.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-cli ---
[INFO] Installing
/data/hive-ptest/working/apache-svn-trunk-source/cli/target/hive-cli-0.14.0-SNAPSHOT.jar
to
/data/hive-ptest/working/maven/org/apache/hive/hive-cli/0.14.0-SNAPSHOT/hive-cli-0.14.0-SNAPSHOT.jar
[INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/cli/pom.xml
to
/data/hive-ptest/working/maven/org/apache/hive/hive-cli/0.14.0-SNAPSHOT/hive-cli-0.14.0-SNAPSHOT.pom
[INFO]
[INFO]
[INFO] Building Hive Contrib 0.14.0-SNAPSHOT
[INFO]
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-contrib ---
[INFO] Deleting /data/hive-ptest/working/apache-svn-trunk-source/contrib
(includes = [datanucleus.log, derby.log], excludes = [])
[INFO]
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @
hive-contrib ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/data/hive-ptest/working/apache-svn-trunk-source/contrib/src/main/resources
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-contrib ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-contrib
---
[INFO] Compiling 39 source files to
/data/hive-ptest/working/apache-svn-trunk-source/contrib/target/classes
[WARNING] Note: Some input files use or override a deprecated API.
[WARNING] Note: Recompile with -Xlint:deprecation for details.
[WARNING] Note:
/data/hive-ptest/working/apache-svn-trunk-source/contrib/src/java/org/apache/hadoop/hive/contrib/udf/example/UDFExampleStructPrint.java
uses unchecked or unsafe operations.
[WARNING] Note: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @
hive-contrib ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/data/hive-ptest/working/apache-svn-trunk-source/contrib/src/test/resources
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-contrib ---
[INFO] Executing tasks
main:
[mkdir] Created dir:
/data/hive-ptest/working/apache-svn-trunk-source/contrib/target/tmp
[mkdir] Created dir:
/data/hive-ptest/working/apache-svn-trunk-source/contrib/target/warehouse
[mkdir] Created dir:
/data/hive-ptest/working/apache-svn-trunk-source/contrib/target/tmp/conf
[copy] Copying 5 files to
/data/hive-ptest/w
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13919872#comment-13919872
]
Hive QA commented on HIVE-6411:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12632418/HIVE-6411.3.patch.txt
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 5240 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_custom_key2
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_mapreduce_stack_trace_hadoop20
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1616/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1616/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12632418
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt,
> HIVE-6411.3.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13916603#comment-13916603
]
Hive QA commented on HIVE-6411:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12631672/HIVE-6411.2.patch.txt
{color:green}SUCCESS:{color} +1 5181 tests passed
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1559/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1559/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12631672
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt, HIVE-6411.2.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13901558#comment-13901558
]
Swarnim Kulkarni commented on HIVE-6411:
[~navis] [~brocknoland] I created a RB review here to make it easier to look at
the diff. I'll start digging into the changes.
[1] https://reviews.apache.org/r/18122/
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13901166#comment-13901166
]
Brock Noland commented on HIVE-6411:
I also don't think the test failures are related. [~xuefuz] also looked at the
hbase composite key changes IIRC.
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13901164#comment-13901164
]
Navis commented on HIVE-6411:
-
Cannot reproduce fail of TestWriteLock. authorization_revoke_table_priv is
resolved in other issue. [~brocknoland], [~swarnim], Could you review this? (I
cannot make review board entry by 502 error)
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13900191#comment-13900191
]
Hive QA commented on HIVE-6411:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12628439/HIVE-6411.1.patch.txt
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 5090 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_revoke_table_priv
org.apache.hcatalog.hbase.snapshot.lock.TestWriteLock.testRun
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1303/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1303/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12628439
> Support more generic way of using composite key for HBaseHandler
>
>
> Key: HIVE-6411
> URL: https://issues.apache.org/jira/browse/HIVE-6411
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6411.1.patch.txt
>
>
> HIVE-2599 introduced using custom object for the row key. But it forces key
> objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
> If user provides proper Object and OI, we can replace internal key and keyOI
> with those.
> Initial implementation is based on factory interface.
> {code}
> public interface HBaseKeyFactory {
> void init(SerDeParameters parameters, Properties properties) throws
> SerDeException;
> ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
> LazyObjectBase createObject(ObjectInspector inspector) throws
> SerDeException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
