[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13867602#comment-13867602 ] Viraj Bhat commented on HIVE-5260: -- Hi Brock, Sorry for getting back to you late on this. Thanks for cleaning the code on this one. I think during the merge of the HBaseStorageHandler, some of the code might have got copied in 2 places, before I created a function for it. So it is redundant to call: _addHBaseResources_ in 331 which in turn calls later calls _HBaseConfiguration.addHbaseResources_ and also do the same while configuring both input and output job properties. You can remove line 345. I think this is a typo. Viraj Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.12.0 Attachments: HIVE-5260.2.patch, HIVE-5260.patch This is a task being created to address the hive-side of HIVE-4331. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13857548#comment-13857548 ] Brock Noland commented on HIVE-5260: Hey guys, I am struggling to understand this code. Specifically: 1) We call addHBaseResource three separate times, twice for each code path: https://github.com/apache/hive/blob/3b96b3204e5298bcc252bf6f0f5180bb6119bff1/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java#L331 https://github.com/apache/hive/blob/3b96b3204e5298bcc252bf6f0f5180bb6119bff1/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java#L337 https://github.com/apache/hive/blob/3b96b3204e5298bcc252bf6f0f5180bb6119bff1/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java#L346 2) We copy the jobConf, call addHBaseResources a second time, and then never use the object? https://github.com/apache/hive/blob/3b96b3204e5298bcc252bf6f0f5180bb6119bff1/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java#L345 Any idea what is the purpose of doing these things? Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.12.0 Attachments: HIVE-5260.2.patch, HIVE-5260.patch This is a task being created to address the hive-side of HIVE-4331. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13765664#comment-13765664 ] Hudson commented on HIVE-5260: -- FAILURE: Integrated in Hive-trunk-hadoop1-ptest #163 (See [https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/163/]) HIVE-5260 : Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers (Viraj Bhat via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1522574) * /hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java * /hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableOutputFormat.java * /hive/trunk/hbase-handler/src/test/results/positive/external_table_ppd.q.out * /hive/trunk/hbase-handler/src/test/results/positive/hbase_binary_storage_queries.q.out * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HivePassThroughOutputFormat.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HivePassThroughRecordWriter.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/TableDesc.java Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.13.0 Attachments: HIVE-5260.2.patch, HIVE-5260.patch This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766243#comment-13766243 ] Viraj Bhat commented on HIVE-5260: -- Sushanth I think the patch to apply to Hive 0.12 as well. I would be good to have this in Hive 0.12 Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.13.0 Attachments: HIVE-5260.2.patch, HIVE-5260.patch This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766335#comment-13766335 ] Hudson commented on HIVE-5260: -- ABORTED: Integrated in Hive-trunk-hadoop2 #425 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/425/]) HIVE-5260 : Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers (Viraj Bhat via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1522574) * /hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java * /hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableOutputFormat.java * /hive/trunk/hbase-handler/src/test/results/positive/external_table_ppd.q.out * /hive/trunk/hbase-handler/src/test/results/positive/hbase_binary_storage_queries.q.out * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HivePassThroughOutputFormat.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HivePassThroughRecordWriter.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/TableDesc.java Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.13.0 Attachments: HIVE-5260.2.patch, HIVE-5260.patch This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13767271#comment-13767271 ] Thejas M Nair commented on HIVE-5260: - Patch committed to 0.12 branch as requested. Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.12.0 Attachments: HIVE-5260.2.patch, HIVE-5260.patch This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13767275#comment-13767275 ] Viraj Bhat commented on HIVE-5260: -- Thejas; Thanks for the commit to branch 0.12 Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.12.0 Attachments: HIVE-5260.2.patch, HIVE-5260.patch This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13765293#comment-13765293 ] Sushanth Sowmyan commented on HIVE-5260: I'm +1 on the latest changes - they are in accordance with the requirements discussion we had yesterday. Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.12.0 Attachments: HIVE-5260.2.patch, HIVE-5260.patch This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[
https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13765407#comment-13765407
]
Hive QA commented on HIVE-5260:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12602727/HIVE-5260.2.patch
{color:green}SUCCESS:{color} +1 3091 tests passed
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/709/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/709/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
Introduce HivePassThroughOutputFormat that allows Hive to use general purpose
OutputFormats instead of HiveOutputFormats in StorageHandlers
---
Key: HIVE-5260
URL: https://issues.apache.org/jira/browse/HIVE-5260
Project: Hive
Issue Type: Sub-task
Components: HBase Handler, HCatalog
Affects Versions: 0.12.0
Reporter: Sushanth Sowmyan
Assignee: Viraj Bhat
Fix For: 0.12.0
Attachments: HIVE-5260.2.patch, HIVE-5260.patch
This is a task being created to address the hive-side of HIVE-4331.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13765507#comment-13765507 ] Viraj Bhat commented on HIVE-5260: -- Thanks Ashutosh and Sushanth for your help. Viraj Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.13.0 Attachments: HIVE-5260.2.patch, HIVE-5260.patch This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13765647#comment-13765647 ] Hudson commented on HIVE-5260: -- FAILURE: Integrated in Hive-trunk-hadoop2-ptest #96 (See [https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/96/]) HIVE-5260 : Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers (Viraj Bhat via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1522574) * /hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java * /hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableOutputFormat.java * /hive/trunk/hbase-handler/src/test/results/positive/external_table_ppd.q.out * /hive/trunk/hbase-handler/src/test/results/positive/hbase_binary_storage_queries.q.out * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HivePassThroughOutputFormat.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HivePassThroughRecordWriter.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/TableDesc.java Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.13.0 Attachments: HIVE-5260.2.patch, HIVE-5260.patch This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13765896#comment-13765896 ] Sushanth Sowmyan commented on HIVE-5260: [~thejas]/[~ashutoshc] : Could we get this on to 0.12 as well? We need it there to be able to deprecate the hbase storage handler as of 0.12. Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.13.0 Attachments: HIVE-5260.2.patch, HIVE-5260.patch This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766037#comment-13766037 ] Hudson commented on HIVE-5260: -- FAILURE: Integrated in Hive-trunk-h0.21 #2327 (See [https://builds.apache.org/job/Hive-trunk-h0.21/2327/]) HIVE-5260 : Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers (Viraj Bhat via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1522574) * /hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java * /hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableOutputFormat.java * /hive/trunk/hbase-handler/src/test/results/positive/external_table_ppd.q.out * /hive/trunk/hbase-handler/src/test/results/positive/hbase_binary_storage_queries.q.out * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HivePassThroughOutputFormat.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HivePassThroughRecordWriter.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/TableDesc.java Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.13.0 Attachments: HIVE-5260.2.patch, HIVE-5260.patch This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764759#comment-13764759 ] Viraj Bhat commented on HIVE-5260: -- Hi Sushanth and Ashutosh, Thanks for taking time to discuss the changes at length. I have not attached the patch on the Hive side to take care of things as suggested in the previous comment. We are skipping phabricator. I will make this patch available and let the PRE-COMMIT take over. Thanks again Viraj Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.12.0 This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764958#comment-13764958 ] Ashutosh Chauhan commented on HIVE-5260: +1 Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.12.0 Attachments: HIVE-5260.patch This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[
https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13765055#comment-13765055
]
Hive QA commented on HIVE-5260:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12602666/HIVE-5260.patch
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/702/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/702/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests failed with: NonZeroExitCodeException: Command 'bash
/data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and
output '+ [[ -n '' ]]
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-702/source-prep.txt
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
++ egrep -v '^X|^Performing status on external'
++ awk '{print $2}'
++ svn status --no-ignore
+ rm -rf
+ svn update
Fetching external item into 'hcatalog/src/test/e2e/harness'
External at revision 1522097.
At revision 1522097.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0 to p2
+ exit 1
'
{noformat}
This message is automatically generated.
Introduce HivePassThroughOutputFormat that allows Hive to use general purpose
OutputFormats instead of HiveOutputFormats in StorageHandlers
---
Key: HIVE-5260
URL: https://issues.apache.org/jira/browse/HIVE-5260
Project: Hive
Issue Type: Sub-task
Components: HBase Handler, HCatalog
Affects Versions: 0.12.0
Reporter: Sushanth Sowmyan
Assignee: Viraj Bhat
Fix For: 0.12.0
Attachments: HIVE-5260.patch
This is a task being created to address the hive-side of HIVE-4331.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13765115#comment-13765115 ] Viraj Bhat commented on HIVE-5260: -- The previous message was caused because the git apply patch worked well but the patch -p1 command does not. Attaching another patch. Viraj Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.12.0 Attachments: HIVE-5260.2.patch, HIVE-5260.patch This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13763418#comment-13763418 ] Ashutosh Chauhan commented on HIVE-5260: I don't see the feedback provided on https://reviews.facebook.net/D12063#comment-2 https://reviews.facebook.net/D12063#comment-3 getting addressed in latest patch. Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.12.0 This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13763343#comment-13763343 ] Sushanth Sowmyan commented on HIVE-5260: Phabricator link put up by viraj : https://reviews.facebook.net/D12063 Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.12.0 This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13763344#comment-13763344 ] Sushanth Sowmyan commented on HIVE-5260: I'm +1 on this. [~ashutoshc], could you please check this? Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.12.0 This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13763632#comment-13763632 ] Sushanth Sowmyan commented on HIVE-5260: Ashutosh, FYI, I had issues with the diff between versions of diffs by Phabricator not showing correct diffs. I had to download the unified patch Viraj put up (HIVE-4331.patch) to see the changes. Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.12.0 This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13763773#comment-13763773 ] Sushanth Sowmyan commented on HIVE-5260: Per discussions between Viraj, Ashutosh and myself, we need a couple more changes on this patch. Viraj is currently respinning his patch with these changes: Line numbers addendum to Ashutosh's original review: 73 : ok 105 : can be improved by doing inner-if once, but will not require respinning by itself. If we have to respin though, we should do this. configureTableJobProperties : HBaseConfiguration.addHbaseResources seems to take Configuration, which makes it unnecessary to create copyOfConf. In this case, avoid copy, and avoid the if two lines later as well, where we check if it's an instance of JobConf or not. Respin. possible to change addHBaseDelegationToken to accept Configuration, and instantiate a JobContext to pass into User.getCurrent().obtainAuthTokenForJob(conf,job) form. 341 : similar comment Refactor to do similar things outside the if(), and only different things inside the if-else for inputjobprops/outputjobprops 100 : fine 123 : replace concurrenthashmap with string, keep in hivefileformatutils Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.12.0 This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
