[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[ https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14025376#comment-14025376 ] Nick Dimiduk commented on HIVE-6473: Made a small change. Thanks Sushanth! > Allow writing HFiles via HBaseStorageHandler table > -- > > Key: HIVE-6473 > URL: https://issues.apache.org/jira/browse/HIVE-6473 > Project: Hive > Issue Type: Improvement > Components: HBase Handler >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.14.0 > > Attachments: HIVE-6473.0.patch.txt, HIVE-6473.1.patch, > HIVE-6473.1.patch.txt, HIVE-6473.2.patch, HIVE-6473.3.patch, > HIVE-6473.4.patch, HIVE-6473.5.patch, HIVE-6473.6.patch > > > Generating HFiles for bulkload into HBase could be more convenient. Right now > we require the user to register a new table with the appropriate output > format. This patch allows the exact same functionality, but through an > existing table managed by the HBaseStorageHandler. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[ https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14025349#comment-14025349 ] Sushanth Sowmyan commented on HIVE-6473: I'm okay with this final change, and thanks for opening a new ticket for a follow up. I'll try to verify that on my end too. +1, will go ahead and commit. > Allow writing HFiles via HBaseStorageHandler table > -- > > Key: HIVE-6473 > URL: https://issues.apache.org/jira/browse/HIVE-6473 > Project: Hive > Issue Type: Improvement > Components: HBase Handler >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.14.0 > > Attachments: HIVE-6473.0.patch.txt, HIVE-6473.1.patch, > HIVE-6473.1.patch.txt, HIVE-6473.2.patch, HIVE-6473.3.patch, > HIVE-6473.4.patch, HIVE-6473.5.patch, HIVE-6473.6.patch > > > Generating HFiles for bulkload into HBase could be more convenient. Right now > we require the user to register a new table with the appropriate output > format. This patch allows the exact same functionality, but through an > existing table managed by the HBaseStorageHandler. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[ https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14025318#comment-14025318 ] Nick Dimiduk commented on HIVE-6473: bq. Removed enabling of hbase_bulk.m; it mostly passes but is flakey for me. Will address it in a follow-on ticket. Opened HIVE-7197. > Allow writing HFiles via HBaseStorageHandler table > -- > > Key: HIVE-6473 > URL: https://issues.apache.org/jira/browse/HIVE-6473 > Project: Hive > Issue Type: Improvement > Components: HBase Handler >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.14.0 > > Attachments: HIVE-6473.0.patch.txt, HIVE-6473.1.patch, > HIVE-6473.1.patch.txt, HIVE-6473.2.patch, HIVE-6473.3.patch, > HIVE-6473.4.patch, HIVE-6473.5.patch, HIVE-6473.6.patch > > > Generating HFiles for bulkload into HBase could be more convenient. Right now > we require the user to register a new table with the appropriate output > format. This patch allows the exact same functionality, but through an > existing table managed by the HBaseStorageHandler. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[
https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14020796#comment-14020796
]
Hive QA commented on HIVE-6473:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12648784/HIVE-6473.6.patch
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 5587 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_optimization
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_load_dyn_part1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_scriptfile1
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_ctas
org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataPrimitiveTypes
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY
org.apache.hive.hcatalog.templeton.tool.TestTempletonUtils.testPropertiesParsing
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/405/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/405/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-405/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12648784
> Allow writing HFiles via HBaseStorageHandler table
> --
>
> Key: HIVE-6473
> URL: https://issues.apache.org/jira/browse/HIVE-6473
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.14.0
>
> Attachments: HIVE-6473.0.patch.txt, HIVE-6473.1.patch,
> HIVE-6473.1.patch.txt, HIVE-6473.2.patch, HIVE-6473.3.patch,
> HIVE-6473.4.patch, HIVE-6473.5.patch, HIVE-6473.6.patch
>
>
> Generating HFiles for bulkload into HBase could be more convenient. Right now
> we require the user to register a new table with the appropriate output
> format. This patch allows the exact same functionality, but through an
> existing table managed by the HBaseStorageHandler.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[
https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017709#comment-14017709
]
Hive QA commented on HIVE-6473:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12648248/HIVE-6473.5.patch
{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 5588 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_bulk
org.apache.hadoop.hive.cli.TestHBaseMinimrCliDriver.testCliDriver_hbase_bulk
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_optimization
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_insert1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_load_dyn_part1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadata_only_queries
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_ptf
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_scriptfile1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_schema_evolution
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin6
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_ctas
org.apache.hadoop.hive.ql.exec.tez.TestTezTask.testSubmit
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/386/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/386/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-386/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12648248
> Allow writing HFiles via HBaseStorageHandler table
> --
>
> Key: HIVE-6473
> URL: https://issues.apache.org/jira/browse/HIVE-6473
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: HIVE-6473.0.patch.txt, HIVE-6473.1.patch,
> HIVE-6473.1.patch.txt, HIVE-6473.2.patch, HIVE-6473.3.patch,
> HIVE-6473.4.patch, HIVE-6473.5.patch
>
>
> Generating HFiles for bulkload into HBase could be more convenient. Right now
> we require the user to register a new table with the appropriate output
> format. This patch allows the exact same functionality, but through an
> existing table managed by the HBaseStorageHandler.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[
https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017420#comment-14017420
]
Swarnim Kulkarni commented on HIVE-6473:
[~ndimiduk] I ran the test locally with the latest patch and was able to
reproduce the failure.
{noformat}
Failed tests:
TestHBaseMinimrCliDriver.testCliDriver_hbase_bulk:92->runTest:122 Unexpected
exception
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 3:01.311s
[INFO] Finished at: Wed Jun 04 01:26:30 CDT 2014
[INFO] Final Memory: 27M/87M
[INFO]
{noformat}
Also as a side note, it seems like the latest patch does not apply cleanly. May
be it would be regnerated with "--no-prefix" option.
> Allow writing HFiles via HBaseStorageHandler table
> --
>
> Key: HIVE-6473
> URL: https://issues.apache.org/jira/browse/HIVE-6473
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: HIVE-6473.0.patch.txt, HIVE-6473.1.patch,
> HIVE-6473.1.patch.txt, HIVE-6473.2.patch, HIVE-6473.3.patch,
> HIVE-6473.4.patch, HIVE-6473.5.patch
>
>
> Generating HFiles for bulkload into HBase could be more convenient. Right now
> we require the user to register a new table with the appropriate output
> format. This patch allows the exact same functionality, but through an
> existing table managed by the HBaseStorageHandler.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[ https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017421#comment-14017421 ] Swarnim Kulkarni commented on HIVE-6473: You might need to rebase the patch with master to reproduce the failure locally. > Allow writing HFiles via HBaseStorageHandler table > -- > > Key: HIVE-6473 > URL: https://issues.apache.org/jira/browse/HIVE-6473 > Project: Hive > Issue Type: Improvement > Components: HBase Handler >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HIVE-6473.0.patch.txt, HIVE-6473.1.patch, > HIVE-6473.1.patch.txt, HIVE-6473.2.patch, HIVE-6473.3.patch, > HIVE-6473.4.patch, HIVE-6473.5.patch > > > Generating HFiles for bulkload into HBase could be more convenient. Right now > we require the user to register a new table with the appropriate output > format. This patch allows the exact same functionality, but through an > existing table managed by the HBaseStorageHandler. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[ https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017386#comment-14017386 ] Swarnim Kulkarni commented on HIVE-6473: [~ndimiduk] I'll try running the test locally as well and then post back. > Allow writing HFiles via HBaseStorageHandler table > -- > > Key: HIVE-6473 > URL: https://issues.apache.org/jira/browse/HIVE-6473 > Project: Hive > Issue Type: Improvement > Components: HBase Handler >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HIVE-6473.0.patch.txt, HIVE-6473.1.patch, > HIVE-6473.1.patch.txt, HIVE-6473.2.patch, HIVE-6473.3.patch, > HIVE-6473.4.patch, HIVE-6473.5.patch > > > Generating HFiles for bulkload into HBase could be more convenient. Right now > we require the user to register a new table with the appropriate output > format. This patch allows the exact same functionality, but through an > existing table managed by the HBaseStorageHandler. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[ https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017258#comment-14017258 ] Nick Dimiduk commented on HIVE-6473: Ping [~sushanth], [~swarnim] mind taking another look? > Allow writing HFiles via HBaseStorageHandler table > -- > > Key: HIVE-6473 > URL: https://issues.apache.org/jira/browse/HIVE-6473 > Project: Hive > Issue Type: Improvement > Components: HBase Handler >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HIVE-6473.0.patch.txt, HIVE-6473.1.patch, > HIVE-6473.1.patch.txt, HIVE-6473.2.patch, HIVE-6473.3.patch, > HIVE-6473.4.patch, HIVE-6473.5.patch > > > Generating HFiles for bulkload into HBase could be more convenient. Right now > we require the user to register a new table with the appropriate output > format. This patch allows the exact same functionality, but through an > existing table managed by the HBaseStorageHandler. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[
https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14007018#comment-14007018
]
Hive QA commented on HIVE-6473:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12646293/HIVE-6473.4.patch
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 5459 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_5
org.apache.hadoop.hive.cli.TestHBaseMinimrCliDriver.testCliDriver_hbase_bulk
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/268/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/268/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-268/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12646293
> Allow writing HFiles via HBaseStorageHandler table
> --
>
> Key: HIVE-6473
> URL: https://issues.apache.org/jira/browse/HIVE-6473
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: HIVE-6473.0.patch.txt, HIVE-6473.1.patch,
> HIVE-6473.1.patch.txt, HIVE-6473.2.patch, HIVE-6473.3.patch, HIVE-6473.4.patch
>
>
> Generating HFiles for bulkload into HBase could be more convenient. Right now
> we require the user to register a new table with the appropriate output
> format. This patch allows the exact same functionality, but through an
> existing table managed by the HBaseStorageHandler.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[
https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14001576#comment-14001576
]
Nick Dimiduk commented on HIVE-6473:
[~sushanth] This test was disabled before I came along; I'm not sure why. It
passes for me locally and when I run with {{-Dtest.output.overwrite=true}} the
output file is unchanged. Mind giving it a go yourself?
{noformat}
MAVEN_OPTS="-Xmx2g" mvn test -Dtest=TestHBaseCliDriver -Dqfile=hbase_snapshot.q
-Dtest.output.overwrite=true -Phadoop-2,dist -Dhive.root.logger=DEBUG,console
{noformat}
> Allow writing HFiles via HBaseStorageHandler table
> --
>
> Key: HIVE-6473
> URL: https://issues.apache.org/jira/browse/HIVE-6473
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: HIVE-6473.0.patch.txt, HIVE-6473.1.patch,
> HIVE-6473.1.patch.txt, HIVE-6473.2.patch, HIVE-6473.3.patch
>
>
> Generating HFiles for bulkload into HBase could be more convenient. Right now
> we require the user to register a new table with the appropriate output
> format. This patch allows the exact same functionality, but through an
> existing table managed by the HBaseStorageHandler.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[
https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13999343#comment-13999343
]
Hive QA commented on HIVE-6473:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12645054/HIVE-6473.2.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/205/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/205/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12645054
> Allow writing HFiles via HBaseStorageHandler table
> --
>
> Key: HIVE-6473
> URL: https://issues.apache.org/jira/browse/HIVE-6473
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: HIVE-6473.0.patch.txt, HIVE-6473.1.patch,
> HIVE-6473.1.patch.txt, HIVE-6473.2.patch, HIVE-6473.3.patch
>
>
> Generating HFiles for bulkload into HBase could be more convenient. Right now
> we require the user to register a new table with the appropriate output
> format. This patch allows the exact same functionality, but through an
> existing table managed by the HBaseStorageHandler.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[ https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14000168#comment-14000168 ] Sushanth Sowmyan commented on HIVE-6473: Most of those tests seem unrelated, except for org.apache.hadoop.hive.cli.TestHBaseMinimrCliDriver.testCliDriver_hbase_bulk Nick, could you please look into that to see if the test needs updating? > Allow writing HFiles via HBaseStorageHandler table > -- > > Key: HIVE-6473 > URL: https://issues.apache.org/jira/browse/HIVE-6473 > Project: Hive > Issue Type: Improvement > Components: HBase Handler >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HIVE-6473.0.patch.txt, HIVE-6473.1.patch, > HIVE-6473.1.patch.txt, HIVE-6473.2.patch, HIVE-6473.3.patch > > > Generating HFiles for bulkload into HBase could be more convenient. Right now > we require the user to register a new table with the appropriate output > format. This patch allows the exact same functionality, but through an > existing table managed by the HBaseStorageHandler. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[ https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13999452#comment-13999452 ] Brock Noland commented on HIVE-6473: Same patch for testing.. > Allow writing HFiles via HBaseStorageHandler table > -- > > Key: HIVE-6473 > URL: https://issues.apache.org/jira/browse/HIVE-6473 > Project: Hive > Issue Type: Improvement > Components: HBase Handler >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HIVE-6473.0.patch.txt, HIVE-6473.1.patch, > HIVE-6473.1.patch.txt, HIVE-6473.2.patch, HIVE-6473.3.patch > > > Generating HFiles for bulkload into HBase could be more convenient. Right now > we require the user to register a new table with the appropriate output > format. This patch allows the exact same functionality, but through an > existing table managed by the HBaseStorageHandler. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[
https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13999598#comment-13999598
]
Hive QA commented on HIVE-6473:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12645134/HIVE-6473.3.patch
{color:red}ERROR:{color} -1 due to 18 failed/errored test(s), 5452 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_java_method
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_reflect
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_math_funcs
org.apache.hadoop.hive.cli.TestHBaseMinimrCliDriver.testCliDriver_hbase_bulk
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_stats_counter
org.apache.hadoop.hive.common.metrics.TestMetrics.testScopeConcurrency
org.apache.hadoop.hive.metastore.TestRetryingHMSHandler.testRetryingHMSHandler
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input20
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input4
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input5
org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataPrimitiveTypes
org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHadoopVersion
org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion
org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getPigVersion
org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getStatus
org.apache.hive.hcatalog.templeton.TestWebHCatE2e.invalidPath
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/207/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/207/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 18 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12645134
> Allow writing HFiles via HBaseStorageHandler table
> --
>
> Key: HIVE-6473
> URL: https://issues.apache.org/jira/browse/HIVE-6473
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: HIVE-6473.0.patch.txt, HIVE-6473.1.patch,
> HIVE-6473.1.patch.txt, HIVE-6473.2.patch, HIVE-6473.3.patch
>
>
> Generating HFiles for bulkload into HBase could be more convenient. Right now
> we require the user to register a new table with the appropriate output
> format. This patch allows the exact same functionality, but through an
> existing table managed by the HBaseStorageHandler.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[ https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13997920#comment-13997920 ] Sushanth Sowmyan commented on HIVE-6473: Patch looks good to me. I'll try to kick off some tests on this myself. One more thing though - you remove hbase-handler/src/test/queries/positive/hbase_bulk.m in this patch, but you do not remove the corresponding hbase-handler/src/test/results/positive/hbase_bulk.m.out file. Could you add that removal as well? I'm +1 on it otherwise though, and will commit once we have a test run. > Allow writing HFiles via HBaseStorageHandler table > -- > > Key: HIVE-6473 > URL: https://issues.apache.org/jira/browse/HIVE-6473 > Project: Hive > Issue Type: Improvement > Components: HBase Handler >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HIVE-6473.0.patch.txt, HIVE-6473.1.patch, > HIVE-6473.1.patch.txt > > > Generating HFiles for bulkload into HBase could be more convenient. Right now > we require the user to register a new table with the appropriate output > format. This patch allows the exact same functionality, but through an > existing table managed by the HBaseStorageHandler. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[ https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13924393#comment-13924393 ] Nick Dimiduk commented on HIVE-6473: [~swarnim] thanks for the notes, I'll have a look. I presume the test failures have to do with the test infrastructure not running an online HBase instance. Perhaps this is why hbase_bulk.m was named thusly? That's just a guess, I'm entirely ignorant of the buildbot's capabilities. > Allow writing HFiles via HBaseStorageHandler table > -- > > Key: HIVE-6473 > URL: https://issues.apache.org/jira/browse/HIVE-6473 > Project: Hive > Issue Type: Improvement > Components: HBase Handler >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HIVE-6473.0.patch.txt > > > Generating HFiles for bulkload into HBase could be more convenient. Right now > we require the user to register a new table with the appropriate output > format. This patch allows the exact same functionality, but through an > existing table managed by the HBaseStorageHandler. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[ https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13924346#comment-13924346 ] Swarnim Kulkarni commented on HIVE-6473: [~ndimiduk] The patch looks pretty good to me. I left minor comments on the RB. Also do we know the reason for the test failures here? They seem related. > Allow writing HFiles via HBaseStorageHandler table > -- > > Key: HIVE-6473 > URL: https://issues.apache.org/jira/browse/HIVE-6473 > Project: Hive > Issue Type: Improvement > Components: HBase Handler >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HIVE-6473.0.patch.txt > > > Generating HFiles for bulkload into HBase could be more convenient. Right now > we require the user to register a new table with the appropriate output > format. This patch allows the exact same functionality, but through an > existing table managed by the HBaseStorageHandler. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[ https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13912291#comment-13912291 ] Nick Dimiduk commented on HIVE-6473: Sure think. I've opened https://reviews.apache.org/r/18492/ . > Allow writing HFiles via HBaseStorageHandler table > -- > > Key: HIVE-6473 > URL: https://issues.apache.org/jira/browse/HIVE-6473 > Project: Hive > Issue Type: Improvement > Components: HBase Handler >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HIVE-6473.0.patch.txt > > > Generating HFiles for bulkload into HBase could be more convenient. Right now > we require the user to register a new table with the appropriate output > format. This patch allows the exact same functionality, but through an > existing table managed by the HBaseStorageHandler. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[ https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13910031#comment-13910031 ] Swarnim Kulkarni commented on HIVE-6473: Thanks for looking into this Nick. This should be very helpful. Would you mind opening a RB review for detailed review of the patch? Thanks, > Allow writing HFiles via HBaseStorageHandler table > -- > > Key: HIVE-6473 > URL: https://issues.apache.org/jira/browse/HIVE-6473 > Project: Hive > Issue Type: Improvement > Components: HBase Handler >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HIVE-6473.0.patch.txt > > > Generating HFiles for bulkload into HBase could be more convenient. Right now > we require the user to register a new table with the appropriate output > format. This patch allows the exact same functionality, but through an > existing table managed by the HBaseStorageHandler. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[
https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13909582#comment-13909582
]
Hive QA commented on HIVE-6473:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12630117/HIVE-6473.0.patch.txt
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 5178 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_bulk
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_handler_bulk
org.apache.hadoop.hive.cli.TestHBaseMinimrCliDriver.testCliDriver_hbase_bulk
org.apache.hadoop.hive.cli.TestHBaseNegativeCliDriver.testCliDriver_generatehfiles_require_family_path
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1446/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1446/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12630117
> Allow writing HFiles via HBaseStorageHandler table
> --
>
> Key: HIVE-6473
> URL: https://issues.apache.org/jira/browse/HIVE-6473
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: HIVE-6473.0.patch.txt
>
>
> Generating HFiles for bulkload into HBase could be more convenient. Right now
> we require the user to register a new table with the appropriate output
> format. This patch allows the exact same functionality, but through an
> existing table managed by the HBaseStorageHandler.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6473) Allow writing HFiles via HBaseStorageHandler table
[ https://issues.apache.org/jira/browse/HIVE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13908068#comment-13908068 ] Lefty Leverenz commented on HIVE-6473: -- When this is committed, it should be documented in the HBase Bulk Load design doc with a release note and link back to this JIRA. * [Hive HBase Bulk Load |https://cwiki.apache.org/confluence/display/Hive/HBaseBulkLoad] > Allow writing HFiles via HBaseStorageHandler table > -- > > Key: HIVE-6473 > URL: https://issues.apache.org/jira/browse/HIVE-6473 > Project: Hive > Issue Type: Improvement > Components: HBase Handler >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HIVE-6473.0.patch.txt > > > Generating HFiles for bulkload into HBase could be more convenient. Right now > we require the user to register a new table with the appropriate output > format. This patch allows the exact same functionality, but through an > existing table managed by the HBaseStorageHandler. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
