[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783830#comment-16783830 ] Daniel Dai commented on HIVE-1555: -- [~Tagar], yes, you can store password in a Hadoop credential store, and point hive.sql.dbcp.password.keystore there. See https://cwiki.apache.org/confluence/display/Hive/JdbcStorageHandler. > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner >Priority: Major > Labels: TODOC2.2 > Fix For: 2.3.0 > > Attachments: HIVE-1555.7.patch, HIVE-1555.8.patch, HIVE-1555.9.patch, > JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783654#comment-16783654 ] Ruslan Dautkhanov commented on HIVE-1555: - Is this possible to store password in a Hadoop credential store? Metadata is visible to all users.. Alternatively Hive should always return password field redacted for commands like `show create table`. > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner >Priority: Major > Labels: TODOC2.2 > Fix For: 2.3.0 > > Attachments: HIVE-1555.7.patch, HIVE-1555.8.patch, HIVE-1555.9.patch, > JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889718#comment-15889718 ] Lefty Leverenz commented on HIVE-1555: -- Should this be documented in the wiki? If so, please add a TODOC2.2 label. > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Fix For: 2.2.0 > > Attachments: HIVE-1555.7.patch, HIVE-1555.8.patch, HIVE-1555.9.patch, > JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15887005#comment-15887005 ] Jason Dere commented on HIVE-1555: -- +1 > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Attachments: HIVE-1555.7.patch, HIVE-1555.8.patch, HIVE-1555.9.patch, > JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884282#comment-15884282
]
Hive QA commented on HIVE-1555:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12854637/HIVE-1555.9.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 10284 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=224)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3783/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3783/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3783/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12854637 - PreCommit-HIVE-Build
> JDBC Storage Handler
>
>
> Key: HIVE-1555
> URL: https://issues.apache.org/jira/browse/HIVE-1555
> Project: Hive
> Issue Type: New Feature
> Components: JDBC
>Reporter: Bob Robertson
>Assignee: Gunther Hagleitner
> Attachments: HIVE-1555.7.patch, HIVE-1555.8.patch, HIVE-1555.9.patch,
> JDBCStorageHandler Design Doc.pdf
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> With the Cassandra and HBase Storage Handlers I thought it would make sense
> to include a generic JDBC RDBMS Storage Handler so that you could import a
> standard DB table into Hive. Many people must want to perform HiveQL joins,
> etc against tables in other systems etc.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880424#comment-15880424
]
Hive QA commented on HIVE-1555:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12854126/HIVE-1555.8.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10279 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[jdbc_handler]
(batchId=136)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=224)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=224)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3723/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3723/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3723/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12854126 - PreCommit-HIVE-Build
> JDBC Storage Handler
>
>
> Key: HIVE-1555
> URL: https://issues.apache.org/jira/browse/HIVE-1555
> Project: Hive
> Issue Type: New Feature
> Components: JDBC
>Reporter: Bob Robertson
>Assignee: Gunther Hagleitner
> Attachments: HIVE-1555.3.patch, HIVE-1555.4.patch, HIVE-1555.5.patch,
> HIVE-1555.6.patch, HIVE-1555.7.patch, HIVE-1555.8.patch, JDBCStorageHandler
> Design Doc.pdf
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> With the Cassandra and HBase Storage Handlers I thought it would make sense
> to include a generic JDBC RDBMS Storage Handler so that you could import a
> standard DB table into Hive. Many people must want to perform HiveQL joins,
> etc against tables in other systems etc.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879733#comment-15879733
]
Hive QA commented on HIVE-1555:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12854037/HIVE-1555.7.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10274 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[jdbc_handler]
(batchId=52)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=140)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=224)
org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver
(batchId=231)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3705/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3705/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3705/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12854037 - PreCommit-HIVE-Build
> JDBC Storage Handler
>
>
> Key: HIVE-1555
> URL: https://issues.apache.org/jira/browse/HIVE-1555
> Project: Hive
> Issue Type: New Feature
> Components: JDBC
>Reporter: Bob Robertson
>Assignee: Gunther Hagleitner
> Attachments: HIVE-1555.3.patch, HIVE-1555.4.patch, HIVE-1555.5.patch,
> HIVE-1555.6.patch, HIVE-1555.7.patch, JDBCStorageHandler Design Doc.pdf
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> With the Cassandra and HBase Storage Handlers I thought it would make sense
> to include a generic JDBC RDBMS Storage Handler so that you could import a
> standard DB table into Hive. Many people must want to perform HiveQL joins,
> etc against tables in other systems etc.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877579#comment-15877579
]
Hive QA commented on HIVE-1555:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853848/HIVE-1555.6.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10278 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[jdbc_handler]
(batchId=52)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=224)
org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testTupleInBagInTupleInBag[1]
(batchId=173)
org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testTupleInBagInTupleInBag[2]
(batchId=173)
org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteSmallint
(batchId=173)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3684/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3684/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3684/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12853848 - PreCommit-HIVE-Build
> JDBC Storage Handler
>
>
> Key: HIVE-1555
> URL: https://issues.apache.org/jira/browse/HIVE-1555
> Project: Hive
> Issue Type: New Feature
> Components: JDBC
>Reporter: Bob Robertson
>Assignee: Gunther Hagleitner
> Attachments: HIVE-1555.3.patch, HIVE-1555.4.patch, HIVE-1555.5.patch,
> HIVE-1555.6.patch, JDBCStorageHandler Design Doc.pdf
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> With the Cassandra and HBase Storage Handlers I thought it would make sense
> to include a generic JDBC RDBMS Storage Handler so that you could import a
> standard DB table into Hive. Many people must want to perform HiveQL joins,
> etc against tables in other systems etc.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15875376#comment-15875376 ] Brock Noland commented on HIVE-1555: I think the resource closing in GenericJdbcDatabaseAccessor.cleanupResources could be a littler tigher. For example if ResultSet.close throws, the statement and connection are not cleaned up. > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Attachments: HIVE-1555.3.patch, HIVE-1555.4.patch, HIVE-1555.5.patch, > JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15875371#comment-15875371 ] Gunther Hagleitner commented on HIVE-1555: -- [~thejas] do you have a chance to take a look? > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Attachments: HIVE-1555.3.patch, HIVE-1555.4.patch, HIVE-1555.5.patch, > JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868626#comment-15868626 ] Gunther Hagleitner commented on HIVE-1555: -- Test failures aren't related. [~ashutoshc] do you have some feedback? > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Attachments: HIVE-1555.3.patch, HIVE-1555.4.patch, HIVE-1555.5.patch, > JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862547#comment-15862547
]
Hive QA commented on HIVE-1555:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12852219/HIVE-1555.5.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10261 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=224)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=224)
org.apache.hive.hcatalog.pig.TestRCFileHCatStorer.testWriteDecimalXY
(batchId=173)
org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteDecimal
(batchId=173)
org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteDecimalXY
(batchId=173)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3507/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3507/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3507/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12852219 - PreCommit-HIVE-Build
> JDBC Storage Handler
>
>
> Key: HIVE-1555
> URL: https://issues.apache.org/jira/browse/HIVE-1555
> Project: Hive
> Issue Type: New Feature
> Components: JDBC
>Reporter: Bob Robertson
>Assignee: Gunther Hagleitner
> Attachments: HIVE-1555.3.patch, HIVE-1555.4.patch, HIVE-1555.5.patch,
> JDBCStorageHandler Design Doc.pdf
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> With the Cassandra and HBase Storage Handlers I thought it would make sense
> to include a generic JDBC RDBMS Storage Handler so that you could import a
> standard DB table into Hive. Many people must want to perform HiveQL joins,
> etc against tables in other systems etc.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862370#comment-15862370
]
Hive QA commented on HIVE-1555:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12852198/HIVE-1555.4.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10261 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestHCatNonPartitioned - did not produce a TEST-*.xml file (likely timed out)
(batchId=175)
TestHCatPartitionPublish - did not produce a TEST-*.xml file (likely timed out)
(batchId=175)
TestHCatPartitioned - did not produce a TEST-*.xml file (likely timed out)
(batchId=175)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[jdbc_handler]
(batchId=149)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=224)
org.apache.hive.hcatalog.cli.TestPermsGrp.testCustomPerms (batchId=175)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3505/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3505/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3505/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12852198 - PreCommit-HIVE-Build
> JDBC Storage Handler
>
>
> Key: HIVE-1555
> URL: https://issues.apache.org/jira/browse/HIVE-1555
> Project: Hive
> Issue Type: New Feature
> Components: JDBC
>Reporter: Bob Robertson
>Assignee: Gunther Hagleitner
> Attachments: HIVE-1555.3.patch, HIVE-1555.4.patch, JDBCStorageHandler
> Design Doc.pdf
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> With the Cassandra and HBase Storage Handlers I thought it would make sense
> to include a generic JDBC RDBMS Storage Handler so that you could import a
> standard DB table into Hive. Many people must want to perform HiveQL joins,
> etc against tables in other systems etc.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862349#comment-15862349
]
Hive QA commented on HIVE-1555:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12852192/HIVE-1555.3.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 10272 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[jdbc_handler]
(batchId=149)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=224)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testConnectionSchemaAPIs (batchId=218)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testHttpHeaderSize (batchId=218)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3504/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3504/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3504/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12852192 - PreCommit-HIVE-Build
> JDBC Storage Handler
>
>
> Key: HIVE-1555
> URL: https://issues.apache.org/jira/browse/HIVE-1555
> Project: Hive
> Issue Type: New Feature
> Components: JDBC
>Reporter: Bob Robertson
>Assignee: Gunther Hagleitner
> Attachments: HIVE-1555.1.patch, HIVE-1555.2.patch, HIVE-1555.3.patch,
> JDBCStorageHandler Design Doc.pdf
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> With the Cassandra and HBase Storage Handlers I thought it would make sense
> to include a generic JDBC RDBMS Storage Handler so that you could import a
> standard DB table into Hive. Many people must want to perform HiveQL joins,
> etc against tables in other systems etc.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862331#comment-15862331
]
Hive QA commented on HIVE-1555:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12852188/HIVE-1555.2.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 10272 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[jdbc_handler]
(batchId=52)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[jdbc_handler]
(batchId=149)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=224)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vector_count_distinct]
(batchId=106)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3503/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3503/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3503/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12852188 - PreCommit-HIVE-Build
> JDBC Storage Handler
>
>
> Key: HIVE-1555
> URL: https://issues.apache.org/jira/browse/HIVE-1555
> Project: Hive
> Issue Type: New Feature
> Components: JDBC
>Reporter: Bob Robertson
>Assignee: Gunther Hagleitner
> Attachments: HIVE-1555.1.patch, HIVE-1555.2.patch, JDBCStorageHandler
> Design Doc.pdf
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> With the Cassandra and HBase Storage Handlers I thought it would make sense
> to include a generic JDBC RDBMS Storage Handler so that you could import a
> standard DB table into Hive. Many people must want to perform HiveQL joins,
> etc against tables in other systems etc.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862315#comment-15862315
]
Hive QA commented on HIVE-1555:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12852188/HIVE-1555.2.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 10257 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=131)
[union_remove_15.q,bucket_map_join_tez1.q,scriptfile1.q,groupby7_noskew.q,bucketmapjoin1.q,subquery_multiinsert.q,auto_join8.q,auto_join6.q,groupby2_map_skew.q,lateral_view_explode2.q,join28.q,load_dyn_part1.q,skewjoinopt17.q,union_remove_20.q,bucketmapjoin5.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[jdbc_handler]
(batchId=52)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[jdbc_handler]
(batchId=149)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=224)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3502/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3502/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3502/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12852188 - PreCommit-HIVE-Build
> JDBC Storage Handler
>
>
> Key: HIVE-1555
> URL: https://issues.apache.org/jira/browse/HIVE-1555
> Project: Hive
> Issue Type: New Feature
> Components: JDBC
>Reporter: Bob Robertson
>Assignee: Gunther Hagleitner
> Attachments: HIVE-1555.1.patch, HIVE-1555.2.patch, JDBCStorageHandler
> Design Doc.pdf
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> With the Cassandra and HBase Storage Handlers I thought it would make sense
> to include a generic JDBC RDBMS Storage Handler so that you could import a
> standard DB table into Hive. Many people must want to perform HiveQL joins,
> etc against tables in other systems etc.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862266#comment-15862266 ] Gunther Hagleitner commented on HIVE-1555: -- Can I take this over? It's been a few months w/o update again. I've brought some code. Well I took [~charithe]'s recommendation and took https://github.com/QubitProducts/hive-jdbc-storage-handler I haven't tested w/ mysql or any other db yet. But you can write queries against derby in q files now. > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Attachments: JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15681496#comment-15681496
]
Dmitry Zagorulkin commented on HIVE-1555:
-
[~xuefuz] I've implemented simple solution with some hard code by one.
It's tested with oracle database.
beeline> !connect jdbc:hive2://localhost:1
Connecting to jdbc:hive2://localhost:1
Enter username for jdbc:hive2://localhost:1:
Enter password for jdbc:hive2://localhost:1:
Connected to: Apache Hive (version 2.2.0-SNAPSHOT)
Driver: Hive JDBC (version 2.2.0-SNAPSHOT)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://localhost:1>
0: jdbc:hive2://localhost:1> SET
hive.metastore.warehouse.dir=${env:HOME}/Documents/hive-warehouse;
No rows affected (0.158 seconds)
0: jdbc:hive2://localhost:1>
0: jdbc:hive2://localhost:1> CREATE EXTERNAL TABLE books3 (
. . . . . . . . . . . . . . . .> book_idINT,
. . . . . . . . . . . . . . . .> book_name STRING,
. . . . . . . . . . . . . . . .> author_nameSTRING,
. . . . . . . . . . . . . . . .> book_isbn STRING
. . . . . . . . . . . . . . . .> )
. . . . . . . . . . . . . . . .> STORED BY
"org.apache.hive.storagehandler.JDBCStorageHandler"
. . . . . . . . . . . . . . . .> TBLPROPERTIES (
. . . . . . . . . . . . . . . .> "mapred.jdbc.driver.class" =
"oracle.jdbc.OracleDriver",
. . . . . . . . . . . . . . . .> "mapred.jdbc.url" =
"jdbc:oracle:thin:@//localhost:49161/XE",
. . . . . . . . . . . . . . . .> "mapred.jdbc.username" = "*",
. . . . . . . . . . . . . . . .> "mapred.jdbc.password" = "*",
. . . . . . . . . . . . . . . .> "hive.jdbc.update.on.duplicate" = "true",
. . . . . . . . . . . . . . . .> "mapreduce.jdbc.input.table.name" = "books"
. . . . . . . . . . . . . . . .> );
No rows affected (2.297 seconds)
0: jdbc:hive2://localhost:1>
0: jdbc:hive2://localhost:1>
0: jdbc:hive2://localhost:1> select * from books3;
+-+---+-+---+
| books3.book_id | books3.book_name | books3.author_name | books3.book_isbn
|
+-+---+-+---+
| 124123 | name | author | 132321adsaf31
|
| 13 | name2 | author2 | asd213fadsf
|
| 2345236 | name3 | author3 | asdfds1234123
|
+-+---+-+---+
3 rows selected (2.146 seconds)
0: jdbc:hive2://localhost:1> explain select * from books3;
++
| Explain |
++
| STAGE DEPENDENCIES:|
| Stage-0 is a root stage |
||
| STAGE PLANS: |
| Stage: Stage-0 |
| Fetch Operator |
| limit: -1|
| Processor Tree: |
| TableScan |
| alias: books3|
| Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE |
| Select Operator |
| expressions: book_id (type: string), book_name (type: string),
author_name (type: string), book_isbn (type: string) |
| outputColumnNames: _col0, _col1, _col2, _col3 |
| Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE |
| ListSink |
||
++
17 rows selected (0.508 seconds)
What do you think about add information about external table inside serde
properties section?
> JDBC Storage Handler
>
>
> Key: HIVE-1555
> URL: https://issues.apache.org/jira/browse/HIVE-1555
> Project: Hive
> Issue Type: New Feature
> Components: JDBC
>Reporter: Bob Robertson
>Assignee: Dmitry Zagorulkin
> Attachments: JDBCStorageHandler Design Doc.pdf
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> With the Cassandra and HBase Storage Handlers I thought it would make sense
> to include a generic JDBC RDBMS Storage Handler so that you could import a
> standard DB table into Hive. Many people must want to perform HiveQL joins,
> etc against tables in other systems etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15506789#comment-15506789 ] Dmitry Zagorulkin commented on HIVE-1555: - Actually it works fine with mysql and postgres. For oracle it does not work. See here: http://stackoverflow.com/questions/39594861/creating-sparksql-jdbc-federation-with-oracle-db-table-fails-with-strange-error#comment66500144_39594861 > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Teddy Choi > Attachments: JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15401447#comment-15401447
]
Yuming Wang commented on HIVE-1555:
---
Spark SQL support it.
https://community.hortonworks.com/articles/4671/sparksql-jdbc-federation.html
{code:sql}
CREATE TABLE mysql_federated_sample
USING org.apache.spark.sql.jdbc
OPTIONS (
driver "com.mysql.jdbc.Driver",
url "jdbc:mysql://localhost/hive?user=hive=hive",
dbtable "TBLS"
);
{code}
> JDBC Storage Handler
>
>
> Key: HIVE-1555
> URL: https://issues.apache.org/jira/browse/HIVE-1555
> Project: Hive
> Issue Type: New Feature
> Components: JDBC
>Reporter: Bob Robertson
>Assignee: Teddy Choi
> Attachments: JDBCStorageHandler Design Doc.pdf
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> With the Cassandra and HBase Storage Handlers I thought it would make sense
> to include a generic JDBC RDBMS Storage Handler so that you could import a
> standard DB table into Hive. Many people must want to perform HiveQL joins,
> etc against tables in other systems etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
