Fucun Chu has uploaded a new patch set (#7). (
http://gerrit.cloudera.org:8080/17842 )
Change subject: IMPALA-5741: Support reading tiny RDBMS tables
......................................................................
IMPALA-5741: Support reading tiny RDBMS tables
This patch uses the "external data source" mechanism in Impala and
writes a data source for querying jdbc. It has some limitations:
- It is not distributed.
- Only support binary predicates with operators =, !=, <=, >=, <, > to
be pushed to RDBMS
In order to query the RDBMS tables, the following steps should be
followed (note that existing data source table will be rebuilt):
1. Make sure that the database driver package has been added to the
classpath and the minicluster cluster has been started.
2. Copy the data source library into HDFS.
${IMPALA_HOME}/testdata/bin/copy-data-sources.sh
3. Create an `alltypes` table in the postgres database.
${IMPALA_HOME}/testdata/bin/load-data-sources.sh
4. Create data sources table(alltypes_jdbc_datasource).
${IMPALA_HOME}/bin/impala-shell.sh -i ${IMPALAD} -f\
${IMPALA_HOME}/testdata/bin/create-data-source-table.sql
Testing:
- Ran core tests successfully.
Change-Id: I8244e978c7717c6f1452f66f1630b6441392e7d2
---
M
fe/src/main/java/org/apache/impala/extdatasource/ExternalDataSourceExecutor.java
M fe/src/test/java/org/apache/impala/service/FrontendTest.java
A java/ext-data-source/jdbc/pom.xml
A
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/JdbcDataSource.java
A
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/conf/DatabaseType.java
A
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/conf/JdbcStorageConfig.java
A
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/conf/JdbcStorageConfigManager.java
A
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/DB2DatabaseAccessor.java
A
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/DatabaseAccessor.java
A
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/DatabaseAccessorFactory.java
A
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java
A
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/JdbcRecordIterator.java
A
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/JethroDatabaseAccessor.java
A
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/MsSqlDatabaseAccessor.java
A
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/MySqlDatabaseAccessor.java
A
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/OracleDatabaseAccessor.java
A
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/PostgresDatabaseAccessor.java
A
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/exception/JdbcDatabaseAccessException.java
A
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/util/QueryConditionUtil.java
A
java/ext-data-source/jdbc/src/test/java/org/apache/impala/extdatasource/jdbc/JdbcDataSourceTest.java
A java/ext-data-source/jdbc/src/test/resources/log4j.properties
A java/ext-data-source/jdbc/src/test/resources/test_script.sql
M java/ext-data-source/pom.xml
M testdata/bin/copy-data-sources.sh
M testdata/bin/create-data-source-table.sql
M testdata/bin/create-load-data.sh
A testdata/bin/load-data-sources.sh
M testdata/workloads/functional-query/queries/QueryTest/data-source-tables.test
28 files changed, 2,003 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/17842/7
--
To view, visit http://gerrit.cloudera.org:8080/17842
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8244e978c7717c6f1452f66f1630b6441392e7d2
Gerrit-Change-Number: 17842
Gerrit-PatchSet: 7
Gerrit-Owner: Fucun Chu <chufu...@hotmail.com>
Gerrit-Reviewer: Fucun Chu <chufu...@hotmail.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
