[Impala-ASF-CR] IMPALA-12378: Add jar file of JdbcDataSource library to classpath
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20971 ) Change subject: IMPALA-12378: Add jar file of JdbcDataSource library to classpath .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15128/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20971 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0daff8db6231f161ec27b45b51d78e21733d9b1f Gerrit-Change-Number: 20971 Gerrit-PatchSet: 3 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: gaurav singh Gerrit-Comment-Date: Wed, 31 Jan 2024 18:11:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12378: Add jar file of JdbcDataSource library to classpath
Wenzhe Zhou has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/20971 ) Change subject: IMPALA-12378: Add jar file of JdbcDataSource library to classpath .. IMPALA-12378: Add jar file of JdbcDataSource library to classpath This patch adds the jar file of JdbcDataSource library to classpath. With the change, We can get the instance of JdbcDataSourceData class in current ClassLoader and don't need to download the jar file from the given HDFS location to local file system in runtime. Changes CMakeLists.txt to install jar file of JdbcDataSource library impala-data-source-jdbc-$IMPALA_VERSION.jar to destination directory. Updates docker/setup_build_context.py to create symlink for impala-data-source-jdbc-$IMPALA_VERSION.jar so that the jar file is bundled in docker image and added to classpath. With these changes, impala-data-source-jdbc-$IMPALA_VERSION.jar is automatically shipped and added to classpath. Impala will try to load the jar file from the specified location in runtime. If the jar file does not exist, Impala will try to create the instance of JdbcDataSource with current class loader, i.e. assume it's in classpath. In a following patch, we will allow user to create DataSource without specifying HDFS location. Testing: - Passed core test - Passed dockerised-tests Change-Id: I0daff8db6231f161ec27b45b51d78e21733d9b1f --- M CMakeLists.txt M be/src/exec/external-data-source-executor.cc M bin/impala-config.sh M bin/set-classpath.sh M docker/setup_build_context.py M fe/src/main/java/org/apache/impala/extdatasource/ExternalDataSourceExecutor.java M fe/src/main/java/org/apache/impala/planner/DataSourceScanNode.java M java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java M testdata/bin/copy-ext-data-sources.sh 9 files changed, 64 insertions(+), 26 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/71/20971/3 -- To view, visit http://gerrit.cloudera.org:8080/20971 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I0daff8db6231f161ec27b45b51d78e21733d9b1f Gerrit-Change-Number: 20971 Gerrit-PatchSet: 3 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: gaurav singh
[Impala-ASF-CR] IMPALA-12378: Add jar file of JdbcDataSource library to classpath
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20971 ) Change subject: IMPALA-12378: Add jar file of JdbcDataSource library to classpath .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15124/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20971 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0daff8db6231f161ec27b45b51d78e21733d9b1f Gerrit-Change-Number: 20971 Gerrit-PatchSet: 2 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: gaurav singh Gerrit-Comment-Date: Wed, 31 Jan 2024 04:51:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12378: Add jar file of JdbcDataSource library to classpath
Wenzhe Zhou has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/20971 ) Change subject: IMPALA-12378: Add jar file of JdbcDataSource library to classpath .. IMPALA-12378: Add jar file of JdbcDataSource library to classpath This patch adds the jar file of JdbcDataSource library to classpath. With the change, We can get the instance of JdbcDataSourceData class in current ClassLoader and don't need to download the jar file from the given HDFS location to local file system in runtime. Changes CMakeLists.txt to install jar file of JdbcDataSource library impala-data-source-jdbc-$IMPALA_VERSION.jar to destination directory. Updates docker/setup_build_context.py to create symlink for impala-data-source-jdbc-$IMPALA_VERSION.jar so that the jar file is built into docker image and added to classpath. With these changes, impala-data-source-jdbc-$IMPALA_VERSION.jar is automatically shipped in Impala image and added to classpath. In a following patch, we should allow user to create a Data Source without specifying HDFS location. Testing: - Passed core test - Passed dockerised-tests Change-Id: I0daff8db6231f161ec27b45b51d78e21733d9b1f --- M CMakeLists.txt M be/src/exec/external-data-source-executor.cc M bin/impala-config.sh M bin/set-classpath.sh M docker/setup_build_context.py M fe/src/main/java/org/apache/impala/extdatasource/ExternalDataSourceExecutor.java M fe/src/main/java/org/apache/impala/planner/DataSourceScanNode.java M java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java M testdata/bin/copy-ext-data-sources.sh 9 files changed, 64 insertions(+), 26 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/71/20971/2 -- To view, visit http://gerrit.cloudera.org:8080/20971 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I0daff8db6231f161ec27b45b51d78e21733d9b1f Gerrit-Change-Number: 20971 Gerrit-PatchSet: 2 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: gaurav singh
[Impala-ASF-CR] IMPALA-12378: Add jar file of JdbcDataSource library to classpath
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20971 ) Change subject: IMPALA-12378: Add jar file of JdbcDataSource library to classpath .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/20971 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0daff8db6231f161ec27b45b51d78e21733d9b1f Gerrit-Change-Number: 20971 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: gaurav singh Gerrit-Comment-Date: Mon, 29 Jan 2024 21:48:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12378: Add jar file of JdbcDataSource library to classpath
gaurav singh has posted comments on this change. ( http://gerrit.cloudera.org:8080/20971 ) Change subject: IMPALA-12378: Add jar file of JdbcDataSource library to classpath .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/20971 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0daff8db6231f161ec27b45b51d78e21733d9b1f Gerrit-Change-Number: 20971 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: gaurav singh Gerrit-Comment-Date: Mon, 29 Jan 2024 18:53:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12378: Add jar file of JdbcDataSource library to classpath
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/20971 ) Change subject: IMPALA-12378: Add jar file of JdbcDataSource library to classpath .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/20971/1/java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java File java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java: http://gerrit.cloudera.org:8080/#/c/20971/1/java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java@285 PS1, Line 285: FileSystemUtil.copyFileFromUriToLocal(driverUrl); > Remove this copy ? No, this is jar file of jdbc driver, not the jar file of jdbc data source library. -- To view, visit http://gerrit.cloudera.org:8080/20971 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0daff8db6231f161ec27b45b51d78e21733d9b1f Gerrit-Change-Number: 20971 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: gaurav singh Gerrit-Comment-Date: Mon, 29 Jan 2024 18:14:23 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12378: Add jar file of JdbcDataSource library to classpath
gaurav singh has posted comments on this change. ( http://gerrit.cloudera.org:8080/20971 ) Change subject: IMPALA-12378: Add jar file of JdbcDataSource library to classpath .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/20971/1/java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java File java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java: http://gerrit.cloudera.org:8080/#/c/20971/1/java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java@285 PS1, Line 285: FileSystemUtil.copyFileFromUriToLocal(driverUrl); Remove this copy ? -- To view, visit http://gerrit.cloudera.org:8080/20971 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0daff8db6231f161ec27b45b51d78e21733d9b1f Gerrit-Change-Number: 20971 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: gaurav singh Gerrit-Comment-Date: Mon, 29 Jan 2024 18:09:17 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12378: Add jar file of JdbcDataSource library to classpath
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20971 ) Change subject: IMPALA-12378: Add jar file of JdbcDataSource library to classpath .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10205/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/20971 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0daff8db6231f161ec27b45b51d78e21733d9b1f Gerrit-Change-Number: 20971 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 29 Jan 2024 17:10:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12378: Add jar file of JdbcDataSource library to classpath
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20971 ) Change subject: IMPALA-12378: Add jar file of JdbcDataSource library to classpath .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15088/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20971 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0daff8db6231f161ec27b45b51d78e21733d9b1f Gerrit-Change-Number: 20971 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 29 Jan 2024 17:06:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12378: Add jar file of JdbcDataSource library to classpath
Wenzhe Zhou has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20971 Change subject: IMPALA-12378: Add jar file of JdbcDataSource library to classpath .. IMPALA-12378: Add jar file of JdbcDataSource library to classpath This patch adds the jar file of JdbcDataSource library to classpath. With the change, We can get the instance of JdbcDataSourceData class in current ClassLoader and don't need to download the jar file from the given HDFS location to local file system. Testing: - Passed core test - Passed dockerised-tests Change-Id: I0daff8db6231f161ec27b45b51d78e21733d9b1f --- M CMakeLists.txt M be/src/exec/external-data-source-executor.cc M bin/set-classpath.sh M docker/setup_build_context.py M fe/src/main/java/org/apache/impala/extdatasource/ExternalDataSourceExecutor.java M fe/src/main/java/org/apache/impala/planner/DataSourceScanNode.java M java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java M testdata/bin/copy-ext-data-sources.sh 8 files changed, 63 insertions(+), 26 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/71/20971/1 -- To view, visit http://gerrit.cloudera.org:8080/20971 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I0daff8db6231f161ec27b45b51d78e21733d9b1f Gerrit-Change-Number: 20971 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou
