[jira] [Commented] (FLINK-14667) flink1.9,1.8.2 run flinkSql fat jar ,can't load the right tableFactory (TableSourceFactory) for the kafka
[
https://issues.apache.org/jira/browse/FLINK-14667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16971112#comment-16971112
]
Jark Wu commented on FLINK-14667:
-
I will close this issue then, as it is not a bug.
> flink1.9,1.8.2 run flinkSql fat jar ,can't load the right tableFactory
> (TableSourceFactory) for the kafka
> --
>
> Key: FLINK-14667
> URL: https://issues.apache.org/jira/browse/FLINK-14667
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Client
>Affects Versions: 1.8.2, 1.9.1
>Reporter: chun11
>Priority: Major
>
> [root@mj flink-1.9.1]# ./bin/flink run -m yarn-cluster
> /root/cp19/streaming-view-0.0.1-SNAPSHOT-jar-with-dependencies.jar
> 2019-11-07 16:48:57,616 INFO org.apache.hadoop.yarn.client.RMProxy
> - Connecting to ResourceManager at /0.0.0.0:8032
> 2019-11-07 16:48:57,789 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli
> - No path for the flink jar passed. Using the location of class
> org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
> 2019-11-07 16:48:57,789 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli
> - No path for the flink jar passed. Using the location of class
> org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
> 2019-11-07 16:48:57,986 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Cluster
> specification: ClusterSpecification\{masterMemoryMB=1024,
> taskManagerMemoryMB=1024, numberTaskManagers=1, slotsPerTaskManager=1}
> 2019-11-07 16:48:58,657 WARN
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - The
> configuration directory ('/opt/app/flink-1.9.1/conf') contains both LOG4J and
> Logback configuration files. Please delete or rename one of them.
> 2019-11-07 16:49:00,954 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Submitting
> application master application_1573090964983_0039
> 2019-11-07 16:49:00,986 INFO
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted
> application application_1573090964983_0039
> 2019-11-07 16:49:00,986 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Waiting for
> the cluster to be allocated
> 2019-11-07 16:49:00,988 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Deploying
> cluster, current state ACCEPTED
> 2019-11-07 16:49:06,534 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - YARN
> application has been deployed successfully.
> Starting execution of program
>
>
> The program finished with the following exception:
>
> org.apache.flink.client.program.ProgramInvocationException: The main method
> caused an error: findAndCreateTableSource failed.
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:593)
> at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438)
> at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274)
> at
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)
> at
> org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:273)
> at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:205)
> at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1010)
> at
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1083)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1083)
> Caused by: org.apache.flink.table.api.TableException:
> findAndCreateTableSource failed.
> at
> org.apache.flink.table.factories.TableFactoryUtil.findAndCreateTableSource(TableFactoryUtil.java:67)
> at
> org.apache.flink.table.factories.TableFactoryUtil.findAndCreateTableSource(TableFactoryUtil.java:54)
> at
> org.apache.flink.table.descriptors.ConnectTableDescriptor.registerTableSource(ConnectTableDescriptor.java:69)
> at
> com.streaming.activity.task.PaymentViewIndex.registerKafkaTable(PaymentViewIndex.java:205)
> at
> com.streaming.activity.task.PaymentViewIndex.registerSourceTable(PaymentViewIndex.java:106)
> at
>
[jira] [Commented] (FLINK-14667) flink1.9,1.8.2 run flinkSql fat jar ,can't load the right tableFactory (TableSourceFactory) for the kafka
[
https://issues.apache.org/jira/browse/FLINK-14667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1697#comment-1697
]
Jark Wu commented on FLINK-14667:
-
Thanks [~wenlong.lwl] for the tip. I think the reason is service file is not
merged and {{ServicesResourceTransformer}} can solve this.
> flink1.9,1.8.2 run flinkSql fat jar ,can't load the right tableFactory
> (TableSourceFactory) for the kafka
> --
>
> Key: FLINK-14667
> URL: https://issues.apache.org/jira/browse/FLINK-14667
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Client
>Affects Versions: 1.8.2, 1.9.1
>Reporter: chun11
>Priority: Major
>
> [root@mj flink-1.9.1]# ./bin/flink run -m yarn-cluster
> /root/cp19/streaming-view-0.0.1-SNAPSHOT-jar-with-dependencies.jar
> 2019-11-07 16:48:57,616 INFO org.apache.hadoop.yarn.client.RMProxy
> - Connecting to ResourceManager at /0.0.0.0:8032
> 2019-11-07 16:48:57,789 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli
> - No path for the flink jar passed. Using the location of class
> org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
> 2019-11-07 16:48:57,789 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli
> - No path for the flink jar passed. Using the location of class
> org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
> 2019-11-07 16:48:57,986 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Cluster
> specification: ClusterSpecification\{masterMemoryMB=1024,
> taskManagerMemoryMB=1024, numberTaskManagers=1, slotsPerTaskManager=1}
> 2019-11-07 16:48:58,657 WARN
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - The
> configuration directory ('/opt/app/flink-1.9.1/conf') contains both LOG4J and
> Logback configuration files. Please delete or rename one of them.
> 2019-11-07 16:49:00,954 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Submitting
> application master application_1573090964983_0039
> 2019-11-07 16:49:00,986 INFO
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted
> application application_1573090964983_0039
> 2019-11-07 16:49:00,986 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Waiting for
> the cluster to be allocated
> 2019-11-07 16:49:00,988 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Deploying
> cluster, current state ACCEPTED
> 2019-11-07 16:49:06,534 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - YARN
> application has been deployed successfully.
> Starting execution of program
>
>
> The program finished with the following exception:
>
> org.apache.flink.client.program.ProgramInvocationException: The main method
> caused an error: findAndCreateTableSource failed.
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:593)
> at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438)
> at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274)
> at
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)
> at
> org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:273)
> at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:205)
> at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1010)
> at
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1083)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1083)
> Caused by: org.apache.flink.table.api.TableException:
> findAndCreateTableSource failed.
> at
> org.apache.flink.table.factories.TableFactoryUtil.findAndCreateTableSource(TableFactoryUtil.java:67)
> at
> org.apache.flink.table.factories.TableFactoryUtil.findAndCreateTableSource(TableFactoryUtil.java:54)
> at
> org.apache.flink.table.descriptors.ConnectTableDescriptor.registerTableSource(ConnectTableDescriptor.java:69)
> at
> com.streaming.activity.task.PaymentViewIndex.registerKafkaTable(PaymentViewIndex.java:205)
> at
>
[jira] [Commented] (FLINK-14667) flink1.9,1.8.2 run flinkSql fat jar ,can't load the right tableFactory (TableSourceFactory) for the kafka
[
https://issues.apache.org/jira/browse/FLINK-14667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969965#comment-16969965
]
chun11 commented on FLINK-14667:
You are right,Thank you very much (y):)
> flink1.9,1.8.2 run flinkSql fat jar ,can't load the right tableFactory
> (TableSourceFactory) for the kafka
> --
>
> Key: FLINK-14667
> URL: https://issues.apache.org/jira/browse/FLINK-14667
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Client
>Affects Versions: 1.8.2, 1.9.1
>Reporter: chun11
>Priority: Major
>
> [root@mj flink-1.9.1]# ./bin/flink run -m yarn-cluster
> /root/cp19/streaming-view-0.0.1-SNAPSHOT-jar-with-dependencies.jar
> 2019-11-07 16:48:57,616 INFO org.apache.hadoop.yarn.client.RMProxy
> - Connecting to ResourceManager at /0.0.0.0:8032
> 2019-11-07 16:48:57,789 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli
> - No path for the flink jar passed. Using the location of class
> org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
> 2019-11-07 16:48:57,789 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli
> - No path for the flink jar passed. Using the location of class
> org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
> 2019-11-07 16:48:57,986 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Cluster
> specification: ClusterSpecification\{masterMemoryMB=1024,
> taskManagerMemoryMB=1024, numberTaskManagers=1, slotsPerTaskManager=1}
> 2019-11-07 16:48:58,657 WARN
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - The
> configuration directory ('/opt/app/flink-1.9.1/conf') contains both LOG4J and
> Logback configuration files. Please delete or rename one of them.
> 2019-11-07 16:49:00,954 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Submitting
> application master application_1573090964983_0039
> 2019-11-07 16:49:00,986 INFO
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted
> application application_1573090964983_0039
> 2019-11-07 16:49:00,986 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Waiting for
> the cluster to be allocated
> 2019-11-07 16:49:00,988 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Deploying
> cluster, current state ACCEPTED
> 2019-11-07 16:49:06,534 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - YARN
> application has been deployed successfully.
> Starting execution of program
>
>
> The program finished with the following exception:
>
> org.apache.flink.client.program.ProgramInvocationException: The main method
> caused an error: findAndCreateTableSource failed.
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:593)
> at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438)
> at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274)
> at
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)
> at
> org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:273)
> at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:205)
> at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1010)
> at
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1083)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1083)
> Caused by: org.apache.flink.table.api.TableException:
> findAndCreateTableSource failed.
> at
> org.apache.flink.table.factories.TableFactoryUtil.findAndCreateTableSource(TableFactoryUtil.java:67)
> at
> org.apache.flink.table.factories.TableFactoryUtil.findAndCreateTableSource(TableFactoryUtil.java:54)
> at
> org.apache.flink.table.descriptors.ConnectTableDescriptor.registerTableSource(ConnectTableDescriptor.java:69)
> at
> com.streaming.activity.task.PaymentViewIndex.registerKafkaTable(PaymentViewIndex.java:205)
> at
> com.streaming.activity.task.PaymentViewIndex.registerSourceTable(PaymentViewIndex.java:106)
> at
>
[jira] [Commented] (FLINK-14667) flink1.9,1.8.2 run flinkSql fat jar ,can't load the right tableFactory (TableSourceFactory) for the kafka
[
https://issues.apache.org/jira/browse/FLINK-14667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969952#comment-16969952
]
chun11 commented on FLINK-14667:
Thank you for your answer(y) ！
You mean I need to setup flink-kafka-connector.jar in flink cluster ?
If I want to support anther MQ source for flinkSql ，I need to setup the jar
with user-defined-tableFactory . in the flink cluster even my fat jar contains
all of the class ?
My paltform is going to support flink SQL ,when my paltform user will submit
his flinksql fat jar program with another kind MQ source across my web
paltform . Setup his user defined jar in my flink cluster ?
:) :) :) :) :)
> flink1.9,1.8.2 run flinkSql fat jar ,can't load the right tableFactory
> (TableSourceFactory) for the kafka
> --
>
> Key: FLINK-14667
> URL: https://issues.apache.org/jira/browse/FLINK-14667
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Client
>Affects Versions: 1.8.2, 1.9.1
>Reporter: chun11
>Priority: Major
>
> [root@mj flink-1.9.1]# ./bin/flink run -m yarn-cluster
> /root/cp19/streaming-view-0.0.1-SNAPSHOT-jar-with-dependencies.jar
> 2019-11-07 16:48:57,616 INFO org.apache.hadoop.yarn.client.RMProxy
> - Connecting to ResourceManager at /0.0.0.0:8032
> 2019-11-07 16:48:57,789 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli
> - No path for the flink jar passed. Using the location of class
> org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
> 2019-11-07 16:48:57,789 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli
> - No path for the flink jar passed. Using the location of class
> org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
> 2019-11-07 16:48:57,986 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Cluster
> specification: ClusterSpecification\{masterMemoryMB=1024,
> taskManagerMemoryMB=1024, numberTaskManagers=1, slotsPerTaskManager=1}
> 2019-11-07 16:48:58,657 WARN
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - The
> configuration directory ('/opt/app/flink-1.9.1/conf') contains both LOG4J and
> Logback configuration files. Please delete or rename one of them.
> 2019-11-07 16:49:00,954 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Submitting
> application master application_1573090964983_0039
> 2019-11-07 16:49:00,986 INFO
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted
> application application_1573090964983_0039
> 2019-11-07 16:49:00,986 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Waiting for
> the cluster to be allocated
> 2019-11-07 16:49:00,988 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Deploying
> cluster, current state ACCEPTED
> 2019-11-07 16:49:06,534 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - YARN
> application has been deployed successfully.
> Starting execution of program
>
>
> The program finished with the following exception:
>
> org.apache.flink.client.program.ProgramInvocationException: The main method
> caused an error: findAndCreateTableSource failed.
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:593)
> at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438)
> at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274)
> at
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)
> at
> org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:273)
> at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:205)
> at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1010)
> at
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1083)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1083)
> Caused by: org.apache.flink.table.api.TableException:
> findAndCreateTableSource failed.
> at
> org.apache.flink.table.factories.TableFactoryUtil.findAndCreateTableSource(TableFactoryUtil.java:67)
> at
>
[jira] [Commented] (FLINK-14667) flink1.9,1.8.2 run flinkSql fat jar ,can't load the right tableFactory (TableSourceFactory) for the kafka
[
https://issues.apache.org/jira/browse/FLINK-14667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969947#comment-16969947
]
Wenlong Lyu commented on FLINK-14667:
-
[~chun11] I think the problem is the way you pack the fat jar is wrong. You
could try maven shade plugin with resource transformer， like following:
{code:java}
// code placeholder
org.apache.maven.plugins
maven-shade-plugin
3.1.0
package
shade
false
true
jar-with-dependencies
{code}
> flink1.9,1.8.2 run flinkSql fat jar ,can't load the right tableFactory
> (TableSourceFactory) for the kafka
> --
>
> Key: FLINK-14667
> URL: https://issues.apache.org/jira/browse/FLINK-14667
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Client
>Affects Versions: 1.8.2, 1.9.1
>Reporter: chun11
>Priority: Major
>
> [root@mj flink-1.9.1]# ./bin/flink run -m yarn-cluster
> /root/cp19/streaming-view-0.0.1-SNAPSHOT-jar-with-dependencies.jar
> 2019-11-07 16:48:57,616 INFO org.apache.hadoop.yarn.client.RMProxy
> - Connecting to ResourceManager at /0.0.0.0:8032
> 2019-11-07 16:48:57,789 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli
> - No path for the flink jar passed. Using the location of class
> org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
> 2019-11-07 16:48:57,789 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli
> - No path for the flink jar passed. Using the location of class
> org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
> 2019-11-07 16:48:57,986 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Cluster
> specification: ClusterSpecification\{masterMemoryMB=1024,
> taskManagerMemoryMB=1024, numberTaskManagers=1, slotsPerTaskManager=1}
> 2019-11-07 16:48:58,657 WARN
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - The
> configuration directory ('/opt/app/flink-1.9.1/conf') contains both LOG4J and
> Logback configuration files. Please delete or rename one of them.
> 2019-11-07 16:49:00,954 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Submitting
> application master application_1573090964983_0039
> 2019-11-07 16:49:00,986 INFO
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted
> application application_1573090964983_0039
> 2019-11-07 16:49:00,986 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Waiting for
> the cluster to be allocated
> 2019-11-07 16:49:00,988 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Deploying
> cluster, current state ACCEPTED
> 2019-11-07 16:49:06,534 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - YARN
> application has been deployed successfully.
> Starting execution of program
>
>
> The program finished with the following exception:
>
> org.apache.flink.client.program.ProgramInvocationException: The main method
> caused an error: findAndCreateTableSource failed.
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:593)
> at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438)
> at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274)
> at
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)
> at
> org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:273)
> at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:205)
> at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1010)
> at
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1083)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1083)
> Caused by: org.apache.flink.table.api.TableException:
> findAndCreateTableSource failed.
> at
> org.apache.flink.table.factories.TableFactoryUtil.findAndCreateTableSource(TableFactoryUtil.java:67)
> at
>
[jira] [Commented] (FLINK-14667) flink1.9,1.8.2 run flinkSql fat jar ,can't load the right tableFactory (TableSourceFactory) for the kafka
[
https://issues.apache.org/jira/browse/FLINK-14667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969939#comment-16969939
]
Jark Wu commented on FLINK-14667:
-
Hi [~chun11], the reason is that there is no kafka sql connector jar in
your cluster setup.
You can read documentation here and download your Kafka version SQL CLI JAR :
https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/table/connect.html#connectors
> flink1.9,1.8.2 run flinkSql fat jar ,can't load the right tableFactory
> (TableSourceFactory) for the kafka
> --
>
> Key: FLINK-14667
> URL: https://issues.apache.org/jira/browse/FLINK-14667
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Client
>Affects Versions: 1.8.2, 1.9.1
>Reporter: chun11
>Priority: Major
>
> [root@mj flink-1.9.1]# ./bin/flink run -m yarn-cluster
> /root/cp19/streaming-view-0.0.1-SNAPSHOT-jar-with-dependencies.jar
> 2019-11-07 16:48:57,616 INFO org.apache.hadoop.yarn.client.RMProxy
> - Connecting to ResourceManager at /0.0.0.0:8032
> 2019-11-07 16:48:57,789 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli
> - No path for the flink jar passed. Using the location of class
> org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
> 2019-11-07 16:48:57,789 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli
> - No path for the flink jar passed. Using the location of class
> org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
> 2019-11-07 16:48:57,986 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Cluster
> specification: ClusterSpecification\{masterMemoryMB=1024,
> taskManagerMemoryMB=1024, numberTaskManagers=1, slotsPerTaskManager=1}
> 2019-11-07 16:48:58,657 WARN
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - The
> configuration directory ('/opt/app/flink-1.9.1/conf') contains both LOG4J and
> Logback configuration files. Please delete or rename one of them.
> 2019-11-07 16:49:00,954 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Submitting
> application master application_1573090964983_0039
> 2019-11-07 16:49:00,986 INFO
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted
> application application_1573090964983_0039
> 2019-11-07 16:49:00,986 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Waiting for
> the cluster to be allocated
> 2019-11-07 16:49:00,988 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Deploying
> cluster, current state ACCEPTED
> 2019-11-07 16:49:06,534 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - YARN
> application has been deployed successfully.
> Starting execution of program
>
>
> The program finished with the following exception:
>
> org.apache.flink.client.program.ProgramInvocationException: The main method
> caused an error: findAndCreateTableSource failed.
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:593)
> at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438)
> at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274)
> at
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)
> at
> org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:273)
> at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:205)
> at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1010)
> at
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1083)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1083)
> Caused by: org.apache.flink.table.api.TableException:
> findAndCreateTableSource failed.
> at
> org.apache.flink.table.factories.TableFactoryUtil.findAndCreateTableSource(TableFactoryUtil.java:67)
> at
> org.apache.flink.table.factories.TableFactoryUtil.findAndCreateTableSource(TableFactoryUtil.java:54)
> at
> org.apache.flink.table.descriptors.ConnectTableDescriptor.registerTableSource(ConnectTableDescriptor.java:69)
> at
>
[jira] [Commented] (FLINK-14667) flink1.9,1.8.2 run flinkSql fat jar ,can't load the right tableFactory (TableSourceFactory) for the kafka
[
https://issues.apache.org/jira/browse/FLINK-14667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969870#comment-16969870
]
chun11 commented on FLINK-14667:
The Kafka011TableSourceSinkFactory is packaged in the fat jar .
> flink1.9,1.8.2 run flinkSql fat jar ,can't load the right tableFactory
> (TableSourceFactory) for the kafka
> --
>
> Key: FLINK-14667
> URL: https://issues.apache.org/jira/browse/FLINK-14667
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Client
>Affects Versions: 1.8.2, 1.9.1
>Reporter: chun11
>Priority: Major
>
> [root@mj flink-1.9.1]# ./bin/flink run -m yarn-cluster
> /root/cp19/streaming-view-0.0.1-SNAPSHOT-jar-with-dependencies.jar[root@mj
> flink-1.9.1]# ./bin/flink run -m yarn-cluster
> /root/cp19/streaming-view-0.0.1-SNAPSHOT-jar-with-dependencies.jar2019-11-07
> 16:48:57,616 INFO org.apache.hadoop.yarn.client.RMProxy
> - Connecting to ResourceManager at /0.0.0.0:80322019-11-07 16:48:57,789
> INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - No path
> for the flink jar passed. Using the location of class
> org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2019-11-07
> 16:48:57,789 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli
> - No path for the flink jar passed. Using the location of class
> org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2019-11-07
> 16:48:57,986 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor
> - Cluster specification: ClusterSpecification\{masterMemoryMB=1024,
> taskManagerMemoryMB=1024, numberTaskManagers=1,
> slotsPerTaskManager=1}2019-11-07 16:48:58,657 WARN
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - The
> configuration directory ('/opt/app/flink-1.9.1/conf') contains both LOG4J and
> Logback configuration files. Please delete or rename one of them.2019-11-07
> 16:49:00,954 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor
> - Submitting application master application_1573090964983_00392019-11-07
> 16:49:00,986 INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl
> - Submitted application application_1573090964983_00392019-11-07
> 16:49:00,986 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor
> - Waiting for the cluster to be allocated2019-11-07 16:49:00,988 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - Deploying
> cluster, current state ACCEPTED2019-11-07 16:49:06,534 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor - YARN
> application has been deployed successfully.Starting execution of program
> The program
> finished with the following exception:
> org.apache.flink.client.program.ProgramInvocationException: The main method
> caused an error: findAndCreateTableSource failed. at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:593)
> at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438)
> at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274)
> at
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)
> at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:273)
> at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:205) at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1010)
> at
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1083)
> at java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1083)Caused
> by: org.apache.flink.table.api.TableException: findAndCreateTableSource
> failed. at
> org.apache.flink.table.factories.TableFactoryUtil.findAndCreateTableSource(TableFactoryUtil.java:67)
> at
> org.apache.flink.table.factories.TableFactoryUtil.findAndCreateTableSource(TableFactoryUtil.java:54)
> at
> org.apache.flink.table.descriptors.ConnectTableDescriptor.registerTableSource(ConnectTableDescriptor.java:69)
> at
> com.streaming.activity.task.PaymentViewIndex.registerKafkaTable(PaymentViewIndex.java:205)
> at
> com.streaming.activity.task.PaymentViewIndex.registerSourceTable(PaymentViewIndex.java:106)
> at
>
