[jira] [Commented] (FLINK-16662) Blink Planner failed to generate JobGraph for POJO DataStream converting to Table (Cannot determine simple type name)
[
https://issues.apache.org/jira/browse/FLINK-16662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091155#comment-17091155
]
chenxyz commented on FLINK-16662:
-
[~jark] thx, you're right, end-to-end test is a better way for running in flink
cluster and testing the whole lifecycle
> Blink Planner failed to generate JobGraph for POJO DataStream converting to
> Table (Cannot determine simple type name)
> -
>
> Key: FLINK-16662
> URL: https://issues.apache.org/jira/browse/FLINK-16662
> Project: Flink
> Issue Type: Bug
> Components: Client / Job Submission, Table SQL / Planner
>Affects Versions: 1.10.0
>Reporter: chenxyz
>Assignee: LionelZ
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.10.1, 1.11.0
>
>
> When using Blink Palnner to convert a POJO DataStream to a Table, Blink will
> generate and compile the SourceConversion$1 code. If the Jar task is
> submitted to Flink, since the UserCodeClassLoader is not used when generating
> the JobGraph, the ClassLoader(AppClassLoader) of the compiled code cannot
> load the POJO class in the Jar package, so the following error will be
> reported:
>
> {code:java}
> Caused by: org.codehaus.commons.compiler.CompileException: Line 27, Column
> 174: Cannot determine simple type name "net"Caused by:
> org.codehaus.commons.compiler.CompileException: Line 27, Column 174: Cannot
> determine simple type name "net" at
> org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12124) at
> org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6746) at
> org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6507) at
> org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
> org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
> org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
> org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
> org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
> org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
> org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
> org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
> org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6486) at
> org.codehaus.janino.UnitCompiler.access$13800(UnitCompiler.java:215) at
> org.codehaus.janino.UnitCompiler$21$1.visitReferenceType(UnitCompiler.java:6394)
> at
> org.codehaus.janino.UnitCompiler$21$1.visitReferenceType(UnitCompiler.java:6389)
> at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3917) at
> org.codehaus.janino.UnitCompiler$21.visitType(UnitCompiler.java:6389) at
> org.codehaus.janino.UnitCompiler$21.visitType(UnitCompiler.java:6382) at
> org.codehaus.janino.Java$ReferenceType.accept(Java.java:3916) at
> org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6382) at
> org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:7009) at
> org.codehaus.janino.UnitCompiler.access$15200(UnitCompiler.java:215) at
> org.codehaus.janino.UnitCompiler$21$2.visitCast(UnitCompiler.java:6425) at
> org.codehaus.janino.UnitCompiler$21$2.visitCast(UnitCompiler.java:6403) at
> org.codehaus.janino.Java$Cast.accept(Java.java:4887) at
> org.codehaus.janino.UnitCompiler$21.visitRvalue(UnitCompiler.java:6403) at
> org.codehaus.janino.UnitCompiler$21.visitRvalue(UnitCompiler.java:6382) at
> org.codehaus.janino.Java$Rvalue.accept(Java.java:4105) at
> org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6382) at
> org.codehaus.janino.UnitCompiler.findMostSpecificIInvocable(UnitCompiler.java:9150)
> at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:9036) at
> org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:8938) at
> org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5060) at
> org.codehaus.janino.UnitCompiler.access$9100(UnitCompiler.java:215) at
> org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(UnitCompiler.java:4421)
> at
> org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(UnitCompiler.java:4394)
> at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:5062) at
> org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4394) at
> org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5575) at
> org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5019) at
> org.codehaus.janino.UnitCompiler.access$8600(UnitCompiler.java:215) at
> org.codehaus.janino.UnitCompiler$16.visitCast(UnitCompiler.java:4416) at
> org.codehaus.janino.UnitCompiler$16.visitCast(
[jira] [Commented] (FLINK-16662) Blink Planner failed to generate JobGraph for POJO DataStream converting to Table (Cannot determine simple type name)
[
https://issues.apache.org/jira/browse/FLINK-16662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17090816#comment-17090816
]
chenxyz commented on FLINK-16662:
-
Hi,[~kkl0u], [~zhanglibing1...@126.com], thx for fix. I'm reporter of this
issue.I think I can add a test. The root cause is current classloader
(AppClassLoader, jvm default) cannot find user class, but find in
UserCodeClassLoader. so we can generate job graph from a blink jar, which
contains user code (udf, customer pojos,...).The test class execute in
AppClassLoader (default), user class load in UserCodeClassLoader. It can be
recured locally.
I submited PR for the
test[11889|[https://github.com/apache/flink/pull/11889]],
[11890|[https://github.com/apache/flink/pull/11890]]
> Blink Planner failed to generate JobGraph for POJO DataStream converting to
> Table (Cannot determine simple type name)
> -
>
> Key: FLINK-16662
> URL: https://issues.apache.org/jira/browse/FLINK-16662
> Project: Flink
> Issue Type: Bug
> Components: Client / Job Submission, Table SQL / Planner
>Affects Versions: 1.10.0
>Reporter: chenxyz
>Assignee: LionelZ
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.10.1, 1.11.0
>
>
> When using Blink Palnner to convert a POJO DataStream to a Table, Blink will
> generate and compile the SourceConversion$1 code. If the Jar task is
> submitted to Flink, since the UserCodeClassLoader is not used when generating
> the JobGraph, the ClassLoader(AppClassLoader) of the compiled code cannot
> load the POJO class in the Jar package, so the following error will be
> reported:
>
> {code:java}
> Caused by: org.codehaus.commons.compiler.CompileException: Line 27, Column
> 174: Cannot determine simple type name "net"Caused by:
> org.codehaus.commons.compiler.CompileException: Line 27, Column 174: Cannot
> determine simple type name "net" at
> org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12124) at
> org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6746) at
> org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6507) at
> org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
> org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
> org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
> org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
> org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
> org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
> org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
> org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
> org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6486) at
> org.codehaus.janino.UnitCompiler.access$13800(UnitCompiler.java:215) at
> org.codehaus.janino.UnitCompiler$21$1.visitReferenceType(UnitCompiler.java:6394)
> at
> org.codehaus.janino.UnitCompiler$21$1.visitReferenceType(UnitCompiler.java:6389)
> at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3917) at
> org.codehaus.janino.UnitCompiler$21.visitType(UnitCompiler.java:6389) at
> org.codehaus.janino.UnitCompiler$21.visitType(UnitCompiler.java:6382) at
> org.codehaus.janino.Java$ReferenceType.accept(Java.java:3916) at
> org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6382) at
> org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:7009) at
> org.codehaus.janino.UnitCompiler.access$15200(UnitCompiler.java:215) at
> org.codehaus.janino.UnitCompiler$21$2.visitCast(UnitCompiler.java:6425) at
> org.codehaus.janino.UnitCompiler$21$2.visitCast(UnitCompiler.java:6403) at
> org.codehaus.janino.Java$Cast.accept(Java.java:4887) at
> org.codehaus.janino.UnitCompiler$21.visitRvalue(UnitCompiler.java:6403) at
> org.codehaus.janino.UnitCompiler$21.visitRvalue(UnitCompiler.java:6382) at
> org.codehaus.janino.Java$Rvalue.accept(Java.java:4105) at
> org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6382) at
> org.codehaus.janino.UnitCompiler.findMostSpecificIInvocable(UnitCompiler.java:9150)
> at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:9036) at
> org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:8938) at
> org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5060) at
> org.codehaus.janino.UnitCompiler.access$9100(UnitCompiler.java:215) at
> org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(UnitCompiler.java:4421)
> at
> org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(UnitCompiler.java:4394)
> at org.codehaus.janino.Java$
[jira] [Created] (FLINK-16662) Blink Planner failed to generate JobGraph for POJO DataStream converting to Table (Cannot determine simple type name)
chenxyz created FLINK-16662:
---
Summary: Blink Planner failed to generate JobGraph for POJO
DataStream converting to Table (Cannot determine simple type name)
Key: FLINK-16662
URL: https://issues.apache.org/jira/browse/FLINK-16662
Project: Flink
Issue Type: Bug
Components: API / DataStream
Affects Versions: 1.10.0
Reporter: chenxyz
When using Blink Palnner to convert a POJO DataStream to a Table, Blink will
generate and compile the SourceConversion$1 code. If the Jar task is submitted
to Flink, since the UserCodeClassLoader is not used when generating the
JobGraph, the ClassLoader(AppClassLoader) of the compiled code cannot load the
POJO class in the Jar package, so the following error will be reported:
{code:java}
Caused by: org.codehaus.commons.compiler.CompileException: Line 27, Column 174:
Cannot determine simple type name "net"Caused by:
org.codehaus.commons.compiler.CompileException: Line 27, Column 174: Cannot
determine simple type name "net" at
org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12124) at
org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6746) at
org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6507) at
org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6520) at
org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6486) at
org.codehaus.janino.UnitCompiler.access$13800(UnitCompiler.java:215) at
org.codehaus.janino.UnitCompiler$21$1.visitReferenceType(UnitCompiler.java:6394)
at
org.codehaus.janino.UnitCompiler$21$1.visitReferenceType(UnitCompiler.java:6389)
at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3917) at
org.codehaus.janino.UnitCompiler$21.visitType(UnitCompiler.java:6389) at
org.codehaus.janino.UnitCompiler$21.visitType(UnitCompiler.java:6382) at
org.codehaus.janino.Java$ReferenceType.accept(Java.java:3916) at
org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6382) at
org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:7009) at
org.codehaus.janino.UnitCompiler.access$15200(UnitCompiler.java:215) at
org.codehaus.janino.UnitCompiler$21$2.visitCast(UnitCompiler.java:6425) at
org.codehaus.janino.UnitCompiler$21$2.visitCast(UnitCompiler.java:6403) at
org.codehaus.janino.Java$Cast.accept(Java.java:4887) at
org.codehaus.janino.UnitCompiler$21.visitRvalue(UnitCompiler.java:6403) at
org.codehaus.janino.UnitCompiler$21.visitRvalue(UnitCompiler.java:6382) at
org.codehaus.janino.Java$Rvalue.accept(Java.java:4105) at
org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6382) at
org.codehaus.janino.UnitCompiler.findMostSpecificIInvocable(UnitCompiler.java:9150)
at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:9036) at
org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:8938) at
org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5060) at
org.codehaus.janino.UnitCompiler.access$9100(UnitCompiler.java:215) at
org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(UnitCompiler.java:4421)
at
org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(UnitCompiler.java:4394)
at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:5062) at
org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4394) at
org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5575) at
org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5019) at
org.codehaus.janino.UnitCompiler.access$8600(UnitCompiler.java:215) at
org.codehaus.janino.UnitCompiler$16.visitCast(UnitCompiler.java:4416) at
org.codehaus.janino.UnitCompiler$16.visitCast(UnitCompiler.java:4394) at
org.codehaus.janino.Java$Cast.accept(Java.java:4887) at
org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4394) at
org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5575) at
org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5019) at
org.codehaus.janino.UnitCompiler.access$8600(UnitCompiler.java:215) at
org.codehaus.janino.UnitCompiler$16.visitCast(UnitCompiler.java:4416) at
org.codehaus.janino.UnitCompiler$16.visitCast(UnitCompiler.java:4394) at
org.codehaus.janino.Java$Cast.accept(Java.java:4887) at
org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4394) at
org.codehaus.janino.UnitCompiler.compileGetValue(Un
