Hi everyone, why does flink xonsider densevector or vectors as a raw
types'features: RAW('org.apache.flink.ml.linalg.Vector', '...')'?
I'm trying to deploy my job on flink and i have this error
Server Response:
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not execute
application. at
org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$handleRequest$1(JarRunHandler.java:113)
at java.base/java.util.concurrent.CompletableFuture.uniHandle(Unknown Source)
at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(Unknown
Source) at
java.base/java.util.concurrent.CompletableFuture.postComplete(Unknown Source)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown
Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by:
java.util.concurrent.CompletionException:
org.apache.flink.util.FlinkRuntimeException: Could not execute application. at
java.base/java.util.concurrent.CompletableFuture.encodeThrowable(Unknown
Source) at
java.base/java.util.concurrent.CompletableFuture.completeThrowable(Unknown
Source) ... 2 more Caused by: org.apache.flink.util.FlinkRuntimeException:
Could not execute application. at
org.apache.flink.client.deployment.application.DetachedApplicationRunner.tryExecuteJobs(DetachedApplicationRunner.java:88)
at
org.apache.flink.client.deployment.application.DetachedApplicationRunner.run(DetachedApplicationRunner.java:70)
at
org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$handleRequest$0(JarRunHandler.java:107)
... 2 more Caused by:
org.apache.flink.client.program.ProgramInvocationException: The main method
caused an error: Column types of query result and sink for
'*anonymous_datastream_sink$16*' do not match. Cause: Incompatible types for
sink column 'features' at position 15. Query schema: [tripId: INT, stopId: INT,
routeId: INT, stopSequence: INT, speed: DOUBLE, currentStatus: INT, temp_max:
DOUBLE, temp_min: DOUBLE, visibility: INT, dayOfWeek: INT, distance: DOUBLE,
hour: DOUBLE, weatherConditionId: INT, bearing: DOUBLE, arrival_delay: DOUBLE,
features: RAW('org.apache.flink.ml.linalg.Vector', '...')] Sink schema:
[tripId: INT, stopId: INT, routeId: INT, stopSequence: INT, speed: DOUBLE,
currentStatus: INT, temp_max: DOUBLE, temp_min: DOUBLE, visibility: INT,
dayOfWeek: INT, distance: DOUBLE, hour: DOUBLE, weatherConditionId: INT,
bearing: DOUBLE, arrival_delay: DOUBLE, features:
RAW('org.apache.flink.ml.linalg.Vector', '...')] at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372)
at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114) at
org.apache.flink.client.deployment.application.DetachedApplicationRunner.tryExecuteJobs(DetachedApplicationRunner.java:84)
... 4 more Caused by: org.apache.flink.table.api.ValidationException: Column
types of query result and sink for '*anonymous_datastream_sink$16*' do not
match. Cause: Incompatible types for sink column 'features' at position 15.
Query schema: [tripId: INT, stopId: INT, routeId: INT, stopSequence: INT,
speed: DOUBLE, currentStatus: INT, temp_max: DOUBLE, temp_min: DOUBLE,
visibility: INT, dayOfWeek: INT, distance: DOUBLE, hour: DOUBLE,
weatherConditionId: INT, bearing: DOUBLE, arrival_delay: DOUBLE, features:
RAW('org.apache.flink.ml.linalg.Vector', '...')] Sink schema: [tripId: INT,
stopId: INT, routeId: INT, stopSequence: INT, speed: DOUBLE, currentStatus:
INT, temp_max: DOUBLE, temp_min: DOUBLE, visibility: INT, dayOfWeek: INT,
distance: DOUBLE, hour: DOUBLE, weatherConditionId: INT, bearing: DOUBLE,
arrival_delay: DOUBLE, features: RAW('org.apache.flink.ml.linalg.Vector',
'...')] at
org.apache.flink.table.planner.connectors.DynamicSinkUtils.createSchemaMismatchException(DynamicSinkUtils.java:453)
at
org.apache.flink.table.planner.connectors.DynamicSinkUtils.validateSchemaAndApplyImplicitCast(DynamicSinkUtils.java:265)
at
org.apache.flink.table.planner.connectors.DynamicSinkUtils.convertSinkToRel(DynamicSinkUtils.java:208)
at
org.apache.flink.table.planner.connectors.DynamicSinkUtils.convertExternalToRel(DynamicSinkUtils.java:151)
at
org.apache.flink.table.planner.delegation.PlannerBase.translateToRel(PlannerBase.scala:268)
at
org.apache.flink.table.planner.delegation.PlannerBase.$anonfun$translate$1(PlannerBase.scala:186)
at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:233)
at scala.collection.Iterator.foreach(Iterator.scala:937) at
scala.collection.Iterator.foreach$(Iterator.scala:937) at
scala.collection.AbstractIterator.foreach(Iterator.scala:1425) at
scala.collection.IterableLike.foreach(IterableLike.scala:70) at
scala.collection.IterableLike.foreach$(IterableLike.scala:69) at
scala.collection.AbstractIterable.foreach(Iterable.scala:54) at
scala.collection.TraversableLike.map(TraversableLike.scala:233) at
scala.collection.TraversableLike.map$(TraversableLike.scala:226) at
scala.collection.AbstractTraversable.map(Traversable.scala:104) at
org.apache.flink.table.planner.delegation.PlannerBase.translate(PlannerBase.scala:186)
at
org.apache.flink.table.api.bridge.internal.AbstractStreamTableEnvironmentImpl.toStreamInternal(AbstractStreamTableEnvironmentImpl.java:223)
at
org.apache.flink.table.api.bridge.internal.AbstractStreamTableEnvironmentImpl.toStreamInternal(AbstractStreamTableEnvironmentImpl.java:218)
at
org.apache.flink.table.api.bridge.java.internal.StreamTableEnvironmentImpl.toDataStream(StreamTableEnvironmentImpl.java:245)
at
org.apache.flink.table.api.bridge.java.internal.StreamTableEnvironmentImpl.toDataStream(StreamTableEnvironmentImpl.java:218)
at org.apache.flink.Main.main(Main.java:88) at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown
Source) at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
... 7 more
