[
https://issues.apache.org/jira/browse/FLINK-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14538348#comment-14538348
]
Stephan Ewen commented on FLINK-1989:
-------------------------------------
Storing the type information breaks with the current design principles, where
the {{TypeInformation}} is a pure pre-flight concept, and the
{{TypeSerializer}} and {{TypeComparator}} are the runtime handles.
> Sorting of POJO data set from TableEnv yields NotSerializableException
> ----------------------------------------------------------------------
>
> Key: FLINK-1989
> URL: https://issues.apache.org/jira/browse/FLINK-1989
> Project: Flink
> Issue Type: Bug
> Components: Table API
> Affects Versions: 0.9
> Reporter: Fabian Hueske
> Assignee: Aljoscha Krettek
> Fix For: 0.9
>
>
> Sorting or grouping (or probably any other key operation) on a POJO data set
> that was created by a {{TableEnvironment}} yields a
> {{NotSerializableException}} due to a non-serializable
> {{java.lang.reflect.Field}} object.
> I traced the error back to the {{ExpressionSelectFunction}}. I guess that a
> {{TypeInformation}} object is stored in the generated user-code function. A
> {{PojoTypeInfo}} holds Field objects, which cannot be serialized.
> The following test can be pasted into the {{SelectITCase}} and reproduces the
> problem.
> {code}
> @Test
> public void testGroupByAfterTable() throws Exception {
> ExecutionEnvironment env =
> ExecutionEnvironment.getExecutionEnvironment();
> TableEnvironment tableEnv = new TableEnvironment();
> DataSet<Tuple3<Integer, Long, String>> ds =
> CollectionDataSets.get3TupleDataSet(env);
> Table in = tableEnv.toTable(ds, "a,b,c");
> Table result = in
> .select("a, b, c");
> DataSet<ABC> resultSet = tableEnv.toSet(result, ABC.class);
> resultSet
> .sortPartition("a", Order.DESCENDING)
> .writeAsText(resultPath,
> FileSystem.WriteMode.OVERWRITE);
> env.execute();
> expected = "1,1,Hi\n" + "2,2,Hello\n" + "3,2,Hello world\n" +
> "4,3,Hello world, " +
> "how are you?\n" + "5,3,I am fine.\n" + "6,3,Luke
> Skywalker\n" + "7,4," +
> "Comment#1\n" + "8,4,Comment#2\n" + "9,4,Comment#3\n" +
> "10,4,Comment#4\n" + "11,5," +
> "Comment#5\n" + "12,5,Comment#6\n" + "13,5,Comment#7\n"
> + "14,5,Comment#8\n" + "15,5," +
> "Comment#9\n" + "16,6,Comment#10\n" +
> "17,6,Comment#11\n" + "18,6,Comment#12\n" + "19," +
> "6,Comment#13\n" + "20,6,Comment#14\n" +
> "21,6,Comment#15\n";
> }
> public static class ABC {
> public int a;
> public long b;
> public String c;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
