jiabao sun created FLINK-21172:
----------------------------------
Summary: canal-json format include es field
Key: FLINK-21172
URL: https://issues.apache.org/jira/browse/FLINK-21172
Project: Flink
Issue Type: Improvement
Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
Affects Versions: 1.12.1, 1.12.0
Reporter: jiabao sun
Canal flat message json format has an 'es' field extracted from mysql binlog
which means the row data real change time in mysql. It expressed the event time
naturally but is ignored during deserialization.
org.apache.flink.formats.json.canal. CanalJsonDeserializationSchema
{code:java}
private static RowType createJsonRowType(DataType databaseSchema) {
// Canal JSON contains other information, e.g. "ts", "sql", but we
don't need them
return (RowType)
DataTypes.ROW(
DataTypes.FIELD("data",
DataTypes.ARRAY(databaseSchema)),
DataTypes.FIELD("old",
DataTypes.ARRAY(databaseSchema)),
DataTypes.FIELD("type", DataTypes.STRING()),
DataTypes.FIELD("database", DataTypes.STRING()),
DataTypes.FIELD("table", DataTypes.STRING()))
.getLogicalType();
}
{code}
[Canal Flat Message|https://github.com/alibaba/canal/issues/2170]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
