Liu Chong created FLINK-33197:
---------------------------------
Summary: PyFlink support for ByteArraySchema
Key: FLINK-33197
URL: https://issues.apache.org/jira/browse/FLINK-33197
Project: Flink
Issue Type: New Feature
Components: API / Python
Affects Versions: 1.17.0
Reporter: Liu Chong
Currently in Python Flink API, when reading messages from a Kafka source, only
SimpleStringSchema is available.
If the data is in arbitary binary format(e.g. marshalled Protocol Buffer msg)
it may not be decodable with the default 'utf-8' encoding.
There's currently a workaround which is to manually set the encoding to
'ISO-8859-1' which supports all possible byte combinations.
However this is not an elegant solution.
We should support ByteArraySchema which outputs a raw byte array for subsequent
unmarshalling.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
