[
https://issues.apache.org/jira/browse/FLINK-36769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-36769:
-----------------------------------
Labels: pull-request-available (was: )
> Suport fury Serializer for pyflink
> ----------------------------------
>
> Key: FLINK-36769
> URL: https://issues.apache.org/jira/browse/FLINK-36769
> Project: Flink
> Issue Type: New Feature
> Components: API / Python
> Affects Versions: 1.16.1
> Environment: flink 1.16.1
> Reporter: xingyuan cheng
> Priority: Major
> Labels: pull-request-available
>
> Hi, community. Currently, in the batch verification scenario of our algorithm
> data, we use pyflink and encounter low transmission efficiency caused by low
> performance of pickle4-based encoding. After research, we decided to adopt
> Apache fury, a serialization framework based on pickle5 encoding. The
> implementation of fury in python will define the transmission buffer size in
> the protocol for transmission to improve the performance of large data
> transmission. To this end, I Prepared a draft pull request. What do friends
> in the community think about this?
>
> Pickle protocol 5 with out-of-band data: https://peps.python.org/pep-0574/
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
