[ https://issues.apache.org/jira/browse/FLINK-26793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518077#comment-17518077 ]
Etienne Chauchot commented on FLINK-26793: ------------------------------------------ [~bumblebee], in the meantime, I made a [pure-flink test pipeline|https://github.com/echauchot/flink/blob/FLINK-26793-cassandra-perf/flink-connectors/flink-connector-cassandra/src/test/java/org/apache/flink/streaming/connectors/cassandra/example/CassandraPojoSinkStreamingExample.java] that runs in streaming mode (so it is infinite) with a fake source that provides Pojos every 100 ms and write them to Cassandra. I've run this pipeline locally for 2 hours with no perf log messages. I'll try to run them for 72 hours or more on my Azure environment (hoping that I can get a Cassandra instance there). That way, I'll try to reproduce the performance issue logs you had. If I can't reproduce after some days of running this pipeline, I think the good way to proceed is to take a look at your Flink restore stats hence my question above or take a look at the Scylla connector you use. > Flink Cassandra connector performance issue > -------------------------------------------- > > Key: FLINK-26793 > URL: https://issues.apache.org/jira/browse/FLINK-26793 > Project: Flink > Issue Type: Improvement > Components: Connectors / Cassandra > Affects Versions: 1.14.4 > Reporter: Jay Ghiya > Assignee: Etienne Chauchot > Priority: Major > > A warning is observed during long runs of flink job stating “Insertions into > scylla might be suffering. Expect performance problems unless this is > resolved.” > Upon initial analysis - “flink cassandra connector is not keeping instance of > mapping manager that is used to convert a pojo to cassandra row. Ideally the > mapping manager should have the same life time as cluster and session objects > which are also created once when the driver is initialized” > Reference: > https://stackoverflow.com/questions/59203418/cassandra-java-driver-warning -- This message was sent by Atlassian Jira (v8.20.1#820001)