[ 
https://issues.apache.org/jira/browse/FLINK-26793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518077#comment-17518077
 ] 

Etienne Chauchot commented on FLINK-26793:
------------------------------------------

[~bumblebee], in the meantime, I made a [pure-flink test 
pipeline|https://github.com/echauchot/flink/blob/FLINK-26793-cassandra-perf/flink-connectors/flink-connector-cassandra/src/test/java/org/apache/flink/streaming/connectors/cassandra/example/CassandraPojoSinkStreamingExample.java]
 that runs in streaming mode (so it is infinite) with a fake source that 
provides Pojos every 100 ms and write them to Cassandra. I've run this pipeline 
locally for 2 hours with no perf log messages. I'll try to run them for 72 
hours or more  on my Azure environment (hoping that I can get a Cassandra 
instance there). That way, I'll try to reproduce the performance issue logs you 
had. If I can't reproduce after some days of running this pipeline, I think the 
good way to proceed is to take a look at your Flink restore stats hence my 
question above or take a look at the Scylla connector you use.

> Flink Cassandra connector performance issue 
> --------------------------------------------
>
>                 Key: FLINK-26793
>                 URL: https://issues.apache.org/jira/browse/FLINK-26793
>             Project: Flink
>          Issue Type: Improvement
>          Components: Connectors / Cassandra
>    Affects Versions: 1.14.4
>            Reporter: Jay Ghiya
>            Assignee: Etienne Chauchot
>            Priority: Major
>
> A warning is observed during long runs of flink job stating “Insertions into 
> scylla might be suffering. Expect performance problems unless this is 
> resolved.”
> Upon initial analysis - “flink cassandra connector is not keeping instance of 
> mapping manager that is used to convert a pojo to cassandra row. Ideally the 
> mapping manager should have the same life time as cluster and session objects 
> which are also created once when the driver is initialized”
> Reference: 
> https://stackoverflow.com/questions/59203418/cassandra-java-driver-warning



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Reply via email to