[
https://issues.apache.org/jira/browse/FLINK-13479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17323212#comment-17323212
]
Flink Jira Bot commented on FLINK-13479:
----------------------------------------
This issue is assigned but has not received an update in 7 days so it has been
labeled "stale-assigned". If you are still working on the issue, please give an
update and remove the label. If you are no longer working on the issue, please
unassign so someone else may work on it. In 7 days the issue will be
automatically unassigned.
> Cassandra POJO Sink - Prepared Statement query does not have deterministic
> ordering of columns - causing prepared statement cache overflow
> ------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-13479
> URL: https://issues.apache.org/jira/browse/FLINK-13479
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Cassandra
> Affects Versions: 1.7.2
> Reporter: Ronak Thakrar
> Assignee: Ronak Thakrar
> Priority: Major
> Labels: pull-request-available, stale-assigned
> Time Spent: 20m
> Remaining Estimate: 0h
>
> While using Cassandra POJO Sink as part of Flink Jobs - prepared statements
> query string which is automatically generated while inserting the data(using
> Mapper.saveQuery method), Cassandra entity does not have deterministic
> ordering enforced-so every time column position is changed a new prepared
> statement is generated and used. As an effect of that prepared statement
> query cache is overflown because every time when insert statement query
> string is generated by - columns are in random order.
> Following is the detailed explanation for what happens inside the Datastax
> java driver([https://datastax-oss.atlassian.net/browse/JAVA-1587]):
> The current Mapper uses random ordering of columns when it creates prepared
> queries. This is fine when only 1 java client is accessing a cluster (and
> assuming the application developer does the correct thing by re-using a
> Mapper), since each Mapper will reused prepared statement. However when you
> have many java clients accessing a cluster, they will each create their own
> permutations of column ordering, and can thrash the prepared statement cache
> on the cluster.
> I propose that the Mapper uses a TreeMap instead of a HashMap when it builds
> its set of AliasedMappedProperty - sorted by the column name
> (col.mappedProperty.getMappedName()). This would create a deterministic
> ordering of columns, and all java processes accessing the same cluster would
> end up with the same prepared queries for the same entities.
> This issue is already fixed in the Datastax java driver update version(3.3.1)
> which is not used by Flink Cassandra connector (using 3.0.0).
> I upgraded the driver version to 3.3.1 locally in Flink Cassandra connector
> and tested, it stopped creating new prepared statements with different
> ordering of column for the same entity. I have the fix for this issue and
> would like to contribute the change and will raise the PR request for the
> same.
> Flink Cassandra Connector Version: flink-connector-cassandra_2.11
> Flink Version: 1.7.1
> I am creating PR request for the same and which can be merged accordingly and
> re released in new minor release or patch release as required.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
