[jira] (SPARK-19383) Spark Sql Fails with Cassandra 3.6 and later PER PARTITION LIMIT option

[Herman van Hovell (JIRA)]

Title: Message Title

Herman van Hovell commented on SPARK-19383   Re: Spark Sql Fails with Cassandra 3.6 and later PER PARTITION LIMIT option The Cassandra connector currently uses the DataSources API. The API in its current form does support limit pushdown. This means that we cannot implement this at this time, without extending the DataSources API (which is quite an undertaking). We are going to release a new version of the DataSource API this year, and limit push down is on the requirements list for that one. Generally I am neutral on per partition pushdown. On the one hand it is quite a low level detail; Spark for instance plans a limit using local (partition level) and global limits. It is, on the other hand, sometimes useful to be able to control such a thing. Add Comment   This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d)