[ https://issues.apache.org/jira/browse/IGNITE-12054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937715#comment-16937715 ]
Aleksey Zinoviev edited comment on IGNITE-12054 at 9/25/19 1:18 PM: -------------------------------------------------------------------- I've started the R&D about 2.4 version, fixed a few minor changes and posted here a few thoughts about current problems here, in Jira comments. An ExternalCatalog was [refactored |[https://github.com/apache/spark/commit/f38ea00e83099a5ae8d3afdec2e896e43c2db612]]and all listener properties were inherited in ExternalCatalogWithListener. Nobody yet inherited from this class on Github, the known implementations are HiveExternalCatalog and MemoryExternalCatalog (both of them doesn't support listeners and events) Also, people in Spark ML couldn't solve the same problem [http://mail-archives.apache.org/mod_mbox/spark-issues/201812.mbox/%3cjira.13144856.1520975543000.147283.1544598241...@atlassian.jira%3E] was (Author: zaleslaw): I've started the R&D about 2.4 version, fixed a few minor changes and posted here a few thoughts about current problems here, in Jira comments. An ExternalCatalog was refactored and all listener properties were inherited in ExternalCatalogWithListener. Nobody yet inherited from this class on Github, the known implementations are HiveExternalCatalog and MemoryExternalCatalog (both of them doesn't support listeners and events) Also, people in Spark ML couldn't solve the same problem [http://mail-archives.apache.org/mod_mbox/spark-issues/201812.mbox/%3cjira.13144856.1520975543000.147283.1544598241...@atlassian.jira%3E] > Upgrade Spark module to 2.4 > --------------------------- > > Key: IGNITE-12054 > URL: https://issues.apache.org/jira/browse/IGNITE-12054 > Project: Ignite > Issue Type: Task > Components: spark > Affects Versions: 2.7.5 > Reporter: Denis Magda > Assignee: Nikolay Izhikov > Priority: Blocker > Fix For: 2.8 > > > Users can't use APIs that are already available in Spark 2.4: > https://stackoverflow.com/questions/57392143/persisting-spark-dataframe-to-ignite > Let's upgrade Spark from 2.3 to 2.4 until we extract the Spark Integration as > a separate module that can support multiple Spark versions. -- This message was sent by Atlassian Jira (v8.3.4#803005)