Hello DataFu, The fix for DATAFU-167 (which allows us to support newer Spark 2.x versions) requires the use of a py4j API that isn't used in the py4j versions in older Spark versions. Therefore, we need to decide whether we're fine with dropping support for 2.2.0, 2.2.1, and 2.3.0.
In exchange, we'll be able to support 2.2.2 - 2.2.3, 2.3.1 - 2.3.4 and 2.4.0 - 2.4.5. I think this is fine, and the solution proposed in DATAFU-167 (by new contributor Arpit!) should be accepted, and we can put the fix in upcoming release 1.7.0. You can look at the proposed solution easily here: https://github.com/apache/datafu/compare/master...arpitbhardwaj:datafu:spark_upgrade_fix Does anyone have any reservations? If not, I'd like to release 1.7.0 soon. Cheers, Eyal
