[ https://issues.apache.org/jira/browse/SPARK-11812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tathagata Das updated SPARK-11812: ---------------------------------- Assignee: David Tolpin > pyspark reduceByKeyAndWindow does not handle unspecified invFunc > (invFunc=None) > ------------------------------------------------------------------------------- > > Key: SPARK-11812 > URL: https://issues.apache.org/jira/browse/SPARK-11812 > Project: Spark > Issue Type: Bug > Components: PySpark > Affects Versions: 1.5.2 > Reporter: David Tolpin > Assignee: David Tolpin > Fix For: 1.3.2, 1.4.2, 1.5.3, 1.6.0 > > > invFunc is optional and can be None. Instead of invFunc (the parameter) > invReduceFunc (a local function) was checked for trueness (that is, not None, > in this context). A local function is never None, > thus the case of invFunc=none (a common one when inverse reduction is not > defined) is treated incorrectly, resulting in loss of data. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org