Chuck Connell created SPARK-37183: ------------------------------------- Summary: pyspark.pandas.DataFrame.map() should support .fillna() Key: SPARK-37183 URL: https://issues.apache.org/jira/browse/SPARK-37183 Project: Spark Issue Type: Improvement Components: PySpark Affects Versions: 3.2.0 Reporter: Chuck Connell
In regular pandas you can say {quote}DF["new_column"] = DF["column"].map(some_map).fillna(DF["column"]) {quote} In order to use the existing value if the mapping key is not found. But this does not work in pyspark.pandas. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org