[ https://issues.apache.org/jira/browse/SPARK-35659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
L. C. Hsieh updated SPARK-35659: -------------------------------- Description: According to {{get}} method doc in StateStore API, it returns non-null row if the key exists. So basically we should avoid write null to StateStore. You cannot distinguish if the returned null row is because the key doesn't exist, or the value is actually null. And due to the defined behavior of {{get}}, it is quite easy to cause NPE error if the caller doesn't expect to get a null if the caller believes the key exists. (was: According to {{get}} metho doc in StateStore API, it returns non-null row if the key exists. So basically we should avoid write null to StateStore. You cannot distinguish if the returned null row is because the key doesn't exist, or the value is actually null. And due to the defined behavior of {{get}}, it is quite easy to cause NPE error if the caller doesn't expect to get a null if the caller believes the key exists.) > Avoid write null to StateStore > ------------------------------ > > Key: SPARK-35659 > URL: https://issues.apache.org/jira/browse/SPARK-35659 > Project: Spark > Issue Type: Bug > Components: Structured Streaming > Affects Versions: 3.0.2, 3.1.2, 3.2.0 > Reporter: L. C. Hsieh > Assignee: L. C. Hsieh > Priority: Major > > According to {{get}} method doc in StateStore API, it returns non-null row if > the key exists. So basically we should avoid write null to StateStore. You > cannot distinguish if the returned null row is because the key doesn't exist, > or the value is actually null. And due to the defined behavior of {{get}}, it > is quite easy to cause NPE error if the caller doesn't expect to get a null > if the caller believes the key exists. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org