[
https://issues.apache.org/jira/browse/SPARK-13027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaditya Ramesh updated SPARK-13027:
-----------------------------------
Description:
wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwThe StateDStream currently
does not provide the batch time as input to the state update function. This is
required in cases where the behavior depends on the batch start time.
We (Conviva) have been patching it manually for the past several Spark versions
but we thought it might be useful for others as well.
was:
The StateDStream currently does not provide the batch time as input to the
state update function. This is required in cases where the behavior depends on
the batch start time.
We (Conviva) have been patching it manually for the past several Spark versions
but we thought it might be useful for others as well.
> Add API for updateStateByKey to provide batch time as input
> -----------------------------------------------------------
>
> Key: SPARK-13027
> URL: https://issues.apache.org/jira/browse/SPARK-13027
> Project: Spark
> Issue Type: Improvement
> Components: Streaming
> Affects Versions: 1.6.0
> Reporter: Aaditya Ramesh
>
> wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwThe StateDStream currently
> does not provide the batch time as input to the state update function. This
> is required in cases where the behavior depends on the batch start time.
> We (Conviva) have been patching it manually for the past several Spark
> versions but we thought it might be useful for others as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
