[
https://issues.apache.org/jira/browse/SPARK-24396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16491415#comment-16491415
]
Tathagata Das edited comment on SPARK-24396 at 5/26/18 12:07 AM:
-----------------------------------------------------------------
TaskContext.getLocalProperty (SPARK-24397) in Python is needed for getting the
batchId/epochId that is passed on by StreamExecution as Spark job local
property.
was (Author: tdas):
TaskContext.getLocalProperty in Python is needed for getting the
batchId/epochId that is passed on by StreamExecution as Spark job local
property.
> Add Structured Streaming ForeachWriter for python
> -------------------------------------------------
>
> Key: SPARK-24396
> URL: https://issues.apache.org/jira/browse/SPARK-24396
> Project: Spark
> Issue Type: New Feature
> Components: Structured Streaming
> Affects Versions: 2.3.0
> Reporter: Tathagata Das
> Assignee: Tathagata Das
> Priority: Major
>
> Users should be able to write ForeachWriter code in python, that is, they
> should be able to use the partitionid and the version/batchId/epochId to
> conditionally process rows.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
