[
https://issues.apache.org/jira/browse/BEAM-9451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136872#comment-17136872
]
Beam JIRA Bot commented on BEAM-9451:
-------------------------------------
This issue was marked "stale-P2" and has not received a public comment in 14
days. It is now automatically moved to P3. If you are still affected by it, you
can comment and move it back to P2.
> Optimize translation when Schema information is available in Spark Structured
> Streaming runner
> ----------------------------------------------------------------------------------------------
>
> Key: BEAM-9451
> URL: https://issues.apache.org/jira/browse/BEAM-9451
> Project: Beam
> Issue Type: Improvement
> Components: runner-spark
> Reporter: Ismaël Mejía
> Priority: P3
> Labels: structured-streaming
>
> Spark Structured Streaming runner supports Datasets that already have Schema
> information. This is used by Spark to optimize jobs (via Catalyst). This
> issue is to implement optimized translations of the transforms for the runner
> so we can benefit of the performance improvements internally done by Spark.
> Notice that we also may need to map Beam's core internal representations like
> WindowedValue so we can have intermediary optimizations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
