[ https://issues.apache.org/jira/browse/BEAM-15?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242606#comment-15242606 ]
Amit Sela commented on BEAM-15: ------------------------------- This will happen in native Spark (DStream) code as well, so not sure yet on how to deal with that. Again, let's see how Structured Streaming behaves. > Applying windowing to cached RDDs fails > --------------------------------------- > > Key: BEAM-15 > URL: https://issues.apache.org/jira/browse/BEAM-15 > Project: Beam > Issue Type: Bug > Components: runner-spark > Reporter: Amit Sela > Assignee: Amit Sela > > The Spark runner caches RDDs that are accessed more than once. If applying > window operations to a cached RDD, it will fail because windowed RDDs will > try to cache with a different cache level - windowing cache level is > StorageLevel.MEMORY_ONLY_SER and RDD cache level is StorageLevel.MEMORY_ONLY. -- This message was sent by Atlassian JIRA (v6.3.4#6332)