[jira] [Commented] (SPARK-33507) Improve and fix cache behavior in v1 and v2
[ https://issues.apache.org/jira/browse/SPARK-33507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17268063#comment-17268063 ] Chao Sun commented on SPARK-33507: -- [~aokolnychyi] could you elaborate on the question? currently Spark doesn't support caching streaming tables yet. > Improve and fix cache behavior in v1 and v2 > --- > > Key: SPARK-33507 > URL: https://issues.apache.org/jira/browse/SPARK-33507 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.1.0 >Reporter: Chao Sun >Priority: Critical > > This is an umbrella JIRA to track fixes & improvements for caching behavior > in Spark datasource v1 and v2, which includes: > - fix existing cache behavior in v1 and v2. > - fix inconsistent cache behavior between v1 and v2 > - implement missing features in v2 to align with those in v1. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-33507) Improve and fix cache behavior in v1 and v2
[ https://issues.apache.org/jira/browse/SPARK-33507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17266479#comment-17266479 ] Anton Okolnychyi commented on SPARK-33507: -- [~csun], shall we also handle streaming writes? > Improve and fix cache behavior in v1 and v2 > --- > > Key: SPARK-33507 > URL: https://issues.apache.org/jira/browse/SPARK-33507 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.1.0 >Reporter: Chao Sun >Priority: Critical > > This is an umbrella JIRA to track fixes & improvements for caching behavior > in Spark datasource v1 and v2, which includes: > - fix existing cache behavior in v1 and v2. > - fix inconsistent cache behavior between v1 and v2 > - implement missing features in v2 to align with those in v1. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-33507) Improve and fix cache behavior in v1 and v2
[ https://issues.apache.org/jira/browse/SPARK-33507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17264608#comment-17264608 ] Chao Sun commented on SPARK-33507: -- Thanks [~hyukjin.kwon]. From my side, there is no regression. Although I feel SPARK-34052 is a bit important since it concerns correctness. I'n working on a fix but got delayed by a few other issues found during the process :( > Improve and fix cache behavior in v1 and v2 > --- > > Key: SPARK-33507 > URL: https://issues.apache.org/jira/browse/SPARK-33507 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.1.0 >Reporter: Chao Sun >Priority: Critical > > This is an umbrella JIRA to track fixes & improvements for caching behavior > in Spark datasource v1 and v2, which includes: > - fix existing cache behavior in v1 and v2. > - fix inconsistent cache behavior between v1 and v2 > - implement missing features in v2 to align with those in v1. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-33507) Improve and fix cache behavior in v1 and v2
[ https://issues.apache.org/jira/browse/SPARK-33507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17264601#comment-17264601 ] Hyukjin Kwon commented on SPARK-33507: -- Hey guys, I think we should just start RC regardless of these issues here. Looks like it's going to take too long and the release schedule will have to be delayed. These are non-regressions, right? Please directly ping me and let me know if there are regressions. > Improve and fix cache behavior in v1 and v2 > --- > > Key: SPARK-33507 > URL: https://issues.apache.org/jira/browse/SPARK-33507 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.1.0 >Reporter: Chao Sun >Priority: Critical > > This is an umbrella JIRA to track fixes & improvements for caching behavior > in Spark datasource v1 and v2, which includes: > - fix existing cache behavior in v1 and v2. > - fix inconsistent cache behavior between v1 and v2 > - implement missing features in v2 to align with those in v1. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-33507) Improve and fix cache behavior in v1 and v2
[ https://issues.apache.org/jira/browse/SPARK-33507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262836#comment-17262836 ] Dongjoon Hyun commented on SPARK-33507: --- Sure, no problem, [~csun]. > Improve and fix cache behavior in v1 and v2 > --- > > Key: SPARK-33507 > URL: https://issues.apache.org/jira/browse/SPARK-33507 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.1.0 >Reporter: Chao Sun >Assignee: Chao Sun >Priority: Critical > > This is an umbrella JIRA to track fixes & improvements for caching behavior > in Spark datasource v1 and v2, which includes: > - fix existing cache behavior in v1 and v2. > - fix inconsistent cache behavior between v1 and v2 > - implement missing features in v2 to align with those in v1. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-33507) Improve and fix cache behavior in v1 and v2
[ https://issues.apache.org/jira/browse/SPARK-33507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262826#comment-17262826 ] Chao Sun commented on SPARK-33507: -- [~dongjoon], [~hyukjin.kwon]: can either of you remove me as assignee from this JIRA since it has multiple contributors. Thanks. > Improve and fix cache behavior in v1 and v2 > --- > > Key: SPARK-33507 > URL: https://issues.apache.org/jira/browse/SPARK-33507 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.1.0 >Reporter: Chao Sun >Assignee: Chao Sun >Priority: Critical > > This is an umbrella JIRA to track fixes & improvements for caching behavior > in Spark datasource v1 and v2, which includes: > - fix existing cache behavior in v1 and v2. > - fix inconsistent cache behavior between v1 and v2 > - implement missing features in v2 to align with those in v1. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-33507) Improve and fix cache behavior in v1 and v2
[ https://issues.apache.org/jira/browse/SPARK-33507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17236521#comment-17236521 ] Chao Sun commented on SPARK-33507: -- [~dongjoon] Yes this is for Spark 3.1 _mostly_ (some JIRAs are in 2.4.x though such as SPARK-33290) > Improve and fix cache behavior in v1 and v2 > --- > > Key: SPARK-33507 > URL: https://issues.apache.org/jira/browse/SPARK-33507 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.1.0 >Reporter: Chao Sun >Assignee: Chao Sun >Priority: Major > > This is an umbrella JIRA to track fixes & improvements for caching behavior > in Spark datasource v1 and v2, which includes: > - fix some existing cache behavior in v1. > - fix inconsistent cache behaviors between v1 and v2 > - implement new features in v2 to align with those in v1. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-33507) Improve and fix cache behavior in v1 and v2
[ https://issues.apache.org/jira/browse/SPARK-33507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17236515#comment-17236515 ] Dongjoon Hyun commented on SPARK-33507: --- BTW, are you targeting this on Apache Spark 3.1? > Improve and fix cache behavior in v1 and v2 > --- > > Key: SPARK-33507 > URL: https://issues.apache.org/jira/browse/SPARK-33507 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.1.0 >Reporter: Chao Sun >Assignee: Chao Sun >Priority: Major > > This is an umbrella JIRA to track fixes & improvements for caching behavior > in Spark datasource v1 and v2, which includes: > - fix some existing cache behavior in v1. > - fix inconsistent cache behaviors between v1 and v2 > - implement new features in v2 to align with those in v1. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-33507) Improve and fix cache behavior in v1 and v2
[ https://issues.apache.org/jira/browse/SPARK-33507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17236512#comment-17236512 ] Dongjoon Hyun commented on SPARK-33507: --- Thank you for working on this area, [~csun]. This looks very important to me for Apache Iceberg use-cases, too. > Improve and fix cache behavior in v1 and v2 > --- > > Key: SPARK-33507 > URL: https://issues.apache.org/jira/browse/SPARK-33507 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.1.0 >Reporter: Chao Sun >Priority: Major > > This is an umbrella JIRA to track fixes & improvements for caching behavior > in Spark datasource v1 and v2, which includes: > - fix some existing cache behavior in v1. > - fix inconsistent cache behaviors between v1 and v2 > - implement new features in v2 to align with those in v1. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
