[jira] [Commented] (SPARK-15689) Data source API v2

2019-10-09 Thread Rafael (Jira)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16947794#comment-16947794 ] Rafael commented on SPARK-15689: Could you clarify whether 4 item from the original list was

[jira] [Commented] (SPARK-15689) Data source API v2

2018-10-02 Thread Geoff Freeman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16636307#comment-16636307 ] Geoff Freeman commented on SPARK-15689: --- Thanks Wenchen. I was hoping that we might be able to

[jira] [Commented] (SPARK-15689) Data source API v2

2018-10-01 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16634871#comment-16634871 ] Wenchen Fan commented on SPARK-15689: - So {{SupportsReportPartitioning}} is not powerful enough to

[jira] [Commented] (SPARK-15689) Data source API v2

2018-10-01 Thread Geoff Freeman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16634650#comment-16634650 ] Geoff Freeman commented on SPARK-15689: --- I'm having trouble figuring out how to expose a custom

[jira] [Commented] (SPARK-15689) Data source API v2

2018-07-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549724#comment-16549724 ] Reynold Xin commented on SPARK-15689: - Is there an umbrella ticket on improving the dsv2 api? If

[jira] [Commented] (SPARK-15689) Data source API v2

2017-11-01 Thread Russell Spitzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234899#comment-16234899 ] Russell Spitzer commented on SPARK-15689: - I think knowing whether or not the count was occurring

[jira] [Commented] (SPARK-15689) Data source API v2

2017-11-01 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234892#comment-16234892 ] Wenchen Fan commented on SPARK-15689: - Spark wants to get `unhandledFilters` first so that it can

[jira] [Commented] (SPARK-15689) Data source API v2

2017-11-01 Thread Russell Spitzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234820#comment-16234820 ] Russell Spitzer commented on SPARK-15689: - It does not, we can tell that a count (or similar

[jira] [Commented] (SPARK-15689) Data source API v2

2017-11-01 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234811#comment-16234811 ] Wenchen Fan commented on SPARK-15689: - how would a count(agg function) exist in filter expression? >

[jira] [Commented] (SPARK-15689) Data source API v2

2017-11-01 Thread Russell Spitzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234640#comment-16234640 ] Russell Spitzer commented on SPARK-15689: - Something I just noticed, it may be helpful to also

[jira] [Commented] (SPARK-15689) Data source API v2

2017-10-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224224#comment-16224224 ] Wenchen Fan commented on SPARK-15689: - ah missed this one, reopen this ticket. My concern is that,

[jira] [Commented] (SPARK-15689) Data source API v2

2017-10-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224122#comment-16224122 ] Reynold Xin commented on SPARK-15689: - Why not put all of them as subtasks here? Also

[jira] [Commented] (SPARK-15689) Data source API v2

2017-10-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16223988#comment-16223988 ] Wenchen Fan commented on SPARK-15689: - The basic read/write interfaces are done, I'm resolving this

[jira] [Commented] (SPARK-15689) Data source API v2

2017-09-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153834#comment-16153834 ] Apache Spark commented on SPARK-15689: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-15689) Data source API v2

2017-09-05 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153709#comment-16153709 ] Yanbo Liang commented on SPARK-15689: - [~cloud_fan] Great doc! I like the direction this is going in,

[jira] [Commented] (SPARK-15689) Data source API v2

2017-08-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145309#comment-16145309 ] Reynold Xin commented on SPARK-15689: - That seems like an issue orthogonal to the API described here.

[jira] [Commented] (SPARK-15689) Data source API v2

2017-08-29 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145304#comment-16145304 ] Thomas Graves commented on SPARK-15689: --- I haven't had a chance to look at this, someone just

[jira] [Commented] (SPARK-15689) Data source API v2

2017-08-23 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139479#comment-16139479 ] Wenchen Fan commented on SPARK-15689: - yea, `LogicalPlan` is an internal concept and we can't use it

[jira] [Commented] (SPARK-15689) Data source API v2

2017-08-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138607#comment-16138607 ] Reynold Xin commented on SPARK-15689: - Not the author but my guess is that the other approach

[jira] [Commented] (SPARK-15689) Data source API v2

2017-08-23 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138509#comment-16138509 ] Andrew Ash commented on SPARK-15689: Can the authors of this document add a section contrasting the

[jira] [Commented] (SPARK-15689) Data source API v2

2017-08-23 Thread Liang Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137998#comment-16137998 ] Liang Chen commented on SPARK-15689: Cool! Very look forward to API V2 > Data source API v2 >

[jira] [Commented] (SPARK-15689) Data source API v2

2017-08-23 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137985#comment-16137985 ] Wenchen Fan commented on SPARK-15689: - yea, we plan to. > Data source API v2 > -- >

[jira] [Commented] (SPARK-15689) Data source API v2

2017-08-23 Thread Liang Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137982#comment-16137982 ] Liang Chen commented on SPARK-15689: Thanks for the doc. all these features will be considered in

[jira] [Commented] (SPARK-15689) Data source API v2

2017-08-17 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130863#comment-16130863 ] Wenchen Fan commented on SPARK-15689: - good doc attached! > Data source API v2 > --

[jira] [Commented] (SPARK-15689) Data source API v2

2017-08-17 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130494#comment-16130494 ] Dongjoon Hyun commented on SPARK-15689: --- Thank you for the document, too! > Data source API v2 >

[jira] [Commented] (SPARK-15689) Data source API v2

2017-08-17 Thread Russell Spitzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130355#comment-16130355 ] Russell Spitzer commented on SPARK-15689: - Thanks [~cloud_fan] for posting the design doc it was

[jira] [Commented] (SPARK-15689) Data source API v2

2017-06-16 Thread Russell Spitzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16052321#comment-16052321 ] Russell Spitzer commented on SPARK-15689: - I've been trying to work with making Catalyst