[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16185169#comment-16185169
]
Hyukjin Kwon commented on SPARK-21190:
--
Will keep in mind and suggest to fix it or f
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16185166#comment-16185166
]
Reynold Xin commented on SPARK-21190:
-
OK it would be great to have a better error me
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16185164#comment-16185164
]
Hyukjin Kwon commented on SPARK-21190:
--
[~rxin], I suggested to disallow it
[here|h
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16185155#comment-16185155
]
Reynold Xin commented on SPARK-21190:
-
Where did we settle on 0-arg UDFs? I think we
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16177886#comment-16177886
]
Reynold Xin commented on SPARK-21190:
-
Maybe create an umbrella ticket so it is easie
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16177885#comment-16177885
]
Wenchen Fan commented on SPARK-21190:
-
yea, let's do that in a separated ticket.
> S
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16177846#comment-16177846
]
Li Jin commented on SPARK-21190:
[~cloud_fan], do we want to track other vectorized udf e
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16162646#comment-16162646
]
Takuya Ueshin commented on SPARK-21190:
---
[~icexelloss] Thank you for your suggestio
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16162205#comment-16162205
]
Bryan Cutler commented on SPARK-21190:
--
Thanks [~icexelloss]. I definitely think co
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158845#comment-16158845
]
Li Jin commented on SPARK-21190:
To [~bryanc]'s point, PR [#18659|https://github.com/apac
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156082#comment-16156082
]
Bryan Cutler commented on SPARK-21190:
--
I attached my PR because it had already been
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156068#comment-16156068
]
Apache Spark commented on SPARK-21190:
--
User 'BryanCutler' has created a pull reques
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155271#comment-16155271
]
Apache Spark commented on SPARK-21190:
--
User 'ueshin' has created a pull request for
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155270#comment-16155270
]
Takuya Ueshin commented on SPARK-21190:
---
[~leif], [~bryanc] Thanks for the instruct
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16154361#comment-16154361
]
Bryan Cutler commented on SPARK-21190:
--
Thanks [~ueshin], I think having an optional
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16153986#comment-16153986
]
Leif Walsh commented on SPARK-21190:
I think the size parameter is confusing: if a 1-
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16153978#comment-16153978
]
Takuya Ueshin commented on SPARK-21190:
---
[~leif] Thank you for your proposal.
I'm s
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16153955#comment-16153955
]
Takuya Ueshin commented on SPARK-21190:
---
[~bryanc] Thank you for your suggestion. {
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151377#comment-16151377
]
Leif Walsh commented on SPARK-21190:
You can also make a Series with no content and a
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151376#comment-16151376
]
Leif Walsh commented on SPARK-21190:
Yep, that's totally a thing:
{noformat}In [1]:
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151261#comment-16151261
]
Leif Walsh commented on SPARK-21190:
I'm not 100% sure this is legal pandas but I thi
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150924#comment-16150924
]
Bryan Cutler commented on SPARK-21190:
--
I'm good with the API summary proposed by [~
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16143491#comment-16143491
]
Wenchen Fan commented on SPARK-21190:
-
hmmm, your proposal has a weird usage: users n
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16140094#comment-16140094
]
Li Jin commented on SPARK-21190:
[~ueshin], Got it. I'd actually prefer doing it this way
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16139507#comment-16139507
]
Takuya Ueshin commented on SPARK-21190:
---
[~icexelloss]
We can know the length of in
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138340#comment-16138340
]
Li Jin commented on SPARK-21190:
[~ueshin], thanks for the summary. +1 for this API.
Alt
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16137897#comment-16137897
]
Takuya Ueshin commented on SPARK-21190:
---
Hi all,
I'd like to summarize this discus
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16105095#comment-16105095
]
Li Jin commented on SPARK-21190:
I think the use case 2 of what [~rxin] proposed original
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16105072#comment-16105072
]
Li Jin commented on SPARK-21190:
[~cloud_fan], thanks for pointing out `ArrowColumnVector
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102420#comment-16102420
]
Bryan Cutler commented on SPARK-21190:
--
Hi [~icexelloss], yes I think there is defin
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101014#comment-16101014
]
Wenchen Fan commented on SPARK-21190:
-
I think (2) is already done by {{ArrowColumnVe
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16100712#comment-16100712
]
Li Jin commented on SPARK-21190:
[~bryanc],
I have looked at your PR at https://github.c
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16100582#comment-16100582
]
Li Jin commented on SPARK-21190:
I have created this PR for the groupby().apply() use cas
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16083018#comment-16083018
]
Bryan Cutler commented on SPARK-21190:
--
[~cloud_fan] yes, I know not every function
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16079954#comment-16079954
]
Wenchen Fan commented on SPARK-21190:
-
[~bryanc] You example works because Python is
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078837#comment-16078837
]
Bryan Cutler commented on SPARK-21190:
--
[~rxin] I was talking about 2 different thin
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16077484#comment-16077484
]
Reynold Xin commented on SPARK-21190:
-
[~bryanc] Sorry I don't think it makes sense t
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16077390#comment-16077390
]
Bryan Cutler commented on SPARK-21190:
--
This is a great discussion so far and I wou
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16074857#comment-16074857
]
Leif Walsh commented on SPARK-21190:
If the user specifies an int return type but pro
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16074848#comment-16074848
]
Li Jin commented on SPARK-21190:
> I have 2 thoughts:
> 1. How should we handle null valu
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072429#comment-16072429
]
Leif Walsh commented on SPARK-21190:
I believe we could also compute window indexes w
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072424#comment-16072424
]
Leif Walsh commented on SPARK-21190:
I figure we could address that by using shared m
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072019#comment-16072019
]
Wenchen Fan commented on SPARK-21190:
-
> I think we can get away with doing windowing
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16071002#comment-16071002
]
Wenchen Fan commented on SPARK-21190:
-
Thanks for your proposal!
I have 2 thoughts:
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16070809#comment-16070809
]
Leif Walsh commented on SPARK-21190:
I think we can get away with doing windowing (de
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16070618#comment-16070618
]
Li Jin commented on SPARK-21190:
I have some APIs design written down here:
Here is how t
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16069665#comment-16069665
]
Wenchen Fan commented on SPARK-21190:
-
For aggregate, I think it makes more sense to
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16069490#comment-16069490
]
Reynold Xin commented on SPARK-21190:
-
That makes a lot of sense. So to design APIs s
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16069269#comment-16069269
]
Leif Walsh commented on SPARK-21190:
I agree with [~icexelloss] that we should aim to
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063873#comment-16063873
]
Li Jin commented on SPARK-21190:
[~r...@databricks.com],
The use case of seeing entire p
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063515#comment-16063515
]
Reynold Xin commented on SPARK-21190:
-
[~icexelloss] Thanks. Your proposal brings up
[
https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063103#comment-16063103
]
Li Jin commented on SPARK-21190:
Very excited to see this.
I created https://issues.apa
52 matches
Mail list logo
