Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19443
Let's resolve it as `Later` for now. Will keep my eyes on similar JIRAs and
ping / cc you in the future. Thanks for bearing with me @jsnowacki.
---
Github user jsnowacki commented on the issue:
https://github.com/apache/spark/pull/19443
OK, closing than. Should I leave the JIRA issue or close it as well.
---
-
To unsubscribe, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19443
I could consider going ahead if the small fix makes all the things in
`functions.py` consistent, but I guess it is not. I think I am less sure
because, IIUC, we are not even clear on what to do
Github user jsnowacki commented on the issue:
https://github.com/apache/spark/pull/19443
This PR fixes only the functions created using `_create_function`, which to
what I found, were the only ones affected by the issue. Rest of the functions
either have different assumption or
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19443
Yea, I mean, I just wonder if this PR targets to fix all the same cases and
instances within this Python API, `functions.py`. Are these all cases that need
to support string case within this
Github user jsnowacki commented on the issue:
https://github.com/apache/spark/pull/19443
@HyukjinKwon It takes the argument the functions imported via
`_create_function` and tries to cast it to `Column` via `_to_java_column`
functions. Before it was passing the argument as is, and,
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19443
@jsnowacki, does the current fix make all other Python functions, except
non-applucable ones, take string parameters BTW? I could check it by myself but
I guess you took a look already.
---
Github user jsnowacki commented on the issue:
https://github.com/apache/spark/pull/19443
The only other reason for Python I can think of, if the above are not
compelling enough, is that the issue with function not having
call-by-column-name option is that we'll get the error only at
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19443
I think it's okay to wait for a few days more and for other committers who
might support or like this idea before closing this. I won't stay against.
Providing more compelling reasons
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19443
> I think the argument about consistency here is valid, though, I agree
with @jaceklaskowski that changes should go one way or the other, i.e. allow
string column names or remove this option
Github user jsnowacki commented on the issue:
https://github.com/apache/spark/pull/19443
@HyukjinKwon Thanks for pointing that out. I think the argument about
consistency here is valid, though, I agree with @jaceklaskowski that changes
should go one way or the other, i.e. allow
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19443
This might look okay within Python side because the fix looks minimised and
does not actually increase complexity much; however, I think we focus on API
consistency between other languages in
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19443
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19443
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82507/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19443
**[Test build #82507 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82507/testReport)**
for PR 19443 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19443
**[Test build #82507 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82507/testReport)**
for PR 19443 at commit
16 matches
Mail list logo
