Github user petermaxlee commented on the issue:
https://github.com/apache/spark/pull/13969
I am going to close this one since #14138 has been merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user petermaxlee commented on the issue:
https://github.com/apache/spark/pull/13969
I submitted a new pull request https://github.com/apache/spark/pull/14138
and in that version only static methods are supported.
---
If your project is set up for it, you can reply to this
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/13969
You can also remove half of the test cases.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/13969
One thing ... it might be better to remove the ability to call non-static
methods. At least to me it'd make the things slightly simpler and more clear.
---
If your project is set up for it, you can
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/13969
what's hive's behaviour if calling a non-static method but the class
doesn't have no-arg constructor? null or exception?
---
If your project is set up for it, you can reply to this email and
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13969
**[Test build #3174 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3174/consoleFull)**
for PR 13969 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13969
**[Test build #3174 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3174/consoleFull)**
for PR 13969 at commit
Github user petermaxlee commented on the issue:
https://github.com/apache/spark/pull/13969
I've pushed a new commit that addresses the review comments.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13969
If you mean the following mention, what @rxin said was not about the return
type. It's about `function` and `method` name, isn't it? Is there any other
mentions about `String`?
> Majority
Github user petermaxlee commented on the issue:
https://github.com/apache/spark/pull/13969
@hvanhovell in its current form we'd need some refactoring to StaticInvoke
to work with this, due to Hive allowing both static invocation and dynamic
invocation.
Also - does
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/13969
@petermaxlee are you sure that we shouldn't implement this using
`RuntimeReplaceable`, using NewInstance -> Invoke or a StaticInvoke? The added
value is that this support code generation, and
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13969
Oh, sorry, @petermaxlee .
It seems we are calling `Utils.classForName(className)` three places.
Could we do that once? I'll leave a comment about this in code, too.
---
If your
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13969
Hi, @petermaxlee .
Sorry for late response.
LGTM except three minor style changes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user petermaxlee commented on the issue:
https://github.com/apache/spark/pull/13969
Ping!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
Github user petermaxlee commented on the issue:
https://github.com/apache/spark/pull/13969
Alright - I've updated it. The expression is now called Reflect, and it
prints different messages depending on whether it is a class not found or a
method not found.
---
If your project is
Github user petermaxlee commented on the issue:
https://github.com/apache/spark/pull/13969
JavaReflectMethod isn't the best, but calling it Reflect is pretty bad
because there are many "Reflect" classes in various libraries -- just within
Spark dependencies there are 3 classes called
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13969
Oh, if then, no problem. :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/13969
@dongjoon-hyun it's ok to not support the hive case here. Majority of the
use cases will be calling some literal functions anyway.
---
If your project is set up for it, you can reply to this email
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13969
I've done my first pass.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13969
In general, the noticeable big difference from Hive seems to be the
limitation on `first` and `second` arguments. Hive supports columns for them,
too.
IMHO, we can achieve that by putting
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13969
By the way, could we change the expression name?
I know the reason why you choose `JavaMethodReflect`, but we can call Scala
functions, too.
```
select
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13969
Oh, sure. @cloud-fan .
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/13969
cc @dongjoon-hyun to take a look
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user petermaxlee commented on the issue:
https://github.com/apache/spark/pull/13969
I brought this up to date. Any comments on the pull request?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13969
**[Test build #3152 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3152/consoleFull)**
for PR 13969 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13969
**[Test build #3152 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3152/consoleFull)**
for PR 13969 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13969
**[Test build #3146 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3146/consoleFull)**
for PR 13969 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13969
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13969
**[Test build #3146 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3146/consoleFull)**
for PR 13969 at commit
Github user petermaxlee commented on the issue:
https://github.com/apache/spark/pull/13969
cc @rxin @cloud-fan @dongjoon-hyun
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
30 matches
Mail list logo
