[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

2018-05-12 Thread mgaido91 
Github user mgaido91 commented on the issue:

https://github.com/apache/spark/pull/21309
  
@rxin I also always had some problems in understanding which functions are 
available and which not. I think it is quite confusing for a user. I am 
wondering: what do you think if we automatically generate this file in order to 
make all them available to users? In this way probably we don't have the huge 
maintenance effort exposing all of them to users.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

2018-05-11 Thread HyukjinKwon 
Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/21309
  
Merged to master.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

2018-05-11 Thread HyukjinKwon 
Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/21309
  
Just had a short talk with Reynold. LGTM too. thanks for bearing with me.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

2018-05-11 Thread rxin 
Github user rxin commented on the issue:

https://github.com/apache/spark/pull/21309
  
Better compile time error. Plus a lot of people are already using these.

On Fri, May 11, 2018 at 7:35 PM Hyukjin Kwon 
wrote:

> Yup, then why not just deprecate other functions in other APIs for 3.0.0,
> and promote the usage of expr?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> , or 
mute
> the thread
> 

> .
>



---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

2018-05-11 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21309
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90537/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

2018-05-11 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21309
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

2018-05-11 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/21309
  
**[Test build #90537 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90537/testReport)**
 for PR 21309 at commit 
[`ce2c305`](https://github.com/apache/spark/commit/ce2c305169d90c4d7803338d85d2d4c92a8e1d3c).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

2018-05-11 Thread HyukjinKwon 
Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/21309
  
Yup, then why not just deprecate other functions in other APIs for 3.0.0, 
and promote the usage of expr?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

2018-05-11 Thread rxin 
Github user rxin commented on the issue:

https://github.com/apache/spark/pull/21309
  
Adding it to sql would allow it to be available everywhere (through expr)
right?

On Fri, May 11, 2018 at 7:30 PM Hyukjin Kwon 
wrote:

> Thing is, I am a bit confused when to add it to other APIs. I thought if
> it's expected to be less commonly used, it shouldn't be added at the first
> place. We have UDFs.
>
> I have been a bit confused of some functions specifically not added into
> other APIs. If that's worth being added in an API, I thought it makes 
sense
> to add it to other APIs too. Is there a reason to add them to SQL side
> specifically?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> , or 
mute
> the thread
> 

> .
>



---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

2018-05-11 Thread HyukjinKwon 
Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/21309
  
I am asking this to use the same judgement for when to add it.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

2018-05-11 Thread HyukjinKwon 
Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/21309
  
Thing is, I am a bit confused when to add it to other APIs. I thought if 
it's expected to be less commonly used, it shouldn't be added at the first 
place. We have UDFs.

I have been a bit confused of some functions specifically not added into 
other APIs. If that's worth being added in an API, I thought it makes sense to 
add it to other APIs too. Is there a reason to add them to SQL side 
specifically?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

2018-05-11 Thread rxin 
Github user rxin commented on the issue:

https://github.com/apache/spark/pull/21309
  
Btw it’s been always the case that the less commonly used functions are 
not
part of this file. There is just a lot of overhead to maintaining all of
them.

I’m not even sure if the regr_* expressions should be added in the first
place.

On Fri, May 11, 2018 at 7:20 PM Hyukjin Kwon 
wrote:

> @rxin , how about splitting up this file by the
> group or something, or deprecating all the functions that can be called 
via
> expr for 3.0.0? To me, it looked a bit odd when some functions exist and
> some did not. It was an actual use case and I had to check which function
> exists or not every time.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> , or 
mute
> the thread
> 

> .
>



---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

2018-05-11 Thread HyukjinKwon 
Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/21309
  
@rxin, how about splitting up this file by the group or something, or 
deprecating all the functions that can be called via expr for 3.0.0? To me, it 
looked a bit odd when some functions exist and some did not. It was an actual 
use case and I had to check which function exists or not every time.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

2018-05-11 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21309
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 

https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3164/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

2018-05-11 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21309
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

2018-05-11 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/21309
  
**[Test build #90537 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90537/testReport)**
 for PR 21309 at commit 
[`ce2c305`](https://github.com/apache/spark/commit/ce2c305169d90c4d7803338d85d2d4c92a8e1d3c).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

2018-05-11 Thread rxin 
Github user rxin commented on the issue:

https://github.com/apache/spark/pull/21309
  
cc @gatorsmile @mgaido91 


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org