[GitHub] spark issue #14523: [SPARK-16936] [SQL] Case Sensitivity Support for Refresh...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14523
  
**[Test build #63319 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63319/consoleFull)**
 for PR 14523 at commit 
[`fb0dd0b`](https://github.com/apache/spark/commit/fb0dd0b03640c9456313d8b7a63203607940e683).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14523: [SPARK-16936] [SQL] Case Sensitivity Support for Refresh...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14523
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14523: [SPARK-16936] [SQL] Case Sensitivity Support for Refresh...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14523
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63319/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14175: [SPARK-16522][MESOS] Spark application throws exception ...

[sun-rui]

Github user sun-rui commented on the issue:

https://github.com/apache/spark/pull/14175
  
rebased to master


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14175: [SPARK-16522][MESOS] Spark application throws exception ...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14175
  
**[Test build #63321 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63321/consoleFull)**
 for PR 14175 at commit 
[`f848530`](https://github.com/apache/spark/commit/f848530cbc4fe5372e912a62d11e9cc8bf646522).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14477: [SPARK-16870][docs]Summary:add "spark.sql.broadcastTimeo...

[srowen]

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14477
  
Jenkins test this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14175: [SPARK-16522][MESOS] Spark application throws exception ...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14175
  
**[Test build #63321 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63321/consoleFull)**
 for PR 14175 at commit 
[`f848530`](https://github.com/apache/spark/commit/f848530cbc4fe5372e912a62d11e9cc8bf646522).
 * This patch **fails to build**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14175: [SPARK-16522][MESOS] Spark application throws exception ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14175
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63321/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14175: [SPARK-16522][MESOS] Spark application throws exception ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14175
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14477: [SPARK-16870][docs]Summary:add "spark.sql.broadcastTimeo...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14477
  
**[Test build #63322 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63322/consoleFull)**
 for PR 14477 at commit 
[`52911bf`](https://github.com/apache/spark/commit/52911bf0f288270c11ba7d59ff70c459f7eabb4e).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14514: document that Mesos cluster mode supports python

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/14514


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14514: document that Mesos cluster mode supports python

[srowen]

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14514
  
Merged to master/2.0


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14516: [SPARK-16932][DOCS] Changed programming guide to not ref...

[srowen]

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14516
  
Merged to master/2.0


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14516: [SPARK-16932][DOCS] Changed programming guide to ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/14516


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14522: [Spark-16508][SparkR] Split docs for arrange and ...

[felixcheung]

Github user felixcheung commented on a diff in the pull request:

https://github.com/apache/spark/pull/14522#discussion_r73797672
  
--- Diff: R/pkg/R/generics.R ---
@@ -551,7 +551,15 @@ setGeneric("merge")
 #' @export
 setGeneric("mutate", function(.data, ...) {standardGeneric("mutate") })
 
-#' @rdname arrange
+#' Ordering Columns in a WindowSpec
--- End diff --

please put this block in windowsspec.R


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14477: [SPARK-16870][docs]Summary:add "spark.sql.broadcastTimeo...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14477
  
**[Test build #63322 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63322/consoleFull)**
 for PR 14477 at commit 
[`52911bf`](https://github.com/apache/spark/commit/52911bf0f288270c11ba7d59ff70c459f7eabb4e).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14477: [SPARK-16870][docs]Summary:add "spark.sql.broadcastTimeo...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14477
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14477: [SPARK-16870][docs]Summary:add "spark.sql.broadcastTimeo...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14477
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63322/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14522: [Spark-16508][SparkR] Split docs for arrange and orderBy...

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/spark/pull/14522
  
Could you elaborate how this help with Spark-16508?
Generally we put functions with the same name even if they have different 
parameters in the same doc - but in this case it seems ok since their behavior 
are quite different.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14502: [SPARK-16909][Spark Core] - Streaming for postgre...

[princejwesley]

Github user princejwesley commented on a diff in the pull request:

https://github.com/apache/spark/pull/14502#discussion_r73797732
  
--- Diff: core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala ---
@@ -79,14 +79,19 @@ class JdbcRDD[T: ClassTag](
 val conn = getConnection()
 val stmt = conn.prepareStatement(sql, ResultSet.TYPE_FORWARD_ONLY, 
ResultSet.CONCUR_READ_ONLY)
 
-// setFetchSize(Integer.MIN_VALUE) is a mysql driver specific way to 
force streaming results,
-// rather than pulling entire resultset into memory.
-// see 
http://dev.mysql.com/doc/refman/5.0/en/connector-j-reference-implementation-notes.html
-if (conn.getMetaData.getURL.matches("jdbc:mysql:.*")) {
+val url = conn.getMetaData.getURL
+if (url.startsWith("jdbc:mysql:")) {
+  // setFetchSize(Integer.MIN_VALUE) is a mysql driver specific way to 
force streaming results,
--- End diff --

@srowen Updated.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14489: [MINOR][SparkR] R API documentation for "coltypes...

[felixcheung]

Github user felixcheung commented on a diff in the pull request:

https://github.com/apache/spark/pull/14489#discussion_r73797758
  
--- Diff: R/pkg/R/DataFrame.R ---
@@ -411,7 +411,7 @@ setMethod("coltypes",
 #'
 #' Set the column types of a SparkDataFrame.
 #'
-#' @param x A SparkDataFrame
+#' @param y A SparkDataFrame whose column types to be set
--- End diff --

is there a reason you change this from x to y?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14477: [SPARK-16870][docs]Summary:add "spark.sql.broadca...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/14477


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14503: [#SPARK-16911] Fix the links in the programming guide

[srowen]

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14503
  
Merged to master/2.0


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14477: [SPARK-16870][docs]Summary:add "spark.sql.broadcastTimeo...

[srowen]

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14477
  
Merged to master/2.0


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14503: [#SPARK-16911] Fix the links in the programming g...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/14503


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14502: [SPARK-16909][Spark Core] - Streaming for postgreSQL JDB...

[srowen]

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14502
  
Jenkins retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13932: [SPARK-15354] [CORE] [WIP] Topology aware block replicat...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13932
  
Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14502: [SPARK-16909][Spark Core] - Streaming for postgreSQL JDB...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14502
  
**[Test build #63323 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63323/consoleFull)**
 for PR 14502 at commit 
[`bc1b318`](https://github.com/apache/spark/commit/bc1b318c43bbdfee70f1b650861d187b93ac0107).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14454: [Minor] [ML] Rename TreeEnsembleModels to TreeEnsembleMo...

[yanboliang]

Github user yanboliang commented on the issue:

https://github.com/apache/spark/pull/14454
  
@srowen Would you mind to have a look when you are available? Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #11228: [SPARK-13356][Streaming]WebUI missing input informations...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/11228
  
**[Test build #63324 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63324/consoleFull)**
 for PR 11228 at commit 
[`f7c41ff`](https://github.com/apache/spark/commit/f7c41ff5b19117c91a8ab51ebad1fe470e00b645).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14454: [Minor] [ML] Rename TreeEnsembleModels to TreeEnsembleMo...

[srowen]

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14454
  
Doesn't this change an API though?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #11228: [SPARK-13356][Streaming]WebUI missing input informations...

[jeanlyn]

Github user jeanlyn commented on the issue:

https://github.com/apache/spark/pull/11228
  
@vanzin Sorry for the late reply, I had solved the conflicts.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14454: [Minor] [ML] Rename TreeEnsembleModels to TreeEnsembleMo...

[yanboliang]

Github user yanboliang commented on the issue:

https://github.com/apache/spark/pull/14454
  
In Scala, we define ```TreeEnsembleModel``` as private.
```
private[ml] trait TreeEnsembleModel[M <: DecisionTreeModel] { ... }
```
So I think Python users should not use ```TreeEnsembleModels``` as public 
API.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14454: [Minor] [ML] Rename TreeEnsembleModels to TreeEnsembleMo...

[srowen]

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14454
  
Hm, though it ends up being exposed in Python, no? that's what I'm 
concerned about. If it's just a rename, yeah I agree ideally we'd do it but am 
not clear it's worth the possible breakage. Unless someone does confirm that 
really nobody should be using this directly in Python.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14371: [SPARK-16736] Core+ SQL superfluous fs calls

[steveloughran]

Github user steveloughran commented on the issue:

https://github.com/apache/spark/pull/14371
  
Pulled the WiP; happy for full reviews —though I'm on vacation right now, 
so can't handle feedback just yet


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14454: [Minor] [ML] Rename TreeEnsembleModels to TreeEnsembleMo...

[yanboliang]

Github user yanboliang commented on the issue:

https://github.com/apache/spark/pull/14454
  
It should not exposed in Python. But unfortunately we did not make it 
private by naming it with underscore prefix which is the official Python 
convention for 'internal' symbols. I think if there are users using this 
directly in Python, we can regard it as hitting bug.
Further more, I think we should rename it to ```_ TreeEnsembleModel``` to 
make it really private in Python. I'm open to hear your thoughts.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14454: [Minor] [ML] Rename TreeEnsembleModels to TreeEnsembleMo...

[srowen]

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14454
  
@jkbradley @davies what do you think of that?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14502: [SPARK-16909][Spark Core] - Streaming for postgreSQL JDB...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14502
  
**[Test build #63323 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63323/consoleFull)**
 for PR 14502 at commit 
[`bc1b318`](https://github.com/apache/spark/commit/bc1b318c43bbdfee70f1b650861d187b93ac0107).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14502: [SPARK-16909][Spark Core] - Streaming for postgreSQL JDB...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14502
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63323/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14502: [SPARK-16909][Spark Core] - Streaming for postgreSQL JDB...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14502
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #11228: [SPARK-13356][Streaming]WebUI missing input informations...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/11228
  
**[Test build #63324 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63324/consoleFull)**
 for PR 11228 at commit 
[`f7c41ff`](https://github.com/apache/spark/commit/f7c41ff5b19117c91a8ab51ebad1fe470e00b645).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14524: [SPARK-16832] [ML] [WIP] CrossValidator and Train...

[srowen]

GitHub user srowen opened a pull request:

https://github.com/apache/spark/pull/14524

[SPARK-16832] [ML] [WIP] CrossValidator and TrainValidationSplit are not 
random without seed

## What changes were proposed in this pull request?

Default random seed for ML classes should be random not fixed to class 
`hashCode`

## How was this patch tested?

Existing Jenkins tests

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/srowen/spark SPARK-16832

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/14524.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #14524


commit fbc93eb3241dbbbc7f7525a803444e1ded390978
Author: Sean Owen 
Date:   2016-08-07T11:14:57Z

Default random seed for ML classes should be random not fixed to class 
hashCode




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14524: [SPARK-16832] [ML] [WIP] CrossValidator and TrainValidat...

[srowen]

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14524
  
Want to check with @jkbradley on this before proceeding, but seems to pass 
tests


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #11228: [SPARK-13356][Streaming]WebUI missing input informations...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/11228
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14524: [SPARK-16832] [ML] [WIP] CrossValidator and TrainValidat...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14524
  
**[Test build #63325 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63325/consoleFull)**
 for PR 14524 at commit 
[`fbc93eb`](https://github.com/apache/spark/commit/fbc93eb3241dbbbc7f7525a803444e1ded390978).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #11228: [SPARK-13356][Streaming]WebUI missing input informations...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/11228
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63324/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14502: [SPARK-16909][Spark Core] - Streaming for postgreSQL JDB...

[srowen]

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14502
  
Merged to master


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14504: [SPARK-16409] [SQL] regexp_extract with optional groups ...

[srowen]

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14504
  
Merged to master/2.0/1.6


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14504: [SPARK-16409] [SQL] regexp_extract with optional ...

[srowen]

Github user srowen closed the pull request at:

https://github.com/apache/spark/pull/14504


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14502: [SPARK-16909][Spark Core] - Streaming for postgre...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/14502


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14500: [SPARK-16905] SQL DDL: MSCK REPAIR TABLE

[hvanhovell]

Github user hvanhovell commented on a diff in the pull request:

https://github.com/apache/spark/pull/14500#discussion_r73800495
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala ---
@@ -425,6 +430,110 @@ case class AlterTableDropPartitionCommand(
 
 }
 
+/**
+ * Recover Partitions in ALTER TABLE: recover all the partition in the 
directory of a table and
+ * update the catalog.
+ *
+ * The syntax of this command is:
+ * {{{
+ *   ALTER TABLE table RECOVER PARTITIONS;
+ *   MSCK REPAIR TABLE table;
+ * }}}
+ */
+case class AlterTableRecoverPartitionsCommand(
+tableName: TableIdentifier,
+cmd: String = "ALTER TABLE RECOVER PARTITIONS") extends 
RunnableCommand {
+  override def run(spark: SparkSession): Seq[Row] = {
+val catalog = spark.sessionState.catalog
+if (!catalog.tableExists(tableName)) {
+  throw new AnalysisException(s"Table $tableName in $cmd does not 
exist.")
+}
+val table = catalog.getTableMetadata(tableName)
+if (catalog.isTemporaryTable(tableName)) {
+  throw new AnalysisException(
+s"Operation not allowed: $cmd on temporary tables: $tableName")
+}
+if (DDLUtils.isDatasourceTable(table)) {
+  throw new AnalysisException(
+s"Operation not allowed: $cmd on datasource tables: $tableName")
+}
+if (table.tableType != CatalogTableType.EXTERNAL) {
+  throw new AnalysisException(
+s"Operation not allowed: $cmd only works on external tables: 
$tableName")
+}
+if (!DDLUtils.isTablePartitioned(table)) {
+  throw new AnalysisException(
+s"Operation not allowed: $cmd only works on partitioned tables: 
$tableName")
+}
+if (table.storage.locationUri.isEmpty) {
+  throw new AnalysisException(
+s"Operation not allowed: $cmd only works on table with location 
provided: $tableName")
+}
+
+val root = new Path(table.storage.locationUri.get)
+val fs = root.getFileSystem(spark.sparkContext.hadoopConfiguration)
+// Dummy jobconf to get to the pathFilter defined in configuration
+// It's very expensive to create a 
JobConf(ClassUtil.findContainingJar() is slow)
+val jobConf = new JobConf(spark.sparkContext.hadoopConfiguration, 
this.getClass)
+val pathFilter = FileInputFormat.getInputPathFilter(jobConf)
+val partitionSpecsAndLocs = scanPartitions(
+  spark, fs, pathFilter, root, Map(), 
table.partitionColumnNames.map(_.toLowerCase))
+val parts = partitionSpecsAndLocs.map { case (spec, location) =>
+  // inherit table storage format (possibly except for location)
+  CatalogTablePartition(spec, table.storage.copy(locationUri = 
Some(location.toUri.toString)))
+}
+spark.sessionState.catalog.createPartitions(tableName,
+  parts.toArray[CatalogTablePartition], ignoreIfExists = true)
+Seq.empty[Row]
+  }
+
+  @transient private lazy val evalTaskSupport = new 
ForkJoinTaskSupport(new ForkJoinPool(8))
+
+  private def scanPartitions(
+  spark: SparkSession,
+  fs: FileSystem,
+  filter: PathFilter,
+  path: Path,
+  spec: TablePartitionSpec,
+  partitionNames: Seq[String]): GenSeq[(TablePartitionSpec, Path)] = {
+if (partitionNames.length == 0) {
+  return Seq(spec -> path)
+}
+
+val statuses = fs.listStatus(path)
+val threshold = spark.conf.get("spark.rdd.parallelListingThreshold", 
"10").toInt
+val statusPar: GenSeq[FileStatus] =
+  if (partitionNames.length > 1 && statuses.length > threshold || 
partitionNames.length > 2) {
+val parArray = statuses.par
--- End diff --

A new one is created here: 
https://github.com/apache/spark/pull/14500/files#diff-54979ed5797b4a6193cf663dc23baca5R490


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14525: [SPARK-16324] [SQL] regexp_extract should doc tha...

[srowen]

GitHub user srowen opened a pull request:

https://github.com/apache/spark/pull/14525

[SPARK-16324] [SQL] regexp_extract should doc that it returns empty string 
when match fails

## What changes were proposed in this pull request?

Doc that regexp_extract returns empty string when regex or group does not 
match

## How was this patch tested?

Jenkins test, with a few new test cases

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/srowen/spark SPARK-16324

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/14525.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #14525


commit 98036a1e105ffa133e35f6122aacf1679e1d04d3
Author: Sean Owen 
Date:   2016-08-07T11:52:24Z

Doc that regexp_extract returns empty string when regex or group does not 
match




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14525: [SPARK-16324] [SQL] regexp_extract should doc that it re...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14525
  
**[Test build #63326 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63326/consoleFull)**
 for PR 14525 at commit 
[`98036a1`](https://github.com/apache/spark/commit/98036a1e105ffa133e35f6122aacf1679e1d04d3).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14524: [SPARK-16832] [ML] [WIP] CrossValidator and TrainValidat...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14524
  
**[Test build #63325 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63325/consoleFull)**
 for PR 14524 at commit 
[`fbc93eb`](https://github.com/apache/spark/commit/fbc93eb3241dbbbc7f7525a803444e1ded390978).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14524: [SPARK-16832] [ML] [WIP] CrossValidator and TrainValidat...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14524
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14524: [SPARK-16832] [ML] [WIP] CrossValidator and TrainValidat...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14524
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63325/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14218: [SPARK-16563][SQL] fix spark sql thrift server FetchResu...

[alicegugu]

Github user alicegugu commented on the issue:

https://github.com/apache/spark/pull/14218
  
@ericl Hi Eric, two test cases are added for the code change. Please 
review. Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14175: [SPARK-16522][MESOS] Spark application throws exception ...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14175
  
**[Test build #63327 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63327/consoleFull)**
 for PR 14175 at commit 
[`3526f53`](https://github.com/apache/spark/commit/3526f53ad762479b966f69a3c6444cdb1165f1a8).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14525: [SPARK-16324] [SQL] regexp_extract should doc that it re...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14525
  
**[Test build #63326 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63326/consoleFull)**
 for PR 14525 at commit 
[`98036a1`](https://github.com/apache/spark/commit/98036a1e105ffa133e35f6122aacf1679e1d04d3).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14525: [SPARK-16324] [SQL] regexp_extract should doc that it re...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14525
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63326/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14525: [SPARK-16324] [SQL] regexp_extract should doc that it re...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14525
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14422: Add rand(numRows: Int, numCols: Int) functions

[xubo245]

Github user xubo245 commented on the issue:

https://github.com/apache/spark/pull/14422
  
@srowen sorry, please close the issue. I will learning more  before next 
PR. The PR is only because breeze have the function. In spark ,there is no use 
for them.
Could you tell me some issue for starter? Please.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14175: [SPARK-16522][MESOS] Spark application throws exception ...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14175
  
**[Test build #63327 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63327/consoleFull)**
 for PR 14175 at commit 
[`3526f53`](https://github.com/apache/spark/commit/3526f53ad762479b966f69a3c6444cdb1165f1a8).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14175: [SPARK-16522][MESOS] Spark application throws exception ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14175
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14175: [SPARK-16522][MESOS] Spark application throws exception ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14175
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63327/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14526: [SPARK-16939][SQL] Fix build error by using `Tupl...

[dongjoon-hyun]

GitHub user dongjoon-hyun opened a pull request:

https://github.com/apache/spark/pull/14526

[SPARK-16939][SQL] Fix build error by using `Tuple1` explicitly in 
StringFunctionsSuite

## What changes were proposed in this pull request?

This PR aims to fix a build error on branch 1.6 at 
https://github.com/apache/spark/commit/8d8725208771a8815a60160a5a30dc6ea87a7e6a,
 but I think we had better have this consistently in master branch, too. It's 
because there exist other ongoing PR 
(https://github.com/apache/spark/pull/14525) about this.


https://amplab.cs.berkeley.edu/jenkins/job/spark-branch-1.6-compile-maven-with-yarn-2.3/286/console

```scala
[error] 
/home/jenkins/workspace/spark-branch-1.6-compile-maven-with-yarn-2.3/sql/core/src/test/scala/org/apache/spark/sql/StringFunctionsSuite.scala:82:
 value toDF is not a member of Seq[String]
[error] val df = Seq("c").toDF("s")
[error]   ^
```

## How was this patch tested?

After passing Jenkins, run compilation test on branch 1.6.
```
build/mvn -DskipTests -Pyarn -Phadoop-2.3 -Pkinesis-asl -Phive 
-Phive-thriftserver install
```

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dongjoon-hyun/spark SPARK-16939

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/14526.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #14526


commit 1256d67a77301783512f53db6e1ed2dc48da98b5
Author: Dongjoon Hyun 
Date:   2016-08-07T15:51:23Z

[SPARK-16939][SQL] Fix build error by using `Tuple1` explicitly in 
StringFunctionsSuite




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14526: [SPARK-16939][SQL] Fix build error by using `Tuple1` exp...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14526
  
**[Test build #63328 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63328/consoleFull)**
 for PR 14526 at commit 
[`1256d67`](https://github.com/apache/spark/commit/1256d67a77301783512f53db6e1ed2dc48da98b5).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14526: [SPARK-16939][SQL] Fix build error by using `Tuple1` exp...

[srowen]

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14526
  
Ah, shoot, that would be my fault. I am not sure why it would only fail for 
1.6? let me also look into it. Thanks for the catch.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14526: [SPARK-16939][SQL] Fix build error by using `Tuple1` exp...

[dongjoon-hyun]

Github user dongjoon-hyun commented on the issue:

https://github.com/apache/spark/pull/14526
  
Oh, thank you, @srowen !


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14526: [SPARK-16939][SQL] Fix build error by using `Tuple1` exp...

[dongjoon-hyun]

Github user dongjoon-hyun commented on the issue:

https://github.com/apache/spark/pull/14526
  
Until today, I also guessed that master branch does not handle this.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #12872: [SPARK-6339][SQL] Supports CREATE TEMPORARY VIEW tableId...

[gatorsmile]

Github user gatorsmile commented on the issue:

https://github.com/apache/spark/pull/12872
  
Currently, the existing DDL behaviors for temporary views when users do not 
specify the database name:

- `CREATE OR REPLACE TEMPORARY VIEW view_name` alters the TEMPORARY view if 
the temporary view already exists. 

- `DROP VIEW view_name` OR `SELECT... FROM view_name` will be always first 
applied to a TEMPORARY view, if existing. If the temporary view does not exist, 
we will try to drop/fetch the PERSISTENT view, if existing.

- `ALTER VIEW view_name` is only applicable to the PERSISTENT view, even if 
the temporary view with the same name exists. 

@clockfly @rxin @yhuai @liancheng @hvanhovell @cloud-fan  Should we make 
them consistent? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14526: [SPARK-16939][SQL] Fix build error by using `Tupl...

[srowen]

Github user srowen commented on a diff in the pull request:

https://github.com/apache/spark/pull/14526#discussion_r73805425
  
--- Diff: 
sql/core/src/test/scala/org/apache/spark/sql/StringFunctionsSuite.scala ---
@@ -95,7 +95,7 @@ class StringFunctionsSuite extends QueryTest with 
SharedSQLContext {
   }
 
   test("non-matching optional group") {
-val df = Seq("c").toDF("s")
+val df = Seq(Tuple1("c")).toDF("s")
--- End diff --

OK, I guess that Scala 2.11 has an implicit conversion that Scala 2.10 does 
not that would make this conversion automatically, hence the PR builder 
succeeded. We could also just add a dummy column in the example, but I suppose 
this is the best way to ensure that the fix works in 1.6, and also makes the 
test a little more robust. LGTM, thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14411: [SPARK-16804][SQL] Correlated subqueries containing non-...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14411
  
**[Test build #3206 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3206/consoleFull)**
 for PR 14411 at commit 
[`ac43ab4`](https://github.com/apache/spark/commit/ac43ab47907a1ccd6d22f920415fbb4de93d4720).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14411: [SPARK-16804][SQL] Correlated subqueries containing non-...

[hvanhovell]

Github user hvanhovell commented on the issue:

https://github.com/apache/spark/pull/14411
  
@nsyca I have triggered a manual build. I'll merge as soon as it completes 
successfully.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14411: [SPARK-16804][SQL] Correlated subqueries containing non-...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14411
  
**[Test build #3206 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3206/consoleFull)**
 for PR 14411 at commit 
[`ac43ab4`](https://github.com/apache/spark/commit/ac43ab47907a1ccd6d22f920415fbb4de93d4720).
 * This patch **fails Scala style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #12872: [SPARK-6339][SQL] Supports CREATE TEMPORARY VIEW tableId...

[yhuai]

Github user yhuai commented on the issue:

https://github.com/apache/spark/pull/12872
  
Can you be more specific on the inconsistency? Seems `ALTER VIEW view_name` 
is the only inconsistent command?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14522: [Spark-16508][SparkR] Split docs for arrange and orderBy...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14522
  
**[Test build #63329 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63329/consoleFull)**
 for PR 14522 at commit 
[`c2445f3`](https://github.com/apache/spark/commit/c2445f3c6829d9372569ee93b3cf6bf9eaa3da3d).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

[hvanhovell]

Github user hvanhovell commented on the issue:

https://github.com/apache/spark/pull/1
  
First (hopefully dumb) question: why is this needed?

Maybe I am missing something: Shouldn't we just ban CreateStruct (and its 
unsafe brother)? Why keep these around if we are going to rewrite them anyway?

As for visitTable. You could also write this into a `Union` of a number of 
`Project(..., OneRowRelation)` statements, if that makes life easier.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14489: [MINOR][SparkR] R API documentation for "coltypes...

[keypointt]

Github user keypointt commented on a diff in the pull request:

https://github.com/apache/spark/pull/14489#discussion_r73805651
  
--- Diff: R/pkg/R/DataFrame.R ---
@@ -411,7 +411,7 @@ setMethod("coltypes",
 #'
 #' Set the column types of a SparkDataFrame.
 #'
-#' @param x A SparkDataFrame
+#' @param y A SparkDataFrame whose column types to be set
--- End diff --

to avoid duplicate variables, now it has 2 'x' in 'ARGUMENTS' section, or 
is there a better way to do it?

http://spark.apache.org/docs/2.0.0/api/R/coltypes.html


https://cloud.githubusercontent.com/assets/3925641/17386808/effb98ce-59a2-11e6-9657-d477d258a80c.png


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14526: [SPARK-16939][SQL] Fix build error by using `Tupl...

[dongjoon-hyun]

Github user dongjoon-hyun commented on a diff in the pull request:

https://github.com/apache/spark/pull/14526#discussion_r73805655
  
--- Diff: 
sql/core/src/test/scala/org/apache/spark/sql/StringFunctionsSuite.scala ---
@@ -95,7 +95,7 @@ class StringFunctionsSuite extends QueryTest with 
SharedSQLContext {
   }
 
   test("non-matching optional group") {
-val df = Seq("c").toDF("s")
+val df = Seq(Tuple1("c")).toDF("s")
--- End diff --

Thank you, @srowen .


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14522: [Spark-16508][SparkR] Split docs for arrange and orderBy...

[junyangq]

Github user junyangq commented on the issue:

https://github.com/apache/spark/pull/14522
  
Hi @felixcheung Thanks for reviewing this PR. It happened when I was trying 
to fix one of the CRAN warnings (Duplicated \argument entries in documentation 
object 'arrange': ‘x’) and found it confusing to have these two functions 
in the same doc. After the split, this warning got fixed as well.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14526: [SPARK-16939][SQL] Fix build error by using `Tuple1` exp...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14526
  
**[Test build #63328 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63328/consoleFull)**
 for PR 14526 at commit 
[`1256d67`](https://github.com/apache/spark/commit/1256d67a77301783512f53db6e1ed2dc48da98b5).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

[eyalfa]

Github user eyalfa commented on the issue:

https://github.com/apache/spark/pull/1
  
I think we need createStruct first of all for backward compatibility and
second because some constructs (such as row creator) maps directly into its
structure.
I think that the way ast-builder handles the inline table is wrong as it
mixes parsing and validation in a single pass, I think it could produce
some unresolved structure (UnresolvedInlineTable?) Which can be later
resolved by the analyzer,this would also produce better and clearer
validation errors. I.e. by looking at antlr grammar,it seems that
row-constructor isn't really limited to constant expressions, I haven't
tried but I guess it's possible to come up with an input that crashes it in
a bad way.

Regarding the expressionEncoder,the more I look at its usage and the more I
consider the fact that Dataset's suit is passing,the more I believe I
should revert my changes to expressionEncider and actually fix it's test
code (pass the tree to the analyzer prior to comparing/evaluating it).

On Aug 7, 2016 8:14 PM, "Herman van Hovell" 
wrote:

> First (hopefully dumb) question: why is this needed?
>
> Maybe I am missing something: Shouldn't we just ban CreateStruct (and its
> unsafe brother)? Why keep these around if we are going to rewrite them
> anyway?
>
> As for visitTable. You could also write this into a Union of a number of 
Project(...,
> OneRowRelation) statements, if that makes life easier.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> , or 
mute
> the thread
> 

> .
>



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14526: [SPARK-16939][SQL] Fix build error by using `Tuple1` exp...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14526
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63328/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14526: [SPARK-16939][SQL] Fix build error by using `Tuple1` exp...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14526
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #12872: [SPARK-6339][SQL] Supports CREATE TEMPORARY VIEW tableId...

[gatorsmile]

Github user gatorsmile commented on the issue:

https://github.com/apache/spark/pull/12872
  
Yeah, `ALTER VIEW` should be consistent with `DROP VIEW`, if we use the 
same naming rule. Should I submit a PR for it?

Another potential issue to users is the behaviors of `CRAETE VIEW` and 
`CREATE TEMPORARY VIEW` when users do not specify the database names. See the 
following example:

```
sql(s"CREATE TEMPORARY VIEW $viewName AS SELECT * FROM $tabName WHERE ID < 
3")
sql(s"CREATE VIEW $viewName AS SELECT * FROM $tabName") 
```
When we processing the second statement, we simply add `CURRENT_DATABASE` 
to make it a fully qualified view name. However, if users do not specify the 
fully qualified name in the subsequent SELECT/DROP, the persistent view is 
shadowed by the temporary view with the same name. The returned results might 
be a surprise to the Spark users, because they might not realize there exist a 
temporary view in the existing session.

Of course, the existing behavior is right, but I think the better way is to 
force users to specify the database name when creating a persistent view if 
there exists a temporary view with the same name. That means, we can issue an 
error message here in this specific case. 



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14504: [SPARK-16409] [SQL] regexp_extract with optional groups ...

[ericl]

Github user ericl commented on the issue:

https://github.com/apache/spark/pull/14504
  
I think this broke the build: ```[error] 
/home/jenkins/workspace/spark-branch-1.6-compile-maven-with-yarn-2.4/sql/core/src/test/scala/org/apache/spark/sql/StringFunctionsSuite.scala:82:
 value toDF is not a member of Seq[String] [error] val df = 
Seq("c").toDF("s") [error] ^ [warn] two warnings found [error] one error 
found```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14522: [Spark-16508][SparkR] Split docs for arrange and orderBy...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14522
  
**[Test build #63329 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63329/consoleFull)**
 for PR 14522 at commit 
[`c2445f3`](https://github.com/apache/spark/commit/c2445f3c6829d9372569ee93b3cf6bf9eaa3da3d).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14522: [Spark-16508][SparkR] Split docs for arrange and orderBy...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14522
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63329/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14522: [Spark-16508][SparkR] Split docs for arrange and orderBy...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14522
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14218: [SPARK-16563][SQL] fix spark sql thrift server Fe...

[ericl]

Github user ericl commented on a diff in the pull request:

https://github.com/apache/spark/pull/14218#discussion_r73807474
  
--- Diff: 
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala
 ---
@@ -91,6 +93,57 @@ class HiveThriftBinaryServerSuite extends 
HiveThriftJdbcTest {
 }
   }
 
+  test("SPARK-16563 ThriftCLIService FetchResults repeat fetching result") 
{
+withCLIServiceClient { client =>
+  val user = System.getProperty("user.name")
+  val sessionHandle = client.openSession(user, "")
+
+  withJdbcStatement { statement =>
+val queries = Seq(
+  "SET spark.sql.shuffle.partitions=3",
+  "DROP TABLE IF EXISTS test",
+  "CREATE TABLE test_16563(key INT, val STRING)",
+  s"LOAD DATA LOCAL INPATH '${TestData.smallKv}' OVERWRITE INTO 
TABLE test_16563",
+  "CACHE TABLE test_16563")
+
+queries.foreach(statement.execute)
+val confOverlay = new java.util.HashMap[java.lang.String, 
java.lang.String]
+val operationHandle = client.executeStatement(
+  sessionHandle,
+  "SELECT * FROM test_16563",
+  confOverlay)
+
+assertResult(0, "Repeat fetching result from next row") {
+
+  // Fetch first time
+  val rows = client.fetchResults(operationHandle)
--- End diff --

Can you assert this also returns 5?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14218: [SPARK-16563][SQL] fix spark sql thrift server Fe...

[ericl]

Github user ericl commented on a diff in the pull request:

https://github.com/apache/spark/pull/14218#discussion_r73807486
  
--- Diff: 
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala
 ---
@@ -91,6 +93,57 @@ class HiveThriftBinaryServerSuite extends 
HiveThriftJdbcTest {
 }
   }
 
+  test("SPARK-16563 ThriftCLIService FetchResults repeat fetching result") 
{
+withCLIServiceClient { client =>
+  val user = System.getProperty("user.name")
+  val sessionHandle = client.openSession(user, "")
+
+  withJdbcStatement { statement =>
+val queries = Seq(
+  "SET spark.sql.shuffle.partitions=3",
+  "DROP TABLE IF EXISTS test",
--- End diff --

Dropping the wrong table?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14218: [SPARK-16563][SQL] fix spark sql thrift server Fe...

[ericl]

Github user ericl commented on a diff in the pull request:

https://github.com/apache/spark/pull/14218#discussion_r73807494
  
--- Diff: 
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala
 ---
@@ -91,6 +93,57 @@ class HiveThriftBinaryServerSuite extends 
HiveThriftJdbcTest {
 }
   }
 
+  test("SPARK-16563 ThriftCLIService FetchResults repeat fetching result") 
{
+withCLIServiceClient { client =>
+  val user = System.getProperty("user.name")
+  val sessionHandle = client.openSession(user, "")
+
+  withJdbcStatement { statement =>
+val queries = Seq(
+  "SET spark.sql.shuffle.partitions=3",
--- End diff --

Is this needed? Same comment with cache table below.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14218: [SPARK-16563][SQL] fix spark sql thrift server FetchResu...

[ericl]

Github user ericl commented on the issue:

https://github.com/apache/spark/pull/14218
  
Lgtm, just some comments on the tests


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #13818: [SPARK-15968][SQL] Nonempty partitioned metastore...

[erfangc]

Github user erfangc commented on a diff in the pull request:

https://github.com/apache/spark/pull/13818#discussion_r73807925
  
--- Diff: 
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala
 ---
@@ -298,6 +298,7 @@ case class InsertIntoHiveTable(
 
 // Invalidate the cache.
 sqlContext.sharedState.cacheManager.invalidateCache(table)
+
sqlContext.sessionState.catalog.invalidateTable(table.catalogTable.identifier)
--- End diff --

Hi @mallman does this change cause new sessions (ex: external App to the 
ThriftServer via JDBC) to not see the cached tables? I noticed this in the 
released version 2.0.0 whereby `CACHE TABLE` in one session has no effect on 
new sessions. Future SQL statements are still reading the underlying Parquet 
files from Disk (as evidenced by tasks being `NODE_LOCAL` and `RACK_LOCAL` 
instead of `PROCESS_LOCAL`). Sorry if this question is unrelated to your patch, 
but this became a major issue in 2.0.0 for us, where as in 1.6.2 we do not have 
an issue.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14504: [SPARK-16409] [SQL] regexp_extract with optional groups ...

[srowen]

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14504
  
Yes, for 1.6: https://github.com/apache/spark/pull/14526


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14526: [SPARK-16939][SQL] Fix build error by using `Tupl...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/14526


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14526: [SPARK-16939][SQL] Fix build error by using `Tuple1` exp...

[srowen]

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14526
  
Merged to master/2.0/1.6


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14526: [SPARK-16939][SQL] Fix build error by using `Tuple1` exp...

[dongjoon-hyun]

Github user dongjoon-hyun commented on the issue:

https://github.com/apache/spark/pull/14526
  
Thank you for merging, @srowen !


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14411: [SPARK-16804][SQL] Correlated subqueries containing non-...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14411
  
**[Test build #3207 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3207/consoleFull)**
 for PR 14411 at commit 
[`631d396`](https://github.com/apache/spark/commit/631d396031e8bf627eb1f4872a4d3a17c144536c).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org