[GitHub] spark issue #20133: [SPARK-22934] [SQL] Make optional clauses order insensit...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20133
  
**[Test build #85576 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85576/testReport)**
 for PR 20133 at commit 
[`8ae8f18`](https://github.com/apache/spark/commit/8ae8f1832a62caf10a62511f339402c0d94f89ea).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20133: [SPARK-22934] [SQL] Make optional clauses order i...

[gatorsmile]

GitHub user gatorsmile opened a pull request:

https://github.com/apache/spark/pull/20133

[SPARK-22934] [SQL] Make optional clauses order insensitive for CREATE 
TABLE SQL statement

## What changes were proposed in this pull request?
Currently, our CREATE TABLE syntax require the EXACT order of clauses. It 
is pretty hard to remember the exact order. Thus, this PR is to make optional 
clauses order insensitive for `CREATE TABLE` SQL statement.

```
CREATE [TEMPORARY] TABLE [IF NOT EXISTS] [db_name.]table_name
[(col_name1 col_type1 [COMMENT col_comment1], ...)]
USING datasource
[OPTIONS (key1=val1, key2=val2, ...)]
[PARTITIONED BY (col_name1, col_name2, ...)]
[CLUSTERED BY (col_name3, col_name4, ...) INTO num_buckets BUCKETS]
[LOCATION path]
[COMMENT table_comment]
[TBLPROPERTIES (key1=val1, key2=val2, ...)]
[AS select_statement]
```

The proposal is to make the following clauses order insensitive. 
```
[OPTIONS (key1=val1, key2=val2, ...)]
[PARTITIONED BY (col_name1, col_name2, ...)]
[CLUSTERED BY (col_name3, col_name4, ...) INTO num_buckets BUCKETS]
[LOCATION path]
[COMMENT table_comment]
[TBLPROPERTIES (key1=val1, key2=val2, ...)]
```

The same idea is also applicable to Create Hive Table. 
```
CREATE [EXTERNAL] TABLE [IF NOT EXISTS] [db_name.]table_name
[(col_name1[:] col_type1 [COMMENT col_comment1], ...)]
[COMMENT table_comment]
[PARTITIONED BY (col_name2[:] col_type2 [COMMENT col_comment2], ...)]
[ROW FORMAT row_format]
[STORED AS file_format]
[LOCATION path]
[TBLPROPERTIES (key1=val1, key2=val2, ...)]
[AS select_statement]
```

The proposal is to make the following clauses order insensitive. 
```
[COMMENT table_comment]
[PARTITIONED BY (col_name2[:] col_type2 [COMMENT col_comment2], ...)]
[ROW FORMAT row_format]
[STORED AS file_format]
[LOCATION path]
[TBLPROPERTIES (key1=val1, key2=val2, ...)]
```

## How was this patch tested?
Added test cases

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gatorsmile/spark createDataSourceTableDDL

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/20133.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #20133


commit 8ae8f1832a62caf10a62511f339402c0d94f89ea
Author: gatorsmile 
Date:   2018-01-01T07:03:01Z

fix




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20106: [SPARK-21616][SPARKR][DOCS] update R migration guide and...

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/spark/pull/20106
  
Maybe. Other test files have that extra empty line though ;)






---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20127
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20127
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85570/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20127
  
**[Test build #85570 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85570/testReport)**
 for PR 20127 at commit 
[`f158a95`](https://github.com/apache/spark/commit/f158a951b779e56e06d2c73234bac5c79055b2f5).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20095: [SPARK-22126][ML] Added fitMultiple method with default ...

[MrBago]

Github user MrBago commented on the issue:

https://github.com/apache/spark/pull/20095
  
@jkbradley I pushed changes in response to your comments. I think we should 
split the `TrainValidationSplit` memory split into another PR, I may have time 
to work on it tomorrow.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20095: [SPARK-22126][ML] Added fitMultiple method with default ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20095
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20095: [SPARK-22126][ML] Added fitMultiple method with default ...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20095
  
**[Test build #85572 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85572/testReport)**
 for PR 20095 at commit 
[`8c7c8e3`](https://github.com/apache/spark/commit/8c7c8e3e14c6c2d7f64c5a3f54c129e794176672).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20095: [SPARK-22126][ML] Added fitMultiple method with default ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20095
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85572/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18277: [SPARK-20947][PYTHON] Fix encoding/decoding error in pip...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18277
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85574/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18277: [SPARK-20947][PYTHON] Fix encoding/decoding error in pip...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18277
  
**[Test build #85574 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85574/testReport)**
 for PR 18277 at commit 
[`8c88595`](https://github.com/apache/spark/commit/8c88595125fbd328a3ed2383a9e96db7ad96f0e9).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18277: [SPARK-20947][PYTHON] Fix encoding/decoding error in pip...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18277
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/19789
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/19789
  
**[Test build #85575 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85575/testReport)**
 for PR 19789 at commit 
[`7700bf0`](https://github.com/apache/spark/commit/7700bf02c06d6fe3909053a1e146569a67b21de3).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/19789
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85575/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20106: [SPARK-21616][SPARKR][DOCS] update R migration gu...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/20106#discussion_r159149323
  
--- Diff: R/pkg/tests/fulltests/test_Windows.R ---
@@ -14,6 +14,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
+
--- End diff --

We could remove this back tho :).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20131: [MINOR] Fix a bunch of typos

[HyukjinKwon]

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/20131
  
Merged to master


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/19789
  
**[Test build #85575 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85575/testReport)**
 for PR 19789 at commit 
[`7700bf0`](https://github.com/apache/spark/commit/7700bf02c06d6fe3909053a1e146569a67b21de3).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20130: [BUILD] Close stale PRs

[HyukjinKwon]

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/20130
  
I saw this one 19520. Might be good to add too.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...

[HyukjinKwon]

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/19789
  
ok to test


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19693: [MINOR][CORE] Improved statistical shuffle write time

[HyukjinKwon]

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/19693
  
Shall we leave this closed for now @heary-cao?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18424
  
**[Test build #85573 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85573/testReport)**
 for PR 18424 at commit 
[`62f273b`](https://github.com/apache/spark/commit/62f273b0b0e7fec6b80d95c39f49eba32de92c43).
 * This patch **fails Scala style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18424
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85573/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18424
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18424
  
**[Test build #85573 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85573/testReport)**
 for PR 18424 at commit 
[`62f273b`](https://github.com/apache/spark/commit/62f273b0b0e7fec6b80d95c39f49eba32de92c43).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18277: [SPARK-20947][PYTHON] Fix encoding/decoding error in pip...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18277
  
**[Test build #85574 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85574/testReport)**
 for PR 18277 at commit 
[`8c88595`](https://github.com/apache/spark/commit/8c88595125fbd328a3ed2383a9e96db7ad96f0e9).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18277: [SPARK-20947][PYTHON] Fix encoding/decoding error in pip...

[HyukjinKwon]

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/18277
  
ok to test


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...

[HyukjinKwon]

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/18424
  
ok to test


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20095: [SPARK-22126][ML] Added fitMultiple method with default ...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20095
  
**[Test build #85572 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85572/testReport)**
 for PR 20095 at commit 
[`8c7c8e3`](https://github.com/apache/spark/commit/8c7c8e3e14c6c2d7f64c5a3f54c129e794176672).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20131: [MINOR] Fix a bunch of typos

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20131
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85568/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20131: [MINOR] Fix a bunch of typos

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20131
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20131: [MINOR] Fix a bunch of typos

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20131
  
**[Test build #85568 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85568/testReport)**
 for PR 20131 at commit 
[`8eea4de`](https://github.com/apache/spark/commit/8eea4de9c005f52f4e83eaec647cd7e67b44dc2d).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20095: [SPARK-22126][ML] Added fitMultiple method with default ...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20095
  
**[Test build #85571 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85571/testReport)**
 for PR 20095 at commit 
[`4d10c19`](https://github.com/apache/spark/commit/4d10c19e004e6f7b265f7028accbda6d83af8ae4).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20127
  
**[Test build #85570 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85570/testReport)**
 for PR 20127 at commit 
[`f158a95`](https://github.com/apache/spark/commit/f158a951b779e56e06d2c73234bac5c79055b2f5).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext

[kiszk]

Github user kiszk commented on the issue:

https://github.com/apache/spark/pull/20127
  
retest this please


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20129
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85564/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20129
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20129
  
**[Test build #85564 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85564/testReport)**
 for PR 20129 at commit 
[`2d26bd8`](https://github.com/apache/spark/commit/2d26bd84d75cb3e3d7159ae192b014313a0f6fc8).
 * This patch **fails SparkR unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20132: [SPARK-13030][ML] Follow-up cleanups for OneHotEncoderEs...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/20132
  
@jkbradley Thanks for this follow-up!

I've noticed that first issue in original PR. But don't have enough time to 
discuss with you further.

I'll go through this soon.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20132: [SPARK-13030][ML] Follow-up cleanups for OneHotEncoderEs...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20132
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85569/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20132: [SPARK-13030][ML] Follow-up cleanups for OneHotEncoderEs...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20132
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20129
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20129
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85566/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20129
  
**[Test build #85566 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85566/testReport)**
 for PR 20129 at commit 
[`73e0def`](https://github.com/apache/spark/commit/73e0defb92d9d1f7fc8f7bb1f2bf5b637ac83113).
 * This patch **fails SparkR unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20130: [BUILD] Close stale PRs

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20130
  
**[Test build #85565 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85565/testReport)**
 for PR 20130 at commit 
[`d710a54`](https://github.com/apache/spark/commit/d710a543f9c1fdc362b3e4c7adde1b2a8debabf8).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20130: [BUILD] Close stale PRs

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20130
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20130: [BUILD] Close stale PRs

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20130
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85565/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20132: [SPARK-13030][ML] Follow-up cleanups for OneHotEncoderEs...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20132
  
**[Test build #85569 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85569/testReport)**
 for PR 20132 at commit 
[`9bf045d`](https://github.com/apache/spark/commit/9bf045da1adeaa08deeb96eaa0289d8d4cb74bc1).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20132: [SPARK-13030][ML] Follow-up cleanups for OneHotEncoderEs...

[jkbradley]

Github user jkbradley commented on the issue:

https://github.com/apache/spark/pull/20132
  
@viirya This basically has 2 changes:
* configedCategorySize: my mistake!
* encoder: clarify what I meant before


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

[jkbradley]

Github user jkbradley commented on the issue:

https://github.com/apache/spark/pull/19527
  
Thank you for all the work in this PR!  Here's the follow-up: 
https://github.com/apache/spark/pull/20132


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20132: [SPARK-13030][ML] Follow-up cleanups for OneHotEn...

[jkbradley]

GitHub user jkbradley opened a pull request:

https://github.com/apache/spark/pull/20132

[SPARK-13030][ML] Follow-up cleanups for OneHotEncoderEstimator

## What changes were proposed in this pull request?

Follow-up cleanups for the OneHotEncoderEstimator PR.  See some discussion 
in the original PR: https://github.com/apache/spark/pull/19527 or read below 
for what this PR includes:
* configedCategorySize: I reverted this to return an Array.  I realized the 
original setup (which I had recommended in the original PR) caused the whole 
model to be serialized in the UDF.
* encoder: I reorganized the logic to show what I meant in the comment in 
the previous PR.  I think it's simpler but am open to suggestions.

I also made some small style cleanups based on IntelliJ warnings.

## How was this patch tested?

Existing unit tests

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jkbradley/spark viirya-SPARK-13030

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/20132.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #20132


commit 9bf045da1adeaa08deeb96eaa0289d8d4cb74bc1
Author: Joseph K. Bradley 
Date:   2017-12-31T23:25:45Z

updates for final PR




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/19527
  
@jkbradley Thanks for reviewing and merging this. Thanks for all helping 
this too.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator f...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/19527


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

[jkbradley]

Github user jkbradley commented on the issue:

https://github.com/apache/spark/pull/19527
  
Thanks for the updates!  I still think there's some confusion, but since I 
think this code is correct & it doesn't affect APIs, I'll go ahead and merge 
this.  I'll ping you on a follow-up PR to show what I had in mind.

LGTM
Merging with master


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20131: [MINOR] Fix a bunch of typos

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20131
  
**[Test build #85568 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85568/testReport)**
 for PR 20131 at commit 
[`8eea4de`](https://github.com/apache/spark/commit/8eea4de9c005f52f4e83eaec647cd7e67b44dc2d).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20131: [MINOR] Fix a bunch of typos

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20131
  
**[Test build #85567 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85567/testReport)**
 for PR 20131 at commit 
[`a762e19`](https://github.com/apache/spark/commit/a762e19411cd6632e3f683ebaa4dcba3c3fe5bdf).
 * This patch **fails Scala style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20131: [MINOR] Fix a bunch of typos

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20131
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85567/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20131: [MINOR] Fix a bunch of typos

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20131
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20131: [MINOR] Fix a bunch of typos

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20131
  
**[Test build #85567 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85567/testReport)**
 for PR 20131 at commit 
[`a762e19`](https://github.com/apache/spark/commit/a762e19411cd6632e3f683ebaa4dcba3c3fe5bdf).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20131: [MINOR] Fix a bunch of typos

[srowen]

GitHub user srowen opened a pull request:

https://github.com/apache/spark/pull/20131

[MINOR] Fix a bunch of typos

## What changes were proposed in this pull request?

Fix a bunch of typos, to head off more typo PRs.

## How was this patch tested?

Existing tests

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/srowen/spark Spelling

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/20131.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #20131


commit a762e19411cd6632e3f683ebaa4dcba3c3fe5bdf
Author: Sean Owen 
Date:   2017-12-31T23:00:41Z

Fix a bunch of typos




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20129
  
**[Test build #85566 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85566/testReport)**
 for PR 20129 at commit 
[`73e0def`](https://github.com/apache/spark/commit/73e0defb92d9d1f7fc8f7bb1f2bf5b637ac83113).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20129
  
**[Test build #85563 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85563/testReport)**
 for PR 20129 at commit 
[`3843395`](https://github.com/apache/spark/commit/3843395391281f74278346bd518e543a4b86a66f).
 * This patch **fails SparkR unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20129
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85563/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20130: [BUILD] Close stale PRs

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20130
  
**[Test build #85565 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85565/testReport)**
 for PR 20130 at commit 
[`d710a54`](https://github.com/apache/spark/commit/d710a543f9c1fdc362b3e4c7adde1b2a8debabf8).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20129
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20130: [BUILD] Close stale PRs

[srowen]

GitHub user srowen opened a pull request:

https://github.com/apache/spark/pull/20130

[BUILD] Close stale PRs

Closes #18916
Closes #19613
Closes #19739
Closes #19936
Closes #19919
Closes #19933
Closes #19917
Closes #20027
Closes #19035
Closes #20044
Closes #20104

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/srowen/spark StalePRs

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/20130.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #20130


commit d710a543f9c1fdc362b3e4c7adde1b2a8debabf8
Author: Sean Owen 
Date:   2017-12-31T21:45:22Z

Close stale PRs.

Closes #18916
Closes #19613
Closes #19739
Closes #19936
Closes #19919
Closes #19933
Closes #19917
Closes #20027
Closes #19035
Closes #20044
Closes #20104




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19968: [SPARK-22769][CORE] When driver stopping, there is error...

[srowen]

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/19968
  
Ping @KaiXinXiaoLei to update or close


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20108: [MINOR][DOCS] s/It take/It takes/g

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/20108


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20108: [MINOR][DOCS] s/It take/It takes/g

[srowen]

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/20108
  
This time I'll just merge it.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20129
  
**[Test build #85564 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85564/testReport)**
 for PR 20129 at commit 
[`2d26bd8`](https://github.com/apache/spark/commit/2d26bd84d75cb3e3d7159ae192b014313a0f6fc8).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20129
  
**[Test build #85563 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85563/testReport)**
 for PR 20129 at commit 
[`3843395`](https://github.com/apache/spark/commit/3843395391281f74278346bd518e543a4b86a66f).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20129: [SPARK-22933][SPARKR] R Structured Streaming API ...

[felixcheung]

GitHub user felixcheung opened a pull request:

https://github.com/apache/spark/pull/20129

[SPARK-22933][SPARKR] R Structured Streaming API for withWatermark, 
trigger, partitionBy

## What changes were proposed in this pull request?

R Structured Streaming API for withWatermark, trigger, partitionBy

## How was this patch tested?

manual, unit tests

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/felixcheung/spark rwater

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/20129.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #20129


commit 9c980e3987c44c6df87e4d39524e4689babcabda
Author: Felix Cheung 
Date:   2017-08-21T00:09:57Z

withWatermark

commit e440706bfeb819faa1579f27fdf93d740aba0e3e
Author: Felix Cheung 
Date:   2017-12-31T20:48:21Z

trigger, patitionBy, tests

commit 3843395391281f74278346bd518e543a4b86a66f
Author: Felix Cheung 
Date:   2017-12-31T20:57:17Z

more test




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19715: [SPARK-22397][ML]add multiple columns support to Quantil...

[huaxingao]

Github user huaxingao commented on the issue:

https://github.com/apache/spark/pull/19715
  
Thank you all for your help!!


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20127
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85562/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20127
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20127
  
**[Test build #85562 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85562/testReport)**
 for PR 20127 at commit 
[`f158a95`](https://github.com/apache/spark/commit/f158a951b779e56e06d2c73234bac5c79055b2f5).
 * This patch **fails PySpark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20128: [SPARK-21893][SPARK-22142][TESTS][FOLLOWUP] Enables PySp...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20128
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85561/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20128: [SPARK-21893][SPARK-22142][TESTS][FOLLOWUP] Enables PySp...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20128
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20128: [SPARK-21893][SPARK-22142][TESTS][FOLLOWUP] Enables PySp...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20128
  
**[Test build #85561 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85561/testReport)**
 for PR 20128 at commit 
[`3b1533a`](https://github.com/apache/spark/commit/3b1533a400ecf277cf0e18b8cb76a14305b0).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20108: [MINOR][DOCS] s/It take/It takes/g

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20108
  
**[Test build #4032 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4032/testReport)**
 for PR 20108 at commit 
[`daae351`](https://github.com/apache/spark/commit/daae3511d04f5dab1a32790f09b05c9188c4ac5b).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20108: [MINOR][DOCS] s/It take/It takes/g

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20108
  
**[Test build #4032 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4032/testReport)**
 for PR 20108 at commit 
[`daae351`](https://github.com/apache/spark/commit/daae3511d04f5dab1a32790f09b05c9188c4ac5b).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20127
  
**[Test build #85562 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85562/testReport)**
 for PR 20127 at commit 
[`f158a95`](https://github.com/apache/spark/commit/f158a951b779e56e06d2c73234bac5c79055b2f5).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation

[srowen]

Github user srowen commented on a diff in the pull request:

https://github.com/apache/spark/pull/20070#discussion_r159143833
  
--- Diff: 
examples/src/main/scala/org/apache/spark/examples/ml/QuantileDiscretizerExample.scala
 ---
@@ -45,7 +45,7 @@ object QuantileDiscretizerExample {
   .setNumBuckets(3)
 
 val result = discretizer.fit(df).transform(df)
-result.show()
+result.show(false)
--- End diff --

Which other examples? most do not set this, and the Java equivalent doesn't 
either. If there's a good reason that the output needs to be untruncated, 
that's fine, just also change the Java example.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation

[srowen]

Github user srowen commented on a diff in the pull request:

https://github.com/apache/spark/pull/20070#discussion_r159143849
  
--- Diff: 
examples/src/main/scala/org/apache/spark/examples/mllib/LatentDirichletAllocationExample.scala
 ---
@@ -46,7 +46,10 @@ object LatentDirichletAllocationExample {
 val topics = ldaModel.topicsMatrix
 for (topic <- Range(0, 3)) {
   print(s"Topic $topic :")
-  for (word <- Range(0, ldaModel.vocabSize)) { print(s" ${topics(word, 
topic)}") }
+  for (word <- Range(0, ldaModel.vocabSize))
+  {
--- End diff --

No, we put the open brace on the preceding line with for, and don't triple 
indent. See any other for loop in the code.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext

[kiszk]

Github user kiszk commented on the issue:

https://github.com/apache/spark/pull/20127
  
retest this please


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20126: [DO-NOT-MERGE] Investigate if changes in flume.py actual...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20126
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85558/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20126: [DO-NOT-MERGE] Investigate if changes in flume.py actual...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20126
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20126: [DO-NOT-MERGE] Investigate if changes in flume.py actual...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20126
  
**[Test build #85558 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85558/testReport)**
 for PR 20126 at commit 
[`85639dd`](https://github.com/apache/spark/commit/85639dd220e8fcb0489febc0414b51d22c0e41a9).
 * This patch **fails PySpark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20128: [SPARK-21893][SPARK-22142][TESTS][FOLLOWUP] Enables PySp...

[srowen]

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/20128
  
Ah I probably missed that when moving these behind a profile. I wasn't 
fully aware of the dependency mechanism. If it resolves the problem, yes go for 
it.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20076: [SPARK-21786][SQL] When acquiring 'compressionCod...

[jaceklaskowski]

Github user jaceklaskowski commented on a diff in the pull request:

https://github.com/apache/spark/pull/20076#discussion_r159142765
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -323,11 +323,13 @@ object SQLConf {
 .createWithDefault(false)
 
   val PARQUET_COMPRESSION = 
buildConf("spark.sql.parquet.compression.codec")
-.doc("Sets the compression codec use when writing Parquet files. 
Acceptable values include: " +
-  "uncompressed, snappy, gzip, lzo.")
+.doc("Sets the compression codec use when writing Parquet files. If 
other compression codec " +
--- End diff --

s/use when/used when


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20076: [SPARK-21786][SQL] When acquiring 'compressionCod...

[jaceklaskowski]

Github user jaceklaskowski commented on a diff in the pull request:

https://github.com/apache/spark/pull/20076#discussion_r159142783
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -364,7 +366,9 @@ object SQLConf {
   .createWithDefault(true)
 
   val ORC_COMPRESSION = buildConf("spark.sql.orc.compression.codec")
-.doc("Sets the compression codec use when writing ORC files. 
Acceptable values include: " +
+.doc("Sets the compression codec use when writing ORC files. If other 
compression codec " +
--- End diff --

s/use when/used when


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20076: [SPARK-21786][SQL] When acquiring 'compressionCod...

[jaceklaskowski]

Github user jaceklaskowski commented on a diff in the pull request:

https://github.com/apache/spark/pull/20076#discussion_r159142760
  
--- Diff: docs/sql-programming-guide.md ---
@@ -953,8 +953,10 @@ Configuration of Parquet can be done using the 
`setConf` method on `SparkSession
   spark.sql.parquet.compression.codec
   snappy
   
-Sets the compression codec use when writing Parquet files. Acceptable 
values include:
-uncompressed, snappy, gzip, lzo.
+Sets the compression codec use when writing Parquet files. If other 
compression codec
--- End diff --

s/use when/used when


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20127
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85557/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/20127
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20128: [SPARK-21893][SPARK-22142][TESTS][FOLLOWUP] Enables PySp...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20128
  
**[Test build #85561 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85561/testReport)**
 for PR 20128 at commit 
[`3b1533a`](https://github.com/apache/spark/commit/3b1533a400ecf277cf0e18b8cb76a14305b0).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/20127
  
**[Test build #85557 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85557/testReport)**
 for PR 20127 at commit 
[`f158a95`](https://github.com/apache/spark/commit/f158a951b779e56e06d2c73234bac5c79055b2f5).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20128: [SPARK-21893][SPARK-22142][BUILD][FOLLOWUP] Enables PySp...

[HyukjinKwon]

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/20128
  
cc @srowen, @vanzin, @yhuai and @zsxwing who I believe are primarily 
related with this change.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20128: [SPARK-21893][SPARK-22142][BUILD][FOLLOWUP] Enabl...

[HyukjinKwon]

GitHub user HyukjinKwon opened a pull request:

https://github.com/apache/spark/pull/20128

[SPARK-21893][SPARK-22142][BUILD][FOLLOWUP] Enables PySpark tests for Flume 
and Kafka in Jenkins

## What changes were proposed in this pull request?

This PR proposes to enable PySpark tests for Flume and Kafka in Jenkins by 
explicitly setting the environment variables in `modules.py`.

Seems we are not taking the dependencies into account when calculating 
environment variables:


https://github.com/apache/spark/blob/3a07eff5af601511e97a05e6fea0e3d48f74c4f0/dev/run-tests.py#L554-L561


## How was this patch tested?

Manual tests with Jenkins in https://github.com/apache/spark/pull/20126.

**Before** - 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85559/consoleFull

```
[info] Setup the following environment variables for tests: 
...
```

**After** - 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85560/consoleFull

```
[info] Setup the following environment variables for tests: 
ENABLE_KAFKA_0_8_TESTS=1
ENABLE_FLUME_TESTS=1
...
```

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/HyukjinKwon/spark SPARK-21893

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/20128.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #20128


commit 3b1533a400ecf277cf0e18b8cb76a14305b0
Author: hyukjinkwon 
Date:   2017-12-31T15:35:54Z

Enables PySpark tests for Flume and Kafka in Jenkins




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20126: [DO-NOT-MERGE] Investigate if changes in flume.py...

[HyukjinKwon]

Github user HyukjinKwon closed the pull request at:

https://github.com/apache/spark/pull/20126


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org