[GitHub] spark issue #21192: [SPARK-24118][SQL] Flexible format for the lineSep optio...

2018-05-16 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/21192 eh I actually think separated makes it much simpler to look at, compared with an array. Why complicate the API and require users to understand how to specify an array (in all languages)? One

[GitHub] spark pull request #20894: [SPARK-23786][SQL] Checking column names of csv h...

2018-05-16 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/20894#discussion_r188841673 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala --- @@ -206,24 +280,33 @@ object

[GitHub] spark pull request #20894: [SPARK-23786][SQL] Checking column names of csv h...

2018-05-16 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/20894#discussion_r188841632 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala --- @@ -202,28 +263,33 @@ object

[GitHub] spark issue #21350: [SPARK-24303][PYTHON] Update cloudpickle to v0.4.4

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21350 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21028 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3282/

[GitHub] spark issue #21350: [SPARK-24303][PYTHON] Update cloudpickle to v0.4.4

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21350 **[Test build #90709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90709/testReport)** for PR 21350 at commit

[GitHub] spark issue #21350: [SPARK-24303][PYTHON] Update cloudpickle to v0.4.4

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21350 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90709/ Test PASSed. ---

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21028 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20611: [SPARK-23425][SQL]Support wildcard in HDFS path for load...

2018-05-16 Thread sujith71955
Github user sujith71955 commented on the issue: https://github.com/apache/spark/pull/20611 @kevinyu98 Thanks for you feedback, i will test once all my scenario with your fix, but here one more point i need to mention is like my fix has also has impact in the logic of local path

[GitHub] spark issue #21252: [SPARK-24193] Sort by disk when number of limit is big i...

2018-05-16 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21252 can you change the PR title? otherwise LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21028#discussion_r188840260 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -18,15 +18,53 @@ package

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21028 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21028 **[Test build #90711 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90711/testReport)** for PR 21028 at commit

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21028 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90711/ Test FAILed. ---

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-16 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21028 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20936: [SPARK-23503][SS] Enforce sequencing of committed epochs...

2018-05-16 Thread efimpoberezkin
Github user efimpoberezkin commented on the issue: https://github.com/apache/spark/pull/20936 @tdas can't start tests --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21327: [SPARK-24107][CORE][followup] ChunkedByteBuffer.writeFul...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21327 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21327: [SPARK-24107][CORE][followup] ChunkedByteBuffer.writeFul...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21327 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3281/

[GitHub] spark issue #21348: [SPARK-22739][Catalyst Additional Expression Support for...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21348 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90701/ Test PASSed. ---

[GitHub] spark issue #21348: [SPARK-22739][Catalyst Additional Expression Support for...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21348 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21028 **[Test build #90711 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90711/testReport)** for PR 21028 at commit

[GitHub] spark issue #21327: [SPARK-24107][CORE][followup] ChunkedByteBuffer.writeFul...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21327 **[Test build #90710 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90710/testReport)** for PR 21327 at commit

[GitHub] spark issue #21348: [SPARK-22739][Catalyst Additional Expression Support for...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21348 **[Test build #90701 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90701/testReport)** for PR 21348 at commit

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-16 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21028 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21327: [SPARK-24107][CORE][followup] ChunkedByteBuffer.writeFul...

2018-05-16 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21327 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21345: [SPARK-24159] [SS] Enable no-data micro batches for stre...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21345 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90702/ Test PASSed. ---

[GitHub] spark issue #20936: [SPARK-23503][SS] Enforce sequencing of committed epochs...

2018-05-16 Thread efimpoberezkin
Github user efimpoberezkin commented on the issue: https://github.com/apache/spark/pull/20936 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21345: [SPARK-24159] [SS] Enable no-data micro batches for stre...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21345 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21345: [SPARK-24159] [SS] Enable no-data micro batches for stre...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21345 **[Test build #90702 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90702/testReport)** for PR 21345 at commit

[GitHub] spark issue #21086: [SPARK-24002] [SQL] Task not serializable caused by org....

2018-05-16 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21086 @ghoto Sure. Will backport it tonight. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21350: [SPARK-24303][PYTHON] Update cloudpickle to v0.4.4

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21350 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21350: [SPARK-24303][PYTHON] Update cloudpickle to v0.4.4

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21350 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3280/

[GitHub] spark issue #21350: [SPARK-24303][PYTHON] Update cloudpickle to v0.4.4

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21350 **[Test build #90709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90709/testReport)** for PR 21350 at commit

[GitHub] spark pull request #21350: [SPARK-24303][PYTHON] Update cloudpickle to v0.4....

2018-05-16 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/21350 [SPARK-24303][PYTHON] Update cloudpickle to v0.4.4 ## What changes were proposed in this pull request? cloudpickle 0.4.4 is release -

[GitHub] spark issue #21350: [SPARK-24303][PYTHON] Update cloudpickle to v0.4.4

2018-05-16 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21350 cc @ueshin and @BryanCutler --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #21208: [SPARK-23925][SQL] Add array_repeat collection fu...

2018-05-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21208 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21208: [SPARK-23925][SQL] Add array_repeat collection function

2018-05-16 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21208 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21349: [MINOR][PROJECT-INFRA] Check if 'original_head' variable...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21349 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21349: [MINOR][PROJECT-INFRA] Check if 'original_head' variable...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21349 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3279/

[GitHub] spark issue #21349: [MINOR][PROJECT-INFRA] Check if 'original_head' variable...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21349 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21349: [MINOR][PROJECT-INFRA] Check if 'original_head' variable...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21349 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3278/

[GitHub] spark issue #21349: [MINOR][PROJECT-INFRA] Check if 'original_head' variable...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21349 **[Test build #90708 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90708/testReport)** for PR 21349 at commit

[GitHub] spark pull request #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work ...

2018-05-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21267 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21349: [MINOR][PROJECT-INFRA] Check if 'original_head' variable...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21349 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90707/ Test FAILed. ---

[GitHub] spark issue #21349: [MINOR][PROJECT-INFRA] Check if 'original_head' variable...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21349 **[Test build #90707 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90707/testReport)** for PR 21349 at commit

[GitHub] spark issue #21349: [MINOR][PROJECT-INFRA] Check if 'original_head' variable...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21349 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-16 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21267 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21349: [MINOR][PROJECT-INFRA] Check if 'original_head' variable...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21349 **[Test build #90707 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90707/testReport)** for PR 21349 at commit

[GitHub] spark pull request #21349: [MINOR][PROJECT-INFRA] Check if 'original_head' v...

2018-05-16 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/21349 [MINOR][PROJECT-INFRA] Check if 'original_head' variable is defined in clean_up at merge script ## What changes were proposed in this pull request? This PR proposes to check if global

[GitHub] spark issue #21268: [SPARK-24209][SHS] Automatic retrieve proxyBase from Kno...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21268 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90698/ Test PASSed. ---

[GitHub] spark issue #21268: [SPARK-24209][SHS] Automatic retrieve proxyBase from Kno...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21268 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21268: [SPARK-24209][SHS] Automatic retrieve proxyBase from Kno...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21268 **[Test build #90698 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90698/testReport)** for PR 21268 at commit

[GitHub] spark issue #21337: [SPARK-24234][SS] Reader for continuous processing shuff...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21337 **[Test build #90706 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90706/testReport)** for PR 21337 at commit

[GitHub] spark issue #21337: [SPARK-24234][SS] Reader for continuous processing shuff...

2018-05-16 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/21337 @tdas --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21337: [SPARK-24234][SS] Reader for continuous processin...

2018-05-16 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/21337#discussion_r188831887 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/shuffle/ContinuousShuffleReadSuite.scala --- @@ -0,0 +1,122 @@ +/*

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21267 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90704/ Test PASSed. ---

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21267 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-05-16 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20146 ping @dbtsai @jkbradley --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21267 **[Test build #90704 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90704/testReport)** for PR 21267 at commit

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-16 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21193 Thanks @maropu @kiszk @mgaido91 @mgaido91 I overlooked it. Just replied you. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-16 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r188830465 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -114,6 +114,113 @@ object JavaCode { }

[GitHub] spark issue #21337: [SPARK-24234][SS] Reader for continuous processing shuff...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21337 **[Test build #90705 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90705/testReport)** for PR 21337 at commit

[GitHub] spark pull request #21337: [SPARK-24234][SS] Reader for continuous processin...

2018-05-16 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/21337#discussion_r188829894 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/shuffle/ContinuousShuffleReadRDD.scala --- @@ -0,0 +1,64 @@

[GitHub] spark pull request #21337: [SPARK-24234][SS] Reader for continuous processin...

2018-05-16 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/21337#discussion_r188829815 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/shuffle/ContinuousShuffleReadRDD.scala --- @@ -0,0 +1,64 @@

[GitHub] spark pull request #21337: [SPARK-24234][SS] Reader for continuous processin...

2018-05-16 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/21337#discussion_r188829780 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/shuffle/UnsafeRowReceiver.scala --- @@ -0,0 +1,56 @@ +/*

[GitHub] spark pull request #21299: [SPARK-24250][SQL] support accessing SQLConf insi...

2018-05-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21299#discussion_r188828684 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SQLExecution.scala --- @@ -90,13 +92,42 @@ object SQLExecution { * thread from

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21267 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21267 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3277/

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21267 **[Test build #90704 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90704/testReport)** for PR 21267 at commit

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-16 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21267 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21346: [SPARK-6237][NETWORK] Network-layer changes to allow str...

2018-05-16 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/21346 All good questions and stuff I had wondered about too -- I should actually be sure to comment on these on the jira as well: > I recall that the problem with large shuffle blocks was that the

[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21114 **[Test build #90703 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90703/testReport)** for PR 21114 at commit

[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...

2018-05-16 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/21114 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21347: [SPARK-24290][ML] add support for Array input for...

2018-05-16 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21347#discussion_r188820444 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/Instrumentation.scala --- @@ -125,6 +125,19 @@ private[spark] class Instrumentation[E <:

[GitHub] spark issue #21345: [SPARK-24159] [SS] Enable no-data micro batches for stre...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21345 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21345: [SPARK-24159] [SS] Enable no-data micro batches for stre...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21345 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3276/

[GitHub] spark issue #21345: [SPARK-24159] [SS] Enable no-data micro batches for stre...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21345 **[Test build #90702 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90702/testReport)** for PR 21345 at commit

[GitHub] spark issue #21348: [SPARK-22739][Catalyst Additional Expression Support for...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21348 **[Test build #90701 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90701/testReport)** for PR 21348 at commit

[GitHub] spark pull request #21090: [SPARK-24026][ML] Add Power Iteration Clustering ...

2018-05-16 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/21090#discussion_r188813735 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/PowerIterationClustering.scala --- @@ -0,0 +1,256 @@ +/* + * Licensed to the

[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-16 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/21342 I'm also in favor of delaying for a couple of days for more detailed review because historically I think these types of changes have been high risk. The risk calculus might be a bit different if

[GitHub] spark pull request #20973: [SPARK-20114][ML] spark.ml parity for sequential ...

2018-05-16 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20973#discussion_r188813405 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/PrefixSpan.scala --- @@ -0,0 +1,96 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #21183: [SPARK-22210][ML] Add seed for LDA variationalTop...

2018-05-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21183 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20973: [SPARK-20114][ML] spark.ml parity for sequential ...

2018-05-16 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20973#discussion_r188813297 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/PrefixSpan.scala --- @@ -0,0 +1,96 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #21183: [SPARK-22210][ML] Add seed for LDA variationalTopicInfer...

2018-05-16 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/21183 Thanks for checking this manually. Since the test sometimes fails, then let's leave it. LGTM Merging with master Thanks @ludatabricks and @mengxr ! ---

[GitHub] spark issue #21348: [SPARK-22739][Catalyst Additional Expression Support for...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21348 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21348: [SPARK-22739][Catalyst Additional Expression Support for...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21348 **[Test build #90700 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90700/testReport)** for PR 21348 at commit

[GitHub] spark issue #21348: [SPARK-22739][Catalyst Additional Expression Support for...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21348 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90700/ Test FAILed. ---

[GitHub] spark issue #21348: [SPARK-22739][Catalyst Additional Expression Support for...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21348 **[Test build #90700 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90700/testReport)** for PR 21348 at commit

[GitHub] spark pull request #20085: [SPARK-22739][Catalyst][WIP] Additional Expressio...

2018-05-16 Thread bdrillard
Github user bdrillard closed the pull request at: https://github.com/apache/spark/pull/20085 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20085: [SPARK-22739][Catalyst][WIP] Additional Expression Suppo...

2018-05-16 Thread bdrillard
Github user bdrillard commented on the issue: https://github.com/apache/spark/pull/20085 Closing this PR in favor of #21348. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21045: [SPARK-23931][SQL] Adds zip function to sparksql

2018-05-16 Thread DylanGuedes
Github user DylanGuedes commented on the issue: https://github.com/apache/spark/pull/21045 Great! Also I finally discovered the unit-tests log file that logs the generated java code :D Since now it works I'll remove the WIP tag and focus on the other suggestions (such as new

[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers

2018-05-16 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20894 Right, will proceed reviewing this. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21348: [SPARK-22739][Catalyst Additional Expression Support for...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21348 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21348: [SPARK-22739][Catalyst Additional Expression Support for...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21348 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90699/ Test FAILed. ---

[GitHub] spark issue #21348: [SPARK-22739][Catalyst Additional Expression Support for...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21348 **[Test build #90699 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90699/testReport)** for PR 21348 at commit

[GitHub] spark issue #21348: [SPARK-22739][Catalyst Additional Expression Support for...

2018-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21348 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21348: [SPARK-22739][Catalyst Additional Expression Support for...

2018-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21348 **[Test build #90699 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90699/testReport)** for PR 21348 at commit

[GitHub] spark pull request #21348: [SPARK-22739][Catalyst Additional Expression Supp...

2018-05-16 Thread bdrillard
Github user bdrillard commented on a diff in the pull request: https://github.com/apache/spark/pull/21348#discussion_r188810286 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -408,16 +439,19 @@ object NewInstance {

[GitHub] spark pull request #21348: [SPARK-22739][Catalyst Additional Expression Supp...

2018-05-16 Thread bdrillard
GitHub user bdrillard opened a pull request: https://github.com/apache/spark/pull/21348 [SPARK-22739][Catalyst Additional Expression Support for Objects ## What changes were proposed in this pull request? This PR is a working followup to the expression work begun in #20085.

[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-16 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21342 Could we hold this PR? This change sounds risky. `OutOfMemoryError` is not issued by Spark. If we eat it, it could cause serious outcomes. cc @zsxwing @JoshRosen ---

  1   2   3   4   5   >