[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21317 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3425/ ---

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21416 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21317 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3425/ ---

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21317 **[Test build #91098 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91098/testReport)** for PR 21317 at commit

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21416 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91091/ Test FAILed. ---

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21317 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91098/ Test PASSed. ---

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21317 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21331: [SPARK-24276][SQL] Order of literals in IN should not af...

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21331 **[Test build #91099 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91099/testReport)** for PR 21331 at commit

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21317 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21416 **[Test build #91091 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91091/testReport)** for PR 21416 at commit

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21317 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3546/

[GitHub] spark pull request #21331: [SPARK-24276][SQL] Order of literals in IN should...

2018-05-24 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21331#discussion_r190518402 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Canonicalize.scala --- @@ -85,6 +87,10 @@ object Canonicalize {

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-24 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21317 @liyinan926 ready for another round. I think its ready for merge. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21317 **[Test build #91098 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91098/testReport)** for PR 21317 at commit

[GitHub] spark pull request #21317: [SPARK-24232][k8s] Add support for secret env var...

2018-05-24 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21317#discussion_r190514132 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/features/EnvSecretsFeatureStepSuite.scala --- @@ -0,0 +1,59 @@

[GitHub] spark issue #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase w...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21295 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase w...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3545/

[GitHub] spark issue #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase w...

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21295 **[Test build #91097 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91097/testReport)** for PR 21295 at commit

[GitHub] spark issue #21415: [SPARK-24244][SPARK-24368][SQL] Passing only required co...

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21415 **[Test build #91096 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91096/testReport)** for PR 21415 at commit

[GitHub] spark issue #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase w...

2018-05-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21295 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21415: [SPARK-24244][SPARK-24368][SQL] Passing only required co...

2018-05-24 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/21415 jenkins, retest this, please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21260: [SPARK-23529][K8s] Support mounting volumes

2018-05-24 Thread andrusha
Github user andrusha commented on the issue: https://github.com/apache/spark/pull/21260 @liyinan926 yeap, ready for review. If you think it makes sense to make refactors I'm speaking of then I can add those too. ---

[GitHub] spark pull request #21405: [SPARK-24361][SQL] Polish code block manipulation...

2018-05-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21405#discussion_r190502747 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -149,6 +152,34 @@ trait Block extends

[GitHub] spark issue #21422: [Spark-24376][doc]Summary:compiling spark with scala-2.1...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21422 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21422: [Spark-24376][doc]Summary:compiling spark with scala-2.1...

2018-05-24 Thread gentlewangyu
Github user gentlewangyu commented on the issue: https://github.com/apache/spark/pull/21422 I have verified this patch --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #21405: [SPARK-24361][SQL] Polish code block manipulation...

2018-05-24 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21405#discussion_r190502024 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -149,6 +152,34 @@ trait Block extends

[GitHub] spark issue #21246: [SPARK-23901][SQL] Add masking functions

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21246 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21246: [SPARK-23901][SQL] Add masking functions

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21246 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3544/

[GitHub] spark pull request #21422: [Spark-24376][doc]Summary:compiling spark with sc...

2018-05-24 Thread gentlewangyu
GitHub user gentlewangyu opened a pull request: https://github.com/apache/spark/pull/21422 [Spark-24376][doc]Summary:compiling spark with scala-2.10 should use the -P parameter instead of -D What changes were proposed in this pull request? (Please fill in

[GitHub] spark issue #21422: [Spark-24376][doc]Summary:compiling spark with scala-2.1...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21422 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21405: [SPARK-24361][SQL] Polish code block manipulation...

2018-05-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21405#discussion_r190501136 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -149,6 +152,34 @@ trait Block extends

[GitHub] spark issue #21246: [SPARK-23901][SQL] Add masking functions

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21246 **[Test build #91095 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91095/testReport)** for PR 21246 at commit

[GitHub] spark issue #21246: [SPARK-23901][SQL] Add masking functions

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21246 **[Test build #91094 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91094/testReport)** for PR 21246 at commit

[GitHub] spark issue #21246: [SPARK-23901][SQL] Add masking functions

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21246 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91094/ Test FAILed. ---

[GitHub] spark issue #21246: [SPARK-23901][SQL] Add masking functions

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21246 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21246: [SPARK-23901][SQL] Add masking functions

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21246 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3543/

[GitHub] spark issue #21246: [SPARK-23901][SQL] Add masking functions

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21246 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21405: [SPARK-24361][SQL] Polish code block manipulation...

2018-05-24 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21405#discussion_r190497595 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -149,6 +152,34 @@ trait Block extends

[GitHub] spark issue #21246: [SPARK-23901][SQL] Add masking functions

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21246 **[Test build #91094 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91094/testReport)** for PR 21246 at commit

[GitHub] spark issue #21246: [SPARK-23901][SQL] Add masking functions

2018-05-24 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21246 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21398: [SPARK-24338][SQL] Fixed Hive CREATETABLE error i...

2018-05-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21398#discussion_r190494840 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -230,11 +232,29 @@ private[spark] class

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-05-24 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r190494243 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/planning/SelectedFieldSuite.scala --- @@ -0,0 +1,432 @@ +/* + * Licensed to

[GitHub] spark pull request #21369: [SPARK-22713][CORE] ExternalAppendOnlyMap leaks w...

2018-05-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21369#discussion_r190494153 --- Diff: core/src/test/scala/org/apache/spark/util/collection/ExternalAppendOnlyMapSuite.scala --- @@ -414,6 +415,99 @@ class

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-05-24 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r190493689 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruning.scala --- @@ -0,0 +1,154 @@ +/* + *

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-05-24 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r190492424 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruning.scala --- @@ -0,0 +1,154 @@ +/* + *

[GitHub] spark pull request #21421: Myfeature

2018-05-24 Thread gentlewangyu
Github user gentlewangyu closed the pull request at: https://github.com/apache/spark/pull/21421 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21421: Myfeature

2018-05-24 Thread gentlewangyu
GitHub user gentlewangyu opened a pull request: https://github.com/apache/spark/pull/21421 Myfeature ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain how this

[GitHub] spark issue #21391: [SPARK-24343][SQL] Avoid shuffle for the bucketed table ...

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21391 **[Test build #91093 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91093/testReport)** for PR 21391 at commit

[GitHub] spark issue #21391: [SPARK-24343][SQL] Avoid shuffle for the bucketed table ...

2018-05-24 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/21391 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-05-24 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r190491050 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetQuerySuite.scala --- @@ -879,6 +879,15 @@ class

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21416 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3542/

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21416 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19560: [SPARK-22334][SQL] Check table size from filesystem in c...

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19560 **[Test build #91092 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91092/testReport)** for PR 19560 at commit

[GitHub] spark pull request #21419: Branch 2.2

2018-05-24 Thread gentlewangyu
Github user gentlewangyu closed the pull request at: https://github.com/apache/spark/pull/21419 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21419: Branch 2.2

2018-05-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21419 @gentlewangyu, please close this and read https://spark.apache.org/contributing.html. Questions should go to mailing list and issues should be filed in JIRA. ---

[GitHub] spark issue #21420: [SPARK-24377][Spark Submit] make --py-files work in non ...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21420 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3541/

[GitHub] spark issue #21420: [SPARK-24377][Spark Submit] make --py-files work in non ...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21420 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21416 **[Test build #91091 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91091/testReport)** for PR 21416 at commit

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-05-24 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r190486220 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1256,8 +1256,18 @@ object SQLConf { "issues. Turn on

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-24 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21416 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21069: [SPARK-23920][SQL]add array_remove to remove all ...

2018-05-24 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21069#discussion_r190485891 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1882,3 +1882,123 @@ case class

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-05-24 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r190485768 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -286,7 +286,19 @@ case class FileSourceScanExec(

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-24 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21416 test this again --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-05-24 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r190485713 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ColumnarFileFormat.scala --- @@ -0,0 +1,32 @@ +/* + * Licensed to the

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r190485347 --- Diff: python/pyspark/sql/tests.py --- @@ -3040,6 +3040,50 @@ def test_csv_sampling_ratio(self): .csv(rdd, samplingRatio=0.5).schema

[GitHub] spark issue #21420: [SPARK-24377][Spark Submit] make --py-files work in non ...

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21420 **[Test build #91090 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91090/testReport)** for PR 21420 at commit

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-05-24 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r190484386 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/QueryPlanConstraints.scala --- @@ -99,27 +100,28 @@ trait

[GitHub] spark issue #21420: [SPARK-24377][Spark Submit] make --py-files work in non ...

2018-05-24 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21420 CC @HyukjinKwon @vanzin please help to review, thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #21420: [SPARK-24377][Spark Submit] make --py-files work ...

2018-05-24 Thread jerryshao
GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/21420 [SPARK-24377][Spark Submit] make --py-files work in non pyspark application ## What changes were proposed in this pull request? For some Spark applications, though they're a java

[GitHub] spark issue #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase w...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21295 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase w...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21295 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91086/ Test FAILed. ---

[GitHub] spark issue #21069: [SPARK-23920][SQL]add array_remove to remove all element...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21069 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91081/ Test FAILed. ---

[GitHub] spark issue #21069: [SPARK-23920][SQL]add array_remove to remove all element...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21069 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21391: [SPARK-24343][SQL] Avoid shuffle for the bucketed table ...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21391 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21391: [SPARK-24343][SQL] Avoid shuffle for the bucketed table ...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21391 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91088/ Test FAILed. ---

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21390: [SPARK-24340][Core] Clean up non-shuffle disk block mana...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21390 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91083/ Test FAILed. ---

[GitHub] spark issue #21415: [SPARK-24244][SPARK-24368][SQL] Passing only required co...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21415 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91087/ Test FAILed. ---

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21416 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91089/ Test FAILed. ---

[GitHub] spark issue #21390: [SPARK-24340][Core] Clean up non-shuffle disk block mana...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21390 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21390: [SPARK-24340][Core] Clean up non-shuffle disk block mana...

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21390 **[Test build #91083 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91083/testReport)** for PR 21390 at commit

[GitHub] spark issue #21415: [SPARK-24244][SPARK-24368][SQL] Passing only required co...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21415 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21415: [SPARK-24244][SPARK-24368][SQL] Passing only required co...

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21415 **[Test build #91087 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91087/testReport)** for PR 21415 at commit

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21416 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase w...

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21295 **[Test build #91086 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91086/testReport)** for PR 21295 at commit

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21399 **[Test build #91085 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91085/testReport)** for PR 21399 at commit

[GitHub] spark issue #21391: [SPARK-24343][SQL] Avoid shuffle for the bucketed table ...

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21391 **[Test build #91088 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91088/testReport)** for PR 21391 at commit

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21416 **[Test build #91089 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91089/testReport)** for PR 21416 at commit

[GitHub] spark issue #21069: [SPARK-23920][SQL]add array_remove to remove all element...

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21069 **[Test build #91081 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91081/testReport)** for PR 21069 at commit

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91085/ Test FAILed. ---

[GitHub] spark issue #21419: Branch 2.2

2018-05-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21419 could you please close this PR? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21419: Branch 2.2

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21419 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21419: Branch 2.2

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21419 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21419: Branch 2.2

2018-05-24 Thread gentlewangyu
GitHub user gentlewangyu opened a pull request: https://github.com/apache/spark/pull/21419 Branch 2.2 ## What changes were proposed in this pull request? compiling spark with scala-2.10 should use the -P parameter instead of -D ## How was this patch tested?

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21416 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3540/

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21416 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21416 **[Test build #91089 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91089/testReport)** for PR 21416 at commit

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-24 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21416 +@cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-05-24 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r190479026 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -162,7 +162,9 @@ case class FilterExec(condition:

[GitHub] spark issue #21402: SPARK-24355 Spark external shuffle server improvement to...

2018-05-24 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21402 while we are here, could we also add or at least propose some metrics around this, such as number of open block failure, or even number of block threads? we have suffer a lot from the

<    1   2   3   4   5   6   >