[GitHub] spark pull request #20422: [SPARK-23253][Core][Shuffle]Only write shuffle te...

2018-01-31 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/20422#discussion_r165225733 --- Diff: core/src/test/scala/org/apache/spark/shuffle/sort/IndexShuffleBlockResolverSuite.scala --- @@ -123,7 +123,7 @@ class

[GitHub] spark pull request #20461: [SPARK-23289][CORE]OneForOneBlockFetcher.Download...

2018-01-31 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/20461#discussion_r165225508 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/OneForOneBlockFetcher.java --- @@ -171,7 +171,9 @@ private void

[GitHub] spark issue #20460: [SPARK-23285][K8S] Allow fractional values for spark.exe...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20460 **[Test build #86895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86895/testReport)** for PR 20460 at commit

[GitHub] spark pull request #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregati...

2018-01-31 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/19872#discussion_r165224852 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala --- @@ -199,7 +200,7 @@ object ExtractFiltersAndInnerJoins

[GitHub] spark issue #20460: [SPARK-23285][K8S] Allow fractional values for spark.exe...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20460 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20460: [SPARK-23285][K8S] Allow fractional values for spark.exe...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20460 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86894/ Test FAILed. ---

[GitHub] spark issue #20460: [SPARK-23285][K8S] Allow fractional values for spark.exe...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20460 **[Test build #86894 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86894/testReport)** for PR 20460 at commit

[GitHub] spark issue #20460: [SPARK-23285][K8S] Allow fractional values for spark.exe...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20460 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20460: [SPARK-23285][K8S] Allow fractional values for spark.exe...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20460 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/447/

[GitHub] spark issue #20460: [SPARK-23285][K8S] Allow fractional values for spark.exe...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20460 **[Test build #86894 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86894/testReport)** for PR 20460 at commit

[GitHub] spark pull request #20460: [SPARK-23285][K8S] Allow fractional values for sp...

2018-01-31 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20460#discussion_r165223463 --- Diff: docs/configuration.md --- @@ -1220,14 +1220,15 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark pull request #20460: [SPARK-23285][K8S] Allow fractional values for sp...

2018-01-31 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20460#discussion_r165223500 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala --- @@ -267,7 +267,11 @@ private[deploy] class

[GitHub] spark issue #20458: changed scala example from java "style" to scala

2018-01-31 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20458 Yeah, that's what I was thinking of. Subclasses of `App` don't necessarily work in Spark (not just Scala). I don't know if that has changed as Spark and Scala moved forward. While that's an

[GitHub] spark pull request #16989: [SPARK-19659] Fetch big blocks to disk when shuff...

2018-01-31 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16989#discussion_r165222051 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/OneForOneBlockFetcher.java --- @@ -126,4 +150,38 @@ private void

[GitHub] spark pull request #20460: [SPARK-23285][K8S] Allow fractional values for sp...

2018-01-31 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20460#discussion_r165221502 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala --- @@ -267,7 +267,11 @@ private[deploy] class SparkSubmitArguments(args:

[GitHub] spark pull request #20460: [SPARK-23285][K8S] Allow fractional values for sp...

2018-01-31 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20460#discussion_r165221518 --- Diff: docs/configuration.md --- @@ -1220,14 +1220,15 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark issue #20419: [SPARK-23032][SQL][FOLLOW-UP]Add codegenStageId in comme...

2018-01-31 Thread rednaxelafx
Github user rednaxelafx commented on the issue: https://github.com/apache/spark/pull/20419 I did the experiment that I asked about at https://github.com/apache/spark/pull/20419#issuecomment-361359825 , and verified that under the current implementation, this PR will not affect the

[GitHub] spark pull request #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregati...

2018-01-31 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/19872#discussion_r165220142 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala --- @@ -199,7 +200,7 @@ object ExtractFiltersAndInnerJoins

[GitHub] spark pull request #16989: [SPARK-19659] Fetch big blocks to disk when shuff...

2018-01-31 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/16989#discussion_r165214410 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/OneForOneBlockFetcher.java --- @@ -126,4 +150,38 @@ private void

[GitHub] spark pull request #20461: [SPARK-23289][CORE]OneForOneBlockFetcher.Download...

2018-01-31 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20461#discussion_r165213755 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/OneForOneBlockFetcher.java --- @@ -171,7 +171,9 @@ private void

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2018-01-31 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r165207476 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala --- @@ -246,6 +251,38 @@ class BlockManagerMasterEndpoint(

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2018-01-31 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r165207798 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala --- @@ -246,6 +251,38 @@ class BlockManagerMasterEndpoint(

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2018-01-31 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r165210078 --- Diff: core/src/main/scala/org/apache/spark/CacheRecoveryManager.scala --- @@ -0,0 +1,189 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #20458: changed scala example from java "style" to scala

2018-01-31 Thread mantovani
Github user mantovani commented on the issue: https://github.com/apache/spark/pull/20458 I just tested with Scala 2.10.6 and worked perfect. 18/01/31 17:21:28 INFO DAGScheduler: Job 1 finished: count at Bootstrap.scala:14, took 0.024446 s DemoBug: rslt1 = 3 rslt2 = 3

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-01-31 Thread gerashegalov
Github user gerashegalov commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r165206579 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -744,7 +744,9 @@ object SparkSubmit extends CommandLineUtils with

[GitHub] spark pull request #20456: [SPARK-22624][PYSPARK] Expose range partitioning ...

2018-01-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20456#discussion_r165205764 --- Diff: python/pyspark/sql/dataframe.py --- @@ -667,6 +667,55 @@ def repartition(self, numPartitions, *cols): else: raise

[GitHub] spark pull request #20456: [SPARK-22624][PYSPARK] Expose range partitioning ...

2018-01-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20456#discussion_r165205030 --- Diff: python/pyspark/sql/dataframe.py --- @@ -667,6 +667,55 @@ def repartition(self, numPartitions, *cols): else: raise

[GitHub] spark issue #20458: changed scala example from java "style" to scala

2018-01-31 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20458 Did you try it with Scala 2.10? That's still supported by Spark. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20456: [SPARK-22624][PYSPARK] Expose range partitioning shuffle...

2018-01-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20456 cc @ueshin @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20462: [SPARK-23020][core] Fix another race in the in-process l...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20462 **[Test build #86893 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86893/testReport)** for PR 20462 at commit

[GitHub] spark issue #20462: [SPARK-23020][core] Fix another race in the in-process l...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20462 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/446/

[GitHub] spark issue #20462: [SPARK-23020][core] Fix another race in the in-process l...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20462 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20458: changed scala example from java "style" to scala

2018-01-31 Thread mantovani
Github user mantovani commented on the issue: https://github.com/apache/spark/pull/20458 Hello Marcel, This is an old bug and was fixed long time ago, I just runned the following example and got the correct result: DemoBug: rslt1 = 3 rslt2 = 3 Doesn't make

[GitHub] spark issue #20462: [SPARK-23020][core] Fix another race in the in-process l...

2018-01-31 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20462 @cloud-fan @sameeragarwal hopefully the last one. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #20462: [SPARK-23020][core] Fix another race in the in-pr...

2018-01-31 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/20462 [SPARK-23020][core] Fix another race in the in-process launcher test. First the bad news: there's an unfixable race in the launcher code. (By unfixable I mean it would take a lot more effort

[GitHub] spark pull request #20453: [SPARK-23281][SQL] Query produces results in inco...

2018-01-31 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20453#discussion_r165203182 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -18,7 +18,6 @@ package org.apache.spark.sql import

[GitHub] spark pull request #20453: [SPARK-23281][SQL] Query produces results in inco...

2018-01-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20453#discussion_r165201975 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -18,7 +18,6 @@ package org.apache.spark.sql import

[GitHub] spark issue #20461: [SPARK-23289][CORE]OneForOneBlockFetcher.DownloadCallbac...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20461 **[Test build #86892 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86892/testReport)** for PR 20461 at commit

[GitHub] spark issue #20461: [SPARK-23289][CORE]OneForOneBlockFetcher.DownloadCallbac...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20461 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/445/

[GitHub] spark issue #20461: [SPARK-23289][CORE]OneForOneBlockFetcher.DownloadCallbac...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20461 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20461: [SPARK-23289][CORE]OneForOneBlockFetcher.DownloadCallbac...

2018-01-31 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20461 cc @cloud-fan @squito --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20461: [SPARK-23289][CORE]OneForOneBlockFetcher.Download...

2018-01-31 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/20461 [SPARK-23289][CORE]OneForOneBlockFetcher.DownloadCallback.onData should write the bufer fully ## What changes were proposed in this pull request? `channel.write(buf)` may not write the

[GitHub] spark pull request #20453: [SPARK-23281][SQL] Query produces results in inco...

2018-01-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20453 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20453: [SPARK-23281][SQL] Query produces results in incorrect o...

2018-01-31 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/20453 Thanks a LOT @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20453: [SPARK-23281][SQL] Query produces results in incorrect o...

2018-01-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20453 LGTM Thanks! Merged to master/2.3/2.2 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20460: [SPARK-23285][K8S] Allow fractional values for spark.exe...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20460 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/442/ ---

[GitHub] spark pull request #20453: [SPARK-23281][SQL] Query produces results in inco...

2018-01-31 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20453#discussion_r165198141 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1493,7 +1493,7 @@ class Analyzer(

[GitHub] spark issue #20459: [SPARK-23107][ML] ML 2.3 QA: New Scala APIs, docs.

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20459 **[Test build #86891 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86891/testReport)** for PR 20459 at commit

[GitHub] spark issue #20459: [SPARK-23107][ML] ML 2.3 QA: New Scala APIs, docs.

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20459 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20459: [SPARK-23107][ML] ML 2.3 QA: New Scala APIs, docs.

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20459 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/444/

[GitHub] spark issue #20460: [SPARK-23285][K8S] Allow fractional values for spark.exe...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20460 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/442/ ---

[GitHub] spark issue #20460: [SPARK-23285][K8S] Allow fractional values for spark.exe...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20460 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/443/

[GitHub] spark issue #20460: [SPARK-23285][K8S] Allow fractional values for spark.exe...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20460 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20460: [SPARK-23285][K8S] Allow fractional values for sp...

2018-01-31 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/20460#discussion_r165195110 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala --- @@ -267,7 +267,11 @@ private[deploy] class SparkSubmitArguments(args:

[GitHub] spark issue #20460: [SPARK-23285][K8S] Allow fractional values for spark.exe...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20460 **[Test build #86890 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86890/testReport)** for PR 20460 at commit

[GitHub] spark pull request #20460: [SPARK-23285][K8S] Allow fractional values for sp...

2018-01-31 Thread liyinan926
GitHub user liyinan926 opened a pull request: https://github.com/apache/spark/pull/20460 [SPARK-23285][K8S] Allow fractional values for spark.executor.cores ## What changes were proposed in this pull request? K8s treats CPU as a "compressible resource" and can actually

[GitHub] spark issue #20327: [SPARK-12963][CORE] NM host for driver end points

2018-01-31 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20327 cc @vanzin since you are expert on this topic. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #16989: [SPARK-19659] Fetch big blocks to disk when shuff...

2018-01-31 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16989#discussion_r165193691 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/OneForOneBlockFetcher.java --- @@ -126,4 +150,38 @@ private void

[GitHub] spark pull request #20453: [SPARK-23281][SQL] Query produces results in inco...

2018-01-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20453#discussion_r165193579 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1493,7 +1493,7 @@ class Analyzer( //

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-01-31 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r165193521 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -744,7 +744,9 @@ object SparkSubmit extends CommandLineUtils with

[GitHub] spark pull request #20459: [SPARK-23107][ML] ML 2.3 QA: New Scala APIs, docs...

2018-01-31 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/20459#discussion_r165189663 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala --- @@ -93,7 +93,7 @@ private[feature] trait

[GitHub] spark issue #20458: changed scala example from java "style" to scala

2018-01-31 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20458 > It's not just "will" work, it's working. https://issues.apache.org/jira/browse/SPARK-4170 --- - To unsubscribe,

[GitHub] spark issue #20415: [SPARK-23247][SQL]combines Unsafe operations and statist...

2018-01-31 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20415 LGTM, only a minor comment --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20415: [SPARK-23247][SQL]combines Unsafe operations and ...

2018-01-31 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20415#discussion_r165188943 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -326,22 +325,23 @@ case class FileSourceScanExec(

[GitHub] spark issue #20459: [SPARK-23107][ML] ML 2.3 QA: New Scala APIs, docs.

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20459 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20459: [SPARK-23107][ML] ML 2.3 QA: New Scala APIs, docs.

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20459 **[Test build #86889 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86889/testReport)** for PR 20459 at commit

[GitHub] spark issue #20459: [SPARK-23107][ML] ML 2.3 QA: New Scala APIs, docs.

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20459 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86889/ Test FAILed. ---

[GitHub] spark issue #20459: [SPARK-23107][ML] ML 2.3 QA: New Scala APIs, docs.

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20459 **[Test build #86889 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86889/testReport)** for PR 20459 at commit

[GitHub] spark issue #20459: [SPARK-23107][ML] ML 2.3 QA: New Scala APIs, docs.

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20459 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/442/

[GitHub] spark issue #20459: [SPARK-23107][ML] ML 2.3 QA: New Scala APIs, docs.

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20459 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20458: changed scala example from java "style" to scala

2018-01-31 Thread mantovani
Github user mantovani commented on the issue: https://github.com/apache/spark/pull/20458 It's not just "will" work, _it's working_. I never said that it wasn't working, the Scala example was wrote using Java "style" not Scala "style" which . Which you don't create a main method but

[GitHub] spark issue #20385: [SPARK-21396][SQL] Fixes MatchError when UDTs are passed...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20385 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20385: [SPARK-21396][SQL] Fixes MatchError when UDTs are passed...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20385 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86887/ Test PASSed. ---

[GitHub] spark issue #20385: [SPARK-21396][SQL] Fixes MatchError when UDTs are passed...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20385 **[Test build #86887 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86887/testReport)** for PR 20385 at commit

[GitHub] spark pull request #20459: [SPARK-23107][ML] ML 2.3 QA: New Scala APIs, docs...

2018-01-31 Thread yanboliang
GitHub user yanboliang opened a pull request: https://github.com/apache/spark/pull/20459 [SPARK-23107][ML] ML 2.3 QA: New Scala APIs, docs. ## What changes were proposed in this pull request? Audit new APIs and docs in 2.3.0. ## How was this patch tested? No test.

[GitHub] spark issue #20458: changed scala example from java "style" to scala

2018-01-31 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20458 IIRC extending App won't work, or didn't. The example is already correct. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20458: changed scala example from java "style" to scala

2018-01-31 Thread mantovani
Github user mantovani commented on the issue: https://github.com/apache/spark/pull/20458 I just changed to the right section. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20458: changed scala example from java "style" to scala

2018-01-31 Thread mantovani
Github user mantovani commented on a diff in the pull request: https://github.com/apache/spark/pull/20458#discussion_r165180039 --- Diff: docs/quick-start.md --- @@ -299,19 +299,17 @@ We'll create a very simple Spark application, `SimpleApp.java`: import

[GitHub] spark pull request #16989: [SPARK-19659] Fetch big blocks to disk when shuff...

2018-01-31 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/16989#discussion_r165178844 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/OneForOneBlockFetcher.java --- @@ -126,4 +150,38 @@ private void

[GitHub] spark pull request #20458: changed scala example from java "style" to scala

2018-01-31 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20458#discussion_r165178509 --- Diff: docs/quick-start.md --- @@ -299,19 +299,17 @@ We'll create a very simple Spark application, `SimpleApp.java`: import

[GitHub] spark issue #20454: [SPARK-23202][SQL] Add new API in DataSourceWriter: onDa...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20454 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19802: [SPARK-22594][CORE] Handling spark-submit and master ver...

2018-01-31 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19802 I think this is the wrong place for the fix, if one is desired. The transport library doesn't know anything about serialization, that's all handled by the code using it. In this case, the 'RpcEnv'

[GitHub] spark issue #20454: [SPARK-23202][SQL] Add new API in DataSourceWriter: onDa...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20454 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86884/ Test PASSed. ---

[GitHub] spark issue #20454: [SPARK-23202][SQL] Add new API in DataSourceWriter: onDa...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20454 **[Test build #86884 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86884/testReport)** for PR 20454 at commit

[GitHub] spark issue #20458: changed scala example from java "style" to scala

2018-01-31 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20458 the example is written in Java, not in Scala. I think you can close this PR, thanks. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20457: [SPARK-23110][MINOR] Make linearRegressionModel construc...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20457 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86888/ Test PASSed. ---

[GitHub] spark issue #20457: [SPARK-23110][MINOR] Make linearRegressionModel construc...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20457 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20457: [SPARK-23110][MINOR] Make linearRegressionModel construc...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20457 **[Test build #86888 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86888/testReport)** for PR 20457 at commit

[GitHub] spark issue #18423: SPARK-21158: Fix case sensitive of temp table/view name

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18423 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20442: [SPARK-23265][SQL]Update multi-column error handling log...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20442 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20442: [SPARK-23265][SQL]Update multi-column error handling log...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20442 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86886/ Test PASSed. ---

[GitHub] spark issue #20442: [SPARK-23265][SQL]Update multi-column error handling log...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20442 **[Test build #86886 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86886/testReport)** for PR 20442 at commit

[GitHub] spark issue #20458: changed scala example from java "style" to scala

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20458 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20458: changed scala example from java "style" to scala

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20458 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20161: [SPARK-21525][streaming] Check error code from su...

2018-01-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20161 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20458: changed scala example from java "style" to scala

2018-01-31 Thread mantovani
GitHub user mantovani opened a pull request: https://github.com/apache/spark/pull/20458 changed scala example from java "style" to scala ## What changes were proposed in this pull request? I changed an example wrote in Scala using Java "style" to Scala "style". ## How

[GitHub] spark issue #20161: [SPARK-21525][streaming] Check error code from superviso...

2018-01-31 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20161 Alright merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20453: [SPARK-23281][SQL] Query produces results in incorrect o...

2018-01-31 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/20453 cc @gatorsmile @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20422: [SPARK-23253][Core][Shuffle]Only write shuffle te...

2018-01-31 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20422#discussion_r165161260 --- Diff: core/src/test/scala/org/apache/spark/shuffle/sort/IndexShuffleBlockResolverSuite.scala --- @@ -133,4 +133,65 @@ class

[GitHub] spark pull request #20422: [SPARK-23253][Core][Shuffle]Only write shuffle te...

2018-01-31 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20422#discussion_r165159188 --- Diff: core/src/test/scala/org/apache/spark/shuffle/sort/IndexShuffleBlockResolverSuite.scala --- @@ -133,4 +133,65 @@ class

<    1   2   3   4   5   6   7   >