[GitHub] spark issue #20589: [SPARK-23394][UI] In RDD storage page show the executor ...

2018-02-13 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20589 jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20556: [SPARK-23367][Build] Include python document style check...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20556 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20556: [SPARK-23367][Build] Include python document style check...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20556 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87439/ Test FAILed. ---

[GitHub] spark issue #20556: [SPARK-23367][Build] Include python document style check...

2018-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20556 **[Test build #87439 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87439/testReport)** for PR 20556 at commit

[GitHub] spark issue #20605: [SPARK-23419][SPARK-23416][SS] data source v2 write path...

2018-02-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20605 Also cc @tdas @marmbrus --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20555: [SPARK-23366] Improve hot reading path in ReadAheadInput...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20555 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20555: [SPARK-23366] Improve hot reading path in ReadAheadInput...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20555 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87438/ Test PASSed. ---

[GitHub] spark issue #20555: [SPARK-23366] Improve hot reading path in ReadAheadInput...

2018-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20555 **[Test build #87438 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87438/testReport)** for PR 20555 at commit

[GitHub] spark pull request #20424: [Spark-23240][python] Better error message when e...

2018-02-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20424#discussion_r168086187 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala --- @@ -191,7 +192,26 @@ private[spark] class

[GitHub] spark pull request #20424: [Spark-23240][python] Better error message when e...

2018-02-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20424#discussion_r168081472 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala --- @@ -191,7 +192,26 @@ private[spark] class

[GitHub] spark issue #20594: [SPARK-23377][ML] Fixes Bucketizer with multiple columns...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20594 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20556: [SPARK-23367][Build] Include python document styl...

2018-02-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20556#discussion_r168079061 --- Diff: dev/tox.ini --- @@ -17,3 +17,5 @@ ignore=E402,E731,E241,W503,E226,E722,E741,E305 max-line-length=100

[GitHub] spark pull request #20556: [SPARK-23367][Build] Include python document styl...

2018-02-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20556#discussion_r168078916 --- Diff: dev/lint-python --- @@ -21,10 +21,15 @@ SCRIPT_DIR="$( cd "$( dirname "$0" )" && pwd )" SPARK_ROOT_DIR="$(dirname "$SCRIPT_DIR")" #

[GitHub] spark issue #20594: [SPARK-23377][ML] Fixes Bucketizer with multiple columns...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20594 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87440/ Test PASSed. ---

[GitHub] spark issue #20594: [SPARK-23377][ML] Fixes Bucketizer with multiple columns...

2018-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20594 **[Test build #87440 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87440/testReport)** for PR 20594 at commit

[GitHub] spark pull request #20556: [SPARK-23367][Build] Include python document styl...

2018-02-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20556#discussion_r168078994 --- Diff: dev/lint-python --- @@ -21,10 +21,15 @@ SCRIPT_DIR="$( cd "$( dirname "$0" )" && pwd )" SPARK_ROOT_DIR="$(dirname "$SCRIPT_DIR")" #

[GitHub] spark pull request #20424: [Spark-23240][python] Better error message when e...

2018-02-13 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/20424#discussion_r168080320 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala --- @@ -191,7 +192,26 @@ private[spark] class

[GitHub] spark issue #20605: [SPARK-23419][SPARK-23416][SS] data source v2 write path...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20605 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20605: [SPARK-23419][SPARK-23416][SS] data source v2 write path...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20605 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87437/ Test PASSed. ---

[GitHub] spark issue #20605: [SPARK-23419][SPARK-23416][SS] data source v2 write path...

2018-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20605 **[Test build #87437 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87437/testReport)** for PR 20605 at commit

[GitHub] spark pull request #20593: [SPARK-23230][SQL][BRANCH-2.2]When hive.default.f...

2018-02-13 Thread cxzl25
Github user cxzl25 closed the pull request at: https://github.com/apache/spark/pull/20593 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20593: [SPARK-23230][SQL][BRANCH-2.2]When hive.default.fileform...

2018-02-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20593 Thanks! Merged to 2.2. Could you close it? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20593: [SPARK-23230][SQL][BRANCH-2.2]When hive.default.fileform...

2018-02-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20593 Supporting Hadoop 3.0 is being discussed. Now, we do not support it yet. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20424: [Spark-23240][python] Better error message when e...

2018-02-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20424#discussion_r168076901 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala --- @@ -17,7 +17,7 @@ package

[GitHub] spark pull request #20424: [Spark-23240][python] Better error message when e...

2018-02-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20424#discussion_r168075968 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala --- @@ -191,7 +192,26 @@ private[spark] class

[GitHub] spark pull request #20424: [Spark-23240][python] Better error message when e...

2018-02-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20424#discussion_r168075909 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala --- @@ -191,7 +192,26 @@ private[spark] class

[GitHub] spark issue #20424: [Spark-23240][python] Better error message when extraneo...

2018-02-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20424 Thank you @squito .. LGTM otherwise. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20424: [Spark-23240][python] Better error message when e...

2018-02-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20424#discussion_r168075988 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala --- @@ -191,7 +192,26 @@ private[spark] class

[GitHub] spark issue #20594: [SPARK-23377][ML] Fixes Bucketizer with multiple columns...

2018-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20594 **[Test build #87440 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87440/testReport)** for PR 20594 at commit

[GitHub] spark issue #20594: [SPARK-23377][ML] Fixes Bucketizer with multiple columns...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20594 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/888/

[GitHub] spark issue #20594: [SPARK-23377][ML] Fixes Bucketizer with multiple columns...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20594 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20554: [SPARK-23362][SS] Migrate Kafka Microbatch source to v2

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20554 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87436/ Test PASSed. ---

[GitHub] spark issue #20554: [SPARK-23362][SS] Migrate Kafka Microbatch source to v2

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20554 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20554: [SPARK-23362][SS] Migrate Kafka Microbatch source to v2

2018-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20554 **[Test build #87436 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87436/testReport)** for PR 20554 at commit

[GitHub] spark issue #20556: [SPARK-23367][Build] Include python document style check...

2018-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20556 **[Test build #87439 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87439/testReport)** for PR 20556 at commit

[GitHub] spark issue #20556: [SPARK-23367][Build] Include python document style check...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20556 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/887/

[GitHub] spark issue #20556: [SPARK-23367][Build] Include python document style check...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20556 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20555: [SPARK-23366] Improve hot reading path in ReadAheadInput...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20555 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87435/ Test PASSed. ---

[GitHub] spark issue #20556: [SPARK-23367][Build] Include python document style check...

2018-02-13 Thread rekhajoshm
Github user rekhajoshm commented on the issue: https://github.com/apache/spark/pull/20556 build error unrelated to this PR. retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20555: [SPARK-23366] Improve hot reading path in ReadAheadInput...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20555 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20555: [SPARK-23366] Improve hot reading path in ReadAheadInput...

2018-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20555 **[Test build #87435 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87435/testReport)** for PR 20555 at commit

[GitHub] spark issue #20593: [SPARK-23230][SQL][BRANCH-2.2]When hive.default.fileform...

2018-02-13 Thread PandaMonkey
Github user PandaMonkey commented on the issue: https://github.com/apache/spark/pull/20593 @dongjoon-hyun Hi, I have a digression question, the latest version of Hadoop is 3.0.0, why Spark still uses Hadoop 2.6.5? Does spark plan to upgrade Hadoop from 2.6.5 to 3.0.0? I am only a

[GitHub] spark issue #20554: [SPARK-23362][SS] Migrate Kafka Microbatch source to v2

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20554 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87433/ Test PASSed. ---

[GitHub] spark issue #20554: [SPARK-23362][SS] Migrate Kafka Microbatch source to v2

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20554 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20594: [SPARK-23377][ML] Fixes Bucketizer with multiple columns...

2018-02-13 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20594 Because this is a quick fix, my idea is to have a surface patch that doesn't change existing API. The approach of adding parameter to `DefaultParamsWriter.saveMetadata` also sounds good to me, but

[GitHub] spark issue #20554: [SPARK-23362][SS] Migrate Kafka Microbatch source to v2

2018-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20554 **[Test build #87433 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87433/testReport)** for PR 20554 at commit

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20387 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20387 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87434/ Test PASSed. ---

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20387 **[Test build #87434 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87434/testReport)** for PR 20387 at commit

[GitHub] spark pull request #20594: [SPARK-23377][ML] Fixes Bucketizer with multiple ...

2018-02-13 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/20594#discussion_r168069150 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Bucketizer.scala --- @@ -290,6 +293,27 @@ object Bucketizer extends

[GitHub] spark issue #20555: [SPARK-23366] Improve hot reading path in ReadAheadInput...

2018-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20555 **[Test build #87438 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87438/testReport)** for PR 20555 at commit

[GitHub] spark issue #20555: [SPARK-23366] Improve hot reading path in ReadAheadInput...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20555 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/886/

[GitHub] spark issue #20555: [SPARK-23366] Improve hot reading path in ReadAheadInput...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20555 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20599: [SPARK-23407][SQL] add a config to try to inline all mut...

2018-02-13 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20599 @mgaido91 As I said in the PR description, no regression is found so far, just providing a config to be super safe. Actually this PR has a problem: the codegen usually happens at executor

[GitHub] spark pull request #20594: [SPARK-23377][ML] Fixes Bucketizer with multiple ...

2018-02-13 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/20594#discussion_r168067865 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Bucketizer.scala --- @@ -290,6 +293,27 @@ object Bucketizer extends

[GitHub] spark issue #20594: [SPARK-23377][ML] Fixes Bucketizer with multiple columns...

2018-02-13 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/20594 I thought again, instead of "removing default value and restore it again later (which may cause some side effects)", maybe the better way is, add a parameter to

[GitHub] spark pull request #20599: [SPARK-23407][SQL] add a config to try to inline ...

2018-02-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20599#discussion_r168067551 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1461,20 +1465,19 @@ object

[GitHub] spark pull request #20599: [SPARK-23407][SQL] add a config to try to inline ...

2018-02-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20599#discussion_r168067521 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -675,6 +675,16 @@ object SQLConf { "disable logging

[GitHub] spark pull request #20590: [SPARK-23399][SQL] Register a task completion lis...

2018-02-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20590 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20590: [SPARK-23399][SQL] Register a task completion listener f...

2018-02-13 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20590 thanks, merging to master/2.3! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20555: [SPARK-23366] Improve hot reading path in ReadAhe...

2018-02-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20555#discussion_r168066120 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -230,6 +227,7 @@ private void signalAsyncReadComplete() {

[GitHub] spark pull request #20598: [SPARK-23406] [SS] Enable stream-stream self-join...

2018-02-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20598#discussion_r168064805 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingRelation.scala --- @@ -62,7 +64,7 @@ case class

[GitHub] spark issue #20605: [SPARK-23419][SPARK-23416][SS] data source v2 write path...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20605 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20605: [SPARK-23419][SPARK-23416][SS] data source v2 write path...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20605 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/885/

[GitHub] spark issue #20605: [SPARK-23419][SPARK-23416][SS] data source v2 write path...

2018-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20605 **[Test build #87437 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87437/testReport)** for PR 20605 at commit

[GitHub] spark pull request #20605: [SPARK-23419][SPARK-23416][SS] data source v2 wri...

2018-02-13 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/20605 [SPARK-23419][SPARK-23416][SS] data source v2 write path should re-throw interruption exceptions directly ## What changes were proposed in this pull request? Streaming execution has a

[GitHub] spark issue #20605: [SPARK-23419][SPARK-23416][SS] data source v2 write path...

2018-02-13 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20605 cc @zsxwing @jose-torres @mgaido91 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20602: [SPARK-23416][SS] handle streaming interrupts in ...

2018-02-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20602#discussion_r168060518 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -369,7 +370,11 @@ abstract class

[GitHub] spark issue #20604: [WIP][SPARK-23365][CORE] Do not adjust num executors whe...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20604 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87429/ Test PASSed. ---

[GitHub] spark issue #20604: [WIP][SPARK-23365][CORE] Do not adjust num executors whe...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20604 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20589: [SPARK-23394][UI] In RDD storage page show the executor ...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20589 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20589: [SPARK-23394][UI] In RDD storage page show the executor ...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20589 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87430/ Test FAILed. ---

[GitHub] spark issue #20604: [WIP][SPARK-23365][CORE] Do not adjust num executors whe...

2018-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20604 **[Test build #87429 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87429/testReport)** for PR 20604 at commit

[GitHub] spark issue #20589: [SPARK-23394][UI] In RDD storage page show the executor ...

2018-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20589 **[Test build #87430 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87430/testReport)** for PR 20589 at commit

[GitHub] spark pull request #20594: [SPARK-23377][ML] Fixes Bucketizer with multiple ...

2018-02-13 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20594#discussion_r168057330 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Bucketizer.scala --- @@ -290,6 +293,27 @@ object Bucketizer extends

[GitHub] spark issue #20603: [SPARK-23418][SQL]: Fail DataSourceV2 reads when user sc...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20603 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87431/ Test FAILed. ---

[GitHub] spark issue #20596: [SPARK-23404][CORE]When the underlying buffers are direc...

2018-02-13 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20596 Can you please elaborate the case to support your fix here? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20603: [SPARK-23418][SQL]: Fail DataSourceV2 reads when user sc...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20603 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20603: [SPARK-23418][SQL]: Fail DataSourceV2 reads when user sc...

2018-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20603 **[Test build #87431 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87431/testReport)** for PR 20603 at commit

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20387 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87432/ Test FAILed. ---

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20387 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20387 **[Test build #87432 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87432/testReport)** for PR 20387 at commit

[GitHub] spark issue #20554: [SPARK-23362][SS] Migrate Kafka Microbatch source to v2

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20554 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/884/

[GitHub] spark issue #20554: [SPARK-23362][SS] Migrate Kafka Microbatch source to v2

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20554 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20554: [SPARK-23362][SS] Migrate Kafka Microbatch source to v2

2018-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20554 **[Test build #87436 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87436/testReport)** for PR 20554 at commit

[GitHub] spark issue #20555: [SPARK-23366] Improve hot reading path in ReadAheadInput...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20555 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/883/

[GitHub] spark issue #20555: [SPARK-23366] Improve hot reading path in ReadAheadInput...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20555 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20554: [SPARK-23362][SS] Migrate Kafka Microbatch source to v2

2018-02-13 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20554 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20555: [SPARK-23366] Improve hot reading path in ReadAheadInput...

2018-02-13 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20555 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20555: [SPARK-23366] Improve hot reading path in ReadAheadInput...

2018-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20555 **[Test build #87435 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87435/testReport)** for PR 20555 at commit

[GitHub] spark pull request #20555: [SPARK-23366] Improve hot reading path in ReadAhe...

2018-02-13 Thread juliuszsompolski
Github user juliuszsompolski commented on a diff in the pull request: https://github.com/apache/spark/pull/20555#discussion_r168048937 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -78,9 +79,8 @@ // whether there is a read ahead task

[GitHub] spark pull request #20555: [SPARK-23366] Improve hot reading path in ReadAhe...

2018-02-13 Thread juliuszsompolski
Github user juliuszsompolski commented on a diff in the pull request: https://github.com/apache/spark/pull/20555#discussion_r168048795 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -230,24 +227,32 @@ private void signalAsyncReadComplete() {

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.3

2018-02-13 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20373 >Does the hijacking of the namedtuple still cause problems on Python 3.6? I'm not too familiar with the history of this, but I ran PySpark tests that cover namedtuples with 3.6.3 and

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-13 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20387 Okay, I rebased again after SPARK-23303 was reverted. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20387 **[Test build #87434 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87434/testReport)** for PR 20387 at commit

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20387 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/882/

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.3

2018-02-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20373 I think it's fine in cloudpickle but Spark has the hijacking for regular pickling. I was thinking of a possibility for a deduplicated fix but might have to be investigated separately.

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20387 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-13 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20477 Thank you very much @gatorsmile, I promise I will do a proper review of the streaming side when you reopen this PR. --- - To

[GitHub] spark pull request #20474: [SPARK-23235][Core] Add executor Threaddump to ap...

2018-02-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20474 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

  1   2   3   4   5   6   7   >