[GitHub] [spark] BryanCutler commented on issue #23900: [SPARK-23836][PYTHON] Add support for StructType return in Scalar Pandas UDF

2019-03-07 Thread GitBox
BryanCutler commented on issue #23900: [SPARK-23836][PYTHON] Add support for StructType return in Scalar Pandas UDF URL: https://github.com/apache/spark/pull/23900#issuecomment-470606701 merged to master This is an automated

[GitHub] [spark] BryanCutler closed pull request #23900: [SPARK-23836][PYTHON] Add support for StructType return in Scalar Pandas UDF

2019-03-07 Thread GitBox
BryanCutler closed pull request #23900: [SPARK-23836][PYTHON] Add support for StructType return in Scalar Pandas UDF URL: https://github.com/apache/spark/pull/23900 This is an automated message from the Apache Git Service. T

[GitHub] [spark] mgaido91 commented on a change in pull request #23942: [SPARK-27033][SQL]Add Optimize rule RewriteArithmeticFiltersOnIntOrLongColumn

2019-03-07 Thread GitBox
mgaido91 commented on a change in pull request #23942: [SPARK-27033][SQL]Add Optimize rule RewriteArithmeticFiltersOnIntOrLongColumn URL: https://github.com/apache/spark/pull/23942#discussion_r263472345 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimiz

[GitHub] [spark] srowen commented on a change in pull request #23986: [SPARK-27070] Fix performance bug in DefaultPartitionCoalescer

2019-03-07 Thread GitBox
srowen commented on a change in pull request #23986: [SPARK-27070] Fix performance bug in DefaultPartitionCoalescer URL: https://github.com/apache/spark/pull/23986#discussion_r263472161 ## File path: core/src/main/scala/org/apache/spark/rdd/CoalescedRDD.scala ## @@ -213,15

[GitHub] [spark] mgaido91 commented on a change in pull request #23942: [SPARK-27033][SQL]Add Optimize rule RewriteArithmeticFiltersOnIntOrLongColumn

2019-03-07 Thread GitBox
mgaido91 commented on a change in pull request #23942: [SPARK-27033][SQL]Add Optimize rule RewriteArithmeticFiltersOnIntOrLongColumn URL: https://github.com/apache/spark/pull/23942#discussion_r263472027 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimiz

[GitHub] [spark] AmplabJenkins removed a comment on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2 URL: https://github.com/apache/spark/pull/24005#issuecomment-470603312 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs

[GitHub] [spark] AmplabJenkins removed a comment on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2 URL: https://github.com/apache/spark/pull/24005#issuecomment-470603298 Merged build finished. Test PASSed. This is an

[GitHub] [spark] AmplabJenkins commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2 URL: https://github.com/apache/spark/pull/24005#issuecomment-470603312 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkele

[GitHub] [spark] AmplabJenkins commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2 URL: https://github.com/apache/spark/pull/24005#issuecomment-470603298 Merged build finished. Test PASSed. This is an automat

[GitHub] [spark] srowen commented on a change in pull request #23954: [SPARK-27041][PySpark] Use imap() for python 2.x to resolve oom issue

2019-03-07 Thread GitBox
srowen commented on a change in pull request #23954: [SPARK-27041][PySpark] Use imap() for python 2.x to resolve oom issue URL: https://github.com/apache/spark/pull/23954#discussion_r263470130 ## File path: python/pyspark/worker.py ## @@ -45,6 +45,8 @@ if sys.version >=

[GitHub] [spark] srowen commented on a change in pull request #23954: [SPARK-27041][PySpark] Use imap() for python 2.x to resolve oom issue

2019-03-07 Thread GitBox
srowen commented on a change in pull request #23954: [SPARK-27041][PySpark] Use imap() for python 2.x to resolve oom issue URL: https://github.com/apache/spark/pull/23954#discussion_r263470130 ## File path: python/pyspark/worker.py ## @@ -45,6 +45,8 @@ if sys.version >=

[GitHub] [spark] squito commented on a change in pull request #23951: [SPARK-27038][CORE][YARN] Re-implement RackResolver to reduce resolving time

2019-03-07 Thread GitBox
squito commented on a change in pull request #23951: [SPARK-27038][CORE][YARN] Re-implement RackResolver to reduce resolving time URL: https://github.com/apache/spark/pull/23951#discussion_r263469410 ## File path: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala

[GitHub] [spark] SparkQA commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2

2019-03-07 Thread GitBox
SparkQA commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2 URL: https://github.com/apache/spark/pull/24005#issuecomment-470601100 **[Test build #103148 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103148/testReport)** f

[GitHub] [spark] dongjoon-hyun commented on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition

2019-03-07 Thread GitBox
dongjoon-hyun commented on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition URL: https://github.com/apache/spark/pull/23964#issuecomment-470600475 Thank you so much for review, @maropu . All comments are valid. I'll address your comments today.

[GitHub] [spark] srowen commented on a change in pull request #23983: [SPARK-26881][core] Heuristic for tree aggregate depth

2019-03-07 Thread GitBox
srowen commented on a change in pull request #23983: [SPARK-26881][core] Heuristic for tree aggregate depth URL: https://github.com/apache/spark/pull/23983#discussion_r263461500 ## File path: mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/RowMatrix.scala ##

[GitHub] [spark] srowen commented on a change in pull request #23983: [SPARK-26881][core] Heuristic for tree aggregate depth

2019-03-07 Thread GitBox
srowen commented on a change in pull request #23983: [SPARK-26881][core] Heuristic for tree aggregate depth URL: https://github.com/apache/spark/pull/23983#discussion_r263461813 ## File path: mllib/src/test/scala/org/apache/spark/mllib/linalg/distributed/RowMatrixSuite.scala #

[GitHub] [spark] srowen commented on a change in pull request #23983: [SPARK-26881][core] Heuristic for tree aggregate depth

2019-03-07 Thread GitBox
srowen commented on a change in pull request #23983: [SPARK-26881][core] Heuristic for tree aggregate depth URL: https://github.com/apache/spark/pull/23983#discussion_r263462819 ## File path: mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/RowMatrix.scala ##

[GitHub] [spark] attilapiros commented on a change in pull request #23986: [SPARK-27070] Fix performance bug in DefaultPartitionCoalescer

2019-03-07 Thread GitBox
attilapiros commented on a change in pull request #23986: [SPARK-27070] Fix performance bug in DefaultPartitionCoalescer URL: https://github.com/apache/spark/pull/23986#discussion_r263460077 ## File path: core/src/main/scala/org/apache/spark/rdd/CoalescedRDD.scala ## @@ -2

[GitHub] [spark] AmplabJenkins removed a comment on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData URL: https://github.com/apache/spark/pull/24011#issuecomment-470594427 Test PASSed. Refer to this link for build results (access rights to CI server needed): htt

[GitHub] [spark] AmplabJenkins commented on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData URL: https://github.com/apache/spark/pull/24011#issuecomment-470594427 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amp

[GitHub] [spark] AmplabJenkins commented on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData URL: https://github.com/apache/spark/pull/24011#issuecomment-470594409 Merged build finished. Test PASSed. This

[GitHub] [spark] AmplabJenkins removed a comment on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData URL: https://github.com/apache/spark/pull/24011#issuecomment-470594409 Merged build finished. Test PASSed. -

[GitHub] [spark] AmplabJenkins removed a comment on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData URL: https://github.com/apache/spark/pull/24011#issuecomment-470591345 Can one of the admins verify this patch?

[GitHub] [spark] AmplabJenkins commented on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData URL: https://github.com/apache/spark/pull/24011#issuecomment-470592160 Can one of the admins verify this patch?

[GitHub] [spark] AmplabJenkins commented on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData URL: https://github.com/apache/spark/pull/24011#issuecomment-470591345 Can one of the admins verify this patch?

[GitHub] [spark] AmplabJenkins removed a comment on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData URL: https://github.com/apache/spark/pull/24011#issuecomment-470591100 Can one of the admins verify this patch?

[GitHub] [spark] hvanhovell commented on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData

2019-03-07 Thread GitBox
hvanhovell commented on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData URL: https://github.com/apache/spark/pull/24011#issuecomment-470591288 ok to test This is an automated message fro

[GitHub] [spark] AmplabJenkins commented on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData URL: https://github.com/apache/spark/pull/24011#issuecomment-470591100 Can one of the admins verify this patch?

[GitHub] [spark] hvanhovell commented on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData

2019-03-07 Thread GitBox
hvanhovell commented on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData URL: https://github.com/apache/spark/pull/24011#issuecomment-470590794 @vanzin can you take a look? This is an aut

[GitHub] [spark] squito commented on a change in pull request #23951: [SPARK-27038][CORE][YARN] Re-implement RackResolver to reduce resolving time

2019-03-07 Thread GitBox
squito commented on a change in pull request #23951: [SPARK-27038][CORE][YARN] Re-implement RackResolver to reduce resolving time URL: https://github.com/apache/spark/pull/23951#discussion_r263455284 ## File path: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.s

[GitHub] [spark] squito commented on a change in pull request #23951: [SPARK-27038][CORE][YARN] Re-implement RackResolver to reduce resolving time

2019-03-07 Thread GitBox
squito commented on a change in pull request #23951: [SPARK-27038][CORE][YARN] Re-implement RackResolver to reduce resolving time URL: https://github.com/apache/spark/pull/23951#discussion_r263450412 ## File path: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala

[GitHub] [spark] squito commented on a change in pull request #23951: [SPARK-27038][CORE][YARN] Re-implement RackResolver to reduce resolving time

2019-03-07 Thread GitBox
squito commented on a change in pull request #23951: [SPARK-27038][CORE][YARN] Re-implement RackResolver to reduce resolving time URL: https://github.com/apache/spark/pull/23951#discussion_r263447342 ## File path: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.s

[GitHub] [spark] tomvanbussel opened a new pull request #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData

2019-03-07 Thread GitBox
tomvanbussel opened a new pull request #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData URL: https://github.com/apache/spark/pull/24011 ## What changes were proposed in this pull request? This PR exposes additional metrics in `status.api.v1.StageData`.

[GitHub] [spark] SparkQA commented on issue #23797: [SPARK-26856][PYSPARK] Python support for from_avro and to_avro APIs

2019-03-07 Thread GitBox
SparkQA commented on issue #23797: [SPARK-26856][PYSPARK] Python support for from_avro and to_avro APIs URL: https://github.com/apache/spark/pull/23797#issuecomment-470586487 **[Test build #103147 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103147/testRe

[GitHub] [spark] gaborgsomogyi commented on a change in pull request #23797: [SPARK-26856][PYSPARK] Python support for from_avro and to_avro APIs

2019-03-07 Thread GitBox
gaborgsomogyi commented on a change in pull request #23797: [SPARK-26856][PYSPARK] Python support for from_avro and to_avro APIs URL: https://github.com/apache/spark/pull/23797#discussion_r263450416 ## File path: dev/sparktestsupport/modules.py ## @@ -178,6 +178,10 @@ def

[GitHub] [spark] WangGuangxin commented on a change in pull request #23942: [SPARK-27033][SQL]Add Optimize rule RewriteArithmeticFiltersOnIntOrLongColumn

2019-03-07 Thread GitBox
WangGuangxin commented on a change in pull request #23942: [SPARK-27033][SQL]Add Optimize rule RewriteArithmeticFiltersOnIntOrLongColumn URL: https://github.com/apache/spark/pull/23942#discussion_r263449619 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/opt

[GitHub] [spark] AmplabJenkins removed a comment on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery URL: https://github.com/apache/spark/pull/23783#issuecomment-470582880 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/j

[GitHub] [spark] gaborgsomogyi commented on a change in pull request #23797: [SPARK-26856][PYSPARK] Python support for from_avro and to_avro APIs

2019-03-07 Thread GitBox
gaborgsomogyi commented on a change in pull request #23797: [SPARK-26856][PYSPARK] Python support for from_avro and to_avro APIs URL: https://github.com/apache/spark/pull/23797#discussion_r263449003 ## File path: dev/sparktestsupport/modules.py ## @@ -178,6 +178,10 @@ def

[GitHub] [spark] AmplabJenkins removed a comment on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery URL: https://github.com/apache/spark/pull/23783#issuecomment-470582871 Merged build finished. Test FAILed. This is an automated mes

[GitHub] [spark] SparkQA removed a comment on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery

2019-03-07 Thread GitBox
SparkQA removed a comment on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery URL: https://github.com/apache/spark/pull/23783#issuecomment-470534488 **[Test build #103138 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103138/testReport)** for PR

[GitHub] [spark] AmplabJenkins commented on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery URL: https://github.com/apache/spark/pull/23783#issuecomment-470582871 Merged build finished. Test FAILed. This is an automated message fro

[GitHub] [spark] AmplabJenkins commented on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery URL: https://github.com/apache/spark/pull/23783#issuecomment-470582880 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//

[GitHub] [spark] SparkQA commented on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery

2019-03-07 Thread GitBox
SparkQA commented on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery URL: https://github.com/apache/spark/pull/23783#issuecomment-470582588 **[Test build #103138 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103138/testReport)** for PR 23783 at

[GitHub] [spark] WangGuangxin commented on a change in pull request #23942: [SPARK-27033][SQL]Add Optimize rule RewriteArithmeticFiltersOnIntOrLongColumn

2019-03-07 Thread GitBox
WangGuangxin commented on a change in pull request #23942: [SPARK-27033][SQL]Add Optimize rule RewriteArithmeticFiltersOnIntOrLongColumn URL: https://github.com/apache/spark/pull/23942#discussion_r263195273 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/opt

[GitHub] [spark] AmplabJenkins removed a comment on issue #23004: [SPARK-26004][SQL] InMemoryTable support StartsWith predicate push down

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #23004: [SPARK-26004][SQL] InMemoryTable support StartsWith predicate push down URL: https://github.com/apache/spark/pull/23004#issuecomment-470579601 Test PASSed. Refer to this link for build results (access rights to CI server needed): http

[GitHub] [spark] AmplabJenkins removed a comment on issue #23004: [SPARK-26004][SQL] InMemoryTable support StartsWith predicate push down

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #23004: [SPARK-26004][SQL] InMemoryTable support StartsWith predicate push down URL: https://github.com/apache/spark/pull/23004#issuecomment-470579592 Merged build finished. Test PASSed. --

[GitHub] [spark] abellina commented on issue #23521: [SPARK-26604][CORE] Clean up channel registration for StreamManager

2019-03-07 Thread GitBox
abellina commented on issue #23521: [SPARK-26604][CORE] Clean up channel registration for StreamManager URL: https://github.com/apache/spark/pull/23521#issuecomment-470580968 I am taking a look, but it may not be until later today that I'd have a PR up. It's mostly clean but the 2.4 branch

[GitHub] [spark] gaborgsomogyi edited a comment on issue #23991: [SPARK-24621][WEBUI] Show secure URLs on web pages

2019-03-07 Thread GitBox
gaborgsomogyi edited a comment on issue #23991: [SPARK-24621][WEBUI] Show secure URLs on web pages URL: https://github.com/apache/spark/pull/23991#issuecomment-470579418 Self-signed certificate is the fake but that's also not available. It can be either: * Checked in as a binary which i

[GitHub] [spark] attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-07 Thread GitBox
attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#discussion_r263444270 ## File path: core/src/test/scala/org/apache/spark/deploy/worker/WorkerSuite.scala ## @@ -243,4

[GitHub] [spark] gaborgsomogyi edited a comment on issue #23991: [SPARK-24621][WEBUI] Show secure URLs on web pages

2019-03-07 Thread GitBox
gaborgsomogyi edited a comment on issue #23991: [SPARK-24621][WEBUI] Show secure URLs on web pages URL: https://github.com/apache/spark/pull/23991#issuecomment-470579418 Self-signed certificate the is fake but that's also not available. It can be either: * Checked in as a binary which i

[GitHub] [spark] AmplabJenkins commented on issue #23004: [SPARK-26004][SQL] InMemoryTable support StartsWith predicate push down

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #23004: [SPARK-26004][SQL] InMemoryTable support StartsWith predicate push down URL: https://github.com/apache/spark/pull/23004#issuecomment-470579592 Merged build finished. Test PASSed. This

[GitHub] [spark] AmplabJenkins commented on issue #23004: [SPARK-26004][SQL] InMemoryTable support StartsWith predicate push down

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #23004: [SPARK-26004][SQL] InMemoryTable support StartsWith predicate push down URL: https://github.com/apache/spark/pull/23004#issuecomment-470579601 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://ampl

[GitHub] [spark] gaborgsomogyi commented on issue #23991: [SPARK-24621][WEBUI] Show secure URLs on web pages

2019-03-07 Thread GitBox
gaborgsomogyi commented on issue #23991: [SPARK-24621][WEBUI] Show secure URLs on web pages URL: https://github.com/apache/spark/pull/23991#issuecomment-470579418 Self-signed certificate the is fake but that's also not available. I can be either: * Checked in as a binary which is wrong

[GitHub] [spark] attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-07 Thread GitBox
attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#discussion_r263442459 ## File path: core/src/test/scala/org/apache/spark/deploy/worker/WorkerSuite.scala ## @@ -243,4

[GitHub] [spark] squito commented on a change in pull request #23951: [SPARK-27038][CORE][YARN] Re-implement RackResolver to reduce resolving time

2019-03-07 Thread GitBox
squito commented on a change in pull request #23951: [SPARK-27038][CORE][YARN] Re-implement RackResolver to reduce resolving time URL: https://github.com/apache/spark/pull/23951#discussion_r263442029 ## File path: resource-managers/yarn/src/main/scala/org/apache/spark/scheduler/clu

[GitHub] [spark] SparkQA commented on issue #23004: [SPARK-26004][SQL] InMemoryTable support StartsWith predicate push down

2019-03-07 Thread GitBox
SparkQA commented on issue #23004: [SPARK-26004][SQL] InMemoryTable support StartsWith predicate push down URL: https://github.com/apache/spark/pull/23004#issuecomment-470577254 **[Test build #103146 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103146/tes

[GitHub] [spark] wangyum commented on a change in pull request #23004: [SPARK-26004][SQL] InMemoryTable support StartsWith predicate push down

2019-03-07 Thread GitBox
wangyum commented on a change in pull request #23004: [SPARK-26004][SQL] InMemoryTable support StartsWith predicate push down URL: https://github.com/apache/spark/pull/23004#discussion_r263439936 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemory

[GitHub] [spark] squito commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-07 Thread GitBox
squito commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#discussion_r263430434 ## File path: core/src/test/scala/org/apache/spark/deploy/worker/WorkerSuite.scala ## @@ -243,4 +244,

[GitHub] [spark] squito commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-07 Thread GitBox
squito commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#discussion_r263429105 ## File path: core/src/test/scala/org/apache/spark/deploy/worker/WorkerSuite.scala ## @@ -243,4 +244,

[GitHub] [spark] squito commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-07 Thread GitBox
squito commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#discussion_r263431416 ## File path: core/src/test/scala/org/apache/spark/deploy/worker/WorkerSuite.scala ## @@ -243,4 +244,

[GitHub] [spark] squito commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-07 Thread GitBox
squito commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#discussion_r263434370 ## File path: core/src/test/scala/org/apache/spark/deploy/worker/WorkerSuite.scala ## @@ -243,4 +244,

[GitHub] [spark] squito commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-07 Thread GitBox
squito commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#discussion_r263431651 ## File path: core/src/test/scala/org/apache/spark/deploy/worker/WorkerSuite.scala ## @@ -243,4 +244,

[GitHub] [spark] attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-07 Thread GitBox
attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#discussion_r263413655 ## File path: core/src/test/scala/org/apache/spark/deploy/ExternalShuffleServiceDbSuite.scala ##

[GitHub] [spark] attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-07 Thread GitBox
attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#discussion_r263433683 ## File path: core/src/test/scala/org/apache/spark/deploy/worker/WorkerSuite.scala ## @@ -243,4

[GitHub] [spark] AmplabJenkins removed a comment on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet URL: https://github.com/apache/spark/pull/24010#issuecomment-470570186 Test PASSed. Refer to this link for build results (access rights to CI server n

[GitHub] [spark] AmplabJenkins removed a comment on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet URL: https://github.com/apache/spark/pull/24010#issuecomment-470570175 Merged build finished. Test PASSed. -

[GitHub] [spark] AmplabJenkins commented on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet URL: https://github.com/apache/spark/pull/24010#issuecomment-470570175 Merged build finished. Test PASSed. -

[GitHub] [spark] AmplabJenkins commented on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet URL: https://github.com/apache/spark/pull/24010#issuecomment-470570186 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet URL: https://github.com/apache/spark/pull/24010#issuecomment-470569758 Can one of the admins verify this patch?

[GitHub] [spark] AmplabJenkins removed a comment on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2 URL: https://github.com/apache/spark/pull/24005#issuecomment-470566246 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.

[GitHub] [spark] AmplabJenkins removed a comment on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2 URL: https://github.com/apache/spark/pull/24005#issuecomment-470566235 Merged build finished. Test FAILed. This is an

[GitHub] [spark] SparkQA commented on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet

2019-03-07 Thread GitBox
SparkQA commented on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet URL: https://github.com/apache/spark/pull/24010#issuecomment-470567943 **[Test build #103145 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestB

[GitHub] [spark] Ngone51 commented on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet

2019-03-07 Thread GitBox
Ngone51 commented on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet URL: https://github.com/apache/spark/pull/24010#issuecomment-470567602 @jiangxb1987 @cloud-fan any thoughts on this ? ---

[GitHub] [spark] Ngone51 opened a new pull request #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet

2019-03-07 Thread GitBox
Ngone51 opened a new pull request #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet URL: https://github.com/apache/spark/pull/24010 ## What changes were proposed in this pull request? Currently, Barrier TaskSet has a hard requirement tha

[GitHub] [spark] Ngone51 commented on issue #24007: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions

2019-03-07 Thread GitBox
Ngone51 commented on issue #24007: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/24007#issuecomment-470567141 tests from 2.4 -

[GitHub] [spark] AmplabJenkins commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2 URL: https://github.com/apache/spark/pull/24005#issuecomment-470566235 Merged build finished. Test FAILed. This is an automat

[GitHub] [spark] SparkQA removed a comment on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2

2019-03-07 Thread GitBox
SparkQA removed a comment on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2 URL: https://github.com/apache/spark/pull/24005#issuecomment-470522175 **[Test build #103136 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103136/testRepo

[GitHub] [spark] AmplabJenkins commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2 URL: https://github.com/apache/spark/pull/24005#issuecomment-470566246 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley

[GitHub] [spark] SparkQA commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2

2019-03-07 Thread GitBox
SparkQA commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2 URL: https://github.com/apache/spark/pull/24005#issuecomment-470565972 **[Test build #103136 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103136/testReport)**

[GitHub] [spark] Ngone51 closed pull request #23377: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet

2019-03-07 Thread GitBox
Ngone51 closed pull request #23377: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet URL: https://github.com/apache/spark/pull/23377 This is an automated message from the Apache Git Se

[GitHub] [spark] squito commented on issue #24007: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions

2019-03-07 Thread GitBox
squito commented on issue #24007: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/24007#issuecomment-470564954 just curious, what was the conflict? I'm wondering if there is some other

[GitHub] [spark] AmplabJenkins removed a comment on issue #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotation

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotations URL: https://github.com/apache/spark/pull/24009#issuecomment-470563820 Can one of the admins veri

[GitHub] [spark] AmplabJenkins removed a comment on issue #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotation

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotations URL: https://github.com/apache/spark/pull/24009#issuecomment-470561751 Can one of the admins veri

[GitHub] [spark] AmplabJenkins commented on issue #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotations

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotations URL: https://github.com/apache/spark/pull/24009#issuecomment-470564064 Can one of the admins verify this

[GitHub] [spark] AmplabJenkins commented on issue #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotations

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotations URL: https://github.com/apache/spark/pull/24009#issuecomment-470563820 Can one of the admins verify this

[GitHub] [spark] SparkQA commented on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery

2019-03-07 Thread GitBox
SparkQA commented on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery URL: https://github.com/apache/spark/pull/23783#issuecomment-470561947 **[Test build #103144 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103144/testReport)** for PR 23783 at

[GitHub] [spark] AmplabJenkins commented on issue #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotations

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotations URL: https://github.com/apache/spark/pull/24009#issuecomment-470561751 Can one of the admins verify this

[GitHub] [spark] hehuiyuan opened a new pull request #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotations

2019-03-07 Thread GitBox
hehuiyuan opened a new pull request #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotations URL: https://github.com/apache/spark/pull/24009 ## What changes were proposed in this pull request?

[GitHub] [spark] AmplabJenkins removed a comment on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery URL: https://github.com/apache/spark/pull/23783#issuecomment-470561001 Merged build finished. Test PASSed. This is an automated mes

[GitHub] [spark] AmplabJenkins removed a comment on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery URL: https://github.com/apache/spark/pull/23783#issuecomment-470561018 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/

[GitHub] [spark] AmplabJenkins commented on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery URL: https://github.com/apache/spark/pull/23783#issuecomment-470561018 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/

[GitHub] [spark] AmplabJenkins commented on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery URL: https://github.com/apache/spark/pull/23783#issuecomment-470561001 Merged build finished. Test PASSed. This is an automated message fro

[GitHub] [spark] jashgala commented on issue #23748: [SPARK-23619][SQL] Added default column names for explode & posexplode in comments

2019-03-07 Thread GitBox
jashgala commented on issue #23748: [SPARK-23619][SQL] Added default column names for explode & posexplode in comments URL: https://github.com/apache/spark/pull/23748#issuecomment-470560739 So far I've addressed the following (also updated in PR description): * stack * inline * exp

[GitHub] [spark] francis0407 commented on a change in pull request #23783: [SPARK-26854][SQL] Support ANY/SOME subquery

2019-03-07 Thread GitBox
francis0407 commented on a change in pull request #23783: [SPARK-26854][SQL] Support ANY/SOME subquery URL: https://github.com/apache/spark/pull/23783#discussion_r263420600 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala #

[GitHub] [spark] AmplabJenkins removed a comment on issue #23067: [SPARK-26097][Web UI] Add the new partitioning description to the exchange node name

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #23067: [SPARK-26097][Web UI] Add the new partitioning description to the exchange node name URL: https://github.com/apache/spark/pull/23067#issuecomment-439589244 Can one of the admins verify this patch?

[GitHub] [spark] AmplabJenkins commented on issue #23067: [SPARK-26097][Web UI] Add the new partitioning description to the exchange node name

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #23067: [SPARK-26097][Web UI] Add the new partitioning description to the exchange node name URL: https://github.com/apache/spark/pull/23067#issuecomment-470558164 Can one of the admins verify this patch?

[GitHub] [spark] AmplabJenkins removed a comment on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception URL: https://github.com/apache/spark/pull/24008#issuecomment-470557343 Merged build finished. Test PASSed. -

[GitHub] [spark] AmplabJenkins removed a comment on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception URL: https://github.com/apache/spark/pull/24008#issuecomment-470557352 Test PASSed. Refer to this link for build results (access rights to CI server needed): htt

[GitHub] [spark] srowen commented on a change in pull request #23958: [SPARK-27045] SQL tab in UI shows actual SQL instead of callsite

2019-03-07 Thread GitBox
srowen commented on a change in pull request #23958: [SPARK-27045] SQL tab in UI shows actual SQL instead of callsite URL: https://github.com/apache/spark/pull/23958#discussion_r263418543 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SQLExecution.scala #

[GitHub] [spark] maropu commented on a change in pull request #23942: [SPARK-27033][SQL]Add Optimize rule RewriteArithmeticFiltersOnIntOrLongColumn

2019-03-07 Thread GitBox
maropu commented on a change in pull request #23942: [SPARK-27033][SQL]Add Optimize rule RewriteArithmeticFiltersOnIntOrLongColumn URL: https://github.com/apache/spark/pull/23942#discussion_r263376060 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer

<    4   5   6   7   8   9   10   11   12   13   >