[GitHub] [spark] qudade commented on a change in pull request #26118: [SPARK-24915][PySpark] Fix Row handling with Schema.

2019-10-15 Thread GitBox
qudade commented on a change in pull request #26118: [SPARK-24915][PySpark] Fix Row handling with Schema. URL: https://github.com/apache/spark/pull/26118#discussion_r335145385 ## File path: python/pyspark/sql/types.py ## @@ -599,6 +599,8 @@ def toInternal(self, obj):

[GitHub] [spark] vanzin commented on a change in pull request #22952: [SPARK-20568][SS] Provide option to clean up completed files in streaming query

2019-10-15 Thread GitBox
vanzin commented on a change in pull request #22952: [SPARK-20568][SS] Provide option to clean up completed files in streaming query URL: https://github.com/apache/spark/pull/22952#discussion_r335145149 ## File path:

[GitHub] [spark] MaxGekk commented on issue #26092: [SPARK-29440][SQL] Support java.time.Duration as an external type of CalendarIntervalType

2019-10-15 Thread GitBox
MaxGekk commented on issue #26092: [SPARK-29440][SQL] Support java.time.Duration as an external type of CalendarIntervalType URL: https://github.com/apache/spark/pull/26092#issuecomment-542375895 > I don't think we can map CalendarInterval to Duration as it's kind of a truncation.

[GitHub] [spark] MaxGekk commented on issue #26102: [SPARK-29448][SQL] Support the `INTERVAL` type by Parquet datasource

2019-10-15 Thread GitBox
MaxGekk commented on issue #26102: [SPARK-29448][SQL] Support the `INTERVAL` type by Parquet datasource URL: https://github.com/apache/spark/pull/26102#issuecomment-542374895 In general, are you ok with the proposed changes?

[GitHub] [spark] MaxGekk commented on a change in pull request #26102: [SPARK-29448][SQL] Support the `INTERVAL` type by Parquet datasource

2019-10-15 Thread GitBox
MaxGekk commented on a change in pull request #26102: [SPARK-29448][SQL] Support the `INTERVAL` type by Parquet datasource URL: https://github.com/apache/spark/pull/26102#discussion_r335142560 ## File path:

[GitHub] [spark] MaxGekk commented on issue #25981: [SPARK-28420][SQL] Support the `INTERVAL` type in `date_part()`

2019-10-15 Thread GitBox
MaxGekk commented on issue #25981: [SPARK-28420][SQL] Support the `INTERVAL` type in `date_part()` URL: https://github.com/apache/spark/pull/25981#issuecomment-542373439 What else can I do here? This is an automated message

[GitHub] [spark] ifilonenko commented on issue #26093: [SPARK-27812][K8s] Bump client version

2019-10-15 Thread GitBox
ifilonenko commented on issue #26093: [SPARK-27812][K8s] Bump client version URL: https://github.com/apache/spark/pull/26093#issuecomment-542373452 @erikerlandson I think there should be. This bug will cause pyspark resources to hang unless some external process does resource reaping.

[GitHub] [spark] vanzin closed pull request #25648: [SPARK-28947][K8S] Status logging not happens at an interval for liveness

2019-10-15 Thread GitBox
vanzin closed pull request #25648: [SPARK-28947][K8S] Status logging not happens at an interval for liveness URL: https://github.com/apache/spark/pull/25648 This is an automated message from the Apache Git Service. To

[GitHub] [spark] vanzin commented on issue #25648: [SPARK-28947][K8S] Status logging not happens at an interval for liveness

2019-10-15 Thread GitBox
vanzin commented on issue #25648: [SPARK-28947][K8S] Status logging not happens at an interval for liveness URL: https://github.com/apache/spark/pull/25648#issuecomment-542370054 Merging to master. This is an automated

[GitHub] [spark] AmplabJenkins removed a comment on issue #25863: [SPARK-28945][SPARK-29037][CORE][SQL] Fix the issue that spark gives duplicate result and support concurrent file source write operati

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #25863: [SPARK-28945][SPARK-29037][CORE][SQL] Fix the issue that spark gives duplicate result and support concurrent file source write operations write to different partitions in the same table. URL:

[GitHub] [spark] AmplabJenkins removed a comment on issue #25863: [SPARK-28945][SPARK-29037][CORE][SQL] Fix the issue that spark gives duplicate result and support concurrent file source write operati

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #25863: [SPARK-28945][SPARK-29037][CORE][SQL] Fix the issue that spark gives duplicate result and support concurrent file source write operations write to different partitions in the same table. URL:

[GitHub] [spark] erikerlandson commented on issue #26093: [SPARK-27812][K8s] Bump client version

2019-10-15 Thread GitBox
erikerlandson commented on issue #26093: [SPARK-27812][K8s] Bump client version URL: https://github.com/apache/spark/pull/26093#issuecomment-542365607 > @mccheah @erikerlandson @felixcheung @holdenk we should backport this fix to branch 2.4 as well and cut a new release with this patch.

[GitHub] [spark] AmplabJenkins commented on issue #25863: [SPARK-28945][SPARK-29037][CORE][SQL] Fix the issue that spark gives duplicate result and support concurrent file source write operations writ

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #25863: [SPARK-28945][SPARK-29037][CORE][SQL] Fix the issue that spark gives duplicate result and support concurrent file source write operations write to different partitions in the same table. URL:

[GitHub] [spark] AmplabJenkins commented on issue #25863: [SPARK-28945][SPARK-29037][CORE][SQL] Fix the issue that spark gives duplicate result and support concurrent file source write operations writ

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #25863: [SPARK-28945][SPARK-29037][CORE][SQL] Fix the issue that spark gives duplicate result and support concurrent file source write operations write to different partitions in the same table. URL:

[GitHub] [spark] SparkQA removed a comment on issue #25863: [SPARK-28945][SPARK-29037][CORE][SQL] Fix the issue that spark gives duplicate result and support concurrent file source write operations wr

2019-10-15 Thread GitBox
SparkQA removed a comment on issue #25863: [SPARK-28945][SPARK-29037][CORE][SQL] Fix the issue that spark gives duplicate result and support concurrent file source write operations write to different partitions in the same table. URL:

[GitHub] [spark] SparkQA commented on issue #25863: [SPARK-28945][SPARK-29037][CORE][SQL] Fix the issue that spark gives duplicate result and support concurrent file source write operations write to d

2019-10-15 Thread GitBox
SparkQA commented on issue #25863: [SPARK-28945][SPARK-29037][CORE][SQL] Fix the issue that spark gives duplicate result and support concurrent file source write operations write to different partitions in the same table. URL: https://github.com/apache/spark/pull/25863#issuecomment-542364663

[GitHub] [spark] erikerlandson commented on a change in pull request #25870: [SPARK-27936][K8S] support python deps

2019-10-15 Thread GitBox
erikerlandson commented on a change in pull request #25870: [SPARK-27936][K8S] support python deps URL: https://github.com/apache/spark/pull/25870#discussion_r335131121 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala

[GitHub] [spark] igozali commented on issue #25910: [SPARK-28762][CORE] Read JAR main class if JAR is not located in local file system

2019-10-15 Thread GitBox
igozali commented on issue #25910: [SPARK-28762][CORE] Read JAR main class if JAR is not located in local file system URL: https://github.com/apache/spark/pull/25910#issuecomment-542363360 @srowen @vanzin I've updated my approach, would like to see if this is more along the lines of what

[GitHub] [spark] viirya commented on a change in pull request #26117: [SPARK-29470][BUILD] Update plugins to latest versions

2019-10-15 Thread GitBox
viirya commented on a change in pull request #26117: [SPARK-29470][BUILD] Update plugins to latest versions URL: https://github.com/apache/spark/pull/26117#discussion_r335128757 ## File path: project/plugins.sbt ## @@ -18,7 +18,7 @@ addSbtPlugin("com.etsy" %

[GitHub] [spark] BryanCutler commented on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0

2019-10-15 Thread GitBox
BryanCutler commented on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0 URL: https://github.com/apache/spark/pull/26133#issuecomment-542358825 My vote is to increase the minimum version of PyArrow since it will be the cleanest change, which is done

[GitHub] [spark] AmplabJenkins removed a comment on issue #26044: [SPARK-29375][SQL] Exchange reuse across all subquery levels

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #26044: [SPARK-29375][SQL] Exchange reuse across all subquery levels URL: https://github.com/apache/spark/pull/26044#issuecomment-542355933 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #26044: [SPARK-29375][SQL] Exchange reuse across all subquery levels

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #26044: [SPARK-29375][SQL] Exchange reuse across all subquery levels URL: https://github.com/apache/spark/pull/26044#issuecomment-542355924 Merged build finished. Test PASSed. This is

[GitHub] [spark] AmplabJenkins commented on issue #26044: [SPARK-29375][SQL] Exchange reuse across all subquery levels

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #26044: [SPARK-29375][SQL] Exchange reuse across all subquery levels URL: https://github.com/apache/spark/pull/26044#issuecomment-542355933 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] dongjoon-hyun closed pull request #26117: [SPARK-29470][BUILD] Update plugins to latest versions

2019-10-15 Thread GitBox
dongjoon-hyun closed pull request #26117: [SPARK-29470][BUILD] Update plugins to latest versions URL: https://github.com/apache/spark/pull/26117 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins commented on issue #26044: [SPARK-29375][SQL] Exchange reuse across all subquery levels

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #26044: [SPARK-29375][SQL] Exchange reuse across all subquery levels URL: https://github.com/apache/spark/pull/26044#issuecomment-542355924 Merged build finished. Test PASSed. This is an

[GitHub] [spark] dongjoon-hyun commented on issue #26117: [SPARK-29470][BUILD] Update plugins to latest versions

2019-10-15 Thread GitBox
dongjoon-hyun commented on issue #26117: [SPARK-29470][BUILD] Update plugins to latest versions URL: https://github.com/apache/spark/pull/26117#issuecomment-542355489 Thank you so much for review and approval, @viirya ! Merged to master.

[GitHub] [spark] BryanCutler commented on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0

2019-10-15 Thread GitBox
BryanCutler commented on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0 URL: https://github.com/apache/spark/pull/26133#issuecomment-542355414 I'm marking this as a WIP because we need to decide if we will increase the minimum required version of

[GitHub] [spark] erikerlandson commented on a change in pull request #26093: [SPARK-27812][K8s] Bump client version

2019-10-15 Thread GitBox
erikerlandson commented on a change in pull request #26093: [SPARK-27812][K8s] Bump client version URL: https://github.com/apache/spark/pull/26093#discussion_r335122057 ## File path: resource-managers/kubernetes/core/pom.xml ## @@ -29,7 +29,7 @@ Spark Project

[GitHub] [spark] SparkQA removed a comment on issue #26044: [SPARK-29375][SQL] Exchange reuse across all subquery levels

2019-10-15 Thread GitBox
SparkQA removed a comment on issue #26044: [SPARK-29375][SQL] Exchange reuse across all subquery levels URL: https://github.com/apache/spark/pull/26044#issuecomment-542265664 **[Test build #112112 has

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #26117: [SPARK-29470][BUILD] Update plugins to latest versions

2019-10-15 Thread GitBox
dongjoon-hyun commented on a change in pull request #26117: [SPARK-29470][BUILD] Update plugins to latest versions URL: https://github.com/apache/spark/pull/26117#discussion_r335121950 ## File path: project/plugins.sbt ## @@ -18,7 +18,7 @@ addSbtPlugin("com.etsy" %

[GitHub] [spark] SparkQA commented on issue #26044: [SPARK-29375][SQL] Exchange reuse across all subquery levels

2019-10-15 Thread GitBox
SparkQA commented on issue #26044: [SPARK-29375][SQL] Exchange reuse across all subquery levels URL: https://github.com/apache/spark/pull/26044#issuecomment-542355186 **[Test build #112112 has

[GitHub] [spark] erikerlandson commented on issue #25648: [SPARK-28947][K8S] Status logging not happens at an interval for liveness

2019-10-15 Thread GitBox
erikerlandson commented on issue #25648: [SPARK-28947][K8S] Status logging not happens at an interval for liveness URL: https://github.com/apache/spark/pull/25648#issuecomment-542354128 @yaooqinn @vanzin LGTM This is an

[GitHub] [spark] viirya commented on a change in pull request #26117: [SPARK-29470][BUILD] Update plugins to latest versions

2019-10-15 Thread GitBox
viirya commented on a change in pull request #26117: [SPARK-29470][BUILD] Update plugins to latest versions URL: https://github.com/apache/spark/pull/26117#discussion_r335119451 ## File path: project/plugins.sbt ## @@ -18,7 +18,7 @@ addSbtPlugin("com.etsy" %

[GitHub] [spark] AmplabJenkins removed a comment on issue #26112: [SPARK-29364][SQL] Return an interval from date subtract according to SQL standard

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #26112: [SPARK-29364][SQL] Return an interval from date subtract according to SQL standard URL: https://github.com/apache/spark/pull/26112#issuecomment-542351389 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #26112: [SPARK-29364][SQL] Return an interval from date subtract according to SQL standard

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #26112: [SPARK-29364][SQL] Return an interval from date subtract according to SQL standard URL: https://github.com/apache/spark/pull/26112#issuecomment-542351389 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #26112: [SPARK-29364][SQL] Return an interval from date subtract according to SQL standard

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #26112: [SPARK-29364][SQL] Return an interval from date subtract according to SQL standard URL: https://github.com/apache/spark/pull/26112#issuecomment-542351407 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins commented on issue #26112: [SPARK-29364][SQL] Return an interval from date subtract according to SQL standard

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #26112: [SPARK-29364][SQL] Return an interval from date subtract according to SQL standard URL: https://github.com/apache/spark/pull/26112#issuecomment-542351407 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] HeartSaVioR commented on issue #26127: [SPARK-29348][SQL] Add observable Metrics for Streaming queries

2019-10-15 Thread GitBox
HeartSaVioR commented on issue #26127: [SPARK-29348][SQL] Add observable Metrics for Streaming queries URL: https://github.com/apache/spark/pull/26127#issuecomment-542351523 I like an idea in general. Just a sake of understanding, is this meant to be an alternative of #21721 or we still

[GitHub] [spark] SparkQA commented on issue #26112: [SPARK-29364][SQL] Return an interval from date subtract according to SQL standard

2019-10-15 Thread GitBox
SparkQA commented on issue #26112: [SPARK-29364][SQL] Return an interval from date subtract according to SQL standard URL: https://github.com/apache/spark/pull/26112#issuecomment-542350857 **[Test build #112127 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0 URL: https://github.com/apache/spark/pull/26133#issuecomment-542349690 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0 URL: https://github.com/apache/spark/pull/26133#issuecomment-542349687 Merged build finished. Test FAILed.

[GitHub] [spark] SparkQA removed a comment on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0

2019-10-15 Thread GitBox
SparkQA removed a comment on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0 URL: https://github.com/apache/spark/pull/26133#issuecomment-542347828 **[Test build #112126 has

[GitHub] [spark] AmplabJenkins commented on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0 URL: https://github.com/apache/spark/pull/26133#issuecomment-542349690 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0

2019-10-15 Thread GitBox
SparkQA commented on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0 URL: https://github.com/apache/spark/pull/26133#issuecomment-542349668 **[Test build #112126 has

[GitHub] [spark] AmplabJenkins commented on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0 URL: https://github.com/apache/spark/pull/26133#issuecomment-542349687 Merged build finished. Test FAILed. This

[GitHub] [spark] AmplabJenkins removed a comment on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0 URL: https://github.com/apache/spark/pull/26133#issuecomment-542348344 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0 URL: https://github.com/apache/spark/pull/26133#issuecomment-542348359 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0 URL: https://github.com/apache/spark/pull/26133#issuecomment-542348344 Merged build finished. Test PASSed. This

[GitHub] [spark] AmplabJenkins commented on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0 URL: https://github.com/apache/spark/pull/26133#issuecomment-542348359 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #24851: [SPARK-27303][GRAPH] Add Spark Graph API

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #24851: [SPARK-27303][GRAPH] Add Spark Graph API URL: https://github.com/apache/spark/pull/24851#issuecomment-542347921 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] AmplabJenkins removed a comment on issue #24851: [SPARK-27303][GRAPH] Add Spark Graph API

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #24851: [SPARK-27303][GRAPH] Add Spark Graph API URL: https://github.com/apache/spark/pull/24851#issuecomment-542347926 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24851: [SPARK-27303][GRAPH] Add Spark Graph API

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #24851: [SPARK-27303][GRAPH] Add Spark Graph API URL: https://github.com/apache/spark/pull/24851#issuecomment-542347926 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24851: [SPARK-27303][GRAPH] Add Spark Graph API

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #24851: [SPARK-27303][GRAPH] Add Spark Graph API URL: https://github.com/apache/spark/pull/24851#issuecomment-542347921 Merged build finished. Test PASSed. This is an automated message from

[GitHub] [spark] SparkQA removed a comment on issue #24851: [SPARK-27303][GRAPH] Add Spark Graph API

2019-10-15 Thread GitBox
SparkQA removed a comment on issue #24851: [SPARK-27303][GRAPH] Add Spark Graph API URL: https://github.com/apache/spark/pull/24851#issuecomment-542287300 **[Test build #112117 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/112117/testReport)** for PR

[GitHub] [spark] SparkQA commented on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0

2019-10-15 Thread GitBox
SparkQA commented on issue #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0 URL: https://github.com/apache/spark/pull/26133#issuecomment-542347828 **[Test build #112126 has

[GitHub] [spark] SparkQA commented on issue #24851: [SPARK-27303][GRAPH] Add Spark Graph API

2019-10-15 Thread GitBox
SparkQA commented on issue #24851: [SPARK-27303][GRAPH] Add Spark Graph API URL: https://github.com/apache/spark/pull/24851#issuecomment-542347182 **[Test build #112117 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/112117/testReport)** for PR 24851 at

[GitHub] [spark] BryanCutler opened a new pull request #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0

2019-10-15 Thread GitBox
BryanCutler opened a new pull request #26133: [WIP][SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.0 URL: https://github.com/apache/spark/pull/26133 ### What changes were proposed in this pull request? Upgrade Apache Arrow to version 0.15.0. This includes

[GitHub] [spark] squito commented on a change in pull request #25943: [WIP][SPARK-29261][SQL][CORE] Support recover live entities from KVStore for (SQL)AppStatusListener

2019-10-15 Thread GitBox
squito commented on a change in pull request #25943: [WIP][SPARK-29261][SQL][CORE] Support recover live entities from KVStore for (SQL)AppStatusListener URL: https://github.com/apache/spark/pull/25943#discussion_r335110917 ## File path:

[GitHub] [spark] AmplabJenkins removed a comment on issue #26131: [SPARK-29483][BUILD] Bump Jackson to 2.10.0

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #26131: [SPARK-29483][BUILD] Bump Jackson to 2.10.0 URL: https://github.com/apache/spark/pull/26131#issuecomment-542345314 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #26131: [SPARK-29483][BUILD] Bump Jackson to 2.10.0

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #26131: [SPARK-29483][BUILD] Bump Jackson to 2.10.0 URL: https://github.com/apache/spark/pull/26131#issuecomment-542345306 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] AmplabJenkins commented on issue #26131: [SPARK-29483][BUILD] Bump Jackson to 2.10.0

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #26131: [SPARK-29483][BUILD] Bump Jackson to 2.10.0 URL: https://github.com/apache/spark/pull/26131#issuecomment-542345314 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #26131: [SPARK-29483][BUILD] Bump Jackson to 2.10.0

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #26131: [SPARK-29483][BUILD] Bump Jackson to 2.10.0 URL: https://github.com/apache/spark/pull/26131#issuecomment-542345306 Merged build finished. Test PASSed. This is an automated message

[GitHub] [spark] SparkQA commented on issue #26131: [SPARK-29483][BUILD] Bump Jackson to 2.10.0

2019-10-15 Thread GitBox
SparkQA commented on issue #26131: [SPARK-29483][BUILD] Bump Jackson to 2.10.0 URL: https://github.com/apache/spark/pull/26131#issuecomment-542344831 **[Test build #112125 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/112125/testReport)** for PR 26131 at

[GitHub] [spark] dongjoon-hyun commented on issue #26119: [SPARK-29107][SQL][TESTS] Port window.sql (Part 1)

2019-10-15 Thread GitBox
dongjoon-hyun commented on issue #26119: [SPARK-29107][SQL][TESTS] Port window.sql (Part 1) URL: https://github.com/apache/spark/pull/26119#issuecomment-542343787 According to the failures, we need to update the PR. ``` - Cannot safely cast 'enroll_date': StringType to DateType;

[GitHub] [spark] dongjoon-hyun commented on issue #26131: [SPARK-29483][BUILD] Bump Jackson to 2.10.0

2019-10-15 Thread GitBox
dongjoon-hyun commented on issue #26131: [SPARK-29483][BUILD] Bump Jackson to 2.10.0 URL: https://github.com/apache/spark/pull/26131#issuecomment-542343282 BTW, @Fokko . We use `[BUILD]` instead of `[CORE]` for this kind of PRs~

[GitHub] [spark] dongjoon-hyun commented on issue #26117: [SPARK-29470][BUILD] Update plugins to latest versions

2019-10-15 Thread GitBox
dongjoon-hyun commented on issue #26117: [SPARK-29470][BUILD] Update plugins to latest versions URL: https://github.com/apache/spark/pull/26117#issuecomment-542342887 Hi, @viirya . Could you review this, please? This is an

[GitHub] [spark] dongjoon-hyun commented on issue #26131: [SPARK-29483][CORE] Bump Jackson to 2.10.0

2019-10-15 Thread GitBox
dongjoon-hyun commented on issue #26131: [SPARK-29483][CORE] Bump Jackson to 2.10.0 URL: https://github.com/apache/spark/pull/26131#issuecomment-542342472 Please polish the PR description, @Fokko . Also, cc @srowen since this is `Scala 2.13`-related.

[GitHub] [spark] AmplabJenkins removed a comment on issue #26132: [SPARK-29387][SQL] Support `*` and `/` operators for intervals

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #26132: [SPARK-29387][SQL] Support `*` and `/` operators for intervals URL: https://github.com/apache/spark/pull/26132#issuecomment-542342186 Merged build finished. Test PASSed. This

[GitHub] [spark] AmplabJenkins removed a comment on issue #26132: [SPARK-29387][SQL] Support `*` and `/` operators for intervals

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #26132: [SPARK-29387][SQL] Support `*` and `/` operators for intervals URL: https://github.com/apache/spark/pull/26132#issuecomment-542342193 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #26131: [SPARK-29483][CORE] Bump Jackson to 2.10.0

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #26131: [SPARK-29483][CORE] Bump Jackson to 2.10.0 URL: https://github.com/apache/spark/pull/26131#issuecomment-542339052 Can one of the admins verify this patch? This is an automated

[GitHub] [spark] AmplabJenkins commented on issue #26132: [SPARK-29387][SQL] Support `*` and `/` operators for intervals

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #26132: [SPARK-29387][SQL] Support `*` and `/` operators for intervals URL: https://github.com/apache/spark/pull/26132#issuecomment-542342186 Merged build finished. Test PASSed. This is an

[GitHub] [spark] dongjoon-hyun commented on issue #26131: [SPARK-29483][CORE] Bump Jackson to 2.10.0

2019-10-15 Thread GitBox
dongjoon-hyun commented on issue #26131: [SPARK-29483][CORE] Bump Jackson to 2.10.0 URL: https://github.com/apache/spark/pull/26131#issuecomment-542342202 ok to test This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins commented on issue #26132: [SPARK-29387][SQL] Support `*` and `/` operators for intervals

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #26132: [SPARK-29387][SQL] Support `*` and `/` operators for intervals URL: https://github.com/apache/spark/pull/26132#issuecomment-542342193 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #26093: [SPARK-27812][K8s] Bump client version

2019-10-15 Thread GitBox
dongjoon-hyun commented on a change in pull request #26093: [SPARK-27812][K8s] Bump client version URL: https://github.com/apache/spark/pull/26093#discussion_r335106636 ## File path: resource-managers/kubernetes/core/pom.xml ## @@ -29,7 +29,7 @@ Spark Project

[GitHub] [spark] SparkQA commented on issue #26132: [SPARK-29387][SQL] Support `*` and `/` operators for intervals

2019-10-15 Thread GitBox
SparkQA commented on issue #26132: [SPARK-29387][SQL] Support `*` and `/` operators for intervals URL: https://github.com/apache/spark/pull/26132#issuecomment-542341652 **[Test build #112124 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #26119: [SPARK-29107][SQL][TESTS] Port window.sql (Part 1)

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #26119: [SPARK-29107][SQL][TESTS] Port window.sql (Part 1) URL: https://github.com/apache/spark/pull/26119#issuecomment-542340411 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #25910: [SPARK-28762][CORE] Read JAR main class if JAR is not located in local file system

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #25910: [SPARK-28762][CORE] Read JAR main class if JAR is not located in local file system URL: https://github.com/apache/spark/pull/25910#issuecomment-542340541 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #26119: [SPARK-29107][SQL][TESTS] Port window.sql (Part 1)

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #26119: [SPARK-29107][SQL][TESTS] Port window.sql (Part 1) URL: https://github.com/apache/spark/pull/26119#issuecomment-542340411 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #25910: [SPARK-28762][CORE] Read JAR main class if JAR is not located in local file system

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #25910: [SPARK-28762][CORE] Read JAR main class if JAR is not located in local file system URL: https://github.com/apache/spark/pull/25910#issuecomment-542340541 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #25910: [SPARK-28762][CORE] Read JAR main class if JAR is not located in local file system

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #25910: [SPARK-28762][CORE] Read JAR main class if JAR is not located in local file system URL: https://github.com/apache/spark/pull/25910#issuecomment-542340555 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] SparkQA removed a comment on issue #26119: [SPARK-29107][SQL][TESTS] Port window.sql (Part 1)

2019-10-15 Thread GitBox
SparkQA removed a comment on issue #26119: [SPARK-29107][SQL][TESTS] Port window.sql (Part 1) URL: https://github.com/apache/spark/pull/26119#issuecomment-542329654 **[Test build #112123 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/112123/testReport)**

[GitHub] [spark] AmplabJenkins commented on issue #26119: [SPARK-29107][SQL][TESTS] Port window.sql (Part 1)

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #26119: [SPARK-29107][SQL][TESTS] Port window.sql (Part 1) URL: https://github.com/apache/spark/pull/26119#issuecomment-542340400 Merged build finished. Test FAILed. This is an automated

[GitHub] [spark] AmplabJenkins removed a comment on issue #26119: [SPARK-29107][SQL][TESTS] Port window.sql (Part 1)

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #26119: [SPARK-29107][SQL][TESTS] Port window.sql (Part 1) URL: https://github.com/apache/spark/pull/26119#issuecomment-542340400 Merged build finished. Test FAILed. This is an

[GitHub] [spark] SparkQA commented on issue #26119: [SPARK-29107][SQL][TESTS] Port window.sql (Part 1)

2019-10-15 Thread GitBox
SparkQA commented on issue #26119: [SPARK-29107][SQL][TESTS] Port window.sql (Part 1) URL: https://github.com/apache/spark/pull/26119#issuecomment-542340364 **[Test build #112123 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/112123/testReport)** for PR

[GitHub] [spark] AmplabJenkins commented on issue #25910: [SPARK-28762][CORE] Read JAR main class if JAR is not located in local file system

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #25910: [SPARK-28762][CORE] Read JAR main class if JAR is not located in local file system URL: https://github.com/apache/spark/pull/25910#issuecomment-542340555 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] MaxGekk opened a new pull request #26132: [SPARK-29387][SQL] Support `*` and `\` operators for intervals

2019-10-15 Thread GitBox
MaxGekk opened a new pull request #26132: [SPARK-29387][SQL] Support `*` and `\` operators for intervals URL: https://github.com/apache/spark/pull/26132 ### What changes were proposed in this pull request? Added new expressions `MultiplyInterval` and `DivideInterval` to multiply/divide

[GitHub] [spark] SparkQA removed a comment on issue #25910: [SPARK-28762][CORE] Read JAR main class if JAR is not located in local file system

2019-10-15 Thread GitBox
SparkQA removed a comment on issue #25910: [SPARK-28762][CORE] Read JAR main class if JAR is not located in local file system URL: https://github.com/apache/spark/pull/25910#issuecomment-542276782 **[Test build #112114 has

[GitHub] [spark] SparkQA commented on issue #25910: [SPARK-28762][CORE] Read JAR main class if JAR is not located in local file system

2019-10-15 Thread GitBox
SparkQA commented on issue #25910: [SPARK-28762][CORE] Read JAR main class if JAR is not located in local file system URL: https://github.com/apache/spark/pull/25910#issuecomment-542339608 **[Test build #112114 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #26131: [SPARK-29483][CORE] Bump Jackson to 2.10.0

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #26131: [SPARK-29483][CORE] Bump Jackson to 2.10.0 URL: https://github.com/apache/spark/pull/26131#issuecomment-542338556 Can one of the admins verify this patch? This is an automated

[GitHub] [spark] AmplabJenkins commented on issue #26131: [SPARK-29483][CORE] Bump Jackson to 2.10.0

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #26131: [SPARK-29483][CORE] Bump Jackson to 2.10.0 URL: https://github.com/apache/spark/pull/26131#issuecomment-542339052 Can one of the admins verify this patch? This is an automated message

[GitHub] [spark] AmplabJenkins commented on issue #26131: [SPARK-29483][CORE] Bump Jackson to 2.10.0

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #26131: [SPARK-29483][CORE] Bump Jackson to 2.10.0 URL: https://github.com/apache/spark/pull/26131#issuecomment-542338556 Can one of the admins verify this patch? This is an automated message

[GitHub] [spark] Fokko opened a new pull request #26131: [SPARK-29483][BUILD] Bump Jackson to 2.10.0

2019-10-15 Thread GitBox
Fokko opened a new pull request #26131: [SPARK-29483][BUILD] Bump Jackson to 2.10.0 URL: https://github.com/apache/spark/pull/26131 Fixes the following CVE's: https://www.cvedetails.com/cve/CVE-2019-16942/ https://www.cvedetails.com/cve/CVE-2019-16943/ Release notes:

[GitHub] [spark] BryanCutler commented on issue #26130: [SPARK-29464][PYTHON][ML] PySpark ML should expose Params.clear() to unset a user supplied Param

2019-10-15 Thread GitBox
BryanCutler commented on issue #26130: [SPARK-29464][PYTHON][ML] PySpark ML should expose Params.clear() to unset a user supplied Param URL: https://github.com/apache/spark/pull/26130#issuecomment-542331326 @huaxingao I think there is a problem when the Java object has the param set, then

[GitHub] [spark] AmplabJenkins removed a comment on issue #26121: [SPARK-29108][SQL][TESTS] Port window.sql (Part 2)

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #26121: [SPARK-29108][SQL][TESTS] Port window.sql (Part 2) URL: https://github.com/apache/spark/pull/26121#issuecomment-542330239 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #26121: [SPARK-29108][SQL][TESTS] Port window.sql (Part 2)

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #26121: [SPARK-29108][SQL][TESTS] Port window.sql (Part 2) URL: https://github.com/apache/spark/pull/26121#issuecomment-542330239 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #26121: [SPARK-29108][SQL][TESTS] Port window.sql (Part 2)

2019-10-15 Thread GitBox
AmplabJenkins removed a comment on issue #26121: [SPARK-29108][SQL][TESTS] Port window.sql (Part 2) URL: https://github.com/apache/spark/pull/26121#issuecomment-542330236 Merged build finished. Test PASSed. This is an

[GitHub] [spark] srowen commented on a change in pull request #26088: [SPARK-29436][K8S] Support executor for selecting scheduler through scheduler name in the case of k8s multi-scheduler scenario

2019-10-15 Thread GitBox
srowen commented on a change in pull request #26088: [SPARK-29436][K8S] Support executor for selecting scheduler through scheduler name in the case of k8s multi-scheduler scenario URL: https://github.com/apache/spark/pull/26088#discussion_r335093176 ## File path:

[GitHub] [spark] AmplabJenkins commented on issue #26121: [SPARK-29108][SQL][TESTS] Port window.sql (Part 2)

2019-10-15 Thread GitBox
AmplabJenkins commented on issue #26121: [SPARK-29108][SQL][TESTS] Port window.sql (Part 2) URL: https://github.com/apache/spark/pull/26121#issuecomment-542330236 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #26123: [SPARK-27259][CORE] Allow setting -1 as split size for InputFileBlock

2019-10-15 Thread GitBox
dongjoon-hyun commented on a change in pull request #26123: [SPARK-27259][CORE] Allow setting -1 as split size for InputFileBlock URL: https://github.com/apache/spark/pull/26123#discussion_r335092902 ## File path: core/src/main/scala/org/apache/spark/rdd/InputFileBlockHolder.scala

[GitHub] [spark] SparkQA commented on issue #26121: [SPARK-29108][SQL][TESTS] Port window.sql (Part 2)

2019-10-15 Thread GitBox
SparkQA commented on issue #26121: [SPARK-29108][SQL][TESTS] Port window.sql (Part 2) URL: https://github.com/apache/spark/pull/26121#issuecomment-542329663 **[Test build #112122 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/112122/testReport)** for PR

<    1   2   3   4   5   6   7   8   >