[GitHub] [spark] HyukjinKwon commented on a change in pull request #26747: [SPARK-29188][PySpark] toPandas gets wrong dtypes when applied on empty DF

2019-12-03 Thread GitBox
HyukjinKwon commented on a change in pull request #26747: [SPARK-29188][PySpark] toPandas gets wrong dtypes when applied on empty DF URL: https://github.com/apache/spark/pull/26747#discussion_r353500751 ## File path: python/pyspark/sql/dataframe.py ## @@ -2305,16 +2305,15

[GitHub] [spark] ifilonenko opened a new pull request #26753: [SPARK-30111] Apt-get update to fix debian issues

2019-12-03 Thread GitBox
ifilonenko opened a new pull request #26753: [SPARK-30111] Apt-get update to fix debian issues URL: https://github.com/apache/spark/pull/26753 ### What changes were proposed in this pull request? Added apt-get update as per [docker best-practices](https://docs.docker.com/develop/develop

[GitHub] [spark] vanzin commented on issue #26502: [SPARK-29876][SS] Delete/archive file source completed files in separate thread

2019-12-03 Thread GitBox
vanzin commented on issue #26502: [SPARK-29876][SS] Delete/archive file source completed files in separate thread URL: https://github.com/apache/spark/pull/26502#issuecomment-561422379 Do the tests need any adjustment given that the cleaning is asynchronous now? (Maybe to disable the clean

[GitHub] [spark] vanzin commented on a change in pull request #26502: [SPARK-29876][SS] Delete/archive file source completed files in separate thread

2019-12-03 Thread GitBox
vanzin commented on a change in pull request #26502: [SPARK-29876][SS] Delete/archive file source completed files in separate thread URL: https://github.com/apache/spark/pull/26502#discussion_r353497386 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/

[GitHub] [spark] vanzin commented on a change in pull request #26502: [SPARK-29876][SS] Delete/archive file source completed files in separate thread

2019-12-03 Thread GitBox
vanzin commented on a change in pull request #26502: [SPARK-29876][SS] Delete/archive file source completed files in separate thread URL: https://github.com/apache/spark/pull/26502#discussion_r353497464 ## File path: docs/structured-streaming-programming-guide.md ## @@ -55

[GitHub] [spark] viirya commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-03 Thread GitBox
viirya commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#issuecomment-561420886 cc @cloud-fan This is an autom

[GitHub] [spark] AmplabJenkins removed a comment on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#issuecomment-561420189 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins removed a comment on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#issuecomment-561420188 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#issuecomment-561420188 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#issuecomment-561420189 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-03 Thread GitBox
SparkQA commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#issuecomment-561419802 **[Test build #114806 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequest

[GitHub] [spark] HyukjinKwon commented on a change in pull request #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
HyukjinKwon commented on a change in pull request #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#discussion_r353494396 ## File path: python/pyspark/sql/tests/test_types.py ## @@ -968,6 +968,19 @@

[GitHub] [spark] viirya commented on a change in pull request #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
viirya commented on a change in pull request #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#discussion_r353473381 ## File path: python/pyspark/sql/types.py ## @@ -1432,10 +1434,20 @@ class Row(tup

[GitHub] [spark] viirya commented on a change in pull request #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
viirya commented on a change in pull request #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#discussion_r353474462 ## File path: python/pyspark/sql/types.py ## @@ -1474,21 +1486,40 @@ class Row(tup

[GitHub] [spark] viirya opened a new pull request #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-03 Thread GitBox
viirya opened a new pull request #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752 ### What changes were proposed in this pull request? This patch proposes to allow insert over

[GitHub] [spark] HeartSaVioR commented on a change in pull request #26258: [SPARK-29604][SQL] Force initialize SessionState before initializing HiveClient in SparkSQLEnv

2019-12-03 Thread GitBox
HeartSaVioR commented on a change in pull request #26258: [SPARK-29604][SQL] Force initialize SessionState before initializing HiveClient in SparkSQLEnv URL: https://github.com/apache/spark/pull/26258#discussion_r353484736 ## File path: sql/hive-thriftserver/src/test/scala/org/apac

[GitHub] [spark] jiangxb1987 commented on issue #26614: [SPARK-29976][CORE] Trigger speculation for stages with too few tasks

2019-12-03 Thread GitBox
jiangxb1987 commented on issue #26614: [SPARK-29976][CORE] Trigger speculation for stages with too few tasks URL: https://github.com/apache/spark/pull/26614#issuecomment-561410074 I think I get your concern now, we might have two stages running concurrently, the expected task duration for

[GitHub] [spark] srowen commented on a change in pull request #26748: [SPARK-29392][CORE][SQL][FOLLOWUP] Avoid deprecated (in 2.13) Symbol syntax 'foo in favor of simpler expression, where it generate

2019-12-03 Thread GitBox
srowen commented on a change in pull request #26748: [SPARK-29392][CORE][SQL][FOLLOWUP] Avoid deprecated (in 2.13) Symbol syntax 'foo in favor of simpler expression, where it generated deprecation warnings URL: https://github.com/apache/spark/pull/26748#discussion_r353482378 ## Fil

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #26748: [SPARK-29392][CORE][SQL][FOLLOWUP] Avoid deprecated (in 2.13) Symbol syntax 'foo in favor of simpler expression, where it g

2019-12-03 Thread GitBox
dongjoon-hyun commented on a change in pull request #26748: [SPARK-29392][CORE][SQL][FOLLOWUP] Avoid deprecated (in 2.13) Symbol syntax 'foo in favor of simpler expression, where it generated deprecation warnings URL: https://github.com/apache/spark/pull/26748#discussion_r353482272

[GitHub] [spark] dongjoon-hyun commented on issue #26748: [SPARK-29392][CORE][SQL][FOLLOWUP] Avoid deprecated (in 2.13) Symbol syntax 'foo in favor of simpler expression, where it generated deprecatio

2019-12-03 Thread GitBox
dongjoon-hyun commented on issue #26748: [SPARK-29392][CORE][SQL][FOLLOWUP] Avoid deprecated (in 2.13) Symbol syntax 'foo in favor of simpler expression, where it generated deprecation warnings URL: https://github.com/apache/spark/pull/26748#issuecomment-561406386 Also, please fix the foll

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #26748: [SPARK-29392][CORE][SQL][FOLLOWUP] Avoid deprecated (in 2.13) Symbol syntax 'foo in favor of simpler expression, where it g

2019-12-03 Thread GitBox
dongjoon-hyun commented on a change in pull request #26748: [SPARK-29392][CORE][SQL][FOLLOWUP] Avoid deprecated (in 2.13) Symbol syntax 'foo in favor of simpler expression, where it generated deprecation warnings URL: https://github.com/apache/spark/pull/26748#discussion_r353478869

[GitHub] [spark] AmplabJenkins commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561396927 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs

[GitHub] [spark] AmplabJenkins removed a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561396924 Merged build finished. Test PASSed. T

[GitHub] [spark] AmplabJenkins removed a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561396927 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://a

[GitHub] [spark] AmplabJenkins commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561396924 Merged build finished. Test PASSed. This is a

[GitHub] [spark] SparkQA removed a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
SparkQA removed a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561387263 **[Test build #114804 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114804

[GitHub] [spark] SparkQA commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
SparkQA commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561396593 **[Test build #114804 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114804/testRe

[GitHub] [spark] AmplabJenkins removed a comment on issue #22198: [SPARK-25121][SQL] Supports multi-part table names for broadcast hint resolution

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #22198: [SPARK-25121][SQL] Supports multi-part table names for broadcast hint resolution URL: https://github.com/apache/spark/pull/22198#issuecomment-561393977 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #22198: [SPARK-25121][SQL] Supports multi-part table names for broadcast hint resolution

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #22198: [SPARK-25121][SQL] Supports multi-part table names for broadcast hint resolution URL: https://github.com/apache/spark/pull/22198#issuecomment-561393971 Build finished. Test FAILed.

[GitHub] [spark] SparkQA removed a comment on issue #22198: [SPARK-25121][SQL] Supports multi-part table names for broadcast hint resolution

2019-12-03 Thread GitBox
SparkQA removed a comment on issue #22198: [SPARK-25121][SQL] Supports multi-part table names for broadcast hint resolution URL: https://github.com/apache/spark/pull/22198#issuecomment-561389783 **[Test build #114805 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestB

[GitHub] [spark] AmplabJenkins commented on issue #22198: [SPARK-25121][SQL] Supports multi-part table names for broadcast hint resolution

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #22198: [SPARK-25121][SQL] Supports multi-part table names for broadcast hint resolution URL: https://github.com/apache/spark/pull/22198#issuecomment-561393977 Test FAILed. Refer to this link for build results (access rights to CI server needed): http

[GitHub] [spark] AmplabJenkins commented on issue #22198: [SPARK-25121][SQL] Supports multi-part table names for broadcast hint resolution

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #22198: [SPARK-25121][SQL] Supports multi-part table names for broadcast hint resolution URL: https://github.com/apache/spark/pull/22198#issuecomment-561393971 Build finished. Test FAILed. Thi

[GitHub] [spark] SparkQA commented on issue #22198: [SPARK-25121][SQL] Supports multi-part table names for broadcast hint resolution

2019-12-03 Thread GitBox
SparkQA commented on issue #22198: [SPARK-25121][SQL] Supports multi-part table names for broadcast hint resolution URL: https://github.com/apache/spark/pull/22198#issuecomment-561393967 **[Test build #114805 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #26751: [SPARK-30107][SQL] Expose nested schema pruning to all V2 sources

2019-12-03 Thread GitBox
dongjoon-hyun commented on a change in pull request #26751: [SPARK-30107][SQL] Expose nested schema pruning to all V2 sources URL: https://github.com/apache/spark/pull/26751#discussion_r353464069 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataS

[GitHub] [spark] dongjoon-hyun commented on issue #26751: [SPARK-30107][SQL] Expose nested schema pruning to all V2 sources

2019-12-03 Thread GitBox
dongjoon-hyun commented on issue #26751: [SPARK-30107][SQL] Expose nested schema pruning to all V2 sources URL: https://github.com/apache/spark/pull/26751#issuecomment-561391894 The 5 failures look relevant. Could you fix the UT failure first? https://user-images.githubusercontent.com/97

[GitHub] [spark] dongjoon-hyun commented on issue #26751: [SPARK-30107][SQL] Expose nested schema pruning to all V2 sources

2019-12-03 Thread GitBox
dongjoon-hyun commented on issue #26751: [SPARK-30107][SQL] Expose nested schema pruning to all V2 sources URL: https://github.com/apache/spark/pull/26751#issuecomment-561391422 Thank you for pinging me, @aokolnychyi ! This i

[GitHub] [spark] SparkQA commented on issue #22198: [SPARK-25121][SQL] Supports multi-part table names for broadcast hint resolution

2019-12-03 Thread GitBox
SparkQA commented on issue #22198: [SPARK-25121][SQL] Supports multi-part table names for broadcast hint resolution URL: https://github.com/apache/spark/pull/22198#issuecomment-561389783 **[Test build #114805 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/1

[GitHub] [spark] dongjoon-hyun closed pull request #26742: [SPARK-30051][BUILD] Clean up hadoop-3.2 dependency

2019-12-03 Thread GitBox
dongjoon-hyun closed pull request #26742: [SPARK-30051][BUILD] Clean up hadoop-3.2 dependency URL: https://github.com/apache/spark/pull/26742 This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [spark] AmplabJenkins removed a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561387663 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://

[GitHub] [spark] AmplabJenkins commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561387663 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.c

[GitHub] [spark] AmplabJenkins commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561387655 Merged build finished. Test PASSed. This is a

[GitHub] [spark] AmplabJenkins removed a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561387655 Merged build finished. Test PASSed. T

[GitHub] [spark] dongjoon-hyun commented on issue #26742: [SPARK-30051][BUILD] Clean up hadoop-3.2 dependency

2019-12-03 Thread GitBox
dongjoon-hyun commented on issue #26742: [SPARK-30051][BUILD] Clean up hadoop-3.2 dependency URL: https://github.com/apache/spark/pull/26742#issuecomment-561387746 Thank you for review. Merged to master. This is an automated

[GitHub] [spark] dongjoon-hyun closed pull request #26692: [SPARK-30060][CORE] Rename metrics enable/disable configs

2019-12-03 Thread GitBox
dongjoon-hyun closed pull request #26692: [SPARK-30060][CORE] Rename metrics enable/disable configs URL: https://github.com/apache/spark/pull/26692 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
SparkQA commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561387263 **[Test build #114804 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114804/testRep

[GitHub] [spark] dongjoon-hyun closed pull request #26744: [SPARK-30106][SQL][TEST] Fix the test of DynamicPartitionPruningSuite

2019-12-03 Thread GitBox
dongjoon-hyun closed pull request #26744: [SPARK-30106][SQL][TEST] Fix the test of DynamicPartitionPruningSuite URL: https://github.com/apache/spark/pull/26744 This is an automated message from the Apache Git Service. To res

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #26744: [SPARK-30106][SQL][MINOR][TEST] Fix the test of DynamicPartitionPruningSuite

2019-12-03 Thread GitBox
dongjoon-hyun commented on a change in pull request #26744: [SPARK-30106][SQL][MINOR][TEST] Fix the test of DynamicPartitionPruningSuite URL: https://github.com/apache/spark/pull/26744#discussion_r353456144 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DynamicPartition

[GitHub] [spark] AmplabJenkins commented on issue #26734: [SPARK-30099][SQL] Improve Analyzed Logical Plan

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26734: [SPARK-30099][SQL] Improve Analyzed Logical Plan URL: https://github.com/apache/spark/pull/26734#issuecomment-561380670 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenki

[GitHub] [spark] AmplabJenkins removed a comment on issue #26734: [SPARK-30099][SQL] Improve Analyzed Logical Plan

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26734: [SPARK-30099][SQL] Improve Analyzed Logical Plan URL: https://github.com/apache/spark/pull/26734#issuecomment-561380664 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] AmplabJenkins removed a comment on issue #26734: [SPARK-30099][SQL] Improve Analyzed Logical Plan

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26734: [SPARK-30099][SQL] Improve Analyzed Logical Plan URL: https://github.com/apache/spark/pull/26734#issuecomment-561380670 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.e

[GitHub] [spark] AmplabJenkins commented on issue #26734: [SPARK-30099][SQL] Improve Analyzed Logical Plan

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26734: [SPARK-30099][SQL] Improve Analyzed Logical Plan URL: https://github.com/apache/spark/pull/26734#issuecomment-561380664 Merged build finished. Test PASSed. This is an automated message

[GitHub] [spark] SparkQA removed a comment on issue #26734: [SPARK-30099][SQL] Improve Analyzed Logical Plan

2019-12-03 Thread GitBox
SparkQA removed a comment on issue #26734: [SPARK-30099][SQL] Improve Analyzed Logical Plan URL: https://github.com/apache/spark/pull/26734#issuecomment-561294287 **[Test build #114800 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114800/testReport)** for

[GitHub] [spark] SparkQA commented on issue #26734: [SPARK-30099][SQL] Improve Analyzed Logical Plan

2019-12-03 Thread GitBox
SparkQA commented on issue #26734: [SPARK-30099][SQL] Improve Analyzed Logical Plan URL: https://github.com/apache/spark/pull/26734#issuecomment-561379983 **[Test build #114800 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114800/testReport)** for PR 267

[GitHub] [spark] tgravescs commented on issue #26696: [WIP][SPARK-18886][CORE] Only reset scheduling delay timer if allocated slots are fully utilized

2019-12-03 Thread GitBox
tgravescs commented on issue #26696: [WIP][SPARK-18886][CORE] Only reset scheduling delay timer if allocated slots are fully utilized URL: https://github.com/apache/spark/pull/26696#issuecomment-561378106 yeah my approach changes it to be slot delay vs task delay, but it also depends on wh

[GitHub] [spark] srowen commented on a change in pull request #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large

2019-12-03 Thread GitBox
srowen commented on a change in pull request #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large URL: https://github.com/apache/spark/pull/26722#discussion_r353439839 ## File path: mllib/src/main/scala/org/apache/spark/mllib/feature/Word2

[GitHub] [spark] jiangxb1987 commented on issue #26714: [SPARK-25100][CORE] Fix no registering TaskCommitMessage bug

2019-12-03 Thread GitBox
jiangxb1987 commented on issue #26714: [SPARK-25100][CORE] Fix no registering TaskCommitMessage bug URL: https://github.com/apache/spark/pull/26714#issuecomment-56136 Do you plan to address https://github.com/apache/spark/pull/22093#issuecomment-412893675 ? --

[GitHub] [spark] AmplabJenkins commented on issue #26080: [SPARK-29425][SQL] The ownership of a database should be respected

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26080: [SPARK-29425][SQL] The ownership of a database should be respected URL: https://github.com/apache/spark/pull/26080#issuecomment-561353876 Merged build finished. Test PASSed. This is an

[GitHub] [spark] AmplabJenkins commented on issue #26080: [SPARK-29425][SQL] The ownership of a database should be respected

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26080: [SPARK-29425][SQL] The ownership of a database should be respected URL: https://github.com/apache/spark/pull/26080#issuecomment-561353890 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.

[GitHub] [spark] AmplabJenkins removed a comment on issue #26080: [SPARK-29425][SQL] The ownership of a database should be respected

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26080: [SPARK-29425][SQL] The ownership of a database should be respected URL: https://github.com/apache/spark/pull/26080#issuecomment-561353890 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://am

[GitHub] [spark] AmplabJenkins removed a comment on issue #26080: [SPARK-29425][SQL] The ownership of a database should be respected

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26080: [SPARK-29425][SQL] The ownership of a database should be respected URL: https://github.com/apache/spark/pull/26080#issuecomment-561353876 Merged build finished. Test PASSed. Th

[GitHub] [spark] SparkQA removed a comment on issue #26080: [SPARK-29425][SQL] The ownership of a database should be respected

2019-12-03 Thread GitBox
SparkQA removed a comment on issue #26080: [SPARK-29425][SQL] The ownership of a database should be respected URL: https://github.com/apache/spark/pull/26080#issuecomment-561254537 **[Test build #114796 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114796/

[GitHub] [spark] SparkQA commented on issue #26080: [SPARK-29425][SQL] The ownership of a database should be respected

2019-12-03 Thread GitBox
SparkQA commented on issue #26080: [SPARK-29425][SQL] The ownership of a database should be respected URL: https://github.com/apache/spark/pull/26080#issuecomment-561353216 **[Test build #114796 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114796/testRep

[GitHub] [spark] AmplabJenkins removed a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561351949 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://a

[GitHub] [spark] AmplabJenkins removed a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561351942 Merged build finished. Test FAILed. T

[GitHub] [spark] AmplabJenkins commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561351942 Merged build finished. Test FAILed. This is a

[GitHub] [spark] AmplabJenkins commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561351949 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs

[GitHub] [spark] SparkQA removed a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
SparkQA removed a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561347549 **[Test build #114803 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114803

[GitHub] [spark] SparkQA commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
SparkQA commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561351865 **[Test build #114803 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114803/testRe

[GitHub] [spark] BryanCutler edited a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
BryanCutler edited a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561349805 Ok, I changed to @HyukjinKwon suggestion of using the env var to control sorting only, and also creation

[GitHub] [spark] BryanCutler commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
BryanCutler commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561349805 Ok, I changed to @HyukjinKwon suggestion of using the env var to control sorting only, and also creation of the

[GitHub] [spark] AmplabJenkins removed a comment on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location URL: https://github.com/apache/spark/pull/26633#issuecomment-561349032 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu

[GitHub] [spark] SparkQA removed a comment on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location

2019-12-03 Thread GitBox
SparkQA removed a comment on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location URL: https://github.com/apache/spark/pull/26633#issuecomment-56130 **[Test build #114802 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114802/testReport)** for P

[GitHub] [spark] AmplabJenkins removed a comment on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location URL: https://github.com/apache/spark/pull/26633#issuecomment-561349028 Merged build finished. Test FAILed. This is an automated m

[GitHub] [spark] SparkQA commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location

2019-12-03 Thread GitBox
SparkQA commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location URL: https://github.com/apache/spark/pull/26633#issuecomment-561349002 **[Test build #114802 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114802/testReport)** for PR 26633

[GitHub] [spark] AmplabJenkins commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location URL: https://github.com/apache/spark/pull/26633#issuecomment-561349032 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins

[GitHub] [spark] AmplabJenkins commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location URL: https://github.com/apache/spark/pull/26633#issuecomment-561349028 Merged build finished. Test FAILed. This is an automated message f

[GitHub] [spark] AmplabJenkins removed a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561347992 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://

[GitHub] [spark] AmplabJenkins removed a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561347983 Merged build finished. Test PASSed. T

[GitHub] [spark] AmplabJenkins commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561347992 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.c

[GitHub] [spark] AmplabJenkins commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561347983 Merged build finished. Test PASSed. This is a

[GitHub] [spark] BryanCutler edited a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
BryanCutler edited a comment on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-553274505 WIP still need to do: - [ ] Add note to migration guide - [ ] Check docs - [x] Fix remaining

[GitHub] [spark] SparkQA commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark

2019-12-03 Thread GitBox
SparkQA commented on issue #26496: [WIP][SPARK-29748][PYTHON][SQL] Remove Row field sorting in PySpark URL: https://github.com/apache/spark/pull/26496#issuecomment-561347549 **[Test build #114803 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114803/testRep

[GitHub] [spark] AmplabJenkins removed a comment on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location URL: https://github.com/apache/spark/pull/26633#issuecomment-561345145 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.ed

[GitHub] [spark] AmplabJenkins removed a comment on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location URL: https://github.com/apache/spark/pull/26633#issuecomment-561345127 Merged build finished. Test PASSed. This is an automated m

[GitHub] [spark] AmplabJenkins commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location URL: https://github.com/apache/spark/pull/26633#issuecomment-561345127 Merged build finished. Test PASSed. This is an automated message f

[GitHub] [spark] AmplabJenkins commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location URL: https://github.com/apache/spark/pull/26633#issuecomment-561345145 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkin

[GitHub] [spark] maryannxue commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location

2019-12-03 Thread GitBox
maryannxue commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location URL: https://github.com/apache/spark/pull/26633#issuecomment-561345400 @tgravescs Our benchmark comparing AQE w/ LSR (local shuffle reader) with AQE w/o LSR showed that before locality wait fix, there were

[GitHub] [spark] SparkQA commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location

2019-12-03 Thread GitBox
SparkQA commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location URL: https://github.com/apache/spark/pull/26633#issuecomment-56130 **[Test build #114802 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114802/testReport)** for PR 26633

[GitHub] [spark] AmplabJenkins removed a comment on issue #26751: [SPARK-30107][SQL] Expose nested schema pruning to all V2 sources

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26751: [SPARK-30107][SQL] Expose nested schema pruning to all V2 sources URL: https://github.com/apache/spark/pull/26751#issuecomment-561340775 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amp

[GitHub] [spark] AmplabJenkins commented on issue #26751: [SPARK-30107][SQL] Expose nested schema pruning to all V2 sources

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26751: [SPARK-30107][SQL] Expose nested schema pruning to all V2 sources URL: https://github.com/apache/spark/pull/26751#issuecomment-561340775 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.b

[GitHub] [spark] AmplabJenkins commented on issue #26751: [SPARK-30107][SQL] Expose nested schema pruning to all V2 sources

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26751: [SPARK-30107][SQL] Expose nested schema pruning to all V2 sources URL: https://github.com/apache/spark/pull/26751#issuecomment-561340767 Merged build finished. Test FAILed. This is an

[GitHub] [spark] AmplabJenkins removed a comment on issue #26751: [SPARK-30107][SQL] Expose nested schema pruning to all V2 sources

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26751: [SPARK-30107][SQL] Expose nested schema pruning to all V2 sources URL: https://github.com/apache/spark/pull/26751#issuecomment-561340767 Merged build finished. Test FAILed. Thi

[GitHub] [spark] SparkQA removed a comment on issue #26751: [SPARK-30107][SQL] Expose nested schema pruning to all V2 sources

2019-12-03 Thread GitBox
SparkQA removed a comment on issue #26751: [SPARK-30107][SQL] Expose nested schema pruning to all V2 sources URL: https://github.com/apache/spark/pull/26751#issuecomment-561291167 **[Test build #114799 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114799/t

[GitHub] [spark] SparkQA commented on issue #26751: [SPARK-30107][SQL] Expose nested schema pruning to all V2 sources

2019-12-03 Thread GitBox
SparkQA commented on issue #26751: [SPARK-30107][SQL] Expose nested schema pruning to all V2 sources URL: https://github.com/apache/spark/pull/26751#issuecomment-561340606 **[Test build #114799 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114799/testRepo

[GitHub] [spark] viirya commented on a change in pull request #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large

2019-12-03 Thread GitBox
viirya commented on a change in pull request #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large URL: https://github.com/apache/spark/pull/26722#discussion_r353396038 ## File path: mllib/src/main/scala/org/apache/spark/mllib/feature/Word2

[GitHub] [spark] AmplabJenkins removed a comment on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large URL: https://github.com/apache/spark/pull/26722#issuecomment-561330778 Merged build finished. Test PASSed. -

[GitHub] [spark] AmplabJenkins removed a comment on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large

2019-12-03 Thread GitBox
AmplabJenkins removed a comment on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large URL: https://github.com/apache/spark/pull/26722#issuecomment-561330783 Test PASSed. Refer to this link for build results (access rights to CI server n

[GitHub] [spark] AmplabJenkins commented on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large URL: https://github.com/apache/spark/pull/26722#issuecomment-561330783 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large

2019-12-03 Thread GitBox
SparkQA removed a comment on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large URL: https://github.com/apache/spark/pull/26722#issuecomment-561303560 **[Test build #114801 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPull

[GitHub] [spark] AmplabJenkins commented on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large

2019-12-03 Thread GitBox
AmplabJenkins commented on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large URL: https://github.com/apache/spark/pull/26722#issuecomment-561330778 Merged build finished. Test PASSed. -

<    1   2   3   4   5   6   7   8   9   10   >