[GitHub] peter-toth commented on a change in pull request #23531: [SPARK-24497][SQL] Support recursive SQL query

2019-01-24 Thread GitBox
peter-toth commented on a change in pull request #23531: [SPARK-24497][SQL] Support recursive SQL query URL: https://github.com/apache/spark/pull/23531#discussion_r249968739 ## File path: sql/core/src/test/resources/sql-tests/inputs/recursion.sql ## @@ -0,0 +1,291 @@ +-- L

[GitHub] gatorsmile commented on issue #23645: [SPARK-26725][TEST] Fix the input values of UnifiedMemoryManager constructor in test suites

2019-01-24 Thread GitBox
gatorsmile commented on issue #23645: [SPARK-26725][TEST] Fix the input values of UnifiedMemoryManager constructor in test suites URL: https://github.com/apache/spark/pull/23645#issuecomment-457485737 Also cc @cloud-fan @zsxwing Could you take a look? --

[GitHub] gatorsmile commented on issue #23645: [SPARK-26725][TEST] Fix the input values of UnifiedMemoryManager constructor in test suites

2019-01-24 Thread GitBox
gatorsmile commented on issue #23645: [SPARK-26725][TEST] Fix the input values of UnifiedMemoryManager constructor in test suites URL: https://github.com/apache/spark/pull/23645#issuecomment-457485616 It sounds like there are more : https://github.com/apache/spark/pull/23457/files#r2508912

[GitHub] gatorsmile commented on a change in pull request #23457: [SPARK-26539][CORE] Remove spark.memory.useLegacyMode and StaticMemoryManager

2019-01-24 Thread GitBox
gatorsmile commented on a change in pull request #23457: [SPARK-26539][CORE] Remove spark.memory.useLegacyMode and StaticMemoryManager URL: https://github.com/apache/spark/pull/23457#discussion_r250891226 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/Ha

[GitHub] xubo245 commented on a change in pull request #23633: [SPARK-26710][TEST] Fix some errors in image test case

2019-01-24 Thread GitBox
xubo245 commented on a change in pull request #23633: [SPARK-26710][TEST] Fix some errors in image test case URL: https://github.com/apache/spark/pull/23633#discussion_r250891254 ## File path: mllib/src/test/scala/org/apache/spark/ml/image/ImageSchemaSuite.scala ## @@ -17

[GitHub] HyukjinKwon commented on a change in pull request #23641: [SPARK-26719][SQL] Get rid of java.util.Calendar in DateTimeUtils

2019-01-24 Thread GitBox
HyukjinKwon commented on a change in pull request #23641: [SPARK-26719][SQL] Get rid of java.util.Calendar in DateTimeUtils URL: https://github.com/apache/spark/pull/23641#discussion_r250889125 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeTes

[GitHub] MaxGekk commented on a change in pull request #23641: [SPARK-26719][SQL] Get rid of java.util.Calendar in DateTimeUtils

2019-01-24 Thread GitBox
MaxGekk commented on a change in pull request #23641: [SPARK-26719][SQL] Get rid of java.util.Calendar in DateTimeUtils URL: https://github.com/apache/spark/pull/23641#discussion_r250888103 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeTestUti

[GitHub] MaxGekk commented on a change in pull request #23641: [SPARK-26719][SQL] Get rid of java.util.Calendar in DateTimeUtils

2019-01-24 Thread GitBox
MaxGekk commented on a change in pull request #23641: [SPARK-26719][SQL] Get rid of java.util.Calendar in DateTimeUtils URL: https://github.com/apache/spark/pull/23641#discussion_r250888103 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeTestUti

[GitHub] gatorsmile commented on issue #23622: [SPARK-26677][SQL] Disable dictionary filtering by default at Parquet

2019-01-24 Thread GitBox
gatorsmile commented on issue #23622: [SPARK-26677][SQL] Disable dictionary filtering by default at Parquet URL: https://github.com/apache/spark/pull/23622#issuecomment-457480947 We will not upgrade the major version of Parquet in 2.4.1 version. If Parquet has 0.10.1 with various bug fixes

[GitHub] gatorsmile edited a comment on issue #23622: [SPARK-26677][SQL] Disable dictionary filtering by default at Parquet

2019-01-24 Thread GitBox
gatorsmile edited a comment on issue #23622: [SPARK-26677][SQL] Disable dictionary filtering by default at Parquet URL: https://github.com/apache/spark/pull/23622#issuecomment-457480947 We will not upgrade the major version of Parquet in 2.4.1. If Parquet release 0.10.1 with various bug fi

[GitHub] gatorsmile edited a comment on issue #23622: [SPARK-26677][SQL] Disable dictionary filtering by default at Parquet

2019-01-24 Thread GitBox
gatorsmile edited a comment on issue #23622: [SPARK-26677][SQL] Disable dictionary filtering by default at Parquet URL: https://github.com/apache/spark/pull/23622#issuecomment-457480947 We will not upgrade the major version of Parquet in 2.4.1. If Parquet release 0.10.1 with various bug fi

[GitHub] HyukjinKwon commented on a change in pull request #23641: [SPARK-26719][SQL] Get rid of java.util.Calendar in DateTimeUtils

2019-01-24 Thread GitBox
HyukjinKwon commented on a change in pull request #23641: [SPARK-26719][SQL] Get rid of java.util.Calendar in DateTimeUtils URL: https://github.com/apache/spark/pull/23641#discussion_r250886526 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeTes

[GitHub] gengliangwang commented on a change in pull request #23639: [SPARK-26716][SQL] FileFormat: the supported types of read/write should be consistent

2019-01-24 Thread GitBox
gengliangwang commented on a change in pull request #23639: [SPARK-26716][SQL] FileFormat: the supported types of read/write should be consistent URL: https://github.com/apache/spark/pull/23639#discussion_r250886350 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/cat

[GitHub] AmplabJenkins removed a comment on issue #23648: [BRANCH-2.3][SPARK-26709][SQL] OptimizeMetadataOnlyQuery does not handle empty records correctly

2019-01-24 Thread GitBox
AmplabJenkins removed a comment on issue #23648: [BRANCH-2.3][SPARK-26709][SQL] OptimizeMetadataOnlyQuery does not handle empty records correctly URL: https://github.com/apache/spark/pull/23648#issuecomment-457477553 Test PASSed. Refer to this link for build results (access rights to CI

[GitHub] AmplabJenkins commented on issue #23648: [BRANCH-2.3][SPARK-26709][SQL] OptimizeMetadataOnlyQuery does not handle empty records correctly

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #23648: [BRANCH-2.3][SPARK-26709][SQL] OptimizeMetadataOnlyQuery does not handle empty records correctly URL: https://github.com/apache/spark/pull/23648#issuecomment-457477553 Test PASSed. Refer to this link for build results (access rights to CI server n

[GitHub] AmplabJenkins removed a comment on issue #23648: [BRANCH-2.3][SPARK-26709][SQL] OptimizeMetadataOnlyQuery does not handle empty records correctly

2019-01-24 Thread GitBox
AmplabJenkins removed a comment on issue #23648: [BRANCH-2.3][SPARK-26709][SQL] OptimizeMetadataOnlyQuery does not handle empty records correctly URL: https://github.com/apache/spark/pull/23648#issuecomment-457477548 Merged build finished. Test PASSed. -

[GitHub] AmplabJenkins commented on issue #23648: [BRANCH-2.3][SPARK-26709][SQL] OptimizeMetadataOnlyQuery does not handle empty records correctly

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #23648: [BRANCH-2.3][SPARK-26709][SQL] OptimizeMetadataOnlyQuery does not handle empty records correctly URL: https://github.com/apache/spark/pull/23648#issuecomment-457477548 Merged build finished. Test PASSed. -

[GitHub] SparkQA commented on issue #23648: [BRANCH-2.3][SPARK-26709][SQL] OptimizeMetadataOnlyQuery does not handle empty records correctly

2019-01-24 Thread GitBox
SparkQA commented on issue #23648: [BRANCH-2.3][SPARK-26709][SQL] OptimizeMetadataOnlyQuery does not handle empty records correctly URL: https://github.com/apache/spark/pull/23648#issuecomment-457477237 **[Test build #101667 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPull

[GitHub] HyukjinKwon commented on a change in pull request #23622: [SPARK-26677][SQL] Disable dictionary filtering by default at Parquet

2019-01-24 Thread GitBox
HyukjinKwon commented on a change in pull request #23622: [SPARK-26677][SQL] Disable dictionary filtering by default at Parquet URL: https://github.com/apache/spark/pull/23622#discussion_r250883971 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/par

[GitHub] felixcheung commented on a change in pull request #22954: [SPARK-25981][R] Enables Arrow optimization from R DataFrame to Spark DataFrame

2019-01-24 Thread GitBox
felixcheung commented on a change in pull request #22954: [SPARK-25981][R] Enables Arrow optimization from R DataFrame to Spark DataFrame URL: https://github.com/apache/spark/pull/22954#discussion_r250883467 ## File path: R/pkg/R/SQLContext.R ## @@ -147,6 +147,73 @@ getDef

[GitHub] gengliangwang opened a new pull request #23648: [BRANCH-2.3][SPARK-26709][SQL] OptimizeMetadataOnlyQuery does not handle empty records correctly

2019-01-24 Thread GitBox
gengliangwang opened a new pull request #23648: [BRANCH-2.3][SPARK-26709][SQL] OptimizeMetadataOnlyQuery does not handle empty records correctly URL: https://github.com/apache/spark/pull/23648 ## What changes were proposed in this pull request? When reading from empty tables, the opt

[GitHub] gengliangwang commented on issue #23648: [BRANCH-2.3][SPARK-26709][SQL] OptimizeMetadataOnlyQuery does not handle empty records correctly

2019-01-24 Thread GitBox
gengliangwang commented on issue #23648: [BRANCH-2.3][SPARK-26709][SQL] OptimizeMetadataOnlyQuery does not handle empty records correctly URL: https://github.com/apache/spark/pull/23648#issuecomment-457476253 This PR is to port https://github.com/apache/spark/pull/23635 to branch 2.3 -

[GitHub] AmplabJenkins commented on issue #23647: [SPARK-26712]Support multi directories for executor shuffle info recovery in yarn shuffle serivce

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #23647: [SPARK-26712]Support multi directories for executor shuffle info recovery in yarn shuffle serivce URL: https://github.com/apache/spark/pull/23647#issuecomment-457474810 Can one of the admins verify this patch? ---

[GitHub] AmplabJenkins removed a comment on issue #23647: [SPARK-26712]Support multi directories for executor shuffle info recovery in yarn shuffle serivce

2019-01-24 Thread GitBox
AmplabJenkins removed a comment on issue #23647: [SPARK-26712]Support multi directories for executor shuffle info recovery in yarn shuffle serivce URL: https://github.com/apache/spark/pull/23647#issuecomment-457474153 Can one of the admins verify this patch? ---

[GitHub] AmplabJenkins removed a comment on issue #23647: [SPARK-26712]Support multi directories for executor shuffle info recovery in yarn shuffle serivce

2019-01-24 Thread GitBox
AmplabJenkins removed a comment on issue #23647: [SPARK-26712]Support multi directories for executor shuffle info recovery in yarn shuffle serivce URL: https://github.com/apache/spark/pull/23647#issuecomment-457473848 Can one of the admins verify this patch? ---

[GitHub] AmplabJenkins commented on issue #23647: [SPARK-26712]Support multi directories for executor shuffle info recovery in yarn shuffle serivce

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #23647: [SPARK-26712]Support multi directories for executor shuffle info recovery in yarn shuffle serivce URL: https://github.com/apache/spark/pull/23647#issuecomment-457474153 Can one of the admins verify this patch? ---

[GitHub] AmplabJenkins commented on issue #23647: [SPARK-26712]Support multi directories for executor shuffle info recovery in yarn shuffle serivce

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #23647: [SPARK-26712]Support multi directories for executor shuffle info recovery in yarn shuffle serivce URL: https://github.com/apache/spark/pull/23647#issuecomment-457473848 Can one of the admins verify this patch? ---

[GitHub] liupc opened a new pull request #23647: [SPARK-26712]Support multi directories for executor shuffle info recovery in yarn shuffle serivce

2019-01-24 Thread GitBox
liupc opened a new pull request #23647: [SPARK-26712]Support multi directories for executor shuffle info recovery in yarn shuffle serivce URL: https://github.com/apache/spark/pull/23647 ## What changes were proposed in this pull request? Currently, `ExecutorShuffleInfo` can be recove

[GitHub] AmplabJenkins removed a comment on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-01-24 Thread GitBox
AmplabJenkins removed a comment on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-457473660 Merged build finished. Test PASSed. This is an automated

[GitHub] AmplabJenkins removed a comment on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-01-24 Thread GitBox
AmplabJenkins removed a comment on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-457473665 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.e

[GitHub] AmplabJenkins commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-457473665 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenki

[GitHub] AmplabJenkins commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-457473660 Merged build finished. Test PASSed. This is an automated message

[GitHub] SparkQA removed a comment on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-01-24 Thread GitBox
SparkQA removed a comment on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-457430690 **[Test build #101659 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101659/testReport)** for

[GitHub] SparkQA commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-01-24 Thread GitBox
SparkQA commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-457473298 **[Test build #101659 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101659/testReport)** for PR 233

[GitHub] dongjoon-hyun commented on issue #23646: [SPARK-26427][BUILD][BACKPORT-2.4] Upgrade Apache ORC to 1.5.4

2019-01-24 Thread GitBox
dongjoon-hyun commented on issue #23646: [SPARK-26427][BUILD][BACKPORT-2.4] Upgrade Apache ORC to 1.5.4 URL: https://github.com/apache/spark/pull/23646#issuecomment-457468461 Oh, thank you for informing that and approval, @srowen ! --

[GitHub] srowen commented on issue #23646: [SPARK-26427][BUILD][BACKPORT-2.4] Upgrade Apache ORC to 1.5.4

2019-01-24 Thread GitBox
srowen commented on issue #23646: [SPARK-26427][BUILD][BACKPORT-2.4] Upgrade Apache ORC to 1.5.4 URL: https://github.com/apache/spark/pull/23646#issuecomment-457468091 I think a maintenance release update of a dependency is OK in a maintenance branch, if there are specific issues it would

[GitHub] dongjoon-hyun commented on a change in pull request #23628: [SPARK-26702][TEST] Create a test trait for Parquet and Orc test

2019-01-24 Thread GitBox
dongjoon-hyun commented on a change in pull request #23628: [SPARK-26702][TEST] Create a test trait for Parquet and Orc test URL: https://github.com/apache/spark/pull/23628#discussion_r250875953 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/Or

[GitHub] Ngone51 commented on issue #23497: [SPARK-26578][CORE] Synchronize putBytes's memory allocation and putting block on memoryManager

2019-01-24 Thread GitBox
Ngone51 commented on issue #23497: [SPARK-26578][CORE] Synchronize putBytes's memory allocation and putting block on memoryManager URL: https://github.com/apache/spark/pull/23497#issuecomment-457466509 > If we put some blocks into entries before putting block1, task 2 will evict them all f

[GitHub] AmplabJenkins removed a comment on issue #23646: [SPARK-26427][BUILD][BACKPORT-2.4] Upgrade Apache ORC to 1.5.4

2019-01-24 Thread GitBox
AmplabJenkins removed a comment on issue #23646: [SPARK-26427][BUILD][BACKPORT-2.4] Upgrade Apache ORC to 1.5.4 URL: https://github.com/apache/spark/pull/23646#issuecomment-457466236 Merged build finished. Test PASSed. This i

[GitHub] AmplabJenkins removed a comment on issue #23646: [SPARK-26427][BUILD][BACKPORT-2.4] Upgrade Apache ORC to 1.5.4

2019-01-24 Thread GitBox
AmplabJenkins removed a comment on issue #23646: [SPARK-26427][BUILD][BACKPORT-2.4] Upgrade Apache ORC to 1.5.4 URL: https://github.com/apache/spark/pull/23646#issuecomment-457466245 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://ampla

[GitHub] AmplabJenkins commented on issue #23646: [SPARK-26427][BUILD][BACKPORT-2.4] Upgrade Apache ORC to 1.5.4

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #23646: [SPARK-26427][BUILD][BACKPORT-2.4] Upgrade Apache ORC to 1.5.4 URL: https://github.com/apache/spark/pull/23646#issuecomment-457466236 Merged build finished. Test PASSed. This is an aut

[GitHub] AmplabJenkins commented on issue #23646: [SPARK-26427][BUILD][BACKPORT-2.4] Upgrade Apache ORC to 1.5.4

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #23646: [SPARK-26427][BUILD][BACKPORT-2.4] Upgrade Apache ORC to 1.5.4 URL: https://github.com/apache/spark/pull/23646#issuecomment-457466245 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.ber

[GitHub] dongjoon-hyun commented on issue #23646: [SPARK-26427][BUILD][BACKPORT-2.4] Upgrade Apache ORC to 1.5.4

2019-01-24 Thread GitBox
dongjoon-hyun commented on issue #23646: [SPARK-26427][BUILD][BACKPORT-2.4] Upgrade Apache ORC to 1.5.4 URL: https://github.com/apache/spark/pull/23646#issuecomment-457465781 Hi, @srowen , @cloud-fan , @gatorsmile , @dbtsai , @HyukjinKwon . In general, we don't upgrade dependency in

[GitHub] AmplabJenkins removed a comment on issue #23645: [SPARK-26725][TEST] Fix the input values of UnifiedMemoryManager constructor in test suites

2019-01-24 Thread GitBox
AmplabJenkins removed a comment on issue #23645: [SPARK-26725][TEST] Fix the input values of UnifiedMemoryManager constructor in test suites URL: https://github.com/apache/spark/pull/23645#issuecomment-457465318 Test PASSed. Refer to this link for build results (access rights to CI serve

[GitHub] AmplabJenkins removed a comment on issue #23645: [SPARK-26725][TEST] Fix the input values of UnifiedMemoryManager constructor in test suites

2019-01-24 Thread GitBox
AmplabJenkins removed a comment on issue #23645: [SPARK-26725][TEST] Fix the input values of UnifiedMemoryManager constructor in test suites URL: https://github.com/apache/spark/pull/23645#issuecomment-457465312 Merged build finished. Test PASSed. --

[GitHub] AmplabJenkins commented on issue #23645: [SPARK-26725][TEST] Fix the input values of UnifiedMemoryManager constructor in test suites

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #23645: [SPARK-26725][TEST] Fix the input values of UnifiedMemoryManager constructor in test suites URL: https://github.com/apache/spark/pull/23645#issuecomment-457465318 Test PASSed. Refer to this link for build results (access rights to CI server needed

[GitHub] AmplabJenkins commented on issue #23645: [SPARK-26725][TEST] Fix the input values of UnifiedMemoryManager constructor in test suites

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #23645: [SPARK-26725][TEST] Fix the input values of UnifiedMemoryManager constructor in test suites URL: https://github.com/apache/spark/pull/23645#issuecomment-457465312 Merged build finished. Test PASSed. --

[GitHub] SparkQA commented on issue #23646: [SPARK-26427][BUILD][BACKPORT-2.4] Upgrade Apache ORC to 1.5.4

2019-01-24 Thread GitBox
SparkQA commented on issue #23646: [SPARK-26427][BUILD][BACKPORT-2.4] Upgrade Apache ORC to 1.5.4 URL: https://github.com/apache/spark/pull/23646#issuecomment-457464989 **[Test build #101665 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101665/testReport)*

[GitHub] SparkQA commented on issue #23645: [SPARK-26725][TEST] Fix the input values of UnifiedMemoryManager constructor in test suites

2019-01-24 Thread GitBox
SparkQA commented on issue #23645: [SPARK-26725][TEST] Fix the input values of UnifiedMemoryManager constructor in test suites URL: https://github.com/apache/spark/pull/23645#issuecomment-457464990 **[Test build #101666 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullReque

[GitHub] dongjoon-hyun opened a new pull request #23646: [SPARK-26427][BUILD][BACKPORT-2.4] Upgrade Apache ORC to 1.5.4

2019-01-24 Thread GitBox
dongjoon-hyun opened a new pull request #23646: [SPARK-26427][BUILD][BACKPORT-2.4] Upgrade Apache ORC to 1.5.4 URL: https://github.com/apache/spark/pull/23646 ## What changes were proposed in this pull request? This is a backport of #23364. To make Apache Spark 2.4.1 more robu

[GitHub] srowen opened a new pull request #23645: [SPARK-26725][TEST] Fix the input values of UnifiedMemoryManager constructor in test suites

2019-01-24 Thread GitBox
srowen opened a new pull request #23645: [SPARK-26725][TEST] Fix the input values of UnifiedMemoryManager constructor in test suites URL: https://github.com/apache/spark/pull/23645 ## What changes were proposed in this pull request? Adjust mem settings in UnifiedMemoryManager used in

[GitHub] AmplabJenkins removed a comment on issue #23628: [SPARK-26702][TEST] Create a test trait for Parquet and Orc test

2019-01-24 Thread GitBox
AmplabJenkins removed a comment on issue #23628: [SPARK-26702][TEST] Create a test trait for Parquet and Orc test URL: https://github.com/apache/spark/pull/23628#issuecomment-457464389 Merged build finished. Test PASSed. This

[GitHub] AmplabJenkins commented on issue #23628: [SPARK-26702][TEST] Create a test trait for Parquet and Orc test

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #23628: [SPARK-26702][TEST] Create a test trait for Parquet and Orc test URL: https://github.com/apache/spark/pull/23628#issuecomment-457464389 Merged build finished. Test PASSed. This is an a

[GitHub] AmplabJenkins commented on issue #23628: [SPARK-26702][TEST] Create a test trait for Parquet and Orc test

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #23628: [SPARK-26702][TEST] Create a test trait for Parquet and Orc test URL: https://github.com/apache/spark/pull/23628#issuecomment-457464392 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.b

[GitHub] AmplabJenkins removed a comment on issue #23628: [SPARK-26702][TEST] Create a test trait for Parquet and Orc test

2019-01-24 Thread GitBox
AmplabJenkins removed a comment on issue #23628: [SPARK-26702][TEST] Create a test trait for Parquet and Orc test URL: https://github.com/apache/spark/pull/23628#issuecomment-457464392 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amp

[GitHub] SparkQA commented on issue #23628: [SPARK-26702][TEST] Create a test trait for Parquet and Orc test

2019-01-24 Thread GitBox
SparkQA commented on issue #23628: [SPARK-26702][TEST] Create a test trait for Parquet and Orc test URL: https://github.com/apache/spark/pull/23628#issuecomment-457464004 **[Test build #101664 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101664/testReport

[GitHub] srowen commented on a change in pull request #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI

2019-01-24 Thread GitBox
srowen commented on a change in pull request #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI URL: https://github.com/apache/spark/pull/23637#discussion_r250873365 ## File path: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala ## @@ -857,8 +857,

[GitHub] viirya commented on a change in pull request #23628: [SPARK-26702][TEST] Create a test trait for Parquet and Orc test

2019-01-24 Thread GitBox
viirya commented on a change in pull request #23628: [SPARK-26702][TEST] Create a test trait for Parquet and Orc test URL: https://github.com/apache/spark/pull/23628#discussion_r250873312 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcQueryS

[GitHub] srowen commented on a change in pull request #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI

2019-01-24 Thread GitBox
srowen commented on a change in pull request #23637: [SPARK-26714][CORE][WEBUI] Show 0 partition job in WebUI URL: https://github.com/apache/spark/pull/23637#discussion_r250872948 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -693,6

[GitHub] viirya commented on issue #20146: [SPARK-11215][ML] Add multiple columns support to StringIndexer

2019-01-24 Thread GitBox
viirya commented on issue #20146: [SPARK-11215][ML] Add multiple columns support to StringIndexer URL: https://github.com/apache/spark/pull/20146#issuecomment-457462999 @srowen Thanks for continuing the review! I think I addressed your latest comments. Please take a look at this. Thanks.

[GitHub] srowen commented on a change in pull request #23625: [SPARK-26700][CORE] enable fetch-big-block-to-memory by default

2019-01-24 Thread GitBox
srowen commented on a change in pull request #23625: [SPARK-26700][CORE] enable fetch-big-block-to-memory by default URL: https://github.com/apache/spark/pull/23625#discussion_r250871989 ## File path: docs/configuration.md ## @@ -1519,6 +1519,17 @@ Apart from these, the fo

[GitHub] SparkQA removed a comment on issue #23638: [SPARK-26713][CORE] Interrupt pipe IO threads in PipedRDD when task is finished

2019-01-24 Thread GitBox
SparkQA removed a comment on issue #23638: [SPARK-26713][CORE] Interrupt pipe IO threads in PipedRDD when task is finished URL: https://github.com/apache/spark/pull/23638#issuecomment-457417533 **[Test build #4529 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestB

[GitHub] SparkQA commented on issue #23638: [SPARK-26713][CORE] Interrupt pipe IO threads in PipedRDD when task is finished

2019-01-24 Thread GitBox
SparkQA commented on issue #23638: [SPARK-26713][CORE] Interrupt pipe IO threads in PipedRDD when task is finished URL: https://github.com/apache/spark/pull/23638#issuecomment-457461904 **[Test build #4529 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/

[GitHub] AmplabJenkins commented on issue #23618: [SPARK-26694][CORE] Progress bar should be enabled by default for spark-shell

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #23618: [SPARK-26694][CORE] Progress bar should be enabled by default for spark-shell URL: https://github.com/apache/spark/pull/23618#issuecomment-457461794 Test PASSed. Refer to this link for build results (access rights to CI server needed): https:/

[GitHub] AmplabJenkins removed a comment on issue #23618: [SPARK-26694][CORE] Progress bar should be enabled by default for spark-shell

2019-01-24 Thread GitBox
AmplabJenkins removed a comment on issue #23618: [SPARK-26694][CORE] Progress bar should be enabled by default for spark-shell URL: https://github.com/apache/spark/pull/23618#issuecomment-457461794 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] AmplabJenkins commented on issue #23618: [SPARK-26694][CORE] Progress bar should be enabled by default for spark-shell

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #23618: [SPARK-26694][CORE] Progress bar should be enabled by default for spark-shell URL: https://github.com/apache/spark/pull/23618#issuecomment-457461792 Merged build finished. Test PASSed.

[GitHub] AmplabJenkins removed a comment on issue #23618: [SPARK-26694][CORE] Progress bar should be enabled by default for spark-shell

2019-01-24 Thread GitBox
AmplabJenkins removed a comment on issue #23618: [SPARK-26694][CORE] Progress bar should be enabled by default for spark-shell URL: https://github.com/apache/spark/pull/23618#issuecomment-457461792 Merged build finished. Test PASSed.

[GitHub] SparkQA commented on issue #23618: [SPARK-26694][CORE] Progress bar should be enabled by default for spark-shell

2019-01-24 Thread GitBox
SparkQA commented on issue #23618: [SPARK-26694][CORE] Progress bar should be enabled by default for spark-shell URL: https://github.com/apache/spark/pull/23618#issuecomment-457461481 **[Test build #101657 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101

[GitHub] bersprockets commented on issue #23642: [SPARK-26680][SPARK-25767][SQL][BACKPORT-2.3] Eagerly create inputVars while conditions are appropriate

2019-01-24 Thread GitBox
bersprockets commented on issue #23642: [SPARK-26680][SPARK-25767][SQL][BACKPORT-2.3] Eagerly create inputVars while conditions are appropriate URL: https://github.com/apache/spark/pull/23642#issuecomment-457461455 @dongjoon-hyun Thanks much! --

[GitHub] SparkQA removed a comment on issue #23618: [SPARK-26694][CORE] Progress bar should be enabled by default for spark-shell

2019-01-24 Thread GitBox
SparkQA removed a comment on issue #23618: [SPARK-26694][CORE] Progress bar should be enabled by default for spark-shell URL: https://github.com/apache/spark/pull/23618#issuecomment-457419075 **[Test build #101657 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuil

[GitHub] AmplabJenkins removed a comment on issue #20146: [SPARK-11215][ML] Add multiple columns support to StringIndexer

2019-01-24 Thread GitBox
AmplabJenkins removed a comment on issue #20146: [SPARK-11215][ML] Add multiple columns support to StringIndexer URL: https://github.com/apache/spark/pull/20146#issuecomment-457461060 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://ampla

[GitHub] AmplabJenkins removed a comment on issue #20146: [SPARK-11215][ML] Add multiple columns support to StringIndexer

2019-01-24 Thread GitBox
AmplabJenkins removed a comment on issue #20146: [SPARK-11215][ML] Add multiple columns support to StringIndexer URL: https://github.com/apache/spark/pull/20146#issuecomment-457461055 Merged build finished. Test PASSed. This

[GitHub] AmplabJenkins commented on issue #20146: [SPARK-11215][ML] Add multiple columns support to StringIndexer

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #20146: [SPARK-11215][ML] Add multiple columns support to StringIndexer URL: https://github.com/apache/spark/pull/20146#issuecomment-457461060 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.ber

[GitHub] AmplabJenkins commented on issue #20146: [SPARK-11215][ML] Add multiple columns support to StringIndexer

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #20146: [SPARK-11215][ML] Add multiple columns support to StringIndexer URL: https://github.com/apache/spark/pull/20146#issuecomment-457461055 Merged build finished. Test PASSed. This is an au

[GitHub] AmplabJenkins removed a comment on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan

2019-01-24 Thread GitBox
AmplabJenkins removed a comment on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan URL: https://github.com/apache/spark/pull/23644#issuecomment-457460809 Test PASSed. Refer to this link for build results (a

[GitHub] AmplabJenkins removed a comment on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan

2019-01-24 Thread GitBox
AmplabJenkins removed a comment on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan URL: https://github.com/apache/spark/pull/23644#issuecomment-457460808 Merged build finished. Test PASSed. --

[GitHub] AmplabJenkins commented on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan URL: https://github.com/apache/spark/pull/23644#issuecomment-457460809 Test PASSed. Refer to this link for build results (access rig

[GitHub] SparkQA removed a comment on issue #20146: [SPARK-11215][ML] Add multiple columns support to StringIndexer

2019-01-24 Thread GitBox
SparkQA removed a comment on issue #20146: [SPARK-11215][ML] Add multiple columns support to StringIndexer URL: https://github.com/apache/spark/pull/20146#issuecomment-457415886 **[Test build #101656 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101656/tes

[GitHub] AmplabJenkins commented on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan URL: https://github.com/apache/spark/pull/23644#issuecomment-457460808 Merged build finished. Test PASSed. ---

[GitHub] SparkQA commented on issue #20146: [SPARK-11215][ML] Add multiple columns support to StringIndexer

2019-01-24 Thread GitBox
SparkQA commented on issue #20146: [SPARK-11215][ML] Add multiple columns support to StringIndexer URL: https://github.com/apache/spark/pull/20146#issuecomment-457460741 **[Test build #101656 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101656/testReport

[GitHub] SparkQA commented on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan

2019-01-24 Thread GitBox
SparkQA commented on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan URL: https://github.com/apache/spark/pull/23644#issuecomment-457460532 **[Test build #101663 has started](https://amplab.cs.berkeley.edu/jen

[GitHub] dongjoon-hyun commented on a change in pull request #23639: [SPARK-26716][SQL] FileFormat: the supported types of read/write should be consistent

2019-01-24 Thread GitBox
dongjoon-hyun commented on a change in pull request #23639: [SPARK-26716][SQL] FileFormat: the supported types of read/write should be consistent URL: https://github.com/apache/spark/pull/23639#discussion_r250869623 ## File path: sql/core/src/main/scala/org/apache/spark/sql/executi

[GitHub] dongjoon-hyun commented on a change in pull request #23639: [SPARK-26716][SQL] FileFormat: the supported types of read/write should be consistent

2019-01-24 Thread GitBox
dongjoon-hyun commented on a change in pull request #23639: [SPARK-26716][SQL] FileFormat: the supported types of read/write should be consistent URL: https://github.com/apache/spark/pull/23639#discussion_r250869623 ## File path: sql/core/src/main/scala/org/apache/spark/sql/executi

[GitHub] dongjoon-hyun commented on a change in pull request #23639: [SPARK-26716][SQL] FileFormat: the supported types of read/write should be consistent

2019-01-24 Thread GitBox
dongjoon-hyun commented on a change in pull request #23639: [SPARK-26716][SQL] FileFormat: the supported types of read/write should be consistent URL: https://github.com/apache/spark/pull/23639#discussion_r250869623 ## File path: sql/core/src/main/scala/org/apache/spark/sql/executi

[GitHub] dongjoon-hyun commented on a change in pull request #23639: [SPARK-26716][SQL] FileFormat: the supported types of read/write should be consistent

2019-01-24 Thread GitBox
dongjoon-hyun commented on a change in pull request #23639: [SPARK-26716][SQL] FileFormat: the supported types of read/write should be consistent URL: https://github.com/apache/spark/pull/23639#discussion_r250869623 ## File path: sql/core/src/main/scala/org/apache/spark/sql/executi

[GitHub] SparkQA commented on issue #23260: [SPARK-26311][CORE] New feature: apply custom log URL pattern for executor log URLs in SHS

2019-01-24 Thread GitBox
SparkQA commented on issue #23260: [SPARK-26311][CORE] New feature: apply custom log URL pattern for executor log URLs in SHS URL: https://github.com/apache/spark/pull/23260#issuecomment-457458049 **[Test build #101662 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullReques

[GitHub] viirya commented on a change in pull request #23638: [SPARK-26713][CORE] Interrupt pipe IO threads in PipedRDD when task is finished

2019-01-24 Thread GitBox
viirya commented on a change in pull request #23638: [SPARK-26713][CORE] Interrupt pipe IO threads in PipedRDD when task is finished URL: https://github.com/apache/spark/pull/23638#discussion_r250867842 ## File path: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherI

[GitHub] viirya commented on a change in pull request #23638: [SPARK-26713][CORE] Interrupt pipe IO threads in PipedRDD when task is finished

2019-01-24 Thread GitBox
viirya commented on a change in pull request #23638: [SPARK-26713][CORE] Interrupt pipe IO threads in PipedRDD when task is finished URL: https://github.com/apache/spark/pull/23638#discussion_r250861770 ## File path: core/src/main/scala/org/apache/spark/rdd/PipedRDD.scala #

[GitHub] SongYadong commented on issue #23497: [SPARK-26578][CORE] Synchronize putBytes's memory allocation and putting block on memoryManager

2019-01-24 Thread GitBox
SongYadong commented on issue #23497: [SPARK-26578][CORE] Synchronize putBytes's memory allocation and putting block on memoryManager URL: https://github.com/apache/spark/pull/23497#issuecomment-457453790 > In your test, no blocks in entries when task 2 try to evict blocks, right ? So, how

[GitHub] dongjoon-hyun commented on issue #23622: [SPARK-26677][SQL] Disable dictionary filtering by default at Parquet

2019-01-24 Thread GitBox
dongjoon-hyun commented on issue #23622: [SPARK-26677][SQL] Disable dictionary filtering by default at Parquet URL: https://github.com/apache/spark/pull/23622#issuecomment-457453633 Hi, @rdblue . Parquet 0.10 is released on April 2018. And, PARQUET-1309 seems to be resolved Jun 5, 2018. I'

[GitHub] AmplabJenkins removed a comment on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan

2019-01-24 Thread GitBox
AmplabJenkins removed a comment on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan URL: https://github.com/apache/spark/pull/23644#issuecomment-457453012 Test PASSed. Refer to this link for build results (a

[GitHub] AmplabJenkins removed a comment on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan

2019-01-24 Thread GitBox
AmplabJenkins removed a comment on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan URL: https://github.com/apache/spark/pull/23644#issuecomment-457453010 Merged build finished. Test PASSed. --

[GitHub] dongjoon-hyun commented on a change in pull request #23622: [SPARK-26677][SQL] Disable dictionary filtering by default at Parquet

2019-01-24 Thread GitBox
dongjoon-hyun commented on a change in pull request #23622: [SPARK-26677][SQL] Disable dictionary filtering by default at Parquet URL: https://github.com/apache/spark/pull/23622#discussion_r250864643 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/p

[GitHub] AmplabJenkins commented on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan URL: https://github.com/apache/spark/pull/23644#issuecomment-457453010 Merged build finished. Test PASSed. ---

[GitHub] AmplabJenkins commented on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan

2019-01-24 Thread GitBox
AmplabJenkins commented on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan URL: https://github.com/apache/spark/pull/23644#issuecomment-457453012 Test PASSed. Refer to this link for build results (access rig

[GitHub] dongjoon-hyun commented on issue #23622: [SPARK-26677][SQL] Disable dictionary filtering by default at Parquet

2019-01-24 Thread GitBox
dongjoon-hyun commented on issue #23622: [SPARK-26677][SQL] Disable dictionary filtering by default at Parquet URL: https://github.com/apache/spark/pull/23622#issuecomment-457452603 cc @dbtsai This is an automated message fr

[GitHub] SparkQA commented on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan

2019-01-24 Thread GitBox
SparkQA commented on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan URL: https://github.com/apache/spark/pull/23644#issuecomment-457451977 **[Test build #101661 has started](https://amplab.cs.berkeley.edu/jen

[GitHub] maryannxue commented on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan

2019-01-24 Thread GitBox
maryannxue commented on issue #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan URL: https://github.com/apache/spark/pull/23644#issuecomment-457451818 cc @gatorsmile ---

[GitHub] maryannxue opened a new pull request #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan

2019-01-24 Thread GitBox
maryannxue opened a new pull request #23644: [SPARK-26708][SQL] Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan URL: https://github.com/apache/spark/pull/23644 ## What changes were proposed in this pull request? When performing non-cas

[GitHub] dongjoon-hyun commented on a change in pull request #23641: [SPARK-26719][SQL] Get rid of java.util.Calendar in DateTimeUtils

2019-01-24 Thread GitBox
dongjoon-hyun commented on a change in pull request #23641: [SPARK-26719][SQL] Get rid of java.util.Calendar in DateTimeUtils URL: https://github.com/apache/spark/pull/23641#discussion_r250862263 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeT

  1   2   3   4   5   6   7   8   9   >