[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21266 **[Test build #91017 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91017/testReport)** for PR 21266 at commit

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21266 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21266 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3492/

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-22 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21266 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21266: [SPARK-24206][SQL] Improve DataSource read benchm...

2018-05-22 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21266#discussion_r190129285 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/DataSourceReadBenchmark.scala --- @@ -0,0 +1,827 @@ +/* + * Licensed to

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21389 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3491/

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21389 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21266 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91012/ Test FAILed. ---

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21266 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21389 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91010/ Test FAILed. ---

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21389 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21266 **[Test build #91012 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91012/testReport)** for PR 21266 at commit

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21389 **[Test build #91016 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91016/testReport)** for PR 21389 at commit

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21389 **[Test build #91010 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91010/testReport)** for PR 21389 at commit

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-22 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/21389 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate the new s...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21311 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91009/ Test FAILed. ---

[GitHub] spark issue #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate the new s...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21311 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate the new s...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21311 **[Test build #91009 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91009/testReport)** for PR 21311 at commit

[GitHub] spark pull request #21372: [SPARK-24322][BUILD] Upgrade Apache ORC to 1.4.4

2018-05-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21372#discussion_r190123342 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala --- @@ -169,6 +170,14 @@ abstract class OrcSuite

[GitHub] spark issue #21390: [SPARK-24340][Core] Clean up non-shuffle disk block mana...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21390 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21390: [SPARK-24340][Core] Clean up non-shuffle disk block mana...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21390 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91006/ Test FAILed. ---

[GitHub] spark issue #21390: [SPARK-24340][Core] Clean up non-shuffle disk block mana...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21390 **[Test build #91006 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91006/testReport)** for PR 21390 at commit

[GitHub] spark issue #21401: [SPARK-24350][SQL] "array_position" error fix

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21401 **[Test build #91015 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91015/testReport)** for PR 21401 at commit

[GitHub] spark issue #21401: [SPARK-24350][SQL] "array_position" error fix

2018-05-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21401 good catch, thanks --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21401: [SPARK-24350][SQL] "array_position" error fix

2018-05-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21401 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21069: [SPARK-23920][SQL]add array_remove to remove all element...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21069 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91007/ Test FAILed. ---

[GitHub] spark issue #21069: [SPARK-23920][SQL]add array_remove to remove all element...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21069 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21069: [SPARK-23920][SQL]add array_remove to remove all element...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21069 **[Test build #91007 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91007/testReport)** for PR 21069 at commit

[GitHub] spark issue #21401: [SPARK-24350][SQL] "array_position" error fix

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21401 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21401: [SPARK-24350][SQL] "array_position" error fix

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21401 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91005/ Test FAILed. ---

[GitHub] spark issue #21401: [SPARK-24350][SQL] "array_position" error fix

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21401 **[Test build #91005 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91005/testReport)** for PR 21401 at commit

[GitHub] spark issue #21372: [SPARK-24322][BUILD] Upgrade Apache ORC to 1.4.4

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21372 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21372: [SPARK-24322][BUILD] Upgrade Apache ORC to 1.4.4

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21372 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3490/

[GitHub] spark issue #21405: [SPARK-24361][SQL] Polish code block manipulation API

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21405 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3489/

[GitHub] spark issue #21405: [SPARK-24361][SQL] Polish code block manipulation API

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21405 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21404: [SPARK-24360][SQL] Support Hive 3.0 metastore

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21404 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21404: [SPARK-24360][SQL] Support Hive 3.0 metastore

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21404 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91008/ Test FAILed. ---

[GitHub] spark issue #21404: [SPARK-24360][SQL] Support Hive 3.0 metastore

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21404 **[Test build #91008 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91008/testReport)** for PR 21404 at commit

[GitHub] spark pull request #21396: [SPARK-24349][SQL] Ignore setting token if using ...

2018-05-22 Thread LantaoJin
Github user LantaoJin closed the pull request at: https://github.com/apache/spark/pull/21396 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21405: [SPARK-24361][SQL] Polish code block manipulation API

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21405 **[Test build #91014 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91014/testReport)** for PR 21405 at commit

[GitHub] spark issue #21405: [SPARK-24361][SQL] Polish code block manipulation API

2018-05-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21405 cc @cloud-fan @hvanhovell @kiszk @mgaido91 @maropu @rednaxelafx @ueshin --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21396: [SPARK-24349][SQL] Ignore setting token if using ...

2018-05-22 Thread LantaoJin
Github user LantaoJin commented on a diff in the pull request: https://github.com/apache/spark/pull/21396#discussion_r190113497 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HiveDelegationTokenProvider.scala --- @@ -85,7 +85,10 @@ private[spark] class

[GitHub] spark pull request #21405: [SPARK-24361][SQL] Polish code block manipulation...

2018-05-22 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/21405 [SPARK-24361][SQL] Polish code block manipulation API ## What changes were proposed in this pull request? Current code block manipulation API is immature and hacky. We need a formal API to

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21399 @vanzin, I will leave it to you as I believe you are the most appropriate one to review this - I assume the approach you initially took is the same with this. I feel quite sure on the current

[GitHub] spark issue #21372: [SPARK-24322][BUILD] Upgrade Apache ORC to 1.4.4

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21372 **[Test build #91013 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91013/testReport)** for PR 21372 at commit

[GitHub] spark issue #21385: [SPARK-24234][SS] Support multiple row writers in contin...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21385 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91002/ Test PASSed. ---

[GitHub] spark pull request #21392: [SPARK-24063][SS] Control maximum epoch backlog f...

2018-05-22 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/21392#discussion_r190112873 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -233,9 +235,15 @@ class

[GitHub] spark issue #21372: [SPARK-24322][BUILD] Upgrade Apache ORC to 1.4.4

2018-05-22 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21372 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21372: [SPARK-24322][BUILD] Upgrade Apache ORC to 1.4.4

2018-05-22 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21372 The failure is irrelevant to this PR. ``` org.apache.spark.sql.execution.datasources.csv.UnivocityParserSuite.(It is not a test it is a sbt.testing.SuiteSelector) ``` ---

[GitHub] spark pull request #21396: [SPARK-24349][SQL] Ignore setting token if using ...

2018-05-22 Thread LantaoJin
Github user LantaoJin commented on a diff in the pull request: https://github.com/apache/spark/pull/21396#discussion_r190112872 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HiveDelegationTokenProvider.scala --- @@ -85,7 +85,10 @@ private[spark] class

[GitHub] spark issue #21385: [SPARK-24234][SS] Support multiple row writers in contin...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21385 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21392: [SPARK-24063][SS] Control maximum epoch backlog f...

2018-05-22 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/21392#discussion_r190112179 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/EpochCoordinator.scala --- @@ -45,6 +45,11 @@ private[sql] case

[GitHub] spark issue #21385: [SPARK-24234][SS] Support multiple row writers in contin...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21385 **[Test build #91002 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91002/testReport)** for PR 21385 at commit

[GitHub] spark pull request #21396: [SPARK-24349][SQL] Ignore setting token if using ...

2018-05-22 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21396#discussion_r190112524 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HiveDelegationTokenProvider.scala --- @@ -85,7 +85,10 @@ private[spark] class

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91000/ Test PASSed. ---

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21399 **[Test build #91000 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91000/testReport)** for PR 21399 at commit

[GitHub] spark issue #21396: [SPARK-24349][SQL] Ignore setting token if using JDBC

2018-05-22 Thread LantaoJin
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/21396 In our current settings, when we onboard a new cluster, the default is connect to DB directly, it's much simpler than access metastore. And we are going to update to access metastore by default.

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21266 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3488/

[GitHub] spark pull request #21396: [SPARK-24349][SQL] Ignore setting token if using ...

2018-05-22 Thread LantaoJin
Github user LantaoJin commented on a diff in the pull request: https://github.com/apache/spark/pull/21396#discussion_r190110376 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HiveDelegationTokenProvider.scala --- @@ -85,7 +85,10 @@ private[spark] class

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21266 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21401: [SPARK-24350][SQL] "array_position" error fix

2018-05-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21401 `Fixes ClassCastException in the array_position function` - this seems like a better PR title than the current one. --- - To

[GitHub] spark pull request #21396: [SPARK-24349][SQL] Ignore setting token if using ...

2018-05-22 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21396#discussion_r190109928 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HiveDelegationTokenProvider.scala --- @@ -85,7 +85,10 @@ private[spark] class

[GitHub] spark pull request #21392: [SPARK-24063][SS] Control maximum epoch backlog f...

2018-05-22 Thread yanlin-Lynn
Github user yanlin-Lynn commented on a diff in the pull request: https://github.com/apache/spark/pull/21392#discussion_r190109933 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/EpochCoordinator.scala --- @@ -153,9 +161,13 @@

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21389 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3487/

[GitHub] spark issue #21396: [SPARK-24349][SQL] Ignore setting token if using JDBC

2018-05-22 Thread LantaoJin
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/21396 Also, why still needs #20784 or #21343 to extends to #17335 may be caused by: 1. Some DDL operation in local mode is much faster than launching a AM in yarn. 2. Nodes in YARN cluster have

[GitHub] spark issue #21396: [SPARK-24349][SQL] Ignore setting token if using JDBC

2018-05-22 Thread LantaoJin
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/21396 @jerryshao Simply speaking, in a security environment, if we use JDBC to connect to mysql directly instead of accessing hive metastore, current implementation blocks job execution.

[GitHub] spark pull request #21392: [SPARK-24063][SS] Control maximum epoch backlog f...

2018-05-22 Thread yanlin-Lynn
Github user yanlin-Lynn commented on a diff in the pull request: https://github.com/apache/spark/pull/21392#discussion_r190108352 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -233,9 +235,15 @@ class

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-22 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21266 I'll update the benchmark results soon. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21266 **[Test build #91012 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91012/testReport)** for PR 21266 at commit

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread galv
Github user galv commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r190107244 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -306,9 +318,14 @@ private[yarn] class YarnAllocator(

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread galv
Github user galv commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r190106824 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,185 @@ +/* + * Licensed to

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread galv
Github user galv commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r190105256 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed to the

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread galv
Github user galv commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r190105917 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,185 @@ +/* + * Licensed to

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread galv
Github user galv commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r190106056 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,185 @@ +/* + * Licensed to

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread galv
Github user galv commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r190106859 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,185 @@ +/* + * Licensed to

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21389 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread galv
Github user galv commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r190107444 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -306,9 +318,14 @@ private[yarn] class YarnAllocator(

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread galv
Github user galv commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r190104166 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,150 @@ +/* + * Licensed to the

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread galv
Github user galv commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r190107502 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -306,9 +318,14 @@ private[yarn] class YarnAllocator(

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21389 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3486/

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21389 **[Test build #91011 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91011/testReport)** for PR 21389 at commit

[GitHub] spark issue #21402: SPARK-24355 Spark external shuffle server improvement to...

2018-05-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21402 what's the strategy here? if the number of block fetch requests has reached the limitation, shall we fail following block fetch requests immediately? ---

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21389 **[Test build #91010 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91010/testReport)** for PR 21389 at commit

[GitHub] spark pull request #21266: [SPARK-24206][SQL] Improve DataSource read benchm...

2018-05-22 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21266#discussion_r190107103 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/DataSourceReadBenchmark.scala --- @@ -0,0 +1,827 @@ +/* + * Licensed to

[GitHub] spark pull request #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate th...

2018-05-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21311#discussion_r190106818 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/joins/HashedRelationSuite.scala --- @@ -254,6 +254,30 @@ class HashedRelationSuite

[GitHub] spark issue #21396: [SPARK-24349][SQL] Ignore setting token if using JDBC

2018-05-22 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21396 Can you please describe your scenario @LantaoJin ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #21390: [SPARK-24340][Core] Clean up non-shuffle disk blo...

2018-05-22 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/21390#discussion_r190105740 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala --- @@ -97,6 +97,10 @@ private[deploy] class Worker( private val

[GitHub] spark issue #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate the new s...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21311 **[Test build #91009 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91009/testReport)** for PR 21311 at commit

[GitHub] spark issue #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate the new s...

2018-05-22 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21311 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21402: SPARK-24355 Spark external shuffle server improvement to...

2018-05-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21402 to confirm, `StreamRequest` will not block the server netty handler thread, right? --- - To unsubscribe, e-mail:

[GitHub] spark issue #21372: [SPARK-24322][BUILD] Upgrade Apache ORC to 1.4.4

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21372 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90999/ Test FAILed. ---

[GitHub] spark issue #21372: [SPARK-24322][BUILD] Upgrade Apache ORC to 1.4.4

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21372 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21404: [SPARK-24360][SQL] Support Hive 3.0 metastore

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21404 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21404: [SPARK-24360][SQL] Support Hive 3.0 metastore

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21404 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91003/ Test FAILed. ---

[GitHub] spark issue #21372: [SPARK-24322][BUILD] Upgrade Apache ORC to 1.4.4

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21372 **[Test build #90999 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90999/testReport)** for PR 21372 at commit

[GitHub] spark issue #21404: [SPARK-24360][SQL] Support Hive 3.0 metastore

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21404 **[Test build #91003 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91003/testReport)** for PR 21404 at commit

[GitHub] spark pull request #21390: [SPARK-24340][Core] Clean up non-shuffle disk blo...

2018-05-22 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/21390#discussion_r190104118 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala --- @@ -732,6 +736,9 @@ private[deploy] class Worker(

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread galv
Github user galv commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r190104076 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -249,6 +259,10 @@ private[spark] class Client( val

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread galv
Github user galv commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r190103364 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed to the

  1   2   3   4   5   6   7   8   9   >