[GitHub] spark pull request #18213: [SPARK-20996][YARN] Better handling AM reattempt ...

Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/18213#discussion_r120804119 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -229,8 +229,17 @@ private[spark] class

[GitHub] spark issue #15326: [SPARK-17759] [CORE] Avoid adding duplicate schedulables

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15326 I am sorry I misunderstood and thought it is almost (or already) ready. Will read the comments carefilly next time. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #15326: [SPARK-17759] [CORE] Avoid adding duplicate schedulables

Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/15326 @HyukjinKwon what's the ping here for? It looks like I left some comments that @erenavsarogullari will address when he has time. --- If your project is set up for it, you can reply to this

[GitHub] spark issue #18148: [SPARK-20926][SQL] Removing exposures to guava library c...

Github user yhuai commented on the issue: https://github.com/apache/spark/pull/18148 @vanzin Seems merging to branch-2.2 was an accident? Since it is not really a bug fix, should we revert it from branch-2.2 and just keep it in the master? --- If your project is set up for it, you

[GitHub] spark issue #18118: SPARK-20199 : Provided featureSubsetStrategy to GBTClass...

Github user pralabhkumar commented on the issue: https://github.com/apache/spark/pull/18118 @sethah agree with you . Sorry if I unnecessary bother , was eager to get reviews on pull request. Thanks for the suggestion , will keep in mind --- If your project is set up for it, you

[GitHub] spark issue #18226: [SPARK-21006][TESTS] Create rpcEnv and run later needs s...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18226 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18226: [SPARK-21006][TESTS] Create rpcEnv and run later needs s...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18226 **[Test build #77803 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77803/testReport)** for PR 18226 at commit

[GitHub] spark issue #18226: [SPARK-21006][TESTS] Create rpcEnv and run later needs s...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18226 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77803/ Test PASSed. ---

[GitHub] spark issue #16171: [SPARK-18739][ML][PYSPARK] Classification and regression...

Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/16171 @holdenk @MLnick Can you help reveiwing this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16171: [SPARK-18739][ML][PYSPARK] Classification and regression...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16171 **[Test build #77807 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77807/testReport)** for PR 16171 at commit

[GitHub] spark issue #16171: [SPARK-18739][ML][PYSPARK] Classification and regression...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16171 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16171: [SPARK-18739][ML][PYSPARK] Classification and regression...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16171 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77807/ Test PASSed. ---

[GitHub] spark issue #16171: [SPARK-18739][ML][PYSPARK] Classification and regression...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16171 **[Test build #77807 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77807/testReport)** for PR 16171 at commit

[GitHub] spark issue #18231: [WIP][SPARK-20994] Remove reduant characters in OpenBloc...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18231 **[Test build #77806 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77806/testReport)** for PR 18231 at commit

[GitHub] spark issue #18230: [SPARK-21008] [STREAMING] Not to read `spark.yarn.creden...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18230 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18235: [SPARK-21012][Submit] Add glob support for resources add...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18235 **[Test build #77805 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77805/testReport)** for PR 18235 at commit

[GitHub] spark issue #18230: [SPARK-21008] [STREAMING] Not to read `spark.yarn.creden...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18230 **[Test build #77804 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77804/testReport)** for PR 18230 at commit

[GitHub] spark issue #18230: [SPARK-21008] [STREAMING] Not to read `spark.yarn.creden...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18230 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77804/ Test PASSed. ---

[GitHub] spark pull request #18235: [SPARK-21012][Submit] Add glob support for resour...

GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/18235 [SPARK-21012][Submit] Add glob support for resources adding to Spark Current "--jars (spark.jars)", "--files (spark.files)", "--py-files (spark.submit.pyFiles)" and "--archives

[GitHub] spark issue #18064: [SPARK-20213][SQL] Fix DataFrameWriter operations in SQL...

Github user yhuai commented on the issue: https://github.com/apache/spark/pull/18064 I just case across this pr. I have one general feedback. It will be great if we can make a pr have a single purpose. This pr contains different kinds of changes in order to fix the UI. If refactoring

[GitHub] spark issue #15326: [SPARK-17759] [CORE] Avoid adding duplicate schedulables

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15326 Gentle ping @kayousterhout. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18223: [INFRA] Close stale PRs

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18223 Took out 15326 per https://github.com/apache/spark/pull/15326#issuecomment-306974580 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #15326: [SPARK-17759] [CORE] Avoid adding duplicate schedulables

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15326 I will take this out in the list. Thanks for your input. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #18230: [SPARK-21008] [STREAMING] Not to read `spark.yarn.creden...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18230 **[Test build #77804 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77804/testReport)** for PR 18230 at commit

[GitHub] spark issue #18230: [SPARK-21008] [STREAMING] Not to read `spark.yarn.creden...

Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18230 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15326: [SPARK-17759] [CORE] Avoid adding duplicate schedulables

Github user erenavsarogullari commented on the issue: https://github.com/apache/spark/pull/15326 Hi @HyukjinKwon, thanks for the following this PR again. This looks required but i am too busy for a while. Fix is already ready and will address the last comments asap. Sorry for delay

[GitHub] spark issue #18226: [SPARK-21006][TESTS] Create rpcEnv and run later needs s...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18226 **[Test build #77803 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77803/testReport)** for PR 18226 at commit

[GitHub] spark issue #18226: [SPARK-21006][TESTS] Create rpcEnv and run later needs s...

Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18226 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #18234: [SPARK-19185][DSTREAM] Make Kafka consumer cache configu...

Github user markgrover commented on the issue: https://github.com/apache/spark/pull/18234 Thanks all. On Jun 7, 2017 6:41 PM, "Cody Koeninger" wrote: > LGTM, thanks Mark > > — > You are receiving this because you authored the

[GitHub] spark issue #18234: [SPARK-19185][DSTREAM] Make Kafka consumer cache configu...

Github user koeninger commented on the issue: https://github.com/apache/spark/pull/18234 LGTM, thanks Mark --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18234: [SPARK-19185][DSTREAM] Make Kafka consumer cache configu...

Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18234 If there are no more comments I'll push this in the morning. I'd have filed a separate bug since now SPARK-19185 will forever be "in progress" (unless an admin sees this and changes its

[GitHub] spark issue #18226: [SPARK-21006][TESTS] Create rpcEnv and run later needs s...

Github user wangjiaochun commented on the issue: https://github.com/apache/spark/pull/18226 ok, I have re submit, Thanks for reviewing @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #18234: [SPARK-19185][DSTREAM] Make Kafka consumer cache configu...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18234 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18234: [SPARK-19185][DSTREAM] Make Kafka consumer cache configu...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18234 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77802/ Test PASSed. ---

[GitHub] spark issue #18234: [SPARK-19185][DSTREAM] Make Kafka consumer cache configu...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18234 **[Test build #77802 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77802/testReport)** for PR 18234 at commit

[GitHub] spark issue #18234: [SPARK-19185][DSTREAM] Make Kafka consumer cache configu...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18234 **[Test build #77802 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77802/testReport)** for PR 18234 at commit

[GitHub] spark pull request #18234: [SPARK-19185][DSTREAM] Make Kafka consumer cache ...

Github user markgrover commented on a diff in the pull request: https://github.com/apache/spark/pull/18234#discussion_r120778946 --- Diff: docs/streaming-kafka-0-10-integration.md --- @@ -91,7 +91,7 @@ The new Kafka consumer API will pre-fetch messages into buffers. Therefore it

[GitHub] spark pull request #17723: [SPARK-20434][YARN][CORE] Move kerberos delegatio...

Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17723#discussion_r120769140 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/security/YARNConfigurableCredentialManager.scala --- @@ -0,0 +1,87 @@ +/*

[GitHub] spark pull request #17723: [SPARK-20434][YARN][CORE] Move kerberos delegatio...

Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17723#discussion_r120760632 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/security/YARNConfigurableCredentialManagerSuite.scala --- @@ -0,0 +1,61 @@

[GitHub] spark pull request #17723: [SPARK-20434][YARN][CORE] Move kerberos delegatio...

Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17723#discussion_r120759448 --- Diff: core/src/test/scala/org/apache/spark/deploy/security/ConfigurableCredentialManagerSuite.scala --- @@ -104,7 +96,9 @@ class

[GitHub] spark pull request #17723: [SPARK-20434][YARN][CORE] Move kerberos delegatio...

Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17723#discussion_r120760412 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/security/YARNConfigurableCredentialManager.scala --- @@ -0,0 +1,87 @@ +/*

[GitHub] spark pull request #17723: [SPARK-20434][YARN][CORE] Move kerberos delegatio...

Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17723#discussion_r120760489 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/security/YARNConfigurableCredentialManagerSuite.scala --- @@ -0,0 +1,61 @@

[GitHub] spark pull request #17723: [SPARK-20434][YARN][CORE] Move kerberos delegatio...

Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17723#discussion_r120768942 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HiveCredentialProvider.scala --- @@ -0,0 +1,122 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #17723: [SPARK-20434][YARN][CORE] Move kerberos delegatio...

Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17723#discussion_r120758091 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/security/YARNConfigurableCredentialManager.scala --- @@ -0,0 +1,87 @@ +/*

[GitHub] spark pull request #17723: [SPARK-20434][YARN][CORE] Move kerberos delegatio...

Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17723#discussion_r120760010 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala --- @@ -103,6 +111,21 @@ class YarnSparkHadoopUtil

[GitHub] spark pull request #17723: [SPARK-20434][YARN][CORE] Move kerberos delegatio...

Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17723#discussion_r120760716 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/security/YARNConfigurableCredentialManagerSuite.scala --- @@ -0,0 +1,61 @@

[GitHub] spark pull request #17723: [SPARK-20434][YARN][CORE] Move kerberos delegatio...

Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17723#discussion_r120758198 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/security/YARNConfigurableCredentialManager.scala --- @@ -0,0 +1,87 @@ +/*

[GitHub] spark pull request #17723: [SPARK-20434][YARN][CORE] Move kerberos delegatio...

Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17723#discussion_r120760131 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala --- @@ -103,6 +111,21 @@ class YarnSparkHadoopUtil

[GitHub] spark pull request #17723: [SPARK-20434][YARN][CORE] Move kerberos delegatio...

Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17723#discussion_r120759036 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HiveCredentialProvider.scala --- @@ -0,0 +1,122 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18231: [WIP][SPARK-20994] Remove reduant characters in O...

Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18231#discussion_r120768040 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockHandler.java --- @@ -209,4 +190,47 @@ private

[GitHub] spark pull request #18231: [WIP][SPARK-20994] Remove reduant characters in O...

Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18231#discussion_r120767926 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockHandler.java --- @@ -209,4 +190,47 @@ private

[GitHub] spark issue #18004: [SPARK-18838][CORE] Introduce blocking strategy for Live...

Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18004 Sorry, forgot one comment. > With what you propose, the implementation will be much more complex and I think that it is not valuable yet while there is no concrete need of this mechanism.

[GitHub] spark issue #18220: [SPARK-21000][MESOS] Add Mesos labels support to the Spa...

Github user mgummelt commented on the issue: https://github.com/apache/spark/pull/18220 @srowen Can we get a merge? @ArtRand is a engineer working on Spark here at Mesosphere, and has approved these changes. Thanks. --- If your project is set up for it, you can reply to

[GitHub] spark pull request #18220: [SPARK-21000][MESOS] Add Mesos labels support to ...

Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/18220#discussion_r120764076 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosProtoUtils.scala --- @@ -0,0 +1,94 @@ +/* + *

[GitHub] spark issue #18004: [SPARK-18838][CORE] Introduce blocking strategy for Live...

Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18004 > But I clearly explain what I think is the best option for this potential issue Well, you stated your opinion, and I disagree with it. I've tried to explain why I disagree with it and

[GitHub] spark issue #18004: [SPARK-18838][CORE] Introduce blocking strategy for Live...

Github user bOOm-X commented on the issue: https://github.com/apache/spark/pull/18004 > Wait, which comment? You mention the issue with the external listener as I did not mention this case. But I clearly explain what I think is the best option for this potential issue: log very

[GitHub] spark issue #17373: [SPARK-12664] Expose probability in mlp model

Github user alwaysprep commented on the issue: https://github.com/apache/spark/pull/17373 In which version this is going to be available on PySpark? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #18221: [SPARK-20655][core] In-memory KVStore implementation.

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18221 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77801/ Test PASSed. ---

[GitHub] spark issue #18221: [SPARK-20655][core] In-memory KVStore implementation.

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18221 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18221: [SPARK-20655][core] In-memory KVStore implementation.

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18221 **[Test build #77801 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77801/testReport)** for PR 18221 at commit

[GitHub] spark issue #11887: [SPARK-13041][Mesos]add driver sandbox uri to the dispat...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/11887 @skonto Sure, I would not propose to close. Thank you for your input. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #11887: [SPARK-13041][Mesos]add driver sandbox uri to the dispat...

Github user skonto commented on the issue: https://github.com/apache/spark/pull/11887 @HyukjinKwon I will have a look and let you know, please don't close it for now. There was finally progress at the mesos side: https://reviews.apache.org/r/58872/ --- If your project is

[GitHub] spark issue #17645: [SPARK-20348] [ML] Support squared hinge loss (L2 loss) ...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17645 I took out this in the list. Though, should we maybe close this for now and reopen again when it's ready if it takes quite long? It'd be probably better than leaving this open without further

[GitHub] spark issue #18223: [INFRA] Close stale PRs

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18223 I took out 17645 per https://github.com/apache/spark/pull/17645#issuecomment-306907150 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #18233: [SPARK-20342][core] Update task accumulators before send...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18233 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77798/ Test PASSed. ---

[GitHub] spark issue #18233: [SPARK-20342][core] Update task accumulators before send...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18233 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18233: [SPARK-20342][core] Update task accumulators before send...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18233 **[Test build #77798 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77798/testReport)** for PR 18233 at commit

[GitHub] spark issue #17645: [SPARK-20348] [ML] Support squared hinge loss (L2 loss) ...

Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/17645 Hi @HyukjinKwon I think this is a feature we need, but currently we are still having some discussion about optimizer interface. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #18200: [SPARK-20978][SQL] Set null for malformed column ...

Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18200#discussion_r120724004 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -53,7 +53,8 @@ class UnivocityParser(

[GitHub] spark pull request #18200: [SPARK-20978][SQL] Set null for malformed column ...

Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18200#discussion_r120723655 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -53,7 +53,8 @@ class UnivocityParser(

[GitHub] spark issue #18223: [INFRA] Close stale PRs

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18223 Sure. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #18200: [SPARK-20978][SQL] Set null for malformed column ...

Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18200#discussion_r120720274 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -53,7 +53,8 @@ class UnivocityParser(

[GitHub] spark issue #18223: [INFRA] Close stale PRs

Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18223 @HyukjinKwon How about keep #17716 open? I think we still need this and Herman will continue working on this. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request #18234: [SPARK-19185][DSTREAM] Make Kafka consumer cache ...

Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/18234#discussion_r120716157 --- Diff: docs/streaming-kafka-0-10-integration.md --- @@ -91,7 +91,7 @@ The new Kafka consumer API will pre-fetch messages into buffers. Therefore it i

[GitHub] spark issue #18223: [INFRA] Close stale PRs

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18223 #18130 and #18044 are duplicates. I added the others assuming they are inactive or reviewed. Just to make sure, https://github.com/apache/spark/pull/12835 was also reviewed and then

[GitHub] spark issue #18223: [INFRA] Close stale PRs

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18223 #18130, #18044 are duplicates. I added all assuming they are inactive or reviewed. Just to make sure, https://github.com/apache/spark/pull/12835 was also reviewed and then suggested to

[GitHub] spark issue #18118: SPARK-20199 : Provided featureSubsetStrategy to GBTClass...

Github user sethah commented on the issue: https://github.com/apache/spark/pull/18118 I don't think there's any point in pinging every day :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #18004: [SPARK-18838][CORE] Introduce blocking strategy for Live...

Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18004 > Honestly, your last comment is so unfair. Wait, which comment? I pointed out places where you current design has limitations. I'm asking for a way to make it easy to create asynchronous

[GitHub] spark issue #18004: [SPARK-18838][CORE] Introduce blocking strategy for Live...

Github user bOOm-X commented on the issue: https://github.com/apache/spark/pull/18004 Honestly, your last comment is so unfair. You commented on a simple phrase out of its context as I did not mentioned what is in my mind the reasonable first step for the external listeners. For the

[GitHub] spark pull request #17882: [SPARK-20079][yarn] Re registration of AM hangs s...

Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17882#discussion_r120710864 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala --- @@ -307,6 +301,9 @@ private[spark]

[GitHub] spark pull request #18234: [SPARK-19185][DSTREAM] Make Kafka consumer cache ...

Github user markgrover commented on a diff in the pull request: https://github.com/apache/spark/pull/18234#discussion_r120703106 --- Diff: docs/streaming-kafka-0-10-integration.md --- @@ -91,7 +91,7 @@ The new Kafka consumer API will pre-fetch messages into buffers. Therefore it

[GitHub] spark pull request #18234: [SPARK-19185][DSTREAM] Make Kafka consumer cache ...

Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/18234#discussion_r120701187 --- Diff: docs/streaming-kafka-0-10-integration.md --- @@ -91,7 +91,7 @@ The new Kafka consumer API will pre-fetch messages into buffers. Therefore it i

[GitHub] spark issue #18221: [SPARK-20655][core] In-memory KVStore implementation.

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18221 **[Test build #77801 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77801/testReport)** for PR 18221 at commit

[GitHub] spark issue #18234: [SPARK-19185][DSTREAM] Make Kafka consumer cache configu...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18234 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77799/ Test PASSed. ---

[GitHub] spark issue #18234: [SPARK-19185][DSTREAM] Make Kafka consumer cache configu...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18234 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18234: [SPARK-19185][DSTREAM] Make Kafka consumer cache configu...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18234 **[Test build #77799 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77799/testReport)** for PR 18234 at commit

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

Github user mallman commented on the issue: https://github.com/apache/spark/pull/16578 Hmmm... this failed again for the same reason. I'll see if I can reproduce locally. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #18223: [INFRA] Close stale PRs

Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18223 Add: #12835 #17141 #18044 #18130 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #18223: [INFRA] Close stale PRs

Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18223 Add: #16291 #17480 #14995 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17694: [SPARK-12717][PYSPARK] Resolving race condition with pys...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17694 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17694: [SPARK-12717][PYSPARK] Resolving race condition with pys...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17694 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77800/ Test FAILed. ---

[GitHub] spark issue #17694: [SPARK-12717][PYSPARK] Resolving race condition with pys...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17694 **[Test build #77800 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77800/testReport)** for PR 17694 at commit

[GitHub] spark issue #17694: [SPARK-12717][PYSPARK] Resolving race condition with pys...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17694 **[Test build #77800 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77800/testReport)** for PR 17694 at commit

[GitHub] spark issue #18234: [SPARK-19185][DSTREAM] Make Kafka consumer cache configu...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18234 **[Test build #77799 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77799/testReport)** for PR 18234 at commit

[GitHub] spark issue #18234: [SPARK-19185][DSTREAM] Make Kafka consumer cache configu...

Github user markgrover commented on the issue: https://github.com/apache/spark/pull/18234 This is related to but is a stripped down version of #16629. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #17694: [SPARK-12717][PYSPARK] Resolving race condition with pys...

Github user vanzin commented on the issue: https://github.com/apache/spark/pull/17694 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #18234: [SPARK-19185][DSTREAM] Make Kafka consumer cache ...

GitHub user markgrover opened a pull request: https://github.com/apache/spark/pull/18234 [SPARK-19185][DSTREAM] Make Kafka consumer cache configurable ## What changes were proposed in this pull request? Add a new property `spark.streaming.kafka.consumer.cache.enabled` that

[GitHub] spark issue #18233: [SPARK-20342][core] Update task accumulators before send...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18233 **[Test build #77798 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77798/testReport)** for PR 18233 at commit

[GitHub] spark pull request #18233: [SPARK-20342][core] Update task accumulators befo...

GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/18233 [SPARK-20342][core] Update task accumulators before sending task end event. This makes sures that listeners get updated task information; otherwise it's possible to write incomplete task

[GitHub] spark issue #17882: [SPARK-20079][yarn] Re registration of AM hangs spark cl...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17882 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77797/ Test FAILed. ---

  1   2   >