[GitHub] spark pull request #19466: [SPARK-22237] [CORE] Fix spark submit file downlo...

2017-10-11 Thread loneknightpy
Github user loneknightpy commented on a diff in the pull request: https://github.com/apache/spark/pull/19466#discussion_r144100747 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -366,6 +366,16 @@ object SparkSubmit extends CommandLineUtils

[GitHub] spark pull request #19466: [SPARK-22237] [CORE] Fix spark submit file downlo...

2017-10-11 Thread loneknightpy
Github user loneknightpy closed the pull request at: https://github.com/apache/spark/pull/19466 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #19466: [SPARK-22237] [CORE] Fix spark submit file downlo...

2017-10-10 Thread loneknightpy
Github user loneknightpy commented on a diff in the pull request: https://github.com/apache/spark/pull/19466#discussion_r143883926 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -366,6 +366,16 @@ object SparkSubmit extends CommandLineUtils

[GitHub] spark pull request #19466: [CORE] Fix spark submit file download for standal...

2017-10-10 Thread loneknightpy
GitHub user loneknightpy opened a pull request: https://github.com/apache/spark/pull/19466 [CORE] Fix spark submit file download for standalone client mode ## What changes were proposed in this pull request? This PR makes spark-submit script to use downloaded files

[GitHub] spark pull request #19074: [SPARK-21714][CORE][BACKPORT-2.2] Avoiding re-upl...

2017-09-15 Thread loneknightpy
Github user loneknightpy commented on a diff in the pull request: https://github.com/apache/spark/pull/19074#discussion_r139248521 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -311,12 +317,16 @@ object SparkSubmit extends CommandLineUtils

[GitHub] spark pull request #19074: [SPARK-21714][CORE][BACKPORT-2.2] Avoiding re-upl...

2017-09-15 Thread loneknightpy
Github user loneknightpy commented on a diff in the pull request: https://github.com/apache/spark/pull/19074#discussion_r139247692 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -366,7 +376,7 @@ object SparkSubmit extends CommandLineUtils

[GitHub] spark pull request #19074: [SPARK-21714][CORE][BACKPORT-2.2] Avoiding re-upl...

2017-09-15 Thread loneknightpy
Github user loneknightpy commented on a diff in the pull request: https://github.com/apache/spark/pull/19074#discussion_r139248109 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -376,8 +386,8 @@ object SparkSubmit extends CommandLineUtils

[GitHub] spark issue #18078: [SPARK-10643] [Core] Make spark-submit download remote f...

2017-05-26 Thread loneknightpy
Github user loneknightpy commented on the issue: https://github.com/apache/spark/pull/18078 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #18078: [SPARK-10643] [Core] Make spark-submit download r...

2017-05-26 Thread loneknightpy
Github user loneknightpy commented on a diff in the pull request: https://github.com/apache/spark/pull/18078#discussion_r118641792 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -825,6 +837,43 @@ object SparkSubmit extends CommandLineUtils

[GitHub] spark pull request #18078: [SPARK-10643] [Core] Make spark-submit download r...

2017-05-25 Thread loneknightpy
Github user loneknightpy commented on a diff in the pull request: https://github.com/apache/spark/pull/18078#discussion_r118627131 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -308,6 +311,15 @@ object SparkSubmit extends CommandLineUtils

[GitHub] spark pull request #18078: [SPARK-10643] Make spark-submit download remote f...

2017-05-25 Thread loneknightpy
Github user loneknightpy commented on a diff in the pull request: https://github.com/apache/spark/pull/18078#discussion_r118583624 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala --- @@ -535,7 +538,7 @@ class SparkSubmitSuite test

[GitHub] spark pull request #18078: [SPARK-20860] Make spark-submit download remote f...

2017-05-23 Thread loneknightpy
GitHub user loneknightpy opened a pull request: https://github.com/apache/spark/pull/18078 [SPARK-20860] Make spark-submit download remote files to local in client mode ## What changes were proposed in this pull request? This PR makes spark-submit script download remote

[GitHub] spark pull request #15537: [SPARK-17711] [test-hadoop2.2] Fix hadoop2.2 comp...

2016-10-18 Thread loneknightpy
GitHub user loneknightpy opened a pull request: https://github.com/apache/spark/pull/15537 [SPARK-17711] [test-hadoop2.2] Fix hadoop2.2 compilation error ## What changes were proposed in this pull request? Fix hadoop2.2 compilation error. ## How was this patch

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-17 Thread loneknightpy
Github user loneknightpy commented on the issue: https://github.com/apache/spark/pull/15285 @tdas Addressed your comments, please take a look. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-13 Thread loneknightpy
Github user loneknightpy commented on the issue: https://github.com/apache/spark/pull/15285 @tdas Add `fileUncompressedLengthCacheSize` to executor conf. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-12 Thread loneknightpy
Github user loneknightpy commented on the issue: https://github.com/apache/spark/pull/15285 @tdas Based on our offline discussion, I added file size cache for the compressed log files. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-10 Thread loneknightpy
Github user loneknightpy commented on the issue: https://github.com/apache/spark/pull/15285 @tdas Addressed your comments --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-05 Thread loneknightpy
Github user loneknightpy commented on the issue: https://github.com/apache/spark/pull/15285 ping @tdas --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-09-29 Thread loneknightpy
Github user loneknightpy commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r81202273 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1455,7 +1456,11 @@ private[spark] object Utils extends Logging { val

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-09-29 Thread loneknightpy
Github user loneknightpy commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r81201000 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1455,7 +1456,11 @@ private[spark] object Utils extends Logging { val

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-09-29 Thread loneknightpy
Github user loneknightpy commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r81200830 --- Diff: core/src/main/scala/org/apache/spark/util/logging/RollingFileAppender.scala --- @@ -76,15 +79,40 @@ private[spark] class RollingFileAppender

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-09-29 Thread loneknightpy
Github user loneknightpy commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r81200190 --- Diff: core/src/main/scala/org/apache/spark/util/logging/RollingFileAppender.scala --- @@ -97,11 +125,11 @@ private[spark] class RollingFileAppender

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-09-28 Thread loneknightpy
GitHub user loneknightpy opened a pull request: https://github.com/apache/spark/pull/15285 [SPARK-17711] Compress rolled executor log ## What changes were proposed in this pull request? This PR adds support for executor log compression. ## How was this patch