[GitHub] spark pull request #21856: [SPARK-24738] [HistoryServer] FsHistoryProvider c...

2018-11-04 Thread LiShuMing
Github user LiShuMing closed the pull request at: https://github.com/apache/spark/pull/21856 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21856: [SPARK-24738] [HistoryServer] FsHistoryProvider c...

2018-07-24 Thread LiShuMing
GitHub user LiShuMing opened a pull request: https://github.com/apache/spark/pull/21856 [SPARK-24738] [HistoryServer] FsHistoryProvider clean outdated event … Now FsHistoryProvider's policy : 1. FsHistoryProvider create checkEvent thread and cleanLog thread, they

[GitHub] spark pull request #18659: [SPARK-21190][PYSPARK][WIP] Python Vectorized UDF...

2017-09-20 Thread LiShuMing
Github user LiShuMing commented on a diff in the pull request: https://github.com/apache/spark/pull/18659#discussion_r139947140 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/EvalPythonExec.scala --- @@ -0,0 +1,142 @@ +/* + * Licensed to the

[GitHub] spark pull request #18659: [SPARK-21190][PYSPARK][WIP] Python Vectorized UDF...

2017-09-20 Thread LiShuMing
Github user LiShuMing commented on a diff in the pull request: https://github.com/apache/spark/pull/18659#discussion_r139942881 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/EvalPythonExec.scala --- @@ -0,0 +1,142 @@ +/* + * Licensed to the

[GitHub] spark issue #19052: [Doc] ExchangeCoordinator description num error

2017-08-25 Thread LiShuMing
Github user LiShuMing commented on the issue: https://github.com/apache/spark/pull/19052 Sorry, don't pull this request. I will close this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #19052: [Doc] ExchangeCoordinator description num error

2017-08-25 Thread LiShuMing
Github user LiShuMing closed the pull request at: https://github.com/apache/spark/pull/19052 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #19052: [Doc] ExchangeCoordinator description num error

2017-08-25 Thread LiShuMing
GitHub user LiShuMing opened a pull request: https://github.com/apache/spark/pull/19052 [Doc] ExchangeCoordinator description num error ## What changes were proposed in this pull request? In `ExchangeCoordinator.scala` , this maybe just a mistake occasionally which is

[GitHub] spark issue #18905: [SPARK-21660][YARN][Shuffle] Yarn ShuffleService failed ...

2017-08-23 Thread LiShuMing
Github user LiShuMing commented on the issue: https://github.com/apache/spark/pull/18905 See another approach to solve this problem: https://github.com/apache/spark/pull/19032 and I will close this pr. Thanks @jerryshao @tgravescs . --- If your project is set up for it

[GitHub] spark pull request #18905: [SPARK-21660][YARN][Shuffle] Yarn ShuffleService ...

2017-08-23 Thread LiShuMing
Github user LiShuMing closed the pull request at: https://github.com/apache/spark/pull/18905 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18905: [SPARK-21660][YARN][Shuffle] Yarn ShuffleService failed ...

2017-08-19 Thread LiShuMing
Github user LiShuMing commented on the issue: https://github.com/apache/spark/pull/18905 ping @jerryshao I found a method to check disk in hadoop: https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util

[GitHub] spark issue #18905: [SPARK-21660][YARN][Shuffle] Yarn ShuffleService failed ...

2017-08-18 Thread LiShuMing
Github user LiShuMing commented on the issue: https://github.com/apache/spark/pull/18905 Sorry, busy recently, I will update it today... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18905: [SPARK-21660][YARN][Shuffle] Yarn ShuffleService failed ...

2017-08-14 Thread LiShuMing
Github user LiShuMing commented on the issue: https://github.com/apache/spark/pull/18905 @jerryshao Thanks for your replies! I will do such things then: 1. "it is good to change to other directories (is yarn internally relying on it)?" I think the recovery

[GitHub] spark pull request #18905: [SPARK-21660] [YARN] [Shuffle] Yarn ShuffleServic...

2017-08-10 Thread LiShuMing
GitHub user LiShuMing opened a pull request: https://github.com/apache/spark/pull/18905 [SPARK-21660] [YARN] [Shuffle] Yarn ShuffleService failed to start when the chosen dir… ## What changes were proposed in this pull request? See [SPARK-21660](https

[GitHub] spark pull request #18130: [Web UI] Remove no need loop in JobProgressListen...

2017-05-27 Thread LiShuMing
GitHub user LiShuMing opened a pull request: https://github.com/apache/spark/pull/18130 [Web UI] Remove no need loop in JobProgressListener ## What changes were proposed in this pull request? I found it's unnecessary to use the same loop twice, so changed them into