[GitHub] spark pull request: [SPARK-3636][CORE]:It is not friendly to inter...

2014-09-22 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/2488 [SPARK-3636][CORE]:It is not friendly to interrupt a Job when user passe... ... different storageLevels to a RDD You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: [SPARK-3636][CORE]:It is not friendly to inter...

2014-09-23 Thread uncleGen
Github user uncleGen closed the pull request at: https://github.com/apache/spark/pull/2488 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-3636][CORE]:It is not friendly to inter...

2014-09-23 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/2488#issuecomment-56505578 @pwendell ah, make sense, I will close this PR. Thank you! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-1774] Respect SparkSubmit --jars on YAR...

2014-09-23 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/710#discussion_r17951094 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala --- @@ -192,15 +236,17 @@ class SparkSubmitSuite extends FunSuite

[GitHub] spark pull request: [SPARK-3712][STREAMING]: add a new UpdateDStre...

2014-09-28 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/2562 [SPARK-3712][STREAMING]: add a new UpdateDStream to update a rdd dynamically Maybe, we can achieve the aim by using forEachRdd function. But it is weird in this way, because I need to pass

[GitHub] spark pull request: [SPARK-3712][STREAMING]: add a new UpdateDStre...

2014-09-28 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/2562#issuecomment-57083416 Test failure appears to be unrelated to my patch. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [SPARK-3712][STREAMING]: add a new UpdateDStre...

2014-09-28 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/2562#issuecomment-57087576 @jerryshao Thanks for your comments! I want to abstract an independent DStream to achieve the aim. I feel it is weird to update a rdd by passing a closure. Maybe

[GitHub] spark pull request: [SPARK-3712][STREAMING]: add a new UpdateDStre...

2014-09-28 Thread uncleGen
Github user uncleGen closed the pull request at: https://github.com/apache/spark/pull/2562 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-3719][CORE]:complete/failed stages is...

2014-09-29 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/2574 [SPARK-3719][CORE]:complete/failed stages is better to show the total ... ...number of stages You can merge this pull request into a Git repository by running: $ git pull https://github.com

[GitHub] spark pull request: [SPARK-2506]: In yarn-cluster mode, Applicatio...

2014-07-22 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/1429#issuecomment-49751553 @tgravescs ok, and any suggestions? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: Spark Shuffle:use growth rate to predict if ...

2014-07-31 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/1696 Spark Shuffle:use growth rate to predict if need to spill You can merge this pull request into a Git repository by running: $ git pull https://github.com/uncleGen/spark master

[GitHub] spark pull request: [SPARK-2773]use growth rate to predict if need...

2014-08-02 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/1696#issuecomment-50958278 @rxin Thanks for your attention, I have updated my jira. https://issues.apache.org/jira/browse/SPARK-2773 --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-3719][CORE]:complete/failed stages is...

2014-10-07 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/2574#discussion_r18561760 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobProgressPage.scala --- @@ -70,11 +72,11 @@ private[ui] class JobProgressPage(parent

[GitHub] spark pull request: [SPARK-3719][CORE]:complete/failed stages is...

2014-10-08 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/2574#discussion_r18622733 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobProgressPage.scala --- @@ -70,11 +72,11 @@ private[ui] class JobProgressPage(parent

[GitHub] spark pull request: [SPARK-3719][CORE]:complete/failed stages is...

2014-10-10 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/2574#issuecomment-58735669 @JoshRosen Sorry for my misunderstanding, I will correct it as soon as possible. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-3818] Graph coarsening

2014-10-13 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/2679#issuecomment-58885039 @ankurdave I have some doubts, but not about this patch. In [GraphX OSDI paper](http://ankurdave.com/dl/graphx-osdi14.pdf) , I find that you have implemented a memory

[GitHub] spark pull request: [SPARK-3818] Graph coarsening

2014-10-14 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/2679#issuecomment-59037906 @ankurdave I see. And I think it is worthy to provide a memory-based shuffle manager in some cases, like sufficient memory resources, stringent performance requirement

[GitHub] spark pull request: Bug Fix: LiveListenerBus Queue Overflow

2014-07-10 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/1356 Bug Fix: LiveListenerBus Queue Overflow As we know, the size of eventQueue is fixed. When event comes faster than consume speed of listener, overflow events will be thrown away with throwing

[GitHub] spark pull request: Bug Fix: LiveListenerBus Queue Overflow

2014-07-10 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/1356#issuecomment-48691344 @pwendell yeah, this is not a handsome way to resolve the bug. My fix is a compromised way. Actually, there are no frequent get/put opertions in blockManager when

[GitHub] spark pull request: Bug Fix: LiveListenerBus Queue Overflow

2014-07-10 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/1356#issuecomment-48691872 @pwendell yeah, it is not a handsome way to resolve the bug. My fix is a compromise way. Actualy, is will not cause frequent put/get opertions in blockManager when

[GitHub] spark pull request: [GraphX]: override the setName function to s...

2014-08-19 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/2033 [GraphX]: override the setName function to set EdgeRDD's name manually just as VertexRDD does. You can merge this pull request into a Git repository by running: $ git pull https

[GitHub] spark pull request: [SPARK-3170][CORE]: Bug Fix in Storage UI

2014-08-20 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/2076 [SPARK-3170][CORE]: Bug Fix in Storage UI current compeleted stage only need to remove its own partitions that are no longer cached. Currently, Storage in Spark UI may lost some rdds which

[GitHub] spark pull request: [SPARK-3170][CORE]: Bug Fix in Storage UI

2014-08-21 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/2076#issuecomment-52908740 @srowen yes! Not only in StorageTab, ExectutorTab may also lose some rdd-infos which have been overwritten by following rdd in a same task. StorageTab: when

[GitHub] spark pull request: [SPARK-3170][CORE]: Bug Fix in Storage UI

2014-08-23 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/2076#issuecomment-53144395 @pwendell Okay! I will add them as soon as possible and pay more attention. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-3170][CORE][BUG]:RDD info loss in Stor...

2014-08-26 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/2131 [SPARK-3170][CORE][BUG]:RDD info loss in StorageTab and ExecutorTab compeleted stage only need to remove its own partitions that are no longer cached. However, StorageTab may lost some rdds which

[GitHub] spark pull request: [SPARK-3170][CORE]: Bug Fix in Storage UI

2014-08-26 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/2076#issuecomment-53383270 @andrewor14 @pwendell @srowen As my branch is not up to date, I decide to close this and submit a new PR. Please Review It : https://github.com/apache/spark

[GitHub] spark pull request: [SPARK-3170][CORE]: Bug Fix in Storage UI

2014-08-26 Thread uncleGen
Github user uncleGen closed the pull request at: https://github.com/apache/spark/pull/2076 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-3170][CORE][BUG]:RDD info loss in Stor...

2014-08-26 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/2131#issuecomment-53521662 Hi @andrewor14, test it again please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-3170][CORE][BUG]:RDD info loss in Stor...

2014-08-27 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/2131#discussion_r16761636 --- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala --- @@ -68,7 +68,9 @@ private[spark] class CacheManager(blockManager: BlockManager

[GitHub] spark pull request: [SPARK-3170][CORE][BUG]:RDD info loss in Stor...

2014-08-27 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/2131#discussion_r16762096 --- Diff: core/src/test/scala/org/apache/spark/CacheManagerSuite.scala --- @@ -87,4 +99,12 @@ class CacheManagerSuite extends FunSuite with BeforeAndAfter

[GitHub] spark pull request: [SPARK-3170][CORE][BUG]:RDD info loss in Stor...

2014-08-27 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/2131#issuecomment-53549872 @andrewor14 sorry for my poor coding. Unit test passed locally, test it again pls. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-2675]LiveListenerBus Queue Overflow

2014-08-30 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/1356#issuecomment-53977557 okay! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-2773]use growth rate to predict if need...

2014-09-01 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/1696#issuecomment-54102947 @pwendell OK! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2506]: In yarn-cluster mode, Applicatio...

2014-09-02 Thread uncleGen
Github user uncleGen closed the pull request at: https://github.com/apache/spark/pull/1429 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [GraphX]: trim some useless informations of Ve...

2014-09-03 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/2249 [GraphX]: trim some useless informations of VertexRDD in some cases You can merge this pull request into a Git repository by running: $ git pull https://github.com/uncleGen/spark

[GitHub] spark pull request: [SPARK-3373][GraphX]: trim some useless inform...

2014-09-03 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/2249#issuecomment-54399143 @ankurdave Thanks for you comments, I will update it as soon as possible. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-3373][GraphX]: trim some useless inform...

2014-09-03 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/2249#discussion_r17095835 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/Graph.scala --- @@ -262,13 +262,61 @@ abstract class Graph[VD: ClassTag, ED: ClassTag] protected

[GitHub] spark pull request: [SPARK-4488][PySpark] Add control over map-sid...

2014-11-19 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/3365 [SPARK-4488][PySpark] Add control over map-side aggregation You can merge this pull request into a Git repository by running: $ git pull https://github.com/uncleGen/spark master-clean-141119

[GitHub] spark pull request: [SPARK-4488][PySpark] Add control over map-sid...

2014-11-19 Thread uncleGen
Github user uncleGen closed the pull request at: https://github.com/apache/spark/pull/3365 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-4488][PySpark] Add control over map-sid...

2014-11-19 Thread uncleGen
GitHub user uncleGen reopened a pull request: https://github.com/apache/spark/pull/3365 [SPARK-4488][PySpark] Add control over map-side aggregation You can merge this pull request into a Git repository by running: $ git pull https://github.com/uncleGen/spark master-clean

[GitHub] spark pull request: [SPARK-4488][PySpark] Add control over map-sid...

2014-11-19 Thread uncleGen
Github user uncleGen closed the pull request at: https://github.com/apache/spark/pull/3365 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-4488][PySpark] Add control over map-sid...

2014-11-19 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/3366 [SPARK-4488][PySpark] Add control over map-side aggregation You can merge this pull request into a Git repository by running: $ git pull https://github.com/uncleGen/spark master-pyspark

[GitHub] spark pull request: [SPARK-3719][CORE][UI]:complete/failed stages...

2014-11-19 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/2574#issuecomment-63651117 @JoshRosen [[SPARK-4168][WebUI] ](https://github.com/apache/spark/commit/97a466eca0a629f17e9662ca2b59eeca99142c54) The patch solved the same problem, and I will close

[GitHub] spark pull request: [SPARK-3719][CORE][UI]:complete/failed stages...

2014-11-19 Thread uncleGen
Github user uncleGen closed the pull request at: https://github.com/apache/spark/pull/2574 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-3373][GraphX]: Filtering operations sho...

2014-11-19 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/2249#issuecomment-63654065 @ankurdave Hi, can you review it again. Thank you! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [SPARK-4488][PySpark] Add control over map-sid...

2014-11-20 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/3366#issuecomment-63831692 @davies Could you help reviewing this patch? Thank you! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [SPARK-4920][UI]:current spark version in UI i...

2014-12-22 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/3763 [SPARK-4920][UI]:current spark version in UI is not striking. It is not convenient to see the Spark version. We can keep the same style with Spark website. !https

[GitHub] spark pull request: [SPARK-4920][UI]: back port the PR-3763 to bra...

2014-12-22 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/3768 [SPARK-4920][UI]: back port the PR-3763 to branch 1.1 You can merge this pull request into a Git repository by running: $ git pull https://github.com/uncleGen/spark branch-1.1-1223

[GitHub] spark pull request: [SPARK-4920][UI]: back port the PR-3763 to bra...

2014-12-22 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/3768#issuecomment-67926060 There are two irrelevant test failures. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-01-15 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/4008#discussion_r23064794 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -23,10 +23,20 @@ import scala.language.existentials

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-01-15 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/4008#discussion_r23064804 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -274,6 +284,7 @@ class ReceiverTracker(ssc

[GitHub] spark pull request: [SPARK-4920][UI]: back port the PR-3763 to bra...

2015-01-18 Thread uncleGen
Github user uncleGen closed the pull request at: https://github.com/apache/spark/pull/3768 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-4920][UI]: back port the PR-3763 to bra...

2015-01-15 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/3768#issuecomment-70198185 @JoshRosen OK, thank you! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-01-21 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/4135 [SPARK-5205][Streaming]:Inconsistent behaviour between Streaming job and others, when click kill link in WebUI The kill link is used to kill a stage in job. It works in any kinds of Spark job

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-01-21 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4008#issuecomment-70804703 @JoshRosen Since my local branch is out of date and this PR contains merge conflicts, I will open a [new PR](https://github.com/apache/spark/pull/4135) and close

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-01-21 Thread uncleGen
Github user uncleGen closed the pull request at: https://github.com/apache/spark/pull/4008 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...

2015-02-11 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/4522#discussion_r24495488 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala --- @@ -413,10 +413,13 @@ private[spark] class SparkSubmitArguments(args

[GitHub] spark pull request: [SPARK-5461] [graphx] Add isCheckpointed, getC...

2015-02-02 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4253#issuecomment-72421951 @jkbradley IMHO, we do not need to override the `isCheckpointed()` in EdgeRDDImpl and VertexRDDImpl, and only define a normal `isCheckpointed()`. IIUC, the func

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-01-12 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/4008 [SPARK-5205][Streaming]:Inconsistent behaviour between Streaming job and others, when click kill link in WebUI The kill link is used to kill a stage in job. It works in any kinds of Spark job

[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...

2015-01-06 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/3912#discussion_r22572535 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/receiver/ReceiverSupervisorImpl.scala --- @@ -73,14 +73,16 @@ private[streaming] class

[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...

2015-01-06 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/3912#discussion_r22572546 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -138,7 +140,17 @@ class ReceiverTracker(ssc

[GitHub] spark pull request: [SPARK-5131][Streaming][DOC]: There is a discr...

2015-01-07 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/3930 [SPARK-5131][Streaming][DOC]: There is a discrepancy in WAL implementation and configuration doc. There is a discrepancy in WAL implementation and configuration doc. You can merge this pull

[GitHub] spark pull request: [SPARK-5131][Streaming][DOC]: There is a discr...

2015-01-07 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/3930#issuecomment-68996611 @srowen oops, I missed. . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: Distinguish the ...

2015-01-07 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/3912#discussion_r22577352 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -137,8 +141,24 @@ class ReceiverTracker(ssc

[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: Distinguish the ...

2015-01-07 Thread uncleGen
Github user uncleGen closed the pull request at: https://github.com/apache/spark/pull/3912 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: Distinguish the ...

2015-01-07 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/3912#discussion_r22575229 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -137,8 +141,24 @@ class ReceiverTracker(ssc

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-01-12 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4008#issuecomment-69680034 @JoshRosen Thanks for your patience, I will optimize my code based on your comments. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-01-13 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4008#issuecomment-69719446 @JoshRosen how to fix these two errors? According to the hint, I need to add ProblemFilters.exclude[MissingMethodProblem

[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...

2015-02-10 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4522#issuecomment-73839883 An irrelevant test failure in `DirectKafkaStreamSuite` introduced by [PR](4384) --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...

2015-02-10 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/4522 [SPARK-5732][CORE]:Add an option to print the spark version in spark script. Naturally, we need to add an option to print the spark version in spark script. It is pretty common in script tool

[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...

2015-02-11 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/4522#discussion_r24490981 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala --- @@ -413,10 +413,13 @@ private[spark] class SparkSubmitArguments(args

[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...

2015-02-11 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/4522#discussion_r24491713 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala --- @@ -413,10 +413,13 @@ private[spark] class SparkSubmitArguments(args

[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...

2015-01-06 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/3912 [SPARK-5107][Streaming][Log]: A trick log info for the start of Receiver Receiver will register itself whenever it begins to start. But, it is trick to log the same information. Especially

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-18 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4135#issuecomment-82844745 @JoshRosen Could you please take a look again, thank you! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [SPARK-3373][GraphX]: Filtering operations sho...

2015-03-16 Thread uncleGen
Github user uncleGen closed the pull request at: https://github.com/apache/spark/pull/2249 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-17 Thread uncleGen
Github user uncleGen closed the pull request at: https://github.com/apache/spark/pull/4135 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-17 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4135#issuecomment-82363317 close and resolve the merge conflict. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-17 Thread uncleGen
GitHub user uncleGen reopened a pull request: https://github.com/apache/spark/pull/4135 [SPARK-5205][Streaming]:Inconsistent behaviour between Streaming job and others, when click kill link in WebUI The kill link is used to kill a stage in job. It works in any kinds of Spark job

[GitHub] spark pull request: [SPARK-3373][GraphX]: Filtering operations sho...

2015-03-15 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/2249#issuecomment-81433144 @ankurdave This PR has gone stale. Since GraphX has graduated from alpha, do we need to close this? --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-10 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4135#issuecomment-78000658 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-05 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4135#issuecomment-77514182 @pwendell Could you please take a look again, thank you! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-10 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4135#issuecomment-78201163 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-12 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4135#issuecomment-78468550 @JoshRosen, thanks for your patience. It occurred to me that we may check when to terminate the `receiver` in `ReceiverSupervisor`. Then the condition to stop

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-12 Thread uncleGen
Github user uncleGen closed the pull request at: https://github.com/apache/spark/pull/4135 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-12 Thread uncleGen
GitHub user uncleGen reopened a pull request: https://github.com/apache/spark/pull/4135 [SPARK-5205][Streaming]:Inconsistent behaviour between Streaming job and others, when click kill link in WebUI The kill link is used to kill a stage in job. It works in any kinds of Spark job

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-12 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4135#issuecomment-78736965 @JoshRosen OK, I will roll back to the original approach and do some impromvements :) --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-10 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4135#issuecomment-78015735 Some timeout errors happened. Could you please take a time to check and review it again? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-30 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4135#issuecomment-87908164 @JoshRosen Sorry for my laziness. I will update it as soon as possible --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-31 Thread uncleGen
Github user uncleGen closed the pull request at: https://github.com/apache/spark/pull/4135 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-31 Thread uncleGen
GitHub user uncleGen reopened a pull request: https://github.com/apache/spark/pull/4135 [SPARK-5205][Streaming]:Inconsistent behaviour between Streaming job and others, when click kill link in WebUI The kill link is used to kill a stage in job. It works in any kinds of Spark job

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-31 Thread uncleGen
Github user uncleGen closed the pull request at: https://github.com/apache/spark/pull/4135 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-31 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4135#issuecomment-87967820 @JoshRosen Your comments are reasonable, and I have improved related code just as what you pointed. For the test suite, I just check if the state of `Receiver

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-31 Thread uncleGen
GitHub user uncleGen reopened a pull request: https://github.com/apache/spark/pull/4135 [SPARK-5205][Streaming]:Inconsistent behaviour between Streaming job and others, when click kill link in WebUI The kill link is used to kill a stage in job. It works in any kinds of Spark job

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-31 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4135#issuecomment-88013418 wait for a moment --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-31 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4135#issuecomment-88342325 @JoshRosen Your comments are reasonable, and I have improved the relevant codes just what you pointed. And more, I add some unit tests about the behavior about

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-01 Thread uncleGen
GitHub user uncleGen reopened a pull request: https://github.com/apache/spark/pull/4135 [SPARK-5205][Streaming]:Inconsistent behaviour between Streaming job and others, when click kill link in WebUI The kill link is used to kill a stage in job. It works in any kinds of Spark job

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-01 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4135#issuecomment-76663454 @JoshRosen Could you please take a look again, thank you! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-01 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4135#issuecomment-76650327 @JoshRosen Could you please take a look again, thank you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-03-01 Thread uncleGen
Github user uncleGen closed the pull request at: https://github.com/apache/spark/pull/4135 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...

2015-04-07 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4135#issuecomment-90476533 Ha~llo~, @JoshRosen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

  1   2   3   4   5   6   >