Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/16818#discussion_r100049000
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/window/BoundOrdering.scala
---
@@ -25,18 +25,22 @@ import
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16818
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16691
cc @tdas also
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16827
cc @rxin also
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16818
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16818
cc @cloud-fan also
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16818
@hvanhovell After dug deeply into code, I found the range scale has nothing
to do with `RowBuffer`, so there is no need to limit this to allowing long
values for range frames only, and this pr
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16827
@srowen Well, this change may be really radical. Just like what I say, many
fresh users has no consciousness to check if job runs correctly in standalone
mode or yarn mode. Maybe, they just forgot
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/16827#discussion_r99777156
--- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala ---
@@ -779,6 +781,30 @@ private[spark] object SparkConf extends Logging
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/16827#discussion_r99776850
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala ---
@@ -63,6 +63,8 @@ class Checkpoint(ssc: StreamingContext, val
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/16827#discussion_r99776217
--- Diff: core/src/test/scala/org/apache/spark/util/UtilsSuite.scala ---
@@ -845,6 +845,7 @@ class UtilsSuite extends SparkFunSuite
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/16827#discussion_r99775245
--- Diff:
core/src/test/scala/org/apache/spark/deploy/rest/StandaloneRestSubmitSuite.scala
---
@@ -440,7 +440,6 @@ class StandaloneRestSubmitSuite extends
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/16827#discussion_r99776038
--- Diff: core/src/test/scala/org/apache/spark/SparkConfSuite.scala ---
@@ -131,9 +131,8 @@ class SparkConfSuite extends SparkFunSuite
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16827
Working on UT failure.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16827
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/16827#discussion_r99743099
--- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala ---
@@ -779,6 +781,30 @@ private[spark] object SparkConf extends Logging
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/16827#discussion_r99739390
--- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala ---
@@ -779,6 +781,31 @@ private[spark] object SparkConf extends Logging
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16656
cc @tdas also.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/16827#discussion_r99738940
--- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala ---
@@ -779,6 +781,30 @@ private[spark] object SparkConf extends Logging
GitHub user uncleGen opened a pull request:
https://github.com/apache/spark/pull/16827
[SPARK-19482][CORE] Fail it if 'spark.master' is set with different value
## What changes were proposed in this pull request?
First, there is no need to set 'spark.master' multi-times
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16818
@hvanhovell Thanks for your suggestions, it is just what I failed to notice
or consider.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16818
cc @srowen
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16815
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user uncleGen opened a pull request:
https://github.com/apache/spark/pull/16818
[SPARK-19451][SQL][Core] Underlying integer overflow in Window function
## What changes were proposed in this pull request?
reproduce code:
```
val tw = Window.orderBy("
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16815
cc @srowen @zsxwing
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16816
It is too tiny to make a pr, so let us close it, OK?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
GitHub user uncleGen opened a pull request:
https://github.com/apache/spark/pull/16815
[SPARK-19407][SS] defaultFS is used FileSystem.get instead of getting it
from uri scheme
## What changes were proposed in this pull request?
```
Caused
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16656
cc @zsxwing agagin
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16691
@zsxwing That is right, this pr is the improvement for the previous one.
There is no much performance regression, i.e. only wait for
"spark.scheduler.maxRegisteredResourcesWaitin
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16702
cc @srowen take a second review please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
GitHub user uncleGen opened a pull request:
https://github.com/apache/spark/pull/16702
[SPARK-18495][UI] Document meaning of green dot in DAG visualization
## What changes were proposed in this pull request?
A green dot in the DAG visualization apparently means
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16691
cc @zsxwing
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16691
Working on the test failure.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16502
mistakenly opened? close it please!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16656
cc @zsxwing
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user uncleGen closed the pull request at:
https://github.com/apache/spark/pull/16687
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16691
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/16656#discussion_r97519002
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala ---
@@ -536,6 +539,7 @@ abstract class DStream[T: ClassTag
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/16691#discussion_r97518189
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -422,16 +423,36 @@ class ReceiverTracker(ssc
GitHub user uncleGen opened a pull request:
https://github.com/apache/spark/pull/16691
[SPARK-19349][DStreams] Check resource ready to avoid multiple receivers to
be scheduled on the same node.
## What changes were proposed in this pull request?
remove related TODO
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16687
cc @zsxwing
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/16687#discussion_r97483845
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobGenerator.scala
---
@@ -146,6 +147,11 @@ class JobGenerator(jobScheduler
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/16687#discussion_r97483687
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
---
@@ -837,6 +839,29 @@ class StreamingContextSuite extends
GitHub user uncleGen opened a pull request:
https://github.com/apache/spark/pull/16687
[SPARK-19343][DStreams] Do once optimistic checkpoint before stop
## What changes were proposed in this pull request?
Streaming job restarts from checkpoint, and it will rebuild several
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/16658#discussion_r97304642
--- Diff: docs/index.md ---
@@ -15,7 +15,7 @@ It also supports a rich set of higher-level tools
including [Spark SQL](sql-prog
Get Spark from
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16658
@srowen I have checked on one sweep of all the docs.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16673
Sorry, I do not get your point. Do you mean the precessing time detail can
not be shown on line? I can not reproduce it, and everything works well.
---
If your project is set up for it, you can
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16658
cc @srowen
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/14731
@steveloughran Are you still working on this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
GitHub user uncleGen opened a pull request:
https://github.com/apache/spark/pull/16658
[DOCS] Fix typo in docs
## What changes were proposed in this pull request?
Fix typo in docs
cc @srowen
## How was this patch tested?
You can merge this pull
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16656
cc @zsxwing could you please give a review?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16656
process was terminated by signal 9
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16656
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user uncleGen opened a pull request:
https://github.com/apache/spark/pull/16656
[SPARK-18116][DStream] Report stream input information after recover from
checkpoint
## What changes were proposed in this pull request?
Run a streaming application which souce from
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16635
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16635
process was terminated by signal 9
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user uncleGen closed the pull request at:
https://github.com/apache/spark/pull/16629
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16629
@srowen Yes, this pr does not provide a way to support `ConsumerCache` in
multi-thread, but give users a more clear hint to this issue. I think it may be
more complex before we achieve a better
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16629
cc @zsxwing and @koeninger
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
GitHub user uncleGen opened a pull request:
https://github.com/apache/spark/pull/16629
[SPARK-19185][DStream] Add more clear hint for
'ConcurrentModificationExceptions'
## What changes were proposed in this pull request?
When same kafka partition is consumed from multiple
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16601
@zsxwing Take a review please!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user uncleGen closed the pull request at:
https://github.com/apache/spark/pull/16573
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16591
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16591
No errors, but process was terminated by signal 9.
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/16601#discussion_r96358523
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/DStreamGraph.scala ---
@@ -112,12 +112,10 @@ final private[streaming] class DStreamGraph
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16573
@zsxwing I know what you mean, and indeed it can achieve the right result!
IMHO, since we have provided the `SparkContext.setLogLevel`, it is weird to
call
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16573
also cc @tdas
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16601
also cc @tdas
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16591
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/16591#discussion_r96336953
--- Diff:
core/src/main/java/org/apache/spark/api/java/JavaFutureAction.java ---
@@ -17,7 +17,6 @@
package org.apache.spark.api.java
GitHub user uncleGen opened a pull request:
https://github.com/apache/spark/pull/16601
[SPARK-19182][DStream] Optimize the lock in StreamingJobProgressListener to
not block UI when generating Streaming jobs
## What changes were proposed in this pull request?
When
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16573
@zsxwing Could you review this PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16591
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16591
This work does not change any code, but just delete unused imports and fix
some code style issue.
cc @srowen
---
If your project is set up for it, you can reply to this email and have your
GitHub user uncleGen opened a pull request:
https://github.com/apache/spark/pull/16591
[SPARK-19227][CORE] remove ununsed imports and outdated comments in
`org.apache.spark.internal.config.ConfigEntry`
## What changes were proposed in this pull request?
remove ununsed imports
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16573
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user uncleGen opened a pull request:
https://github.com/apache/spark/pull/16573
[SPARK-19210][DStream] Add log level info into checkpoint file
## What changes were proposed in this pull request?
If we set log level by using **SparkContext.setLogLevel()**, after
GitHub user uncleGen opened a pull request:
https://github.com/apache/spark/pull/16569
[SPARK-19206][DOC][DStream]Fix outdated parameter descriptions
## What changes were proposed in this pull request?
Fix outdated parameter descriptions in kafka010
## How
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16414
@vanzin Take a review again please!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16414
cc @vanzin @saturday-shi
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16414
cc @srowen
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16370
@srowen It looks like @zsxwing has no time to feed back.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
GitHub user uncleGen opened a pull request:
https://github.com/apache/spark/pull/16414
[SPARK-19009][DOC] Add streaming rest api doc
## What changes were proposed in this pull request?
add streaming rest api doc
## How was this patch tested?
You can merge
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16370
@zsxwing Is there any farther feedbackï¼
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16370
@zsxwing Thanks for your reminder!!
In some ways, we really can evade this issue, just like not use `-cp`. But
this is an user-side behaviour, we can not ensure every users know and use
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16370
unrelated errors, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16370
@AmplabJenkins retest it please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16370
cc @zsxwing
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user uncleGen opened a pull request:
https://github.com/apache/spark/pull/16370
[SPARK-18960][SQL][SS] Avoid double reading file which is being copied.
## What changes were proposed in this pull request?
In HDFS, when we copy a file into target directory
Github user uncleGen closed the pull request at:
https://github.com/apache/spark/pull/16142
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16142
@vanzin I will close this PR, but can't take your point completely.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/15915
@srowen All tests passed, take a look please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/15915
There are some unrelated failures, retest it please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/15915#discussion_r92785420
--- Diff:
core/src/test/scala/org/apache/spark/storage/MemoryStoreSuite.scala ---
@@ -60,7 +60,7 @@ class MemoryStoreSuite
SizeEstimator
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/15915
OK, we reach a consensus, I will give a update as soon as possible.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16142
@srowen It has real demand. I have received some reports from our users:
they care about space cost from job event logs, but not care about logs age, no
matter how old it is.
If you guys
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16142
> The current scan code does not make one request to the NameNode per log
file in the directory. Your code does. That should be avoided.
Make sense, current implementation can be optimi
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/15915
@srowen Looks like it is not very convenient to test this case, with
`java.lang.OutOfMemoryError: Requested array size exceeds VM limit` in jenkins.
Do you think it is necessary to test
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/15915#discussion_r92739704
--- Diff:
core/src/test/scala/org/apache/spark/storage/MemoryStoreSuite.scala ---
@@ -60,7 +60,7 @@ class MemoryStoreSuite
SizeEstimator
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/15915#discussion_r92739151
--- Diff:
core/src/test/scala/org/apache/spark/storage/MemoryStoreSuite.scala ---
@@ -60,7 +60,7 @@ class MemoryStoreSuite
SizeEstimator
201 - 300 of 689 matches
Mail list logo
