Github user michaelmior commented on the issue:
https://github.com/apache/spark/pull/22398
@cloud-fan Thanks for the merge again! :)
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user michaelmior commented on a diff in the pull request:
https://github.com/apache/spark/pull/22398#discussion_r217013397
--- Diff: docs/configuration.md ---
@@ -746,6 +746,13 @@ Apart from these, the following properties are also
available, and may be useful
GitHub user michaelmior opened a pull request:
https://github.com/apache/spark/pull/22398
[SPARK-23820][CORE] Enable use of long form of callsite in logs
This is a rework of #21433 to address some concerns there.
You can merge this pull request into a Git repository by running
Github user michaelmior commented on the issue:
https://github.com/apache/spark/pull/21433
@gatorsmile @cloud-fan I'll just go with a boolean config as there really
is no need for more than two options and this simplifies things quite a bit
Github user michaelmior commented on a diff in the pull request:
https://github.com/apache/spark/pull/21433#discussion_r216285120
--- Diff: core/src/main/scala/org/apache/spark/storage/RDDInfo.scala ---
@@ -53,10 +55,16 @@ class RDDInfo(
}
private[spark] object
Github user michaelmior commented on the issue:
https://github.com/apache/spark/pull/21433
Thanks @srowen!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user michaelmior commented on the issue:
https://github.com/apache/spark/pull/21433
@srowen Yes, I don't expect it will be widely used but I've personally
found it helpful in some performance debugging and it's a fairly low impact
change. I was just hoping to avoid having
Github user michaelmior commented on the issue:
https://github.com/apache/spark/pull/21433
Rebased on top of master. The [failing test
](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4200/testReport/org.apache.spark.sql.hive/HiveExternalCatalogVersionsSuite
Github user michaelmior commented on the issue:
https://github.com/apache/spark/pull/21433
Done!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
GitHub user michaelmior opened a pull request:
https://github.com/apache/spark/pull/21433
[SPARK-23820][CORE] Enable use of long form of callsite in logs
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/michaelmior/spark long
Github user michaelmior commented on the issue:
https://github.com/apache/spark/pull/12162
As best I can tell, the code that was pushed here is incomplete. However,
Spark's default cache eviction policy is LRU. You can find the code which
performs eviction
[here](https://github.com
Github user michaelmior commented on the issue:
https://github.com/apache/spark/pull/9428
@zsxwing Thanks for the pointer. It's not clear to me why this needs to be
supported (and in fact the test no longer exists). However, I'm also not clear
why the tests fails in the first place
Github user michaelmior commented on the issue:
https://github.com/apache/spark/pull/9428
@zsxwing @andrewor14 Would either of you be able to explain briefly why
this approach doesn't work?
---
-
To unsubscribe, e
Github user michaelmior commented on the issue:
https://github.com/apache/spark/pull/19263
Thanks for the merge!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user michaelmior commented on the issue:
https://github.com/apache/spark/pull/19263
@vanzin Made the changes you recommended. Thanks for the feedback!
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user michaelmior commented on a diff in the pull request:
https://github.com/apache/spark/pull/19263#discussion_r145203563
--- Diff:
core/src/main/scala/org/apache/spark/internal/config/package.scala ---
@@ -41,6 +41,22 @@ package object config {
.bytesConf
Github user michaelmior commented on a diff in the pull request:
https://github.com/apache/spark/pull/19263#discussion_r145202921
--- Diff: docs/configuration.md ---
@@ -714,6 +714,13 @@ Apart from these, the following properties are also
available, and may be useful
Github user michaelmior commented on a diff in the pull request:
https://github.com/apache/spark/pull/19263#discussion_r145202877
--- Diff: docs/configuration.md ---
@@ -714,6 +714,13 @@ Apart from these, the following properties are also
available, and may be useful
Github user michaelmior commented on a diff in the pull request:
https://github.com/apache/spark/pull/19263#discussion_r145202605
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/EventLoggingListenerSuite.scala
---
@@ -291,6 +292,7 @@ object EventLoggingListenerSuite
Github user michaelmior commented on the issue:
https://github.com/apache/spark/pull/19263
@jerryshao Agreed on both points. Changed!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user michaelmior commented on the issue:
https://github.com/apache/spark/pull/19263
@vanzin Just wanted to ping since I believe I addressed your concerns. Let
me know if it looks like any other changes are required
Github user michaelmior commented on a diff in the pull request:
https://github.com/apache/spark/pull/19263#discussion_r141916752
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -66,6 +67,7 @@ private[spark] class EventLoggingListener
Github user michaelmior commented on a diff in the pull request:
https://github.com/apache/spark/pull/19263#discussion_r141849398
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -66,6 +67,7 @@ private[spark] class EventLoggingListener
Github user michaelmior commented on the issue:
https://github.com/apache/spark/pull/19263
@jerryshao I agree that the history server itself doesn't provide useful
information but for detailed analysis I've found it helpful to persist the logs
and write my own scripts to analyze them
Github user michaelmior commented on the issue:
https://github.com/apache/spark/pull/19263
Whoops. Sorry about that. I opened the PR via the CLI so I didn't see the
pointer on the web interface. I should have known better though. Updated
GitHub user michaelmior opened a pull request:
https://github.com/apache/spark/pull/19263
Optionally add block updates to log
I see that block updates are not logged to the event log.
This makes sense as a default for performance reasons.
However, I find it helpful when
Github user michaelmior commented on the issue:
https://github.com/apache/spark/pull/16609
Is there any way to create a name without caching the result? I'm parsing
the Spark event logs to get some additional data about jobs and having names
there is very helpful but I don't want
GitHub user michaelmior opened a pull request:
https://github.com/apache/spark/pull/17900
[SPARK-20637][Core] Remove mention of old RDD classes from comments
## What changes were proposed in this pull request?
A few comments around the code mention RDD classes that do
Github user michaelmior commented on the issue:
https://github.com/apache/spark/pull/12162
This branch appears to be incomplete. The configuration parameter
`entryEvictionPolicy` does not exist and there is a good chunk of the code that
does not do anything.
---
If your project
29 matches
Mail list logo
