Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/12052#issuecomment-204123962
Thanks @andrewor14
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
GitHub user nishkamravi2 opened a pull request:
https://github.com/apache/spark/pull/12052
[SPARK-13796] Redirect error message to logWarning
## What changes were proposed in this pull request?
Redirect error message to logWarning
## How was this patch tested
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/10881#issuecomment-175363415
Updated the warning message. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/10881#issuecomment-174411762
@zsxwing Updated the PR after some testing. Most of the stack trace in the
log was coming from postToAll RPC for RemoteProcessDisconnected, so I have
removed
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10881#discussion_r50617860
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2183,6 +2183,13 @@ object SparkContext extends Logging
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/10881#issuecomment-174230956
Ok, makes sense. Btw, we also want to mask the exception being thrown at
line 161 (I see that in the logs as well).
---
If your project is set up for it, you
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/10881#issuecomment-174227577
Thanks for the pointers @zsxwing. Makes sense to just log an error message.
Instead of throwing an exception from postMessage and logging it in send, what
if we
GitHub user nishkamravi2 opened a pull request:
https://github.com/apache/spark/pull/10881
[SPARK-12967][Netty] Avoid NettyRpc error message during sparkContext
shutdown
If the there's an RPC issue while sparkContext is alive but stopped (which
would happen only when executing
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10881#discussion_r50617588
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2183,6 +2183,13 @@ object SparkContext extends Logging
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10881#discussion_r50613468
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2183,6 +2183,13 @@ object SparkContext extends Logging
Github user nishkamravi2 closed the pull request at:
https://github.com/apache/spark/pull/9601
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
GitHub user nishkamravi2 opened a pull request:
https://github.com/apache/spark/pull/9601
[SPARK-11635] [BUILD] Use HashCode instead of HashCodes
@vanzin
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nishkamravi2/spark
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/9460#issuecomment-153966219
@vanzin You're right. Updated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
GitHub user nishkamravi2 opened a pull request:
https://github.com/apache/spark/pull/9460
[SPARK-11501][CORE][YARN] Propagate spark.rpc config to executors
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nishkamravi2/spark
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7532#discussion_r36032098
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -563,32 +570,48 @@ private[master] class Master(
app
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7532#discussion_r36028448
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -563,32 +570,48 @@ private[master] class Master(
app
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7532#discussion_r36028473
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -563,32 +570,48 @@ private[master] class Master(
app
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7532#discussion_r35961635
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -804,6 +826,85 @@ private[master] class Master
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7532#discussion_r35961262
--- Diff:
core/src/main/scala/org/apache/spark/deploy/master/ApplicationInfo.scala ---
@@ -96,6 +101,14 @@ private[spark] class ApplicationInfo
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7532#discussion_r35961491
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -563,32 +570,48 @@ private[master] class Master(
app
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7532#discussion_r35961575
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -563,32 +570,48 @@ private[master] class Master(
app
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/7532#issuecomment-126648582
Hey @andrewor14, the scheduling logic and unit tests look good. Would it
make sense to add a test for the dynamic allocation behavior?
---
If your project is set
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/7274#issuecomment-125471627
Hey @andrewor14, thanks for taking care of this! Sorry, couldn't respond
sooner, was out for a couple of days.
---
If your project is set up for it, you can reply
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/7274#issuecomment-123955633
Can this be retested please..
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7274#discussion_r35271244
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -543,39 +544,73 @@ private[master] class Master(
* multiple
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/7274#issuecomment-123887115
Sure, thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7274#discussion_r35176367
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -543,59 +544,108 @@ private[master] class Master(
* multiple
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7274#discussion_r35072034
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -544,58 +544,82 @@ private[master] class Master(
* has enough
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7274#discussion_r35072994
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -543,39 +544,72 @@ private[master] class Master(
* multiple
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7274#discussion_r35073998
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -543,39 +544,72 @@ private[master] class Master(
* multiple
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7274#discussion_r35074547
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -543,39 +544,72 @@ private[master] class Master(
* multiple
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7274#discussion_r35078109
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -543,39 +544,72 @@ private[master] class Master(
* multiple
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7274#discussion_r35071918
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -543,39 +544,72 @@ private[master] class Master(
* multiple
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7274#discussion_r35071961
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -543,39 +544,72 @@ private[master] class Master(
* multiple
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7274#discussion_r35074750
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -543,39 +544,72 @@ private[master] class Master(
* multiple
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7274#discussion_r34981074
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -543,39 +544,72 @@ private[master] class Master(
* multiple
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/7274#issuecomment-122206575
Fair enough. We can have this PR sit for a little bit until I add a couple
of unit tests. I agree, in general it's good practice to have tests be part of
the PR
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7274#discussion_r34867770
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -544,58 +544,82 @@ private[master] class Master(
* has enough
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/7274#issuecomment-122204421
Thanks @andrewor14 @srowen for the review. Master.scala updated. I do plan
to add tests but would prefer to do it in a separate PR if that's ok (might
take more
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7274#discussion_r34942573
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -544,58 +544,82 @@ private[master] class Master(
* has enough
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/7274#issuecomment-119841645
Can this be retested please?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/7274#issuecomment-120172981
Thanks Imran. I thought my local run had gone through, will check again.
Btw, were you able to make this out from the test result tab or by scanning the
console
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7274#discussion_r34134630
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -544,38 +544,60 @@ private[master] class Master(
* has enough
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/7274#issuecomment-119758308
Can this be retested please?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/7274#issuecomment-119820610
Hey @andrewor14, not sure what to make of these test results. Are you able
to see which tests failed?
---
If your project is set up for it, you can reply
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/7274#issuecomment-119381194
Overview of changes:
1. scheduleExecutorsOnWorkers rewritten and separated out (so it can be
unit tested)
2. allocateWorkerResourceToExecutors modified
GitHub user nishkamravi2 opened a pull request:
https://github.com/apache/spark/pull/7274
[SPARK-8881] Fix algorithm for scheduling executors on workers
Current scheduling algorithm allocates one core at a time and in doing so
ends up ignoring spark.executor.cores. As a result
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7274#discussion_r34107025
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -544,38 +544,60 @@ private[master] class Master(
* has enough
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7274#discussion_r34107034
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -544,38 +544,60 @@ private[master] class Master(
* has enough
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7274#discussion_r34108938
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -544,38 +544,60 @@ private[master] class Master(
* has enough
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7274#discussion_r34109910
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -544,38 +544,60 @@ private[master] class Master(
* has enough
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7274#discussion_r34108869
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -544,38 +544,60 @@ private[master] class Master(
* has enough
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/6607#issuecomment-117527403
Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/6607#issuecomment-114359813
This should contain all the changes we have discussed so far.
ReceiverTrackerSuite now contains two instead of four tests (intentionally
keeping some preferred
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/6607#discussion_r32879522
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -308,12 +336,25 @@ class ReceiverTracker(ssc
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/6607#discussion_r32812336
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/scheduler/SchedulerSuite.scala
---
@@ -0,0 +1,91 @@
+/*
+ * Licensed
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/6607#discussion_r32812244
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -308,12 +336,25 @@ class ReceiverTracker(ssc
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/6607#discussion_r32707963
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -308,12 +336,25 @@ class ReceiverTracker(ssc
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/6607#discussion_r32708027
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/scheduler/SchedulerSuite.scala
---
@@ -0,0 +1,91 @@
+/*
+ * Licensed
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/6607#discussion_r32708156
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/scheduler/SchedulerSuite.scala
---
@@ -0,0 +1,91 @@
+/*
+ * Licensed
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/6607#discussion_r32609616
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/scheduler/SchedulerSuite.scala
---
@@ -0,0 +1,91 @@
+/*
+ * Licensed
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/6607#discussion_r32609818
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/scheduler/SchedulerSuite.scala
---
@@ -0,0 +1,91 @@
+/*
+ * Licensed
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/6607#discussion_r32609582
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -308,12 +336,25 @@ class ReceiverTracker(ssc
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/6607#discussion_r32609608
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/scheduler/SchedulerSuite.scala
---
@@ -0,0 +1,91 @@
+/*
+ * Licensed
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/6607#issuecomment-110130224
Support for mixed scheduling, test cases added.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/6607#discussion_r31800833
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -308,12 +305,41 @@ class ReceiverTracker(ssc
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/6607#issuecomment-109400161
Thanks for the review. Will look into the failing test cases soon.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/6607#discussion_r31841788
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -271,27 +272,64 @@ class ReceiverTracker(ssc
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/6607#discussion_r31841859
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -271,27 +272,64 @@ class ReceiverTracker(ssc
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/6607#discussion_r31859924
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -308,12 +305,41 @@ class ReceiverTracker(ssc
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/6607#discussion_r31781469
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/receiver/Receiver.scala ---
@@ -107,8 +107,8 @@ abstract class Receiver[T](val
GitHub user nishkamravi2 opened a pull request:
https://github.com/apache/spark/pull/6607
[SPARK-7988][STREAMING] Round-robin scheduling of receivers by default
Minimal PR for round-robin scheduling of receivers. Dense scheduling can be
enabled by setting preferredLocation, so
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/6544#issuecomment-107662348
Yeah, the JIRA in the title seems unnecessary (removed). With regards to
the merge history: not sure, will have to look into it.
---
If your project is set up
GitHub user nishkamravi2 opened a pull request:
https://github.com/apache/spark/pull/6544
[SPARK-7959][DOC] Minor modification to Streaming docs with regards to
parallel data receiving
@pwendell @tdas
You can merge this pull request into a Git repository by running:
$ git
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/6198#discussion_r30451188
--- Diff: core/src/test/scala/org/apache/spark/SparkConfSuite.scala ---
@@ -241,6 +241,9 @@ class SparkConfSuite extends FunSuite with
LocalSparkContext
GitHub user nishkamravi2 opened a pull request:
https://github.com/apache/spark/pull/6198
[SPARK-7672][CORE] Use int conversion in translating
spark.kryoserializer.buffer.mb to spark.kryoserializer.buffer
You can merge this pull request into a Git repository by running
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/6198#issuecomment-102525738
Thanks for verifying @andrewor14 @ilganeli. Looked at the failed tests and
they seem unrelated.
---
If your project is set up for it, you can reply to this email
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/6198#issuecomment-102530779
Good suggestion @vanzin.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/5760#issuecomment-97529204
Not an an environment problem. Permissions are generally non-uniform. Not a
recent document, but might help:
https://hadoop.apache.org/docs/r2.5.0/hadoop-project
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/5760#issuecomment-97606549
In my installation container-executor.cfg is root readable only (generated
by CDH/CM). Please see the link in the comment above to help answer your
question (go
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/5760#issuecomment-97608828
I think that's a whole different issue. You could have config files
altogether missing and the same problem applies.
---
If your project is set up for it, you can
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5760#discussion_r29389140
--- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -370,9 +370,11 @@ private[spark] class Client(
try
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/5760#issuecomment-97610998
I think you are mixing two separate issues. This PR attempts to have Spark
work for expected behavior. Sanity checking unexpected behavior (anomalous
Hadoop config
GitHub user nishkamravi2 opened a pull request:
https://github.com/apache/spark/pull/5760
Check for read permissions before copying a Hadoop config file
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nishkamravi2/spark
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/5672#issuecomment-96026365
See Utils.inShutDown discussion in 4690.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/5672#issuecomment-96026386
Can this be tested please. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/5672#issuecomment-96089151
@srowen I think I'd approach this one as why-not (especially with logging
enabled). This particular call site has warranted two separate PRs attempting
to add a try
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/5672#issuecomment-96105072
@vanzin Sounds reasonable.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/5672#issuecomment-96079990
The original PR in 4690 didn't make assumptions about the type of Exception
thrown and logged it, I guess we were thinking slightly differently back then
too
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/5672#issuecomment-96089855
@vanzin Are we suggesting adding a try-catch block inside the while loop
in runAll or deleting throw's in logUncaughtExceptions ?
---
If your project is set up
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/5672#issuecomment-96093469
While we are at it: any particular reason for using PriorityQueue or can we
replace it by ConcurrentLinkedQueue or something and get rid of a few
synchronized's
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/5672#issuecomment-96033664
I would be fine with 'not explicitly throwing an exception' and keeping the
try-catch block.
---
If your project is set up for it, you can reply to this email
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/5672#issuecomment-96062413
With regards to logging, I thought you had recommended not logging (in
4690) . Same reasons apply?
---
If your project is set up for it, you can reply
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/5672#issuecomment-96058855
A case of erring on the conservative side: the implementation of
Utils.removeShutdownHook could change over time. The invocation from
DiskBlockManager should
GitHub user nishkamravi2 opened a pull request:
https://github.com/apache/spark/pull/5672
[SPARK-6014] [core][hotfix] Revamp Spark shutdown hooks, fix shutdown races.
Add a try/catch block around removeShutDownHook else IllegalStateException
thrown in YARN cluster mode (see https
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5504#discussion_r28488106
--- Diff:
launcher/src/main/java/org/apache/spark/launcher/AbstractCommandBuilder.java ---
@@ -186,12 +186,24 @@ void addOptionString(ListString cmd
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/5504#issuecomment-93795238
Maintaining a non-null value for spark_assembly is a good thing unless
there is a good reason to not do so.
---
If your project is set up for it, you can reply
Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/5504#issuecomment-93792328
Will submit a hot fix in a bit
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5504#discussion_r28536990
--- Diff:
launcher/src/main/java/org/apache/spark/launcher/AbstractCommandBuilder.java ---
@@ -186,12 +186,24 @@ void addOptionString(ListString cmd
Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5504#discussion_r28477271
--- Diff:
launcher/src/main/java/org/apache/spark/launcher/AbstractCommandBuilder.java ---
@@ -186,12 +186,24 @@ void addOptionString(ListString cmd
1 - 100 of 191 matches
Mail list logo
