Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22612
Thank you @squito for the reviews. I hope we can merge this soon if there
is no concern from @tgravescs @dhruve @edwinalu @LantaoJin. Thanks every one
in advance
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22612
Same flaky test. I think in the mornings there is probably more load and
the flakiness is higher. Jenkins retest this please
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22612
The failure isn't related to my change. Jenkins retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r238739905
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsMetricsGetter.scala ---
@@ -0,0 +1,223 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r238731915
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsMetricsGetter.scala ---
@@ -0,0 +1,223 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r238380513
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsMetricsGetter.scala ---
@@ -0,0 +1,223 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r238377911
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsMetricsGetter.scala ---
@@ -0,0 +1,223 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r238377498
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsMetricsGetter.scala ---
@@ -0,0 +1,223 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22612
@squito @tgravescs @mccheah let me know if you have other concerns. Thank
you in advance.
---
-
To unsubscribe, e-mail
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r237993972
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsMetricsGetter.scala ---
@@ -0,0 +1,228 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r237990856
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsMetricsGetter.scala ---
@@ -0,0 +1,228 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22612
The test is flaky and not related to my change. It passed when we re run it
last time. Jenkins retest this please
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22612
jenkins retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22612
The test failure isn't related to this change.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r237161182
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsMetricsGetter.scala ---
@@ -0,0 +1,231 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r236828402
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,228 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r236432819
--- Diff:
core/src/main/scala/org/apache/spark/metrics/ExecutorMetricType.scala ---
@@ -84,6 +122,8 @@ case object MappedPoolMemory extends
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r235542735
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,228 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r235541704
--- Diff:
core/src/main/scala/org/apache/spark/metrics/ExecutorMetricType.scala ---
@@ -84,6 +122,8 @@ case object MappedPoolMemory extends
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r234815376
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,228 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22612
@squito @mccheah @dhruve Let me know if there are more comments or this
can be merged. I appreciate it.
---
-
To unsubscribe
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r233254497
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,266 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r233252552
--- Diff:
core/src/main/scala/org/apache/spark/metrics/ExecutorMetricType.scala ---
@@ -19,18 +19,31 @@ package org.apache.spark.metrics
import
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r233250143
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,222 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22614
@tejasapatil @kmanamcheri any update? thank you very much in advance.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r230864147
--- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala ---
@@ -394,9 +394,15 @@ private[spark] object JsonProtocol
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r230852432
--- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala ---
@@ -394,9 +394,15 @@ private[spark] object JsonProtocol
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r230858827
--- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala ---
@@ -394,9 +394,15 @@ private[spark] object JsonProtocol
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r230787617
--- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala ---
@@ -394,9 +394,15 @@ private[spark] object JsonProtocol
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22612
it seems that the test has been terminated in the middle. I don't know why.
I won't be able to work on this for a few days. Please retest whenever you can.
Thank you
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r22741
--- Diff:
core/src/main/scala/org/apache/spark/metrics/ExecutorMetricType.scala ---
@@ -95,10 +135,29 @@ private[spark] object ExecutorMetricType
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r227058059
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r226526015
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r226525661
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r226524522
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r226524183
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r226524080
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r226523830
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,266 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r226523734
--- Diff: core/src/main/scala/org/apache/spark/Heartbeater.scala ---
@@ -59,12 +60,13 @@ private[spark] class Heartbeater
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22612
I would appreciate if there is any other concerns with this patch @squito
@edwinalu @mccheah @dhruve
---
-
To unsubscribe, e
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22612
Jenkins retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22612
Another flaky test failure not related to this change
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22612
Looking at the logs the failure doesn't seem to be related to this change.
It is in HiveVersionSuite.
---
-
To unsubscribe, e
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r223934270
--- Diff:
core/src/main/scala/org/apache/spark/executor/ExecutorMetrics.scala ---
@@ -28,35 +30,63 @@ import org.apache.spark.metrics.ExecutorMetricType
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r223915870
--- Diff:
core/src/main/scala/org/apache/spark/executor/ExecutorMetrics.scala ---
@@ -28,35 +30,63 @@ import org.apache.spark.metrics.ExecutorMetricType
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22612
The test failed becuase it assumes a certain order when metrics are being
reported in json. I will think about changing the test since by using a map in
ExecutorMetricType we don't assume any
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r223786195
--- Diff:
core/src/main/scala/org/apache/spark/metrics/ExecutorMetricType.scala ---
@@ -59,6 +60,43 @@ case object JVMOffHeapMemory extends
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r223486041
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,272 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r223150611
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,274 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r223150568
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,266 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r222792590
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,266 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r222792211
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,266 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r222791450
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,274 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r222528022
--- Diff:
core/src/main/scala/org/apache/spark/metrics/ExecutorMetricType.scala ---
@@ -84,6 +122,8 @@ case object MappedPoolMemory extends
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r222527353
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,274 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22612
The test failure isn't related to this change and it is most likely flaky.
I will push another patch shortly, but it will not address all the comments,
but most of them
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r222527340
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,274 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r222493807
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,274 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r222493759
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,274 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22612
Didn't get this scala check error about using root.Locale when I run the
test locally. Will think about whether I should use that suggestion later
tonight
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r222477655
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,268 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r222476446
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,268 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r222475366
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,268 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r222454774
--- Diff:
core/src/main/scala/org/apache/spark/metrics/ExecutorMetricType.scala ---
@@ -59,6 +60,43 @@ case object JVMOffHeapMemory extends
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r222454193
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,268 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22612#discussion_r222452150
--- Diff:
core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala ---
@@ -0,0 +1,268 @@
+/*
+ * Licensed to the Apache Software
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22614#discussion_r222381843
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala ---
@@ -746,34 +746,20 @@ private[client] class Shim_v0_13 extends
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22614#discussion_r222349989
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala ---
@@ -746,34 +746,20 @@ private[client] class Shim_v0_13 extends
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22612
@squito
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22612
Previously I opened a WIP PR
[here](https://github.com/apache/spark/pull/21916) and
[here](https://github.com/rezasafi/spark/pull/1). I have applied all the
comments that I received
GitHub user rezasafi opened a pull request:
https://github.com/apache/spark/pull/22612
[SPARK-24958] Add executors' process tree total memory information to
heartbeat signals.
Spark executors' process tree total memory information can be really
useful. Currently such information
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/21916
I will close this and open the actual Pr shortly. Thank you everyone for
the great reviews.
---
-
To unsubscribe, e-mail
Github user rezasafi closed the pull request at:
https://github.com/apache/spark/pull/21916
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22325
Thank you very much @vanzin @jiangxb1987
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22325
The above test build is not working. I know that that there was a problem
with jenkins earlier today. I appreciate if one of the committers fire another
test when they think the problem
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22325
Jenkins, retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22325
Flaky again. retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22325
Looking at the unit-test.log and stacktrace the failed test doesn't seem to
be related to this change. There is errors like this
`18/09/19 20:47:53.776 shuffle-server-4270-1 ERROR
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22325#discussion_r219021367
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
---
@@ -444,36 +445,36 @@ final class
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22325#discussion_r219021183
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
---
@@ -444,36 +445,36 @@ final class
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22325#discussion_r218944816
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
---
@@ -444,36 +444,34 @@ final class
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22325#discussion_r218872037
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
---
@@ -444,36 +444,34 @@ final class
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22325
@vanzin well I will send the updated PR as soon as I can finish more tests
and that way my concern of breaking stuff will be more clear.
I think no change will be merged by committers if I
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22325
it seems that it is safer to not merge the two exception handling, since
doing that will require moving around or removing some val that isn't clear why
are there. Merging and removing
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/15673#discussion_r216111838
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala ---
@@ -586,17 +587,31 @@ private[client] class Shim_v0_13 extends
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22325#discussion_r215396772
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
---
@@ -444,10 +444,23 @@ final class
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22325#discussion_r215391696
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
---
@@ -444,10 +444,23 @@ final class
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22325#discussion_r215388601
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
---
@@ -444,10 +444,23 @@ final class
GitHub user rezasafi opened a pull request:
https://github.com/apache/spark/pull/22325
[SPARK-25318]. Add exception handling when wrapping the input stream during
the the fetch or stage retry in response to a corrupted block
SPARK-4105 provided a solution to block corruption issue
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/3
Thank you very much @koeninger
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/3#discussion_r213829668
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/DirectKafkaInputDStream.scala
---
@@ -154,7 +153,8 @@ private[spark
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/3
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/3
Thank you very much @koeninger. I appreciate it.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/3#discussion_r212693216
--- Diff: docs/configuration.md ---
@@ -1925,6 +1925,14 @@ showDF(properties, numRows = 200, truncate = FALSE)
first batch when the backpressure
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/3#discussion_r212692983
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/DirectKafkaInputDStream.scala
---
@@ -141,10 +143,9 @@ private[spark
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/3
@koeninger @arzt will appreciate your feedback here. Thank you in advance.
---
-
To unsubscribe, e-mail: reviews-unsubscr
GitHub user rezasafi opened a pull request:
https://github.com/apache/spark/pull/3
SPARK-25233. Give the user the option of specifying a fixed minimum message
per partition per batch when using kafka direct API with backpressure
After SPARK-18371, it is guaranteed
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/21899#discussion_r211825377
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/BroadcastExchangeExec.scala
---
@@ -118,12 +119,20 @@ case class
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/15673#discussion_r211370552
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala ---
@@ -586,17 +587,31 @@ private[client] class Shim_v0_13 extends
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22072
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
1 - 100 of 151 matches
Mail list logo
