GitHub user gaborgsomogyi opened a pull request:
https://github.com/apache/spark/pull/20019
[SPARK-22361][SQL][TEST] Add unit test for Window Frames
## What changes were proposed in this pull request?
There are already quite a few integration tests using window frames, but
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/20019
cc @jiangxb1987 @gatorsmile @hvanhovell
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user gaborgsomogyi opened a pull request:
https://github.com/apache/spark/pull/20022
Add unit test for Window spilling
## What changes were proposed in this pull request?
There is already test using window spilling, but the test coverage is not
ideal.
In
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/20022
cc @jiangxb1987 @gatorsmile @hvanhovell
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/19893
@squito I mean another jira, because it needs deeper analysis and
discussion.
---
-
To unsubscribe, e-mail: reviews
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/20019
@smurakozi nice catch, added them. Additionally found a nit.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19035#discussion_r158050733
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala
---
@@ -435,6 +435,18 @@ case class
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19035#discussion_r158047414
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala
---
@@ -435,6 +435,18 @@ case class
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20019#discussion_r158289422
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameWindowFramesSuite.scala
---
@@ -0,0 +1,381 @@
+/*
+ * Licensed to the Apache
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/20019
ExpressionParserSuite window related part is a bit reorganized and enhanced.
---
-
To unsubscribe, e-mail: reviews
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20019#discussion_r158290356
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameWindowFramesSuite.scala
---
@@ -0,0 +1,381 @@
+/*
+ * Licensed to the Apache
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20045#discussion_r158298186
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala
---
@@ -246,13 +246,23 @@ class
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20045#discussion_r158298350
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala
---
@@ -246,13 +246,23 @@ class
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20045#discussion_r158292663
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameWindowFunctionsSuite.scala
---
@@ -32,6 +32,217 @@ import
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19431#discussion_r158382365
--- Diff:
external/kafka-0-8/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala
---
@@ -91,9 +91,16 @@ class
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19431#discussion_r158382300
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/DirectKafkaInputDStream.scala
---
@@ -126,7 +129,10 @@ private
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19431#discussion_r158375540
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/DirectKafkaInputDStream.scala
---
@@ -126,7 +129,10 @@ private
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19431#discussion_r158374124
--- Diff:
external/kafka-0-8/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala
---
@@ -91,9 +91,16 @@ class
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19431#discussion_r158375331
--- Diff:
external/kafka-0-8/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala
---
@@ -108,7 +115,9 @@ class
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/19431
Until now it was not fully clear documented in which situation does this
parameter take effect. I would personally add things into the doc to be a bit
more specific
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/19893
gentle ping @jiangxb1987
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20022#discussion_r158525031
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameWindowFunctionsSuite.scala
---
@@ -518,9 +519,46 @@ class
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19893#discussion_r158527212
--- Diff: core/src/test/scala/org/apache/spark/ThreadAudit.scala ---
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19893#discussion_r158527190
--- Diff: core/src/test/scala/org/apache/spark/ThreadAudit.scala ---
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19893#discussion_r158527262
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/test/SharedSQLContext.scala ---
@@ -17,4 +17,17 @@
package
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20022#discussion_r158585754
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameWindowFunctionsSuite.scala
---
@@ -518,9 +519,46 @@ class
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20019#discussion_r159196803
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala
---
@@ -263,21 +263,60 @@ class
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20019#discussion_r159196842
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala
---
@@ -263,21 +263,60 @@ class
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/20022
@gatorsmile @cloud-fan @jiangxb1987 Thanks for the help!
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19893#discussion_r159831366
--- Diff: core/src/test/scala/org/apache/spark/ThreadAudit.scala ---
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19893#discussion_r159831631
--- Diff: core/src/test/scala/org/apache/spark/ThreadAudit.scala ---
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19893#discussion_r159832598
--- Diff: core/src/test/scala/org/apache/spark/ThreadAudit.scala ---
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19893#discussion_r159832729
--- Diff: core/src/test/scala/org/apache/spark/ThreadAudit.scala ---
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19893#discussion_r159834709
--- Diff: core/src/test/scala/org/apache/spark/ThreadAudit.scala ---
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19893#discussion_r159837804
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/SessionStateSuite.scala ---
@@ -39,6 +41,7 @@ class SessionStateSuite extends SparkFunSuite
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19893#discussion_r159838691
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/sources/DataSourceAnalysisSuite.scala
---
@@ -29,16 +29,24 @@ import
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19893#discussion_r159840754
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSessionStateSuite.scala
---
@@ -28,14 +28,18 @@ import
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19893#discussion_r159842007
--- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala ---
@@ -31,15 +31,28 @@ import org.apache.spark.util.AccumulatorContext
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19893#discussion_r159841944
--- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala ---
@@ -31,15 +31,28 @@ import org.apache.spark.util.AccumulatorContext
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19893#discussion_r159851476
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/test/SharedSQLContext.scala ---
@@ -17,4 +17,17 @@
package
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20019#discussion_r160143221
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala
---
@@ -263,21 +263,60 @@ class
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19893#discussion_r160158067
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/test/SharedSQLContext.scala ---
@@ -17,4 +17,22 @@
package
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19893#discussion_r160158241
--- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala ---
@@ -27,19 +27,55 @@ import org.apache.spark.util.AccumulatorContext
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19893#discussion_r160158308
--- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala ---
@@ -27,19 +27,55 @@ import org.apache.spark.util.AccumulatorContext
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19893#discussion_r161222015
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/test/SharedSQLContext.scala ---
@@ -17,4 +17,22 @@
package
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/19893
Checked the failure but seems like unrelated.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/19893
Related hive please see my comment on 11 Dec 2017.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19893#discussion_r161308026
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/test/SharedSQLContext.scala ---
@@ -17,4 +17,37 @@
package
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19893#discussion_r161370006
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/test/SharedSQLContext.scala ---
@@ -17,4 +17,37 @@
package
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/19893
Thread audit disabled in hive.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/19893
Checked and seems like unrelated.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user gaborgsomogyi opened a pull request:
https://github.com/apache/spark/pull/21105
[SPARK-24022][TEST] Make SparkContextSuite not flaky
## What changes were proposed in this pull request?
SparkContextSuite.test("Cancelling stages/jobs with custom reasons.&qu
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/21105
cc @vanzin @jiangxb1987
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/21105
@jiangxb1987 Here it's possible to create a separate context.
---
-
To unsubscribe, e-mail: reviews-uns
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/21105
Yes, please see SPARK-22764.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/20997
Taken a look at the pool options I have the feeling it requires more time
to come up with a proper solution. Switching back to the SQL code provided one
cached consumer approach
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20888#discussion_r183067778
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameRangeSuite.scala ---
@@ -152,39 +154,53 @@ class DataFrameRangeSuite extends
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20888#discussion_r183069064
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameRangeSuite.scala ---
@@ -152,39 +154,53 @@ class DataFrameRangeSuite extends
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/20997
In the meantime found a small glitch in the SQL part. Namely if reattempt
happens this line
https://github.com/apache/spark/blob/1d758dc73b54e802fdc92be204185fe7414e6553/external/kafka-0
GitHub user gaborgsomogyi opened a pull request:
https://github.com/apache/spark/pull/21214
[SPARK-23775][TEST] Make DataFrameRangeSuite not flaky
## What changes were proposed in this pull request?
DataFrameRangeSuite.test("Cancelling stage in a query with Range.&qu
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20997#discussion_r185451746
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaDataConsumer.scala
---
@@ -0,0 +1,359
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20997#discussion_r185451919
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaDataConsumer.scala
---
@@ -0,0 +1,359
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20997#discussion_r185451988
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaDataConsumer.scala
---
@@ -0,0 +1,359
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20997#discussion_r185452496
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaDataConsumer.scala
---
@@ -0,0 +1,359
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/21214
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/21214
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/21214#discussion_r185792166
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameRangeSuite.scala ---
@@ -153,23 +153,17 @@ class DataFrameRangeSuite extends
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/21214
> Just curious how this happened?
I've taken a look at 2 logs you've shown and seen the same pattern. Namely
the test which killed threads passed and right after the upcoming
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/21214#discussion_r185836245
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameRangeSuite.scala ---
@@ -153,23 +153,17 @@ class DataFrameRangeSuite extends
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/20997
Do I need to do any further changes?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user gaborgsomogyi opened a pull request:
https://github.com/apache/spark/pull/21430
[SPARK-23991][DSTREAMS] Fix data loss when WAL write fails in
allocateBlocksToBatch
## What changes were proposed in this pull request?
When blocks tried to get allocated to a batch
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/21430
cc @vanzin
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/21430#discussion_r191130178
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceivedBlockTracker.scala
---
@@ -112,10 +112,13 @@ private[streaming
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/21430#discussion_r191130212
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/ReceivedBlockTrackerSuite.scala
---
@@ -308,12 +354,16 @@ class
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/21430#discussion_r191130186
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/ReceivedBlockTrackerSuite.scala
---
@@ -115,6 +117,50 @@ class
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/21430
Thanks @vanzin @jerryshao for the help.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user gaborgsomogyi opened a pull request:
https://github.com/apache/spark/pull/21450
[SPARK-24319][SPARK SUBMIT] Fix spark-submit execution where no main class
is required.
## What changes were proposed in this pull request?
With [PR 20925](https://github.com/apache
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/21450
test this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/21450
cc @vanzin
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/21450
Updated the description to reflect the no arg case as well.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/21450#discussion_r192385912
--- Diff:
launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java
---
@@ -229,7 +238,7 @@
args.add(join
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/21450#discussion_r192385957
--- Diff:
launcher/src/test/java/org/apache/spark/launcher/SparkSubmitCommandBuilderSuite.java
---
@@ -344,8 +365,15 @@ private
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/21450#discussion_r192386271
--- Diff:
launcher/src/test/java/org/apache/spark/launcher/SparkSubmitCommandBuilderSuite.java
---
@@ -190,6 +194,23 @@ public void testSparkRShell
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/21450
The issue seems unrelated.
```
[error] (kubernetes-integration-tests/*:checkstyle)
java.io.FileNotFoundException: checkstyle-config.xml (No such file or directory)
```
Trying
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20853#discussion_r175521925
--- Diff:
core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala ---
@@ -657,6 +667,31 @@ class SparkSubmitSuite
conf3.get
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20853#discussion_r175523620
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -245,6 +245,19 @@ object SparkSubmit extends CommandLineUtils with
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20853#discussion_r175529369
--- Diff: core/src/main/scala/org/apache/spark/deploy/DependencyUtils.scala
---
@@ -137,16 +137,29 @@ private[deploy] object DependencyUtils
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20853#discussion_r175515840
--- Diff:
core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala ---
@@ -105,11 +105,17 @@ class SparkSubmitSuite
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20853#discussion_r175533621
--- Diff: core/src/main/scala/org/apache/spark/deploy/DependencyUtils.scala
---
@@ -137,16 +137,29 @@ private[deploy] object DependencyUtils
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20853#discussion_r175521372
--- Diff:
core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala ---
@@ -606,9 +612,12 @@ class SparkSubmitSuite
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20853#discussion_r175540696
--- Diff:
core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala ---
@@ -105,11 +105,17 @@ class SparkSubmitSuite
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20853#discussion_r175540847
--- Diff:
core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala ---
@@ -606,9 +612,12 @@ class SparkSubmitSuite
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20853#discussion_r175541913
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -245,6 +245,19 @@ object SparkSubmit extends CommandLineUtils with
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20853#discussion_r175543124
--- Diff: core/src/main/scala/org/apache/spark/deploy/DependencyUtils.scala
---
@@ -137,16 +137,29 @@ private[deploy] object DependencyUtils
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20853#discussion_r175544492
--- Diff: core/src/main/scala/org/apache/spark/deploy/DependencyUtils.scala
---
@@ -137,16 +137,29 @@ private[deploy] object DependencyUtils
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20853#discussion_r175553185
--- Diff:
core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala ---
@@ -105,11 +105,17 @@ class SparkSubmitSuite
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20853#discussion_r175567819
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -245,6 +245,19 @@ object SparkSubmit extends CommandLineUtils with
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20853#discussion_r175585581
--- Diff: core/src/main/scala/org/apache/spark/deploy/DependencyUtils.scala
---
@@ -137,16 +138,36 @@ private[deploy] object DependencyUtils
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20853#discussion_r175585634
--- Diff: core/src/main/scala/org/apache/spark/deploy/DependencyUtils.scala
---
@@ -137,16 +138,36 @@ private[deploy] object DependencyUtils
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/19819
It will create a new consumer for each thread. This could be quite resource
consuming when several topics shared with thread pools
1 - 100 of 467 matches
Mail list logo
