Github user ivoson commented on the issue:
https://github.com/apache/spark/pull/22252
Thanks and got it. @srowen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user ivoson commented on the issue:
https://github.com/apache/spark/pull/22252
Based on the discussion above, I think updating the doc to guide users to
explicitly identify the unit may be necessary. The last commit update the doc.
cc @vanzin @srowen @xuanyuanking
Github user ivoson commented on a diff in the pull request:
https://github.com/apache/spark/pull/22252#discussion_r213546414
--- Diff: docs/configuration.md ---
@@ -152,7 +152,7 @@ of the most common options to set are:
spark.driver.memory
1g
-Amount
Github user ivoson commented on a diff in the pull request:
https://github.com/apache/spark/pull/22252#discussion_r213544524
--- Diff: docs/configuration.md ---
@@ -152,7 +152,7 @@ of the most common options to set are:
spark.driver.memory
1g
-Amount
GitHub user ivoson opened a pull request:
https://github.com/apache/spark/pull/22252
[SPARK-25261][MINOR][DOC] correct the default unit for
spark.executor|driver.memory as described in configuration.md
## What changes were proposed in this pull request?
As described
Github user ivoson commented on the issue:
https://github.com/apache/spark/pull/21400
@jose-torres @xuanyuanking @zsxwing Thanks for reviewing this.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user ivoson commented on the issue:
https://github.com/apache/spark/pull/21400
cc @jose-torres Thanks for the review. The last commit addresses the
comments you left.
---
-
To unsubscribe, e-mail: reviews
Github user ivoson commented on a diff in the pull request:
https://github.com/apache/spark/pull/21400#discussion_r191980682
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala
---
@@ -34,7 +34,8 @@ class ContinuousSuiteBase extends
Github user ivoson commented on the issue:
https://github.com/apache/spark/pull/21400
hi @jose-torres, the last commit add a unit test for this.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user ivoson commented on the issue:
https://github.com/apache/spark/pull/21400
@jose-torres thanks for reply. I will try to add a unit test for this.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user ivoson commented on the issue:
https://github.com/apache/spark/pull/21400
hi, @xuanyuanking @jose-torres can you take a look at this pr please?
---
-
To unsubscribe, e-mail: reviews-unsubscr
GitHub user ivoson opened a pull request:
https://github.com/apache/spark/pull/21400
[SPARK-24351][SS]offsetLog/commitLog purge thresholdBatchId should be
computed with cuâ¦
## What changes were proposed in this pull request?
Compute the thresholdBatchId to purge metadata
Github user ivoson commented on the issue:
https://github.com/apache/spark/pull/20635
thanks for reviewing this @squito
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user ivoson commented on the issue:
https://github.com/apache/spark/pull/20244
thank you for reviewing this @squito
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands
Github user ivoson commented on the issue:
https://github.com/apache/spark/pull/20635
cc @squito
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
GitHub user ivoson opened a pull request:
https://github.com/apache/spark/pull/20635
[SPARK-23053][CORE][BRANCH-2.1] taskBinarySerialization and task partitions
calculate in DagScheduler.submitMissingTasks should keep the same RDD
checkpoint status
## What changes were proposed
Github user ivoson commented on a diff in the pull request:
https://github.com/apache/spark/pull/20244#discussion_r167145734
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -2399,6 +2424,121 @@ class DAGSchedulerSuite extends SparkFunSuite
Github user ivoson commented on a diff in the pull request:
https://github.com/apache/spark/pull/20244#discussion_r166502288
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -2399,6 +2424,121 @@ class DAGSchedulerSuite extends SparkFunSuite
Github user ivoson commented on the issue:
https://github.com/apache/spark/pull/20244
@squito Hi Rashid, thanks for your review and advise. The PR description
and JIRA have been updated , and also put the stack trace on the JIRA.
The last commit addresses the comments you left
Github user ivoson commented on a diff in the pull request:
https://github.com/apache/spark/pull/20244#discussion_r166387840
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -1016,15 +1016,23 @@ class DAGScheduler(
// might modify state
Github user ivoson commented on a diff in the pull request:
https://github.com/apache/spark/pull/20244#discussion_r166387716
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -2399,6 +2424,115 @@ class DAGSchedulerSuite extends SparkFunSuite
Github user ivoson commented on a diff in the pull request:
https://github.com/apache/spark/pull/20244#discussion_r166387660
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -2399,6 +2424,115 @@ class DAGSchedulerSuite extends SparkFunSuite
Github user ivoson commented on a diff in the pull request:
https://github.com/apache/spark/pull/20244#discussion_r166386639
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -2399,6 +2424,115 @@ class DAGSchedulerSuite extends SparkFunSuite
Github user ivoson commented on a diff in the pull request:
https://github.com/apache/spark/pull/20244#discussion_r166386624
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -2399,6 +2424,115 @@ class DAGSchedulerSuite extends SparkFunSuite
Github user ivoson commented on a diff in the pull request:
https://github.com/apache/spark/pull/20244#discussion_r166386259
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -2399,6 +2424,115 @@ class DAGSchedulerSuite extends SparkFunSuite
Github user ivoson commented on a diff in the pull request:
https://github.com/apache/spark/pull/20244#discussion_r166385409
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -2399,6 +2424,115 @@ class DAGSchedulerSuite extends SparkFunSuite
Github user ivoson commented on a diff in the pull request:
https://github.com/apache/spark/pull/20244#discussion_r166385020
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -2399,6 +2424,115 @@ class DAGSchedulerSuite extends SparkFunSuite
Github user ivoson commented on the issue:
https://github.com/apache/spark/pull/20244
This is the stack trace of the Exception.
```
java.lang.ClassCastException: org.apache.spark.rdd.CheckpointRDDPartition
cannot be cast
Github user ivoson commented on a diff in the pull request:
https://github.com/apache/spark/pull/20244#discussion_r161145542
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -2399,6 +2417,93 @@ class DAGSchedulerSuite extends SparkFunSuite
Github user ivoson commented on a diff in the pull request:
https://github.com/apache/spark/pull/20244#discussion_r161145538
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -2399,6 +2417,93 @@ class DAGSchedulerSuite extends SparkFunSuite
Github user ivoson commented on a diff in the pull request:
https://github.com/apache/spark/pull/20244#discussion_r161145547
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -96,6 +98,22 @@ class MyRDD(
override def toString: String
GitHub user ivoson reopened a pull request:
https://github.com/apache/spark/pull/20244
[SPARK-23053][CORE] taskBinarySerialization and task partitions calculate
in DagScheduler.submitMissingTasks should keep the same RDD checkpoint status
â¦d is the same when calculate
Github user ivoson commented on the issue:
https://github.com/apache/spark/pull/20244
@xuanyuanking could review this please?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user ivoson closed the pull request at:
https://github.com/apache/spark/pull/20244
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
GitHub user ivoson opened a pull request:
https://github.com/apache/spark/pull/20244
[SPARK-23053][CORE] taskBinarySerialization and task partitions calculate
in DagScheduler.submitMissingTasks should keep the same RDD checkpoint status
â¦d is the same when calculate
35 matches
Mail list logo
