Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r157111981
--- Diff:
core/src/main/scala/org/apache/spark/internal/io/SparkHadoopWriter.scala ---
@@ -60,17 +60,17 @@ object SparkHadoopWriter extends Logging {
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r157111526
--- Diff:
core/src/main/scala/org/apache/spark/mapred/SparkHadoopMapRedUtil.scala ---
@@ -70,7 +70,8 @@ object SparkHadoopMapRedUtil extends Logging {
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r157110808
--- Diff:
core/src/main/scala/org/apache/spark/mapred/SparkHadoopMapRedUtil.scala ---
@@ -70,7 +70,8 @@ object SparkHadoopMapRedUtil extends Logging {
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r157068238
--- Diff:
core/src/test/scala/org/apache/spark/rdd/PairRDDFunctionsSuite.scala ---
@@ -908,6 +918,40 @@ class NewFakeFormatWithCallback() extends
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r157067651
--- Diff:
core/src/main/scala/org/apache/spark/mapred/SparkHadoopMapRedUtil.scala ---
@@ -70,7 +70,8 @@ object SparkHadoopMapRedUtil extends Logging {
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r157064132
--- Diff:
core/src/test/scala/org/apache/spark/rdd/PairRDDFunctionsSuite.scala ---
@@ -908,6 +918,40 @@ class NewFakeFormatWithCallback() extends
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r157063770
--- Diff:
core/src/main/scala/org/apache/spark/mapred/SparkHadoopMapRedUtil.scala ---
@@ -70,7 +70,8 @@ object SparkHadoopMapRedUtil extends Logging
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19848
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r154403383
--- Diff:
core/src/main/scala/org/apache/spark/mapred/SparkHadoopMapRedUtil.scala ---
@@ -70,7 +70,8 @@ object SparkHadoopMapRedUtil extends Logging {
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r154256205
--- Diff:
core/src/main/scala/org/apache/spark/mapred/SparkHadoopMapRedUtil.scala ---
@@ -70,7 +70,8 @@ object SparkHadoopMapRedUtil extends Logging {
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r154238528
--- Diff:
core/src/main/scala/org/apache/spark/mapred/SparkHadoopMapRedUtil.scala ---
@@ -70,7 +70,8 @@ object SparkHadoopMapRedUtil extends Logging {
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r154237986
--- Diff:
core/src/main/scala/org/apache/spark/internal/io/SparkHadoopWriter.scala ---
@@ -102,14 +103,15 @@ object SparkHadoopWriter extends Logging {
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r154235818
--- Diff:
core/src/main/scala/org/apache/spark/mapred/SparkHadoopMapRedUtil.scala ---
@@ -70,7 +70,8 @@ object SparkHadoopMapRedUtil extends Logging {
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r154236366
--- Diff:
core/src/main/scala/org/apache/spark/internal/io/SparkHadoopWriter.scala ---
@@ -102,14 +103,15 @@ object SparkHadoopWriter extends Logging {
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r154168397
--- Diff:
core/src/main/scala/org/apache/spark/mapred/SparkHadoopMapRedUtil.scala ---
@@ -70,7 +79,14 @@ object SparkHadoopMapRedUtil extends Logging {
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r154156234
--- Diff:
core/src/main/scala/org/apache/spark/internal/io/FileCommitProtocol.scala ---
@@ -106,6 +106,12 @@ abstract class FileCommitProtocol {
*/
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r154157813
--- Diff:
core/src/main/scala/org/apache/spark/mapred/SparkHadoopMapRedUtil.scala ---
@@ -70,7 +79,14 @@ object SparkHadoopMapRedUtil extends Logging {
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r154156530
--- Diff:
core/src/test/scala/org/apache/spark/rdd/PairRDDFunctionsSuite.scala ---
@@ -524,6 +525,13 @@ class PairRDDFunctionsSuite extends SparkFunSuite
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r154156477
--- Diff:
core/src/test/scala/org/apache/spark/rdd/PairRDDFunctionsSuite.scala ---
@@ -524,6 +525,13 @@ class PairRDDFunctionsSuite extends SparkFunSuite
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r154158671
--- Diff:
core/src/test/scala/org/apache/spark/rdd/PairRDDFunctionsSuite.scala ---
@@ -524,6 +525,13 @@ class PairRDDFunctionsSuite extends SparkFunSuite
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r154137939
--- Diff:
core/src/main/scala/org/apache/spark/internal/io/FileCommitProtocol.scala ---
@@ -106,6 +106,12 @@ abstract class FileCommitProtocol {
*/
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r153957791
--- Diff:
core/src/main/scala/org/apache/spark/internal/io/FileCommitProtocol.scala ---
@@ -106,6 +106,13 @@ abstract class FileCommitProtocol {
*/
GitHub user rezasafi opened a pull request:
https://github.com/apache/spark/pull/19848
[SPARK-22162] Executors and the driver should use consistent JobIDs in the
RDD commit protocol
I have modified SparkHadoopWriter so that executors and the driver always
use consistent JobIds
23 matches
Mail list logo
